Citation:

Lin, X., M. Simon, Z. Peng, AND N. Niu. Discovering Metamorphic Relations for Scientific Software from User Forums. IEEE Computing in Science & Engineering. IEEE Computer Society, Washington, DC, 23(2):65-72, (2021). https://doi.org/10.1109/MCSE.2020.3046973

Impact/Purpose:

The purpose of this work is to test updates to urban watershed scientific software

Description:

Scientific software can used for decades and is constantly evolving. The results from updated scientific software must be compared to results from earlier versions to determine their scientific veracity. Numerical accuracy and compounding errors present challenges for testing scientific software due to the lack of a precise mechanism for determining the expected program output under a given input. Recently, metamorphic testing, a property-based testing technique, can be effective in testing scientific software. The development of metamorphic relations is difficult: it requires considerable practical expertise for the software tester. The scientific software end-users have a wealth of expertise in interpreting software results. We report our experience of developing metamorphic relations from a user forum’s questions of the United States Environmental Protection Agency’s Storm Water Management Model. Our results illustrate that metamorphic relations discovered from software users’ questions can support not only correctness checking but also behavioral comprehension and end user debugging.

URLs/Downloads:

Record Details: