Citation:

Lin, X., M. Simon, AND N. Niu. Exploratory Metamorphic Testing for Scientific Software. IEEE Computing in Science & Engineering. IEEE Computer Society, Washington, DC, 22(2):78-87, (2018). https://doi.org/10.1109/MCSE.2018.2880577

Impact/Purpose:

The purpose of this article is to explain to the technical public an exploratory method for testing software when there are no known results to compare the new code results. This methodology will assist scientific programmers in writing and combing new scientific code to established code.

Description:

Scientific model developers are able to verify and validate their software via metamorphic testing, even when the expected output of a given test case is not readily available. The tenet is to check whether certain relations hold among the expected outputs of multiple related inputs. Contemporary approaches require the relations to be defined before tests. Our experience shows that it is often straightforward to first define the multiple iterations of tests for performing continuous simulations, and then keep multiple and even competing metamorphic relations open for investigating the testing-result patterns. We call this new approach exploratory metamorphic testing, and report our experience of applying it to detect bugs, mismatches, and constraints in automatically calibrating parameters for the United States Environmental Protection Agency’s Storm Water Management Model (SWMM).

URLs/Downloads:

Record Details: