You are here:
Hierarchical Metamorphic Relations for Testing Scientific Software
Citation:
Lin, X., M. Simon, AND N. Niu. Hierarchical Metamorphic Relations for Testing Scientific Software. In Proceedings, 2018 International Workshop on Software Engineering for Science, Gothenburg, SWEDEN, June 02 - 08, 2018. Association for Computing Machinery (ACM), New York, NY, 99-99, (2018).
Impact/Purpose:
This proceedings paper is an early effort to performing automated software testing for scientific code.
Description:
Scientist developers have not yet routinely adopted systematic testing techniques to assure software quality. A key challenge is the oracle problem, a situation in which appropriate mechanisms for checking whether the program under test produces the expected output when executed using a set of test cases (TCs). Metamorphic testing alleviates the oracle problem by specifying the relationship that a source TC and its follow-up test cases (TCs). Metamorphic testing alleviates the oracle problem by specifying the relationship that a source TC and its follow-up TC shall meet. Such relationships are called metamorphic relations (MRs) which are necessary properties of the intended program's functionality. Existing approaches handle MRs in a flat manner. This paper introduces a novel way to facilitate a hierarchy of MRs to be developed incrementally. We illustrate our approach by testing the U.S. EPA's Storm Water Management Model (SWMM). The results offer concrete insights into developing effective MRs to systematically test scientific software.