Iterative Optimization of Hyperparameter-based Metamorphic Transformations

Verification and validation of a software system to ensure compliance with the specification and intended functional behaviour often pose a challenge when it lacks an explicit test oracle. We present an efficient black-box metamorphic testing approach in which test cases are automatically generated based on metamorphic transformations. The hyperparameters of several metamorphic transformations are optimized on the fly using a generative AI with a feedback loop for optimal test generation and test suite minimization. The proposed method uses several combined metamorphic relations to define test inputs and to determine the test verdict. The feedback on test quality is evaluated based on the metamorphic relation’s fitness function and used to optimize the next iterations of test generation. The effectiveness of the proposed approach is evaluated on an industrial case study of a crane’s load position system which lacks an explicit test oracle. The experimental results confirm that optimizing the morphing transformations using the feedback loop improves the effectiveness of metamorphic test input generation. The outcome of the study shows that the approach can be potentially applied for functional safety verification in software systems with a test oracle problem.