Validation of dynamic simulation models using metamorphic testing and given-when-then patterns

As the maritime industry evolves, there is a focus on simulation-driven design, testing, and validation using novel technology solutions. Simulation models designed to represent the behaviour and features of real systems are increasingly available for testing during the early phase of the full development, but in many cases, their testing suffers from the availability of test oracles. Metamorphic testing has become increasingly used in different application domains as an approach to test systems when an explicit test oracle is unavailable. In order to increase its adoption by domain experts, we combine metamorphic testing with Behaviour Driven development for the verification and validation of simulation models. The tool-based approach facilitates automated test generation based on domain-specific custom metamorphic transformations to generate meaningful test inputs for metamorphic input relations. The method also uses features and scenarios extracted from system requirements and domain expertise to define metamorphic output relations. By automating test generation based on system behaviours as features,scenarios, metamorphic transformations, and output candidates for metamorphic relations in a Gherkin-like format, the tool enables practitioners to verify models based on domain-specific constraints and metamorphic relation checks. Our preliminary evaluation shows that the tool can detect MR violations in the simulation models under test and that automated test generation provides improved coverage.