Best Practices for Stable miniHIL Test Cases

Self-contained tests: Each test must run successfully when executed individually, without relying on the execution order or state resulting from other tests. During test case creation it is advisable to execute the test case in separation as well as with some test cases executed before it. This ensures correct system initialization, simulation resets, …​

Ensure Configuration State: Tests must explicitly verify and, if necessary, configure the target system to a known, predefined state before execution.

Target System Initialization: Always explicitly initialize the target system (e.g., perform required routines, resets, calibrations) prior to running a test. Do not rely on the initialization performed by previous tests.

In case some of the initialization steps are expensive either time or memory write cycle wise we recommend to add steps which can verify if the expected state is already present. These steps should check for the expected state and should only execute the initialization if the state is not present, yet. We usually prefix theses steps with ensure…​

Test cases should not rely on some kind of tear down performed by previous tests. Previous tear downs might not have been executed correctly or might change in the future. Dependence on earlier tests makes reproducible test runs very complicated and time consuming.

In some cases the configuration might influence the timing and or values to be checked in the test cases. If possible we recommend to validate the configured values as part of the test case. This will lead to explicit test failures in case of unexpected configuration changes instead of random failures somewhere down the line.

Reusable Setup Steps: Create and utilize standardized initialization or setup steps for different test scenarios or configurations.

Clearly define and document these reusable setup sequences.

Keep setup sequences modular and clearly named, e.g., setup for different configurations (Setup_Config1, Setup_Config2, etc.).

Convenience Steps: Use and maintain convenience functions or steps provided to simplify common testing tasks. This ensures cleaner test design, readability, and reduces redundancy.

Balanced Approach to Code Reuse: Code deduplication is not as critical in test development as it is in software development. Test cases should prioritize clarity and maintainability over strict adherence to DRY (Don’t Repeat Yourself) principles.

Use Generic Test Steps: When you have individual requirements that test multiple values of the same functionality (e.g., tests that verify positive and negative results depending on parameters, bus message parameters for different SW configurations), consider using a generic, configuration-agnostic test step, but call it from specific test cases!

One-to-One Test Mapping: Maintain a clear, direct mapping between each miniHIL test case and its respective external tracking or management system (e.g., Codebeamer, Jira, or similar tools).

Ensure exactly one miniHIL test case per external management test case.

Use @MetaInfo annotations to link the miniHIL testcases to ids or names in your traceability tool. This will make automatic sync or import much easier if it is to be used (even if only in the future).

Unambiguous Test Outcomes: If a test scenario requires independent evaluation (pass/fail criteria), define it as a separate, stand-alone test. Avoid bundling multiple scenarios under a single test, as this can complicate failure analysis.