This page describes how digital twins can be qualified. It differentiates the parts which are Software, Tools and Libraries (in the sense of ISO 26262) and proposes corresponding Tool and libary qualification methods in compliance with appropriate safety standards

With the rise of artificial intelligence in autonomous driving systems and robotics, creating a digital twin via a model and a simulator to validate the AI has become crucial. Since these simulators impact the safety of the product, this raises the question:

How can we have confidence in the use of a model and a simulator for the purpose of supporting validation?

There are functional safety standards and regulations that provide guidance on this matter:

• EU 2022 R 1426
• UNECE #171
• ISO 21448

The regulation EU 2022 R 1426.

The regulation EU 2022 R 1426 introduces comprehensive guidelines for the credibility assessment of modelling and simulation in ADS (Automated Driving Systems) validation. It is especially important since compliance with this regulation is a prerequisite for cars to be legally allowed to operate on roads in the European Union.

Part 4 of the regulation demands ensuring that models and simulations are reliable and accurate representations of real-world conditions. It outlines the necessary steps for verifying and validating these virtual environments, including steps like code and calculation verification, the use of high-fidelity models, and the continuous improvement of these tools. By adhering to these principles, it can be ensured that the models and simulators provide credible and trustworthy results and, consequently, enhance the safety and reliability of their automated driving system.

UNECE Regulation No. 171

The UNECE Regulation No. 171 is not concerned with ADS but with DCAS (Driver Control Assistance Systems), this means in particular, that the systems are only supporting the driver but do not completely take over the driving task. However, regarding modeling and simulation for verification of the DCAS, UNECE Regulation #171 aligns with the EU R 2022/1426.

ISO 21448

ISO 21448, also known as Safety of the Intended Functionality (SOTIF), is a crucial standard for ensuring the safety of road vehicles, particularly those equipped with autonomous driving systems (ADS). The standard is particularly important for functionalities that rely on complex sensors and processing algorithms, making it a key component in the development and validation of safe and reliable autonomous vehicles.

It addresses the topic of validation and verification of ADS: Analyzing known scenarios and recommending activities to uncover unknown scenarios. Concerning the software tools with which these validations and verifications shall be executed, it clearly states that ISO 26262-8 clause 11 - Confidence in the use of software tools is assumed to be applied and if necessary adapted to achieve SOTIF.

The Validas Solution

The above listed regulations and standards have, with respect to a simulator being used for validation of an ADS or DCAS, one commonality: The model and simulator are treated as software tools with safety impact on the system and consequently proof must be provided, that it is fit for its purpose, i.e. it must be analyzed and, depending on the result of the analysis, qualified.

Validas is an expert in tool analysis and qualification with a focus on automotive software tools. We have extensive experience in qualifying software tools. This can be applied in diverse situations, not limited to automotive, to make models and simulators compliant and safe. For example, qualifying a model and simulator according to ISO 26262 - 8 clause 11, is a requirement for using them to achieve SOTIF as laid out in ISO 21448.

Additionally, Validas has done a risk analysis for models and simulators. This allows us to apply tool qualifications to make important parts of digital twins compliant and safe. For the remaining software and system part of a digital twin, Validas provides checklists to analyze compliance and safety.

On the topic of the regulations, we have applied our experience to devise a solution on how to support our customers with the specified activities, to assess the credibility of their simulators. It is based on the principle of the regulations and ISO 26262 - 8 clause 11 Confidence in the use of software tools and consists of three pillars.

Pillar 1: Qualifying the Model

When using a simulation model, as a representation of a system, for ADS validation, one needs confidence in its capability to represent the actual target system and that it does so accurately and correctly. Additionally, an intricate model requiring expert operation may be prone to errors due to its inherent complexity.

For each of the three properties of the model, the capability, the accuracy and correctness, there are requirements that need to be met to prove that the model possesses these properties. Giving proof that the model possesses these properties will be done via analyzing and in some cases qualifying the model which includes either a sound, documented argument, or a validation that the model satisfies these requirements.

High-quality models of a system usually require highly skilled professionals to work with them. To mitigate sources of errors that stem from the high demands on the operator, the use of assisting software tools can be employed. Ensuring that these software tools are not themselves sources of errors is crucial, so they will also undergo qualification.

Pillar 2: Qualifying the Simulator

The simulator, provides an imitation of the operation of a real-world system over time. Similar to the model, it is crucial to develop confidence in its capability to represent the real world, and to do so accurately and correctly. Also, it must be ensured that the simulator can perform accurately and correctly over the ODD (Operational Design Domain) and is capable of yielding calculations of sufficient quality to allow conclusions about the model, i.e., that it is fit for its purpose. Finally, as with the model, the simulator being usually as intricate as the model, will also be prone to errors due to its complexity.

Building confidence in the capability, accuracy and correctness of the simulator can be done similarly to the model. For some features of the simulator a sound argument for being confident in their use can be found, other features may need to be validated to gain confidence.

Proving that the simulator is fit for its purpose will be done in the same way as is done for capability, accuracy, and correctness: The simulator is analyzed w.r.t. its requirements to be fit for the purpose, e.g., the ODD. If a sound argument can be made for a feature of the simulator being fit for its purpose it is documented. If no such argument can be made, the simulator will be qualified.

As with software tools that relieve an expert in operating the model, the tools supporting an expert in operating the simulator may be used. Again, ensuring that these software tools are working functionally correct will be shown by means of a qualification.

Pillar 3: Qualifying other Supporting Tools

Ensuring compliance with EU and UNECE regulations involves the use of a comprehensive tool chain, which spans multiple stages of the process. This tool chain encompasses everything from tools used to manage the test scenarios to those that map these scenarios to regulatory requirements, as well as tools designed to measure and output Key Performance Indicators (KPIs) as well as documenting them.

However, using tools in this manner necessitates a thorough analysis to identify any potential sources of error and assess the probability of the error remaining undetected. Depending on the outcome of this analysis, qualification of the tools may be required.

Establishing the Pillars with Validas

With two decades of experience in tool qualification, Validas stands ready to analyze and qualify your models, simulators and supporting tools to your satisfaction, ensuring that all aspects of the tool chain perform reliably and safely, and providing proof thereof, consequently supporting a seamless path to regulatory compliance.

Validas Offering

Validas supports you with:

• qualification of your simulator & models
• qualification of digital twins (based on a risk analysis)
• compliance argumentation for the EU 2022 R 1426, UNECE #171 and ISO 21448

Next Steps

The following steps are are free of charge within a cost free strategy talk:

• Learn from Validas about tool qualification
• Get to know the principal risks in digital twins identifed by Validas
• Plan a roadmap to make your digital twin (model and simulator) compliant and safe

After planning the roadmap, Validas can support you to qualify your digital twin in cooperation with you and ensure compliance with the required standards.

Book your strategy talk with Oscar here