The use of CI/CD tools (Continuous Integration and Continuous Delivery/Deployment) is increasingly common in the development of critical systems. The same CI/CD pipelines can also be used to automate the creation and maintenance of assurance cases including the argument structure and evidence, and also for the argument verification and status data updating.
CI/CD process and assurance cases
Let’s start with the concept of CI/CD automation processes and see how this can be related to the assurance case for the system being developed. The CI/CD process ensures the automatic creation, testing, and deployment of a system, where the software is its main part. Is the data used in the CI/CD pipeline useful for the assurance case of a given system? Undoubtedly, yes, this data includes at least information about the system configuration and the results of automated tests, which gives us information about system safety of security, and affects the structure of the argument and the evidence used. Additional value is provided by the results of the tests, which can be used to update information about the status of the argument.
The CI/CD process can also include all the information needed to build a safety/security case for a given system. The main artifacts for a security case can be threat models (HARA, STRIDE), SBOM (Software Bill of Materials), SAST / DAST / IAST reports. For safety, these will be hazard models, HAZOP, FMEA, FTA, and others. When the CI/CD process ensures that all artifacts are valid for a given version of the system, we get a consistent set of data and evidence to be used in the assurance case. Automation ensures that when creating a new version of the system, we can receive all the data to create an argument, and one of the final steps of the CI/CD process can be the creation of the assurance case itself. That means that we get an assurance case for each new build of the system.
The approach is effective when the entire CI/CD process is designed from the beginning to provide all the information needed for the assurance case. This enables the achievement of a Safety/Security Case driven development process, where the entire system development process is focused on providing the arguments and evidence needed for system certification.
Argument generation process
Technically, the assurance case integration process in the CI/CD process can include the four activities shown in the diagram below.
The first step concerns the argument template. Even if you use simple scripts to generate arguments, you need to verify them before applying them. We use templates, which gives the opportunity to use a universal tool for generating arguments. We recommend preparing a metamodel of the system data tailored for the assurance process. This will enable us to verify the template’s correctness and consistency with the metamodel. When needed, you can develop modular templates.
The second step is crucial for obtaining data to build arguments. Data in a CI/CD process is typically placed in a GIT repository using different file formats. If a file from the repository is to be used directly as evidence in an argument, then a reference to it can be used, but in many cases it is necessary to extract the data from a file. The argumentation generator needs input data that constitutes a consistent ontology according to the metamodel of the system data. Some data may be directly accessible, but it is often necessary to use AST parsing to extract the right data. As a result of the step, we have data about a given version of the system in a model format ready to be used to generate arguments.
The third step is the main step of argument generation for the system version based on its data model according to the assurance case template. Depending on the templates structure, specified conditions and parameters, an argument is created for a given version of the system with references to the relevant evidence. The argument is saved in a JSON format that conforms to the OMG SACM metamodel and can be further processed.
The last, fourth step is to verify the argument. This includes checking for completeness, consistency, and other correctness conditions. As a result, in addition to the argumentation itself, we get a report of its verification.
Subsequent actions related to the generated arguments are carried out when the CI/CD process is completed. For example, the PREMIS tool is used to visualize argumentation in GSN diagrams and generate reports. You can also combine the generated argument modules with other modules that are created manually in PREMIS.
Are you interested in insurance case automation, or do you have any questions? Get in touch!