As the saying goes ‘it is about the journey, not the destination’. For us this also applies to assurance cases. They are dynamic objects which evolve in the system life cycle. What is important is the way how you manage the status and changes of the argument and the evidence to finally get and maintain a valid and convincing assurance case. This especially critical for continuous assurance cases.
This is opposite to the concept of a static assurance case report. Yes, the approved assurance case report is static but it is the process of assurance case development and evolution that we should focus on. It is hard to add quality at the last stage. It should be built from the very beginning. The quality of the final assurance case report should result from the quality of the whole process. A practice we can advise is continuous argument reviews with the use of the assessment.
The assessment is a way of communicating the argument quality between parties involved in the process. The assessment contains two pieces of information: the assessment value and a narrative comment.
In diagrams we use colors to represent the assessment value which allow to quickly see the status in large arguments:
- Green color represents accepted parts of the argument.
- Red color is used to present the rejection of the argument and indicate problems or defeaters.
- Yellow color is used for missing information or uncertain quality.
The assessment comments help to identify problems and give guidance on how to improve the argument structure. Users can share comments, questions and answers and discuss specific parts of the arguments.
You can also use the assessment to track progress in achieving fully accepted argument. The target can be called as the “all green” argument. The red color in the argument assessment means that some parts have been rejected, probably some defeaters had been found. The argument needs to be corrected so that the red color disappears when re-evaluated.
The assessment of an assurance case is often performed in two phases. First you develop the argument and do the self-assessment. The complete argument may be subject to third party review. When possible you can also involve independent reviewers at early stages of the assurance case development.
Does frequent or continuous assessment make sense when the argument is still being modified? Any change to any element of the argument or evidence requires a reassessment. The problem is to determine which assessments are no longer valid. NOR-STA helps with this and identifies areas that need to be reviewed and reassessed.
In modular arguments the assessment mechanism will take into account the dependencies on other modules. You will get acceptance of a given module only if all supporting modules are accepted. Changes to sub-modules have an immediate effect on the assessment of higher-level modules.
The assessment flows through the argument structure up to the top claims and to the high-level argument modules. This mechanism is sensitive to any changes in the argument elements and evidence and it enables continuous control of argument quality and validity.
A quick summary:
- The assessment gives information which parts of the argument are complete and verified.
- In case of any changes the tool will advise to reassess the affected parts of the argument.
- This also works for modular arguments, because evaluation is passed through module interfaces.
The assessment mechanism in NOR-STA is one of the elements enabling the continuous evolution of assurance cases.