Strong assurance cases are achieved by building confidence in the validity of reasoning steps and demonstrating how defeaters are handled. These two approaches present two different directions of argumentation. The first one focuses on providing sound basis for the argument while the second aims at defending it against known defeaters.
Which approach is more effective and should be used? The explicit specification of defeaters seems attractive because it provides direct answers our doubts. When I have a “but…” comment, someone can point to a corresponding defeater in the argument and present how it is handled. In this way an assurance case can address the concerns of doubters. They can be satisfied when they find direct answers for their doubts.
On the other hand handling defeaters creates additional explanations for the argument, which should be sound in the first place. If the argument is weak, the response to a defeater should involve a modification of the argument structure. To avoid this we should develop sound arguments from the outset. The best way is to base on proven argument templates and ensure validity of all reasoning steps. The goal is to avoid weaknesses in the whole argument development process.
There are some basic rules that help in the development of sound arguments.
- Specify a strategy for each reasoning step, avoid direct supporting claims by subclaims. This will help reviewers to check if the argument decomposition is complete and valid.
- Specify a justification why you think the strategy is valid for a given claim in its context. Is it valid in all application scenarios? Have you considered all interactions? Are all subclaims valid in the main claim context? Identify all possible weaknesses. The justification is especially important when you reuse argument structures in a new context.
- When the justification is not straightforward, extend it with a confidence argument.
A confidence argument is an argument that supports the credibility of the main assurance argument. The two arguments form an inseparable pair and should be kept together in one assurance case module. This helps to ensure consistency of argument. An argument module can have one or more confidence top claims. The diagram below presents the general schema for the use of conformance argument. Conformance claim CA1 is used to support strategy S1 in the main assurance argument.
The confidence argument for a reasoning step should justify the validity of the reasoning, taking into account the conditions:
- The inference rule is valid for a given claim within its context.
- The inference rule has been applied correctly and it covers the full required context.
- The premises (sub-claims) are correctly defined and satisfy all conditions.
- The premises are valid and consistent.
- There no unresolved defeaters or rebuttals.
Explicitly specifying strategies and justifications helps to identify weak reasoning early on. Although it may seem to slow down the development process, detecting argument deficiencies enables the argument structure to be adapted early on in the system assurance process. Confidence argument reviews, involving third-party reviewers when possible, are a good system assurance process practice.
You can read more about the use of confidence argument in the paper “Integrating Confidence and Assurance Arguments” published in 2015. The main idea of the confidence argument is still valid.