Auditability is an important aspect of systems used in critical industries, such as automotive, energy, aviation or pharmaceuticals. In the latter, the term audit trail is used, which means an automatically created and secured record of the history of activities performed in the system. PREMIS implements audit trail records in accordance with the ALCOA+ principles, which ensure the effectiveness and usability of these records.
What is an audit trail?
An audit trail involves the automatic collection of data to identify who created, modified or deleted what data and when, while preserving the initial and new values. The aim is to ensure the integrity, traceability and accountability of the data and enable the reproduction of events during an audit or inspection. Below, we will break this down into more detailed requirements and demonstrate their implementation in PREMIS.
This mechanism protects against data manipulation, falsification and unauthorised changes, and enables you to determine responsibility for individual data and changes within the system. This makes it a key aspect of quality management systems, with a wide range of applications. It is also a prerequisite for effective system security (cybersecurity). An audit trail does not protect data against attacks; rather, it provides the basis for analysing events and accounting for activities. In section 3.3, ‘Audit and Accountability‘, NIST SP 800-53 defines ‘audit records’ (AU-3), which are equivalent to audit trails. The audit trail is fundamental to many industries and applications. Therefore, PREMIS has been designed from scratch to ensure its effective use.
The audit trail itself is defined in Annex 11 of the Good Manufacturing Practice (GMP) guidelines, which are the fundamental regulations for the pharmaceutical industry. It is also defined in the ‘Best Practices for Data Management and Data Integrity in GMP/GDP Regulated Environments’ guidelines. In the US, the main relevant regulation is 21 CFR Part 11 (Electronic Records; Electronic Signatures). The use of an audit trail is also related to good practices for ensuring data integrity, namely ALCOA+, which we will discuss in more detail below. Without ensuring data integrity in accordance with ALCOA+, an audit trail alone is not very effective.
How PREMIS implements the audit trail
PREMIS implements a data management mechanism that records changes to the data, saving each subsequent version in the database. This includes all assurance case argumentation data, including assessment and references to evidence. If the evidence is stored in PREMIS-managed repositories, it is also included. This allows the history of changes to be presented for each element of the argument, as well as at the level of the entire argumentation.
The following audit trail requirements are implemented by PREMIS:
- Audit trail records are automatically created for all actions on the argumentation data, and this feature cannot be disabled, even by the PREMIS administrator. The records include all actions relating to the creation, modification and deletion of data.
- The audit trail data is complete. Deleted data is marked as such but is still stored in the system for audit purposes.
- All records are timestamped in UTC to ensure that the time of each event is clearly determined.
- Each version of the data, including earlier versions and the version resulting from the change, is preserved. It is possible to compare the state of the data before and after a change.
- Each event is uniquely assigned to an individual user account.
- Users and administrators can view the history of changes and data versions (including the full argumentation version), but cannot modify this data.
- Access to audit trail data requires dedicated permission.
- The audit trail is an integral part of the PREMIS data and is subject to backup and disaster recovery processes.
Below is an example of an argument element’s change history window. Information about who changed what, when, and how much was changed is presented.
Change history is collected for entire assurance case arguments. The window fragment below shows version 1.3 of the argument as of August 28th. You can review the full argument from this version and create a copy to continue further work from that point.
ALCOA+ principles to ensure data integrity
ALCOA was created in the 1990s as an informal mnemonic tool to help FDA inspectors check data integrity. Dr. Stan W. Woollen originated the tool and wrote down these rules as a checklist, creating the acronym ALCOA based on the main words. These words are: Attributable, Legible, Contemporaneous, Original and Accurate. The checklist was so effective that it became popular in the industry, and over time the term began to be used in regulations as well. However, with technological developments after 2010, the ALCOA rules became insufficient, so they were expanded and a plus sign was added to the end of the name. This is how the name ‘ALCOA+’ was created. The most comprehensive description of the rules can now be found in the PIC/S guide PIC/S PI 041-1.
How does PREMIS support ALCOA?
ALCOA, established in the 1990s, encompasses five principles:
- Attributable. It must be possible to identify the person or system that performed the recorded action and the time of its execution. This also applies to all data creation, modification, and deletion activities.
- Legible. Audit trail data should be legible, meaning it can be easily interpreted. Electronic data should be user-friendly and searchable.
- Contemporaneous. Evidence of actions, events, or decisions should be recorded at the time they occur in the system, not replayed later.
- Original. The original record is the first capture of information. This exact information should be retained as the audit trail.
- Accurate. Audit trail data should contain accurate versions of business data (before and after the change).
The extension marked with a plus sign, included in the ALCOA+ rulebook, includes ALCOA requirements and four new ones:
- Complete. All information relevant to event recovery should be recorded, including available metadata. Do not record only the main data items.
- Consistent. Information should be created, processed and stored maintaining logical connections, in particular the order (chronology) of events, but also a clearly defined time format, units of measurement or rounding.
- Enduring. Data should be stored in such a way that they are available for the entire period in which they may be needed, which formally means maintaining the data throughout the required retention period.
- Available. Data should be available for review at any time during the required retention period, in a readable format, by all authorized persons.
All of the above ALCOA+ requirements are implemented by PREMIS.
Practical benefits
In addition to meeting formal requirements, the audit trail at PREMIS provides real operational and business value:
- makes it easier to detect errors and abuses before they grow into a serious quality problem;
- shortens the time of preparation for audits and inspections, and at the same time reduces stress and the risk of new comments (observations);
- facilitates the analysis of deviations and the determination of the sequence of events;
- increases trust and responsibility in teams when, on the one hand, everyone knows that their actions can be checked, but there is also no risk of unfounded accusations;
- gives the possibility of reconstructing historical data and resuming work on its basis, which makes it possible to reduce the consequences of erroneous actions in business processes;
- It allows you to conduct analyses of the occurrence of errors or user behavior over a long period of time.
The audit trail and the ALCOA+ rules are not only a formal requirement, but a useful business practice that reduces the risk of errors in processes and provides the basis for effective actions in exceptional situations. From the early stages of work, PREMIS has been designed and developed in accordance with these principles.