Experience report LA2 audit support after DOQ introductions
In the course of the DOQ implementation (MES solution), many of our customers/prospective parties want to address not only the topic of digitalization, but also, above all, their (so-called) open regulatory flank on the topic of computer system validation (CSV).
Together with our customers, we are therefore increasingly trying to close this gap during the DOQ implementation. This means that in addition to the 21 CFR Part 11 compliance of our DOQ system, we are also increasingly addressing the topic
„Insufficient validation approach and missing validations of existing IT applications in companies.“ In addition to the lack of spreadsheet validation of existing Excel files, this primarily concerns the lack of compliance of existing ERP and CAQ systems.
A very similar situation can currently be found in many medical technology companies: some software application is not “validated” at all or not sufficiently.
The rule of thumb—the more extensive the application, the greater the effort required to close this validation gap—unfortunately applies in many cases. Some providers of such systems deliberately perpetuate the misconception that one has purchased a validated system and nothing more needs to be done through appropriate wording on their websites and in presentations. However, this usually leads to problems during audits with notified bodies or even during FDA inspections.
Ultimately, buyers of such systems always face the same frightening realization/consequence. They are responsible for their own validation, and suddenly they require at least half a brain to maintain the painstakingly self-created validation annually. Validation projects from recent years at companies in the medical technology sector show that the costs for this (depending on company size and software version) are typically between €20,000 and €200,000 as a one-time fee, with additional annual follow-up costs of €15,000 and €50,000.
To avoid falling into this audit and cost trap, in addition to introducing a validated system (e.g., DOQ), one should also work on certain work/procedure instructions and the software classification process. For customers who actually had CSV problems, however, this was usually feasible with reasonable effort, so that this regulatory flank could be closed after 3-6 months in parallel with the DOQ introduction. Through a one-time process, a basic documentation structure was created that fulfills the essential requirements for computer system validation and which in the future only needs to be updated by the customer.
must be kept “up to date”.
Since some customers were somewhat unsure how to conduct the audit of notified bodies after the DOQ introduction or after the „flank closing“, LA2 asked us to participate in these audits in order to clarify the companies‘ regulatory CSV approach.
At this point, I would like to provide an overview of our audit experiences and report on how our clients performed in the audit.
I would like to mention that all clients passed their respective audits successfully and our support was ultimately required only to a limited extent.
However, it is quite remarkable how differently the individual audits went
and the depth with which the auditors addressed the topic of CSV. Everything from a single question on the topic to being grilled for two hours was included.
For example, one auditor limited himself to the question of whether the validated software was actually installed on the company’s servers, or whether it was not in fact a much higher version of this software, which might then not have been sufficiently validated (answer duration: 3 min).
Another auditor made us wait another 3 hours after a 3-hour journey to ask the following two questions:
Do you have traceability matrices, and can you provide me with an example requirement to show where these requirements can be found in the various validation documents? (Response time: 7 minutes)
In stark contrast to these cases, there was also another type of auditor, as the next short example will show:
During an audit in the Medical Mountains about 12 months ago, our client explained their company’s validation strategy with us. When the auditor asked if they could see the validation documents for a highly rated (critical) piece of software, we presented the folders containing all the documents and received a „Oh, that’s awesome“ response (no joke!). Two hours and what felt like 100 questions later, our client was congratulated on their validation strategy.
The following example gives a good overview of how most auditors approached the topic of CSV.
During an appointment with a notified body, the responsible auditor was given a brief overview of the validation plan and classification scheme (assessment mechanism for software systems with regard to their risk classification). When he discovered that data was being transferred from the ERP system to DOQ via an interface, he proceeded as follows: He checked the criticality classification of both systems in the software inventory. In this case, DOQ was classified as „high“ and two quality-relevant modules of the ERP system as „medium“. He then looked up the validation procedure for software systems classified as „medium“ and „high“ in the validation master plan and was shown the documents required there. In this case, these included the requirements specification and design specification, including associated test documents and traceability matrices, as well as interface validation documents and project summaries (also known as „fact files“ or
(called „project profile“) of the ERP modules. After all documents had been shown and explained within a few minutes, the auditor closed the CSV topic with the comment, „That looks good, I’d rather look for errors elsewhere.“
These were just a few examples of how these audits have been conducted so far. In summary, the younger the auditors, the more important the topic of CSV was to them. The generational shift I described in the past continues unabated. The importance of digitalization and the associated area of CSV is also increasing. The findings of the blog post from January 28, 2020
“Sharp increase in findings in Computer System Validation (CSV) in audits” ( https://doq-digital.de/wp-content/uploads/2020/01/aktikel-starke-zunahme-von-findings.pdf ) still seem to be valid.
A positive finding from many of the meetings was that the auditors were happy to be guided. This means that – if the topic of validation is considered at the beginning of this
If the auditor proactively addressed the audit topic and openly and willingly explained the respective validation strategy, many auditors limited themselves to asking only three or four follow-up questions, and the whole issue was resolved, regardless of how serious the findings had been in previous years. To clarify – validation strategy sounds rather pretentious as a term in this context. It means that clients have and can explain the following documents:
- Suitable validation master plan
- Software inventory
- Classification scheme (based on which they evaluate the software systems from their inventory and define the validation scope depending on the classification result [High-Medium-Low]).
If you were then able to provide the required validation documents for high or medium-rated software, “then the box was usually done.”
Conclusion of the LA2 audit support:
The average duration of the CSV audit across all audits was approximately 20 minutes. The examples mentioned above represent the shortest (3 minutes) and the longest (2 hours) surveys on the topic of CSV to date.
And the most important thing last – due to the fact that DOQ, as a highly rated software solution, bears the regulatory burden for CSV, and at the same time, important regulatory data was transferred from other systems to DOQ, the other software systems involved could be downgraded in terms of their risk criticality. This reduced the audit findings of previous years, or even eliminated CSV findings when auditing existing software solutions (such as ERP, CAQ, MES, or even Excel systems)!
