KENX AI in GxP East 2026

2026-09-16-1010-ai-internal-audit-radar-part11-lecture

37 min read

AI-enabled internal audit lecture on RADAR and Part 11 assessment

Recorded 2026-09-16 at 1010 via Plaud. 29:22.

Summary

A vendor lecture (Zifo materials, KENX AI in Life Sciences Summit) on modernizing internal audit for QC and manufacturing labs around three questions: do procedures meet current requirements, are processes operating in a state of control, and which risks would regulators flag that the company is missing. Three AI tools anchor the answer. RADAR is a dual-agent regulatory-intelligence system that digests FDA, EMA, and MHRA actions against organization-specific markers and redlines impacted SOPs, with human checkpoints before any change. A governance dashboard aggregates deviations, CAPAs, training, and periodic reviews into a compliance-health view (the demo showed an overall risk score of 72, 24 potential gaps, and 36 impacted documents). A checklist-driven Part 11 assessment tool (CFR Assistant and CFR Helper) reviews SOPs and audit trails, flagging non-descriptive modification reasons and training non-compliance, each flag traceable to a checklist ID and SOP entry.

Reports generate in five to ten minutes; humans still classify findings by risk, initiate deviations and CAPAs, and approve every action plan.

Plaud AI notes

<App-generated notes from the Plaud cloud, verbatim.>

Date Time: 2026-09-16 10:10:32 Location: [Insert Location] Instructor: [Insert Name]

Plaud summary

The lecture, created on 2026-09-16 at 10:10:32, presents an AI-enabled framework to modernize internal audit strategy and accelerate compliance and governance for QC and manufacturing labs, with emphasis on 21 CFR Part 11 procedural controls. It focuses on three core audit questions: alignment with current regulatory requirements (FDA, EMA, MHRA, industry trends), state of control of processes (deviations, CAPAs, change controls, training, periodic reviews), and identification of overlooked risks regulators may flag. Central AI tools include: RADAR (Regulatory AI-Driven Alert & Review) for FDA Monitoring & Regulatory Intelligence; a Governance Dashboard aggregating compliance data across deviations, CAPAs, changes, training, periodic reviews, and audit trails; and a Part 11 Assessment & Periodic Review Tool (including CFR Assistant and CFR Helper) that automates document reviews against predefined checklists, identifies gaps, and generates audit-ready reports. These capabilities enable automated SOP change monitoring and redlining, risk-prioritized governance views, and AI-assisted periodic compliance assessments and audit trail reviews. The framework stresses human checkpoints, transparency, explainability, and traceability, delivering rapid reporting (5–10 minutes) without replacing human QA oversight. The session references KENX - AI in Life Sciences Summit and Zifo’s confidential materials (2023, 2026).

Knowledge Points

1. Internal Audit Focus and Scope

  • Three core audit questions:
    • Do procedures and systems meet current requirements? Assess alignment with FDA, EMA, MHRA expectations and industry trends; ensure SOPs, work instructions, validation artifacts reflect up-to-date standards.
    • Are processes operating in a state of control? Evaluate deviations, CAPAs, training compliance, change management (including pending changes), and aging of CAPAs/training records.
    • Are risks being missed that regulators might flag? Incorporate external signals such as FDA warnings and industry trends to detect emergent risks.
  • Audit activities and artifacts reviewed: SOPs, validation/evaluation packages, periodic reviews, audit trail reviews, governance matrices, deviations, CAPAs, training compliance, and change management items; AI support addresses the challenge of documentation volume.

2. AI Application Across Audit Pillars

  • Regulatory Intelligence:
    • Challenge: Keeping pace with observations, warning letters, evolving expectations across FDA, EMA, MHRA.
    • Capability: Analyze regulatory actions, identify compliance themes, assess impact on internal procedures.
    • Tool: RADAR consolidates external sources, filters by organizational markers, provides actionable summaries, redlines SOPs, and generates audit-ready reports.
  • Continuous Governance:
    • Challenge: Monitoring compliance health across deviations, CAPAs, changes, training.
    • Capability: Aggregate compliance data, identify trends, prioritize risks, generate dashboards.
    • Tool: Governance Dashboard shows aging CAPAs, training compliance vs. non-compliance, periodic review status, audit trail insights, and recommended focus areas.
  • Documentation & Compliance Assessment:
    • Challenge: Reviewing SOPs, validation packages, Part 11 compliance, and periodic reviews at scale.
    • Capability: Checklist-based document review, gap detection, automated assessments, and review packages.
    • Tools: Part 11 Assessment & Periodic Review Tool; CFR Assistant and CFR Helper for AI-Assisted 21 CFR Part 11 procedural control assessment.

3. RADAR: Regulatory AI-Driven Alert & Review

  • Data Sources and Scope: FDA Warning Letters, Form 483s, guidance, EMA/MHRA updates, industry trends/news (including paywalled sources via original systems); geography-aware scheduling (US FDA, MHRA/EMA for EU, local systems for India).
  • Configurable Markers: Organization-specific markers (e.g., antibiotic manufacturing) tailor retrieval, relevance scoring, and outputs.
  • Dual-Agent Process with Human Checkpoints:
    • Regulatory Review Agent extracts key findings, themes, risks, trends; produces a Regulatory Summary.
    • User checkpoint: Select applicable topics for contextual relevance.
    • Gap Analysis Agent compares regulatory expectations against internal documents (SOPs, policies, standards, validation docs, work instructions); identifies gaps and impacts.
    • Report generation: Gap assessment, impacted SOPs/documents, risk ranking, recommended actions.
    • User checkpoint: Approve action plan to align with priorities.
  • Outputs and Dashboards:
    • Example dashboard: Overall Risk Score 72 (High), 24 potential gaps, 36 impacted documents; top themes include Data Integrity, Audit Trail Review, Validation, CAPA, Training; supports risk distribution, gap summaries, impacted procedures, recommended actions.
    • Regulatory Alerts view: Filter by topic/source/date; relevance scores (e.g., 95%–97%), severity (HIGH); CSV export; article summaries with rationales.
  • SOP Impact and Redlining:
    • Automated SOP impact analysis proposes changes with tracked comments tied to specific regulatory triggers (e.g., new 483s).
    • Configurable schedules (daily/weekly); traceability maintained via redlined copies and citation of triggers.

4. Governance Dashboard for Continuous Compliance

  • Compliance Health Monitoring: Displays aging CAPAs, training compliance status, periodic review status, audit trail review coverage; highlights overdue items and non-compliance trends for proactive remediation.
  • Risk Prioritization and Trend Detection: Identifies trends across deviations, CAPAs, changes, training; prioritizes remediation and resource allocation; supports internal audit readiness with customizable screens (aging metrics, CAPA details, training insights).
  • Continuous Compliance Process: Data sources → aggregation & analysis → compliance health summary → user checkpoint → risk prioritization agent → dashboard → user checkpoint.

5. Documentation Review and 21 CFR Part 11 Assessment

  • AI-Assisted Compliance Automation:
    • Goal: Reduce manual effort in document and periodic report review; accelerate information retrieval for audits; preserve human QA review/approval.
    • Components: CFR Assistant and CFR Helper Tools for procedural controls; vendor-agnostic; supports structured (Excel, PDF, JSON, XML, text) and unstructured data (images/scans via OCR/processing).
    • Integration: API/SDK-based ingestion from enterprise systems (e.g., Empower); effective across instruments and SaaS with relational databases; handles complex/unstructured audit trails (e.g., alarm systems).
  • Checklist-Based Assessment:
    • Predefined 21 CFR Part 11 checklist (e.g., 11.10(a) validation evidence) customizable by users; plain-English question editing; transparency in how items are evaluated and what evidence is checked.
    • Reporting: Compliance status (Yes/No) per regulation with explanatory remarks to guide SOP updates.
  • Periodic Review Logic and Outputs:
    • Strategy: Risk-based checklists derived from SOPs/work instructions; user approval of scope before execution.
    • Compliance Review Agent evaluates evidence, flags gaps/trends; produces Compliance Assessment reports with Gap Summary, Risk Ranking, Recommended Actions; final user checkpoint for action approval (deviations/CAPAs or acceptance of low-risk items).
    • Examples: Validation deliverables verification, configuration adherence, backup activity checks; audit trail logic includes unauthorized modification counts, duration, repeated actions by same/different users; flags linked to SOP-derived checklist IDs.
    • Sample checklist sections: System Validation Review (PRCD availability; eight validation deliverables), System Configuration Review, Backup & Audit Trail Management.
    • Sample report metrics: Change Detection & Traceback, System Access Approved, Maintenance & Calibration, System Validation Review, Backup & Audit Trail, Periodic review verification; exportable as Excel/PDF and integrable into QMS/repositories.
  • AI Audit Trail Review (AI-ATR) Flags and Traceability:
    • Flags linked to checklist IDs and SOP entries; examples include non-descriptive modification reasons (“t”/“T”), training non-compliance in Empower (users without required training before run timestamps), result review inconsistencies; detailed record excerpts with actions, objects, timestamps, IDs, escalation status; checklist descriptions specify review focus.
  • Efficiency and Human-Driven Decisions: Reports generated in 5–10 minutes, significantly reducing manual review; human reviewers classify risk (high/medium/low), initiate deviations/CAPAs, update SOPs as needed.

Questions

  • [Insert Question/Confusion]

Assignments

  • Configure RADAR markers specific to product lines (e.g., antibiotic manufacturing); schedule runs (daily/weekly); validate source configurations for FDA, EMA, MHRA, and regional authorities.
  • Review the latest RADAR Regulatory Summary; confirm applicable topics; initiate Gap Analysis; verify risk rankings, impacted SOPs/documents; approve recommended actions.
  • Implement SOP changes using redlined copies; document rationale linking each change to specific 483s/warning letters/guidances; review AI-generated SOP redlines for recent guidance changes and accept/reject updates.
  • Customize the Governance Dashboard with aging metrics, CAPA details, training records; use it to conduct internal audits and prioritize remediation activities.
  • Provide SOPs, policies, validation documents, and regulatory requirements to the Assessment Strategy Agent; review/approve checklist and scope; execute AI-assisted periodic compliance assessment; classify findings by risk; approve recommended actions.
  • Initiate deviations/CAPAs for high/medium-risk findings; document rationales for ignoring low-risk items; consider SOP updates where gaps are systemic.
  • Verify monthly data backup activity and retrievability; adjust module review frequency (e.g., validation, backup) based on risk and operational needs.
  • Monitor external high-severity regulatory alerts; export CSV summaries; perform targeted reviews where relevance scores ≥95%; perform targeted audit trail reviews focusing on modification reason quality and user training compliance; trace flags to checklist IDs and SOP entries.
  • Plan processing layers for unstructured sources (e.g., scanned documents) to enable automated review; explore API/SDK availability for other enterprise applications to integrate with the AI-assisted review system.

Highlights (button-flagged moments)

Leveraging AI in Internal Audits: A Three-Pillar Strategy

The presentation slide visually structures the three core pillars of AI application in internal audits, which the speaker then details by explaining the specific functionalities of the corresponding AI-assisted tools for regulatory intelligence, governance, and compliance assessment. • Visual Context: The slide presents a table outlining three AI-powered solutions for internal audits, aligning ‘Regulatory Intelligence’, ‘Continuous Governance’, and ‘Documentation & Compliance Assessment’ with specific AI tools. • Discussion Insight: The speaker elaborates on the three AI-assisted tools from the slide, describing the ‘FDA Monitoring & Regulatory Intelligence Tool’ for tracking regulatory changes, the ‘Governance Dashboard’ for monitoring internal compliance metrics, and the ‘Part 11 Assessment & Periodic Review Tool’ for automating document review. • Key Takeaway: The primary goal of leveraging these AI tools is to accelerate information gathering and reduce manual review time, enabling quality professionals to focus on analysis and decision-making rather than data compilation.

Explaining the RADAR AI Tool’s Workflow for Regulatory Compliance

The speaker verbally details the process flow of the RADAR (Regulatory AI-Driven Alert & Review) tool, directly corresponding to the workflow diagram displayed on the screen which shows how the system transforms external regulatory data into actionable internal SOP gap assessments. • Visual Context: The slide presents a flowchart for RADAR, an AI tool that analyzes regulatory sources, compares them against internal documents, identifies gaps, and generates reports with recommended actions. • Discussion Insight: The speaker explains that RADAR is a ‘regulatory intelligence agent’ that automates the review of sources like FDA letters and compares findings against a company’s SOPs to see if changes are needed. • Key Takeaway: The RADAR system is designed to accelerate the identification of regulatory compliance gaps by automating data gathering and analysis, allowing quality professionals to focus on decision-making rather than manual document review.

AI-Powered Governance Dashboard for Compliance and Risk Monitoring

The presented ‘RADAR Dashboard’ visually summarizes key compliance risks, such as a high risk score of 72, which the speaker explains is a tool to aggregate data and provide a quick snapshot of the organization’s state of control for audit preparation. • Visual Context: The slide displays a ‘RADAR Dashboard’ featuring an ‘Overall Risk Score’ of 72 (High Risk), 24 ‘Potential Gaps’, and 36 ‘Impacted Documents’. • Discussion Insight: The speaker identifies this dashboard as a governance and compliance monitoring tool that aggregates information on deviations, change controls, and training records to assess the ‘state of controls’. • Key Takeaway: This dashboard provides a consolidated, customizable view of compliance data, enabling professionals to quickly understand their risk posture and prioritize actions before external audits, rather than manually collecting data from separate systems.

some of these are paywalled

• Context: The discussion centered on the capabilities of ‘Radar’, a regulatory intelligence tool. The main point was clarifying how its data aggregation differentiates it from a simple search engine. • Key Insight: The tool’s primary value comes from accessing and analyzing information from data sources that are not open-source and require subscriptions. • Key Insight: By accessing source data behind paywalls, the tool provides higher quality and more relevant intelligence than is achievable through public searches alone.

AI-Powered Regulatory Intelligence: The RADAR Tool for Proactive Compliance

The speaker presents the user interface of ‘RADAR’, a regulatory intelligence tool that automatically pulls and scores regulatory alerts from sources like the FDA and EMA to proactively identify necessary updates to a company’s internal procedures. • Visual Context: The screen displays the ‘RADAR - Regulatory Analysis Summary’ interface, which lists regulatory alerts with details on their source, relevance score (e.g., 97%), and severity. • Discussion Insight: The speaker identifies this tool as RADAR, an AI-powered ‘regulatory intelligence agent’ that monitors global health authorities (FDA, EMA, MHRA) for updates, warning letters, and industry trends. • Key Takeaway: RADAR automates the review of external regulatory changes and suggests specific, redlined updates to a company’s internal SOPs, shifting the focus of quality professionals from manual data collection to strategic decision-making.

Workflow for AI-Powered Governance and Compliance Monitoring Dashboard

The speaker details the ‘governance and compliance monitoring’ use case, explaining how an AI-powered dashboarding tool aggregates data from various sources to provide a unified compliance overview, directly narrating the seven-step process illustrated on the slide. • Visual Context: The slide presents a seven-step workflow for ‘Continuous Governance and Compliance Monitoring’, showing how data is aggregated, analyzed, and presented on a dashboard for user review and action. • Discussion Insight: The speaker confirms this workflow is a use case where a dashboarding tool aggregates disparate information (deviations, CAPAs, training records) to assess the ‘state of controls’ and prepare for audits. • Key Takeaway: The primary function of this system is to consolidate data from multiple siloed systems into a single, customizable dashboard, enabling users to quickly assess risk, prioritize actions, and improve audit readiness.

Speaker Details AI-Assisted Compliance Assessment Workflow

The speaker explicitly walks through the third use case of their presentation, detailing the exact AI-assisted periodic compliance assessment process that is visually represented by the flowchart on the screen. • Visual Context: The slide shows a flowchart for ‘AI-Assisted Periodic Compliance Assessment’, outlining a process involving two AI agents, data inputs (SOPs, Policies), and two user approval checkpoints. • Discussion Insight: The speaker explains that this system accelerates periodic reviews by having an ‘Assessment Strategy Agent’ create a checklist from SOPs and a ‘Compliance Review Agent’ evaluate data against it to find gaps. • Key Takeaway: The system is designed to automate the evidence-gathering and initial analysis phase of compliance reviews, allowing human experts to focus on decision-making regarding the AI-flagged discrepancies.

AI-Driven Periodic Compliance Review and Reporting

The speaker presents a slide showing a sample AI-generated periodic review checklist and report, which serves as a visual example of the automated compliance assessment process being described. • Visual Context: The slide displays a ‘Sample Periodic Review Checklist’ and a ‘Sample Periodic Review Report’, showing specific checks for system validation and backup management, along with detailed evidence for a completed review item. • Discussion Insight: Speaker 1 explains that an AI agent automates periodic reviews by generating a checklist from SOPs, executing an assessment strategy, and creating a detailed report with discrepancies, aiming to reduce manual review time. • Key Takeaway: The AI system accelerates the compliance process by transforming lengthy manual reviews (e.g., of validation packages) into a fast, automated task that provides professionals with actionable reports for analysis and decision-making.

Demonstration of an AI-Generated Audit Trail Review (ATR) Report

The presented slide displays a sample AI-generated Audit Trail Report, which visually exemplifies the system’s ability to automatically flag non-compliant audit entries as detailed in the speaker’s explanation of the automated review process. • Visual Context: The image shows a ‘Sample AI-ATR Report’ that lists flagged records, such as ‘Invalid modification reason’ and incomplete user training, demonstrating the AI system’s output. • Discussion Insight: The speaker explains that this AI tool reviews audit trails against company SOPs, creating a report that flags non-compliant entries and makes them traceable back to a specific checklist item and SOP rule. • Key Takeaway: The AI-powered audit trail review automates the flagging of discrepancies, such as inadequate modification reasons or missing user training, allowing quality professionals to focus on analysis and decision-making rather than manual review.

Workflow for AI-Assisted 21 CFR Part 11 Compliance Assessment

The speaker details the functionality of the AI-Assisted 21 CFR Part 11 Assessment tool, whose workflow is visually represented on the slide, explaining how it uses input documents to check procedural controls against a predefined checklist and generate a compliance report. • Visual Context: The slide displays a flowchart for an ‘AI-Assisted 21 CFR Part 11 Assessment,’ showing how a user’s input documents are processed by a ‘CFR Helper Tool’ and ‘CFR Assistant’ to produce compliance feedback. • Discussion Insight: The speaker confirms this tool helps assess procedural controls (not technical ones) by running documents like SOPs and user manuals against a predefined checklist based on official regulations. • Key Takeaway: The primary goal of this tool is to identify gaps where procedural controls required by 21 CFR Part 11 are missing from an organization’s SOPs, enabling them to update documentation for better compliance.

Automating 21 CFR Part 11 Compliance Checks with AI

The speaker uses an on-screen example of a ‘21 CFR Part 11 Assessment Checklist’ to illustrate the manual compliance tasks that their AI-powered tools are designed to automate, such as generating and executing checks to accelerate regulatory review. • Visual Context: The image displays a sample ‘21 CFR Part 11 Assessment Checklist’, detailing specific compliance requirements and questions regarding system validation and user access controls. • Discussion Insight: The speaker explains how their AI-assisted periodic compliance assessment tool ingests an organization’s SOPs to automatically create and execute checklists just like the one shown, identifying discrepancies between procedures and actual data. • Key Takeaway: The primary goal of these AI tools is to significantly reduce the manual effort spent on reviewing documents and audit trails, allowing quality professionals to focus on analyzing data and making decisions rather than on the time-consuming review process itself.

Transcript

[00:00] Speaker 1: We see the end department, internal audits, and lab compliance. I have been working as a first assistant for ten years. I’ve been doing compliance and qualification. We have a QC department as well, and we do instrument qualification and lab compliance for QC and manufacturing labs. When it comes to internal audits, the three questions that usually comes to my mind. Whenever we are supporting internal audits, we say: Do our procedures and system meet current requirements? Are our process operating in a state of control? And are we missing any risk that the regulators and any of the authorities looking elsewhere that we are missing? Right. So these are the three questions that we usually. These are the top three questions that we usually look for when we are doing an internal audit. Most of my discussion today is going to revolve around these three questions and how we are using some of the tools we build along with our customers in helping them support the internal audits and the documentation. So when we are into the internal audits and reviewing things, we do review SOPs. We obviously review the evaluation package. We perform periodic review. We perform audit trail review, and then obviously we have this governance matrix. We check the deviations, CAPAs, training compliance, change management. What are the different changes that are still pending? And the aging of the carpus and the training records. We also look for outside the organization. What are the FDA warnings that are currently present? That is that may be relevant to my in my company organization. What’s the industry trends and what are the regulatory expectation expectations that are currently put in place? So it’s not always about. SOPs, procedures, work instructions that are currently available because we have enormous number. Every organization has a lot of SOPs. They write more procedures, but it’s the volume, the volume of data that needs to be reviewed by humans, and the volume of effort that is being spent in these activities. So how we are leveraging AI? There are three things that we are currently focusing on, and that’s the three things that we wanted to show today. The first is the FDA monitoring and regulatory intelligence tool. So essentially, what this tool does is we keep up with the FDA warning letters and what are the evolving expectations, not just safety, right? MHRA and EMA. The second tool is the governance dashboard. So essentially, this is. Like a dashboard where you can actually see your aging covers, trainings, how many people have compliant trainings, non-compliant trainings, as well as the periodic review and audit trail review. So, how do we do the documentation review using these tools? So, the first use case that I wanted to talk today is about radar. So you can think of radar as a regulatory intelligence agent, right? So essentially, it’s it’s trying to read and understand the FDA warning letters, 483s that are being published, any guidance documents recently that is updated, the draft documents, EMA and MHRA updates, and industry news trends. I just wanted to clarify that this is not. And glorified Google search is not going to just go and search in the open source. Many of these data sources are a paywall, so it’s it’s behind a paywall, and we need to go into the actual source and then get this information. So the system is going to essentially look for these websites and then it’s going to pull the relevant information, and it is going to review whichever is relevant for your article. Say for example, if you want to add a new marker into the search criteria, which would be for your drug product. Say for example, if you are into a antibiotic manufacturing, so you can add that as a marker. So it’s going to search for all the core editors, but it’s going to also focus on whether that has something to do with your organization’s title, and it’s going to create a summary of all the articles that are relevant for your organization. And you can configure this marker, basically these search criteria based on your organization. What we did is we also pushed ourselves little further. We wanted to see if the information that this regulatory agent provides can it be used to verify if your organizational procedures and SOPs has any impact because some of the other companies or organizations went through this. For every and they are going through this observation. Will that be any change that you will have to implement in your in your organization? So we have integrated the CDMs where the agent is going to review your SOPs, policies, work instructions, and is going to give you an idea of which SOPs are impacted and which of the SOPs that needs to be changed because someone else is. Along with this, interestingly, also have introduced an option where the system is going to redline an SOP copy based on the existing copy, and it’s going to keep the track changes on, and it’s also going to keep a comment saying, “Hey, this is this needs to be changed. This particular line has to be changed because there is a new 483 that needs to be accommodated,” and also it gives you a clear snapshot of. All the information that has been pulled in, we run this a schedule, so you can run this a daily schedule, a weekly schedule, and then it will give the results based on where the information is being. We wanted to quickly show you how the screen looks like. So, essentially, after we run the schedule, it’s going to pull in some of the articles that was published based on the the location in which you want, the source from which you want this information. Right, so it could be from the FDA system, or it could be from your reform system for India. It could be MHM for European Union, MPMAs, and it’s going to give you the article, and it’s also going to give you the score, which is going to be the relevance. So, how relevant is this for your argumentation based on the criteria and the markers that we have? So, if you look, so I’m going to also go and show you how this article is going to. So once you click on the link, it’s going to give you what that article was about and why the system thinks that this article is relevant. This where we are trying to show in our this EU1X for 22 guidance and exactly call out the relevance of this article and why it is relevant to me because we said these are the markers to which I need to read this. So this why I feel like you have this the article that. So once the article is being read and you confirm that there is relevance to this article, the system is going to propose some changes to the SOP. So because the system already has already a SOPs and all the policies that are in place, it’s going to give us a suggestion on what the proposed changes and why it is going to be needed. Based on the guidance change, the trend change in the industry, and it’s going to give you a reason why it needs to change. You can either accept or reject this based on your observation, your reading. So once you are okay with this change, it will update the SOP as a draft redline copy, and you can download this, and then you can either accept the changes or then further you can reject the changes. That’s the It’s completely the human and the reviewer’s responsibility to make sure that we don’t change. So the reason why we wanted to do this is essentially to support any new changes that are coming in because the industry is changing. Now the previous speakers have spoken about how quickly the industry is adapting to the CAA, and if there are new changes, new trends that are being published. The system has to be able to accommodate you. The system is going to make use of AI and it’s going to get the information fast, so that we can work on the decision-making process and updating the quality policies, rather than spending more time reviewing these articles manually. So the second use case, as I was talking about, is the governance and the complaints monitoring. So essentially, how many deviations are going to be due in a recent time? What’s the number of reports that are pending? Are going to be overdue? Number of change controls that are in place and any training records. So essentially, this dashboarding tool is going to aggregate all the information. From your system, and it’s going to show you where you currently stand. So, in the beginning of this session, I was talking about second question, which is how control, how many controls that I have, what the state of controls my systems are operating. So, this this particular dashboard is going to give us an idea of where we stand. So that whenever we are preparing for an external audit, so we do an internal audit and quickly see. Let things stand. We can go through this dashboard. We can look at various, and we can always work on things, prioritize rather than going and looking at every single system. Because planning management system is going to be separate from your PMS system, and you need to go and collect data, and it’s going to be different people working on it. Right. So this this is sample screen. Should have found the dashboard will look like. And this customizable, so you can add more information around this, where you can add like the aging, carpal, the trainings, whatever would help to get a quick snapshot at any given point when we are looking at the data to prepare for the external audits. So this going to be a useful government dashboard that we can use to gather information. The third use case. Is DA assisted periodic compliance assessment. So obviously we have all done audit trail reviews and periodic reviews, right? So this is essentially going to help us do the periodic review in a faster and more accurate manner. So the logic is very similar to what I showed or what I presented a minute ago. That is going to be information that we need to provide. The system is going to be a rule-based system. Essentially, it’s going to read your SOPs, your work instruction. Let’s say your checklist that you will use to perform a periodic review or an audit review. It’s going to create a checklist. Essentially, it’s going to be either your SOP. These are the things that you have asked for me, and it will ask the end user if you are okay with it. So we say yes. This is the checklist that I want you to use to perform this compliance. There could be error of this. Assessment report, and then it will go one step further. It create the strategy, so the system is going to explain to the end user, and this where again the transparency and the explainability part is being used. So, system is going to give a little bit more information. Not for every single checklist that I identify using your SOPs. This exactly what I’m going to do. So, it’s going to say. Say for example, it says audit trail. Another is logging items shouldn’t be more than five times or ten times made, and you shouldn’t do this. So it’s when you say, okay, whenever I’m going to review this particular checklist, I’m going to go through the entire audit trail. I’m going to check how many unauthorized items are made. I’m also going to check if there’s a time duration between which people are trying to. Then is it going to be the same person doing this multiple times, or is it going to Different people, so it is going to flag all different combinations of gaps that were identified. And if you can imagine this periodic review standpoint, this is the same logic. It’s when they do periodic review. So please say, okay, whenever I do the periodic review, look at the validation package. Is the validation package intact? Did the software version hasn’t changed since the validation has been performed? So the validation is still. Still in the intact state when it was released, right? So it is going to create a checklist for the validation package. Question: It’s also going to create a strategy. It will say, okay, these are the input documents that you will provide to me, and I’m going to review whether this validation package is still intact for the version that was released, right? So once it creates the strategy, it’s going to now actually perform the assessment. So the agent is going to review all the input documents that we share. And it’s going to give us the deviation or the discrepancy report, essentially to say this what your SOP said, this what I saw in the actual data, and this the gap that I have. Right. So beyond that point, it’s going to be the users. Users are going to take over the information, and they are going to process whether it is a high risk or a low risk deviation or a discrepancy that the system identified, and it’s going to give us. And then you can take it further. Like in option investing, you can go and create deviations, capers, or if you think this a low risk, you don’t have to act on it. Then you can ignore it. So this a sample risk checklist. The system created based on the SOE robust framework. So again, this the management must first check point, and then there is the backup for the market risk management. We have about six to seven different modules in which the PDA requires typically perform. So validation is one of them. Batch of the modules is the second. So you can also do this. Like you, you don’t have to do this six modules every time. I say you are doing it every six months. You might have to do all six, but if you are going to reduce the intensity and the frequency, you can just. The validation factors are just the backup every month, so that you know that the system backup is still working each month, and it is retrievable. So, essentially, it’s going to create this checklist, and once it completes the assessment, it’s going to create a review report. As I know, the screen is too small, but it’s it’s essentially saying based on all the documents that we have shared, I reviewed and I have. I mentioned that everything is intact, but it’s going to give us a reason why it thinks it is not intact because some things have changed. Maybe the version of the software has changed, but then the validation document is still not updated to reflect. Right? So this is how the periodic review checklist is done. Right? Then you can either download this as a complete report and then take it to the downstream, or you can create this report in your own organization’s file. So that we can directly take the QMS other documents. This is how the audit trail report is presented. Like, so it’s very similar to the data retrieval. It’s going to create the checklist and the strategy. It’s going to perform the audit trail report. So, the top is how the system will show you the flags. So it’s essentially saying these are the audit entries that I observed in the audit trail. Which doesn’t align with your SOP, and it has been flagged because of so and so reason. And it’s completely traceable to the checklist ID against which this particular entry was flagged. Right? And the checklist ID is going to be traceable to the SOP entry that we provided, so that we can exactly say which checklist item and which SOP was used to identify and flag this entry. At the bottom of this the export of this report, you can either export this as a an Excel or a PDF. Now, why we are doing this? The essential reason is the amount of time that we spend in repeated review audit review. I’ve done this myself several years. If if we are going to do an audit review or a periodic review every six months, and we are looking at 100 and 200 pages system audit review and functional audit review. Takes our service to something. The system essentially reduces that portion of the effort. It will give us the report in less than five minutes, ten minutes. It’s going to give us the data, and then we take it further, further process it, and then understand what we need to do about it, and give some either act on it if it’s a high risk, medium risk, or if it’s a low risk. You don’t act on it, and maybe possibly update the SOP for the low risk. So essentially, we are trying to reduce the manual effort and make the quality professionals to spend more time in analyzing the data and taking insights, rather than making them spend their quality time in reviewing every line of the origin. Same thing with the periodic report. Where you go through the validation package, you run through this complete checklist. Instead, the system gives you the report, and then you just take it, process it. And consuming, so the so the last tool that we are using currently to help us with the document review is the twenty-first for portal assessment. Again, very similar simple logic that we are used for all the tools. So currently, this tool is going to help us with the procedural controls of the portal and complaints, not the technical controls part. Because whenever there is a joint event internal audit, then we go and run into documents to say, hey, have we made sure that all the controls that Part 11, New 1x11 has asked us is that part of the SOP? Because technical controls we make sure that during the system administration process everything is covered because we have a discussion, but sometimes the technical controls, which the procedural controls, which needs to be put in the SOP. Might not be added, or maybe the revision of the SOP gets updated, and someone did not add the information. So the system is going to get the URS specifications, user manuals as an input. It’s going to run the predefined Indian Safety Protocol and checklist that we have, and it’s going to give us a result of what are the different procedural controls that are in place. Are not in place, so that we can update the SOPs to add the system. So that’s essentially what this tool is going to do, and this how it works. Right. So it creates the checklist based on the predefined ones for Part 11 that is in the FDS website, and it creates this checklist. We review this checklist. We say yes, this what I want you to do today, and the compliance questions, as I mentioned, is something that the system does. To essentially give us a transparent, explainable part of the AI, so we know exactly what the tool is going to do. We don’t have to worry whether the system is going to hallucinate, is going to give me something that I did not intend. Rather, these are the questions that the system is going to ask. And if you think, hey, I need you to ask more questions, you have the option to edit, add questions in the simple English native language, and the system is going to ask those. It’s not going to ask anything other than these three questions per checklist that is being followed. So once the system runs through your documents, it’s going to give us a sample report. So every single item in the part elements, like the regulations that are mentioned, is going to give us a report whether it’s compliant as yes, no. And it’s also going to give us a remark on why it thinks it is compliant, or if it is non-compliant. It’s going to say this is the reason why I think it is non-compliant because the SOP probably doesn’t cover this explicitly. So you have to make sure that these systems, these procedures are updated based on the regulations that are put in place. So to summarize, to conclude. What we are trying to do here is, we are trying to get the information faster. We are not trying to replace the process. We don’t want to. We don’t want to end up being a situation where the the auditor is here tomorrow and we are still trying to put together some information. So, all the use cases and the case studies that I present today are are aiming plus for us to accelerate this. Day-to-day process so that we can get the information faster and make sure that the humans, the QA professionals review the instances for every single use case that we’re paying for. So that is going to be an overall review and overall process for every single use case, and we have to make sure that the the report that is being generated are in compliance, like any other manual process that we are doing. When we create a peer review, we send it to the quality unit and evaluation team. They review this and will approve it. They will either add comments, and then it goes through the approval process. It’s the same thing, except for someone spending eight hours, twenty-four hours doing an evaluation peer review. Instead, the system does that for us, and then we process the information and take it forward to be reviewed and approved. So that’s the end. Aim and goal of this entire.

[24:36] Speaker 2: Exercise. Thank you. Quick question: What systems did you enable audit trail review for?

[24:49] Speaker 1: We implement this for the majority of the instruments and SaaS applications. For example, enterprise applications like Empower, and then there is also NEAP system, FormsNet. We did this tool with almost every instrument, lab instrument that we can have, standalone instruments. We did it with Cromillan, we did it with OpenLab, we did it with Ubi. We still work in that basis. Anything that has an RDB structure data, we are able to do this. So most of this tool. Both periodic review or any of these tools is vendor agnostic, so you can use this for any application. These instruments could be a SAS application as long as it provides structured data.

[25:41] Speaker 2: Another interesting thing is that in the clinical space, a lot of the audit trail is very complex and not straightforward. I was curious.

[25:59] Speaker 1: So we did. Interestingly, we did this a couple of alarm systems, and we had some unstructured data. Like they had thousand, two thousand rows of audit trail to review. It still provided the audit trail report as long as you have an SOP which is very clear and what your human person is doing today. The agent can replicate the information. It’s just that if it’s taking two minutes, it will take some more time so that it. Process to the two hundred or two thousand records. Okay. So, what type of file types are compatible with this system? Structured data, right? Yes. So, almost all data that we can think of, we also so it it also is going to add additional layer if it is not structured, or if it is just an image, or if you are going to scan a paper copy. We will have additional layers to process that information and create that structured data. So, it’s it’s never impossible. It’s all possible. It’s just that we might add. If it is an Excel, PDF format, it’s just easily consumable. But if you say like you are going to have a scan copy, then we will have to add more layers to process that information and then get it into a structured data. But any any JSON file. XML, PDF, text files, everything is.

[27:28] Speaker 2: Read.

[27:48] Speaker 1: Yes. I just repeat your questions for the developers. He was asking about whether these systems or the use cases can integrate with other applications directly and consume the information rather than it being manually uploaded. The answer is yes. We have practiced this with empower cases. As long as the application, as an API or an SDK, works with other applications, we should be able to get it and get the information. The entire use case was built initially for a system which cannot be put in the field, and the only way the data was data can be downloaded is through a USB pipe. So four five years ago, we created this tool for that purpose. Like, and then again, it made it easier for us to deliver this. But then it was originally created when COVID hit, and we had to go perform our experiments because the schedules were not stopping because there was. So we had to work with some automation to perform the RFP. But then, interestingly, now because of A, we are able to do it faster and integrate any system that has an APA connection. Thank you.

Graph View

Backlinks