KENX AI in GxP East 2026

2026-09-16-1521-ai-protection-by-design-gxp-lecture

55 min read

Protection by design lecture on quality and security co-design, part 1

Recorded 2026-09-16 at 1521 via Plaud. 33:06.

Summary

Part 1 of Aarti (Amy) Arora’s protection by design lecture (Principal, RxT; she advises pharma clients on tooling), recorded until the device cut out mid-talk; the talk concludes in part 2. Arora’s arc: R&D IT delivery into a CIO-forced takeover of global compliance in 2016, learning SOX and cyber controls as a self-described GXP nerd, and discovering that quality and security are asking the same questions in different languages. The core claim: a cyber event can be a quality event, and a security failure that changes a validated outcome is a quality failure.

Through that lens she walks the AI-expanded control surface: agent privilege misuse, vendor access, prompt changes outside change control, data tampering, undetected model changes, training drift, and audit-log gaps, with excess permissions named as the biggest current problem. She contrasts the handoff operating model (quality, security, and IT each throwing controls over the wall while executives bonus AI use at 30 percent of the job) with co-design, including her practice of a protected weekly two-hour session where every control owner vets projects early; Eric asked from the floor about org design and the time constraint. She closes part 1 with the translation table (MFA means only the right person approves a GxP action; RBAC means only qualified users perform regulated activities), her 50-control EU AI Act matrix mapped against NIST and SOX, a batch-record review walkthrough where AI holds no signing or release rights and every action is checked rather than just the prompt, the deviation-investigation risk of polished but wrong narratives downplaying human error to avoid CAPAs, and the 200-roles Trackwise story. The recording ends mid-discussion of the roles matrix.

Plaud AI notes

<App-generated notes from the Plaud cloud, verbatim.>

Aarti (Amy) Arora of RxT presented a framework for combining quality, cybersecurity, and IT controls as organizations adopt AI in regulated work. Her central point was that AI expands the control surface beyond the validated system to the full path from information to action. Quality and security should therefore co-design controls early rather than conduct separate reviews after a solution has been built.

Career shift revealed common ground between quality and cybersecurity

Arora began in R&D IT delivery. She worked on CTMS, regulatory information management, safety, and other platforms that had to comply with GXP. She learned compliance so she could make better delivery decisions without repeatedly asking quality for permission.

In 2016, her CIO directed her to take over global compliance. Her experience was then concentrated in GXP, but the new role also covered SOX and cyber controls. She initially disliked the move and continued to favor the field she knew best. Over time, cybersecurity became a passion.

That experience shaped the presentation:

  • RxT’s role: RxT helps large and small pharmaceutical clients determine which tools fit their needs. Arora said the same AI tool can work well for one company and poorly for another.
  • Control-framework reality: She found that a common control framework was not as common as its name implied. Different functions often used distinct terms and methods for controls that served similar purposes.
  • Cross-functional lesson: Quality and cybersecurity often ask the same questions in different language. The answers may drive different actions, but both fields seek testable assurance.
  • Learning across disciplines: Arora adopted risk-and-control matrices and control-effectiveness methods from her SOX work. She now applies those methods to quality instead of relying only on SOPs and periodic reviews.
  • Shared responsibility: Patient safety, data integrity, and cybersecurity may each have a department that owns policy or expertise. Operational responsibility still belongs to everyone who handles the process or data.

Arora stressed that a cyber event can also be a quality event, and a quality event can become a cyber event. If a security failure changes a validated outcome, quality must be involved. In her earlier experience, cybersecurity performed reconnaissance while quality opened a deviation. Each function worked alone, then quality struggled to obtain the security findings needed for its investigation.

AI expands risk across identities, data, models, and monitoring

Traditional validation focused on a defined system and its intended use. Security supplied access, detection, and protection controls around that system. AI changes the scope only somewhat, Arora said, but it changes where controls must be placed. Teams must protect the entire route from source information to an automated or human action.

The main risks span several domains:

  • Identity and access: Organizations are creating agents rapidly. Some builders assume an agent does not need quality or cybersecurity review. This creates privilege misuse, identity mismatches, and excessive access.
  • Vendor access: Vendors form an extended network that needs tighter management. Their presence increases the number of identities, tools, and connections within the control environment.
  • Prompt governance: Prompt changes can alter behavior and should remain under change control when they affect validated use.
  • Data integrity: Inputs can be tampered with deliberately or changed unintentionally. Good intentions do not establish lineage or prove that data remained intact.
  • Model change: A model may change without the organization detecting it. External changes, including changes made by OpenAI, can alter behavior and require a controlled response.
  • Training drift: Drift occurs over time, while many organizations still lack mature ways to control it.
  • Visibility: Audit-log gaps and weak monitoring limit oversight. AI increases transaction volume, data load, and integration exposure.
  • Continuous operation: Historic controls often tested a system at one point in time. AI agents can run 24/7, so verification and monitoring must become continuous.

Arora considered excess permissions one of the most serious current problems. Identity and access management was already difficult when she worked on a SailPoint implementation 10 years earlier. The larger number of agents, vendors, and connections has made the issue harder.

Bad source data illustrates why security can change the GXP state. A bad actor could inject poor clinical data. If an AI system ingests it, later decisions will also be bad and may create reputational harm. Similar harm can follow from undetected model changes or improperly granted permissions.

Co-design should replace serial governance handoffs

Arora described the common operating model as a chain of disconnected handoffs. Quality provides GXP controls, security provides a larger set of NIST-based requirements, and IT adds technical controls. The end user receives overlapping demands separately because the control owners report through different organizations.

At the same time, executives press employees to innovate. Arora has heard top-down goals requiring AI use for 30% of a person’s job, with metrics and bonuses tied to that target. People respond to incentives by finding more uses for AI. Legal, privacy, security, quality, and IT then approach those projects with separate approval requirements. The result feels like a barrier to innovation.

Arora proposed a co-design model with defined responsibilities:

  • Business: Define and own the intended use.
  • Quality: Assure the regulated use and establish the quality boundary.
  • Security: Protect assets, patients, identities, and data.
  • IT or the vendor: Own technical controls and change control.
  • All control owners: Design and test a shared solution rather than passing it from one review to another.

Eric Ma noted that organizational design contributes to the problem. He also cited complaints that each party lacks enough time to work together. Arora agreed that time is a constraint. Yet separate reviews force the end user to visit each control owner and collect approvals like marks on a bingo card.

Her suggested model was to protect a recurring block, such as two hours every Thursday, when all control owners sit together. Project teams could bring ideas for an early, informal control conversation. The session would resemble a pre-ARD but function as a working discussion rather than a review board. Participants would assess intended use, fit-for-purpose controls, and risks before commitments harden.

This model also requires a behavioral change. Control owners cannot claim to enable innovation while handing a project 99 quality controls and another 99 security controls, then making approval slow and difficult. The organization’s shared goal is to bring products to patients faster and more safely.

Arora has experienced both models. She described the difference between throwing work over the wall and co-creating around one shared design record as dramatic. The best time to shape the control environment is before a statement of work has been signed and the solution has been built. Pulling quality in late leaves it little choice but to impose changes that delay the project.

The right partner also depends on organizational culture. A relaxed advisor may work well in one company, while a stricter approach may suit another. Arora recommended choosing partners whose style fits the organization.

One control design can serve several compliance languages

Quality and security often describe equivalent safeguards with different terms. Translating those terms improves collaboration:

  • Least privilege and segregation of duties: Both restrict what a user may do. A reviewer who cannot approve an item is constrained by role and authority.
  • MFA: Security says multifactor authentication is required. Quality describes the outcome as ensuring that only the right person can approve a GXP action.
  • RBAC: Role-based access control means only qualified users can perform regulated activities.
  • Audit trails: Quality is not alone in needing them. Security can aggregate audit trails across systems and analyze them with advanced technology.
  • E-signatures and approvals: Both disciplines need reliable identity, authorization, and evidence.
  • AI change governance: Both need controlled versions and testable changes across prompts, models, tools, and connections.

Arora built a 50-control matrix for the EU AI Act. She mapped it against the FDA’s seven-step AI material, NIST, and some SOX requirements. Not every control applies to every use case, but the matrix reduces duplicate work. She contrasted it with an earlier reference to Steve’s framework, whose stated count she remembered as either 120 or 190 controls.

The aim is one testable design. Quality asks how to validate it. Security asks how to protect it. A shared framework lets each function use the resulting evidence for its own purpose.

Batch-record review shows how controls must follow the workflow

Arora used a GXP batch-review scenario to show what protection by design requires. The approved intended use was for AI to read approved batch records, draft exceptions with source references, and send the output to a qualified human expert. The human must have actual batch-record expertise. Merely placing an unspecified person “in the loop” is not enough.

The intended workflow preserves traceability:

  1. AI reads authorized batch records.
  2. It drafts exceptions from approved evidence.
  3. It cites the supporting sources.
  4. A qualified human reviews the result.

That design can fail if a team treats the AI as a simple business enabler and bypasses security or quality.

  • Source boundaries: The agent should read only task-approved sources within the user’s permissions. Draft, obsolete, or irrelevant batch records must be blocked because they can produce false exceptions.
  • Read-only enforcement: Saying a source is read-only does not make it so. The permission must prevent modification.
  • Change authority: Only authorized roles should change models, system prompts, connections, or related settings. Uncontrolled changes can alter the data or outcome.
  • Rounding and manipulation: A person or tool could round a number until it falls within acceptable limits and no longer appears out of specification. The change may be intentional or unintentional.
  • Version control: Changes require controlled versions and independent approval.
  • Human approval: A named, qualified person should authenticate, preferably through MFA, and approve within the batch system. AI should have no signing or release rights.
  • Service identity: Every AI agent should be recorded in the CMDB with a defined identity, purpose, and permitted actions.
  • Credential limits: Credentials should cover only the task at hand. Tool and API permissions must prevent the agent from editing, closing, or releasing batches.
  • Transaction controls: The organization should check every action in the batch system, not merely inspect the prompt.
  • Continuous monitoring: Controls must watch the running service rather than rely on point-in-time testing.

Arora summarized the division of labor as: quality defines the boundary, cybersecurity and IT enforce it, and the functions test it together.

Deviation investigations need protected evidence and expert review

AI can also support deviation investigations and trend analysis, but polished language can conceal wrong content. Arora said Claude is especially capable of producing text that looks technically correct even when the underlying content is wrong.

In a deviation narrative, AI could downplay human error, reflect bias, or omit critical facts. The result might sound authoritative while avoiding a costly CAPA. The control response has three parts:

  • Human expertise: A qualified reviewer checks the AI’s account against the evidence.
  • Quality controls: Independent approval and regulated process controls govern the result.
  • Cybersecurity controls: Original evidence remains protected from alteration.

If the source evidence has changed, human review cannot recover a trustworthy account. Arora’s broader warning was that wider AI adoption can also multiply AI-related failures unless these three safeguards expand with it.

Final takeaway: test trust across the full control surface

AI has enlarged the control surface to include models, prompts, data, tools, and outputs. All of them must remain controlled. Validation can no longer stop at testing whether a system performs its intended function. Teams must also test whether identities, permissions, evidence, and changes are trustworthy.

Arora said common security terminology made collaboration easier once she learned it. Using terms such as MFA, RBAC, and control surface helped her communicate with security, legal, privacy, finance, and SOX specialists without changing the underlying quality objective.

She closed with an example of excessive complexity. In an earlier system, she recalled more than 200 roles in TruPlus. The role matrix became so complicated that the team did not know how to test it. Cybersecurity partners then analyzed the design and helped simplify it. The example reinforced her main conclusion: quality and security become more effective when they treat each other as design partners rather than separate approval gates.

Highlights (button-flagged moments)

Aarti Arora Introduces ‘Protection by Design’ Framework

The speaker, Aarti Arora, introduces the core concept of her presentation, visually represented on the slide as ‘Protection by Design,’ which advocates for integrating quality and security to build trusted GxP systems. • Visual Context: The slide shows the presentation title ‘Protection by Design,’ with a diagram illustrating the convergence of ‘Quality’ and ‘Security’ into ‘Trusted Systems’ for GxP environments. • Discussion Insight: Aarti Arora provides personal context, explaining her career evolved from an IT role focused solely on GXP to understanding and integrating cybersecurity controls, which is the foundation of her talk. • Key Takeaway: The central theme is that ‘Protection by Design’ is achieved by treating quality and security not as separate silos, but as interconnected partners from the start to ensure robust GxP systems.

cybersecurity can be part of building gxp systems, primarily building trust

• Context: The speaker discussed the realization that cybersecurity and GXP quality controls share underlying principles. This insight stems from bridging the gap between the two traditionally siloed departments to foster better collaboration and build more trustworthy systems. • Key Insight: A security failure that alters a validated outcome is fundamentally a quality failure, linking the two domains directly. • Key Insight: Cybersecurity’s ‘Role-Based Access Controls’ and Quality’s principle that ‘only qualified users can perform regulated activities’ are functionally the same. • Decision: Adopt a ‘Protection by Design’ model where business, quality, security, and IT collaborate from the project’s inception. • Risk Identified: The introduction of AI expands the ‘control surface’ beyond the system to include models and prompts, making integrated controls even more critical.

Cybersecurity and Quality: A Unified Responsibility

The speaker uses a slide posing questions about the ownership of patient safety, data integrity, and cybersecurity to argue that these are not siloed duties but shared responsibilities, culminating in the critical insight that a cybersecurity failure can fundamentally be a quality failure. • Visual Context: The slide presents three questions—‘Who owns Patient Safety?’, ‘Who owns Data Integrity?’, and ‘Who owns Cybersecurity?’—under the main heading ‘Cybersecurity is already a Quality issue’. • Discussion Insight: Speaker 1 explains that while specific departments (like PV, Quality, or Cybersecurity) are often seen as the primary owners, the training and reality dictate that everyone shares responsibility for these critical areas. • Key Takeaway: A security failure that alters a validated outcome is, by definition, also a quality failure, meaning a cyber event and a quality event can be one and the same.

Presenter Outlines Talk on AI Risk and Integrated Security Controls

The speaker, Aarti (Amy) Arora, introduces the agenda for her presentation by verbally outlining the key topics displayed on the ‘What We Will Cover’ slide, which include understanding AI risk, building early protections, and co-designing controls. • Visual Context: The presentation slide lists the session’s agenda, including AI risk in GXP, early protection, collaborative control design, and a ‘SHARE’ framework. • Discussion Insight: The speaker confirms she will cover real-life GXP use cases, designing and testing controls together, and a ‘homework exercise’ called the SHARE framework, directly aligning with the on-screen points. • Key Takeaway: The presentation’s core theme is integrating cybersecurity and quality controls from the start (‘Protection by Design’), moving away from siloed operations toward a collaborative co-design model.

Shift from Siloed ‘Handoffs’ to Collaborative ‘Co-Design’ for Unified Control Implementation

The speaker uses the presentation slide to visually contrast the inefficient ‘handoff’ model—where Quality, Security, and IT operate in silos—with the superior ‘co-design’ model, which she advocates for as a collaborative approach involving all stakeholders from the beginning. • Visual Context: The slide depicts two models: a linear ‘Handoff’ model where Quality, Security, and IT pass requirements over ‘the wall’, and a circular ‘Co-Design’ model where Business, Quality, Security, and IT/Vendor collaborate around a ‘One Shared Design Record’. • Discussion Insight: The speaker argues that the ‘Handoff’ approach creates significant hurdles for innovation, forcing project leaders to go ‘door to door’ for approvals from different compliance groups. • Key Takeaway: The proposed solution is a ‘Co-Design’ framework where all control owners collaborate early and concurrently, treating partners like Security as design partners, not just gatekeepers, to create a single, testable design.

we should do co-design

• Decision: Adopt a ‘co-design’ model where business, Quality, Security, and IT collaborate on projects from the beginning. • Key Insight: The current siloed, sequential approval process is a major barrier to innovation and efficiency, creating an adversarial relationship between business and control functions. • Risk Identified: Organizational design and a perceived lack of time are potential barriers to adopting a more collaborative co-design model. • Action Item: Establish a recurring, collaborative session for all control owners to review new projects in their early stages. • Action Item: Shift the mindset of control teams from being gatekeepers to being enabling design partners who foster innovation.

Integrating Quality and Security Early in the Lifecycle

The speaker uses a presentation slide illustrating an ideal project lifecycle to argue for a ‘co-design’ approach where Quality and Security are integrated from the very beginning, rather than operating as separate, late-stage gates. • Visual Context: The slide shows a lifecycle process starting with ‘Business need’ and argues that security must be included from the start, as discovering a control gap late is the ‘most expensive’ time. • Discussion Insight: The speaker contrasts the inefficient ‘hand-off’ model, where teams go to separate governance bodies (Quality, Security, Legal), with a collaborative ‘co-design’ model where all control owners work together with the business from the start. • Key Takeaway: The core message is that treating security as a ‘design partner’ alongside quality from the initial stages—a concept termed ‘Protection by Design’—is crucial for enabling innovation without creating bottlenecks.

Unifying Quality and Security Controls Through Shared Language

The speaker uses the slide to illustrate her realization that seemingly separate ‘Quality’ and ‘Security’ disciplines are fundamentally asking the same questions, and that by translating their respective terminologies, they can achieve a unified, testable design. • Visual Context: The slide proposes a ‘Shared Design’ where Quality (‘How do we validate?’) and Security (‘How do we protect?’) converge on common controls like ‘Least privilege’ and ‘Identity + authentication’. • Discussion Insight: The speaker explains that she initially struggled to see the overlap, for example, not realizing that security teams care as much about ‘audit trails’ as quality teams do. • Key Takeaway: The goal is to create one testable design by understanding that different teams often have the same underlying requirements, just expressed in different professional languages (e.g., ‘MFA’ vs. ‘the right person can approve’).

Speaker Outlines AI Workflow for Batch Record Review and Associated Security Risks

The speaker uses a slide illustrating an ideal three-step AI workflow for reviewing batch records—Read, Summarize, and Human Review—to explain the critical need for integrating cybersecurity controls to prevent potential failures at each stage. • Visual Context: The slide displays an approved AI workflow where the system reads batch records, drafts exceptions with source references, and a qualified human performs the final review for accuracy. • Discussion Insight: The speaker presents this workflow as a GxP use case, detailing what could go wrong if security controls are bypassed, such as the AI accessing incorrect data or its model being altered without authorization. • Key Takeaway: The intended AI process relies on a ‘human in the lead’ for final decisions, but its integrity depends on embedding security and quality controls by design to protect against data tampering and unauthorized changes throughout the workflow.

Applying Access Controls to an AI-Powered Batch Record Review Workflow

Speaker Aarti (Amy) Arora presents a slide illustrating a GxP use case where AI assists in reviewing batch records, emphasizing that strict access controls and human oversight are essential to protect the validated process. • Visual Context: The slide details a 3-step workflow: AI reads batch records, AI drafts exceptions, and a qualified human performs the final review, with specific controls for read, change, and approval permissions. • Discussion Insight: The speaker highlights the risks of bypassing security, such as an AI agent accessing obsolete data or making unauthorized changes, and stresses that AI should have no signing or release authority. • Key Takeaway: The core principle is that while AI can draft and summarize, the final decision and approval must be made by a named, qualified human, with cybersecurity controls enforcing these boundaries by design.

I heard CMDB. I guess it is an industry wide concept

• Context: The discussion focuses on implementing security by design for AI agents in regulated GXP environments. The speaker outlines critical controls for a workflow where an AI reads and summarizes batch records. • Risk Identified: Unidentified DIY AI agents can become ‘ghost’ entities, making them difficult to govern, audit, and control within the system. • Decision: Every AI agent’s service identity must be logged in a CMDB to track its purpose and authorized permissions. • Key Insight: Credentials and API permissions for AI agents must be strictly limited to the task at hand, preventing unauthorized actions like editing or releasing batches. • Key Insight: Security checks must be applied to every action an AI agent takes within a system, not just the initial prompt.

Applying Security-by-Design to an AI Batch Record Review Use Case

The speaker walks through a slide illustrating a GXP use case—using AI to read batch records and draft exceptions—to explain how specific access controls at each step are critical for preventing security failures that could compromise the validated outcome. • Visual Context: The slide diagrams a six-step process for AI access control in batch record review, specifying permissions for reading, changing, approvals, AI identity, tool access (denying release rights), and continuous monitoring. • Discussion Insight: The speaker emphasizes that without these designed controls, a user creating a ‘DIY’ AI agent could inadvertently cause issues, such as the AI accessing incorrect data or a user altering records without authorization. • Key Takeaway: To safely implement AI in regulated GxP processes, Quality must define the boundaries, Cybersecurity and IT must enforce them with integrated controls, and all parties must test the system together.

Speaker Elaborates on AI Misconduct Risks in GXP Scenarios with Visual Aid

The speaker uses the displayed slide to illustrate specific, real-world GxP scenarios where AI could enable misconduct, highlighting the ‘Deviation investigation’ case as an example of how AI can generate a technically plausible but factually incorrect narrative. • Visual Context: The slide presents a table detailing four GxP scenarios, explaining how AI could facilitate misconduct in each and how cybersecurity measures can protect the process. • Discussion Insight: The speaker explicitly references the ‘Deviation investigation’ scenario, explaining that AI (like Claude) can create a narrative that reads correctly but downplays critical facts like human error to avoid costly CAPAs. • Key Takeaway: The slide serves as a practical, take-home reference that grounds the speaker’s broader discussion on the convergence of Quality, IT, and Cybersecurity controls, showing how AI risks manifest in specific GxP workflows and how to mitigate them.

I really like this talk, it’s superb concrete.

• Key Insight: A security failure is a quality failure; the two disciplines must be integrated rather than treated as separate functions. • Key Insight: AI adoption collapses traditional risk boundaries, expanding the ‘control surface’ with new challenges like prompt manipulation, data tampering, and excessive AI agent permissions. • Risk Identified: Incentivizing AI adoption without an integrated control framework can lead to risky, uncontrolled implementations by teams. • Action Item: Establish cross-functional meetings for all control owners (Quality, Security, Privacy) to co-design a unified set of controls for new projects. • Action Item: Map different control frameworks (e.g., GXP, NIST) to create a single, unified set of testable controls to avoid redundant work.

Speaker Analyzes Real-World GxP Scenarios Where AI Poses Risks and Cybersecurity Provides Protection

While displaying a slide detailing how AI could enable misconduct in GxP scenarios like handling SOPs and safety data, the speaker explains these real-life risks and emphasizes how corresponding cybersecurity controls are essential for protection. • Visual Context: The slide presents a table of ‘Real-life use cases’ illustrating how AI could enable misconduct in GxP scenarios and how cybersecurity measures can protect processes like SOPs, safety data, and audits. • Discussion Insight: The speaker confirms she is discussing the slide’s content, noting how AI, like Claude, can generate technically correct but factually wrong narratives, potentially downplaying human error in deviation investigations. • Key Takeaway: The speaker highlights a critical vulnerability where AI could manipulate information to make safety signals appear less significant or alter narratives to avoid costly corrective actions (CAPAs), reinforcing the need for robust cybersecurity protections.

Translating Cybersecurity Controls into Shared GxP Outcomes

The speaker presents a slide that translates technical security terms into understandable GxP outcomes, explaining that this reframing was crucial for bridging the communication gap between Quality and Cybersecurity teams. • Visual Context: The slide maps cybersecurity jargon to practical goals, such as ‘We require MFA’ translating to ‘Only the right person can approve a GxP action,’ and ‘We require RBAC’ to ‘Only qualified users perform regulated activities.’ • Discussion Insight: The speaker explains this translation was a personal breakthrough, realizing that Quality and Cybersecurity were often asking for the same thing using different language, which initially caused confusion and friction. • Key Takeaway: Adopting a shared language by translating technical controls into common goals is a powerful strategy to foster collaboration between departments like Quality and Cybersecurity, moving them from siloed operations to a co-design partnership.

shared outcomes are where cybersecurity and quality can meet and translate between each other

• Key Insight: A single event, such as a security failure that changes a validated outcome, can be classified as both a cybersecurity and a quality incident. • Key Insight: Translating terminology is key; ‘Role-Based Access Control’ (RBAC) in security is functionally equivalent to ensuring ‘Only qualified users can perform regulated activities’ in quality. • Decision: Adopt a shared language and co-design controls with both security and quality teams upfront to create a single, efficient, testable design. • Risk Identified: Operating in separate silos creates hurdles for innovation and leads to redundant work, as teams unknowingly build separate controls for shared goals.

Unifying Quality and Security Controls to ‘Test Trust’ in AI Systems

The speaker culminates her presentation with a slide that visually encapsulates her core argument: Computer Software Assurance (CSA) must evolve to ‘Test Trust’ by integrating cybersecurity principles like access control and integrity monitoring directly into the GXP validation process. • Visual Context: The slide outlines a framework for testing trust as part of a system’s intended use, focusing on four pillars: Function, Access, Integrity, and Assurance, culminating in the idea that ‘Go-live is the start of continuous assurance’. • Discussion Insight: The speaker explains that quality (GXP) and cybersecurity teams often ask the same fundamental questions using different terminology (e.g., ‘qualified users’ vs. ‘role-based access controls’), advocating for a unified ‘co-design’ approach to avoid silos and streamline innovation. • Key Takeaway: The concept of a ‘control surface’ has expanded with AI to include models, prompts, and data, requiring a shift from point-in-time system validation to continuous, testable assurance of trust and integrity.

Transcript

[00:00] Speaker 1: So we’ll dive right in. I’ll start off. I think it’s important for me to share just a little bit about my career trajectory before we dive in, because I will be sharing it through the lens of that. So my name is Aarti (Amy) Arora. I’m with RxT. What we typically do is we work with clients, large pharma, small, just to help them figure out what tools they need. As we heard, everyone’s trying to adopt AI, and no one knows how or what. And The same tool won’t necessarily work with one company that would work phenomenally with another. So I kind of love this forum. You get to see everything that’s out there, and then you can start recommending the tools that work. So my background, which we’re going to get into for the next 20 minutes or so, it’s not going to be a long, it’s a long presentation. It’s really here to start generating conversation. But my my career started in IT as delivery. So R and D I T delivering platforms like CTMS, regulatory information management systems, safety platforms, you name it on the R and D side, and delivering, delivering, and having to comply to GSP. So for me, I then said, okay, let me learn what GSP is all about. That way, I can do this a little bit better rather than having to always go to a mother may I and like ask quality, like, can I do this? Can I do that? Because the video is not going on the virtual side, so just. Switch it to this real quick. Yeah, I can take this off. Can you guys hear me? Okay. No, you’re good. Down. All right. Right on. So as I went to my journey, all IT learning compliance. So then I, in 2016, I was full. I was full time my whole career up until two years ago. In like 2016, I was approached by the CIO at the company I was working with, and he asked me to take over the compliance organization. As well, that’s like a really neat offer. I appreciate it. Let me think about it. And he said, “Well, that’s cute that you think it’s like a choice. It’s actually you have to. You have to take over the global compliance. And all I knew at that time was GXP. So then I had to take on SOX, cyber controls, and everything. And I have to tell you, I was really, really scared. And I actually hated it. Like when you understand something and you’re good at it, like that’s what you want to do all the time.” And even though now you’re accountable for all these things, you kind of lean on what you know and what you’re good at, and you overindex on it. And I guess that’s part one of my confessions: is like I’m a GXP nerd through and through, and I’ve only had to learn other controls. And we talked to Steve earlier, where he talked about a common control framework. And what I learned is like that’s not so common. So the rest of this is really through a lens of therapy. So thank you guys for being my therapist. I’m just going to share what I learned when I had to understand the other controls. And then I’m always trying to compare them to GXP, GXP. So for the rest of this, we’re just going to talk a little bit about quality controls and cyber controls. And we heard Dory say earlier, like, if you guys are really worried about AI, okay, cool, but you should be way more worried about cybersecurity. That’s your biggest risk. So this is really going to go through what are some of those cyber risks that exist? How can you build in cyber controls by design? But then also, If you guys are GXP nerds like I am, how can you make that real for yourself and say, “Okay, cool, I get it. That’s what it was in GXP. This is how we can partner well with security.” And what I learned at the end of the day, we’re really not that different. It was a really long journey, a really hard journey to understand that. And actually, my CSO at the time, he was going to come present this with me. He’s actually from Raleigh, and he’s like, “What, what a great like full circle moment to to say he forced me into it and.” Choking and screaming. I did not want to do it, and then it became a passion point. So, the rest of this, I just wanted to share that background, so you know the lens. It’s a little bit of therapy and a lot of bit of conversation. So, what I do know to be true is I’m probably not telling you anything you don’t already know. So, this is really here to generate conversation. Jump in anytime. There’s only about six slides, so it’s not going to take up the entire time. So if we think about protection by design, we talk about quality by design. What does that mean? So I’m gonna just layer in some tidbits again, things you probably already know. If not, just framed a little bit differently so you can think about it the way you’ve always thought about quality. And I can’t tell you, I never wanted to talk to the cyber side of the world. I never wanted to. It made me feel really dumb, actually. They use acronyms I didn’t know, and then it just kind of makes you. It puts you on your heels a little bit. So. Spent time learning that as well, and I’ll kind of go through some acronyms here as well. So the three questions that always come up in like any risk assessment and everything, or generally in life, who owns patient safety? Everyone can go ahead and say, “Let go.” That’s the PV department owns it. Your patient safety department owns it. You have a whole thing around it, and then every training tells you everybody owns patient safety. Everybody has to do your adverse event reporting. We all own it. Then you get into data integrity. Same question always comes up: Who owns data integrity? And you say, “Oh, your quality department, because they own the policy on it, they own the SOP on it.” And then the training comes out and it’s like, “Well, everybody owns data integrity. You have a role in it. Quality owns the guidance on it, but everyone owns data integrity.” And then it comes to who owns cybersecurity, and the same question, the same answer: It’s cybersecurity. They own it. And then the training comes out and they’ll tell you, “No, actually, you own it also.” You have to make sure you don’t treat your data poorly. You engage with cybersecurity to get the controls. You own all these things, and ultimately, what took me many years and probably like a couple too many drinks to understand is that a cyber event is a quality event. It can be, I should say, not always, but it certainly can be. And vice versa, a quality event can become a cyber event. So the more and more we can speak the same language, but Here to say, if a security failure can change the validated outcome, it is also a security failure. I mean, a quality failure. So the security team will go off and run there, like their recon, try to figure everything out, and then quality will be like, “Wait, it’s a deviation. You have to work with me to figure out this deviation.” And that was always, at least what I saw early on, is no one ever wanted to have go on that journey alone. Quality would do theirs in a silo. Cybersecurity would do theirs in a silo, and then you’re like, “All right, cyber, what did you learn? Tell me. I need to put in my deviation. I need to put in my deviation.” And they never wanted to. So this is really a story of like all those worlds coming together, and understanding, and helping your cyber organization understand. Hey, a cyber event can be a quality event. How do we partner? And like I said, feel free to chime in at any time if you also want this to be your therapy session. It’s not just about me. But again, I’m going to share where I kind of had a hard time really understanding things, and the full circle moment that I’m here to share today. So, what we’ll cover in the next 20 minutes or so is a couple of real life GXP situations. We can talk in academics and talk about like what is. There’s a few real life ones that hopefully you’ll see yourself in it. We, I’ll hit on some clinical examples, some manufacturing examples. Hopefully, you’ll see yourself in it. This is funny. I love being one of the last speakers because you can just tie in what everyone else has already said. And something I heard, I think, in every single session, is how do you kind of step left? How do you do things upstream so you don’t like see the gaps downstream because they come way more expensive. And then Steve talked about this, and we heard about it in I think six of the sessions I took notes of. How do we design and test? Controls together because they really are the same, and like I didn’t get it when I was doing SOPS testing, and I was looking at like the control effectiveness. I’m like, wait, I never did it that way in GXP. I never had a risk and controls matrix. I always just had SOPs, and now I can share like I have a risk and controls matrix for my quality stuff too. I took that learning from SOPS. Applied it to my quality life and do control effectiveness a little bit differently, which is just like periodic review, but you do it control driven now so that you can share things back and forth. And then because I like to think I’m funny, the last one is a share framework. It’s a evidence to help you go back to the office like like a homework exercise. I am a nerd at the end of the day, so we will be assigning homework or a parting gift, however you want to look at it. So what we know, this an AI conference. This could also just apply in generally, but like AI specifically has collapsed the old boundaries of risk. When we used to look at risk around systems, and the next slide will cover this like way clearer, is okay. The system has this, and this where you do things at this point, and you validate your system. Here, as you think through from an identity and access perspective, with AI and all the agents people are creating, every there’s agents everywhere. People don’t think they need to go through cyber controls. Oh, just create an agent. That’s cool. It’ll do what it needs to do. I don’t need to go through quality. I don’t need to go through cyber. It’s just an agent. That that kind of does create a lot of cross-cutting issues with privilege misuse, and we’ll go through a use case, a real-world use case of this. Identity mismatch, vendor access. This we know. Vendors are everywhere. I mean, heck. But vendors are everywhere, so how do we make sure that we manage that like extended network a little bit better? Then, from a quality perspective, and candidly, the tech folks care about this too. But as we talk about the validated state, prompt changes. How do we make sure prompt changes are under change control? How do we make sure there’s no tampering into your data? Like, sure, people could say there’s things that most people don’t do things like with mal intent, and I believe that to be true. Things do like just happen, but that you can’t tell that to the end. I don’t know my data. Like I can’t, I can’t explain the lineage, but like I think everyone’s trying to do the right thing. It’s it’s that how do we show with AI way more, way more probability of data tampering, and then the training drift that just happens. It’s a true story. Not everyone’s like into how we can control that today. I mean, over the last two days, we’ve gotten really good tips and tricks. And like guidance on how to do this, and ultimately the visibility and oversight, which is shared across all controls, audit log gaps, weak monitoring, everything that we used to do historically would be point in time. Everything’s going continuous now. So that data overload and that integration exposure just becomes way more obvious now. It always existed, but with AI, we’re just doing more volume. When you do more volume, probability of failure kind of goes up. So as I mentioned, AI does change what we have to protect. Before, like I said, typical system, I’m gonna validate the intended use. I talk to my security partner. They’ve given your controls for okay. You need to put in these access controls. You need to put in these detection, these protection controls. We follow the whole Ness framework. Now with AI, the conversation’s changing a little bit. Not a lot, bit though, just a little bit on where can we inject. Certain things now, from a cyber perspective, where you’re not just validating the system, but the path the information from information to action. So, if you look at your data sources, you could have tampered inputs. You can have bad data again. Either you inject it, there are bad actors out there. They could inject you with bad data, so that you do get some reputational risk later on. They’ll give you some really bad clinical data, and if you’re ingesting it, that’s. Yeah, and you’re making your decisions on bad data. You’re making bad decisions. So there are some cyber controls, and I’ll go over a lot of controls in a minute that can help with tampered inputs. Prompting. I won’t exhaust this because you guys have been here as much as I have the last couple of days. Like we know there’s prompts, how we prompt, how we engineer those prompts. Your modeling, you can have uncontrolled changes in your model that go undetected. So if you don’t build in that security by design, your model can change. Heard from Steve, you know what happens when the under when OpenAI changes their entire model. The model changes. Are you like ready to change with it in a controlled manner? And then the one that I think is probably, and this is just a personal opinion. If you see it differently, I’d love to hear. Is the excess permissions that are that are being granted right now? It’s like I said, agents, vendors. That identity and access management is because it was an issue, like with SailPoint. I remember trying to implement SailPoint ten years ago; it was a major issue then. It’s like a bigger issue now because the number is bigger. So ultimately, a cyber event can change the outcome, and ultimately can change the the GXP state. This we talked about the tongue, so again, it’s more of a recap of what we heard. This gets really back into what Steve from GSK talked about a lot. So what you see on your left hand side is the world that I see often. And let me know if you live a different life than I live, because I’d love to hear about it. But what happens now is we do these handoffs, and we do it all the time because we report to different people, and no one’s really talking up front. And quality requires GXP controls. There’s an SOP that says, “Hey, you have to have these controls.” And then security requires a ton of security controls, not just access controls. NIST has a ton of controls. I think Steve’s number was he was up to 120 or 190. I forget which it was. And then IT requires controls. While there is an overlap, they’re giving them to you as an end user separately. Meanwhile, your goal that your CIO is put on you, or your VP in the business function is put on you, is you have to innovate. You have to use AI. I’ve heard stats as much as from the top down from the C-suite, you will use AI for thirty percent of your job, and they get metriced on that and they get bonused on that. So my mom always told me, you know, want to know how someone behaves, just watch how they get paid. Show me how they get paid. They’re going to behave accordingly. So if you say I’m going to give you a bonus if you use AI thirty percent of the time, you’re going to find every kind of way to get AI going. So here we are, the people who are like, okay, I’m going to get my thirty percent AI. I want to do all my goals. I’m going to get it did. And then you have all these like compliance people coming in to say, put my controls in. No, put my controls in. And it just becomes this big hurdle on innovation. Like how could you possibly innovate when everyone’s trying to slow you down? How many governancees do I have to go through? Legal, privacy, security, quality. There’s a lot. There’s a lot going on. So if we get away from that handoff and throwing things over the wall, more towards that co-design, which is okay. The business, you need to care about intended use. We know that to be true. Quality. We need to assure that. We know that to be true. Security. You need to make sure you’re protecting our assets and protecting our patients. By protecting our data, and then the IT vendor owns change control, or IT or the vendor, they own all the technical controls. So why not talk about this together, rather than just throwing it over the wall? And it’s like really easy to say, oh yeah, the one on the right—that’s totally how we should operate. How many people actually operate? I’d love to. Like, does anyone actually operate in this world where all your control owners hang out with the business and IT and like? Try to smooth out those bumps in the road.

[16:39] Eric Ma: But also because we’re org design is part of it, but also we’ve heard complaints that there’s not enough time on each party to hang out.

[16:57] Speaker 1: We’re doing it separately.

[17:00] Eric Ma: That’s true.

[17:01] Speaker 1: End user, user centric. I’m the one that is trying to build it. And now I’m Girl Scouting. I used to say this all the time. I have to go door to door like a Girl Scout, saying, “Can you help me make sure I have my controls? Can you help me because I won’t get signed off my ARV unless everybody I talk to signed off like the bingo card and said, ‘Yeah, Arthy went through her control design with me and we’re all set.’ So yeah, time is an issue, but if you protect that time and say, ‘Okay, guys, every Thursday for two hours, all your control owners are going to be in the same room.‘” Bring your projects there and we’ll vet them like a pre-ARD, but not like a review board, a hangout, a chat through controls. What’s fit for purpose? What makes sense? What are you thinking about? Super, super early. But that does require like behavioral changes. I can’t be a cop. I can’t be like, ah, dude, you can’t do that. Sorry, like, ah, that’s GXP, high risk. Let me give you these 99 controls. P.S. That’s, Facing the world, we need these 99 security. Like, we’re talking through both sides of our mouth here. Yeah, I’m gonna enable you. I wanna make sure you’re controlled, but P.S. It’s gonna take you know, a very long time to bake all these controls. And I’m not gonna make it easy for you, but you won’t get approved unless I say you can do it. puts us all in a really bad spot. It’s like, definitely not what I believe to be true. That that’s what. Organization wants. They want us to do better. They want us to get product to patient faster, safer. That’s a common goal, but we’re not behaving that way. So that one shared design record and that hanging out mentality—it’s—it’s hard to build, but when you do build it, it’s special. It’s really, really cool, and I’ve lived it. And I—I can’t tell you the night and day between the throw over the wall and the hangout to co-create. So this is. I’ve heard this so many times, so it’s really about the quality nerds in the group. Will always say quality by design, pull us in early. We want to do it. Same deal though. Security also feels the same way. So if you build that partnership, and someone comes to me as quality first, but I’m like, hey, did you talk to security and privacy? If they say no, I’m like, hold up, let’s just pull them in. We’ll talk about all this at the same time. Versus. What we people at Quality is, oh, he pulled this in. You already signed the SOW. What do you want me to do about it now? Now you’re going to hate me because I’m going to slow you down. So same deal, just become partners with, and quality by design, security by design becomes protection by design. So the best time to shape that control environment is super super early, and the art of it is not being that cop. It’s like an art. It’s like a science and an art, and it’s. I think that’s why it’s really cool that you have a lot of partners here. You got to go with a personality that matches. You can have someone that’s like a little bit, chilled out that fits the vibe, of your of your org, or you can have someone that’s like really militant about it that fits the vibe of your org. So that’s what I’ll always say: is there are partners abound. Everyone wants to work with you. You just got to pick that culture, that vibes with you and vibes with your organization. And treat security as a design partner, like you do quality. This where I had a really hard time. This my this my therapy session. This where I kick back on the couch, talk a little bit about how hard it was for me to understand. We were asking for the same thing, and I know like, we can all say, “Oh yeah, everyone knows that,” but like, do you? I didn’t. So that’s my. I did not quite understand. That when I said least privilege, and segregation of duties, that meant the same thing in some cases. We were saying different words. You have segregation duties. You can only do this. You can only do that. A reviewer can’t approve. But then look, oh, that’s like least privilege in some ways too. Identity and authentication. Believe it or not, our security team cares a crap ton about audit trails. I didn’t think they did. I thought I was the only person that cared. About audit trails, but then when I saw using like advanced technologies, and this is where quality got to learn from security, they would just pull together all audit trails and then find a way to aggregate them and then spit out something that made sense. I was like, oh wow, that’ll actually like totally help with my audit trail. So learning from each other, and the other thing, as I mentioned in my SaaS world, having a very strict Risk and controls matrix because that’s what like your PwC or EYs audit you on. Taking that now, I apply that across the board. So we heard Steve’s 120. I have a 50 control one that I use just for the EU AI Act. So say okay, EU AI Act, the FDA kind of came up with these seven step things for AI. I took that and I mapped it against Nest, and I mapped it against like some SOX requirements. So that 50 one just for AI is what I use. They’re not all applicable, but it makes me not do things twice. Like, I’m kind of lazy. I don’t like doing things twice. Like, give it to me once, I’ll do it once. E-signatures and approvals, and then AI change governance. So it’s a different expertise, but one testable design is the idea. It’s like the utopia I feel. So quality will always say, how do we validate? Security will always say, how do we protect? You might do different things with the answers you get, but I can tell you the questions you’re asking are the same. All right, so here’s a real life scenario that happened somewhere, where you come up with your intent. Every time, especially with AI, he gets come up with a bunch of low risk things that AI can do, so you can hit that thirty percent, get your bonus. Come up with some use cases, and then we’re gonna we’re gonna do this. We’re gonna POC, we heard POC help. We’re gonna pilot it. We’re gonna do it. And so this again, not telling you anything you don’t know, but with a use case, how can you build security in by design for a GXP use case? So you have an approved approved intended use. You want to read approved batch records, summarize the exceptions with sources, and then a human that is super qualified. I think I heard. Human in the lead earlier, not just human in the loop, but someone who like has actual expertise reviews the output. Not me. I’m not reviewing batch records. That’s not my expertise. Someone who has that expertise. So this is the workflow that you come up with. The AI reads the batch records. It complete complete the task approved evidence. Then AI drafts the exceptions. puts all the exceptions. You get your sources. So you have all that traceability. And then the human reduces it. That’s how the world should work. That’s what we wrote in our requirements. That’s what we tell everyone we’re doing. So what could go wrong? What if you just bypass security and say, “Well, look, humans in the loop. I don’t need to engage with security. I don’t need to engage with quality. I’m gonna just do this because it’s gonna be like a business enabler.” I hear that every day. That’s just enabling me. Don’t worry. You don’t have to like engage. So the first thing, and again, this is a summary now of everything we heard over. It’s not I didn’t design it this way, but I love the fact that I’m almost last. Gets to summarize. The first thing is the AI reads the batch records. Okay, but like which batch records, and who? Like there’s still permissions that need to be associated with that agent with the whatever you’re writing. So only task-approved sources within those user permissions. So here, you could look at things like draft batch records, things that have now been obsolete. So just like any anything else, like a human would know not to do that. But if you’re sending out an agent, you’ll put those boundaries on it. It’s going to look at like older batch records and come up with exceptions actually that aren’t true. So you want to block the sources that are not relevant for the boundaries that you’re setting, and you also want to make sure there’s no modification of the sources. And we have seen that where you just say, oh, bullets read only, but you didn’t set that permission. Sounds like super simple, but the person who’s now doing like no code agents because they can do it on their own, they’re not thinking that way. They’re like, ah, look, what can I do? Why does it take you guys so long? And building on their own, they don’t put those boundaries in there. The second area here is they changing permissions. Only change, only change roles, edit models, models. System prompts or connections. So here you can again, if you’re just sitting at your desk one day and you just like watched people on TV or YouTube create their own agents and you think, wow, I can do it too, and you don’t put all of these controls in here, it could change the data. Especially in it, again, it won’t be malintent or malintent. Oh, I’m just gonna round up that number. I didn’t. It’s cool. I’m just gonna take that number and. Round it up to everything, and then it gets within your acceptable limits, and it’s no longer out of spec. So the versions and the independent approval come in here from the change permissions, approval permissions. You have that name qualified human and multi-factor authentication, hopefully, and approves the batch system. So AI has no signing or release rights. Again, if they don’t build that in by design. Sounds again like super super like straightforward, but again, people created these agents. They just kind of I can do it too situation a DIY. I don’t know about you, but when I do DIY at home, it does not show like the ones I see on YouTube. The AI service identity. This is a big one. So if you’re not logging every AI is in like your CMDB or every AI agent, I should say in your CMDB, what identity do they have? What can they do? What are they allowed to do? So limit credentials to just this task at hand. Here again, making sure that the tool and the API permissions are correct. That they can’t edit, close, or release any of these batches. Check every action in the batch system, not only the prompt. And then this is the thing that we’ve heard at nauseum, and hopefully, like everyone’s cool with it, is that verify and monitor. Monitoring becomes a continuous thing, not a point in time anymore. This thing is continuously running, doing whatever you need to do. You need like our ability to monitor things more continuously becomes critical as there’s more and more AI being introduced, and those are running 24/7. So quality defines the boundary. Cyber and IT force it, and together we test it. These next two slides. Totally went. These are real life use cases. I’m not going to dream these slides. These are really for you to take home. But it gives you what’s the GXP scenario. So that’s the quality piece of your life. How AI could enable misconduct, so intentional or unintentional, AI could really mess up some of these use cases that you have, and then how cyber security protects them. So I’m just going to pick on a few. And feel free if you’ve seen any nightmares in your life or anything like that. Feel free to bring it up. We went through the batch release, well, batch review. Deviation investigation is such a popular one, especially since the FDA is doing some of this as well in the trend analysis. But for GXP scenarios, how can AI enable misconduct? So if you’re doing an investigation, you’re just letting AI like write your deviation for you. It could, and I think I hear a lot of folks say this. Claude is especially good at making things look very right. The technical language is correct, but the content’s wrong. And the narrative, in this case, if there’s like a deviation and there’s some human error that gets in here, it could downplay that. It could have a bias and downplay like what the actual facts are, and then come up with a really good-looking summary, though. It reads really technical, but it missed out on some critical issues. And it avoids costly capers. So here again, cybersecurity protects the original evidence. So the human in the loop is going to need to go back and review it. If the original evidence was altered, that’s not happening. And then also requires independent approval. There’s, like I said, a few examples here of where AI could change the world a little bit. Your world, not the world, but your world, a little bit, and how both. Human, actually, I should say three things. Human in the loop, super critical. Quality controls, super critical, and cybersecurity controls will really help prevent as we proliferate AI, proliferating the issues that we see that come out of them. So this is again, it sounds super simple. When I look back, I wonder why I struggled with life so much. Why it was so hard for me to get out of my GXP life and use cybersecurity terms is if you ask my CSO at the time, he will tell you our one-on-ones were like literally him telling me I have to go get my CISSP, like saying get out of your GXP life. He even threatened to take it away from me at one point, so I focused solely on cyber so that I could get better at it. Luckily, that never happened. Security control languages we require MFA. Okay, cool, cool. What do we say? Only the right person can approve a GXP action. So very simple terminology change is like, okay, now I get it. We require RBAC, role-based access controls. Okay, scary, scary acronym. Until you really like break it down and say, only qualified users can perform regulated activities. And it was the other way around. If I would tell them that, hey guys, only qualified users can do their job, they’d be like, ah, quality’s so picky. And I’m like, but are back? Oh, okay, okay. So now let’s do this together. It’s like when you use their terms with them, that conversation goes like way, way better. So I’ve become like a walking human thesaurus to how to have those conversations if you’re having them legal, with privacy, with security, finance. This sox sox terminology kind of blew my mind. I’m not gonna lie. So AI extends the control surface though as well. So control surface used to be a lot smaller. That control surface is now a lot bigger, and I’m starting to use terms like control surface. I never would have said that 10 years ago, ever. But the models, the prompts, the data, the tools, the outputs—they all have to be under control. Is the ultimate take-home, and everyone likes to talk about CSA, CSP. How do these worlds come together? Rather than just testing the system, it does—you have to test trust as well. Which, again, role-based access. But this is where I would say I was in a, trapwise system back in the day, and before cybersecurity had like business facing folks, I think we had over two hundred roles in TruPlus, and then like that, the whole matrix became so complicated we didn’t even know how to test it. So then cybersecurity partners came in to like help. Let me help you simplify, what you’re doing, and they actually like came in and like did an analysis.

Graph View

Backlinks