Summary of Article
- A framework for AI governance is a well-organized method used to pinpoint, measure, and oversee risk throughout the entire AI lifecycle, from its creation to its deployment and monitoring.
- The AI Risk Management Framework from NIST, which is based on four key functions (Govern, Map, Measure, Manage), is the most popular starting point for organizations currently establishing AI governance.
- The majority of AI risk does not appear at launch, but rather several months later when the product is in use. This is why continuous monitoring is the most commonly missed aspect of governance implementation.
- The AI Act from the EU introduces mandatory requirements for high-risk AI systems that are sold or used in Europe. This means that for global organizations, alignment with regulations is no longer optional.
- Most governance programs do not fail due to bad intentions, but rather because responsibility is divided among committees instead of being given to specific individuals. This guide addresses this fixable structural issue.
Attempting to establish AI governance without a framework is like trying to install a smoke detector after a fire has already started. By the time you realize there is a problem, the damage has already been done.
Companies implementing AI in a large-scale manner are confronted with an increasing number of risks: model drift, data leakage, regulatory exposure, and ethical failures that may emerge months after a system has been activated. There is a significant difference between “we have an AI ethics policy” and “we have a functioning governance program,” and most companies are still somewhere in between these two extremes. Palo Alto Networks offers a wealth of cybersecurity expertise that directly intersects with AI governance, making it a valuable reference for understanding where AI risk management and security controls intersect.
Main Points
This guide provides a comprehensive understanding of not only what frameworks are available, but also how to apply them to actual systems, where governance programs usually fail, and what distinguishes organizations that manage AI risk from those that simply document it.
Without a Framework, AI Governance Is Just for Show
It’s possible to write policies, create ethics boards, and perform the occasional red-team exercise. But without a repeatable structure that links risk identification to accountability and action, all of these efforts simply produce documentation rather than protection. The difference between putting on a show of governance and actual risk management is operationalization – whether or not your framework actually influences decision-making.
The Downfall of Ad-Hoc Governance
Ad-hoc governance is a system that is doomed to fail because it’s reactive, not proactive. Instead of constantly being on the lookout for potential risks, teams wait for incidents to occur, regulations to change, or the public to apply pressure. Each response is isolated, managed by whoever noticed the issue, and rarely results in the accumulation of institutional knowledge that can be applied to future situations.
The real issue is that AI systems are not static. They can change a lot over time. A model that works well when it is first launched can become much less effective over the course of a few months. This can happen if the data that is being inputted changes, if the way users behave changes, or if the data pipelines that are upstream are changed. If you are using an ad-hoc approach, there is no way for you to catch this. But if you are using a framework, you can.
The Three Most Important Frameworks in Real World Application
While there are many AI governance frameworks available, only three have gained significant traction across various industries and jurisdictions. Knowing the differences between these three is the first step in choosing the correct foundation for your program. For developers, understanding the best machine learning frameworks can also be crucial in the implementation of AI governance.
In January 2023, the NIST AI Risk Management Framework (AI RMF) was published. It was voluntary and originated in the US, but it quickly gained the most attention from industries worldwide. This framework is based on the lifecycle of the AI system, from its design to its deployment and ongoing operation. Instead of a checklist, the four core functions — Govern, Map, Measure, Manage — provide organizations with a practical operating structure. In July 2024, a Generative AI Profile was added to the framework.
ISO/IEC 23894 is an international standard specifically designed for managing risks associated with AI. It serves as a companion to the larger family of ISO/IEC AI standards and is the preferred framework for organizations seeking internationally recognized certification or operating in several regulatory jurisdictions.
- NIST AI RMF — Voluntary, US-origin, lifecycle-based, widest adoption across industry sectors
- ISO/IEC 23894 — International standard, certification-compatible, aligned with ISO/IEC AI standards family
- EU AI Act — Mandatory regulation for high-risk AI systems in European markets, with Article 9 risk management obligations that carry legal force
What Separates Real Risk Management From Paperwork
Real risk management produces observable outcomes: flagged model behavior, paused deployments, updated controls, documented incident responses. Paperwork produces risk registers that nobody reads and policies that nobody enforces. The distinction almost always comes down to whether accountability is assigned to named individuals with authority to act, or distributed across committees with authority to deliberate.
The Main Risks Brought About By AI Systems
AI systems bring about a unique type of risk that traditional IT risk frameworks were not designed to manage. The risk area is broader, the failure modes are more nuanced, and the effects can escalate more rapidly than usual software bugs. A working AI governance framework must consider all four major risk categories, not just the ones your security team is currently monitoring.
Interference with Model and Unauthorized Entry
Interference with the model includes malicious attacks aimed at altering the outputs of the model, unauthorized access to the weights or training pipelines of the model, and compromises in the supply chain that introduce vulnerabilities before a model ever reaches production. These are not theoretical attack vectors – they are active areas of exploitation that sit at the intersection of AI risk and cybersecurity. Access controls, model versioning, and integrity verification are fundamental controls here.
Unintentional Data Exposure and Data Accuracy Issues
AI models that are trained on confidential data can unintentionally memorize and replicate that data in their results — a well-known issue with significant privacy and compliance concerns. Data accuracy issues, where the data used for training is incorrect, biased, or out-of-date, result in models that seem to work but consistently perform poorly with real-world data. Both of these risks necessitate data governance controls that most businesses have not yet incorporated into their AI processes.
Moral and Social Risks
Regulators, civil society organizations, and increasingly enterprise customers are focused on ethical risk categories such as bias, discrimination, lack of transparency, and erosion of human oversight. These risks are more difficult to quantify than security vulnerabilities but carry similar reputational and regulatory exposure. A governance framework requires explicit mechanisms — bias audits, explainability requirements, human-in-the-loop checkpoints — to identify and address them.
Potential Legal Trouble
The days of unregulated AI are over. The EU AI Act is in full force. There are AI-specific regulations in place for financial services, healthcare, and essential infrastructure. Companies that operate internationally have the added headache of dealing with differing regulations in different countries — a problem that can be solved by aligning with a framework.
The Four Core Functions of the NIST AI Risk Management Framework
The NIST AI RMF is the most practical starting point for organizations that are developing AI governance from scratch. Its structure is intentionally not prescriptive — it provides the operating model without restricting you to particular tools or processes — which makes it versatile across industries, model types, and organizational sizes. The framework organizes AI risk management into four core functions that work as an integrated system, not a step-by-step checklist.
Oversee: Assigning Named Responsibility
Oversee is the function that makes everything else work. It establishes the organizational structures, policies, and roles that allow risk management to actually happen. This means defining who is responsible for AI risk at the senior leadership level, what authority they have, and how AI risk decisions connect to broader enterprise risk management. Without named responsibility at this level, the other three functions become exercises in documentation rather than operational controls.
Map: Pinpointing Risk in Your AI Stack
Mapping is the process of turning broad risk categories into tangible, specific risks. It involves taking an inventory of all AI systems currently in use, understanding the context in which they are used, and identifying the risks that come with that context. For example, a recommendation engine that customers interact with will have different risks than a tool used for classifying documents internally. Mapping makes you consider these differences instead of just applying a one-size-fits-all policy.
The practical result of Map is an AI system catalog with linked risk profiles. Each entry should include the model type, training data sources, deployment environment, intended use case, affected populations, and known limitations. This catalog becomes the basis for everything that follows — you can’t measure or manage risks you haven’t identified and located.
Step Two: Test the Real Systems, Not Just the Templates
Step two is where governance programs most often produce theater instead of results. Organizations complete risk assessment templates, check boxes against generic criteria, and file the outputs. What Step Two actually requires is empirical testing of real systems against the specific risk profiles generated in Step One. That means bias evaluations on your actual training data, adversarial testing against your deployed model endpoints, performance benchmarking under distribution shift conditions, and explainability assessments for systems where transparency is a regulatory or operational requirement.
Manage: Continuous Risk Mitigation and Monitoring
The core of the framework is the Manage function, which is the operational center where the risks that have been identified and measured are prioritized, addressed, and tracked over time. This involves putting technical controls in place, such as access restrictions and model versioning. It also includes setting up human review processes for high-stakes outputs, creating escalation protocols for anomalous behavior, and having documented response plans in place for foreseeable failure modes.
The term “continuously” is a heavy lifter in this function. The management of AI is not a one-time task to be checked off after deployment. It’s a constant operational discipline. Models change. Data drifts. User behavior evolves. The Manage function requires a monitoring infrastructure that detects these changes as they happen and governance processes that address them before they become incidents.
Understanding ISO/IEC 23894 and the EU AI Act
While the NIST AI RMF provides the operational structure, ISO/IEC 23894 and the EU AI Act serve as the international and regulatory anchors. Knowing how these three frameworks work together, as well as where they differ, will guide you in creating a governance program that is both operationally effective and compliant.
The ISO/IEC 23894 was designed to address AI risk management within the larger ISO/IEC AI standards ecosystem. It is aligned with ISO 31000, which is the general risk management standard. This means that organizations that already have ISO-aligned risk management programs can integrate AI-specific risk management without having to rebuild their governance infrastructure from the ground up.
Unlike NIST or ISO, the EU AI Act is a regulation, not a set of guidelines. It categorizes AI systems based on their risk level and assigns mandatory responsibilities to each category. The strictest requirements and harshest penalties for non-compliance are applied to high-risk AI systems, which are used in areas such as critical infrastructure, education, employment, essential services, law enforcement, and border control.
- Unacceptable risk — Prohibited outright (e.g., social scoring by governments, real-time remote biometric identification in public spaces with limited exceptions)
- High risk — Permitted with mandatory risk management, data governance, transparency, and human oversight obligations under Article 9
- Limited risk — Subject to transparency obligations only (e.g., chatbots must disclose they are AI)
- Minimal risk — No mandatory obligations; voluntary codes of conduct encouraged
ISO/IEC 23894 as the International Risk Management Companion
ISO/IEC 23894 provides the methodological depth that organizations need when NIST’s flexibility becomes a liability. Where NIST deliberately avoids prescribing specific risk management methods, ISO/IEC 23894 provides detailed guidance on risk identification techniques, risk analysis approaches, risk evaluation criteria, and risk treatment options — all scoped specifically to AI systems.
For companies seeking international certification, operating under procurement requirements that specify ISO standards, or building governance programs that need to withstand third-party audits, ISO/IEC 23894 provides the documented rigor that NIST alone can’t provide. It doesn’t replace NIST — instead, it’s the layer of methodological specificity you add when your governance program needs to show process discipline, not just structural alignment.
Both frameworks are designed to work together. NIST provides the operational model (Govern, Map, Measure, Manage), while ISO/IEC 23894 provides the risk management method that is applied within that operational model. When used together, they cover both the organizational governance structure and the technical risk management process.
Framework Comparison at a Glance
Framework Type Origin Primary Use Case Mandatory? NIST AI RMF Guidance United States Lifecycle risk management, widest industry adoption No ISO/IEC 23894 Standard International Certified risk management, audit-ready programs No (unless contractually required) EU AI Act Regulation European Union Compliance for high-risk AI in EU markets Yes (for in-scope systems)
EU AI Act Article 9 Obligations for Sellers in Europe
Article 9 of the EU AI Act requires providers of high-risk AI systems to establish, implement, document, and maintain a risk management system that runs throughout the entire lifecycle of the AI system. Specifically, this means conducting risk identification and analysis for each high-risk system, implementing risk mitigation measures, performing residual risk evaluations, and establishing post-market monitoring procedures. The Article 9 obligations aren’t a one-time compliance exercise — they require ongoing operational risk management with documented evidence of continuous monitoring and periodic review. Organizations selling or deploying high-risk AI systems in European markets that don’t have a functioning lifecycle-based risk management program are already non-compliant.
Creating Your AI Risk Management Framework
Putting a governance framework into practice is where it either becomes a functioning system or simply remains a paper exercise. The five steps below show you how to turn the requirements of your framework into specific actions. They are arranged in order so that each step lays the groundwork for the next one. To better understand the infrastructure needed, consider this comparison of AI infrastructure options.
Step 1: Carry Out a Risk Assessment on Your Existing Models
Begin by taking a candid inventory of all the AI systems your organization is currently running or actively building. For each system, record the model type, training data origin, deployment setting, decision-making power (whether the outputs are advisory or automated), affected groups, and known failure modes. Then, carry out a structured risk assessment — not against a generic model, but against the specific environment each system operates in. A credit decision-making model deployed on a large scale to consumer groups presents a categorically different risk than an internal productivity tool built on a general-purpose language model. Your risk assessment should explicitly reflect this difference.
Step 2: Create Teams that Cut Across Functions for Governance
AI risk is not confined to a single function. The engineering and security teams are responsible for the technical risk (model behavior, security vulnerabilities, infrastructure failures). The legal, compliance, HR, and increasingly external stakeholders are needed to address ethical and societal risk. Regulatory risk is the responsibility of compliance and legal but they need technical context for accurate assessment. If you try to manage all of this through a single committee or a centralized AI ethics board, it will be too slow and too disconnected from the operational reality to function.
What seems to work best is a team that works across different functions with a named individual responsible for each layer. The Chief AI Officer or a designated senior risk owner is ultimately responsible. The product and engineering teams are responsible for identifying technical risks and implementing controls for their specific systems. A central governance function sets policy, manages the framework, and runs risk assessments that cut across different areas. Legal and compliance are responsible for regulatory mapping. This structure allows for expertise to be spread out without accountability being watered down.
Step 3: Implement Ongoing Checks After Launch
Monitoring after launch is often the least funded part of AI governance programs. Companies put a lot of money into testing before launch, then see a successful launch as the end of the process of managing risk. In reality, it’s the start of the riskiest time. The performance of the model gets worse as real-world data distributions move away from the conditions it was trained in. New user behaviors bring up edge cases that weren’t covered in testing before launch. Changes to the data pipeline upstream change the inputs to the model in ways that you can’t immediately see in the metrics for outputs.
For successful continuous monitoring, three elements are needed: tools that track model behavior in production in enough detail to identify drift, established limits that initiate review or intervention, and a working process that links monitoring alerts to governance responses. The monitoring infrastructure must be set up before deployment — it is much more difficult and costly to retrofit observability into AI systems that are already in production.
Step 4: Monitor Risk Throughout the Entire AI Lifecycle
AI risk management is based on the lifecycle because risk evolves at each phase of an AI system’s life. During the development phase, the main risks are the quality of the data, the choices of model architecture, and the integrity of the training process. During the testing phase, the risks become gaps in evaluation coverage and benchmark gaming. During deployment, operational risks — performance under distribution shift, security vulnerabilities, misuse — take the lead. During ongoing operation, drift, accumulated technical debt, and changing regulatory requirements introduce new risk categories that didn’t exist at launch.
Managing risk throughout the entire lifecycle requires a governance process that is explicitly activated at each lifecycle transition, not just at deployment. Risk assessments carried out during development need to be reviewed before the product is launched. Findings from production monitoring need to inform decisions about model updates. Decisions to decommission need to take into account obligations to retain data and the risk of dependency for downstream systems that depend on the model being retired.
Step 5: Make Sure Your Framework Meets Regulatory Standards
Meeting regulatory standards isn’t something you do once and then forget about — it’s a continuous process that ensures your internal governance program is in line with the external requirements that are relevant to your particular AI systems and operating jurisdictions. Start by figuring out which regulatory framework applies to your organization based on where you’re located, what sector you’re in, and how your AI systems are classified for risk under the EU AI Act. Then, compare your existing governance controls to the specific obligations each regulation has. Any gaps you find in this comparison will show you what you need to fix.
If your organization is global, you’ll need a systematic way to handle differing requirements across jurisdictions. The NIST AI RMF is the most practical base layer because of its flexibility. You can add ISO/IEC 23894 requirements and EU AI Act obligations to the NIST structure. This way, you won’t have to start your governance program over every time a new regulation is enforced. Make sure to document every mapping decision and the reasoning behind it. This documentation will serve as your proof if regulators or auditors ask how your governance program meets a certain requirement.
Common Pitfalls in Implementing AI Governance
Regardless of the industry or organization, many AI governance programs tend to fail in the same ways. Despite starting with good intentions, a sound framework, and a decent initial implementation, many programs hit a wall. This isn’t because the framework was flawed, but because the necessary operational disciplines weren’t integrated into the organization’s daily processes from the start. Two common pitfalls account for most governance failures in practice.
Unmonitored Risk Registers
A risk register that is not linked to real-time monitoring is a historical record, not a risk management tool. Organizations often put a lot of effort into creating comprehensive risk registers when they first implement a framework. They catalog AI systems, document risk profiles, and assign likelihood and impact scores. Then they consider the register finished. Six months later, the models it describes have changed, the regulatory environment is different, and the risk profiles are no longer accurate. The register is still there. The governance it was supposed to support is not. For those interested in exploring different machine learning frameworks, understanding these changes can be crucial.
It’s not about the process, it’s about the structure. Risk registers must be linked directly to monitoring outputs so that any anomalies in production will automatically trigger a review of the risk profile. They should also have scheduled refresh cycles that are tied to lifecycle events like model updates, changes in the data pipeline, and regulatory developments, rather than just annual governance reviews. And they need to have owners who will be held accountable for keeping them up to date, instead of committees who are only responsible for reviewing them from time to time. If a risk register needs a governance meeting to be updated, it probably won’t be updated when it should be.
Committees vs. Individuals: Where Should Accountability Lie?
Committees discuss, individuals act. When the responsibility for AI risk management is spread across a governance committee, an AI ethics board, a data privacy working group, and a security review panel, all of which need to agree before any action is taken, the end result is that no one person is held accountable for managing risk. In theory, everyone is responsible. In practice, no one is accountable. This is the number one reason why technically proficient governance frameworks fail to actually reduce risk. The answer is to designate individual accountability at every level of the governance structure, with the authority to act, not just make recommendations.
Most AI Risks Appear Months After Launch, Not Before
Pre-deployment testing only catches the risks you expected. Production operation uncovers the ones you didn’t. The distribution of AI incidents is heavily skewed towards the post-deployment phase – model drift builds up gradually until outputs are significantly degraded, adversarial exploitation targets deployed systems rather than development environments, and edge cases that didn’t appear in test data emerge under real-world usage patterns. The governance programs that concentrate the bulk of their risk management investment on pre-deployment controls are systematically under-protected during the period when risk is at its peak.
Pre-deployment testing isn’t to be brushed aside — it’s a necessary step, but not the only one. The organizations that are most successful in managing AI risk see deployment as the start of their risk management work, not the end. They have monitoring systems in place before the first production request even reaches the model. They have pre-set escalation protocols that activate when performance metrics cross certain thresholds. They have governance procedures that link production observations back to risk profile updates and control modifications. This operational cycle — monitor, detect, respond, update — is the real process of AI risk management. Everything before deployment is just getting ready to run this cycle effectively.
Common Questions
These questions are often asked by organizations when they are first developing or assessing AI governance programs.
What does an AI governance framework mean?
An AI governance framework refers to a structured set of policies, procedures, roles, and controls that companies employ to identify, evaluate, and control risks throughout the entire lifecycle of their AI systems, from the design and development phase to deployment and continuous operation.
Unlike an AI ethics policy or a set of AI principles, a governance framework is operational. It clearly outlines who is responsible for what, how risk assessments are carried out, what kind of monitoring is needed, and how the organization reacts when something doesn’t go as planned. While ethics principles define values, a governance framework establishes the organizational structure to act on these values. For a deeper understanding of AI tools in business, you might explore this comparison of AI services.
What is the NIST AI Risk Management Framework?
The NIST AI Risk Management Framework is a voluntary, lifecycle-based guidance document published by the National Institute of Standards and Technology in January 2023. It organizes AI risk management into four core functions — Govern, Map, Measure, and Manage — that together create an integrated operating model for identifying and responding to AI risk. A Generative AI Profile was added in July 2024, extending the framework’s applicability to large language models and other generative AI systems. It carries the widest industry adoption of any AI risk management framework currently in use.
Do all organizations have to comply with the EU AI Act?
The EU AI Act is relevant to any organization that introduces AI systems to the EU market or uses them within the EU, no matter where the organization is based. It is compulsory for organizations that fall within its scope, with the strictest requirements applying to providers and users of high-risk AI systems as categorized by the Act’s risk classification levels. Organizations that only operate outside EU markets and do not have any AI systems facing the EU are not directly affected by the Act, but many are adjusting their governance programs to meet its requirements in preparation for similar regulation in other areas.
What makes AI risk management unique compared to conventional IT risk management?
Conventional IT risk management was developed for systems that have predictable behavior — systems where the same input consistently produces the same output, and where failures can usually be traced back to specific code or infrastructure components. AI systems, on the other hand, are probabilistic. They produce varying outputs, their behavior can change without any alterations to the underlying code as the distribution of input data changes, and their failure modes are often subtle and gradual rather than sudden and noticeable.
AI risks bring about a whole new set of categories that traditional IT risks don’t cover: biased models and failures in fairness, gaps in explainability, adversarial manipulation of model outputs, risks to the integrity of training data, and the ethical and societal consequences of automated decision-making on a large scale. A traditional IT risk framework, when applied to AI systems, will systematically overlook these categories. This is exactly why we have dedicated AI risk management frameworks.
Who should be responsible for AI risk management in a company?
AI risk management needs to be owned by a specific person at different levels of the company, not by a committee or a single central function that tries to handle everything on its own. The structure that works in real life clearly divides responsibility across roles:
- Chief AI Officer or Senior Risk Owner — Ultimate accountability for the organization’s AI risk posture and governance program effectiveness
- Product and Engineering Teams — Technical risk identification, control implementation, and monitoring for the specific AI systems they build and operate
- Central Governance Function — Framework management, policy setting, cross-cutting risk assessments, and governance reporting
- Legal and Compliance — Regulatory mapping, obligation tracking, and compliance evidence management
- Security Teams — Model security controls, access management, adversarial risk assessment, and incident response integration
The specific titles and organizational structure will vary, but the principle holds regardless of company size: every risk needs a named human being who is accountable for ensuring it is managed, with the authority to take action when it isn’t. For those interested in exploring enterprise AI development, understanding the roles within AI governance can be crucial.
Even for smaller organizations that do not have a dedicated AI risk team, this model of accountability can still be implemented. This can be done by appointing a single senior leader with the responsibility of managing cross-functional AI risk, and this leader can be supported by contributors from the engineering and legal departments. This means that the model of accountability can be scaled down, but it is still necessary to have named ownership.
It’s a real challenge to get AI governance right. It’s not that the frameworks are complex, but rather that turning a framework into an operational discipline demands a consistent commitment from the organization, collaboration across functions, and a readiness to view risk management as a key business function, not just a compliance task. The organizations that do this effectively do more than just prevent incidents. They create AI systems that are more dependable, earn greater user trust, and withstand regulatory examination because their governance is integrated into how they construct and operate AI, rather than being added on later.
As artificial intelligence continues to evolve, businesses are increasingly looking to integrate AI technologies into their operations. This involves choosing the right AI framework that suits their specific needs. Many developers are torn between using TensorFlow or PyTorch as they are among the best machine learning frameworks available. Selecting the appropriate framework can significantly impact the efficiency and effectiveness of AI implementations in various industries.
