AI Governance for Builders: NIST AI RMF & the EU AI Act

June 25, 2026 · Securing AI: How AI Gets Attacked — and Defended (part 34)

▶ Watch on YouTube & subscribe to The Stack Underflow

Every AI system you ship carries risk. Some of that risk is technical — a model that hallucinates, a pipeline that leaks training data, an agent that takes an action nobody authorized. Some of it is legal: the EU AI Act is already enforcing prohibitions as of February 2025, Italy fined OpenAI €15 million for GDPR violations in training data processing (EDRM, 2026), and the high-risk tier pulls concrete obligations that engineers — not just lawyers — will have to implement. Governance is the mechanism that makes all the technical security work you have done on your system accountable. Without it, you have a locked house with no record that you checked the locks.

The problem is that most governance content is written for compliance officers. It talks about committees, risk registers, and PDFs. This tutorial is the version for the engineer who actually has to implement it. Governance is not a binder; it is four verbs you apply to your own project, twelve risk categories you check your system against, and a set of legal tiers that decide how much of that is optional.

The one-sentence version: Governance is the loop that turns your security work into an accountable, documented, continuously re-evaluated system — and the EU AI Act is the external force that decides which parts of that loop are legally required.

The NIST AI RMF: four functions, one loop

The NIST AI Risk Management Framework (AI RMF 1.0), published January 26, 2023 (NIST AI 100-1), organizes AI risk management into four core functions. Three of them — Map, Measure, Manage — run as a working loop on your actual system. The fourth — Govern — is the outer ring that holds the loop in place by setting policy, assigning roles, and ensuring accountability.

┌──────────────────────────────────────────────────────────┐
│                     GOVERN                               │
│   (policy · roles · accountability · culture)            │
│                                                          │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐          │
│   │   MAP    │───▶│ MEASURE  │───▶│  MANAGE  │          │
│   │(identify │    │ (test &  │    │ (act on  │          │
│   │ risks)   │    │  metric) │    │  results)│          │
│   └──────────┘    └──────────┘    └──┬───────┘          │
│        ▲                             │                   │
│        └─────────────────────────────┘  (loop)          │
└──────────────────────────────────────────────────────────┘
    NIST AI RMF 1.0 · January 26, 2023

Govern is cross-cutting. It does not run in sequence with the other three — it wraps them. Govern means your organization has decided: who approves high-risk use cases, how third-party models are introduced, what constitutes an unacceptable risk, and who is accountable when something goes wrong. Without Govern, the loop has no policy to run against.

Map means walking your own pipeline and naming what could go wrong at each stage. This is where the AI attack surface mental model becomes a governance artifact. Stage S0 (training data) carries poisoning risk. Stage S4 (inference) carries injection risk. Stage S5 (agentic action) carries excessive-agency risk. Map turns the attack surface diagram into a documented, dated inventory of risks with owners.

Measure means every mapped risk gets a test. An evaluation suite for confabulation. A red-team result for prompt injection. A metric for bias drift. Measure is the phase that connects your threat model to your CI pipeline — if a risk cannot be measured, you cannot know whether your mitigations are working.

Manage means acting on what you measured: mitigate the risk, formally accept it on the record (with an owner and a date), or gate the system so it cannot proceed until the risk drops below threshold. Then the loop runs again. Risk does not stay static — models get updated, new injection techniques appear, data drifts — so the Map → Measure → Manage cycle is not a one-time exercise.

The GenAI Profile: NIST AI 600-1 and the 12 risk categories

General-purpose AI systems have risks that the base AI RMF does not fully address. NIST AI 600-1, the Generative AI Profile, was published July 26, 2024, under Executive Order 14110. It names twelve risk categories that are unique to or significantly amplified by generative AI, and it provides more than 200 suggested actions organized by the same four RMF functions (NIST AI 600-1, 2024).

The 12-category checklist is most useful during your Map phase. Run each category against your system and ask: is this risk present? If yes, does it have a test (Measure) and a response plan (Manage)? If any row has no test and no plan, you have an open gap. Categories 9 (Information Security) and 4 (Data Privacy) overlap heavily with the technical controls covered elsewhere in this series — the governance layer is what makes those controls documented and owned.

The EU AI Act: four tiers, one question

The EU AI Act (Regulation 2024/1689) uses a risk-tiered model. The tier your system falls into determines what you are legally required to do. The question is not “does this apply to me?” — if you are placing an AI system on the EU market or using one to affect EU residents, it almost certainly does. The question is “which tier am I in?”

        ▲   TIER 1: UNACCEPTABLE (banned)
        │   Social scoring · subliminal manipulation ·
        │   real-time biometric surveillance in public
        │   ENFORCED: Feb 2, 2025
        │
        │   TIER 2: HIGH-RISK (heavy obligations)
        │   Biometrics · critical infrastructure ·
        │   education · employment · essential services ·
        │   law enforcement · migration · justice
        │   ENFORCED: Dec 2, 2027 (Annex III, Digital Omnibus deferral)
        │
        │   TIER 3: LIMITED (transparency duties)
        │   Chatbots · synthetic content generators ·
        │   emotion recognition systems
        │   Must disclose AI nature to users
        │
        ▼   TIER 4: MINIMAL (largely free)
            Spam filters · AI-assisted game NPCs ·
            simple recommenders

The four tiers, exact names per the Act: Unacceptable, High-risk, Limited, and Minimal.

Tier 1 (Unacceptable) is already enforced. Systems in this category are banned outright — no conformity assessment path exists. If you are building real-time biometric surveillance for public spaces, subliminal manipulation systems, or AI used for social scoring by public authorities, you are in violation from day one.

Tier 2 (High-risk) is where governance becomes a full engineering obligation, not optional. High-risk systems — those used in employment screening, credit scoring, educational assessment, medical device integration, law enforcement, or border control — must complete a conformity assessment, register in the EU AI Act database, implement a quality management system (QMS), maintain technical documentation, activate post-market monitoring, and support human oversight. Deployers must retain automated decision logs for at least six months and conduct Fundamental Rights Impact Assessments (FRIAs) where required. A provisional agreement reached May 7, 2026 (the Digital Omnibus) deferred the Annex III high-risk deadline from August 2, 2026, to December 2, 2027, giving compliance teams firmer dates to plan against (Trilateral Research, 2026).

Tier 3 (Limited) requires transparency: chatbots must disclose they are AI, synthetic content (images, audio, video) must be labeled, emotion-recognition outputs must be disclosed. These are engineering requirements, not just policy ones — your system needs a disclosure mechanism that fires reliably.

Tier 4 (Minimal) carries no mandatory obligations under the Act, though best practices under the RMF still apply.

General Purpose AI (GPAI) models — foundation models deployed broadly — have their own obligations under Chapter V of the Act, enforced since August 2, 2025. This includes transparency requirements, copyright summaries, and, for the most capable models, additional systemic risk assessments.

How to map your system to a tier

Before you touch the NIST functions, settle your tier. The tier anchors what is required versus what is best practice.

Step 1 — List what your system does and who it affects
         (employment? healthcare? law enforcement? children?)
         |
         v
Step 2 — Check Annex III of the EU AI Act (high-risk list)
         Does your use case appear?
         |
   YES   |   NO
    v         v
Tier 2     Check Annex I (safety-component list)
(heavy)    and GPAI definition
               |
         Foundation model?
           YES → GPAI rules
           NO  → Tier 3 (transparency) or Tier 4 (minimal)

Getting the tier wrong in either direction is costly. Under-classifying a high-risk system exposes you to fines up to €35 million or 7% of global annual turnover (EU AI Act, Article 99). Over-classifying burns engineering time on conformity overhead that is not required.

How to defend: building the governance layer

Governance is not a separate project. It is the wrapper you put around the security work you are already doing. Here is a concrete layered approach:

Layer 1 — Classify before you build. Determine your EU AI Act tier and your NIST RMF profile (full RMF or lightweight for lower-risk systems) during the design phase, not during audit. The tier shapes your documentation requirements from day one.

Layer 2 — Make Map a structured artifact. For each stage of your pipeline (data ingest, training, evaluation, deployment, inference, agentic action), produce a risk inventory that names the risk, the NIST AI 600-1 category it maps to, the owner, and the date reviewed. This document is your Map output. It is also the foundation for any conformity assessment if you are in Tier 2.

Layer 3 — Wire Measure into CI. Every risk in your Map inventory should have a test or a metric. Evaluation suites for confabulation and bias belong in your CI pipeline, not just in a pre-release checklist. Red-team exercises (see AI Red Teaming) produce findings that feed back into Map. Monitoring dashboards (see AI Monitoring, SIEM and Incident Response) generate the metrics that keep Measure live after deployment.

Layer 4 — Gate high-risk decisions. For Tier 2 systems, human oversight is not optional — it is a legal requirement. For any system, the Manage function should include formal decision records for risks you are accepting: who accepted it, at what risk level, with what planned re-evaluation date. Accepted risks that are never re-evaluated become forgotten vulnerabilities.

Layer 5 — Maintain a GPAI/third-party inventory. Category 12 of the NIST AI 600-1 profile (Value Chain and Component Integration) is one of the most commonly skipped. If your system uses a foundation model, a third-party embedding API, or any pre-trained component, document it: what version, what known limitations, what audit trail. This feeds directly into the AIBOM (AI Bill of Materials) practice covered in AIBOM and Model Signing.

Common misconceptions

• “Governance is only for enterprises shipping regulated products.” GPAI model rules under the EU AI Act apply to any organisation placing a general-purpose model on the EU market, and prohibited practices apply to everyone using AI to affect EU residents. A three-person startup using a foundation model to screen job applicants is in the high-risk tier. Size does not change tier.

• “NIST AI RMF compliance means filling out the NIST playbook subcategories.” The playbook subcategories are suggestions, not requirements. The RMF is a voluntary framework. The value is the four-function loop — Govern, Map, Measure, Manage — applied consistently to your actual system. A lightweight, honest risk inventory beats a comprehensive playbook that nobody maintains.

• “Once the conformity assessment passes, I’m done.” High-risk AI systems under the EU AI Act require ongoing post-market monitoring. Risk does not expire at launch. The NIST loop — Map → Measure → Manage → repeat — is the operational expression of that ongoing obligation. A conformity assessment that has never been re-evaluated against updated model behavior is a liability, not a protection.

• “Governance and security are separate concerns.” Governance is what makes security decisions accountable and auditable. Prompt injection defenses that are not documented, tested, and owned by a named individual are not governance-compliant. The technical security controls in this series are the content of governance; the RMF loop is the process that keeps them live.

Frequently asked questions

What is the difference between NIST AI RMF and NIST AI 600-1? The AI RMF (NIST AI 100-1, January 2023) is the general framework for managing AI risk across any AI system — it provides the four-function structure (Govern, Map, Measure, Manage) and the seven trustworthiness characteristics. NIST AI 600-1 (July 2024) is a profile of the RMF specifically for generative AI systems. It names the twelve risk categories that are unique to or amplified by GenAI and provides more than 200 suggested actions organized by RMF function. You use the RMF as your governance process and AI 600-1 as a checklist for GenAI-specific risks within that process.

Do I have to comply with the EU AI Act if I am not based in the EU? The Act applies extraterritorially: if your AI system is placed on the EU market, or if its outputs are used in the EU, the Act applies to you regardless of where you are incorporated. GPAI models with EU users are covered under Chapter V. The practical implication for US or UK developers is that any system affecting EU residents — including SaaS products — falls within scope. The tier and the specific obligations depend on what the system does, not where you are located.

What does “human oversight” mean as an engineering requirement for high-risk systems? At minimum it means the system cannot take a consequential action affecting an individual without a human having a meaningful opportunity to review and override it. That is an engineering constraint, not just a policy one: you need a review queue, a UI surface for human decisions, audit logging of human overrides, and a fallback path when the AI output is rejected. Automation bias — the human tendency to defer to AI recommendations without scrutiny — is itself addressed in NIST AI 600-1 (category 7, Human-AI Configuration), and designing against it means making the review step genuinely reviewable, not a rubber-stamp checkbox.

How does OWASP LLM Top 10 2025 fit alongside the NIST AI RMF? OWASP LLM Top 10 2025 is a risk catalog — a concrete, named list of the most critical vulnerabilities in LLM applications (prompt injection LLM01, data and model poisoning LLM04, excessive agency LLM06, and so on). It feeds directly into the Map function of the RMF: each OWASP entry is a named risk to add to your risk inventory. NIST AI 600-1 provides the GenAI risk categories at a higher level; OWASP provides the specific attack patterns within those categories. Used together, OWASP gives you the “what to look for” and the RMF gives you the “how to manage it.”

What happens if I mis-classify my system at a lower tier than it should be? Under the EU AI Act, placing a high-risk system on the market without completing the required conformity assessment exposes you to fines of up to €15 million or 3% of global annual turnover for providers, and up to €35 million or 7% for violations of prohibited practices (Article 99). Beyond fines, market withdrawal orders and prohibition of use are available enforcement tools. The practical risk for builders is that mis-classification is not a safe harbor — national market surveillance authorities can re-classify and enforce retroactively.

Where this fits in the series

Governance is the plane that wraps every stage of the pipeline you have been learning to defend. You built the mental model in The AI Attack Surface Explained. You learned how to turn it into a structured threat model in How to Threat Model an AI System: ATLAS, OWASP, MAESTRO and NIST. You practiced finding holes in Red Teaming an AI System. You built the monitoring layer in AI Monitoring, SIEM and Incident Response. Governance is what makes all of that work accountable — the policy that says it must happen, the loop that checks it is happening, and the legal tier that decides which parts are required. Browse all tutorials to follow the complete series.