How to stand up an AI governance committee (without boiling the ocean)

Boards are asking about AI governance, and many organizations are responding the same way: convene a committee, invite everyone with an opinion, and produce a policy document that tries to govern "AI" as if it were one thing.

‍

It isn't. And that's the first mistake to avoid.

‍

Start by bucketing AI use into three domains

Before you assign a single stakeholder or write a single policy, you need to establish the domains of AI use in your organization. There are three distinct domains, and they have almost nothing in common when it comes to visibility, controls, and threats.

1. Building AI into your products and services (development). This is AI in your development lifecycle: embedding models into what you ship. For a small number of organizations, this includes training and running their own models, but most won't take that on. Most will integrate foundation models through APIs and build on top.
2. Running AI in production (runtime). Once AI-powered services leave development, you're operating models in a runtime environment with real customer data, real availability requirements, and real attack surface. Prompt injection, model abuse, data leakage through outputs. These threats show up at runtime.
3. Workforce use of AI. Your employees are using AI tools in their day-to-day business processes and to automate new ones. Sales reps summarizing calls, finance teams building models, engineers pasting code into chatbots. This is happening whether you've sanctioned it or not.

Everything else in your governance program hangs off this structure.

‍

Two of these domains already have control points. One doesn't.

For the first two domains, you're not starting from scratch.

‍

Most modern software development lifecycle (SDLC) pipelines already give you control points you can tap into. Dependency scanning, code review gates, artifact registries, CI/CD approval workflows. Governing AI in development is largely a matter of extending controls you already operate: which models are approved, which libraries are sanctioned, what data can touch training and fine-tuning workflows.

‍

The same is true at runtime. Production environments have mature operational controls (network segmentation, API gateways, observability, WAFs) that you can layer AI-specific protections onto. You're adapting an existing discipline, not inventing one.

‍

The third domain is different. Most organizations have little visibility into the AI tools their workforce actually uses. There's no pipeline to instrument, no gateway everything flows through. Employees adopt AI tools the same way they've adopted SaaS for the last decade: with a credit card, an OAuth grant, or a free trial, and no ticket to IT. This is the Workforce Edge: decentralized, employee-led adoption that bypasses your existing control points. If your governance committee treats this domain like the other two, you'll write policies that govern nothing.

‍

The governance sequence: visibility, risk analysis, policy, alignment

Within each domain, the basics of governance follow the same sequence:

1. Visibility. You can't govern what you can't see. What models are in your codebase? What AI services are running in production? What tools has your workforce adopted?
2. Risk analysis. Not every use of AI carries the same risk. A marketing team drafting copy in ChatGPT is not the same as a finance analyst uploading customer data to an unvetted tool.
3. Policy. Only once you understand what's happening and what matters can you write policy that reflects reality rather than aspiration.
4. Alignment. The last mile: bringing actual behavior in line with the policy you've established.

Many committees start at step three. They write the policy first, then discover they have no way to see whether anyone follows it. Resist that urge. Policy written without visibility is fiction with a letterhead.

‍

Get the right stakeholders for each domain

This framework only works if you match the right stakeholders to each domain. This is where the bucketing pays off, because the people with authority in each domain are different.

‍

For in-house models and AI in development: the engineering organization and the DevOps team. What matters is having a stakeholder with real authority in that domain, someone who owns the pipeline and can actually implement control points, not just attend meetings about them.

‍

For runtime and production: this often overlaps with development, but add your operations team and an architecture representative. The people who own uptime and incident response need a seat, because AI in production fails in operational ways.

‍

For workforce AI: your CIO (or whoever owns IT), plus your identity and security teams. Identity is the underappreciated one here. Workforce AI adoption shows up first in your identity layer, in new OAuth grants, new SSO integrations, new accounts created with corporate email. If your identity team isn't at the table, you're missing your best source of visibility into the domain that has the least of it.

‍

Across all three domains, add legal, privacy, and compliance, and name an executive sponsor who can settle disputes between domains.

‍

The biggest lesson: build constructive outlets

If there's one thing that separates governance programs that work from ones that generate resentment, it's this: for workforce AI, "no" is not a strategy.

‍

Your employees are using AI because it makes them better at their jobs. Block a tool without offering an alternative and you don't stop the behavior; you just push it to personal devices and personal accounts where you have zero visibility. Now you've traded a governance problem you could see for one you can't.

‍

The better approach is to identify the use cases your workforce actually has and make the easy path the right path. Find out what people are trying to accomplish. Stand up sanctioned tools that meet those needs. Make the approved option faster to access than the shadow one. When someone adopts an unsanctioned tool, treat it as a signal about unmet demand, not just a violation to remediate.

‍

Alignment beats enforcement, because enforcement without an outlet just relocates the risk.

‍

Where to start

If you're standing up an AI governance committee now, the sequence is straightforward:

1. Bucket your AI use into the three domains: development, runtime, and workforce.
2. Assign a stakeholder with real authority to each domain.
3. Start with visibility in all three before writing any policy.
4. For workforce AI specifically, plan constructive outlets alongside any restrictions.

The organizations getting this right don't have the longest policy documents. They figured out early that AI governance is three different problems wearing one name, and they staffed each one accordingly.

‍

See which AI tools your workforce is already using. Start a free trial.