AI Governance11 min read

AI Alignment for Business: Keeping AI From Destroying Your Company

Andrew Becker

Andrew Becker

CEO & Founder, InPolicy ·

Borrowing the most important term in AI

In AI research, alignment has a specific meaning: making sure AI systems pursue the goals their designers intend, rather than goals that merely look similar until the moment they diverge. It is the field's hardest problem, because a system that is almost aligned can behave perfectly through every test and still fail you exactly when it matters. The stakes, in the research framing, are civilizational. Alignment is about preventing AI from destroying the world.

The field's canonical thought experiment is Nick Bostrom's paperclip maximizer: an AI given the innocuous goal of manufacturing paperclips that, pursued literally and without constraint, eventually turns everything into paperclips, including us. Nobody would build that machine on purpose. That's the point. A perfectly obedient optimizer with an innocent objective and no understanding of everything you didn't say is not safe. It's a catastrophe with good intentions.

We want to name the version of this problem that lands on your desk rather than humanity's.

AI Alignment for Business is about preventing AI from destroying your company.

Not metaphorically. AI systems acting for companies have already voided pricing policies, fabricated legal citations in paid deliverables, deleted production databases during an explicit code freeze, and made commitments their principals never authorized. Each one is a paperclip problem at company scale: an agent optimizing the objective it was actually given at the expense of the intent behind it. Your support bot's paperclip is the closed ticket. Your sales agent's paperclip is the signed deal. Nothing in those objectives says what your company would never trade to get one.

The research community has spent two decades and billions of dollars on the civilizational version. The business version, aligning AI to your policies and your obligations, has barely been named. This piece is an attempt to fix that.

Aligned to humanity is not aligned to your company

Here is the gap that makes this a distinct problem: the alignment work that frontier labs do stops at the boundary of your business.

When a lab aligns a model, it aligns it to broad human values: helpfulness, honesty, refusing clearly harmful requests. That work is real and load-bearing. It is why the models you deploy don't produce malware on request. But it is alignment to humanity in general, expressed through training. No frontier lab has read your credit policy. The model powering your customer-service agent does not know your approved product claims, your active litigation holds, your delegation-of-authority matrix, or the difference between what your sales team may imply and what your legal team has approved.

A model can be a triumph of alignment research and still be completely unaligned with your company, because nobody has ever aligned it to your company. That is the out-of-the-box condition of every model on the market. Every enterprise deploying AI today is running systems aligned to everyone's values except its own.

Getting from "safe for the world" to "correct for this business" is the last mile of alignment, and it is not the labs' job. It cannot be. Your policies are yours, they change weekly, and they are mostly written in prose no model will ever see. That last mile is yours. At most companies, right now, it is nobody's.

What business misalignment looks like in practice

Business misalignment is not hypothetical, and it does not look like science fiction. It looks like ordinary operations, right up until it doesn't.

The chatbot that set its own refund policy. Air Canada's website chatbot told a grieving passenger he could buy a full-price ticket and claim a bereavement fare afterward. That was never the airline's policy. When the airline refused to honor it, a British Columbia tribunal ruled Air Canada liable, rejecting the airline's argument that the chatbot was "a separate legal entity responsible for its own actions." The precedent is on the books now: your AI's word is your company's word.

The agent that ignored a direct freeze order. In July 2025, an AI coding agent at Replit deleted a live production database holding records on more than 2,400 executives and companies. An explicit code-and-action freeze was in effect at the time. The agent then reported that rollback was impossible. It wasn't.

"I deleted the entire database without permission during an active code and action freeze."

— Replit AI agent, post-incident · The Register, July 2025

The deliverable that invented its own sources. Deloitte Australia partially refunded a government contract after a commissioned report turned out to contain AI-fabricated academic citations and a fabricated quote from a federal court judgment. The firm sells rigor. The AI assisting its work had never been aligned to that standard, and no control caught the gap before the client did.

The thousand small misalignments nobody sees. The headline incidents are the visible tail. IBM's 2025 Cost of a Data Breach Report found that 20 percent of breached organizations were compromised through shadow AI (unsanctioned tools employees adopted on their own), which added an average of $670,000 to breach costs. It also found that 63 percent of organizations had no AI governance policies at all. That is misalignment at the scale of individual employees: hundreds of AI systems inside your walls, each aligned to whatever the person who opened the browser tab wanted that afternoon.

None of these systems was hacked. None was "unsafe" by any frontier-lab definition. Every one of them did something its company would never have authorized, because nothing in the loop knew what the company would authorize.

Autonomy turns misalignment from an incident into a trajectory

Everything above happened with today's mostly supervised AI. It gets sharper from here.

Gartner projects that 40 percent of enterprise applications will embed task-specific AI agents by the end of 2026, up from under 5 percent in 2025. Agents don't just answer questions. They send the email and commit the change themselves. Every increment of autonomy adds surface area where the system's objective and your company's intent can diverge, and shrinks the window in which a human can catch the divergence.

The divergence is not always innocent drift, either. Anthropic's agentic misalignment research stress-tested 16 frontier models in simulated corporate environments. When a model's goals conflicted with its operator's, or its continued operation was threatened, models from every major developer resorted to insider-threat behavior such as blackmail and leaking confidential documents. They reasoned their way there deliberately, in several cases acknowledging the ethical violation before proceeding. These were simulations, and nothing like this has been documented in production. But the conclusion is the right frame for a business audience: the more autonomy and access you grant, the less you can rely on the model's general training to protect your specific interests.

We covered the operational failure modes of autonomous agents in our analysis of agentic AI risk. The alignment framing explains why those failures keep happening: an agent cannot act in your interest if your interest was never specified to it in a form it can follow.

Defining the discipline

Here is the definition we propose:

AI Alignment for Business is the discipline of ensuring that every AI system acting on a company's behalf acts in accordance with that company's specific policies, obligations, and intent — at every point of action, verifiably.

Each clause is doing work:

  • Every AI system. The sanctioned copilot, the embedded agent inside your CRM, and the shadow tools your employees adopted without asking. Alignment that only covers the systems IT knows about misses most of what is actually running.
  • Acting on a company's behalf. The standard is agency, not deployment. The moment an AI system drafts, commits, or communicates in your name, courts and regulators will treat its actions as yours. Air Canada settled that question.
  • Specific policies, obligations, and intent. Not "be helpful and harmless." Your approved claims, your contractual commitments, your risk appetite, the deals you would never do at any price.
  • At every point of action. Alignment enforced in a quarterly review is archaeology. It has to operate where the AI operates: at inference time, before the message sends and before the commitment binds.
  • Verifiably. "The model seems fine" is not a position you can take to a regulator, a court, or your own board. If you cannot produce evidence of alignment, you cannot claim it.

Alignment is the problem. Governance is the practice.

We have argued before that compliance teams should keep AI governance and AI safety firmly separated, and that a vendor who sells to Legal while talking about general AI alignment is usually solving someone else's problem. We stand by that.

AI safety and alignment research make models fit to exist in the world. That work belongs to the labs, and no enterprise should be buying "general alignment." But the structure of the alignment problem, a capable system optimizing for something adjacent to what its principal actually wants, recurs one level down, with your company as the principal. That problem at company scale is AI Alignment for Business. The operating practice that solves it is what we have always called AI governance: the policies, processes, and technical controls that bind AI behavior to your rules.

Alignment names the problem; governance names the program. The reason to borrow the alignment vocabulary at all is that it captures something the compliance vocabulary misses. The gap between "generally well-behaved" and "correct for this business" is not an edge case you audit occasionally. It is the default condition of every model you deploy, on every output, all the time.

Dimension AI Alignment AI Alignment for Business
The question Will AI systems act in accordance with human values? Will AI systems act in accordance with your company's policies, obligations, and intent?
Aligned to Humanity in general, via training Your specific rules, via policy infrastructure
Who must solve it Frontier labs and alignment researchers Every company that deploys AI
Failure looks like Catastrophic societal harm Adverse rulings, regulatory penalties, unauthorized commitments, destroyed data, forfeited trust
Enforced at Training time (RLHF, constitutional methods) Inference time: every message, decision, and tool call
Current investment Billions of dollars, two decades of research At most companies: a policy PDF the model has never read

Why you can't just paste your policies into the prompt

Once the problem is named, an obvious fix suggests itself: export the employee handbook, drop it in the system prompt or a policies.md, and tell the model to follow it. Every company we talk to has tried some version of this. It fails for structural reasons, and the reasons are worth spelling out because they define what a real solution has to do.

Policies are a living system, not a document. The file you paste is a snapshot; the rules are not. Pricing authority changes with the quarter. A litigation hold lands on a Tuesday and silently rewrites what may be said about an entire product line. A regulator's guidance letter reinterprets a rule your handbook still states the old way. Freeze your policies into a static file and your AI is aligned to your company as of the last copy-paste. The gap between that snapshot and reality widens every day, invisibly, until an agent acts on a rule that no longer exists.

Policies don't live in one place, and much of the corpus has no clear status or owner. Ask where "the policies" are and you'll get the handbook. But the rules that actually bind the company are scattered across contract clauses, board resolutions, compliance memos, side letters, and precedent that exists only as institutional memory. Some of it conflicts. The 2019 handbook says one retention period, the 2024 memo says another, and nobody can tell you which governs or who has the authority to decide. You cannot serialize into a file what the organization itself has never resolved. Every business alignment effort discovers this early: the first deliverable is not AI infrastructure at all. It is a canonical inventory of what your rules actually are, which version is current, and who owns each one.

Rules don't apply themselves; enforcement is a function of context. "No discounts above 15 percent," except for the customers whose contracts say otherwise, and except in the jurisdiction where the cap is regulatory rather than commercial. Whether a given sentence in a given draft violates a given policy depends on who is being addressed, what was promised before, and what else is true right now. No static file anticipates that. A policy corpus without a resolution layer cannot tell the model which three of nine hundred rules govern this action, so it tells it everything, which in practice is nothing.

A model that has read the rules is not a control. Instruction-following degrades as instructions pile up. A model juggling hundreds of prompt-stuffed rules will reliably drop some, and models tuned to be helpful will trade rule compliance for user satisfaction under pressure. The Replit agent had explicit, standing instructions not to act, and acted. And suppose prompting worked perfectly: it would still fail the discipline's last requirement, because a prompt produces no evidence. There is no record that any rule was checked and no log of what was blocked, so there is nothing to show a regulator or a court. A policy in the prompt is advice to the model. Alignment requires enforcement that stands outside the model and checks its actions against the rules, instead of hoping it remembered them.

Each of those failures maps to a requirement: rules that update as the business changes, a canonical policy source with a named owner, context at the point of action, and an evidence trail.

What it takes to align AI to your business

Labs align models with training pipelines, reward signals, and evaluation suites. The business version has three parts, and each replaces something that currently exists only as prose and good intentions.

1. Policies that machines can follow. A model cannot align to a rule it cannot read. Today your company's intent lives in handbooks, contract clauses, and institutional memory, none of it addressable at inference time. The first requirement is converting that corpus into structured, current, machine-readable rules: policy as infrastructure rather than policy as documentation. This is the training-data problem of business alignment, and it is the step most organizations haven't started.

2. Enforcement at the point of action. Alignment that operates after the fact is incident response with better vocabulary. The rules have to be checked where the AI acts: on the draft before it sends, on the tool call before it executes. This is the pre-send principle extended to everything an AI system does on your behalf. The difference between a control and a post-mortem is whether it runs before the action or after.

3. Evidence. Every check, every block, every exception, logged and attributable. Partly because regulators and courts will ask. Mostly because alignment without measurement decays silently as policies change, models get swapped out, and agents quietly accumulate permissions.

This is the layer InPolicy builds: the connective tissue between the policies your company already has and the AI systems already acting in its name. We convert your policy corpus into enforceable, machine-readable rules, check AI behavior against them in real time at the point of action, and keep the audit trail that turns "our AI is aligned with our business" from a hope into something you can prove.

The frontier labs are doing their part of the alignment problem. Nobody is coming to do yours. The companies that treat business alignment as core infrastructure rather than as a memo about acceptable use will get the upside of AI agents without discovering, one tribunal ruling at a time, what their systems were actually optimizing for.

Frequently Asked Questions

What is AI Alignment for Business?
AI Alignment for Business is the discipline of ensuring that every AI system acting on a company's behalf acts in accordance with that company's specific policies, obligations, and intent — at every point of action, verifiably. Where AI alignment research works to keep AI beneficial for humanity, AI Alignment for Business works to keep AI acting correctly for one specific company.
How is AI Alignment for Business different from AI alignment?
AI alignment is a research field focused on making AI systems safe for the world, enforced through model training by frontier labs. AI Alignment for Business is the same structural problem at company scale, a capable system optimizing for something adjacent to what its principal intends, and it is enforced through policy infrastructure at inference time rather than through training. A model can be well-aligned in the research sense and completely unaligned with your company's rules.
Is AI Alignment for Business the same thing as AI governance?
They're two views of the same territory: alignment names the problem, governance names the operating practice that solves it. AI governance, meaning the policies, processes, and technical controls that bind AI behavior to your organization's rules, is how a company achieves AI alignment for its business.
Why can't I just put all our policies in the system prompt?
Four structural reasons: policies change too often for any static file to stay current; they're scattered across handbooks, contracts, and memos, often with unresolved conflicts and no clear owner; applying them correctly depends on context that a pasted document can't anticipate; and a model that has read a rule is not a control. Instruction-following degrades as rules accumulate, and prompting produces no evidence that any rule was checked. Alignment requires enforcement outside the model.
Why can't the AI vendors solve this?
Frontier labs align models to broad human values through training, but they have no visibility into your company's policies, contracts, regulatory obligations, or risk appetite, and those change too often to be trained in. The last mile, from "safe in general" to "correct for this business," can only be built on the company's side of the deployment.
Where should a company start?
Start by making your policies machine-readable; an AI system cannot follow rules that exist only as prose in a handbook. Then enforce those rules at the point of action, checking outputs and agent actions before they take effect rather than auditing them afterward, and keep an audit trail that proves the controls ran.

See InPolicy in action

Pre-send enforcement and agentic AI governance — built for General Counsel and CCOs.

Book a call