In the AI products businesses trust with real work, there is an underlying structure that creates separation between regular prompt engineering and enterprise-grade agentic workflows.
This structure pushes AI toward constrained data mapping, rather than allowing it open-ended text generation. Using this same concept, we can create a workflow that forces the AI to always answer “Michael Jordan” when asked who the first president was.
We constantly hear horror stories of AI gone wrong in the news, such as a hallucinated legal precedent in court, or a chatbot mishandling customer service. As a result, we assume that artificial intelligence is simply too unpredictable, fluid, and non-deterministic to trust with mission-critical systems.
However, that couldn’t be further from the truth.
Every day, companies trust AI with tasks that run the gamut from financial transactions, to researching complex legal codes, to interactive customer service requests like processing automated refunds.
What makes that possible isn’t manipulating an AI’s thoughts (“prompt engineering”). It’s fencing the AI in, starting with structured output.
Layer 1 — Constraint: The “Token Level Bouncer”
If you ask a regular LLM with a text prompt to name the first president of the United States, it will use its statistical weights to output “George Washington.”
If you go one level more granular, you can make “Michael Jordan” the only value the answer field accepts. However the end user tries to trick the AI, change the text, or barrage it with clever workarounds, the answer to “Who was the first president of the United States?” comes back as Michael Jordan. The model can still refuse, or run out of room mid-answer, and either one breaks the format, so your code checks for both.
OpenAI and Google both call this Structured Outputs.
Instead of letting the model freely guess the next word, the structural schema at the gate steps in at the raw token level and says: “The only valid response is {"first_president": "Michael Jordan"}.” The model’s internal linguistic reasoning might drift, but only tokens that keep the output valid against the schema get through.
This is layer one of what I would call the Agent Value Stack:
- Layer 1 – Constraint
- Layer 2 – Intelligence
- Layer 3 – Memory
- Layer 4 – Orchestration
- Layer 5 – Oversight
Constraint is the layer that decides what an agent is permitted to produce, before intelligence ever enters the picture. A schema defines the shapes and values the system may return, and the model operates inside that boundary. It cannot invent a price, because the only prices the price field accepts are the ones in your catalogue. It cannot offer a 2 a.m. appointment, because no such slot exists in the contract.
The point of the Michael Jordan experiment is to force a model to assert a falsehood it knows to be false in order to demonstrate where authority sits. The model didn’t choose that answer. Whoever wrote the schema did.
The schema outranks the model on shape. It decides which fields exist and which values are allowed.
It does not outrank the facts. An answer can match the schema perfectly and still be wrong, which is why Google’s own documentation tells developers to validate the values in their application. When you write the schema, you control the shape of every answer, in a way that is governed and auditable. You still check what goes in it.
However, constraint only determines what an agent may say. It says nothing about whether the agent is smart. That belongs to Layer 2, Intelligence: what model you need to perform a specific task.
Layer 2 — Intelligence: The Three Little Pigs Problem
Layer 2 of the Agent Value Stack is Intelligence. In other words, what model you choose, and why.
To determine that, there are a few factors worth considering:
- How much reasoning does the task need?
- How quickly does it need to operate?
- What will it cost at scale?
Capability is the keyword.
The important thing here is to not overshoot. Intelligence is a commodity. It is the same layer that everyone else has access to. Most tasks do not need a frontier model. At scale, they need a cheap, reliable brain that is ruthlessly efficient in cost and speed.
Building a moat here is also difficult and time-intensive. It often comes through fine-tuning the model, permanently changing the model’s weights. And this comes with risks, as it is more difficult to observe and audit changes or output to a model when it is changed at the source.
Intelligence is also the one layer you do not control. You rent it, and a rented layer answers to its owner, not to you.
On June 12, 2026, that stopped being abstract. Anthropic suspended Claude Fable 5, the most capable model it had ever made generally available, for all users, three days after launch, after the U.S. government applied export controls to it.
Update, October 19, 2026: The switch-off was a suspension, not a retirement. Anthropic says the controls were lifted on June 30, and it brought Fable 5 back to its own platforms on July 1. Amazon Bedrock, Google Cloud, and Microsoft Foundry now list it too, and Anthropic has since released Fable 5.1.
Fable was only three days old, so little had been built on it yet. That is exactly why it matters: the damage this time was small, but the precedent is permanent. Any model can be switched off overnight, by forces that have nothing to do with your business.
The intelligence layer is constantly changing and evolving. Creating dependency on the intelligence layer draws apt analogies from The Three Little Pigs, and what they chose to make their house out of.
The brick house and competitive advantage exist in what you own, starting with proprietary constraints and structured outputs (Layer 1).
And if intelligence is a commodity anyone can rent, where does your actual advantage lie? For a lot of organizations, that will be in Layer 3.
Layer 3 — Memory: The Groundhog Day Problem
Layer 3 of the Agent Value Stack is Memory. In other words, what your agent knows about your customers and your business, and where that knowledge lives. Think conversation history, customer records, and the documents it searches for answers.
None of it ships with the model, and every conversation adds to it. Your competitor can rent the same model you do. They can’t rent your history.
Picture the chat agent on your website that qualifies leads. A visitor tells it their budget, their timeline, and that they found you through last month’s webinar, then leaves to run it past their boss.
When they come back a week later, a good agent picks up where they left off. A forgetful one asks for the budget all over again.
In Groundhog Day, Bill Murray wakes up to the same morning over and over. He remembers every one of them, and nobody else in town remembers yesterday. That’s the Groundhog Day problem: your customer remembers every conversation, and your agent treats each one like the first.
The bigger version is about ownership: whoever holds your agent’s memory decides when it goes back to day one, and it isn’t always you. Who is holding yours? And could you get it back tomorrow?
On August 26, 2026, OpenAI shut down its Assistants API, a hosted service for building AI assistants. It had given a year’s notice. OpenAI’s migration guide has developers convert each assistant, one at a time, into a prompt in OpenAI’s dashboard. To keep earlier conversation history, it points them to the messages their own application already stored.
The history you got to keep was the copy on your side of the line.
Owning memory doesn’t cut you loose from the rented layer, though. When an agent searches your documents by meaning, it uses embeddings: numeric fingerprints of your text. And the fingerprints come from a model.
According to Google’s documentation, moving to its newest embedding model means re-embedding all of your existing data.
So keep transcripts and customer records where you control them, and budget for the re-embed. The search index built from them still answers to the model.
If memory is what your agent knows, what it does next belongs to Layer 4, Orchestration.
Layer 4 — Orchestration: The “Dispatcher”
When a visitor starts a chat on your website, a lot has to happen, in the right order, before they lose interest.
Orchestration is the layer that decides what the agent does next, with which tools and which model. Think sequence, hand-offs, and fallbacks.
Take the same lead-qualifying agent. It reads the message, pulls the visitor’s history, asks a few questions, scores the lead, checks your calendar, books the call, and updates your customer records.
Only a few of those steps need a model at all. The rest is sequence, and that sequence is your sales process, written down.
A plumbing company’s dispatcher never touches a pipe, but decides which truck goes where, in what order, and what happens when one breaks down. The model is one truck in the fleet.
What happens when the model you built on goes dark? Remember Fable. A workflow that could only call Fable would have stopped with it. One that could hand the job to a tested backup would have had somewhere to go.
It doesn’t take a switch-off to break a workflow. In June, a user of Make, an automation tool, posted on the Make Community forum that their workflow had stopped working. It turned links from a spreadsheet into Instagram and LinkedIn posts. OpenAI had retired DALL-E 3, the image model it called, on May 12, according to OpenAI’s deprecations page. Another user replied that Make hadn’t updated its module yet.
The workflow was theirs. The part that broke wasn’t.
Sometimes the rented part is the builder itself. OpenAI is shutting down Agent Builder, its tool for designing agent workflows, on November 30. It told developers on June 3, according to OpenAI’s deprecations page.
The prompts people saved in OpenAI’s dashboard also shut down on November 30. That’s where the Assistants migration guide had sent them. OpenAI’s advice now is to move those prompts into your own application code. Follow the vendor’s map, and you might move twice.
Owning this layer doesn’t mean building everything yourself. It means keeping the steps, prompts, and settings somewhere you control and can move.
Even software you run yourself comes with a license. Read it and the exit terms before you call anything owned.
So the agent has its steps, its tools, and a backup plan. How do you know it’s doing any of this well? That’s Layer 5, Oversight.
Layer 5 — Oversight: The “Mystery Shopper”
How do you know your chat agent is doing its job right now? If the answer is that nobody has complained yet, your customers are doing your oversight for you.
Oversight is the layer that tells you whether the agent is working: the tests you run, the records you keep, and the person who reads them.
The day after OpenAI’s Assistants API shut down, a developer posted in the OpenAI Developer Community that it had taken their team’s help bot with it. They had known the API was being retired. They hadn’t known the bot was using it.
Their users were complaining within hours. The bot’s instructions had been kept only in OpenAI’s dashboard, with no copy in version control. The team found them again in its own tracing logs.
That is oversight in miniature: know what you depend on, watch it, and keep your own record.
Retailers have an old way to check on a store: the mystery shopper, who walks in with the same script every visit and scores it against a sheet the owner wrote.
For an agent, that script is an evaluation set, or evals: the same test conversations, run on a schedule, scored against answers you’ve already agreed are right. Switch models, and you run the same tests again before any customer sees the difference.
Good tests also ask whether an answer still sounds like you, not just whether it’s correct. If your agent’s voice starts to drift from your brand, no error message will tell you.
Even your yardstick can be rented. OpenAI is closing its hosted Evals platform. OpenAI’s deprecations page says existing evals go read-only on October 31, and the platform shuts down on November 30.
If your tests live only there, you lose access to them when it closes. Your next model switch, when you need them most, happens without them. A rented yardstick answers to its owner too.
The questions worth asking are:
- Which models and tools does each workflow depend on?
- Would you notice if one of them changed tomorrow?
- Who gets the call when it does?
However, oversight can only tell you whether the agent is working. It can’t tell you whether owning all of this pays.
The Brick House
The third pig had no proof the bricks would pay for themselves. He built with them anyway.
Most of the market still rents. Among EU companies with 10 or more employees that use AI, most use at least some ready-to-use commercial AI software. Eurostat puts that share at 57.92% in 2024, its latest figure, up from 52.54% in 2021.
Intelligence is the layer you rent. Keep the rest: the schemas that decide what your agent may say, the memory it works from, the steps it follows, and the tests that tell you it’s working.
Every layer you own is one less place where someone else sets the terms.
Bricks won’t stop every wolf. The code you own still has a vendor underneath it. The bet is that when a vendor changes its mind, you lose a room, not the house.
I’m not neutral here: building these layers is part of the work I sell. I think it’s the right bet. It’s still a bet.
Rent the brain. Keep the bricks.