Stop Buying AI Agents. Start Building the Layer They Run On.
Most enterprises are buying AI the way they once bought SaaS — one tool at a time. An assistant for the sales team here, a document bot for legal there, a clever demo from a vendor that wowed a VP last quarter. Eighteen months in, the drawer is full of pilots that never reached production, and none of them talk to each other.
The companies pulling ahead did something different. They stopped buying agents and started building the layer the agents run on.
That distinction is the whole game. A single agent is a demo. Twenty agents that hand work to each other, share what they know, and answer to one audit trail is an operating model. The thing that separates the two is not the agents. It is two shared layers underneath them: an orchestration layer and a governed data layer, built once and reused by everything that comes after.
One Agent is Easy. The Eleventh is the Test.
Standing up a first agent has gotten almost trivial. Pick a use case, wire a language model to an API, drop it where people already work. A capable team ships that in weeks.
The trouble starts at agent two. Build it the same standalone way and you now have two systems that each authenticate separately, each carry their own copy of "what is a customer," each log to their own place, each get governed by their own rules. By agent five you are maintaining five integrations into the same ERP and five different answers to the question an auditor will eventually ask: who did what, against which record, and who approved it.
That is how AI initiatives stall. Not because the models are bad. Because every new use case pays full freight again, and the cost curve goes the wrong way.
An operating model bends the curve. The first use case carries the weight — it stands up the orchestration layer and the data layer. The second reuses both and ships faster. By the fifth, a new agent is mostly configuration: define how it routes, what it can touch, what a finished answer looks like. We have watched build effort on a real program fall to roughly 40% of the first use case by the fifth one. The plumbing was already there.
What the Orchestration Layer Actually Does
Strip away the marketing and an orchestration layer is four concrete things.
- A typed message bus: Agents pass structured, validated messages, not free text thrown over a wall. Every system gets a contract. A new agent plugs into the bus without anyone touching the agents already running — the property that lets the model scale instead of ossify.
- A handoff protocol: Real work crosses boundaries. A procurement question turns into a finance question turns into an approval. With agent-to-agent handoff, one agent escalates to another and the result comes back carrying the intent and the state, so the user sees one conversation instead of five chatbots.
- Shared session memory: Resolved entities persist across agents and across turns. Look up a customer once and every downstream agent already knows which customer you mean. Without this, every agent re-interrogates the user, and the experience collapses back into form-filling.
- Audit and lineage: Every call — which agent, which model, which record, which human approved the write — is traced end to end. In a regulated shop this is not a nice-to-have. It is the thing that lets you put an agent near a system of record at all.
None of that is exotic. It is the difference between a layer you can grow on and a pile of integrations you have to babysit.
The Data Layer is Half the Answer, and the Half People Skip
Orchestration gets the attention. The data layer is what keeps the whole thing honest, and it is the part most pilots quietly skip.
Agents are only as good as the ground they stand on. If your procurement agent and your finance agent define "open order" two different ways, they will confidently disagree, and your CFO will stop trusting both. A governed data and semantic layer fixes that at the source: one definition of a customer, an order, a part, exposed to every agent, with lineage tracked so you can see where a number came from.
Crucially, this does not mean a migration. The systems of record stay where they are. Agents read and write in place through service APIs and OData, under the user's own permissions, with no shadow copy of sensitive data spun up to feed a model. A lakehouse on object storage handles retrieval and analytics. The ERP stays the system of record. Nobody rips anything out.
When orchestration and data are both shared, interoperability stops being a feature you bolt on later. It is a property of the architecture. Agents share a vocabulary because the semantic layer gave them one. They share context because the bus carries it. They stay governed because the lineage was there from the first call.
Build Cloud-Agnostic, or Build a Cage
One more decision separates the operating models that last from the ones that lock you in: keep every layer vendor-neutral.
Event streaming, object storage, a model gateway that can route across more than one LLM vendor, Kubernetes underneath — all of it has open or managed equivalents on any cloud. Build to those and you can swap a component without rewriting an agent, and you can move the whole thing if your cloud economics change. Build to one vendor's proprietary stack and you have traded a fast start for a long-term cage. The model gateway matters most here. The frontier moves every few months; an operating model that can only call one provider's models is already behind.
What This Means for Enterprise Leaders
The pitch for an AI-first operating model is not "more AI." It is a different shape for the spend. The build cost is one-time. The orchestration layer and the data layer get paid for once and then carry every agent that follows — the second, the tenth, the one you have not thought of yet. The value recurs every year, for every employee, and grows as adoption does.
The trade is sharp when you say it plainly. Roughly the cost of a single ERP specialist, spent once, gives every employee in the building a way to ask any system a question and get an answer the audit trail can stand behind. The specialist leaves; the layer does not.
You can keep buying pilots. Or you can build the layer once and let the org scale on top of it. The first path looks cheaper this quarter. The second is the one still compounding three years from now.