Skip to content
Try IQ Chat

12 min read

Multi-Model Is Now the Default. Governance Is Not.

Most enterprises already run three or more AI model families. Far fewer can say who owns them, what data reaches them, or what they cost. Here is what the 2026 data shows, why multiple models make the gap wider, and what a governed setup looks like.

By IQNOVO
Diagram of four scattered AI model chips converging through one governed gateway into an ordered list.

Two years ago, "Which model should we standardize on?" was a reasonable question for a CIO to ask. In 2026, it is mostly moot. The typical enterprise already uses several model families, from several vendors, chosen by several teams, often without anyone deciding that this was the plan.

That part is fine. Different models are genuinely better at different jobs, and the market keeps shifting under everyone's feet. The problem is what did not happen alongside it. Governance was designed, where it was designed at all, for one vendor and one contract. It has not been redesigned for five.

This article lays out what the 2026 surveys actually say, why running multiple models makes the governance gap wider rather than just bigger, and what a workable answer looks like.

Multi-model happened whether anyone planned it or not

The clearest data point comes from Andreessen Horowitz's January 2026 survey of 100 Global 2000 companies. 81% now use three or more model families in testing or production, up from 68% less than a year earlier. OpenAI is in production at 78% of those companies, Anthropic at 44% (63% including testing), and Google Gemini showing strong momentum across enterprise workloads. In a16z's earlier round in May 2025, 37% of CIOs were already using five or more models, up from 29% the year before.

The Harris Poll, surveying 600 CIOs for Dataiku between December 2025 and January 2026, found the same thing from the buyer's side:

Finding (Dataiku / Harris Poll, 600 CIOs) Share
Expect to rely on two or more LLM providers in 2026 to stay competitive 81%
Say different LLMs perform better for different use cases, requiring continual evaluation and switching 93%
Have already switched LLMs at least once (cost cited as the main driver) 55%

The vendor market itself is fragmenting, which pushes in the same direction. Menlo Ventures' November 2025 survey of 495 US enterprise AI decision-makers put Anthropic at 40% of enterprise LLM API usage, OpenAI at 27% (down from 50% in 2023), and Google at 21%. Enterprise generative AI spend hit $37 billion in 2025. When the leader changes roughly every year and the money triples, nobody sensible bets the whole company on one API.

The reasons enterprises give are practical, not ideological. In a16z's interviews, buyers described choosing models by task: one for coding, one for system design, one for complex question answering. One leader summed up the pricing side bluntly: "All the models perform well enough now, so pricing has become a much more important factor."

So multi-model is the rational outcome. It is also, in most organizations, an accident. It arrived through individual teams, individual subscriptions, and individual API keys. This brings us to the second set of numbers.

Governance stayed where it was

Every 2026 governance survey we could find tells the same story with different respondents.

Source (2026) Who was asked The gap
Compliance Week / konaAI, April 2026 193 compliance, risk, and audit leaders 83% use AI tools; about 25% have a strong governance framework
Larridin / TrendCandy, January 2026 365 senior leaders at companies with 1,000+ employees 25% have fully implemented AI governance; 58% cite unclear or fragmented ownership; 62% lack a complete inventory of the applications in use
Grant Thornton AI Impact Survey, early 2026 Nearly 1,000 senior business leaders 78% are not fully confident they could pass an independent AI governance audit within 90 days; 74% of boards approved major AI investment but only 52% set clear governance expectations
Kiteworks, closed July 2026 459 security, compliance, and technology professionals 43% have centralized AI into a single gateway; 50% cannot produce complete AI access records within one business day; 33% have tamper-evident audit trails

The Compliance Week report's own summary is worth quoting: "Executive leadership is driving adoption from the top down, faster than compliance teams can keep up."

Notice what these numbers are not saying. They are not saying companies lack an AI policy document. Many have one. They are saying companies lack the operational pieces that turn a policy into something enforceable: an inventory, an owner, an access record, an audit trail. Larridin's respondents average 23 AI tools per organization and, in most cases, cannot list them all.

Why multiple models make the gap wider, not just bigger

If you govern one vendor and add a second, the work does not double. It multiplies, because most governance controls were built assuming there was one place to look.

Inventory. With a single vendor, "What are we using?" has a short answer. With five vendors and hundreds of models behind aggregators, the honest answer is usually "we're not sure." Torii's 2026 SaaS benchmark found 61% of applications in the average organization are shadow IT, and 26 of the top 50 shadow IT apps are pure-play AI tools. Their discovery data logged 694 new AI applications entering customer environments in 2025 alone. You cannot write a data-handling rule for a tool you do not know exists.

Data-handling terms. Each provider has its own position on retention, training on customer data, and residency. Kiteworks found that 27% of organizations have never evaluated whether their AI vendors use their data for training. That is a manageable oversight with one vendor. With five, it is a standing exposure that changes every time a team adds a new model.

Identity and access. Separate vendors mean separate consoles, separate keys, and separate seat management. SSO and SCIM often cover the first vendor and none of the rest. The IBM 2026 Cost of a Data Breach report found that 92% of organizations that suffered an AI-related security incident lacked proper AI access controls when it happened.

Audit. A question like "Which model saw this customer's record, who asked, and when?" requires a single log across providers. Only a third of Kiteworks' respondents have tamper-evident audit trails at all, and half cannot produce access records within a day. Fragmented logs across vendors are the reason.

Cost. Managing five vendors means five invoices, each with token pricing, none attributed to a team or a use case. The Harris Poll's CIOs say cost is the top reason they switch models, but switching intelligently requires knowing what each model costs per team and per feature, which most organizations cannot see.

Continuity. Models get deprecated, rate-limited, and occasionally go down. With a single vendor, an outage is an incident. With a governed multi-model setup, it should be a failover. Most organizations are somewhere in between: multiple vendors, but no defined backup path.

Switching cost. Multi-model was supposed to reduce lock-in. In practice, a16z's interviewees reported the opposite: because "prompts have been tuned for OpenAI," moving a workflow takes "lots of engineering time" to re-tune guardrails and multi-step logic. Without an abstraction layer, every model is its own lock-in.

The pattern is the same in each case. The control exists in principle. It was implemented for one vendor. The second through fifth vendors run outside it.

What the gap costs

The 2026 IBM Cost of a Data Breach report, covering 602 breached organizations between March 2025 and February 2026, is the most direct evidence that this is a financial issue rather than a compliance nicety.

IBM Cost of a Data Breach 2026 Figure
Global average breach cost $4.99M, up 12% year over year
Breached organizations with a shadow AI incident 43%, up from 20% the prior year
Average cost of a breach involving shadow AI $5.39M
Shadow AI incidents that led to a regulatory fine 21%
Breached organizations with AI policies in place 32%, down from 37%
Organizations that coordinate AI governance with security teams 19%

That last line matters. Even where an AI policy exists, fewer than one in five organizations connect it to the people who can enforce it.

Shadow AI is not going to be trained away. Teramind's 2026 Shadow AI Report compiles worker surveys in which 48% say they would keep using their preferred AI tools even if explicitly banned, 51% report receiving conflicting AI usage guidance, and a third say IT simply does not offer the capabilities they need. That is a demand signal. People use unsanctioned models because the sanctioned option is worse, slower, or absent.

Regulators, meanwhile, have delayed some deadlines but not removed them. The EU's Digital Omnibus, approved by the Council on June 29, 2026, pushed the AI Act's Annex III high-risk obligations from August 2, 2026 to December 2, 2027. The Article 50 transparency obligations were not moved and took effect on August 2, 2026. Colorado's AI law was rewritten in May 2026 into a narrower disclosure framework, now effective January 1, 2027. Each delay has been paired with a narrower, more transparency-focused rule, which means the practical obligation is converging on the same thing: know which AI system did what, and be able to show it.

Gartner's March 2026 data and analytics predictions put the near-term risk in one sentence: "Ungoverned decisions using LLMs will cause financial or reputational loss for enterprises." Its longer-range prediction is that by 2030, half of AI agent deployment failures will trace to insufficient governance enforcement at runtime.

What governed multi-model looks like

The market has started to answer this. Kiteworks' finding that 43% of organizations have centralized AI traffic through a single gateway is low, but it is the fastest-moving number in the survey. Palo Alto Networks bought Portkey, an AI gateway, in a deal announced April 30, 2026 and closed a month later, on the explicit reasoning that the gateway is where AI spend, security, and observability now converge.

Tooling alone does not close the gap, though. From our own work with healthcare, financial services, life sciences, and software companies, a governed multi-model setup has eight parts. Most organizations have two or three.

  1. One front door. Every model, from every vendor, is reached through the same interface or gateway. If a model can be used without passing through it, the rest of the list cannot be enforced.
  2. One identity layer. SSO and SCIM govern who can use AI at all, with 2FA enforced. Offboarding a person removes their access to every model at once, not one console at a time.
  3. An owned model catalog. An administrator, not each user, decides which models are available, which is the default, and how they are ordered. Adding a model is a configuration change with an approver, not an ad hoc credit-card signup.
  4. Data-class routing rules. Clear policies establish which categories of data may reach which models—decided before rollout and enforced by routing logic rather than by memo.
  5. Connectors that are off by default. Company data reaches a model only through integrations the user has explicitly authorized and that IT can inspect and revoke.
  6. A unified audit trail. A tamper-evident log records who asked what, of which model, and with which data—kept in one place across all vendors.
  7. Granular cost attribution. Spend is tracked internally by model, user, team, and feature—rather than relying on opaque, aggregated vendor invoices—so leaders can see real ROI and act on anomalies.
  8. A defined failover. Each primary model has a designated backup that takes over automatically before visible output if the primary fails, turning a vendor outage or deprecation into a routine routing event rather than a business interruption.

Two principles underpin this list. First, none of it depends on picking the "right" model. The best model changes every quarter. Governance that only works for one vendor is governance that expires. Second, this is not a hurdle to adoption. The organizations with the worst shadow AI problem are the ones whose sanctioned option is unusable. A governed front door with good models in it is how you win people back from the unsanctioned ones.

How we approach it at IQNOVO

Our AI adoption services practice starts every engagement the same way: governance comes before tools. We assess what is already in use (usually more than the client expects), establish data-handling rules and platform risk reviews, prove priority use cases in a sandbox, and only then select and roll out a platform. Evaluating platforms against an existing policy is a very different exercise from starting with a vendor demo.

We built IQ Chat because we kept needing the same thing on the product side. It is a governed AI workspace that gives teams Claude, GPT, Gemini, Llama, and hundreds more models in one interface, switchable per conversation, behind a unified layer for identity, visibility, and control. Administrators own the model catalog, defaults, and routing policies. Each model has a designated backup that takes over before visible output if the primary fails. Administrative actions are secured in a tamper-evident, hash-chained audit log, while detailed session records track usage and calculated cost by model, user, and feature. Microsoft 365 and Google Workspace connectors are authorized per user and stay off by default until explicitly enabled. SSO, SCIM user provisioning, and 2FA are included, and organizations with strict isolation requirements can choose a dedicated deployment.

None of that is exotic. It is the eight-item list above, implemented once, so that the answer to "Which models are we using, who is using them, and what did they cost?" is a report rather than an investigation.

The short version

Multi-model is not a strategy most companies deliberately chose. It is where they ended up, and the data shows it is the right place to be. What is missing is the operational layer that makes it safe: an inventory, an owner, one identity, a unified audit trail, granular cost visibility, and a defined failover plan when a model goes away.

If your organization uses more than one model and cannot produce a complete record of who used which one last week, you are in the majority. That is the problem the 2026 numbers describe—and it is an entirely solvable one.

Learn more about IQ Chat or get in touch with our AI adoption team to establish a governed multi-model setup for your organization.

Sources

From the field

Keep up with the work.

Occasional notes on data, automation, and applied AI, sent when we have something useful to share.

By subscribing, you agree to receive occasional IQNOVO updates. You can unsubscribe at any time. Privacy policy

Next step

Want this kind of work done for your team?

Start with the business problem and we will help determine the delivery path that makes the most sense.

Book intro call