AI-Ready Data Does Not Mean Consolidated Data

AI-ready data means an agent can resolve a business question against governed, current data wherever that data lives. It does not mean the data has been consolidated into a single warehouse first. Readiness is a contract: consistent definitions, permissions that travel, and access to every data shape you own.

Quick Summary (TL;DR)

• Most AI-readiness guidance describes a consolidation project: land everything in one lakehouse, model it, then point the agent at it.
• Gartner predicted in February 2025 that through 2026, organizations would abandon 60% of AI projects unsupported by AI-ready data, based on a 2024 survey of 248 data management leaders.
• Consolidation does not make data AI-ready on its own, and for some data shapes it destroys the exact information the agent needed.
• Microsoft’s own documentation states that Direct Lake tables do not support complex Delta column types and that you must convert them into strings, which is the flattening step teams adopted a lakehouse to escape.
• Elastic’s mapping reference explains that flattening arrays of objects loses the association between values inside each object, which produces answers that are wrong rather than merely slow.
• The second failure is semantic drift: every BI tool, warehouse and agent framework now ships its own modelling layer, so the same metric gets defined in four places and diverges.
• Four properties make data AI-ready: definitions that resolve in one place, permissions that travel with the dataset, native access to non-tabular sources, and model portability. None of them is a storage location.
• Use the seven-question readiness test below to find which of the four your estate actually fails.

Table of Contents

What Does AI-Ready Data Actually Mean?

AI-ready data is data an agent can use to produce an answer somebody will sign their name to. That requires the agent to know which definition of a metric is the approved one, to be restricted to the rows the requesting user is allowed to see, and to reach every source that holds relevant evidence. Storage location is not part of the definition.

This is a meaningful departure from how readiness was framed for traditional BI. A BI tool needed data in a shape it could read, so readiness genuinely was a movement problem. An agent needs data it can reason about correctly, which is a semantics and governance problem that movement does not address.

The gap shows up in the failure rate. Gartner’s February 2025 analysis predicted that through 2026, organizations would abandon 60% of AI projects unsupported by AI-ready data, drawing on a 2024 survey of 248 data management leaders. The instructive part is that many of the stalled programs did the consolidation work. They centralized, they modelled, they catalogued, and the agent still returned answers nobody would put in front of a regulator.

Why Do Enterprise AI Projects Stall After the Data Is Centralized?

A mature enterprise data platform in 2026 tends to look similar from the outside. Several years of investment, a lakehouse on Delta or Iceberg, a migration toward serverless and fully managed services, an established BI estate, and generative AI already running in production somewhere in the business. By every conventional scorecard the consolidation work is done.

The agent programme still stalls, and it stalls on four specific things.

The data that will not flatten

Operational estates contain document stores, search indices, time series and event streams. That data is nested by design, and reporting on it through a tabular tool requires collapsing it. The collapse is the hop teams are trying to eliminate, and it survives most modernization programmes intact.

Semantic definitions rebuilt per tool

Every layer of the stack now ships its own modelling layer. The BI tool has one, the warehouse has one, the transformation framework has one, and the agent framework has one. The same metric is therefore defined four times, maintained by different people, and the four definitions drift apart quietly.

The model is wired into the data logic

When semantics, joins and access rules are expressed inside a specific vendor’s AI feature, changing model becomes a migration. Most enterprises are already on their second or third model decision, so this is not a hypothetical cost.

Analytics moved into operations

Decisions that used to run on yesterday’s batch now run on current state, monitored continuously, often across more than one region because disruption is treated as expected rather than exceptional. A readiness architecture built on nightly ingestion cannot serve that, no matter how well governed the destination is.

None of the four is a storage problem. Every one is a contract problem, which is why adding more consolidation does not resolve them.

Why Does Consolidating Data Break AI Accuracy?

The case against warehouse-first as an AI-readiness strategy is not that centralizing is slow or expensive, though it is frequently both. It is that the conversion step discards information the agent needed, and does so silently.

What flattening actually destroys

Consider a record holding an array of related objects, which is the shape most operational, event and log data arrives in. Take a document with two people on it:

{
  "group": "operations",
  "user": [
    { "first": "John",  "last": "Smith" },
    { "first": "Alice", "last": "White" }
  ]
}

Indexed or loaded as a standard object rather than a nested type, that structure is internally flattened into parallel lists of values. The document becomes, in effect:

{
  "group":      "operations",
  "user.first": [ "john", "alice" ],
  "user.last":  [ "smith", "white" ]
}

The two names are still present. The fact that John belongs with Smith and Alice belongs with White is gone. Elastic’s own nested field type documentation spells out the consequence: a query for Alice AND Smith now matches this document, even though no such person exists in it.

That is not a performance issue or a rounding error. It is a false positive produced by a correctly executed query against correctly loaded data. Ask an agent how many operations records involve Alice Smith and it will answer one, show its working, and be wrong.

The same class of loss applies to any structure where the relationship between fields carries meaning. Line items inside an order, events inside a session, readings inside a device payload, parties inside a transaction. Flattening preserves the values and deletes the relationships, and relationships are most of what an analyst was asking about.

The lakehouse does not fully solve this

Teams generally assume Delta and Iceberg removed this constraint, because both formats support complex types. The formats do. The reporting layer often does not.

Microsoft’s Direct Lake documentation states it directly in its considerations and limitations: “Direct Lake storage mode tables don’t support complex Delta table column types. Binary and GUID semantic types are also unsupported. You must convert these data types into strings or other supported data types.” The limitation applies to both Direct Lake on OneLake and Direct Lake on SQL endpoints.

So a team can migrate to Delta, adopt Iceberg, go serverless, and still be instructed to stringify their complex columns in order to report on them. The flattening hop survived the modernization. It moved from the ingestion layer to the semantic model layer, which is harder to see and easier to forget is there.

Why Does Every Tool Have Its Own Semantic Layer?

Because each vendor solved the same problem locally. A BI tool needs to know what revenue means to render a chart. A warehouse needs it to serve a metrics API. An agent framework needs it to stop the model reading raw schema, which is the failure examined in why AI analytics fails without governed data layers. Each built its own, and each is individually reasonable.

The result at enterprise scale is that a semantic layer is no longer a differentiator, it is an accumulation. Most large estates now hold three or four, none of which is authoritative, and the drift between them is invisible until two executives bring different numbers to the same meeting.

Agents make the drift consequential rather than annoying. A human analyst who gets an odd number checks it against what they know about the business. An agent has no such prior. It returns whichever definition its layer holds, with equal confidence either way, at a volume no human is reviewing.

The consolidation worth doing here is of definitions, not of storage. One resolution path that dashboards, embedded applications, APIs and agents all query against, rather than one physical location that everything is copied into.

Why Does Model Portability Matter for the Data Layer?

Because model choice is now the fastest-moving decision in the stack, and the data layer is the slowest. Coupling them means the slow layer gets rebuilt at the speed of the fast one.

When semantics live inside a model integration rather than beneath it, a model change means rebuilding, at minimum: the schema descriptions embedded in prompts, the few-shot examples that taught the previous model your join paths, the glossary mapping business terms to columns, the access filters expressed as prompt instructions rather than enforced policy, and the evaluation suite that proved the whole thing was accurate. None of that transfers.

When semantics live in the data layer, the model is a setting. The definitions, joins, permissions and lineage stay where they are, and the reasoning engine in front of them is swapped without a data project. This is the same argument as why an MCP server needs a semantic layer beneath it, generalized past any one protocol: the model and the protocol are both replaceable, and the data contract should outlive both.

Want to see how AI agents work with your data across SQL, NoSQL, APIs and documents? Request a demo at knowi.com.

How Do You Know If Your Data Is AI-Ready?

Readiness is testable, and the test is not a maturity score. Run these seven questions against your own estate. Each has a pass condition that is either true or not.

1. Name the single place a core metric is defined. Pass: one system, and you can point to it. Fail: the answer starts with “depends which tool”.
2. Ask an agent for that metric and inspect the generated query. Pass: it resolved the certified definition. Fail: it read raw schema and guessed a column, which will be right often enough to be dangerous.
3. Have two users with different entitlements ask the same question. Pass: the answers differ correctly, because row-level security applied to the agent path. Fail: they match, or the pilot is running under a service account with blanket access.
4. Count the sources the agent can actually reach, then count the sources that hold decision-relevant data. Pass: those numbers are close. Fail: your document stores, search indices and vendor APIs are outside the perimeter and the agent will not tell you it is reasoning on a subset.
5. List every dataset that must be flattened, stringified, or pre-aggregated to become reportable. Pass: the list is empty or deliberate. Fail: the list exists and nobody has checked what the conversion discarded.
6. Write down what you would rebuild if you changed model vendor next quarter. Pass: a configuration change. Fail: the list from the previous section.
7. Take any agent-generated answer and trace it to source. Pass: you can show the query, the datasets, and the lineage. Fail: you can show the answer.

Most enterprises pass one and two after a semantic layer project, then fail three, four and five, because those depend on where governance is enforced and how many data shapes the layer can actually reach.

When Should You Still Centralize Data?

Federating everything is not a universal position, and arguing otherwise would be dishonest. Consolidation earns its place in several real cases.

Centralize when you need historical depth the source system does not retain, when the source cannot survive analytical query load, when regulatory retention requires an immutable copy, or when a single heavy aggregation is read thousands of times a day and recomputing it live is simply waste. Those are engineering facts, not ideology.

The honest framing is per dataset rather than per architecture. Some datasets should be queried live, some cached against a query fingerprint, some materialized on a schedule. The decision factors, including where federation genuinely breaks down, are worked through in federated analytics vs data warehouse. What changes with AI is only this: governance and semantics have to be identical across all three modes, because the agent neither knows nor cares which mode a given dataset is running in.

How Do the Approaches Compare?

Concessions worth stating plainly. Warehouse-native semantic layers are more mature at pure metric governance than anything federated, and if your data genuinely all lives in one warehouse they are the shorter path. Federation engines outperform on large distributed SQL workloads. The multi-source case only becomes compelling when a meaningful part of your estate refuses to be tabular.

How Knowi Solves This

Knowi puts the contract below the tools rather than inside one of them. Each source is queried in its own language, so a search index is queried with its native syntax and a document store keeps its nested structure, and the results are joined at query time into a reusable dataset. Nothing has to be flattened to become reportable, which removes the failure shown earlier rather than mitigating it. The approach is set out in more detail for search and operational store analytics.

Semantics, permissions, row-level security and lineage attach to that dataset, not to a downstream tool. The same definitions therefore resolve whether the consumer is a dashboard, an embedded application, an API call or an agent, which is what makes questions three and seven of the readiness test pass. Each dataset independently runs live, cached against a query fingerprint, or materialized on a schedule, and the governance is identical in all three modes.

On the model question, the platform ships its own AI engine that can run inside your own environment, or you supply your own model key and change it later. Either way the reasoning engine sits above the contract instead of holding it, so a model decision does not become a data project.

Where Knowi fits best

The strong fit is an estate where operational data spans document stores, search indices, REST APIs and relational systems at once, and moving all of it into one platform first is either lossy or organizationally impossible. Deployment matters at this tier, so the platform runs as managed cloud, inside your own cloud tenancy, or fully on-premises, with the AI engine inside your boundary in self-hosted deployments.

It is a weaker fit if your estate is already consolidated and uniformly relational, if you have a strong existing investment in a warehouse-native semantic layer that is working, or if your primary requirement is raw distributed SQL throughput rather than heterogeneity.

Frequently Asked Questions