Operational databases and the lakehouse: the two memories of an AI-native system

An order-to-cash assistant goes live. It answers questions about open orders, credit limits and late invoices, and it reads them straight from the ERP’s production database, because that is where the truth is. Within a month the database team has throttled its queries, the assistant is reading from a replica that is twenty minutes behind, and a customer asks why it quoted a limit that was changed last Tuesday. Nobody can say what the assistant saw at the time, because the table it read no longer looks like that.

Nothing in that story is a model problem. The system was given one memory and asked to do the work of two.

Why one memory is not enough

The operational database under an ERP, CRM, WMS or MES is a system of record. It is built to hold the current state of the business and to change it safely: one order, one lock, one transaction, indexes tuned for finding a single row fast. That is exactly what it should be, and nothing in this article suggests replacing it.

An AI-native system asks the same data different questions, and each of them strains the system of record in a different way.

Scale of reading. An agent preparing a credit decision reads thousands of rows across orders, invoices and payments. A model that scores delivery risk reads millions. Analytical reads of that size compete with the transactions the database exists to protect, which is why the first response of any database team is a throttle.

History. The database holds what is true now. Most questions an agent is asked are about change: what the limit was when the order was accepted, how long this supplier has been slipping, whether the margin on this customer has been falling for three quarters. A system of record overwrites; a second memory has to keep the versions.

More than one system. The order lives in the ERP, the complaint in the CRM, the pallet in the WMS, the machine stop in the MES. The question “why is this customer unhappy” crosses all four, and no single production database can answer it without becoming the integration layer it was never designed to be.

Data that is not rows. Contracts, drawings, inspection photos, emails. A retrieval system over documents needs them next to the structured records they refer to, with the same identifiers and the same access rules.

What the agent saw. When an answer is challenged, by a customer, an auditor or a regulator, the question is not only “was the answer right” but “was it right given the data at that moment”. That requires a snapshot of the data as it was, which an overwritten table cannot give back.

A reporting replica solves the first point and none of the others. A warehouse in a vendor’s closed format solves several, at the price of a second copy of the truth that only one engine can read. The second memory an AI-native system needs is a lakehouse: tables in open formats on object storage, with transactions, history and a schema, readable by every engine that has to touch them.

What the second memory is made of

A lakehouse is three decisions, and they are separate ones.

The file format decides how one file lays out its bytes and therefore what a read costs. Parquet is columnar and is where analytical data lands; Avro is row-oriented, carries its schema and is what events look like while they travel between systems. The two usually appear together, one before the table and one inside it. The comparison is in Parquet or Avro: the contract between systems and the cost of the answer.

The table format decides which files make up a table at a given moment, what its schema and partitions are, and how several writers change it without corrupting it. Delta Lake, Apache Iceberg, Apache Hudi and Apache Paimon all do this over immutable Parquet files by keeping versioned metadata beside them; they differ in how that metadata is kept and in which engines and platforms read it natively. This is the decision that gives the lakehouse transactions, time travel, DELETE for a data-subject request and schema changes without rewriting data. How to choose is in Delta Lake, Iceberg or Hudi: choose by catalogue, not by feature list.

The catalogue decides which tables exist, where their current metadata is and who may read them. Unity Catalog, Apache Polaris, Lakekeeper, AWS Glue or the Fabric catalogue sit here. Open files under a closed catalogue are still half-owned by a platform, and permissions are the part of the design that the open formats do not carry for you.

Infographic

Two memories, one system

1. 01Agents and applicationsRead the second memory; act on the first only through its own APIs.
2. 02Semantic layer and ontologyNames, relations and definitions the whole company agrees on.
3. 03Lakehouse tablesOpen table formats over Parquet: history, transactions, schema, time travel.
4. 04Change capture and event contractsCDC from the database, events between systems, schemas that are versioned.
5. 05System of recordThe operational database under ERP, CRM, WMS, MES. Current state, protected.

How data gets from the database to the lakehouse

The path from the system of record to the second memory is where most of the engineering lives.

Change data capture reads the database’s own transaction log, with Debezium or the platform’s equivalent, and publishes each insert, update and delete as an event. Those events travel through Kafka or a comparable log, each with a schema that is registered and versioned, which is what Avro is for. A processing job, in Flink or Spark, lands them in a table format on Parquet, where they become the versioned history the database never kept.

The question to settle before any of this is built is latency, and it should be settled per decision rather than for the whole platform. Three tiers cover almost every case.

Nightly. Management reporting, model training, monthly closes. A daily load is cheap to run, easy to test and easy to repair.

Minutes. Credit limits, stock promises, order status for a service desk. Change data capture into a table format, with compaction running behind it, is the usual design.

Seconds. Fraud, machine alarms, an agent that has to act while a customer is on the line. This tier is expensive in small files and compute, and it belongs to the few decisions that genuinely change within a minute. How the stream-first formats handle it is in The state your agents act on.

A platform designed for the seconds tier everywhere costs the seconds tier everywhere. Most enterprises find that one or two decisions need it.

Where agents read, and where they may write

The rule we design to is short. Agents read from the second memory, through the semantic layer, at whatever scale the question needs. They act on the first memory only through the interfaces the system of record already exposes to people, an order API, a workflow, a posting, with the same validation and the same permissions.

That rule has three consequences. The production database never sees an agent’s analytical load. Every action an agent proposes passes through the controls the business already trusts, and is recorded under an identity that can be audited. And the agent’s own decisions, what it was asked, what it read, what it proposed, who approved, become a table in the lakehouse like any other, with history, so that the question “what did the agent see” has an answer a year later.

AI SDLC for the data layer

Pipelines are software, and in an AI-native system most of their code will be drafted by coding agents. That changes how the code is written and does not change how it is accepted.

Specification and contract first. Before a pipeline exists, the data contract exists: the schema of the source event or table, the schema of the target, the rules for evolution, the freshness the consumer is promised, the owner on each side. An agent that is given the contract produces a usable pipeline; an agent that is asked to “ingest the orders” produces a plausible one.

Acceptance by tests on real samples. Generated pipelines are accepted the way any code is: tests over a sample of production data, row counts and reconciliations against the source, schema checks, and an evaluation set for the answers the downstream system gives. A pipeline that passes on the vendor’s sample and fails on yours was never accepted.

Versioning together. The schema, the transformation code, the prompt, the model version and the data snapshot that produced an answer are versioned as a set. Time travel in the table format is what makes the data part of that set reproducible, which is what lets a team roll an answer back, not just the code.

Rollback planned before the change. Every schema evolution, every change of a merge rule, every new source ships with the step that undoes it. In a table format that step is usually a restore to a previous version, which is cheap, provided somebody tested it.

AI in data engineering: what to delegate and what to keep

The same agents that write application code are now useful inside the data platform, and the line between what they do well and what they must not decide is clearer than the marketing suggests.

Agents are good at drafting a pipeline from a contract, proposing a schema evolution when a source adds a field, explaining why a load failed by reading the log and the lineage, planning compaction and vacuum windows from table statistics, and writing the tests described above. They do this faster than people and with fewer slips, and they are tireless about documentation.

Engineers and owners keep the decisions that bind the company: whether a contract may change and who pays for the consumers it breaks; retention and deletion, especially for personal data; which table format and which catalogue the company commits to; which latency tier a decision really needs; and whether an agent’s proposed fix is a fix or a workaround that hides a source problem. An agent that proposes is an asset. An agent that silently changes a contract at two in the morning is an incident.

What a good outcome looks like

The system of record is quieter than before, because nothing analytical reads it. One table in the lakehouse is the agreed history of each business object, and any engine the company uses can read it without a copy. A question about an answer the assistant gave in March can be reproduced with the data as it stood in March. Every pipeline has a contract, an owner, a test suite and a rollback, and most of its code was drafted by an agent and accepted by a person. The number of copies of the same data in the company goes down, and the first published freshness figure is usually worse than anyone assumed, which is the moment the budget conversation changes, as described in Data work gets funded for answers, not for platforms.

Series map

This article is the entry point to a series on the data layer of AI-native enterprise systems. The comparisons: Parquet or Avro, Delta Lake, Iceberg or Hudi, the state your agents act on: Paimon, one copy of the data, three formats: XTable and UniForm. The profiles: Apache Avro, Apache Parquet, Delta Lake, Apache Iceberg, Apache Hudi, Apache Paimon.

Our part

In AI-native apps engineering we build the system around the two memories: the agents and applications that read the lakehouse through a semantic layer and act on the system of record through its own interfaces, with the contracts, tests and decision log that make the whole thing auditable. We do not resell platforms. More about predictes is on the home page.