The state your agents act on: Paimon between the database and the report

An agent that prepares credit decisions for a distributor’s sales desk goes live on a Monday. By Wednesday the database administrator asks who is running the same query against the ERP every fifteen seconds. It is the agent. It was told to use current data, and the only place current data existed was the production database.

That is the usual first design of an operational agent, and it fails in two directions at once. It loads the system of record with reads it was never sized for, and it still answers from a state that is a mixture: part ERP, part what the warehouse system knows, part what the agent remembers from its last call. The operational database and the lakehouse are two different memories of one company. An operational agent needs a third thing between them: a table of current state, fed from the database, that it can read as often as it likes.

Why the agent must not poll the production database

The operational database is tuned for transactions: short writes, indexed lookups by key, locks held for milliseconds. An agent asks different questions. It wants the state of one customer joined with open orders, recent returns, the credit limit and the payment history, and it wants the same for the next customer a second later. On the OLTP side that is a wide join repeated without end.

Three things go wrong. The database slows down for the people who actually book orders. The agent sees a state that is correct for one system and ignores the others, because the warehouse system and the e-commerce platform have their own databases. And nobody can say afterwards what the agent saw when it made a given suggestion, because a production database has no memory of its own past states.

A read replica solves the first problem and none of the others.

The path from a change to a state

The pattern that holds up in enterprise systems is to let the database announce its changes and to build the state somewhere else.

Change data capture. A connector such as Debezium reads the database’s own transaction log and emits one event per changed row: order inserted, status updated, limit changed. The database does not run any extra query for this; it is already writing that log.

An event contract. Each event type has a schema, kept in a schema registry, with compatibility rules. This is the point where a field added by the e-commerce team does not break the agent downstream. Apache Avro is the usual carrier here; we cover it in its own article.

A stream processor. Apache Flink consumes the events, joins them by key and writes the result into a table. The join is where “customer state” is assembled from the ERP, the warehouse system and the shop.

A primary-key table. Apache Paimon stores that result as a table keyed by customer or order, organised as a log-structured merge-tree: new versions of a row land in a small top level, and compaction merges them downwards in the background. A row that changes every minute does not turn the table into a pile of small files, and a reader always gets the latest merged version. The table is also a changelog source, so the next step in the chain can react to changes rather than scan.

A lookup join. Flink can look up a Paimon table from inside a stream, so the enrichment of a new order with the customer’s current state is one operation, not a call to the database.

Infographic

From a database change to an agent's decision

1. 01Change in the system of recordAn order status or a credit limit changes in the ERP; the transaction log records it.
2. 02Change data captureDebezium reads the log and emits one event per changed row.
3. 03Event contractThe event carries a registered schema with compatibility rules.
4. 04State table in Apache PaimonFlink joins events by key into one primary-key table of current state.
5. 05Agent reads and decidesThe agent reads the state table and prepares a suggestion, with the snapshot it used.
6. 06Write-back through the APIThe accepted decision goes to the system of record through its own interface, under a named user.

Which decisions need seconds

The chain above can deliver a state that is seconds behind the database. Before building it, decide which decisions need that. In most companies the list is short.

A credit check at order entry needs the current limit and today’s open orders: seconds matter, because the salesperson is on the phone. An allocation of scarce stock between channels needs minutes. A recommendation of what to offer a customer next week needs yesterday’s state and is better served from the lakehouse’s history than from a live table.

Building everything at the fastest tier is the common mistake. Each tier down costs less: fewer events to process, less compaction, a simpler failure mode. Write the latency requirement into the specification of each agent decision, with the person who confirmed it, and let the slow ones be slow.

What the agent reads and where it writes

The state table is read-only for the agent. It reads it as often as the decision requires and records, with every suggestion, the snapshot identifier of the table it used. Paimon keeps snapshots and tags for exactly this: a tag pins the files of a snapshot so that a state from last Tuesday can be read again after newer snapshots have expired.

Writing is a different matter. The agent’s output is a proposal: release the order, hold it, ask for a prepayment. When a person accepts it, the change goes into the system of record through its own API or its own screen, under a named user, with the agent’s reasoning attached. The state table then learns about the change the same way it learns about every other change: through the transaction log. There is one writer of truth, and the agent is not it.

This closes the loop that polling never closed. The agent sees the consequence of its own accepted suggestion through the same path as everything else, so its next decision starts from a consistent state.

Real time as a goal in itself

A state table that is seconds fresh invites a dashboard that refreshes every five seconds, an alerting rule on every change, and an agent that re-evaluates every customer on every event. None of these is a business decision. Each adds load to Flink and compaction to Paimon, and each makes it harder to say what the system was doing at a given moment.

Keep the fast path for the decisions that need it. Everything else reads from the lakehouse, where the same events arrive in Parquet under a table format and where history, not freshness, is the point. How Paimon itself works inside, and what it costs to run, is in the profile of Apache Paimon.

How this is tested

The state table makes a kind of testing possible that a polling agent never had. A tag on the Paimon table and a day’s worth of events from the topic are a complete, repeatable fixture. A new version of the agent can be run against that fixture, its suggestions compared with the previous version’s and with the decisions people actually made, before it sees a live order.

In an AI development lifecycle this becomes a standard step: the evaluation set is a snapshot plus a replay, the acceptance criterion is a difference report, and the release note says which tag the comparison was run against. Coding agents are good at generating the replay harness and the comparison queries. The decision about what difference is acceptable stays with the owner of the process.

Our part

In AI agents engineering we build the part between the database and the agent: the event contracts, the state table and its latency tiers, the read path with snapshot identifiers, and the write-back through the system of record under a named user. We do not resell platforms. More about predictes is on the home page.