After Every Agent Rewrite, What's Actually Worth Keeping?

This article introduces Alibaba Cloud's EventHouse as a unified, real-time semantic data pipeline that provides AI Agents with actionable, trustworthy business context.

The pace of change in Agent development has outrun what many teams expected.

Many teams have lived the same arc: hand-rolling an Agent Loop, then adopting frameworks like LangChain and LangGraph, then turning to model vendors' Agent SDKs and Harness Runtimes. Every leap in model capability seems to bring another architectural adjustment.

That doesn't mean earlier choices were wrong — the underlying capabilities simply changed too fast. The Planner, Memory, Router, retries, and state management teams once built themselves are becoming common capabilities of the model or platform. So a practical question: if models, frameworks, and runtimes keep changing, what part of the Agent you invest in today actually endures?

Our answer: orthogonal growth — keep pace with general intelligence, but put long-term investment in your own business context.

1、Preserve business context instead of redundantly building general-purpose intelligence

1.1 Two Growth Axes: General Intelligence Forward, Business Depth Upward

Agent development splits into two growth axes:

The horizontal axis is general intelligence — Memory, Planner, Managed Runtime, Agent collaboration. Model vendors and platforms invest here continuously; enterprises should keep pace and reuse rather than rebuild what is becoming a common capability. But capabilities on this axis are not necessarily durable enterprise assets: what a business team builds itself today may be a default setting of the model or platform tomorrow.

The other axis runs orthogonal to it — the vertical axis of business depth. It comprises:

• the business facts an enterprise continuously generates;

• the relationships among business objects;

• a unified business semantics internal to the enterprise;

• concrete processes and rules;

• industry knowledge and business experience.

None of this appears automatically with a model upgrade, and no model vendor can produce it for you. Models can help organize, infer, and generate candidates, but your own business definitions, data relationships, and process boundaries must be built up inside the business itself.

This is "orthogonal growth": model vendors push forward along general intelligence while the enterprise climbs along business depth. The more perpendicular the two axes, the larger the "area of intelligence" they enclose.

To judge whether an investment is worth making long term, ask one question: after the next model upgrade, will it be replaced — or amplified?

1.2 Business Context Must Come from Continuously Running Data

Given the importance of business depth, the next question is: where does that context come from?

Not from a static document compiled once, but from the data an enterprise generates every day as it runs. Orders and transactions keep occurring, customer behavior keeps shifting, devices keep emitting events and alerts, tickets keep moving, applications and logs keep growing, and external SaaS and APIs keep returning new information.

Business facts have three characteristics:

They come from systems actually running, not material assembled after the fact. Every order step, customer follow-up, device alert, and process transition generates a new fact.
A single answer usually requires several systems together. A simple question may span customer, order, inventory, fulfillment, service, and after-sales at once. Look at one system alone and the picture is often incomplete.
Business state never stops changing. An exported spreadsheet represents only the moment of export; as the business runs on, it drifts from reality. Even documentary knowledge needs continuous maintenance and updates.

So for an Agent to reach into the business, it must see what is happening in the enterprise right now.

What needs building is a continuous data-supply "pipeline": always-on ingestion, multi-source confluence, incremental updates, cross-system composition, and refresh as business state changes. A one-time import is not continuous business context.

2、Messaging Services: The Natural Foundation for Enterprise Real-Time Data Integration

2.1 Why Messaging Services Are Naturally Suited to Data Integration

Mention "pipeline" and most people think of the Linux pipe: one process feeds its output into another's input. The first doesn't need to know the second; the second doesn't care how the data was produced; the two cooperate through the pipe.

A messaging service extends this model from a single machine to a distributed environment: one end can be a business application, database, SaaS, log system, or IoT device; the other a business system, data platform, analytics tool, or Agent.

Three reasons make it well suited to continuous data integration.

1. Append-only writes, fast transport, decoupled upstream and downstream

Messaging services typically use an append-only log: data is continuously appended, the write path is short, and in theory it sits closest to the hardware's performance ceiling — ideal for high-throughput streams. Producers only write; consumers read at their own offset and pace; neither waits on the other. Upstream doesn't need to know how many consumers exist downstream; downstream isn't bound to upstream's cadence.

2. Flexible ingestion, built for heterogeneous systems

Upstream can Put (write actively) or the service can Pull from it; downstream can Pull (read actively) or the pipe can Push to it. This lets a messaging service connect databases, SaaS, logs, IoT, business applications, and external APIs alike. Whether a source offers data actively or can only be called through an interface, there's a fitting ingestion mode.

3. Data can be buffered, subscribed to, and replayed

Messages don't have to be forwarded instantly. The pipe can retain data for a window — absorbing traffic spikes and buffering speed differences between upstream and downstream. The same data can be subscribed to by many consumers: ingest once, use many ways. On failure, it can be replayed from a historical offset for retries, backfill, or letting a new system catch up.

A messaging service's "integration capability" isn't a feature bolted on later — it's intrinsic.

2.2 Messaging Services: Now Integrating Data for Agents

Today, messaging services such as RocketMQ, Kafka, and EventBridge already have a rich connector ecosystem.

Upstream connects business applications, ERP, CRM, transaction systems, databases, files, object storage, microservices, APIs, log systems, and IoT and edge devices.

Downstream connects business consumption, data lakes, real-time warehouses, ETL, BI, risk control, recommendation, AI, monitoring, audit, and security-governance systems.

Every new data source reuses the existing downstream; every new consumption scenario reuses the upstream already connected. Upstream doesn't need to rebuild a pipeline for each downstream; downstream doesn't need to interface with every source separately. Along the way, messages can be filtered, cleansed, reformatted, enriched, and routed.

Messaging services are thus upgrading from transport components into the infrastructure for enterprise real-time data integration — organizing once-siloed products into a single real-time data network, on which Agents can become an important class of consumers. Today, though, this pipeline is usually "indirect":

Reading left to right: business and transaction data, documents and logs, device and event data, and external-service data first enter the messaging service.
The messaging service acts as the "data-supply hub," filtering, cleansing, transforming, and routing by scenario — syncing on demand.
Agents typically don't read the raw stream directly; they need data they can "query and understand" around a specific task. So the data flows on into different data services:

• business state and precise records go to relational databases;

• documents and logs go to search or document stores;

• semantic retrieval and RAG use vector databases;

• metrics and trends go to analytical databases.

These data services expose themselves to different Agents in a standardized way through an MCP Server — so Qoder, ChatGPT, Claude, or a home-grown enterprise Agent can all query the relevant data on demand.

But is accessible data the same as data an Agent can use?

3、The Data Is Connected — Why Do Agents Still Use It Poorly?

3.1 Accessibility ≠ Usability: What’s Missing for Agents to Truly Leverage Data?

Accessible data is not data an Agent can use.

After messaging, processing, filtering, transformation, syncing, and distributed storage, new breakpoints can appear in schema, semantics, lineage, and timeliness. An Agent needs more than a value; it needs to know what the value means, where it came from, what transformations it underwent, when it was updated, whether it's trustworthy, and whether the current task may use it.

If that context doesn't arrive with the data, several things go wrong:

• data scattered across databases — the Agent can't find it;

• inconsistent field structures, naming, and business definitions — the Agent can't interpret it;

• source and processing hard to trace after filtering, transformation, and syncing — the Agent can't trust it;

• real-time and offline data hard to align, freshness and consistency can't be guaranteed — the Agent can't use it accurately.

Traditional integration, in short, solves "is the data connected," but not yet "can the Agent find, understand, correlate, and trust it."

On this, we hold three judgments.

3.2 Three Judgments: Semantic Propagation, Unified Understanding, Pipeline Consolidation

Judgment 1: Source data semantics must propagate and evolve all the way through

A data pipeline typically passes through several stages: source system, message ingestion, processing and transformation, sync and storage, and finally delivery to the Agent via MCP or another interface.

The source system defines business entities and the initial schema; in the messaging system, event meaning must be recognized and schema registration, version management, and compatibility checks completed; during processing, fields are mapped, transformed, and enriched, producing new results and transformation records; on sync to storage, scenario-specific tables and fields must be designed; and at the MCP and Agent end, data must become queryable, comprehensible business semantics.

Information can be lost at every handoff.

So each stage should deliver not just data but its definition, version, transformation history, and a trustworthy explanation. Every handoff must confirm three things: consistent meaning, compatible schema, traceable change.

This isn't a one-time job. When a source field changes, the semantics, mappings, storage, and access contracts must all keep evolving along the pipeline.

Judgment 2: Storage can stay distributed, but semantics and retrieval must be unified in advance

Distributed storage is a fact of enterprise life.

Real-time events belong in the streaming system, transactions in the business database, history in the lakehouse, documents and logs in search systems, vectors in vector databases, and some data stays with external SaaS or APIs. Each system has its own freshness, cost, performance, permissions, and accountability boundaries. Copying everything to one place is neither realistic nor always necessary.

But physically distributed data doesn't mean understanding of it can be distributed too. Before an Agent calls, the platform must continuously build a unified understanding of the data:

• a Catalog so data can be found;

• Schema and Semantic layers so data can be understood;

• Entity and Lineage to correlate cross-source entities and relationships;

• Freshness, Quality, and Policy to judge whether data is fresh, trustworthy, and who may use it.

This isn't centralizing copies of data; it's unifying the catalog, structure, semantics, relationships, and trustworthy context in advance. Why in advance rather than interpreting on the fly once the Agent gets a question? Because when an Agent responds, time and context window are limited. It can't traverse every system each time to re-guess field meanings, re-check business definitions, re-trace lineage, and re-judge permissions.

On-the-fly interpretation is not only slow; the same question can yield different interpretations, and risk becomes harder to control. The sounder approach: turn "interpret at call time" into "build up understanding before the call," then fetch trustworthy context on demand through a unified MCP interface.

Judgment 3: The whole integration pipeline will keep consolidating

Stack the first two judgments and the third follows: the whole integration pipeline will keep consolidating.

Not because enterprises must merge every system, but because the cost of stitching many segments keeps rising:

Context breaks easily. Every handoff may require re-translating semantics, definitions, and access contracts; any omission amplifies misunderstanding.
Unified management is hard. With Catalog, Schema, and business semantics spread across systems, the same data easily accrues multiple interpretations.
Copies and alignment multiply. Each stage spawns new copies, real-time and offline keep syncing, and freshness and consistency grow harder to guarantee.
Accountability boundaries shatter. A problem may lie in messaging, processing, storage, query, or the Agent, but diagnosis spans many systems, and no single service owns the result.

So consolidation isn't about forcing all data into one place; it's about letting one service own the data end to end — from ingestion, to understanding, to use.

4、EventHouse: One-Stop Real-Time Data Service for Agents

4.1 Architecture: Message Foundation + Unified Metadata Layer

Building on these judgments, EventHouse's thesis: message integration as the data foundation, a unified metadata layer running through four modules — processing, storage, query, and Agentic retrieval — continuously supplying data to mainstream Agents.

From the bottom up, the base layer is message integration. Upstream can Put, or the platform can Pull from it; downstream can Pull, or the platform can Push to it. Existing ecosystems — Kafka, RocketMQ, MQTT, RabbitMQ, EventBridge — all keep plugging in.

Once data enters, the point isn't to copy it again but to let Catalog, Schema, Semantic, and Lineage accumulate continuously in the unified metadata layer. Then, with each processing or storage step, downstream stages don't need to re-explain the data's meaning from scratch.

The four modules share this metadata:

• Processing: use EventHouse, or flow out to other components;

• Storage: use built-in storage, or open lake formats that stay on the customer's side;

• Query: query EventHouse data, or federate external sources through Trino;

• Agentic retrieval: expose to different Agents via API, CLI, MCP, or IM.

This "one-stop" experience isn't a closed rip-and-replace. Processing results can flow to external components, data can stay in the customer's own object storage, queries can federate existing databases, and the unified metadata can be opened to external systems and Agents.

4.2 The Semantic Layer: Turning Business Definitions into Executable Ones

What truly connects data and Agents isn't an interface — it's the semantic layer.

The semantic layer isn't a field manual. It must distill business understanding into definitions that aid matching, correlation, and execution.

EventHouse divides it into five kinds.

Object semantics: table and column descriptions, business names, synonyms;
Scope semantics: which sources the current scenario selects, which irrelevant or ambiguous fields to hide;
Relationship semantics: join conditions, left and right tables, cardinality;
Compute semantics: Measure, Filter, and executable Field SQL;
Value semantics: Format Assistance and Entity Matching.

Take "show net revenue for paid orders in Beijing this quarter." The Agent must at minimum pin down:

• which metric or field "net revenue" maps to;

• which region code "Beijing" maps to;

• what filter "paid" corresponds to;

• what formula computes net revenue;

• how the order and customer tables join, and the cardinality.

For example, "Beijing" can map to region_code = 'BJ', "paid" to payment_status = 'PAID', and net revenue to a confirmed SUM(order_amount - discount_amount). The more explicit these definitions, the less the model must guess.

Names handle matching, SQL handles exact execution, and relationships plus representative values fill in context.

4.3 Balancing the Cost of Building and Using Semantics

EventHouse avoids Semantic Web Ontology or Knowledge Graph Schema approaches. The judgment: for customers these carry high build-and-maintain costs and, without dedicated FDE (forward-deployed engineer) support, often never get off the ground. Palantir's Ontology likewise depends heavily on white-glove, on-site engineering. Often it isn't the customer that chooses Palantir, but Palantir that chooses customers whose scenarios fit.

EventHouse takes a balanced path. We simplify building semantics: beyond structured definitions, customers can define semantics in weakly structured forms like a wiki — akin to a Skill, more flexible and free-form. And we place semantic understanding and alignment at the point of use, betting that LLM capability and context windows keep growing while LLM-side caching keeps driving down cost.

At the point of use, EventHouse first injects the trustworthy semantics relevant to the scenario — typically just a related set of tables (generally no more than 15), not the whole database. Then a ReAct loop refines the SQL against real execution feedback.

That loop has four steps:

• Reason: infer from business definitions and metadata;

• Act: generate SQL and actually execute it;

• Observe: read the returned rows or errorReason;

• Reflect: correct based on what was observed, and enter the next round.

This isn't the model mechanically retrying; every correction rests on trustworthy knowledge and real execution feedback. The semantic layer gets the first step as right as possible; ReAct corrects, with evidence, once it drifts.

4.4 Semantic Initialization: Inherit Facts First, Then Let AI Propose Candidates

When onboarding a new data source, semantics shouldn't start from a blank slate, nor should AI simply declare the business definitions. EventHouse initializes in four steps:

• Step 1: inherit the upstream schema, table and column descriptions, field types, and data-access scope. These are deterministic — inherit them as-is rather than making the model guess again.

• Step 2: sample representative values. From data formats and distributions, add Format Assistance and Entity Matching so the system learns how names, codes, and value patterns correspond.

• Step 3: discover definite relationships. Prefer existing primary keys, foreign keys, and cardinality in the schema; keep only joins backed by evidence.

• Step 4: only then let AI recommend candidate semantics — reusable SQL Expressions, or supplementary joins not declared in the metadata but worth an expert's verification.

The output is a Semantic Draft v0, pending expert certification. AI's role is to shorten the distance from blank slate to reviewable draft — not to skip business judgment.

4.5 Semantic Certification: The Model Supplies Evidence, Experts Own Responsibility

Semantic certification isn't experts proofreading model-generated descriptions word by word; it's deciding which business responsibilities can't be handed to a model.

• First, definition responsibility: how a metric is actually computed and on what statistical definition.

• Second, relationship responsibility: which tables should join, the cardinality, and whether it double-counts.

• Third, boundary responsibility: how entities map, and which data, time, and business scopes the semantics apply to.

• Fourth, release responsibility: who the Owner is, whether the version is Draft or Certified, and who decides on rollback when something breaks.

The machine supplies source metadata, representative values, PK/FK, candidate SQL, and historical query patterns. Domain experts accept, modify, or reject candidates on that evidence and fix the applicable scope and lifecycle. So a certified semantic asset carries more than a description: the decision, the owner, the version state, and the Benchmark regression result. Only when content, responsibility, and verification are all present does semantics qualify for production.

4.6 Benchmarks Measure Business Answers, Not Just SQL Strings

Executable SQL doesn't mean a correct business answer. The platform needs Benchmarks to gauge general capability; customers need their own around real questions, data definitions, and business risk. Neither suffices alone.

Customer-side cases fall into three layers:

Exploration layer: broaden question coverage

This layer checks whether questions are real and answerable, and whether the Agent can produce an executable SQL. It doesn't need a standard answer from the start. AI can batch-generate candidate questions and SQL for humans to sample and confirm. Even with only questions and candidate SQL and no Gold SQL, the case has value — it reveals whether the Agent understands how users phrase things.

Correctness layer: verify business results

This is the bulk of a customer's Benchmark. Each case needs a Gold SQL, an expected result, and the necessary business rules.

The point isn't whether the generated SQL matches the Gold SQL exactly — one question can have many equivalent forms. What must be verified is the execution result: whether the filters, aggregation, joins, ordering, and metric definitions match the business definition.

Production-reliability layer: verify go-live risk

This layer asks whether the Agent is stable in production: does it understand a rephrased question, proactively clarify an ambiguous one, honestly report when no data is found, and generate SQL that meets permission, performance, and security requirements.

It doesn't need to be the largest, but should focus on high-frequency, high-risk scenarios. The specific permission rules, security requirements, and acceptance criteria still fall to the customer to define.

A higher layer isn't automatically more valuable. In practice the correctness layer is the core, with production-reliability cases added by business risk.

4.7 Treat a Semantic Change Like a Controlled Pull Request

Semantics isn't a prompt you write once.

Business definitions change, metrics get adjusted, tables and fields get added, and users pose new questions. Every semantic change can affect existing queries and answers, so it must be governed like code.

A semantic change should carry at least: what changed and why, the tables, metrics, and questions involved, the cases and synonymous phrasings to regression-test, and the expected benefit and risk.

Once review-ready, set release gates: zero regression on core KPIs, no drop in overall accuracy, latency, error rate, and cost within targets, and Owner approval. If conditions aren't met, keep revising or roll back.

The full flow: Draft → Review → Benchmark → Merge / Rollback. Only by governing semantics like code do you gain versioning, diffing, approval, regression, and rollback.

4.8 Start from Real Q&A, Form a Continuous Upgrade Loop

The most valuable Benchmarks come from real user questions.

The system should store not just the final answer but the full Trace: the original question, the SQL generated, tool calls, query results, and the final response.

Combine user feedback with an LLM Judge, then have domain experts confirm the root cause. A wrong answer may stem from a wrong business definition, a wrong field or table, faulty join/filter/aggregation logic, uncovered synonymous phrasing, or an incomplete multi-turn context.

Classify the error type first, and the fix won't stop at "just try again."

After attribution, two outputs emerge:

• Semantic fixes: add business definitions, correct fields and joins, add trustworthy data assets and example SQL, and produce a new semantic version;

• Benchmark samples: add real failure cases to the test set, with varied phrasings, boundary conditions, and correct SQL or expected results.

Then run offline regression to confirm accuracy hasn't dropped and latency, error rate, and cost meet targets; after human sign-off, canary the rollout and keep collecting production feedback.

This way even failed sessions become reusable, verifiable assets for the next round, and semantics can keep evolving, being evaluated, and being rolled back.

5、Two Scenarios: From "Finding Data" to "Taking Action"

5.1 Scenario 1: Tracing complaints behind refunds and flagging churn

A service or operations lead asks: "Among customers who just requested a refund, who complained within 90 days before it — and about what?"

The question spans time, systems, and data types at once: real-time refund messages tell us who just requested a refund; the online MySQL database holds customers, orders, and after-sales tickets; the search system holds service conversations, complaint text, and call summaries.

Around customer_id and a time window, EventHouse can correlate the three into one customer timeline: first complaint 12 days ago, repeated follow-ups 8 days ago, ticket escalation 4 days ago, refund today.

It can also run multi-source hybrid retrieval across real-time refunds, structured MySQL data, and search text — no more querying each by hand and stitching results together.

Unified metadata matters as much. Define in advance whether "complaint" includes platform tickets and calls, and carry Catalog, Schema, Lineage, Freshness, and permissions into the query. Then the Agent doesn't just find records — it knows whether they can be correlated, are trustworthy, and may be used.

The output isn't just a refund list but complaint themes, the path from complaint to refund, and the high-risk customers to retain first. The business moves beyond the after-the-fact "who refunded" toward "why they refunded" and "how to warn earlier."

5.2 Scenario 2: Manufacturing data ingested once, opened for outflow and reuse

A production and quality lead: battery-cell temperature anomalies on a line have caused problems before; next time they want a real-time alert, a sync to the quality database, long-term archiving, and a maintenance Agent that can use the data.

The data comes from IoT/PLC messages, MES/ERP, and maintenance and quality systems. The traditional approach builds a separate pipeline per goal, ending in multiple copies and multiple definitions.

EventHouse keeps one complete, one-stop main pipeline, yet all four stages open on demand: processing can use EventHouse or hand off to external stream-processing components; storage can use built-in storage or open lake formats like Iceberg and Parquet, keeping data in the enterprise's own object storage; query can federate existing databases like PG and MySQL through Trino; and unified metadata can open via API, CLI, MCP, or IM to Copilot, ChatGPT, Claude, Gemini, Dify, and home-grown Agents.

Enterprises don't need to replace every existing system at once, nor rebuild a pipeline per goal. Processing can hand off externally, storage can stay open, query can federate, and semantics can be reused directly.

Closing: Track Intelligence, Accumulate Context; the Base Is Swappable, the Asset Isn't Lost

Back to the opening question: when models, frameworks, and runtimes keep changing, what should Agent development leave behind?

First, keep pace with general intelligence — but what's worth accumulating long term is the enterprise's own business context. The stronger the model, the more that context is worth.

Second, messaging services are already a mature real-time data pipeline. The next step isn't just connecting data but making schema, semantics, lineage, and timeliness arrive with it.

Third, the pipeline must consolidate while capabilities stay open. Enterprises need a one-stop service that owns the whole pipeline — fewer handoffs, less rebuilding — while retaining external processing, open storage, federated query, and connectivity to many kinds of Agent.

An Agent's long-term competitiveness depends not only on how strong the model and Agent Infra are, but on whether it can continuously see the real business, understand the enterprise's definitions, trust the data, and act on it.