Tablestore:Architecture and technology selection

System architecture

Memory Storage is fully managed — you interact through APIs without provisioning or managing servers. Data flows through the following steps:

1. Call the AddMemories API with conversation messages (messages) or plain text (text).

2. The service saves the raw messages as short-term memories.

3. The service extracts searchable long-term memory units from the messages and persists them to Tablestore.

4. When you call the SearchMemories API, the service combines semantic search and keyword search with optional reranking to return relevant long-term memories.

Data model

A memory store organizes data into the following entities.

Scope multi-tenancy

Scope defines the ownership and isolation boundary of memory data. Memory Storage uses a four-level Scope hierarchy: appId / tenantId / agentId / runId.

The Scope hierarchy is:

appId > tenantId > agentId > runId

Core rules:

• On write, appId is required. Other fields default to __default__ if left blank. Wildcards (*) are not allowed on write.

• When searching long-term memories, appId and tenantId are required. agentId and runId accept the wildcard * for cross-agent or cross-session retrieval.

• When querying short-term memories, all four fields are required and wildcards are not allowed.

Search examples:

• agentId="", runId="": Retrieve all memories across all agents and sessions for the specified tenant.

• agentId="sales_assistant", runId="*": Retrieve all session memories for the specified agent under the tenant.

Memory ingestion

Memory Storage supports two ingestion methods:

• Conversation messages: Call the AddMemories API with a messages array. The service saves the raw messages as short-term memories and extracts long-term memory units from them.

• Plain text: Call the AddMemories API with the text field to submit raw text. The service extracts searchable long-term memories from the text.

Ingestion runs asynchronously by default (sync=false): raw messages are persisted immediately and become queryable as short-term memories, while long-term extraction runs in the background. Use the requestId returned by AddMemories to call GetMemoryTask and track extraction status together with the memory units produced. Each store can also carry custom extractInstructions, set at store creation or update time, that steer long-term extraction toward business-critical information.

Memory retrieval

SearchMemories retrieves long-term memories using natural language queries. The service combines semantic search and keyword search with optional reranking. Reranking is enabled by default.

Each result carries a fused ranking score and a normalized similarity. Use minSimilarity to filter out low-relevance hits. Set includeEvidence=true to attach the raw short-term message snippets (evidence) behind each hit — this improves explainability for downstream answer models.

Metadata

Long-term memories support custom metadata for business-specific information such as source, tags, and priority. Metadata is stored alongside each memory and returned in search results.

Request auditing

All requests to a memory store are logged. Call ListMemoryStoreRequests to query audit records, which include operation type, response status, latency, and failure reason.

Memory consolidation (Dream)

Memory consolidation (Dream) is an asynchronous capability that refines the data already stored in a memory store. The service analyzes long-term memories and raw messages in the background and produces actionable consolidation actions. Actions fall into three categories:

• Memory refinement (memory): deduplicate, rewrite, merge, and remove outdated long-term memories through ADD, UPDATE, DELETE, MERGE, and NOOP actions.

• Skill extraction (skill): distill reusable skills from historical interactions through the EMIT_SKILL action.

• Profile extraction (profile): build structured user profiles through the EMIT_PROFILE action.

Consolidation supports two modes: proposal generates suggestions that require manual or programmatic confirmation before they are applied, while safe_auto applies actions automatically once a confidence threshold is met.

Async tasks and observability

Both memory ingestion and consolidation run as asynchronous tasks. The service exposes matching query APIs:

• GetMemoryTask and ListMemoryTasks: inspect the status and output of memory extraction tasks.

• GetMemoryDreamTask, ListMemoryDreamTasks, and ListMemoryDreamActions: inspect consolidation tasks and the actions they produce.

• ListMemoryStoreScopes: list the Scopes that already exist in a memory store — useful for discovering which agents and sessions under a given app or tenant have produced memories.

• ListMemoryStoreRequests: query request audit logs.

Applicability assessment

Recommended scenarios:

• Conversational applications such as customer service bots, sales assistants, and enterprise assistants.

• Personal assistants that remember user preferences, settings, and past choices across sessions.

• Shared memory management in multi-agent collaboration scenarios.

• Agent applications that require raw conversation logs with audit trail support.

• Production systems that need stable retrieval performance at scale with large numbers of tenants and memories.

Scenarios that require evaluation:

• Use cases with strict real-time requirements for memory ingestion. Memory extraction is asynchronous and introduces processing latency.

• Use cases that require custom memory extraction strategies. The current extraction process is fully automated by the service.

Not recommended:

• Pure structured data queries. Use the Tablestore wide column model or a relational database instead.

• Applications that only need simple context window management without persistent memory. Use the LLM context window directly.

Performance benchmarks

Memory Storage was benchmarked against Mem0 across four dimensions: retrieval accuracy, retrieval latency, storage scale, and token cost.

These benchmarks reflect typical performance. Actual results vary with data volume, query patterns, network conditions, model configuration, and data distribution.