Best practices for Tair as AI agent short-term memory - Tair (Redis® OSS-Compatible)

Background

When a user says "Order me a large latte with oat milk, no sugar, delivered to the office," the AI agent must complete intent recognition, address resolution, product search, specification matching, cart addition, and order placement within seconds. Each step depends on accurate memory of prior turns.

In the Taobao Flash Shopping "one-sentence ordering" project, built with Qwen, Tair serves as the core of the agent's short-term memory layer.

Use case

The Taobao Flash Shopping agent lets users complete an end-to-end ordering flow through natural language, compressing a typical 3–5 minute process to under 30 seconds. Across turns, the agent retains full context: Turn 1 captures location, Turn 2 recommends products, Turn 3 confirms the cart and submits the order. All memory lives in Tair.

Why Tair for the memory layer

AI agent memory is extremely latency-sensitive. By Little's Law, concurrency ≈ QPS × latency. When memory latency rises from 5 ms to 50 ms, in-flight requests grow tenfold, exhausting connections, threads, and queues. Because each turn involves multiple reads and writes, latency compounds and can trigger cascading failures.

The difference between 5 ms and 50 ms determines whether the system can scale reliably, not just how fast it feels. Tair's proprietary multi-threaded kernel delivers consistently low latency, keeping memory access within safe limits and preventing runaway concurrency under load.

Architecture overview

The memory service sits between the agent orchestrator and downstream tool services. Tair provides session-level state management.

Memory types and data model design

The Taobao Flash Shopping agent uses Tair as its short-term memory store for the following reasons:

Low latency: Tair's multi-threaded kernel provides microsecond-level reads and writes, meeting the strict response-time requirements of the agent pipeline.
Rich data structures: Each memory type maps to the most suitable data structure, simplifying development.
Elastic scaling: Transparent cluster scaling and burst bandwidth let the system expand during traffic spikes without business impact.
TTL-based lifecycle management: Session memory naturally expires. The TTL mechanism automatically cleans up stale data.

Short-term memory falls into two categories, each mapped to a different Tair data structure:

Model memory - List

Model memory stores conversation history for the LLM. Each turn's user input and agent response are recorded and passed as context in the next inference cycle.

This data is stored in a Tair List, with one key per session:

Key:  memory:model:{sessionId}
Type: List

Sample data:
[
  {"role": "user",      "content": "Order me a milk tea"},
  {"role": "assistant", "content": "Found 3 nearby milk tea shops...", "cards": [...]},
  {"role": "user",      "content": "That one, less sugar, no ice"},
  {"role": "assistant", "content": "Selected: Jasmine Milk Tea, less sugar, no ice, large..."}
]

Key operations:

# Append a new conversation record after each turn
RPUSH memory:model:{sessionId} "{conversation_record_JSON}"

# Read the last N turns of conversation as context before model inference
LRANGE memory:model:{sessionId} -{N} -1

# Set the session expiration time (for example, 30 minutes)
EXPIRE memory:model:{sessionId} 1800

Note

Before writing, the system converts raw data (text and rich media such as cards) into a natural-language format for more efficient model interpretation, reducing token consumption.

Business context memory - Hash

Business context memory stores structured workflow state. The agent's tool layer and intent handlers query and update this data during business logic execution.

This data is split into six submodules by business domain and stored in a Tair Hash:

Key:  memory:context:{sessionId}
Type: Hash

Field structure:
{
  "session":      "{Session metadata: user ID, channel, session stage, etc.}",
  "search":       "{Search state: current query, search results, recommended product list, etc.}",
  "order":        "{Order state: cart contents, selected SKUs, item quantities, etc.}",
  "conversation": "{Conversation state: current intent, previous intent, intent switch flag, etc.}",
  "coupon":       "{Coupon info: available coupon list, selected coupons, etc.}",
  "bizState":     "{Business state: delivery address, shipping method, payment status, etc.}"
}

Key operations:

# Update a specific submodule (for example, the user confirmed a delivery address)
HSET memory:context:{sessionId} bizState "{updated_business_state_JSON}"

# Read a specific submodule
HGET memory:context:{sessionId} order

# Read all context at once (for scenarios that need global information, such as intent recognition)
HGETALL memory:context:{sessionId}

# Set the expiration time
EXPIRE memory:context:{sessionId} 1800

Note

Hash field-level operations let each module update independently, eliminating the race condition of reading and writing back an entire JSON blob. For example, the search module can update recommendations without affecting the cart data the order module writes concurrently.

Data structure comparison

Memory type	Redis data structure	Rationale
Conversation history	List	Conversations form an ordered time series. List supports ordered appending (RPUSH) and range reads (LRANGE).
Business context	Hash	Split into multiple fields by domain. Supports field-level independent read/write operations that prevent race conditions.
Session status flags	String	Provides atomic status markers (such as session stage) with simple operations.
Distributed lock	String	Implemented with SET NX EX to provide concurrency safety.

Concurrency safety: distributed locks

In production, concurrent writes to the same session are common—a user may send messages in rapid succession or submit input while a streaming response is still in progress.

The Taobao Flash Shopping agent uses Tair distributed locks to protect memory read/write consistency at the session level:

# Acquire a session-level distributed lock (3-second timeout to prevent blocking)
SET lock:memory:{sessionId} {requestId} NX EX 3

# After acquiring the lock, perform memory read/write operations
HSET memory:context:{sessionId} order "{updated_order_state}"
RPUSH memory:model:{sessionId} "{new_conversation_record}"

# Release the lock after operations complete (use a Lua script to release only the lock you hold)
EVAL
  if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
  else
    return 0
  end

Note

Lock granularity is per session (sessionId), not global. Different users have zero lock contention. The short timeout prevents extended blocking if the lock holder exits abnormally.

Handle traffic spikes

During the Qwen Spring Festival red envelope campaign, the Taobao Flash Shopping agent experienced over 10× the estimated peak traffic. Each conversation triggers dozens of Tair operations (read history, update state, lock), amplifying request volume by an order of magnitude.

Compared with self-managed Redis, Tair's kernel performance, elastic scaling, and operations capabilities were critical for handling this spike.

Performance advantages of the Tair proprietary kernel

Tair (Enterprise Edition) DRAM-based instances use a multi-threaded model that delivers 3× the read/write throughput of Redis Open-Source Edition at the same specifications.

This matters for AI agent workloads, where a single conversation triggers dozens of operations. Open-source Redis's single-threaded model bottlenecks under high concurrency. Tair's multi-threaded kernel fully utilizes multi-core CPUs on a single node, handling higher concurrency without adding nodes.

Elastic scaling and transparent cluster operations

Agent memory exhibits a read-heavy, write-light pattern. Before each inference cycle, the agent reads full conversation history and context, while writes occur only once per turn. The read-to-write ratio typically ranges from 5:1 to 10:1.

Tair supports read/write splitting in cluster mode, automatically routing reads to replicas and writes to masters. You can configure 1–9 read replicas per shard and scale from 2–256 shards. Add replicas or shards before a peak to increase throughput linearly; reduce them afterward to lower costs.

Normal traffic:
  8-shard cluster, 1 read replica per shard -> meets daily business requirements

Spring Festival campaign (5x - 10x peak):
  Option 1: Scale to 5 read replicas per shard -> linear increase in read throughput
  Option 2: Scale to 16 shards + 3 read replicas per shard -> double both read and write capacity

These operations are transparent to the application. Traditional Redis cluster slot migration can produce -ASK and -TRYAGAIN errors—any failed request can break an agent conversation or cause memory loss. Tair cloud-native instances use kernel-level transparent scaling: data migrates atomically at the slot level, preventing fragmentation. A centralized control component coordinates migration for higher efficiency and accuracy.

Note

During the final migration phase, write latency for affected slots may increase slightly, but requests do not fail—no data loss or errors.

Elastic bandwidth scaling

The campaign also brought bandwidth challenges. Each agent memory read transfers conversation history and business context, producing much larger payloads than typical cache reads. During peaks, bandwidth can bottleneck before CPU or memory.

The Tair cloud-native architecture provides two tiers of elastic bandwidth:

Horizontal bandwidth scaling through load balancers: Add load balancers (LBs) in cluster mode to increase total bandwidth. Each LB supports up to 20 Gbps. For clusters with more than 8 shards, LBs can be added on demand without interrupting connections.
Burst bandwidth: When traffic exceeds fixed bandwidth, the system auto-scales within seconds (up to 288 MB/s per node) and scales back after the spike. You pay only for actual burst usage. Burst operates at the shard level—when a hot key causes a bottleneck on one shard, only that shard scales up.

Note

Burst bandwidth suits the unpredictable spikes common in agent scenarios and costs less than provisioning high fixed bandwidth in advance.

Automatic TTL-based cleanup

All session keys use a TTL (for example, 30 minutes). After a spike, memory usage drops automatically. Combined with elastic scaling, this delivers fully automatic resource management: scale up at peak, scale down off-peak, and clean up expired data.

With this approach, the memory service remained stable throughout the Spring Festival traffic spike, with P99 latency consistently in the single-digit millisecond range.

Summary and outlook

In the Taobao Flash Shopping "one-sentence ordering" scenario, Tair serves as the core storage for the AI agent's short-term memory layer. The following table summarizes the key capabilities:

Capability	Implementation	Problem solved
Low-latency access	Tair in-memory read/write	Meets the real-time requirements of the agent conversation pipeline
Flexible data modeling	List + Hash + String combination	Supports different memory types such as conversation history and business context
Lifecycle management	TTL-based automatic expiration	Automatically cleans up data after sessions end, which reduces operational costs
Concurrency safety	Distributed lock (SET NX EX)	Maintains data consistency when multiple requests write concurrently
Elastic resilience	Read/write splitting + burst bandwidth	Sustained 10x peak traffic during the Spring Festival campaign with transparent elasticity

As AI agent technology evolves, long-term memory capabilities—user preferences and behavioral patterns—are on the roadmap. The goal: agents that remember not just "what was said" but "who this user is and what they prefer."

As agent memory evolves from conversation-level to user-level, Tair will continue to play a critical role.