Qwen Memory Agent: Treating Memory as a Measurable Engineering Problem

Blog Post Award, Global AI Hackathon Series with Qwen Cloud

Project Introduction

Most agent-memory projects promise the world and measure nothing. Qwen Memory Agent does the opposite. It's an open-source persistent-memory agent built on Qwen Cloud and deployed on Alibaba Cloud ECS that remembers information across sessions, retires superseded facts, and retrieves relevant memories within a configurable token budget — and then publishes the evaluations, including the ones that don't flatter it.

Its companion write-up, the Blog Post Award-winning "Memory as a Measurable Engineering Problem," frames the whole effort: agent memory shouldn't be magic. It should be something you can inspect, benchmark, and watch fail.

Project Value

AI agents need to remember useful information without dragging an entire conversation history into every prompt — that's expensive and dilutes attention. But the harder problem is staleness. An old preference or constraint can remain perfectly relevant to a search while being flat-out wrong for the _next decision_. Retrieving a fact and correctly using a fact are different skills, and most systems conflate them.

Qwen Memory Agent addresses persistent memory, stale information, and limited context together — and its main value is making all three behaviors inspectable and measurable. A live memory inspector, token-usage reporting, and reproducible evaluations expose the failures alongside the successes. It's memory with the lights on.

Solution

Qwen function calling lets the agent decide for itself when to remember, recall, or forget — memory as a set of tools the model chooses to invoke, not a black box bolted on. A persistent memory engine stores typed records and vectors, using exact subject/type matching plus semantic similarity to retire superseded facts rather than letting them pile up.

Retrieval ranks memories by similarity, recency, salience, and type, then packs them into a configurable token budget so the agent never overflows its context. The same engine is exposed over HTTP and through eight MCP tools, with JSON/Markdown export and import for portability. A separate Qwen "dreaming" pass proposes memory consolidation — but, in a deliberate safety choice, changes require human approval. The evaluation suite checks three separate things: retrieval quality, the stored memory state, and whether the agent actually consults memory before making a later decision.

Highlights

Qwen Plus powers the agent's reasoning and function-calling loop, deciding when to invoke memory tools. Qwen text-embedding-v3 provides the vectors for semantic retrieval, supersession checks, and natural-language forgetting — all through the Model Studio / DashScope API. Alibaba Cloud ECS hosted the FastAPI backend and the live memory inspector in Singapore. Every model call records API-reported token usage, while retrieval keeps separate token-budget accounting.

The honesty of the results is what makes this project stand out. On a six-persona, 24-query synthetic offline fixture, it achieved context recall of 1.000 and staleness of 0.000 across 8–64-token budgets — flawless. But a separate ten-scenario live evaluation achieved just 0.60 task success, when decision outcome, memory-store state, and recall-before-decision all had to pass. Those numbers measure different things, and the builder says so plainly: retrieving the right fact does not guarantee an agent will use it correctly. Publishing that 0.60 next to the 1.000 is the point.

Alibaba Cloud Products Used

About the Team

Ryan Duffy is a UK-based engineer working across site reliability engineering and applied AI systems. He builds Rootweaver, a self-hosted research and knowledge platform focused on retrieval, provenance, evaluation, and reliable agent behavior. He led the architecture, evaluation, verification, and deployment of Qwen Memory Agent himself (with AI coding assistance), driven by a single aim: make agent memory something we can inspect and measure — including where it fails.

Explore the project: https://youtu.be/TOMjFJ4ayYg

MIT-licensed source, evaluation results, and known limitations: https://github.com/rduffyuk/qwen-memory-agent

This project was developed as part of the Global AI Hackathon Series with Qwen Cloud.

Community

Qwen Memory Agent: Treating Memory as a Measurable Engineering Problem

Project Introduction

Project Value

Solution

Highlights

Alibaba Cloud Products Used

About the Team

Read previous post:

Read next post:

Alibaba Cloud Project Hub

You may also like

Comments

Alibaba Cloud Project Hub

Related Products

Token Plan

QwenWork

Qwen

Alibaba Cloud Model Studio