Enterprise knowledge space is a fully managed serverless Retrieval-Augmented Generation (RAG) service provided by PolarDB Agent Express. Built on the PolarDB unified architecture, it encapsulates data ingestion, multimodal data parsing, vectorization, hybrid retrieval, and agent orchestration into managed capabilities, helping you build searchable, queryable, and traceable enterprise knowledge services.
Overview
Building a RAG system from scratch requires assembling multiple components such as document parsing, chunking, vectorization, index construction, hybrid retrieval, reranking, and agent orchestration. This results in a long pipeline with many moving parts, making data consistency difficult to maintain. Enterprise knowledge space converges this pipeline into a single managed service and addresses the following core challenges:
Built-in multimodal data parsing and search: Documents, images, tables, audio, and video are parsed and retrieved across modalities, eliminating the need to integrate multiple parsing toolchains.
Built-in model services: Embedding, reranking, LLM, layout analysis, and OCR models are managed centrally, removing the need to build dedicated model inference clusters.
End-to-end RAG solution: The full pipeline from data ingestion, parsing, chunking, and vectorization to index construction, hybrid retrieval, reranking, and agent orchestration works out of the box.
Serverless pay-as-you-go: Compute resources scale elastically based on actual usage and drop to zero when idle. No cluster pre-provisioning is required.
Zero data synchronization: Raw data serves as the single source of truth. Indexes are automatically maintained as derived projections, keeping data consistent. Multiple data sources such as OSS and NAS are supported.
Agentic RAG: Retrieval is exposed as a tool for agents, enabling them to autonomously decide the number of retrieval rounds, rewrite queries, and evaluate result sufficiency. This goes beyond traditional fixed-pipeline RAG.
Unified architecture
Enterprise knowledge space is built on the PolarDB unified architecture, with each component serving a specific role.
Component | Description |
Relational database (PolarDB for MySQL) | Manages metadata with precision, including task states, space configurations, document lifecycle, and audit logs. Provides transactional consistency guarantees. |
PolarSearch - Search node | Supports storage and retrieval of tens of billions of vector data entries, with BM25 + KNN multi-path recall. Compute and storage are decoupled for independent elastic scaling. |
GPU cluster | Provides document parsing orchestration capabilities, including layout analysis, table structure recognition, OCR, formula recognition, image understanding, and ASR/VLM. |
Core advantages
Advantage | Description |
End-to-end RAG solution | The full pipeline from data ingestion, multimodal parsing, chunking, and vectorization to index construction, hybrid retrieval, reranking, and agent orchestration works out of the box. |
Built-in multimodal capabilities | Multimodal parsing (documents, tables, images, audio, and video) and multimodal search are provided as a unified service. Parsing models run on the managed GPU cluster. |
Zero data synchronization | Raw data is the single source of truth, and indexes are automatically kept consistent. No CDC or ETL pipelines are required. Multiple data sources such as Object Storage Service (OSS) and Apsara File Storage NAS work out of the box. |
Serverless cost efficiency | Pay-as-you-go pricing with elastic compute scaling. Resources scale to zero when idle. No cluster pre-provisioning is required. |
Tens of billions of vector retrieval | PolarSearch supports storage and retrieval of tens of billions of vector data entries with BM25 + KNN multi-path recall. Compute and storage are decoupled for independent scaling. |
Built-in model services | Embedding, reranking, LLM, and parsing models are managed centrally. Model switching is transparent to production workloads. |
Agentic-native | Retrieval is exposed as a tool for agents, supporting multi-turn iterative reasoning, query rewriting, and result sufficiency evaluation. |
Architecture
Enterprise knowledge space consists of the following deployment units, organized into three main pipelines: data ingestion, query, and management.
Deployment unit | Description |
Data sources (OSS, NAS, and others) | The raw data layer that stores documents, audio, video, and other unstructured data. Serves as the single source of truth for the system. |
Ingestion layer (Ray cluster) | Consists of an event-triggered gateway and a processing cluster (Ray cluster). Subscribes to data changes and performs periodic reconciliation to ensure consistency. Handles parsing, metadata extraction, and chunking. |
Engine layer | The core of the PolarDB unified architecture. Includes the orchestration engine, PolarSearch index storage (inverted index and vector store), and PolarDB for MySQL (metadata management). |
Agentic service | The query layer entry point. Runs the ReAct reasoning loop, tool registry, and agent memory. |
Control plane + Dashboard | The management layer entry point. Handles data source binding, synchronization status, task center, and audit. |
Model service (GPU cluster) | Hosts general-purpose models such as Embedding, Reranking, and LLM, as well as specialized parsing models such as layout analysis and OCR. |
Three main pipelines:
Data ingestion: Data source changes are captured by the ingestion layer, which parses, chunks, and writes vectorized data to the engine layer indexes. No manual synchronization pipelines are required.
Query: Natural language queries enter the agentic service, which uses a ReAct loop to orchestrate retrieval tool calls to the engine layer and returns results with source attribution.
Management: Declare data source bindings, configure index and model parameters, and monitor synchronization status and audit logs through the control plane.
Core capabilities
Event-driven data ingestion
The ingestion layer subscribes to data source changes (additions, modifications, and deletions) and performs periodic reconciliation as a fallback to ensure eventual consistency between indexes and raw data. You only need to maintain the raw data without manually triggering synchronization or building additional pipelines.
Multimodal document parsing
A built-in GPU cluster runs a suite of specialized parsing models for end-to-end multimodal parsing: layout analysis, table structure recognition, OCR, formula and code recognition, and image understanding. Audio and video are transcribed through ASR/VLM. Parsing results are uniformly chunked into indexable content units.
Hybrid retrieval and multimodal search
The engine layer executes multi-path recall through query plans on PolarSearch nodes: BM25 keyword recall and vector KNN recall run in parallel. Scores are normalized, fused, and then reranked by the reranking model to produce the most relevant results. Cross-modal semantic retrieval and multiple retrieval modes are supported.
Agentic RAG
Retrieval is exposed as a tool to the agentic service. The service orchestrates Q&A through a ReAct loop (Thought, Action, Observation): the agent determines whether retrieval is needed, how to rewrite or decompose queries, and iterates until results are sufficient before generating the final answer.
Built-in model services
Embedding, reranking, LLM, and specialized parsing models are managed centrally. Model services are shared by the ingestion layer, engine layer, and agentic service. Select the desired model in the space configuration.
Decoupled compute and storage
The engine layer follows the PolarDB decoupled compute-storage architecture: index construction and retrieval computation are stateless and can elastically scale across multiple compute nodes. Index data resides in the PolarDB shared storage layer, allowing compute and storage to scale independently.
Use cases
Enterprise knowledge base Q&A
Store product documentation, policies, and technical resources in a data source. Enterprise knowledge space automatically parses, chunks, and indexes the content. Employees can ask questions in natural language and receive answers with source attribution. When documents are updated, the indexes synchronize automatically without manual maintenance.
Multimodal media asset retrieval
For media asset libraries that contain documents, images, audio, and video, enterprise knowledge space performs unified multimodal parsing, chunking, and indexing. Cross-modal semantic retrieval is supported, making it suitable for unified retrieval and Q&A across financial research reports, industry materials, and training media.
Retrieval backend for agent applications
Enterprise knowledge space exposes retrieval capabilities through APIs, serving as the retrieval backend for agent applications. Agents can call the retrieval tool as needed during multi-turn reasoning to solve complex problems iteratively.