Tablestore:Knowledge Storage overview

What is the Knowledge Storage service?

The Knowledge Storage service is a fully managed serverless RAG service built on Alibaba Cloud Tablestore. You do not need to deploy or maintain physical servers or infrastructure.

After you import documents through the console, a software development kit (SDK), or the command-line interface (CLI), the system automatically parses and chunks them, vectorizes them, and builds indexes. During retrieval, the service performs hybrid retrieval and returns relevant Chunks.

All data is stored in the Object Storage Service (OSS) and Tablestore resources in your account. The service does not hold or copy your data. Storage and compute are separated and billed independently. In a typical AI application workflow, the Knowledge Storage service provides knowledge management and retrieval.

For billing information about the Knowledge Storage service, see Product Billing. To quickly try the Knowledge Storage service, see Build your first knowledge base.

The Knowledge Storage service is available in the following regions: China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Hong Kong), and Singapore. Select the nearest region for access.

Features and benefits

• Serverless and elasticity — The fully managed serverless architecture uses pay-as-you-go billing with no minimum charge. Storage and compute are billed independently. A single knowledge base supports up to 10 million documents by default. Built on the distributed architecture of Tablestore, the service scales with your business. For specific limits, see Quotas and limits.

• Fully managed knowledge base pipeline — After a document is uploaded, the system automatically completes document parsing, chunking, embedding, index building, and all other processing. The service supports common document formats, including PDF, Word (doc/docx), Excel (xls/xlsx), PowerPoint (ppt/pptx), plain text (txt), and Markdown (md). You do not need to build a document processing pipeline, manage embedding model deployment and operations, or manually build and maintain vector indexes. The service automatically manages the entire knowledge base lifecycle.

• Data ownership and full control — All raw, intermediate, and result data in the process is stored in your own cloud service account, in an OSS bucket or a Tablestore instance. The service does not hold or copy your data. This architecture keeps data ownership and security fully under your control and meets the stringent data-compliance requirements of industries such as finance, government, and healthcare.

• Flexible customization and precise retrieval — The service supports dual-channel hybrid retrieval with vector retrieval and full-text retrieval (semantic similarity + exact keyword matching). It provides three reranking strategies (RRF, WEIGHT, and MODEL) that you can select based on your scenario. You can precisely control core modules through APIs and parameters, from embedding model selection (built-in Model Studio or custom), retrieval strategy configuration, and metadata filtering conditions to Chunk-level content management.

Typical application scenarios

• Document parsing, chunking, and index building after import are asynchronous and completed automatically by the system. Carefully evaluate whether the Knowledge Storage service fits scenarios that require real-time processing or strictly customized chunking strategies.

• For purely structured data queries, we recommend the wide-column model or a relational database. For real-time streaming data indexing, we recommend using a Tablestore search index directly.

Glossary

Data in a knowledge base is organized into four entity levels:

The system automatically creates the corresponding data tables in Tablestore. In the following table names, the placeholders represent the knowledge base name and ID:

• Document table: {knowledge_base_name}_{knowledge_base_id}

• Chunk table: {knowledge_base_name}_{knowledge_base_id}_chunk

• Index table: {knowledge_base_name}_{knowledge_base_id}_index

Knowledge base data source configuration

The data source for a knowledge base is Alibaba Cloud Object Storage Service (OSS). Regardless of how documents are uploaded, their raw data is stored in your own OSS bucket.

A knowledge base needs read and write access to the OSS bucket: it reads source documents for parsing and chunking, and writes intermediate parsing results back to OSS.

You can import documents into a knowledge base in the following three ways:

• Upload local files — Specify a local file or directory. The SDK automatically uploads it to OSS and then adds it to the knowledge base. If you specify a directory, the SDK recursively traverses all files.

• Add OSS files — If a file is already in OSS, specify its OSS path to add it directly to the knowledge base without uploading it.

• Bulk import an OSS directory — Specify an OSS directory path. The system automatically scans all files in the directory recursively. You can use inclusionFilters and exclusionFilters to filter files by name pattern.

For details about each method, see Document management.

Knowledge base embedding configuration

The embedding configuration determines how document Chunks are vectorized and is specified when you create a knowledge base.

You cannot modify the embedding configuration after you create a knowledge base. To change the model or dimension, you must delete and recreate the knowledge base. Fully evaluate the model selection before you create the knowledge base.

If embeddingConfiguration is not specified, the system uses the built-in Model Studio text-embedding-v4 model with 1,024 dimensions by default. It is suitable for most scenarios.

Selection recommendations

• For general Chinese and English scenarios, we recommend text-embedding-v4 with 1,024 dimensions. It provides a good balance between semantic understanding and performance.

• To integrate an existing self-developed or third-party embedding service, use custom mode to keep your technology stack consistent.

• Higher dimensions provide richer semantic representation, but also increase storage and compute costs. For most scenarios, we recommend 1,024 dimensions.

Knowledge base retrieval strategy configuration

When you call the Retrieve API, the retrieval configuration (retrievalConfiguration) is determined in the following order of priority:

Retrieval types

We recommend enabling both retrieval types and combining their results through the reranking mechanism.

Rerank strategies

For detailed retrieval strategy configuration and tuning methods, see Retrieval and Ranking.

Multi-tenant management

Subspace is a data-isolation partition within a knowledge base. Use it to isolate data by user, department, or tenant within the same knowledge base. Set "subspace": true when you create a knowledge base to enable it.

Core rules:

• You cannot modify the Subspace switch after you create a knowledge base.

• After you enable Subspace, all document operations (add, query, delete, and list) and retrieval operations must specify the subspace field. Otherwise, an error is returned.

• Retrieve supports querying multiple Subspaces at the same time by passing a list of up to 32 Subspaces.

• A Subspace name can be up to 128 bytes long.

In an enterprise environment, each employee in the same department has their own documents. The department administrator creates a knowledge base with Subspace enabled and assigns an independent Subspace to each employee, such as employee_zhangsan and employee_lisi. Each employee's documents are invisible to the others, while the administrator centrally manages the knowledge base configuration. To retrieve data across employees, pass multiple Subspaces to Retrieve for joint retrieval.

Subspace and multiple knowledge bases