Tablestore knowledge storage best practices for design, retrieval, and error handling - Tablestore

Knowledge base design

Embedding model, metadata schema, and subspace settings are immutable after creation. Plan these configurations before you begin.

The embedding model controls vector search quality and cannot be changed after creation.

Scenario	Recommendation
General Chinese and English use cases	`text-embedding-v4` (1024 dimensions). Balances semantic understanding and search performance
In-house model available	Use `custom` mode to integrate your own model
Dimension selection	Higher dimensions improve semantics but increase storage and compute costs. 1024 suits most cases

Metadata fields are immutable after creation. Follow these guidelines:

Plan ahead: Identify all filter dimensions (category, date, author, version, department) and define them at creation time.
Use the correct types: Use date for dates (not string), and long or double for numbers to enable range filtering with greaterThanOrEquals.

Note
Supported date formats: yyyy-MM-dd, yyyy-MM-dd HH:mm:ss, yyyy-MM-dd HH:mm:ss.SSS, yyyyMMdd HHmmss, yyyy-MM-dd'T'HH:mm:ss
Limit the number of fields: Up to 200 fields are supported. Define only what you need.
Mind the total size limit: Total metadata size (keys + values) must not exceed 4 KB. Avoid long text values.

Scenario	Recommendation	Reason
Multi-tenant SaaS with homogeneous data	Subspace	Shares embedding config, reduces management overhead, enables cross-tenant search
Different business lines with heterogeneous data	Separate knowledge bases	May require different embedding models and metadata schemas
Very large tenant count (10,000+)	Subspace	Avoids excess knowledge bases. No subspace limit

Consider per-request limits, OSS permissions, and status verification when importing documents.

Each AddDocuments call accepts a maximum of 10 documents. For large-scale imports:

Batch your uploads: Send up to 10 documents per request. Call AddDocuments sequentially for each batch.
Control concurrency: Respect QPS limits to avoid throttling.
Wait asynchronously: After submitting all batches, poll statuses in a single pass.
Handle errors: Check each document's status individually. For failed entries, log the reason, fix, and retry.

Grant the knowledge base service read and write access to the OSS bucket as described in Quick start.
Each file must not exceed 50 MB.
Use inclusionFilters and exclusionFilters to import by OSS directory. * wildcards are supported, such as *.pdf and *report*.

Documents are indexed automatically. To verify completion, use exponential backoff.

Parameter	Recommended value
Initial interval	3 seconds
Backoff multiplier	2 (doubles each time: 3s → 6s → 12s → ...)
Maximum interval	30 seconds
Stop condition	`status` is `Completed` or `Failed`

Processing time varies by file size and count. Exponential backoff outperforms fixed intervals for both short and long tasks.

Select the search type, rerank strategy, and parameters that best match your queries.

Scenario	Recommended type	Description
Natural-language questions	`DENSE_VECTOR` + `FULL_TEXT` hybrid	Combines semantic and keyword matching
Exact keywords or identifiers	`FULL_TEXT` preferred	Vector search misses exact matches
Pure semantic understanding	`DENSE_VECTOR` preferred	For example, "how to install" matches "deployment steps"

Strategy	Pros	Cons	Use case
WEIGHT	Controls weight of each search type	Requires manual parameter tuning	When one search type dominates
RRF	No extra model call. Low latency and stable results	No query-document interaction	General-purpose scenarios
MODEL	Best ranking accuracy	Higher latency and cost	Accuracy-critical scenarios

The search pipeline has three levels of numberOfResults:

N1 (denseVectorSearchConfiguration.numberOfResults): Vector search recall count
N2 (fullTextSearchConfiguration.numberOfResults): Full-text search recall count
N3 (rerankingConfiguration.numberOfResults): Final count after reranking

N1 and N2 set the candidate pool. N3 sets the final count. Start with N1 = N2 = 20, N3 = 5–10.

Filters narrow the candidate set before search, improving both precision and performance.
Filter fields must be defined as metadata at creation time.
For date range filtering, use the date type to enable range comparison.

Error code	Meaning	Common cause	Solution
`INVALID_PARAMETER`	Parameter validation failed	Field type mismatch, length limit exceeded, or missing required field	Verify that request parameters conform to the specification
`NOT_FOUND`	Resource does not exist	Knowledge base name misspelled or already deleted	Confirm that the resource name and ID exist
`BAD_REQUEST`	Malformed request	Invalid JSON format	Check the JSON format of the request body
`VALIDATION_ERROR`	Business validation failed	RRF k value set to 0 or invalid search parameters	Check parameter value ranges
HTTP 200 + `SUCCESS` but document `failed`	Request succeeded but document processing failed	Metadata format mismatch, such as an incorrect date format	Check `documentDetails` for each entry's `status`

Note

An HTTP 200 with code: SUCCESS does not guarantee all entries succeeded. Check each entry's status individually.

Supported date formats:

Unsupported formats cause a failed status during AddDocuments.

The following operations cannot be undone:

Operation	Impact	Recovery
DeleteKnowledgeBase	Deletes the knowledge base and all its documents and chunks	Unrecoverable
DeleteDocuments	Deletes the specified documents and all their chunks	Unrecoverable
embeddingConfiguration	Cannot be changed after creation	Requires deleting and recreating the knowledge base
metadata schema	Fields cannot be added or removed after creation	Requires deleting and recreating the knowledge base
subspace switch	Cannot be changed after creation	Requires deleting and recreating the knowledge base
UpdateDocument metadata	Full replacement. All existing values are overwritten	Re-submit the complete metadata