Manage knowledge bases - Tablestore - Alibaba Cloud Documentation Center

In the AgentStorage knowledge storage service, a knowledge base is the top-level container for documents, chunks, and retrieval configurations. You can create, query, list, update, and delete knowledge bases from the AgentStorage console, through the SDKs, or by calling API operations.

Create a knowledge base

Before you create a knowledge base, make sure that the following prerequisites are met:

Create an AgentStorage instance and obtain its endpoint and instance name. SDK and API calls in this topic require them.
Prepare an API key. The SDK clients in this topic are initialized with API key authentication.
To import documents from local storage or Object Storage Service (OSS), make sure that your account has an OSS bucket in the same region and that OSS authorization is completed.

Basic settings

The following table describes the settings that you can configure when you create a knowledge base and whether each setting can be modified after creation:

Setting	Description	Modifiable after creation
Name	Must start with a letter and contain only letters, digits, and underscores. The length must be 1–64 characters, and the name must be unique within the same instance.	No
Description and tags	Describes the purpose of the knowledge base. The description can be up to 4 KB, and the total length of tags cannot exceed 4 KB.	Yes
Resource group	The resource group to which the knowledge base belongs.	—
Metadata schema	Defines additional document attributes, such as custom fields for source, category, and version number, that you can use to filter retrieval results by field. Supports `string`, `long`, `double`, `boolean`, and `date`, as well as their corresponding `_list` types. Field names can be up to 128 bytes, cannot contain `.`, and cannot use reserved fields (`_uid`, `_id`, `_type`, and so on). Up to 200 fields are supported.	No
Subspace	Isolates documents in the same knowledge base by business line, project, or tenant. After you enable it, all document operations and retrieval requests must specify a subspace.	No
Multimodal	Enables image ingestion and multimodal retrieval.	No
Embedding model	Determines the semantic understanding capability of vector retrieval. Bailian `text-embedding-v4` (1,024 dimensions) is used by default. Custom models are also supported.	No
Retrieval configuration	Specifies the retrieval type and reranking strategy. The default is hybrid retrieval that combines vector retrieval and full-text retrieval. Reranking uses WEIGHT weighted fusion by default (vector 0.7 : full-text 0.3).	Yes

Important

The metadata field definitions, the embedding configuration, and the subspace switch cannot be modified after creation. Plan them completely before you create a knowledge base. To change the embedding model or add or remove metadata fields, delete and recreate the knowledge base.

Console

Log on to the AgentStorage console, click Knowledge Base Management, and then click Create Knowledge Base.
Enter a name that follows the naming rules described in Basic settings, enter a description as needed, and select a resource group.
Configure the metadata schema: add custom fields for retrieval filtering, such as author, category, and version number, and select field types.
Enable Subspace for multi-tenant data isolation and Multimodal for image ingestion and multimodal retrieval as needed.
In Advanced settings, customize the embedding model, number of retrieved results, retrieval type, and reranking strategy. If you do not have specific tuning requirements, use the default configuration first.
Click OK to create the knowledge base.

After the knowledge base is created, it appears in the knowledge base list.

Note

If an OSS authorization prompt appears, complete the authorization before continuing.

SDK

Python

The following sample code shows how to create a knowledge base:

from tablestore_agent_storage import AgentStorageClient

# Initialize the client (API key authentication)
client = AgentStorageClient(
    api_key="<your-api-key>",
    ots_endpoint="https://<instance>.<region>.ots.aliyuncs.com",
    ots_instance_name="<instance-name>",
)

# Create a knowledge base (Bailian text-embedding-v4 and hybrid vector and full-text retrieval are used by default)
client.create_knowledge_base({
    "knowledgeBaseName": "product_docs_kb",
    "description": "Knowledge base for product documentation",
    "metadata": [
        {"name": "author", "type": "string"},
        {"name": "category", "type": "string"},
        {"name": "publish_date", "type": "date"}
    ],
    "embeddingConfiguration": {
        "provider": "bailian",
        "model": "text-embedding-v4",
        "dimension": 1024
    },
    "retrievalConfiguration": {
        "searchType": ["DENSE_VECTOR", "FULL_TEXT"],
        "denseVectorSearchConfiguration": {"numberOfResults": 10},
        "fullTextSearchConfiguration": {"numberOfResults": 10},
        "rerankingConfiguration": {
            "type": "WEIGHT",
            "numberOfResults": 5,
            "weightConfiguration": {
                "denseVectorSearchWeight": 0.7,
                "fullTextSearchWeight": 0.3
            }
        }
    }
})

TypeScript

The following sample code shows how to create a knowledge base:

import { AgentStorageClient } from '@tablestore/agent-storage';

// Initialize the client (API key authentication)
const client = new AgentStorageClient({
  apiKey: '<your-api-key>',
  endpoint: 'https://<instance>.<region>.ots.aliyuncs.com',
  instanceName: '<instance-name>',
});

// Create a knowledge base
await client.createKnowledgeBase({
  knowledgeBaseName: 'product_docs_kb',
  description: 'Knowledge base for product information',
  metadata: [
    { name: 'author', type: 'string' },
    { name: 'category', type: 'string' },
  ],
});

In this example, embeddingConfiguration and retrievalConfiguration are omitted, so the default embedding model and retrieval configuration are used.

API

Call CreateKnowledgeBase to create a knowledge base. For complete request fields and response descriptions, see CreateKnowledgeBase.

Query knowledge base details

Query the complete configuration of a single specified knowledge base, including its name, description, metadata definitions, embedding configuration, and retrieval configuration. To view all knowledge bases in the current instance at a time, see List knowledge bases.

Console

On the knowledge base list page, choose ··· > Details in the upper-right corner of the target knowledge base card to view its complete configuration.

SDK

Python

The following sample code shows how to query knowledge base details:

from tablestore_agent_storage import AgentStorageClient

client = AgentStorageClient(
    api_key="<your-api-key>",
    ots_endpoint="https://<instance>.<region>.ots.aliyuncs.com",
    ots_instance_name="<instance-name>",
)

resp = client.describe_knowledge_base({
    "knowledgeBaseName": "product_docs_kb"
})

data = resp["data"]
print(f"Name: {data['knowledgeBaseName']}")
print(f"Embedding: {data['embeddingConfiguration']}")
print(f"Retrieval configuration: {data['retrievalConfiguration']}")

TypeScript

The following sample code shows how to query knowledge base details:

import { AgentStorageClient } from '@tablestore/agent-storage';

const client = new AgentStorageClient({
  apiKey: '<your-api-key>',
  endpoint: 'https://<instance>.<region>.ots.aliyuncs.com',
  instanceName: '<instance-name>',
});

const resp = await client.describeKnowledgeBase({
  knowledgeBaseName: 'product_docs_kb',
});
console.log(resp.data);

API

Call DescribeKnowledgeBase to query knowledge base details. For complete fields, see DescribeKnowledgeBase.

List knowledge bases

Use pagination to list all knowledge bases in the current instance.

Note

The maximum value of maxResults is 100. Values greater than 100 return INVALID_PARAMETER.

Console

On the Knowledge Base Management page, view the list of all knowledge bases, including their names, descriptions, and creation times.

SDK

Python

The following sample code shows how to list knowledge bases by page:

from tablestore_agent_storage import AgentStorageClient

client = AgentStorageClient(
    api_key="<your-api-key>",
    ots_endpoint="https://<instance>.<region>.ots.aliyuncs.com",
    ots_instance_name="<instance-name>",
)

# Get the first page
resp = client.list_knowledge_base({"maxResults": 10})
for kb in resp["data"]["knowledgeBases"]:
    print(f"{kb['knowledgeBaseName']} - {kb.get('description', '')}")

# Get the next page
next_token = resp["data"].get("nextToken")
if next_token:
    resp = client.list_knowledge_base({
        "maxResults": 10,
        "nextToken": next_token
    })

TypeScript

The following sample code shows how to list knowledge bases:

import { AgentStorageClient } from '@tablestore/agent-storage';

const client = new AgentStorageClient({
  apiKey: '<your-api-key>',
  endpoint: 'https://<instance>.<region>.ots.aliyuncs.com',
  instanceName: '<instance-name>',
});

const resp = await client.listKnowledgeBase({ maxResults: 10 });
for (const kb of resp.data.knowledgeBases) {
  console.log(kb.knowledgeBaseName, kb.description);
}

API

Call ListKnowledgeBase to list knowledge bases. For complete fields, see ListKnowledgeBase.

Update a knowledge base

Update the description, tags, or retrieval configuration of a knowledge base. You must specify at least one of description, tags, and retrievalConfiguration. embeddingConfiguration and metadata cannot be modified by using this operation. To change the embedding model or add or remove metadata fields, delete and recreate the knowledge base.

Note

After you modify retrievalConfiguration, subsequent Retrieve requests that do not include a retrieval configuration use the new configuration. Requests that explicitly include a configuration are not affected.

Console

On the knowledge base list page, choose ··· > Settings in the upper-right corner of the target knowledge base card, and then click Edit to update the description, tags, and retrieval configuration.

SDK

Python

The following sample code shows how to update a knowledge base:

from tablestore_agent_storage import AgentStorageClient

client = AgentStorageClient(
    api_key="<your-api-key>",
    ots_endpoint="https://<instance>.<region>.ots.aliyuncs.com",
    ots_instance_name="<instance-name>",
)

client.update_knowledge_base({
    "knowledgeBaseName": "product_docs_kb",
    "description": "Updated description",
    "tags": ["live"],
    "retrievalConfiguration": {
        "searchType": ["DENSE_VECTOR", "FULL_TEXT"],
        "rerankingConfiguration": {
            "type": "RRF",
            "numberOfResults": 10,
            "rrfConfiguration": {"k": 60}
        }
    }
})

TypeScript

The following sample code shows how to update a knowledge base:

import { AgentStorageClient } from '@tablestore/agent-storage';

const client = new AgentStorageClient({
  apiKey: '<your-api-key>',
  endpoint: 'https://<instance>.<region>.ots.aliyuncs.com',
  instanceName: '<instance-name>',
});

await client.updateKnowledgeBase({
  knowledgeBaseName: 'product_docs_kb',
  description: 'Updated description',
  tags: ['live'],
});

This example updates only the description and tags, and does not change the retrieval configuration.

API

Call UpdateKnowledgeBase to update a knowledge base. For complete fields, see UpdateKnowledgeBase.

Delete a knowledge base

Warning

This operation is irreversible. Deleting a knowledge base also deletes all documents and chunk data under it.

Console

On the knowledge base list page, click ··· in the upper-right corner of the target knowledge base card and select Delete. Enter the knowledge base name to confirm the deletion.

SDK

Python

The following sample code shows how to delete a knowledge base:

from tablestore_agent_storage import AgentStorageClient

client = AgentStorageClient(
    api_key="<your-api-key>",
    ots_endpoint="https://<instance>.<region>.ots.aliyuncs.com",
    ots_instance_name="<instance-name>",
)

client.delete_knowledge_base({
    "knowledgeBaseName": "product_docs_kb"
})

TypeScript

The following sample code shows how to delete a knowledge base:

import { AgentStorageClient } from '@tablestore/agent-storage';

const client = new AgentStorageClient({
  apiKey: '<your-api-key>',
  endpoint: 'https://<instance>.<region>.ots.aliyuncs.com',
  instanceName: '<instance-name>',
});

await client.deleteKnowledgeBase({ knowledgeBaseName: 'product_docs_kb' });

API

Call DeleteKnowledgeBase to delete a knowledge base. For complete fields, see DeleteKnowledgeBase.