Using a personal knowledge base scenario as an example, this topic describes how to use the Alibaba Cloud Milvus knowledge base to upload data, publish a version, and perform retrieval, and how to connect to a large language model to build a runnable local Q&A page. For 10 to 100 ordinary documents, the process can usually be completed within 10 to 20 minutes.
Solution overview
The overall pipeline consists of three parts: create a knowledge base in the console and import data, publish a version to make the content retrievable, and retrieve content through the SDK and pass the results to a large language model to generate answers. Data import and retrieval can be completed through the SDK, while version publishing must be performed in the console (the corresponding OpenAPI operation is not available yet).
Prerequisites
-
You have created an Alibaba Cloud account and are working in a region that supports knowledge bases (China (Hangzhou), China (Beijing), China (Zhangjiakou), or China (Shenzhen)).
-
You have completed RAM authorization for the calling account: log on to the RAM console with your Alibaba Cloud account, create a RAM user and select Using permanent AccessKey to access, and then grant the system policy
AliyunMilvusFullAccessto the user.NoteCreating a RAM user with an AccessKey pair triggers security verification (MFA code, mobile verification code, or facial recognition). The AccessKey Secret is displayed only once after creation. Use Save Result on the page to download and keep it. If you require stricter least-privilege control, you can create a custom policy that contains only
milvusknowledgebase:*and use it instead of the preceding system policy. -
You have prepared a large language model API that supports the OpenAI
chat/completionsprotocol, such as the OpenAI-compatible endpoint and API key of Alibaba Cloud Model Studio. -
Python 3.8 or later is installed on your local machine.
Store the AccessKey pair and the large language model API key only in local environment variables. Do not write them into documents, chat records, screenshots, or code repositories.
Step 1: Create a knowledge base
-
Log on to the Milvus console, switch to the target region in the region drop-down list in the top navigation bar, and then click Knowledge Base Service in the left-side navigation pane.
-
On the knowledge base page, click Create Knowledge Base and complete the following configurations.
Parameter
Description
Term
2 to 64 characters. Chinese characters are not supported. The name must be unique within the same tenant.
Data type
You can select Full-modal Knowledge Base or Structured Knowledge Base (chunks are generated by table row; only
xls,xlsx, andcsvare supported). The type cannot be changed after creation. This topic selects Full-modal Knowledge Base, which supportsmp4,mp3,png,pdf,ppt,txt,markdown,docs,xlsx,csv,jsonl, andfaq.Vector model
You can select Built-in Model or Private/External Model. The selection cannot be changed after creation. This topic uses Built-in Model, which defaults to
text-embedding-v3, and Vector Dimension is automatically displayed as 1024. If you select Private/External Model, Vector Dimension is displayed as-and cannot be edited.Specification configuration
Required. The current minimum specification is 4 CU (4 vCPUs / 16 GiB). You can adjust it on the details page after creation.
Network configuration
Required. Specify a virtual private cloud and a vSwitch. You can also create them on the spot.
NoteThe chunking strategy is not configured on the creation page. In the Processing Strategy section of the knowledge base details page, you can view the default strategy provided by the system (intelligent chunking, maximum length of 512 characters), or click Create Strategy to customize one. When you register data through the SDK, you can use
strategy_idto specify a strategy. -
Click Create Knowledge Base and wait until the knowledge base status changes from Creating to Running.
-
In the knowledge base list, record the knowledge base ID (in a format similar to
kd-803ae9b10cc31). This ID is required for subsequent SDK calls.
Step 2: Prepare data
We recommend that you first verify the results with a small number of Markdown, TXT, PDF, or Word documents. File names should express the document topic, and the body text should contain complete titles and context.
If you do not have suitable documents, you can use the public Chinese legal case dataset LeCaRDv2. Legal cases contain specific case numbers, case facts, and judgment results, which makes it easier to verify whether answers truly come from the knowledge base than general common-sense content. This dataset is used only to demonstrate retrieval technology and does not constitute legal advice.
Step 3: Configure the local environment
-
Create a virtual environment and install dependencies.
python3 -m venv .venv source .venv/bin/activate pip install alibabacloud_milvusknowledgebase20260604 requests streamlit -
Configure the runtime parameters.
KB_IDis the knowledge base ID that you recorded in Step 1, andKB_REGIONis the region where the knowledge base resides.export ALIBABA_CLOUD_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID" export ALIBABA_CLOUD_ACCESS_KEY_SECRET="YOUR_ACCESS_KEY_SECRET" export KB_REGION="cn-hangzhou" export KB_ID="kd-xxxxxxxxxxxxx" export KB_VERSION="LATEST_PUBLISHED" export LLM_BASE_URL="https://your-openai-compatible-api.example.com/v1" export LLM_API_KEY="YOUR_LLM_TOKEN" export LLM_MODEL="YOUR_MODEL_NAME"
Step 4: Write the sample code
Save the following content as kb_demo.py. It contains four parts: client initialization, data upload, retrieval, and large language model Q&A.
from __future__ import annotations
import os
from pathlib import Path
import requests
from alibabacloud_milvusknowledgebase20260604.client import Client
from alibabacloud_milvusknowledgebase20260604 import models as milvus_kb_models
from alibabacloud_tea_openapi import models as open_api_models
REGION = os.getenv("KB_REGION", "cn-hangzhou")
KB_ID = os.environ["KB_ID"]
KB_VERSION = os.getenv("KB_VERSION", "LATEST_PUBLISHED")
ENDPOINT = f"milvusknowledgebase.{REGION}.aliyuncs.com"
client = Client(open_api_models.Config(
access_key_id=os.environ["ALIBABA_CLOUD_ACCESS_KEY_ID"],
access_key_secret=os.environ["ALIBABA_CLOUD_ACCESS_KEY_SECRET"],
endpoint=ENDPOINT,
region_id=REGION,
connect_timeout=10_000,
read_timeout=60_000,
))
def upload_document(file_path: str) -> dict:
path = Path(file_path).resolve()
size = path.stat().st_size
presigned = client.get_knowledge_base_pre_signed_url(
KB_ID,
milvus_kb_models.GetKnowledgeBasePreSignedUrlRequest(
knowledge_base_id=KB_ID,
documents=[
milvus_kb_models.GetKnowledgeBasePreSignedUrlRequestDocuments(
path=path.name, name=path.name, size=size,
)
],
expires_in=3600,
),
)
upload_url = presigned.body.data.pre_signed_urls[0]
# The presigned URL is signed with an empty Content-Type. Do not include Content-Type in the PUT request.
with path.open("rb") as source:
response = requests.put(upload_url, data=source, timeout=120)
response.raise_for_status()
added = client.add_documents(
KB_ID,
milvus_kb_models.AddDocumentsRequest(
knowledge_base_id=KB_ID,
import_type="LOCAL_UPLOAD",
documents=[
milvus_kb_models.AddDocumentsRequestDocuments(
path=path.name, name=path.name, size=size,
)
],
dedup=milvus_kb_models.AddDocumentsRequestDedup(
doc_name_dedup=True, content_dedup=False,
),
),
)
return added.body
def search_knowledge_base(query: str, page_size: int = 6):
resp = client.search_knowledge_base(
KB_ID,
milvus_kb_models.SearchKnowledgeBaseRequest(
query=query,
version=KB_VERSION,
page_number=1,
page_size=page_size,
retrieval_config=milvus_kb_models.SearchKnowledgeBaseRequestRetrievalConfig(
candidate_count=48,
min_score=0,
semantic_weight=0.5,
enable_query_expansion=False,
),
),
)
return resp.body
def chat(messages: list[dict[str, str]]) -> str:
base_url = os.environ["LLM_BASE_URL"].rstrip("/")
response = requests.post(
f"{base_url}/chat/completions",
headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"},
json={
"model": os.environ["LLM_MODEL"],
"messages": messages,
"temperature": 0.1,
},
timeout=60,
)
response.raise_for_status()
return response.json()["choices"][0]["message"]["content"].strip()
def ask(question: str) -> tuple[str, object]:
search_query = chat([
{
"role": "system",
"content": "Rewrite the user question into a self-contained query that is suitable for knowledge base retrieval. Output only the query.",
},
{"role": "user", "content": question},
])
search_result = search_knowledge_base(search_query)
results = search_result.results or []
context = "\n\n".join(
f"[Source {i}] {item.document_name or ''}\n{item.content}"
for i, item in enumerate(results, start=1)
)
answer = chat([
{
"role": "system",
"content": "Answer only based on the given materials. Mark [Source N] when you cite materials. State clearly when the materials are insufficient.",
},
{"role": "user", "content": f"Question: {question}\n\nMaterials:\n{context}"},
])
return answer, search_result
Step 5: Upload data
The upload consists of three steps: obtain a presigned URL, write the file to OSS, and register the data. upload_document() encapsulates the complete process.
from kb_demo import upload_document
result = upload_document("./documents/example.md")
print(result.request_id)
For batch uploads, we recommend that you start with batches of 10 documents.
from pathlib import Path
from kb_demo import upload_document
for path in sorted(Path("./documents").glob("*.md")):
upload_document(str(path))
print("Submitted:", path.name)
The presigned URL is signed with an empty Content-Type. The PUT request must not include the Content-Type header. Otherwise, OSS returns 403 SignatureDoesNotMatch. If the PUT request fails due to network issues, you can simply retry it.
When you register data, set both path and name to the file name only. Do not use the object path in the presigned URL. Passing an internal path such as direct_upload/… returns 400 Path must not use an internal storage prefix.; leaving path empty returns 400 Path can't be empty.
In the response of a successful registration, data.documents may be an empty array, which does not mean that the registration failed. Refer to the number of documents and number of chunks on the knowledge base details page, or the subsequent retrieval results.
Only sending a PUT request to upload a file to OSS does not add the data to the knowledge base. You must call AddDocuments to complete the registration. After successful registration, run in the response is RUNNING, which indicates that parsing and chunking are in progress.
After the upload is complete, check the data status and the number of chunks on the Data Management page in the console. Publish a version only after the status becomes Processing Completed. Click View Chunks to preview the chunking results.
Step 6: Publish a version
After data is imported, it is in the unpublished state. The data can be retrieved only after you publish a version. Currently, this operation can be performed only in the console.
-
Go to the knowledge base details page, click the Version Management tab, confirm that the page indicates there are changes to be published, and then click Publish Version.
-
In the Confirm Changes step, check the change log and click Next.
-
In the Enter Description step, enter the release description (no more than 200 characters), click Publish, and wait for the Version published successfully message.
-
View the version number in Version Records. The first release is
v1, and its status is Published.
After the version is published, update the local environment variables.
export KB_VERSION="v1"
Step 7: Retrieve data
from kb_demo import search_knowledge_base
result = search_knowledge_base("What conditions must be met to terminate a contract?")
for item in result.results or []:
print(item.document_name, item.score)
print(item.content[:300])
For version, we recommend that you use an explicit version number (such as v1 or v2). You can also use LATEST_PUBLISHED to retrieve the latest published version. Do not use DRAFT to provide a stable external Q&A service.
In the example, min_score=0 also returns chunks with low relevance. In production environments, we recommend that you increase min_score or decrease page_size based on actual results, to prevent irrelevant content from entering the large language model context.
You can also adjust TopK, the semantic retrieval weight, the rerank model, and the filter scope on the Retrieval Verification page in the console to quickly compare retrieval results.
Step 8: Connect to a large language model for Q&A
ask() first lets the large language model rewrite the question into a query suitable for retrieval, then calls SearchKnowledgeBase, and finally lets the large language model summarize an answer based only on the returned chunks.
from kb_demo import ask
answer, search_result = ask("What do these documents stipulate about contract termination?")
print(answer)
print("Request ID:", search_result.request_id)
Step 9: Build a web application
-
Save the following content as
app.py.import streamlit as st from kb_demo import ask st.set_page_config(page_title="Knowledge base Q&A") st.title("Knowledge base Q&A") question = st.chat_input("Enter your question") if question: with st.chat_message("user"): st.write(question) with st.chat_message("assistant"): with st.spinner("Retrieving from the knowledge base…"): answer, result = ask(question) st.write(answer) with st.expander("View retrieval sources"): for item in result.results or []: st.markdown(f"**{item.document_name or 'Unnamed document'}**") st.caption(f"score: {item.score} · Request ID: {result.request_id}") st.write(item.content) -
Start the application.
streamlit run app.py
The browser automatically opens the local knowledge base Q&A page.
FAQ
|
Symptom |
Cause and solution |
|
The API call returns |
The calling account does not have knowledge base permissions. Grant |
|
The PUT upload returns |
The request includes the |
|
Retrieval returns |
No version has been published for the knowledge base, or |
|
The PUT upload occasionally fails to connect |
The OSS connection is unstable. Simply retry the file. |
|
Knowledge Base Service is not displayed in the left-side navigation pane of the console |
The current region is not supported, or the menu has not finished loading. Switch to a supported region and check again. |
Pre-launch checklist
-
Use an independent RAM user with least privileges and rotatable credentials. Do not use the AccessKey pair of your Alibaba Cloud account for a long time.
-
Upload only data that you are authorized to process.
-
Every answer must be supported by evidence that can be found in the expanded source chunks.
-
When you deploy the page externally, add identity authentication, HTTPS, and access log masking.
-
The availability scope of API operations and the permission requirements are subject to the product documentation and the actual console.