All Products
Search
Document Center

Vector Retrieval Service for Milvus:Build an intelligent customer service Q&A application with an Alibaba Cloud Milvus knowledge base

Last Updated:Oct 21, 2026

Using an intelligent customer service scenario as an example, this topic describes how to use an Alibaba Cloud Milvus knowledge base to build a tagged knowledge index for product manuals, FAQs, and return/exchange, warranty, and invoice policies, and how to build a customer service assistant page that displays answer sources by combining tag filtering, a rerank model, and large model generation.

Solution overview

The end-to-end flow is: define tags and create a knowledge base in the console → import customer service materials in batches by tag → publish a version → retrieve through the SDK (with optional tag filtering) → have the large model generate an answer with source annotations → serve a Q&A page with Flask.

This topic focuses on the three things specific to customer service scenarios: tag (metadata) filtering, the rerank model, and answer traceability. For the basic knowledge base flow (presigned upload, data registration, version publishing) and the minimal Q&A implementation, see Build a personal knowledge Q&A application.

We recommend that you prepare 5 to 20 PDF, DOCX, Markdown, or TXT documents in advance. The whole process takes about 20 to 30 minutes (document parsing time depends on the number and size of the documents).

Prerequisites

  • A knowledge base has been created in a region that supports knowledge bases (China (Hangzhou), China (Beijing), China (Zhangjiakou), or China (Shenzhen)), and the knowledge base ID has been recorded (in a format such as kd-803ae9b10cc31).

  • A RAM user has been created with the Alibaba Cloud account, Use a permanent AccessKey to access has been selected, and the system policy AliyunMilvusFullAccess has been granted to the RAM user.

  • A large model endpoint and API key that support the OpenAI chat/completions protocol are ready, for example, the OpenAI-compatible endpoint of Alibaba Cloud Model Studio.

  • Python 3.8 or later is installed locally.

Important

The AccessKey pair and the large model API key are stored only in the local config.json file. Do not commit them to a code repository and do not send them to chat groups. After the demonstration, rotate or delete the temporary credentials promptly.

Step 1: Define tags

Tags are used to label materials with business dimensions (such as material type, product line, and effective date). Tags are written to the knowledge base during import and used for filtering during retrieval. They are the key to distinguishing "policies", "manuals", and "FAQs" in customer service scenarios.

  1. Log on to the Milvus console and go to the details page of the target knowledge base.

  2. Find Tags below Basic Information and click Manage.

  3. In the Tag Management dialog box, enter a tag name, select a field type, and click Add. This topic uses the following three tags, all of the string type:

    Tag name

    Description

    Example

    docType

    Material type

    policy, manual, faq

    productLine

    Applicable product line

    all, phone

    effectiveDate

    Effective date

    2026-01-01

  4. After the three tags are added, click Done. The Tags field on the details page then shows "3 tags".

Note
  • The field type can be string, int64, list, float32, or bool.

  • Tags can be added later to an existing knowledge base, without rebuilding the knowledge base.

  • The dialog box prompts that tag changes affect index building. In practice, however, adding or deleting tag definitions does not rebuild or migrate data that has already been imported, and existing tag values are not cleared. We still recommend that you finalize the tag definitions before batch import, so that console display, option management, and value type conversion are consistent from the start.

  • The Tag Options column is used to preset selectable values for a tag and affects only how values are entered in the console: if it is left empty, you must manually type the tag value when filtering by that tag in the console; if you enter comma-separated values (for example, after-sales,logistics,billing), the console switches to a drop-down list, which avoids typos. It does not validate values written through the API — passing a value outside the options with AddDocuments still succeeds, so the value range must still be guaranteed by the import manifest itself.

Step 2: Prepare materials and the import manifest

  1. Place the customer service materials in the local documents/ directory, for example:

    kb-demo/
    ├── documents/
    │   ├── shipping-policy.md
    │   ├── return-policy.md
    │   ├── warranty-policy.md
    │   ├── phone-manual.md
    │   └── invoice-faq.md
  2. Create documents.jsonl, where each line describes one document and its tags. Tag names must be the same as those defined in Step 1.

    {"path": "documents/shipping-policy.md", "metadata": {"docType": "policy", "productLine": "all", "effectiveDate": "2026-01-01"}}
    {"path": "documents/return-policy.md", "metadata": {"docType": "policy", "productLine": "all", "effectiveDate": "2026-01-01"}}
    {"path": "documents/warranty-policy.md", "metadata": {"docType": "policy", "productLine": "phone", "effectiveDate": "2026-03-01"}}
    {"path": "documents/phone-manual.md", "metadata": {"docType": "manual", "productLine": "phone", "effectiveDate": "2026-03-01"}}
    {"path": "documents/invoice-faq.md", "metadata": {"docType": "faq", "productLine": "all", "effectiveDate": "2026-02-01"}}
Note

File names should convey the topic of the material, and the body text should contain the complete title and context. Import the officially effective policies first, and avoid keeping conflicting older versions at the same time.

Step 3: Configure runtime parameters

  1. Create the project directory and install the dependencies.

    mkdir -p kb-demo/documents kb-demo/templates
    cd kb-demo
    python3 -m venv .venv
    source .venv/bin/activate
    pip install Flask==3.1.1 requests==2.32.4 alibabacloud-milvusknowledgebase20260604==1.0.0
  2. Create config.json and enter your own credentials and knowledge base information.

    {
      "aliyun": {
        "access_key_id": "YOUR_ACCESS_KEY_ID",
        "access_key_secret": "YOUR_ACCESS_KEY_SECRET",
        "region_id": "cn-hangzhou",
        "knowledge_base_id": "kd-xxxxxxxxxxxxx",
        "knowledge_base_version": "LATEST_PUBLISHED"
      },
      "llm": {
        "enabled": true,
        "base_url": "https://YOUR_OPENAI_COMPATIBLE_ENDPOINT/v1",
        "api_key": "YOUR_LLM_API_KEY",
        "model": "YOUR_MODEL_NAME"
      },
      "upload": {
        "default_meta_fields": {}
      },
      "retrieval": {
        "page_size": 6,
        "candidate_count": 48,
        "min_score": 0.35,
        "semantic_weight": 0.7,
        "enable_query_expansion": true,
        "rerank_model_name": "qwen3-rerank",
        "tag_filter": {
          "relation": "and",
          "conditions": []
        }
      },
      "scenario": {
        "title": "Intelligent customer service knowledge base Q&A",
        "system_prompt": "You are an enterprise customer service assistant. Answer only based on the retrieved materials, and use a clear, friendly customer service tone; annotate key conclusions with [Source N]. If the retrieved materials do not directly cover the user's question, you must answer 'The existing materials do not state this explicitly.' You must not infer conclusions from partially relevant materials, and you must not add contact information or service channels that are not in the materials.",
        "image_enabled": false,
        "sample_questions": [
          "How long after payment are goods usually shipped?",
          "Can an item still be repaired after the warranty period expires?",
          "What conditions must be met to apply for a return?"
        ]
      }
    }

Considerations for the retrieval parameter values in this scenario:

  • min_score=0.35: reduces the amount of low-relevance content that enters the answer.

  • semantic_weight=0.7: leans toward semantic retrieval to cover colloquial customer service phrasings.

  • rerank_model_name: performs secondary ranking on candidate chunks to improve the accuracy of the top result.

  • tag_filter.conditions: leave it empty to apply no filtering. For how to filter by tag, see Step 6.

Step 4: Write the knowledge base client

Save the following content as kb_client.py, which encapsulates tagged upload and filtered retrieval.

"""Milvus knowledge base OpenAPI SDK: tagged local upload and retrieval of published versions."""

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping, Sequence

import requests
from alibabacloud_milvusknowledgebase20260604 import models as models
from alibabacloud_milvusknowledgebase20260604.client import Client
from alibabacloud_tea_openapi import models as openapi_models


@dataclass(frozen=True)
class LocalDocument:
    path: Path
    object_path: str

    @classmethod
    def from_path(cls, value: str | Path) -> "LocalDocument":
        path = Path(value).expanduser().resolve()
        if not path.is_file():
            raise FileNotFoundError(path)
        return cls(path=path, object_path=path.name)


@dataclass(frozen=True)
class TagCondition:
    field: str
    op: str
    value: Any


@dataclass(frozen=True)
class SearchOptions:
    version: str = "LATEST_PUBLISHED"
    page_size: int = 6
    candidate_count: int = 48
    min_score: float = 0.0
    semantic_weight: float = 0.5
    enable_query_expansion: bool = True
    rerank_model_name: str | None = None
    tag_relation: str = "and"
    tag_conditions: tuple[TagCondition, ...] = ()


class KnowledgeBaseClient:
    def __init__(
        self,
        access_key_id: str,
        access_key_secret: str,
        region_id: str,
    ) -> None:
        endpoint = f"milvusknowledgebase.{region_id}.aliyuncs.com"
        self.client = Client(
            openapi_models.Config(
                access_key_id=access_key_id,
                access_key_secret=access_key_secret,
                region_id=region_id,
                endpoint=endpoint,
                connect_timeout=10_000,
                read_timeout=60_000,
            )
        )
        self.http = requests.Session()

    @staticmethod
    def _check(body: Any, action: str) -> None:
        # The code field is not returned in a successful response (its value is None), so only an explicitly non-zero code is treated as a failure.
        if body is None:
            raise RuntimeError(f"{action} returned an empty response")
        if getattr(body, "success", None) is False or (
            getattr(body, "code", None) not in (None, 0, "0")
        ):
            raise RuntimeError(
                f"{action} failed: {getattr(body, 'message', 'unknown')}; "
                f"requestId={getattr(body, 'request_id', '')}"
            )

    def upload(
        self,
        knowledge_base_id: str,
        file_paths: Sequence[str | Path],
        meta_fields: Mapping[str, Any] | None = None,
    ) -> dict[str, Any]:
        docs = [LocalDocument.from_path(path) for path in file_paths]
        presign_docs = [
            models.GetKnowledgeBasePreSignedUrlRequestDocuments(
                path=doc.object_path,
                name=doc.path.name,
                size=doc.path.stat().st_size,
            )
            for doc in docs
        ]
        response = self.client.get_knowledge_base_pre_signed_url(
            knowledge_base_id,
            models.GetKnowledgeBasePreSignedUrlRequest(
                knowledge_base_id=knowledge_base_id,
                documents=presign_docs,
                expires_in=3600,
            ),
        )
        body = response.body
        self._check(body, "GetKnowledgeBasePreSignedUrl")

        urls = list(body.data.pre_signed_urls or [])
        if len(urls) != len(docs):
            raise RuntimeError("The number of presigned URLs does not match the number of files")

        for doc, url in zip(docs, urls, strict=True):
            with doc.path.open("rb") as source:
                # The presigned URL is signed with an empty Content-Type, so the request must not carry a Content-Type header.
                self.http.put(url, data=source, timeout=120).raise_for_status()

        add_docs = [
            models.AddDocumentsRequestDocuments(
                path=doc.object_path,
                name=doc.path.name,
                size=doc.path.stat().st_size,
            )
            for doc in docs
        ]
        response = self.client.add_documents(
            knowledge_base_id,
            models.AddDocumentsRequest(
                knowledge_base_id=knowledge_base_id,
                import_type="LOCAL_UPLOAD",
                documents=add_docs,
                meta_fields=dict(meta_fields) if meta_fields else None,
                dedup=models.AddDocumentsRequestDedup(
                    doc_name_dedup=True,
                    content_dedup=False,
                ),
            ),
        )
        body = response.body
        self._check(body, "AddDocuments")

        errors = list(getattr(body.data, "errors", None) or [])
        if errors:
            raise RuntimeError(f"Data registration failed: {errors}")
        return body.to_map()

    def search(
        self,
        knowledge_base_id: str,
        query: str,
        options: SearchOptions,
        image_url: str | None = None,
    ) -> dict[str, Any]:
        tag_filter = None
        if options.tag_conditions:
            tag_filter = models.SearchKnowledgeBaseRequestTagFilter(
                relation=options.tag_relation,
                conditions=[
                    models.SearchKnowledgeBaseRequestTagFilterConditions(
                        field=condition.field,
                        op=condition.op,
                        value=condition.value,
                    )
                    for condition in options.tag_conditions
                ],
            )
        response = self.client.search_knowledge_base(
            knowledge_base_id,
            models.SearchKnowledgeBaseRequest(
                query=query,
                version=options.version,
                page_number=1,
                page_size=options.page_size,
                rerank_model_name=options.rerank_model_name,
                tag_filter=tag_filter,
                image=(
                    models.SearchKnowledgeBaseRequestImage(url=image_url)
                    if image_url
                    else None
                ),
                retrieval_config=models.SearchKnowledgeBaseRequestRetrievalConfig(
                    candidate_count=options.candidate_count,
                    min_score=options.min_score,
                    semantic_weight=options.semantic_weight,
                    enable_query_expansion=options.enable_query_expansion,
                ),
            ),
        )
        body = response.body
        self._check(body, "SearchKnowledgeBase")
        return body.to_map()

Step 5: Write the batch upload script

Save the following content as upload.py. The MetaFields parameter of AddDocuments applies to an entire batch, so the script first groups documents by tag and then submits them in batches of 100.

"""Upload local materials in batches by tag; after the upload, go to the console to wait for parsing and then publish a version."""

from __future__ import annotations

import argparse
import json
from collections import defaultdict
from dataclasses import dataclass
from pathlib import Path
from typing import Any

from kb_client import KnowledgeBaseClient


@dataclass(frozen=True)
class ManifestEntry:
    path: Path
    metadata: dict[str, Any]


def load_manifest(path: Path) -> list[ManifestEntry]:
    entries: list[ManifestEntry] = []
    for line_number, raw_line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        if not raw_line.strip():
            continue
        value = json.loads(raw_line)
        file_path = (path.parent / str(value["path"])).resolve()
        metadata = value.get("metadata") or {}
        if not isinstance(metadata, dict):
            raise ValueError(f"metadata on line {line_number} of the manifest must be an object")
        entries.append(ManifestEntry(file_path, metadata))
    return entries


def discover(paths: list[str], default_metadata: dict[str, Any]) -> list[ManifestEntry]:
    entries: list[ManifestEntry] = []
    for value in paths:
        path = Path(value).expanduser()
        files = sorted(item for item in path.rglob("*") if item.is_file()) if path.is_dir() else [path]
        entries.extend(ManifestEntry(item.resolve(), default_metadata) for item in files)
    return entries


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("paths", nargs="*", help="Files or directories; multiple values can be passed at the same time")
    parser.add_argument("--config", default="config.json")
    parser.add_argument("--manifest", help="JSONL file; each line contains path and metadata")
    args = parser.parse_args()

    config = json.loads(Path(args.config).read_text(encoding="utf-8"))
    aliyun = config["aliyun"]
    upload_config = config.get("upload") or {}
    default_metadata = upload_config.get("default_meta_fields") or {}
    entries = (
        load_manifest(Path(args.manifest).expanduser().resolve())
        if args.manifest
        else discover(args.paths, default_metadata)
    )
    if not entries:
        raise SystemExit("No files found to upload")

    grouped: dict[str, list[ManifestEntry]] = defaultdict(list)
    for entry in entries:
        key = json.dumps(entry.metadata, ensure_ascii=False, sort_keys=True)
        grouped[key].append(entry)

    client = KnowledgeBaseClient(
        aliyun["access_key_id"], aliyun["access_key_secret"], aliyun["region_id"]
    )
    submitted = 0
    for metadata_key, group in grouped.items():
        metadata = json.loads(metadata_key)
        for start in range(0, len(group), 100):
            batch = group[start : start + 100]
            client.upload(
                aliyun["knowledge_base_id"],
                [entry.path for entry in batch],
                meta_fields=metadata,
            )
            submitted += len(batch)
            print(f"Submitted {submitted}/{len(entries)} documents; metadata={metadata}")


if __name__ == "__main__":
    main()

Run the upload:

python upload.py --manifest documents.jsonl

The output echoes each batch grouped by tag. For example, with 5 documents and 4 tag combinations, 4 batches are submitted:

Submitted 2/5 documents; metadata={'docType': 'policy', 'effectiveDate': '2026-01-01', 'productLine': 'all'}
Submitted 3/5 documents; metadata={'docType': 'policy', 'effectiveDate': '2026-03-01', 'productLine': 'phone'}
Submitted 4/5 documents; metadata={'docType': 'manual', 'effectiveDate': '2026-03-01', 'productLine': 'phone'}
Submitted 5/5 documents; metadata={'docType': 'faq', 'effectiveDate': '2026-02-01', 'productLine': 'all'}
Important

A successful response from the upload operation only means that asynchronous parsing has been submitted. Go to the Data Management page in the console and confirm that the material status changes to Processed (the tag column shows the written tags, such as productLine=all, docType=faq +1) before you proceed to the next step.

Step 6: Publish a version

After parsing is complete, the materials are in the unpublished state and can be retrieved only after a version is published. Currently, this operation can be performed only in the console.

  1. Go to the knowledge base details page, click the Version Management tab, confirm that there are changes pending publication, and then click Publish Version.

  2. In the Confirm Changes step, review the change log (which lists each newly added data item) and click Next.

  3. In the Enter Description step, enter the release description (no more than 200 characters) and click Publish.

  4. In Version Records, confirm that the status of the new version is Published and record the version number (the first release is v1, followed by v2 and v3).

When knowledge_base_version in config.json is set to LATEST_PUBLISHED, the latest published version is retrieved automatically. You can also change it to an explicit version number. Do not use DRAFT to provide a stable Q&A service externally.

Important

By default, at most 3 published versions can exist at the same time. This limit is calculated per tenant and does not increase with the instance CU specification. To raise it, you must submit a ticket for evaluation; there is currently no self-service quota application entry for users. Once the limit is reached, the Publish Version button is dimmed while the page still shows "There are N changes pending publication". You must delete old versions that are no longer needed in Version Records before you can continue publishing. Version deletion cannot be undone, and a deleted version immediately becomes unavailable for retrieval (the data is then cleaned up asynchronously in the background). Therefore, before deleting a version, you must confirm that no application is pinned to that version number, and switch any application still using it to the new version. If materials are updated frequently, we recommend that you keep only "the currently effective version plus the most recent historical version".

Each retrieval result also returns scoreDetails, in a format such as {"keywordScore": 0.368, "semanticScore": 0.819}, which is used to determine whether the hit comes mainly from keyword matching or semantic matching.

The three scores mean the following: keywordScore is the keyword similarity; semanticScore is the vector similarity when reranking is not enabled, and the score of the rerank model when reranking is enabled; score is the final ranking and filtering score obtained by weighting the two, calculated as score ≈ (1-semantic_weight) × keywordScore + semantic_weight × semanticScore (rank features may also be added). min_score filters on this final score.

The score scale changes with the model, with whether reranking is enabled, and with different weights. Scores cannot be compared across configurations, and there is no universally recommended threshold. We recommend that you first set min_score to 0 to retrieve a batch of results and manually annotate their relevance, then choose a threshold based on the distribution of recall and false recall. Each time you adjust semantic_weight or turn reranking on or off, you must recalibrate.

Step 7: Filter retrieval by tag

Configure filter conditions in retrieval.tag_filter.conditions in config.json to limit the retrieval scope to the specified tags. For example, to retrieve only policy materials:

"tag_filter": {
  "relation": "and",
  "conditions": [
    {"field": "docType", "op": "=", "value": "policy"}
  ]
}

relation supports and (all conditions must be met) and or (any condition must be met). op supports the following operators:

Operator

Actual observed behavior

=

✅ Exact match (for a tag of the int64 type, either a number or a string can be passed)

in, not in

✅ Belongs to or does not belong to a given set

≠, >, <, ≥, ≤, empty, not empty, start with, end with

❌ Currently ineffective: the condition is ignored and all data is returned, without an error

contains, not contains

❌ An alias for in/not in; it is not string containment, and is not recommended

Important

Only three operators are actually usable: =, in, and not in. Although the other operators appear in the "Supported operators" list of the 400 Unsupported tag filter operator error, tests show that when they are passed, the condition is silently ignored and all unfiltered data is returned. Among them, contains and not contains are merely aliases for in/not in and are not string containment; passing a string fragment returns 0 results. Therefore: to filter by a numeric or date range, use in to enumerate the values; to check whether a tag is empty, use = "". After configuring filter conditions, be sure to compare the total number of results with the unfiltered results to confirm that filtering actually takes effect. In addition, if field is set to an undefined tag name, the operation likewise does not return an error and simply returns 0 results; setting op to an alias such as eq, ==, equal, or like returns 400 Unsupported tag filter operator.

Using the corpus in this topic as an example, the retrieval scope under different filter conditions is as follows:

Filter condition

Matched materials

conditions not set

All materials

docType = policy

Shipping, return and exchange, and warranty policies

docType = faq

Invoice FAQ

docType = policy and productLine = phone

Warranty policy

docType = faq or docType = manual (relation is or)

Invoice FAQ and phone manual

effectiveDate = 2026-01-01

The two policies effective on that date

Step 8: Write the Q&A service

Save the following content as app.py.

"""Knowledge base retrieval + OpenAI-compatible large model Q&A service."""

from __future__ import annotations

import json
from pathlib import Path
from typing import Any

import requests
from flask import Flask, jsonify, render_template, request

from kb_client import KnowledgeBaseClient, SearchOptions, TagCondition


CONFIG = json.loads(Path("config.json").read_text(encoding="utf-8"))
ALIYUN = CONFIG["aliyun"]
LLM = CONFIG.get("llm", {})
SCENARIO = CONFIG.get("scenario", {})
RETRIEVAL = CONFIG.get("retrieval", {})
KB = KnowledgeBaseClient(
    ALIYUN["access_key_id"], ALIYUN["access_key_secret"], ALIYUN["region_id"]
)
app = Flask(__name__)


def search_options() -> SearchOptions:
    raw_conditions = (RETRIEVAL.get("tag_filter") or {}).get("conditions") or []
    return SearchOptions(
        version=ALIYUN.get("knowledge_base_version", "LATEST_PUBLISHED"),
        page_size=int(RETRIEVAL.get("page_size", 6)),
        candidate_count=int(RETRIEVAL.get("candidate_count", 48)),
        min_score=float(RETRIEVAL.get("min_score", 0.0)),
        semantic_weight=float(RETRIEVAL.get("semantic_weight", 0.5)),
        enable_query_expansion=bool(RETRIEVAL.get("enable_query_expansion", True)),
        rerank_model_name=str(RETRIEVAL.get("rerank_model_name") or "") or None,
        tag_relation=str((RETRIEVAL.get("tag_filter") or {}).get("relation", "and")),
        tag_conditions=tuple(
            TagCondition(str(item["field"]), str(item["op"]), item.get("value"))
            for item in raw_conditions
        ),
    )


def find_results(payload: Any) -> list[dict[str, Any]]:
    """Extract the list of retrieved chunks from the response."""
    if isinstance(payload, list):
        return [item for item in payload if isinstance(item, dict)]
    if not isinstance(payload, dict):
        return []
    for key in ("results", "Results"):
        if isinstance(payload.get(key), list):
            return payload[key]
    for key in ("data", "Data"):
        found = find_results(payload.get(key))
        if found:
            return found
    return []


def field(item: dict[str, Any], *names: str) -> Any:
    for name in names:
        if item.get(name) not in (None, ""):
            return item[name]
    return ""


def llm_answer(question: str, results: list[dict[str, Any]]) -> str:
    if not LLM.get("enabled", True):
        return "The large model is not enabled; see the retrieval results below."
    if not results:
        return "The existing materials do not state this explicitly."
    context = "\n\n".join(
        f"[Source {index}] {field(item, 'documentName', 'DocumentName')}\n"
        f"{field(item, 'content', 'Content')}"
        for index, item in enumerate(results, 1)
    )
    url = str(LLM["base_url"]).rstrip("/") + "/chat/completions"
    response = requests.post(
        url,
        headers={"Authorization": f"Bearer {LLM['api_key']}"},
        json={
            "model": LLM["model"],
            "temperature": 0.1,
            "messages": [
                {"role": "system", "content": SCENARIO.get("system_prompt", "Answer only based on the materials.")},
                {"role": "user", "content": f"Question: {question}\n\nRetrieved materials:\n{context}"},
            ],
        },
        timeout=90,
    )
    response.raise_for_status()
    return response.json()["choices"][0]["message"]["content"].strip()


@app.get("/")
def index():
    return render_template(
        "index.html",
        title=SCENARIO.get("title", "Knowledge base Q&A"),
        sample_questions=SCENARIO.get("sample_questions", []),
        image_enabled=bool(SCENARIO.get("image_enabled", False)),
    )


@app.post("/api/ask")
def ask():
    payload = request.get_json(silent=True) or {}
    question = str(payload.get("question", "")).strip()
    image_url = str(payload.get("image_url", "")).strip() or None
    if not question:
        return jsonify({"error": "The question cannot be empty"}), 400
    try:
        raw = KB.search(
            ALIYUN["knowledge_base_id"],
            question,
            options=search_options(),
            image_url=image_url,
        )
        results = find_results(raw)
        return jsonify({"answer": llm_answer(question, results), "sources": results})
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


if __name__ == "__main__":
    app.run(host="127.0.0.1", port=7860, debug=False)

Save the following content as templates/index.html.

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <title>{{ title }}</title>
  <style>
    body{margin:0;background:#f6f7fb;color:#1f2937;font:15px system-ui,sans-serif}
    main{max-width:860px;margin:0 auto;padding:42px 18px}
    .card{background:#fff;border:1px solid #e5e7eb;border-radius:18px;padding:24px}
    textarea{box-sizing:border-box;width:100%;min-height:100px;border:1px solid #d1d5db;border-radius:12px;padding:14px;font:inherit}
    button{margin-top:12px;border:0;border-radius:10px;padding:11px 18px;background:#4f46e5;color:#fff;cursor:pointer}
    .chip{background:#eef2ff;color:#3730a3;margin:4px;padding:7px 10px}
    pre{white-space:pre-wrap;line-height:1.65}.muted{color:#6b7280}.source{border-top:1px solid #eee;padding:12px 0}
  </style>
</head>
<body><main><h1>{{ title }}</h1><p class="muted">Answers are generated from the published knowledge base content, with the retrieval evidence displayed.</p>
  <div>{% for q in sample_questions %}<button class="chip" onclick='setQ({{ q|tojson }})'>{{ q }}</button>{% endfor %}</div>
  <section class="card">
    <textarea id="q" placeholder="Enter your question"></textarea>
    <button id="ask" onclick="ask()">Send</button>
    <pre id="answer"></pre>
    <div id="sources"></div>
  </section>
</main><script>
const q=document.querySelector('#q'), answer=document.querySelector('#answer'), sources=document.querySelector('#sources');
function setQ(value){q.value=value;q.focus()}
async function ask(){
  const text=q.value.trim(); if(!text) return;
  answer.textContent='Retrieving and generating…'; sources.innerHTML='';
  const res=await fetch('/api/ask',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({question:text})});
  const data=await res.json();
  answer.textContent=data.answer||('Error: '+data.error);
  for(const [i,item] of (data.sources||[]).entries()){
    const div=document.createElement('div'); div.className='source';
    div.textContent=`Source ${i+1}: ${item.documentName||''}\n${item.content||''}`;
    sources.appendChild(div);
  }
}
</script></body></html>
Important

The onclick attribute of the sample question buttons must be wrapped in single quotation marks (onclick='setQ({{ q|tojson }})'). If you use double quotation marks, the double quotation marks in the JSON output by tojson close the HTML attribute prematurely, and clicking the button has no effect.

Step 9: Start and verify

  1. Start the service.

    python app.py
  2. In a browser, go to http://127.0.0.1:7860. Click a sample question, or enter a question manually, and then click Send.

  3. You can also call the API directly to verify.

curl -sS http://127.0.0.1:7860/api/ask \
  -H 'Content-Type: application/json' \
  -d '{"question":"How long after payment are orders usually shipped?"}'

Check the following four items for acceptance:

  • The page opens properly, and after you submit a question, both the answer and the retrieval sources are returned.

  • For each [Source N] in the answer, you can find the corresponding content chunk in the sources section below.

  • When you ask about content that the materials do not cover, the answer is "The existing materials do not state this explicitly" instead of fabricated facts.

  • After you add materials and publish a new version in the console, the page can retrieve the new content.

Tuning recommendations

  • Adjust the chunk length based on the material format: Customer service materials are mostly short FAQ entries and policy clauses. On the knowledge base details page, in Processing Strategy, click Create Strategy and set the chunking method to Intelligent Chunking or Chunk by Length. Note that the unit of the maximum chunk length is characters (512 characters by default). For short-entry scenarios, we recommend 380 to 580 characters (roughly equivalent to 256 to 384 tokens). After the strategy is created, you can specify it during import by using AddDocumentsRequest.strategy_id.

  • Reranking and min_score must be tuned together: After you enable rerank_model_name, the rerank score and the vector similarity score are not on the same scale. Keeping the original min_score may trim the results too aggressively (in tests, each of the three customer service questions retained only one hit after reranking was enabled). If a question requires combining multiple materials to answer, lower min_score appropriately when reranking is enabled.

  • Beware of incorrect conclusions caused by "partially relevant" materials: When a question is literally related to a piece of material but is not semantically covered by it (for example, the material only describes shipping lead times while the user asks "Is cash on delivery supported?"), the large model may draw an incorrect conclusion from it. In system_prompt, explicitly require that "if the materials do not directly cover the question, you must answer that you do not know, and must not infer from partially relevant materials", and spot-check with real customer phrasings before going live.

  • Test with real customer phrasings instead of using only manual titles as queries. A higher semantic_weight (such as 0.7) better covers colloquial expressions.

  • Import only the policies that are officially in effect and avoid keeping conflicting old versions at the same time. After a policy is updated, publish a new version and use the effectiveDate tag to distinguish versions.

FAQ

Symptom

Cause and solution

Retrieval returns 400 Unsupported tag filter operator

op uses an alias such as eq, ==, or like. Use =, in, or not in instead.

A tag filter is applied, but the number of results is exactly the same as without the filter

An operator that does not take effect is used (such as >, ≥, ≠, or empty). Only =, in, and not in take effect.

Filtering by tag always returns 0 results

field is set to an undefined tag name (the API does not report an error). On the knowledge base details page, go to Tags > Manage to check the spelling of the tag name. Also confirm that the tag was actually written to these materials during import.

Note: Defining a tag in the console is not a prerequisite for writing tag values. A field that has not been defined in advance can still be written with a document and used for filtering, and deleting a tag definition does not clear the historical values that have already been written. The actual purposes of a definition are console display, management of the available options for list-type tags, and value type conversion during filtering based on string, int64, float32, bool, and list. It is not an independent index field declaration, and adding or deleting a definition does not trigger a rebuild of historical data. Therefore, the recommended practice is to define tags first to gain type validation and manageability, rather than treating it as a mandatory prerequisite step.

The Publish Version button is dimmed, but the page shows that there are changes pending publication

The upper limit of 3 published versions has been reached (hover over the button to see the tip). Delete an old version in Version Records to publish.

Retrieval returns 404 Knowledge base version ... does not exist

No version has been published, or knowledge_base_version does not match the actual version number. Publish a version in the console first.

The upload succeeds, but the content cannot be found

Uploading and parsing are asynchronous. Wait until the Data Management page in the console shows Processing Completed, and then publish a new version.

The call returns 401 or 403

The AccessKey pair is invalid, or the RAM user has not been granted AliyunMilvusFullAccess.

Occasional connection failures when performing a PUT to OSS during upload

This is caused by network jitter. Simply retry the file.

Historical materials have no tags and cannot be found when filtering

Tags are written during import, so materials imported before the tag definition are not matched by tag filters. You can use Set Tags for an individual material on the Data Management page, or import the material again.

Pre-launch checklist

  • Use a dedicated RAM user with least privileges and rotatable credentials. Do not use the AccessKey pair of your Alibaba Cloud account on a long-term basis.

  • Import only materials that you are authorized to process, and make sure every answer can be traced to evidence in the source chunks.

  • For external deployment, do not continue to use the Flask development server. Switch to a production WSGI service, and add authentication, HTTPS, access log masking, throttling, and auditing.

  • For Q&A about critical policies, we recommend that you keep a manual fallback entry point.