All Products
Search
Document Center

Vector Retrieval Service for Milvus:Build an enterprise policy Q&A application with an Alibaba Cloud Milvus knowledge base

Last Updated:Oct 27, 2026

An Alibaba Cloud Milvus knowledge base lets you turn the policies, processes, and SOPs scattered across departments into a single internal Q&A entry point. Employees ask questions in natural language and get the process steps together with the original supporting text, and you can use department tags to limit retrieval to a specified scope.

Solution overview

The overall pipeline is exactly the same as in Build an intelligent customer service Q&A application with an Alibaba Cloud Milvus knowledge base: define tags in the console → import documents in batches by tag → publish a version → retrieve with the SDK (optionally filtering by tag) → have the large model generate an answer with source citations → serve the Q&A page with Flask. Reuse the engineering code (kb_client.py, upload.py, app.py, templates/index.html, and start.sh) directly from that topic. This topic only describes the parts that must change for the policy scenario: the department tag system, retrieval parameters, the policy assistant prompt, and policy version management.

For the first iteration, we recommend that you choose only 5 to 20 documents from a single department and expand the scope after verification. The whole process takes about 20 to 30 minutes.

Prerequisites

  • You have created a knowledge base and recorded the knowledge base ID (in a form such as kd-803ae9b10cc31).

  • You have used the Alibaba Cloud account to create a RAM user, selected Access with a permanent AccessKey, and granted the system policy AliyunMilvusFullAccess.

  • You have prepared the endpoint and API key of a large model that supports the OpenAI chat/completions protocol.

  • Python 3.8 or later is installed locally, and the project directory has been created as described in the preceding tutorial.

Step 1: Define department and policy tags

On the knowledge base details page, below Basic Information, click Tags > Manage and add the following three tags (all of type string):

Tag name

Description

Example

department

Department that owns the policy

Finance, Administration, and IT

docType

Document type

Policy, Process

effectiveDate

Effective date

2026-01-01

Tag values support Chinese, so you can directly use the Chinese names of departments and policy types.

Important

The tag management dialog box is saved as a whole. After the dialog box opens, the tag list is loaded asynchronously. Wait until all existing tags are displayed before you add new tags and click Done. Otherwise, the save may overwrite everything with "an empty list plus the new tag", which clears the existing tag definitions. Tag values that have already been written to data are not lost as a result, but we recommend that you verify the number of tags on the details page after adding a tag.

Note

Tag definitions mainly serve to standardize value conventions. Undefined tag names can also be written along with data and used for filtering, but we still recommend that you define tags before importing to facilitate team collaboration and subsequent maintenance.

Step 2: Organize the policy documents and the import manifest

  1. Organize the documents by department and place them in the documents/ directory. We recommend that file names include the policy name and the version or effective date, so that they are easy to identify in the sources section of an answer.

  2. Create documents.jsonl and annotate each document with its department, type, and effective date.

    {"path": "documents/finance-travel.md", "metadata": {"department": "Finance", "docType": "Policy", "effectiveDate": "2026-01-01"}}
    {"path": "documents/finance-reimbursement.md", "metadata": {"department": "Finance", "docType": "Process", "effectiveDate": "2026-02-01"}}
    {"path": "documents/hr-leave.md", "metadata": {"department": "Administration", "docType": "Policy", "effectiveDate": "2026-01-01"}}
    {"path": "documents/it-troubleshoot.md", "metadata": {"department": "IT", "docType": "Process", "effectiveDate": "2026-03-01"}}
    Note

    For policy documents, we recommend that you state the effective date and the policy owner at the beginning of the body text, and mark in the body of older versions that they have been "superseded by version X". By default, the large model can only see the body content of the retrieved chunks; the effectiveDate tag is not automatically included in the prompt and is carried only after you make the modification described in Step 4.

  3. Policy documents often consist of short entries. You can adjust the chunking granularity by clicking Create Policy under Processing Policy on the knowledge base details page. The maximum chunk length is measured in characters (512 by default). We recommend that you set it to 580 to 770 characters (roughly equivalent to 384 to 512 tokens), so that a single process step stays intact as much as possible.

Step 3: Configure retrieval parameters and the prompt

In config.json, adjust retrieval and scenario for the policy scenario.

{
  "retrieval": {
    "page_size": 6,
    "candidate_count": 48,
    "min_score": 0.35,
    "semantic_weight": 0.6,
    "enable_query_expansion": true,
    "rerank_model_name": "",
    "tag_filter": {
      "relation": "and",
      "conditions": []
    }
  },
  "scenario": {
    "title": "Enterprise policy and process Q&A",
    "system_prompt": "You are an internal enterprise policy assistant. Answer only based on the retrieved published policies; organize processes into steps, state the applicable conditions and the required materials, and mark citations as [Source N]; when the materials conflict or are insufficient, clearly advise the user to contact the policy owner.",
    "image_enabled": false,
    "sample_questions": [
      "What materials must I submit for business travel reimbursement?",
      "Who must approve leave longer than three days?",
      "Which incident reporting process should I follow when my computer cannot connect to the network?"
    ]
  }
}

Parameters:

  • min_score: There is no recommended value that applies across corpora. You must calibrate it against your own corpus by using real questions. The recommended approach is to first set min_score=0, retrieve a batch of results, manually label their relevance, and then choose a threshold based on the distribution of recall and false recall. You must recalibrate whenever you change the corpus, switch the rerank model, or adjust semantic_weight. Measured on the corpus used in this topic: at 0.2, policies that are completely unrelated to the question (scoring 0.36 to 0.43) also enter the large model context, which increases both cost and the risk of incorrect answers; after raising the value to 0.35, these results were filtered out.

  • semantic_weight=0.6: Policy questions often mix proper terms with colloquial expressions, so both semantics and keywords are taken into account.

    This parameter is the weighting coefficient of the final score, calculated as score ≈ (1-semantic_weight) × keywordScore + semantic_weight × semanticScore (a rank feature may also be added on top), and min_score performs post-filtering on this final score.

    When reranking is not enabled, semanticScore is the vector similarity; when reranking is enabled, it is the rerank model score. The two have different scales, so after you adjust the weight or turn reranking on or off, you must recalibrate the threshold based on scoreDetails. Lowering the weight does not necessarily lower the total score: it decreases only when semanticScore is higher than keywordScore for the current batch.

  • tag_filter.conditions is left empty by default, which means all departments are retrieved. Configure filter conditions only when you want to restrict a specific entry point to a single department. For more information, see Step 5.

Important

Only three operators actually work: =, in, and not in. Although other operators appear in the "Supported operators" list shown in the 400 Unsupported tag filter operator error message, tests show that such conditions are silently ignored and all unfiltered data is returned. Among them, contains and not contains are merely aliases of the set operations in and not in; they are not string containment, so passing a string fragment returns 0 results. Therefore: when you need to filter by a numeric or date range, use in to enumerate the values; when you need to check whether a tag is empty, use = "". After you configure the filter conditions, be sure to compare the total number of results with the unfiltered results to confirm that the filter actually takes effect. In addition, if field is set to an undefined tag name, the API likewise does not report an error and simply returns 0 results; setting op to aliases such as eq, ==, equal, or like returns 400 Unsupported tag filter operator.

Step 4: Include the department and effective date in answers

Policy Q&A requires determining "which version is in effect and which department it applies to", so the tag values must also be concatenated into the large model context. The tags field in the retrieval results does not return the tag values you wrote: this field comes from the service's internal chunk tag enhancement features, which is a different field from the MetaFields written by AddDocuments, and there is no request parameter that enables returning them. Therefore, an empty value does not mean that the upload failed or that tags were lost. You must maintain the mapping of tag values on the client side: associate the documentId of the retrieval results with the local import manifest or business-side metadata, and then concatenate the values into the large model context.

In app.py, add the mapping after app = Flask(__name__):

META_BY_NAME: dict[str, dict[str, Any]] = {}
_manifest = Path("documents.jsonl")
if _manifest.is_file():
    for _line in _manifest.read_text(encoding="utf-8").splitlines():
        if _line.strip():
            _entry = json.loads(_line)
            META_BY_NAME[Path(str(_entry["path"])).name] = _entry.get("metadata") or {}

Then modify llm_answer() to carry the tag values when assembling the context, and to tell the model the current date in the question:

def llm_answer(question: str, results: list[dict[str, Any]]) -> str:
    if not LLM.get("enabled", True):
        return "The large model is not enabled. See the retrieval results below."
    if not results:
        return "No relevant provision exists in the current policy documents. Contact the corresponding policy owner for confirmation."
    blocks = []
    for index, item in enumerate(results, 1):
        title = field(item, "documentName", "DocumentName")
        meta = META_BY_NAME.get(title) or {}
        label = ", ".join(f"{key}={value}" for key, value in meta.items())
        header = f"[Source {index}] {title}" + (f" ({label})" if label else "")
        blocks.append(f"{header}\n{field(item, 'content', 'Content')}")
    context = "\n\n".join(blocks)
    today = datetime.date.today().isoformat()
    url = str(LLM["base_url"]).rstrip("/") + "/chat/completions"
    response = requests.post(
        url,
        headers={"Authorization": f"Bearer {LLM['api_key']}"},
        json={
            "model": LLM["model"],
            "temperature": 0.1,
            "messages": [
                {"role": "system", "content": SCENARIO.get("system_prompt", "Answer based only on the materials.")},
                {"role": "user", "content": f"Current date: {today}\nQuestion: {question}\n\nRetrieved materials:\n{context}"},
            ],
        },
        timeout=90,
    )
    response.raise_for_status()
    return response.json()["choices"][0]["message"]["content"].strip()

You must add import datetime at the top of the file.

Important

The step of telling the model the current date cannot be omitted. The large model does not know today's date. If you provide only effectiveDate, it assumes a current date on its own and may reach the opposite conclusion—for example, judging a new version that is already in effect as "not yet effective" and instead citing a repealed older standard. Only when both the tag values and the current date are provided can the model correctly select the version currently in force and state that the older version has been superseded.

Step 5: Restrict the retrieval scope by department

When you need to provide a dedicated entry point for a specific department, configure the corresponding filter condition:

"tag_filter": {
  "relation": "and",
  "conditions": [
    {"field": "department", "op": "=", "value": "Finance"}
  ]
}

You can also combine multiple conditions, for example, to query only process documents of the Finance Department:

"conditions": [
  {"field": "department", "op": "=", "value": "Finance"},
  {"field": "docType", "op": "=", "value": "Process"}
]
Important

Once a department filter is configured, that entry point can answer questions only for that department. In this case, you must adjust sample_questions accordingly; otherwise, employees who ask about matters of other departments will only get "no relevant provision". When you switch departments, you must modify three places at the same time: the metadata in documents.jsonl, the tag_filter in config.json, and the sample questions.

Step 6: Upload, publish, and verify

  1. Upload the documents (MetaFields applies to the entire batch, so the script first groups documents by tag and then submits them in batches).

    python upload.py --manifest documents.jsonl
  2. On the Data Management page of the console, confirm that the document status is Processed and that the tag column shows the written tags (for example, docType=Process, department=IT +1).

  3. On the Version Management page, click Publish Version, complete the three-step wizard, and confirm that the new version is in the Published state.

    Important

    By default, at most 3 published versions can exist at the same time. This limit is counted per tenant and does not increase with the instance CU specification. When the limit is reached, the Publish Version button is dimmed, while the page still shows "There are N pending changes to publish". You must delete obsolete versions from the version records before you can continue publishing. The limit can be adjusted on the server side, but there is currently no self-service quota application entry for users, so you must submit a ticket for evaluation. Version deletion is irreversible, and a deleted version becomes immediately unretrievable (the data is then cleaned up asynchronously in the background). Therefore, before deleting, you must confirm that no application is pinned to that version number and switch any application still using it to a new version.

  4. Start the service and verify.

    python app.py
    curl -sS http://127.0.0.1:7860/api/ask \
      -H 'Content-Type: application/json' \
      -d '{"question":"What materials must I submit for business travel reimbursement?"}'

Perform acceptance against the following four checks:

  • The page opens properly, and after a question is submitted, both the answer and the retrieval sources are returned.

  • Each [Source N] in the answer can be matched to the corresponding policy chunk in the sources section below.

  • When you ask about a matter that the policies do not cover, the answer clearly states that no relevant provision exists and advises contacting the policy owner.

  • After a policy is updated and the version is republished, the page can retrieve the content of the new version.

Notes for the policy scenario

  • Choose only one department for the first iteration, and expand the scope after you verify the retrieval quality and the prompt.

  • Handling older policy versions: If you need to retain historical versions for traceability, be sure to mark "superseded by version X" in the body text and distinguish versions by using effectiveDate. If traceability is not required, we recommend that you delete the old versions on the Data Management page and republish, to prevent the model from wavering between multiple versions.

  • You must republish a version after a policy is updated. Only then can an application that uses LATEST_PUBLISHED retrieve the new content.

  • The version quota is only 3: Policy knowledge bases are updated frequently, so we recommend that you keep only "the currently effective version plus the most recent historical version" and clean up earlier versions before publishing a new one.

  • Uploading files with duplicate names fails: When doc_name_dedup=True, if all files in a batch are deduplicated because of duplicate names, the API returns 400 No OSS document can be registered.. When updating policies, we recommend that you include the version number in the file name or delete the old data in the console first.

  • For critical policy Q&A, we recommend that you keep a human fallback entry point and remind employees that the officially published policy documents prevail.

FAQ

Symptom

Cause and solution

A question returns 500, and the log shows 400 Unsupported tag filter operator

op uses an alias such as eq, ==, or like. Use =, in, or not in instead.

A tag filter is added, but the number of results is exactly the same as without the filter

An operator that does not take effect is used (such as >, ≥, ≠, or empty). Only =, in, and not in take effect.

After a department filter is configured, most questions are answered with "no relevant provision"

The entry point has been restricted to a single department. Check whether the scope of the questions matches tag_filter, or leave conditions empty.

Filtering by tag always returns 0 results

The tag name spelling is inconsistent with the one used when writing the data (the API does not report an error and simply returns 0 results). Verify the tag name on the knowledge base details page → Tags → Manage.

The answer mixes the old and new versions of a policy

The retrieved chunks lack version information. Follow Step 4 to bring the tag values into the context, and mark in the body of the older version that it has been superseded.

The upload returns 400 No OSS document can be registered.

All files in the batch were deduplicated because of duplicate names. Change the file names or delete the old data first.

Retrieval returns 404 Knowledge base version ... does not exist

No version has been published, or knowledge_base_version does not match the actual version number.

The Publish Version button is dimmed, but the page shows that there are pending changes to publish

The number of published versions has reached the limit of 3 (hover over the button to see the tip). You can publish after deleting an old version from the version records.

You want to display tags in the answer, but the tags field of the retrieval results is empty

Retrieval results currently do not backfill tag values. Look them up from documents.jsonl as described in Step 4.

After adding a tag, the existing tag definitions disappear

The tag management dialog box is saved as a whole. Reopen the dialog box, wait until the list finishes loading, and then add the missing tag definitions back. Tag values already written to data are not affected.