This topic describes how to use the DATA_INSPECTION capability of Alibaba Cloud Milvus to add a content safety guardrail to AI Q&A applications: by adding a single data_inspection switch to the parameters of an existing AI Function, you can check the input before the model call and check the output before the result is returned, and converge interception results on the client side into three mutually exclusive dispositions: policy_blocked, manual_review, and operational_error.
Solution overview
Once a large model serves external users, content safety changes from a nice-to-have into a prerequisite for going live. As soon as real users enter prompts on one end and the model outputs text on the other end, two risk exposures open up at the same time:
Input side (user → model): Users may enter illegal or non-compliant content, or use jailbreak phrasing to induce the model to break through its safety boundaries. User-generated content (UGC) such as posts, comments, and nicknames submitted by users must also pass a safety gate before being stored.
Output side (model → user): Even if the input looks normal, the model may still generate non-compliant content, such as marketing copy with exaggerated promises or biased statements. This is especially evident in scenarios such as copywriting and script creation, where the model is free to improvise.
Both ends need a gate, and the business value is straightforward: content safety is a mandatory requirement for filing and launching large model applications; after machines block the vast majority of clearly non-compliant content, human reviewers only need to re-check a small number of ambiguous samples, so the review team shifts from "reviewing everything" to "reviewing hard cases"; a single screenshot of a successful jailbreak or a snippet of inappropriate speech can turn into a public opinion incident, and the guardrail blocks the risk before the content is ever sent out.
Assembling this set of capabilities with traditional solutions runs into several difficulties:
The two ends are checked at different times: The input check must be completed before the model call, and the output check must be completed before the result is returned. A single business call spans two points in time, before and after the model call.
You must integrate an additional content safety service and chain it yourself: The typical approach is for the business code to first call a content safety API to check the input, then call the large model, and then call the check again for the output. That means three remote calls, three sets of timeout and retry logic, and three places to handle authentication.
Interception results are hard to read programmatically: When a safety policy is triggered, the server may return an HTTP error, may return a non-zero business code, or may return a clearly refusing piece of text normally within an HTTP 200 response. Checking only the HTTP status code misses refusals inside 200 responses, while checking only keywords causes false positives on words such as "risk" or "refuse" appearing in normal business text.
Trade-off between false interception and missed interception: If the gate is too tight, normal business is affected; if it is too loose, compliance risk increases. A guardrail cannot have only two states—pass and block—it also needs a middle ground of manual review to absorb ambiguous samples.
Log compliance: Troubleshooting requires logs, but once high-risk original text and personal sensitive information land in business logs in plaintext, the logging system itself becomes a new leakage surface.
The approach of Alibaba Cloud Milvus is: simply add a data_inspection switch to the params of an existing AI Function (such as text generation ai_text_generate). It has only three valid values:
Value | Check timing | Typical scenarios | Description |
| Before the model call | Intelligent customer service and AI assistants receiving user prompts; UGC content before being stored | Blocks jailbreak attempts and illegal or non-compliant input. When triggered, the model is not called, which saves computing power and is also more secure. |
| Before the result is returned | Publishing marketing and campaign copy, script generation | Suitable for scenarios where the input is trusted and the only concern is that the model may generate non-compliant content. |
| Once before and once after | High-risk open-ended conversations, free-form Q&A for the general public | The strongest gate when neither end is trusted, and also the most costly. |
The guardrail and the model call are completed in a single pass inside Milvus. Data never leaves the Milvus instance throughout the process, credentials are configured centrally by the administrator on the Provider side and are never written into any request body, and the business side no longer needs to build or chain an external content safety service.
data_inspection adds a gate to an existing call; it cannot replace the required parameters of the task itself. For example, text generation must still provide texts; if it is missing, the error texts is required for task [ai_text_generate] is reported.
Prerequisites
A Milvus 2.6 instance has been created. AI Function relies on the 2.6 kernel, and no separate model service binding is required after creation.
To access the instance over the Internet, you have enabled Public Access on the Security Configuration tab of the instance details page and added the client egress IP address to the public access whitelist.
pymilvus is installed. The examples in this topic are verified with pymilvus 3.0.0.
The RESTful interface and gRPC share port 19530. You must explicitly specify the port when making calls, for example, http://c-xxx.milvus.aliyuncs.com:19530. If the port is omitted, port 80 is accessed by default, which causes a connection timeout.
Procedure
Prepare the common code
The following code includes the connection configuration, a REST wrapper, a compatibility fallback for the TEXTTRANSFORM type, and a utility function that creates a Collection with a guardrail. The guardrail switch is the single data_inspection line in params.
from __future__ import annotations
import json
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
from pymilvus import DataType, Function, FunctionType, MilvusClient
from pymilvus.exceptions import MilvusException
# ==================== Connection configuration ====================
MILVUS_URI = "http://c-xxx.milvus.aliyuncs.com:19530" # The port must be written as 19530
MILVUS_TOKEN = "root:xxx"
MILVUS_REST_BASE_URL = MILVUS_URI
MODEL_NAME = "qwen3.7-max" # Text model, must already be configured in the Provider
TEXTTRANSFORM_FUNCTION_TYPE = 9
# The contract markers returned by the Provider when a safety policy is triggered;
# they are the only reliable basis for determining "whether it was intercepted"
BLOCK_MARKERS = ("DataInspectionFailed", "inappropriate content")
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)
def texttransform_function_type() -> Any:
"""Get the FunctionType for TEXTTRANSFORM; dynamically add a member if the enum is missing in older versions."""
for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
ft = getattr(FunctionType, type_name, None)
if ft is not None:
return ft
existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
if existing is not None:
return existing
extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
extension._name_ = "TEXTTRANSFORM"
extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
FunctionType._member_map_["TEXTTRANSFORM"] = extension
return extension
def post_json(path: str, body: dict[str, Any], timeout: int = 120) -> tuple[int, dict[str, Any]]:
"""REST interface wrapper: returns (http_status, data); still tries to parse the response body for non-2xx HTTP responses."""
request = Request(
f"{MILVUS_REST_BASE_URL.rstrip('/')}{path}",
data=json.dumps(body, ensure_ascii=False).encode("utf-8"),
headers={"Authorization": f"Bearer {MILVUS_TOKEN}", "Content-Type": "application/json"},
method="POST",
)
try:
with urlopen(request, timeout=timeout) as response:
return response.status, json.loads(response.read().decode("utf-8"))
except HTTPError as exc:
return exc.code, json.loads(exc.read().decode("utf-8"))
def build_guard_collection(name: str, func_name: str, in_field: str, out_field: str,
prompt: str, data_inspection: str) -> None:
"""Create a write-oriented Collection with a TextTransform guardrail."""
if client.has_collection(name):
client.drop_collection(name)
schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field(in_field, DataType.VARCHAR, max_length=1024)
schema.add_field(out_field, DataType.VARCHAR, max_length=4096)
# A Collection must have at least one vector field; this example does not perform vector retrieval,
# so a 2-dimensional placeholder field is used to satisfy the constraint.
# After nullable=True is declared, the field does not need to be passed on insert.
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=2, nullable=True)
schema.add_function(
Function(
name=func_name,
function_type=texttransform_function_type(),
input_field_names=[in_field],
output_field_names=[out_field],
params={
"provider": "aliyun_milvus",
"model_name": MODEL_NAME,
"task": "ai_text_generate",
"prompt": prompt,
"data_inspection": data_inspection, # ← Guardrail switch
"temperature": "0.2",
"enable_thinking": "false",
"timeout_sec": "45",
},
)
)
index_params = client.prepare_index_params()
index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX",
metric_type="COSINE")
client.create_collection(collection_name=name, schema=schema, index_params=index_params)A Collection must contain at least one vector field; otherwise, creating the table reports schema does not contain vector field. This example does not perform vector retrieval, so a 2-dimensional placeholder field is used to satisfy the constraint, and it is declared as nullable=True so that a value does not need to be passed on every write.
Converge interception results into three dispositions
This is the engineering difficulty that is most easily underestimated when implementing guardrails. When a safety policy is triggered, the server may return an HTTP error, a non-zero business code, or a clearly refusing piece of text within an HTTP 200 response. Therefore, the client needs to uniformly converge the observed signals into three mutually exclusive dispositions:
Disposition | Description | Recommended action |
| A safety policy was triggered, the guardrail is working as intended, and this is an expected business outcome. | Record an audit log and return a compliance notice to the user. No need to alert operations. |
| Cannot be determined at the protocol layer, for example, HTTP 200 with a normal structure but suspicious content. | Deliver to the manual review queue. "No error thrown" does not mean "passed the safety check". |
| A real service failure, such as the model being unavailable, a parameter error, or a network failure. | Alert the on-duty engineer and stop retrying. Do not retry indefinitely. |
# ==================== Three-state disposition classification ====================
# Both call paths (gRPC exception / REST response body) first check the Provider contract markers,
# then fall back to the HTTP status and business code, ensuring that the same interception
# yields a consistent disposition conclusion on both paths.
def classify_grpc(exc: MilvusException) -> str:
"""Converge a gRPC exception into one of the three dispositions."""
msg = str(exc)
if any(marker in msg for marker in BLOCK_MARKERS):
return "policy_blocked" # A safety policy was triggered; an expected business outcome
return "operational_error" # Everything else is treated as an operational failure
def classify_rest(status: int, data: dict[str, Any]) -> str:
"""Converge a REST response into one of the three dispositions."""
raw = json.dumps(data, ensure_ascii=False)
if any(marker in raw for marker in BLOCK_MARKERS):
return "policy_blocked"
if status >= 400 or data.get("code", 0) != 0:
return "operational_error"
# HTTP 200 with a normal structure: this may be an "explicit refusal" given by the model in the body,
# which cannot be determined at the protocol layer, so always route to manual review
# instead of treating it directly as having passed the safety check.
return "manual_review"Why you must judge by the Provider contract markers rather than by the HTTP status code alone. In actual tests, "a non-compliant input intercepted by the guardrail" and "a service failure caused by a nonexistent model name" returned exactly the same combination of status codes:
Scenario | HTTP status | Business code | Response body contains DataInspectionFailed | Correct disposition |
Non-compliant input intercepted | 500 | 65535 | ✅ |
|
Nonexistent model name | 500 | 65535 | ❌ |
|
Missing required parameter texts | 400 | 1100 | ❌ |
|
Normal input | 200 | 0 | ❌ | Pass |
The HTTP status codes and business codes in the first two rows are exactly identical, so they alone cannot distinguish "content intercepted" from "service error". The operational meanings of the two are completely different—misjudging a policy interception as an operational failure would generate a false service failure alert for every content safety interception, which over time would drown out real failures. Therefore, the criterion must be the DataInspectionFailed contract marker in the response body. Likewise, do not rely on specific HTTP status code values, as they may change with the gateway version.
Do not use keywords such as "risk", "refuse", or "cannot" to determine whether content was intercepted—these words may also appear in normal business text and can easily cause false positives. At most, keywords can be used for supplementary tracking and should not change the final disposition conclusion.
Steps 1 and 2: the input and output modes pass compliant content
The input mode checks user input before the model call, and the output mode checks the model output before the result is returned. Compliant content passes normally.
# ==================== Step (a): input mode, pre-request check ====================
build_guard_collection(
"guard_input", "inspect_customer_request", "request", "response",
"Answer in one sentence in a customer service tone: ${request}", "input",
)
client.insert("guard_input", [{"request": "How long does a refund take to arrive after the review is approved?"}])
client.flush("guard_input")
for row in client.query("guard_input", filter="",
output_fields=["request", "response"], limit=1):
print(f"Input: {row.get('request')}")
print(f"Output: {row.get('response')}")
# The guardrail was not triggered → passed, and the model returns normally.
# If replaced with a jailbreak prompt, the model would not be called at all; the guardrail blocks it up front.
# ==================== Step (b): output mode, check model output before publishing ====================
build_guard_collection(
"guard_output", "inspect_generated_copy", "draft_request", "publish_copy",
"Generate a membership campaign introduction suitable for publishing in the app: ${draft_request}", "output",
)
client.insert("guard_output",
[{"draft_request": "Generate a benefits introduction of no more than 30 characters for the Member Day campaign. Exaggerated promises are prohibited."}])
client.flush("guard_output")
for row in client.query("guard_output", filter="",
output_fields=["draft_request", "publish_copy"], limit=1):
print(f"Output: {row.get('publish_copy')}")
# The output passed the safety check → it enters the pending-publication queue.Actual pass-through results:
Mode | Input | Output |
| How long does a refund take to arrive after the review is approved? | Dear customer, after the refund is approved, the funds are usually returned to your original payment account within 1 to 7 business days. The exact arrival time depends on the processing speed of the payment channel, so please keep an eye out for it. |
| Generate a benefits introduction of no more than 30 characters for the Member Day campaign. Exaggerated promises are prohibited. | Member Day exclusive: points redemption, member-only discounts, and priority customer service. Details are subject to the page. |
In input mode, the model is not called when the guardrail is triggered, so it both blocks the risk and saves the cost of the model call. The output mode is suitable for scenarios where the input is trusted and the only concern is the model improvising.
Step 3: the both mode intercepts non-compliant input
Use both when neither end is trusted. The following example uses a clearly non-compliant test input to verify that the guardrail actually takes effect—this is a guardrail effectiveness verification that should be performed before going live.
# ==================== Step (c): both mode, double checks on both ends ====================
# Use a clearly non-compliant test input (canary) to verify that the guardrail really blocks it.
UNSAFE_INPUT = "<a clearly non-compliant test input>"
# (c-1) gRPC path: the guardrail is triggered on write
build_guard_collection("guard_both", "inspect_both", "request", "response",
"Please answer: ${request}", "both")
try:
client.insert("guard_both", [{"request": UNSAFE_INPUT}])
client.flush("guard_both")
rows = client.query("guard_both", filter="",
output_fields=["request", "response"], limit=1)
# No error thrown does not mean it passed the safety check: if the structure is normal but the content is suspicious, route to manual review
print("Disposition: manual_review", [r.get("response") for r in rows])
except MilvusException as exc:
print(f"Disposition: {classify_grpc(exc)}")
print(f"Server returned: {exc.message}")
# (c-2) REST path: the same input through the real-time interface
status, data = post_json("/v2/vectordb/ai/text_generate", {
"model_name": MODEL_NAME,
"texts": [UNSAFE_INPUT],
"params": {"data_inspection": "both"},
})
disposition = classify_rest(status, data)
print(f"Observed signals: HTTP={status} provider_code={data.get('code')}")
print(f"Disposition: {disposition}")
# Route by disposition: a policy interception is an expected outcome and does not require an operations alert;
# only operational failures need alerts, and they should not be retried indefinitely
if disposition == "policy_blocked":
pass # Record an audit log and return a compliance notice to the user
elif disposition == "manual_review":
pass # Deliver to the manual review queue
else:
pass # Alert the on-duty engineer and stop retryingIn actual tests, both paths successfully intercepted the input, and the server returned consistent contract markers:
code: DataInspectionFailed
message: Input data may contain inappropriate content.
For details, see: https://www.alibabacloud.com/help/zh/model-studio/error-code#inappropriate-contentThe guardrail was triggered on the input side, the write was blocked, and the model did not generate any content. After being processed by the classification functions above, both the gRPC and REST paths reach the consistent conclusion of policy_blocked.
Actual tests also confirm that the both mode does not falsely intercept normal business input—when a normal customer service question is passed to the same interface, HTTP 200 and a complete answer are returned.
Step 4: logging and disposition recommendations
After the guardrail is in place, logs must be able to help locate problems without becoming a new data leakage surface. Recommendations:
Record only machine-readable, desensitized fields: trace ID, detection stage, HTTP status, provider code, disposition enum, and disposition action.
Always redact or desensitize original text and personal sensitive information; during review, retrieve it through an authorized channel using the trace ID, and do not retain the original text in business logs.
Stop and raise an alert when a policy is triggered or
operational_erroroccurs. Do not retry indefinitely.Can be combined with
AI_PII_MASK: desensitize before logging to minimize the exposure surface of sensitive data.
{
"trace_id": "req-20260808-abc123",
"stage": "both",
"http_status": 500,
"provider_code": 65535,
"disposition": "policy_blocked",
"note": "blocked by data inspection on input side"
}Solution value
Dimension | Before adopting the guardrail | After adopting |
Jailbreak and non-compliant prompts on the input side | Relies on post-hoc manual review, with delayed handling | Blocked before the model call |
Non-compliant content sent out on the output side | Discovered after the fact, handled passively | Intercepted before the return, so it cannot be sent out |
Manual review volume | Full manual review of everything | Only a small number of |
Number of systems | Business system + external content safety service, requiring three chained remote calls | 1 system (Milvus, with the guardrail attached to the call) |
Data and credentials | Original text flows out to external services, and authentication is scattered | Data never leaves the instance, and credentials are configured centrally by the Provider |
After the content safety guardrail is converged into a single switch, you can choose flexibly based on the scenario:
Intelligent customer service Q&A: Add
inputto text generation to block jailbreak attempts and non-compliant questions.Marketing copy and script generation: Add
outputto text generation to intercept non-compliant content before publishing.Open-ended AI assistants for the general public: Use
bothfor double protection on both ends.
Directions for extension:
Combine with
AI_PII_MASK: Desensitize first, then check, then log, to minimize the exposure surface of sensitive data.Cover more tasks: As more AI Functions support
data_inspection, the sameinput/output/bothmental model can be transferred to more generative scenarios.Close the policy loop: Accumulate
manual_reviewsamples into an evaluation set to continuously calibrate the balance between false interception and missed interception, making the guardrail more accurate over time.