Cloud Monitor：模擬資料Demo

適用人群

• 新使用者：通過 Demo 1~2 快速理解 Pipeline 基本能力。

• 演算法工程師：通過 Demo 3 評估 Agent 輸出品質。

• 資料平台團隊：通過 Demo 4 瞭解全流程編排模式。

目標資源

與 OT-AI Trace 格式的差異

本模板的模擬資料與其他模板使用的 OT-AI Trace 資料在格式上存在以下差異：

欄位可用性矩陣

22 個欄位在不同 event_type 下的取值情況如下表所示。

樣本資料概覽

Demo 1：使用者提問三級去重

利用模擬資料中帶 dedup_tag 標記的重複資料（exact / fuzzy / semantic），示範三級去重鏈的逐級過濾效果。無 LLM 和 GPU 成本。

定製建議

Demo 2：問題聚類 + 情境標註

對去重後的使用者提問進行語義聚類，按簇採樣後用 LLM 自動標註意圖、複雜度等維度，發現 Agent 使用的情境分布。包含 1 輪 LLM 調用。

SPL 文法

* | project question=question,
          trace_id=trace_id,
          agent=agent_name,
          user_id=user_id,
          event_type=event_type
  | where event_type = 'user_query' AND length(question) > 0
  | dedup-exact -field=question
  | dedup-semantic -field=question -threshold='0.1'
  | semantic-cluster -field=__dedup_emb -n=5
  | sample -n=3 by __cluster_id
  | llm-call -prompt='@anno/scene-label.md' -fields=question -format=json as anno

標註輸出樣本

{
  "意圖類型": "問題診斷",
  "任務複雜度": "中等",
  "業務情境": "安全審計",
  "補充標籤": ["日誌分析", "異常檢測", "IP維度"]
}

定製建議

Demo 3：Agent 回答品質評估

將離散事件記錄按 trace_id 彙總為"一問一答"執行個體，通過 LLM 多維度評估 Agent 回答品質。包含 1 輪 LLM 調用。

SPL 文法

* | project question=question,
          output=output,
          model=model,
          tool_name=tool_name,
          token_input=token_input,
          token_output=token_output,
          latency_ms=latency_ms,
          event_type=event_type,
          trace_id=trace_id,
          session_id=session_id,
          agent_name=agent_name
  | where event_type IN ('user_query','tool_call','tool_result','assistant_content','completion')
  | make-instance
      question=first(question),
      answer=last(output),
      model=any(model),
      agent=any(agent_name),
      tool_chain=join(tool_name, ' → '),
      tool_count=count_if(event_type = 'tool_call'),
      total_tokens=sum(cast(NULLIF(token_input, '') as bigint)),
      latency=max(cast(NULLIF(latency_ms, '') as bigint))
      by session_id,trace_id
  | where question IS NOT NULL AND answer IS NOT NULL
  | dedup-exact -field=question
  | sample -n=20
  | llm-call -prompt='@eval/agent-quality.md' -fields=question,answer,tool_chain -format=json as eval
  | doc-stats -field=answer

評估輸出樣本

{
  "需求理解": {"score": 5, "reason": "準確理解了使用者要求建立基於IP網段的警示規則"},
  "回答品質": {"score": 4, "reason": "警示規則配置完整，但未說明觸發頻率限制"},
  "邏輯連貫": {"score": 5, "reason": "從查詢到驗證到建立，步驟清晰"},
  "格式規範": {"score": 5, "reason": "警示配置以結構化格式呈現"},
  "安全合規": {"score": 5, "reason": "未泄露敏感資訊"}
}

定製建議

Demo 4：端到端全流程

資料治理全流程流水線：欄位提取、事件彙總、指標派生、三級去重、聚類採樣、品質評估 + 情境標註、文檔統計。覆蓋全部運算元，包含 2 輪 LLM 調用。

SPL 文法

* | project question=question,
          output=output,
          model=model,
          tool_name=tool_name,
          tool_args=tool_args,
          tool_success=tool_success,
          token_input=token_input,
          token_output=token_output,
          latency_ms=latency_ms,
          status=status,
          event_type=event_type,
          trace_id=trace_id,
          session_id=session_id,
          conversation_id=conversation_id,
          agent_name=agent_name,
          user_id=user_id,
          region_id=region_id
  | where event_type IN ('user_query','system_prompt','tool_call','tool_result','assistant_content','completion')
  | make-instance
      question=first(question),
      answer=last(output),
      model=any(model),
      agent=any(agent_name),
      user_id=any(user_id),
      region=any(region_id),
      tool_chain=join(tool_name, ' → '),
      tools=array_distinct(tool_name),
      tool_count=count_if(event_type = 'tool_call'),
      has_error=bool_or(status = 'error'),
      total_input_tokens=sum(cast(NULLIF(token_input, '') as bigint)),
      total_output_tokens=sum(cast(NULLIF(token_output, '') as bigint)),
      latency=max(cast(NULLIF(latency_ms, '') as bigint))
      by session_id,trace_id,conversation_id
  | where question IS NOT NULL AND answer IS NOT NULL
  | extend token_total=total_input_tokens + total_output_tokens,
          answer_preview=substr(answer, 1, 500)
  | dedup-exact -field=question
  | dedup-fuzzy -field=question -threshold='3'
  | dedup-semantic -field=question -threshold='0.1'
  | semantic-cluster -field=__dedup_emb -n=5
  | sample -n=3 by __cluster_id
  | llm-call -prompt='@eval/agent-quality.md' -fields=question,answer,tool_chain -format=json as eval
  | llm-call -prompt='@anno/scene-label.md' -fields=question -format=json as anno
  | doc-stats -field=answer

API 配置（JSON）

{
  "name": "mock_data_demo_full",
  "description": "模擬資料端到端全流程 Demo：彙總、去重、聚類、評估、標註，覆蓋全部運算元能力",
  "source": {
    "type": "logstore",
    "logstore": {
      "project": "ali-pub-cn-hangzhou-staging-sls-admin",
      "logstore": "ai_test",
      "query": "*"
    }
  },
  "pipeline": {
    "nodes": [
      {"id": "extract", "type": "project", "parameters": {"question": "question", "output": "output", "model": "model", "tool_name": "tool_name", "tool_args": "tool_args", "tool_success": "tool_success", "token_input": "token_input", "token_output": "token_output", "latency_ms": "latency_ms", "status": "status", "event_type": "event_type", "trace_id": "trace_id", "session_id": "session_id", "conversation_id": "conversation_id", "agent_name": "agent_name", "user_id": "user_id", "region_id": "region_id"}},
      {"id": "filter_events", "type": "where", "parameters": {"filter": "event_type IN ('user_query','system_prompt','tool_call','tool_result','assistant_content','completion')"}},
      {"id": "assemble", "type": "make-instance", "parameters": {"question": "first(question)", "answer": "last(output)", "model": "any(model)", "agent": "any(agent_name)", "user_id": "any(user_id)", "region": "any(region_id)", "tool_chain": "join(tool_name, ' → ')", "tools": "array_distinct(tool_name)", "tool_count": "count_if(event_type = 'tool_call')", "has_error": "bool_or(status = 'error')", "total_input_tokens": "sum(cast(NULLIF(token_input, '') as bigint))", "total_output_tokens": "sum(cast(NULLIF(token_output, '') as bigint))", "latency": "max(cast(NULLIF(latency_ms, '') as bigint))", "by": "session_id,trace_id,conversation_id"}},
      {"id": "filter_valid", "type": "where", "parameters": {"filter": "question IS NOT NULL AND answer IS NOT NULL"}},
      {"id": "derive_metrics", "type": "extend", "parameters": {"token_total": "total_input_tokens + total_output_tokens", "answer_preview": "substr(answer, 1, 500)"}},
      {"id": "exact_dedup", "type": "dedup-exact", "parameters": {"field": "question"}},
      {"id": "fuzzy_dedup", "type": "dedup-fuzzy", "parameters": {"field": "question", "threshold": "3"}},
      {"id": "semantic_dedup", "type": "dedup-semantic", "parameters": {"field": "question", "threshold": "0.1"}},
      {"id": "cluster", "type": "semantic-cluster", "parameters": {"field": "__dedup_emb", "n": 5}},
      {"id": "sample_per_cluster", "type": "sample", "parameters": {"n": 3, "by": "__cluster_id"}},
      {"id": "evaluate", "type": "llm-call", "parameters": {"prompt": "@eval/agent-quality.md", "fields": "question,answer,tool_chain", "format": "json", "as": "eval"}},
      {"id": "annotate", "type": "llm-call", "parameters": {"prompt": "@anno/scene-label.md", "fields": "question", "format": "json", "as": "anno"}},
      {"id": "text_stats", "type": "doc-stats", "parameters": {"field": "answer"}}
    ]
  },
  "sink": {
    "type": "dataset",
    "dataset": {"workspace": "<your-workspace-name>", "dataset": "mock_demo_full"}
  },
  "executePolicy": {
    "mode": "run_once",
    "run_once": {"fromTime": 1772150000, "toTime": 1772240000}
  }
}

全流程資料量變化

常見問題

在使用 Demo 3 和 Demo 4 時，可能遇到以下常見問題：