DataWorks:Data Agent semantic analysis

Overview

When Data Agent performs tasks such as NLQ and SQL generation, it needs to understand the structure and semantics of business data. The semantic analysis feature automatically scans MaxCompute data source tables, extracts inter-table relationships, field definitions, and metric calculation logic, and generates a structured semantic model in YAML format. The model presents analysis results through a visual graph and source code dual-panel view, helping you intuitively understand the relationships among your data assets.

With semantic analysis, you can:

• Automatically organize data asset relationships: The system automatically identifies table structures, field semantics, and inter-table relationships without manual effort.

• Generate visual semantic graphs: Display the relationships between datasets and metrics in a graph format, with dual-panel navigation between the graph and YAML source code.

• Improve AI query accuracy: Load the semantic model into a Data Agent session context (using the /dataworks-semantic command) so that the AI can generate more accurate SQL based on business semantics.

• Support manual corrections and iterations: Analysis results can be edited and saved online. Changes take effect immediately without re-running the task.

From MaxCompute data source to precise data querying, semantic analysis covers five key stages:

Prerequisites

• You have activated Data Agent. If not, see Activation process to complete the activation.

• An available MaxCompute data source is configured in your workspace.

• An available resource group is ready. We recommend a specification of at least 4 CU.

Step 1: Create a semantic analysis task

1. Go to Data Agent Settings. In the left-side navigation pane, click Semantic Analysis.

2. On the Semantic Analysis list page, click Create Task.

3. In the Create Task dialog box, configure the following parameters:

   Parameter	Description
   Name	Required. The format must comply with the requirements described in the usage limits.
   Data Source Type	Required. Currently, only MaxCompute is supported.
   Business Domain & Focus	Required. Use natural language to describe the business domain and table layer you want the analysis to focus on. For example, "E-commerce livestream domain, covering anchor sales and product sales dimensions, DWD to ADS layer." This parameter serves a dual purpose: Guides the analysis direction: Tells the AI which business dimensions to prioritize, guiding it to extract relevant metrics and relationships. Scopes the code scanning range: The AI uses the business domain description to locate relevant code and scheduling tasks through the DataWorks folder structure in the workspace. For example, if you write "e-commerce livestream domain," the AI focuses on analyzing nodes under "e-commerce" and "livestream" folders rather than scanning the entire workspace. More detailed descriptions lead to more accurate table analysis and more focused code scanning.
   Workspace	Required. Select a DataWorks workspace from the drop-down list. This workspace is not just an execution environment but also the AI's business knowledge source. The AI reads SQL scripts, scheduling tasks, and code folder structures from this workspace to interpret actual metric calculation logic. For example, it extracts expressions such as SUM(CASE WHEN order_status='paid' THEN pay_amount END) from SQL code to understand how metrics are actually calculated. Important Make sure to select the workspace that contains your data processing code. If you select the wrong workspace, the AI cannot read the actual processing code and can only infer metric meanings from field comments, leading to inconsistencies between the model and actual calculations.
   Resource Group	Required. Select the resource group used to run the task.
   Pinned Tables	Required. In the cascading selector, expand a MaxCompute project in the left panel and select specific tables in the right panel. You can select tables across projects, up to 30 tables in total. The model will focus on analyzing the structures and relationships of these tables.
   Reference Files	Optional. You can upload files or enter file URLs to provide external reference materials.

4. After completing the configuration, click the ok button at the bottom of the dialog box. After the system displays the "Task created" message, the list refreshes automatically.

Step 2: Run a semantic analysis task

After a task is created, you can run it as follows:

1. On the Semantic Analysis list page, find the target task and click Run in the Actions column.

2. The system displays a "Run submitted" message and automatically opens the task details dialog box.

After the task is submitted, the system runs the semantic analysis in the background. The running time depends on the number and volume of tables being analyzed, and typically takes several minutes. The AI engine performs analysis from two dimensions simultaneously:

• Dimension 1: Reading SQL scripts and scheduling tasks from the workspace. Based on the business domain description, the AI locates relevant code through the DataWorks folder structure and interprets actual metric calculation logic and inter-table processing relationships from the SQL.

• Dimension 2: Scanning metadata of the pinned tables. The AI extracts the business semantics of each field, synonyms, inter-table layer relationships, business metric definitions, derived metric formulas, and example Q&A pairs.

The analysis results from both dimensions are cross-validated and merged into a single comprehensive semantic model.

Step 3: View task details and run status

Click a task name in the task list to open the task details dialog box. The task details contain the following tabs:

Step 4: View and edit semantic model

Go to the Run History tab, and click View Results for a run record with the "Success" status to open the semantic model result viewer.

The result viewer provides a dual-panel view of the graph and source code:

• Semantic model graph: Displays the relationships between datasets in a visual format. Nodes in the graph represent datasets or metrics, and edges represent the relationships between them. Click a node or edge in the graph, and the source code editor on the right automatically scrolls to the corresponding line and highlights it.

• YAML source code editor: Displays the YAML source code of the semantic model. The top-level structure includes dataset definitions, field descriptions, ai_context, and other information. As you browse the source code, the graph on the left automatically highlights and centers on the corresponding node or edge.

The result viewer also supports the following features:

• Fullscreen view: Click the Fullscreen button to view the graph or source code in fullscreen mode.

• Edit and save: Click Edit to enter edit mode. After modifying the YAML content, click Save to write the changes back to the backend. Changes take effect immediately after saving, without re-running the task. To discard changes, click Reset to restore the last saved version. You can also use Compare Changes to view the differences.

Semantic model YAML structure

The semantic model generated by the AI engine uses YAML format with four core modules at the top level:

The following is a simplified YAML source code example:

semantic_model:
  - ai_context:
      instructions: |
        E-commerce livestream data analysis domain. Covers anchor sales
        and product sales dimensions.
        Data layers: ODS → DWD → DWS → ADS
        Partition field: dt. Currency: CNY.
      few_shots:
        - question: Who are the top 10 anchors by GMV yesterday?
          sql: |
            SELECT anchor_name, gmv
            FROM ads_ctlive_anchor_stats
            WHERE stat_period = '1d'
            ORDER BY gmv DESC LIMIT 10;

    metrics:
      - name: GMV
        description: Total transaction amount (CNY)
        ai_context:
          synonyms: [sales amount, transaction volume, revenue]
        expression:
          dialects:
            - dialect: MaxCompute
              expression: SUM(gmv)

    metric_formulas:
      - name: Average Order Value
        description: Average transaction amount per order (CNY)
        formula: GMV / Order Count

    datasets:
      - source: ads_ctlive_anchor_stats
        description: ADS - Anchor transaction statistics
        fields:
          - name: anchor_name
            type: string
            description: Anchor nickname
            synonyms: [anchor, streamer, host]
          - name: gmv
            type: double
            description: Total transaction amount (CNY)
            metric: true
            synonyms: [sales amount, GMV, revenue]

Step 5: Load and use semantic model in Data Agent

After generating a YAML semantic model through the preceding steps, the model resides on the server side. Data Agent chat sessions do not automatically access it. You must explicitly load the semantic model in an Agent session so that the AI can reference the business knowledge it contains when answering questions.

Procedure:

1. Open Data Agent and enter the chat window.

2. Type /dataworks-semantic in the chat input box and send.

3. The Agent automatically executes: environment check → list available tasks → download YAML output → inject into the current session's AI context.

4. After confirming a successful load, you can start querying and generating SQL based on the semantic model.

To manually download result files, go to the Run History tab in the console and click Download Results for a successful run record to obtain the YAML output files.

The following scenarios see significant quality improvements after loading a semantic model:

Scenario example: E-commerce livestream end-to-end walkthrough

The following walks through the complete workflow from task creation to precise querying using an e-commerce livestream scenario.

1. Create a task (Step 1)

On the Semantic Analysis page, click Create Task and fill in the following configuration:

2. Run the task (Step 2)

Click Run. The AI semantic engine automatically performs the following analysis: locates workspace folders matching "e-commerce livestream domain" and reads SQL scripts → extracts real metric calculation logic from code (e.g., discovers that gmv = SUM(CASE WHEN order_status='paid' THEN pay_amount END)) → scans metadata and field relationships across the 4 tables → identifies ADS/DWS layer hierarchies → generates a structured YAML model containing metric definitions, derived formulas, and example SQL.

3. View and edit the model (Steps 3 and 4)

After the task completes, open the Semantic Model tab on the task details page. Use the graph view to verify table relationships, and fine-tune metric definitions or add business context in the YAML editor. Save your changes when satisfied.

4. Load into Data Agent (Step 5)

In a Data Agent session, run /dataworks-semantic download <job-name> followed by /dataworks-semantic index <job-name> to download the model locally and build the index. All subsequent queries in that session will leverage the semantic model for precise answers.

5. Verify the results

After loading, compare Agent response quality for the same questions before and after:

User question: "Who were the top 10 anchors by GMV yesterday?"

-- AI guesses table and field names
SELECT host_name, total_amount
FROM live_stream_sales
WHERE date = '2026-07-27'
ORDER BY total_amount DESC
LIMIT 10;

-- Generated based on semantic model
SELECT anchor_name, gmv
FROM ads_ctlive_anchor_stats
WHERE stat_period = '1d'
ORDER BY gmv DESC
LIMIT 10;

User question: "What is the average order value by category over the last 30 days?"

-- AI misunderstands "average order value"
SELECT category, AVG(price) AS avg_price
FROM products
GROUP BY category;

-- Model knows: avg order value = GMV / order count
SELECT cate_level1_name,
  SUM(gmv) / SUM(order_cnt) AS avg_order_value
FROM ads_ctlive_item_stats
WHERE stat_period = '30d'
GROUP BY cate_level1_name
ORDER BY avg_order_value DESC;

After loading a semantic model, the AI automatically references ai_context (business instructions), metrics (definitions and synonyms), metric_formulas (derived formulas), and datasets (table and field mappings), transforming from "guessing" to "knowledge-based precise generation."

FAQ

Q: What are the possible causes of task failure?

A: Task failures are typically caused by the following reasons:

• Insufficient resource group specification: Tasks may fail due to insufficient resources when the resource group specification is less than 4 CU. We recommend that you select a resource group with a specification of at least 4 CU.

• Insufficient MaxCompute project permissions: The workspace that runs the semantic analysis task must have read permissions on the target MaxCompute project. Verify the binding relationship and permission configuration between the workspace and the MaxCompute project.

• Excessive data volume: Too many tables or too much data in a single analysis run may cause a timeout. We recommend that you reduce the number of pinned tables and try again.

Q: Do I need to re-run the task after editing the YAML?

A: No. After editing and saving the YAML in the result viewer, changes take effect immediately. The next time you download via /dataworks-semantic, the latest version is automatically retrieved.

Q: I get a token expiry error when using the semantic model in Data Agent.

A: Run /dataworks-semantic check to check the environment. If authentication-related errors appear, refresh your authentication credentials and try again.

Q: I see "local edits detected" when downloading. What should I do?

A: This means the YAML file was modified after the last download (hash mismatch). To overwrite local changes, add the --force flag to force download. We recommend backing up your local edits first.

Related documentation

• Data Agent billing

• Data Agent