DataWorks:Create a knowledge base

Overview

Context Graph (knowledge base) is the unified knowledge management module of DataWorks. It provides a unified knowledge foundation for intelligent applications such as Data Agent and AI Assistant. It organizes information scattered across workspace metadata, data sources, and business documents into browsable, searchable, and authorization-controlled knowledge, enabling intelligent applications to have consistent business context during Q&A, data retrieval, and analysis.

Knowledge bases are managed as independent instances. A single knowledge base can be reused by multiple intelligent applications without redundant construction in each application.

Prerequisites

Complete the following preparations before you use a knowledge base:

1. DataWorks is activated and you have entered the tenant in which you want to create a knowledge base.

2. To build knowledge from workspace semantics, make sure that the current account has read permissions on the workspaces and related data assets to be collected.

3. A serverless resource group and its associated project are prepared for running knowledge build tasks. The selected resource group must have network connectivity and access permissions to the data sources or cloud services required for the build, and must have sufficient available capacity to meet the Compute CU requirement of the task (4 CU by default).

Key concepts

Step 1: Create a knowledge base

Step 2: Configure sources (optional)

Step 3: Build knowledge

After you enter the target knowledge base, click Build Knowledge in the upper-right corner of the details page and select a build method. The system provides three build methods:

• Workspace Semantic Build: Select a workspace to create a knowledge build task from scratch.

• Data Source Semantic Build: Select a data source and specify tables to build knowledge with fine-grained control.

• Document Knowledge Build: Synchronize online documents from DingTalk, Feishu, Lark, WeCom, Yuque, or upload local files to build document-based knowledge.

All three build methods allow you to configure the run policy in Update Method. If you select Manual, the task runs only when manually triggered. If you select Periodic, you can set the execution cycle and time on a monthly, weekly, daily, or hourly basis, and the system runs the task automatically.

All three build methods require the following runtime environment parameters:

The following sections describe the parameters specific to each of the three build methods.

Step 4: Build management

After you start knowledge building, you can view build tasks in build management. On the knowledge base details page, click the Build Management tab to manage task definitions and run instances. Each time a task runs, a run instance is generated.

• Task definitions

  Each task definition contains the task name, task type, build scope, trigger method, resource group for running, and task status. You can search by task name, filter by task type, trigger method, and enabled status, and refresh the task definitions.

  Available operations:

  • Run Now: Manually triggers a build.

  • Edit: Modifies configurations such as the task name and trigger method.

  • Enable / Disable: Controls the automatic triggering of scheduled tasks.

  • Delete: Removes a task definition. A task cannot be deleted while it is running.

• Run instances

  Run instances display the task name, type, trigger method, run status, start time, and operations for each execution. You can filter by run status, task type, and trigger method. Knowledge base upgrade tasks also appear in the run instance list, with the task type displayed as Version Upgrade. Available operations:

  Important

  The running time of the instance depends on the size of the knowledge content to be built. The process may take a long time. Wait patiently.

  • View: Opens the run instance details.

  • Source Files: Views the input files used for this build.

  • Cancel: Cancel instances that are still eligible for termination.

  • Retry: Re-initiate execution for failed or terminated instances that allow rerun.

• Running instance details

  When the status of a running instance is "Failed", perform the following steps:

  1. Identify the issue: In the Running Instances list, click View for the failed task. On the details page, locate the red failed step in the execution graph.

  2. Analyze logs: Open the logs of the failed step and troubleshoot based on the error message by referring to the following table.

  3. Resolve and retry: After you fix the issue, go back to the Task Definition page and click Run Now.

After you fix the issue, click Run Now again in the task definition.

Step 5: Use the knowledge base

After the build task succeeds, the knowledge base becomes available. The following operations are supported:

• Browse the wiki, and view the knowledge graph and reasoning paths.

• Use unified knowledge search.

• Use the knowledge base in Data Agent and AI Assistant.

For more information, see Use a knowledge base.

The semantic models and business documents generated during the build can be viewed, reviewed, and deployed in Knowledge Objects. For more information, see Manage knowledge objects.

Step 6: Configure settings and permissions

After you configure the knowledge base, you can assign multiple administrators for daily management. You can also share the knowledge base with users, AI Assistant instances, or Data Agent instances to provide knowledge reasoning capabilities for the agent context.

Content language

Knowledge base content supports Simplified Chinese and English. Each knowledge base maintains only one content language.

• During the first build, the system determines the content language of the knowledge base based on the language setting in the upper-right corner of the DataWorks console. [To be confirmed: whether the rule that the language is locked at the first build and cannot be changed remains unchanged]

• After the language is determined, all subsequent builds use the same language. Switching the console language only changes the console interface and does not change the language of existing knowledge.

• Directly changing the content language of an existing knowledge base is not supported. To use a different language, create a knowledge base and initiate the first build under the target page language.

• Physical table names, column names, SQL statements, code, IDs, original tags, and reference evidence are not translated. Therefore, an English knowledge base may still contain Chinese characters from the source data.

Upgrade legacy knowledge bases

Legacy knowledge bases must be upgraded before you can use Wiki and Knowledge Graph. The upgrade can be initiated by the knowledge base owner or administrator. Regular members can view the knowledge base and use available features but cannot perform the upgrade.

1. Open the knowledge base list, find the legacy knowledge base marked as Upgradable, and click the mark.

2. Read the upgrade instructions and select the resource group and resource group project to use for the upgrade.

3. Click Confirm Upgrade. After the upgrade task is created, you can view the progress and results in Running Instances under Build Management. The task type is displayed as Version Upgrade.

Local knowledge bases (for existing AI Assistant users)

The knowledge base has been separated from AI Assistant as an independent module. For existing AI Assistant users, knowledge bases previously created in AI Assistant are displayed as local knowledge bases in the interface and remain usable. When the knowledge base module is embedded in AI Assistant, local knowledge bases are displayed in the personal knowledge base section of the knowledge base list. You can click a local knowledge base to go to the knowledge base management page of AI Assistant.

• Supported operations for local knowledge bases: Existing local knowledge base features remain available, including regeneration, export, and clearing.

• Differences from the knowledge base module

  Item	Local knowledge base (legacy)	Knowledge base module
  Entry	The Knowledge Base tab on the AI Assistant instance details page	Knowledge Base in the left-side navigation pane of the console
  Whether creation is supported	Newly created AI Assistant Service instances no longer generate local knowledge bases.	Supported
  Ownership	Belongs to a single AI Assistant Service instance	Independent instance that can be reused by multiple intelligent applications
  Knowledge base type	No type distinction	Supports personal knowledge bases and enterprise knowledge bases
  Sharing method	Used only within the current instance	Supports granting access to users, Data Agent, or AI Assistant Service through Share

• Differences from newly created AI Assistant Service instances: Newly created AI Assistant Service instances no longer generate local knowledge bases, which means you can no longer create knowledge bases within AI Assistant Service. To create a knowledge base, go to the Context Graph (knowledge base) module for unified creation and management, and then use it in AI Assistant Service or Data Agent as needed. To use Wiki and knowledge graphs for existing knowledge bases, see Upgrade existing knowledge bases.

FAQ

• Q: Why are source management or knowledge building buttons not clickable?

  A: Your account may have only read-only or shared usage permissions. Contact the knowledge base owner or administrator to verify your permissions.

• Q: Why can't I see all sections in configuration management?

  A: The system displays only the sections that you have permission to manage. For example, having permission to edit basic information does not necessarily mean you can manage the sharing scope.

• Q: Why are there no available workspaces or data sources on the build page?

  A: The source has not been added to the knowledge base. Add it in source management first, and then return to the page. The available options will be refreshed.

• Q: What do I do if DingTalk or Lark source verification fails?

  A: Check the application credentials, read-only permissions, document member permissions, document scope, and the network access configuration of the resource group. After confirming these settings, click Re-verify.

• Q: What do I do if a knowledge base cannot be upgraded?

  A: If you do not see the Upgradable tag, the knowledge base may have already been upgraded, your account may not be the owner or an administrator, or the knowledge base version information may be temporarily unavailable. Refresh the list and verify your account permissions. If the tag is visible but you cannot submit the upgrade, a build task is usually running for the same knowledge base. Cancel or wait for the task to finish, manually refresh the page, and then try again. A failed upgrade does not affect the existing knowledge base data or normal usage.

• Q: Why does deleting a task fail?

  A: The task may still be running, in the process of being deleted, or depended on by other tasks. Terminate active instances and remove dependencies first, and then retry the deletion.

• Q: Why can't I create a knowledge base in AI Assistant Service?

  A: Knowledge bases have been separated into an independent module for unified management. Newly created AI Assistant Service instances no longer generate local knowledge bases. Create a knowledge base in the Context Graph (knowledge base) module. After creation, you can use it in AI Assistant Service and Data Agent.

• Q: Can existing local knowledge bases still be used?

  A: Yes. Knowledge bases previously created in AI Assistant Service by existing users are displayed as local knowledge bases in the interface. You can still regenerate, export, and clear them without affecting their original usage.

• Q: What is the difference between an administrator and sharing authorization?

  A: Administrators can configure, upload to, and build knowledge bases. Users, Data Agent instances, or AI Assistant Service instances that are granted access through Share can only use the knowledge for search or Q&A and cannot modify the knowledge base.

Recommendations

• Plan the source scope before the first build to avoid scanning irrelevant workspaces.

• Assign a clear owner for the knowledge base and configure backup administrators.

• Enable data profiling for knowledge bases that require high semantic quality, and monitor SQL sampling and token consumption.

• After a successful build, check the running instances, knowledge graph coverage, and knowledge search results together to verify usability.

• Unpublished versions of semantic models do not automatically replace the currently deployed version. Review them before you deploy.

• Disabling a task pauses subsequent triggers, while deleting a task removes the task definition. Do not confuse the two operations.

• Enrich source table and column comments, data standards, owners, lineage, and data quality rules to improve automatic build quality. Manually review core metrics, sensitive columns, and key queries.

• Regularly clean up expired materials and test assets to reduce search noise.

• Follow the principle of least privilege and promptly revoke access when a user no longer needs to access the knowledge base.