All Products
Search
Document Center

DataWorks:System management

Last Updated:Apr 03, 2024

The system management feature is used to manage table creation policies and code generation policies that are required during data model design. After you enable a table creation policy or code generation policy, related operations are strictly managed based on the policy. This ensures that tables are created in a standard manner and the displayed DDL statements of a specific type of compute engine meet your business requirements. This topic describes how to configure a table creation policy and a code generation policy on the System Management page in the DataWorks console.

Background information

  • Table creation policy management: A table creation policy defines whether to allow users who do not have models to create MaxCompute physical tables in DataStudio. After you enable a table creation policy, the policy is triggered when users submit or publish MaxCompute physical tables that are created or modified in DataStudio. If the tables are not published by using Data Modeling, the system may block the submission or publishing operation.

  • Code generation policy management: A code generation policy defines the mappings between the table and field comments in a DDL statement and the display names or descriptions of the table and fields in the visualized modeling UI. After you enable a code generation policy, the table and field comments in the DDL statement that is generated during data modeling in Script mode and during export of configurations of a model in Dimensional Modeling are displayed based on the mappings defined in the code generation policy. The table and field comments in the DDL statement that is generated for a table on the table details page in Data Map are also displayed based on the mappings.

    The following table describes the parameter mappings in script mode and visualized modeling mode.

    Script mode

    Visualized modeling mode

    Comment (table)

    Display Name or Description in the Basic Information section of the configuration tab of the table

    Comment (field)

    Display name or Description of a field in the Field Management section of the configuration tab of a table

  • Publishing policy management: A publishing policy is used when you publish a model.

Limits

  • Table creation policy management:

    • You can use a table creation policy to manage the creation of only MaxCompute physical tables. The table creation policy is triggered only when you submit or publish a created MaxCompute physical table.

    • You cannot use a table creation policy to manage the submission or publishing operation performed on tables that are created by calling API operations.

  • Code generation policy management: You can use a code generation policy to define mappings for table and field comments in DDL statements only when you use a MaxCompute, Hive, MySQL, or Hologres compute engine.

Precautions

Table creation policy management: You can use DataStudio to create or modify a table in a workspace for which a table creation policy is enabled. If the creation or modification operation does not meet the conditions defined in the table creation policy, the submission or publishing operation that is performed on the table may be blocked. You can enable a table creation policy for a model design workspace, or a data development workspace that is associated with a model design workspace.

Solution:

  1. View the table creation policy.

    You can use one of the methods that are described in the following table to view the policy that blocks table creation. You can recreate tables based on the table creation policy.

    Method

    Operation

    Method 1

    Move the pointer over Not Passed to view the cause of the block, and follow the on-screen instructions to view the table creation policy for the current workspace.

    Method 2

    Go to the System Management page of Data Warehouse Planning in the DataWorks console to view the table creation policy for the current workspace. If the current workspace is specified as a data development workspace of another model design workspace, you must access the model design workspace to view the table creation policy.

  2. Create a model that has the same name as the table in Data Modeling and then publish the model as a physical table.

    If you want to continue submitting and publishing a table whose submission and publishing are blocked, you must create a model that has the same name as the table in Data Modeling and then publish the model as a physical table.

    Note
    • For information about how to create a model, see Overview of Dimensional Modeling.

    • If a physical table that has the same name already exists in your compute engine instance, the table creation policy is triggered when you modify the table information. In this case, you must use the reverse modeling feature to perform reverse modeling on the physical table, generate a model based on the physical table, modify the model information, and then publish the model as a physical table.

Go to the System Management page

  1. Go to the Data Modeling page.

    Log on to the DataWorks console. In the left-side navigation pane, choose Data Modeling and Development > Data Modeling. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Modeling.

  2. In the top navigation bar of the Data Modeling page, click Data Warehouse Planning. The Data Layer page appears.

  3. In the left-side navigation pane, click System Management to go to the System Management page.

    On the System Management page, you can configure the following policies:

Table creation policy management

Configure and enable a table creation policy

On the Table Creation Policy Management tab of the System Management page, set Mode to Visualization Mode and configure a policy to manage the creation of tables in DataStudio. The following figure shows the configuration details. 建表策略管理Take note of the following items:

  • Policy

    You can configure a policy that prohibits users from directly creating any physical tables in DataStudio or prohibits users from creating physical tables in a specified format.

    After you enable the policy, the policy is triggered to check physical tables that are not generated by publishing models in Data Modeling. If your table creation operation is blocked by the policy, you must go to Data Modeling, create models based on the physical tables, and then publish the models as physical tables with the required names.

  • Policy details

    Configuration item

    Description

    Rule

    The naming rules for the physical tables that you want to create in DataStudio. You can specify whether the name of a table starts with content in a specific format in each naming rule.

    Note

    You can configure naming rules only for the content with which a table name starts.

    Group

    The rule group. The area marked with 3 in the preceding figure indicates a group of rules. You can add multiple rule groups.

    Logic between rules

    You can use Or or And between rules.

    • Or: Only one of the rules must be met.

    • And: Multiple rules must be met.

  • How the policy takes effect

    After you configure the table creation policy, you must manually enable the policy for the policy to take effect in DataStudio.

Sample scenario

If you create tables whose names start with dim or dws in DataStudio, an error is reported when you submit or publish the tables. To create tables whose names start with dim or dws, you must go to the Dimensional Modeling page in the DataWorks console, create models based on the tables, and then publish the models as physical tables.

The following figure shows the sample configurations of the table creation policy.示例

Code generation policy management

Configure a code generation policy

On the Code Generation Management tab of the System Management page, you can configure a code generation policy and define the mappings between the table and field comments in a DDL statement and the display names or descriptions of the table and fields in the visualized modeling UI. 代码生成策略After you enable the policy, the Comment fields in the DDL statements that are generated when you perform the following operations are displayed based on the configurations on the System Management page.

  • Perform data modeling in Script mode: The Comment fields in the DDL statement that is generated by a compute engine instance for a model are displayed based on the mappings defined in the policy.

  • Export the configurations of a model: When you export the configurations of a model as the DDL statement of a specific compute engine instance, the Comment fields in the DDL statement are displayed based on the mappings defined in the policy.

  • Generate the DDL statement of a table: After you click Generate DDL Statement on the Details tab of the details page of a table in Data Map, the Comment fields in the DDL statement are displayed based on the mappings defined in the policy.

Sample scenario

The following figure shows the basic information of a table and the fields in the table. A code generation policy in which Basic Information-Table Display Name and Field Management-Field Display Name are selected is configured. 表信息Display results in Dimensional Modeling:

  • Perform data modeling in Script mode

    The following figure shows the DDL statement that is displayed after you click Script Mode on the configuration tab of the model and select the DDL statement of a specific type of compute engine.代码代码展示

  • Export the configurations of a model

    The following figure shows the DDL statement that is displayed after you click Export in the top toolbar of the configuration tab of the model to export the configurations of the model.导出模型

Publishing policy management

You can define a publishing policy for a model based on your business requirements on the Publish Policy Management tab. After you enable a publishing policy, you can select a publishing mode that meets your business requirements based on the configurations of the policy when you publish a model.

image

  • Valid Value: the publishing modes that you can select. You can select both publishing modes at the same time.

  • Default Setting: the default publishing mode. The default publishing mode must be a selected option displayed in the Valid Value column.

Description for publishing modes:

  • Incremental Publish: When you publish a model, only changes that are made to the model are published to the related compute engine instance.

  • Delete and Recreate: When you publish a model, the system deletes the existing physical table that is already generated based on the model from the related compute engine instance and then recreates a physical table based on the model.

For more information about how to publish a model, see Publish and materialize a table.