After the schema feature is enabled for your MaxCompute project, the logic for using features that involve items such as MaxCompute tables, resources, and functions in each DataWorks module is adjusted based on the schema feature. This topic describes the use scenarios and working principles of the schema feature and support of DataWorks modules for the schema feature.
Background information
After the schema feature is enabled for your MaxCompute projects, the original two-layer model structure project_name.table_name for MaxCompute tables is changed to a three-layer model structure project_default.schema_default.table_name that includes schemas. You can use schemas to classify items such as tables, resources, and functions in your MaxCompute projects. For more information, see Schema-related operations.
Precautions
Take note of the following information if you want to enable the schema feature for your existing projects as a tenant user:
DataWorks displays schema-related interactions only when the schema feature is enabled at the tenant level. For the schema feature to be enabled at the tenant level, the
odps.namespace.schemaparameter must be set totruefor all MaxCompute projects within the related tenant.NoteIf only specific projects use custom schemas for storage and the
odps.namespace.schemaparameter is set tofalseat the tenant level, DataWorks cannot display schema-related interactions because DataWorks does not support use of custom schemas.By default, paths in the
project.tableformat in the original code may be parsed into paths in theproject_default.project.tableformat based on the parsing rules of MaxCompute. As a result, an error is reported because the table path cannot be found.The dependencies that are obtained by using the automatic parsing feature in DataStudio remain unchanged.
Enable the MaxCompute schema feature
You can determine whether to enable the MaxCompute schema feature based on your business requirements. The following table describes information about how to enable the feature in different scenarios.
Scenario | How to enable | Description |
You have a small number of nodes and MaxCompute resources |
| The MaxCompute schema feature takes effect at the tenant level. After you enable the feature as a tenant user, the three-layer model structure that includes schemas is applied to your MaxCompute tables in all regions. You cannot disable the feature after it is enabled. |
You have a large number of nodes and MaxCompute resources | We do not recommend that you enable the MaxCompute schema feature. Therefore, set the | N/A. |
Support of the DataWorks modules for the MaxCompute schema feature
If the schema feature is enabled for your MaxCompute project, you must specify a schema for most operations that are performed in the DataWorks console. The following table describes the support of the DataWorks modules for the MaxCompute schema feature.
Module | Support for the MaxCompute schema feature |
Changes to the operations performed in DataStudio:
| |
You need to specify a schema when you perform the following operations in Dimensional Modeling:
| |
You need to specify a schema when you select a source table and a destination table for batch synchronization nodes, real-time synchronization nodes, and data synchronization solutions. You can also create a schema when you configure a data synchronization node. | |
Table names are displayed in the Note You cannot search for tables by schema. | |
Note You cannot search for tables by schema. | |
You can view the schemas of tables and filter tables by schema. The following figure shows the table information on the Permission Application tab. | |
You can view the schemas of tables and filter tables by schema. The following figure shows information about compute engines on the Processing Details page. | |
You can view and filter tables whose names are in the schema.table format on pages on which MaxCompute tables are displayed. For example, table names are specified in the schema.table format in the Write Query SQL section. | |
You can view and filter tables whose names are in the schema.table format on pages on which MaxCompute tables are displayed. For example, table names are displayed in the schema.table format on the SQL Query page and in the SQL editor. | |
You can view the schemas of tables and filter tables by schema on pages on which MaxCompute tables are displayed. For example, the schemas of tables are displayed on the related pages for sensitive data identification and dynamic and static masking of sensitive data. |
Automatic parsing rules of the MaxCompute schema feature
When you use the automatic parsing feature for MaxCompute tables in a three-layer model structure that includes schemas, the system completes the format of the table names in code as odps_project.schema.table. If the schema parameter is set to default, the table schema is hidden. The following table provides the details.
Schema type | odps_project.schema.table syntax | Automatic parsing result |
default |
|
|
Non-default schema (such as |
|
|
For information about the automatic parsing feature, see Scheduling dependency configuration guide.