DataWorks:MaxCompute data management

1. To the right of the schema, click > Create Table to go to the Create Table page.

2. You can create a table in one of the following ways.

   • Create a table using Copilot:

     1. In the toolbar at the top of the page, click Use Copilot to Create Table to go to the Copilot Chat interface.

     2. Enter an instruction to create a table. For example, Create a daily aggregate table for product sales.

     3. Click Generate and Replace. The system generates a default table name and field information based on your instruction.

     4. If the table name and fields meet your expectations, click Accept.

        Note

        To modify the table information, you can click Accept and then manually edit the system-generated table information.

   • Create a table manually:

     Create a table based on the parameter descriptions.

     Parameter	Description
     Basic Information	Specify a custom Table Name, Lifecycle, and other information.
     Field Information	Edit the field and field annotation information. Manual editing: Click the Insert button above the field information list. Manually specify the number of rows to insert, and then edit the field and its annotation information. Intelligent editing with Copilot: Click Generate Fields or Generate Field Descriptions above the field information list. The system can generate relevant fields and comments based on the table name and description you set.
     Advanced Settings	The Table Type is set to Internal Table by default. You can configure the Cluster Properties.

3. After the configuration is complete, click Publish in the top toolbar and complete the publishing process to create the table.

1. View a table.

   After you create a table in a MaxCompute project, you can click the icon to the left of the project and then click Table to view basic information about the table. You can also click a specific table name to view details, such as its Fields and DDL.

2. Table of operations.

   1. Right-click the internal table on which you want to perform operations. You can select Generate SQL Query Statement, Import Data, Request Permissions, Metadata Mapping to Hologres, or Data Synchronization to Hologres.

   2. Double-click the internal table on which you want to perform operations to go to its details page. Click the Edit button in the upper-right corner. On the table editing page, you can modify the Lifecycle and Table Description, or add, modify, or delete fields.

      Important

      • Before you modify a field name, delete a field, or modify a field type, run the following command in a MaxCompute SQL node to enable schema evolution: setproject odps.schema.evolution.enable=true.

      • To delete a field, select the corresponding row, right-click, and select Delete.

      • After you update the table information, you must click Publish again. Otherwise, the operation does not take effect.

3. Delete a table.

   Right-click the internal table that you want to delete and select Delete.

   Important

   This operation cannot be undone. Proceed with caution.

1. To the right of the schema, click > Create External Table to go to the Create External Table page.

2. You can create a foreign table in either of the following ways.

   • Create a table using Copilot:

     1. In the toolbar, click Use Copilot to Create Table to go to the Copilot Chat interface.

     2. Enter an instruction to create a table. For example, Create a daily aggregate table for product sales.

     3. Click Generate And Replace. The system generates a default table name and field information based on your instruction.

     4. If the table name and fields meet your expectations, click Accept.

        Note

        To modify the table information, you can click Accept and then manually edit the system-generated table information.

   • Create a table manually:

     You can create a foreign table based on the parameter descriptions.

     Parameter	Description
     Basic Information	Specify a custom Table Name, Lifecycle, and other information.
     Field Information	Edit the field and field annotation information. Manual editing: Click the Insert button above the field information list. Manually specify the number of rows to insert, and then edit the field and its annotation information. Intelligent editing with Copilot: Click Generate Fields or Generate Field Descriptions above the field information list. The system can generate relevant fields and comments based on the table name and description you set.
     Advanced Settings	The Table Type is set to Externa Table by default. You must configure the OSS Storage Location and Storage Format. Storage Location: oss://<OSS_Bucket_Name>/ Note You can obtain the OSS bucket name from Bucket overview. Storage Format: Multiple formats are supported, such as CSV, ORC, and TEXTFILE.

3. After the configuration is complete, click Publish in the toolbar to create the table.

1. View a foreign table.

   After you create a foreign table in a MaxCompute project, you can click the icon to the left of the project and then click Externa Table to view its basic information. You can also click a specific table name to view details such as its Fields and DDL.

2. Operations on external tables.

   Double-click the foreign table that you want to modify to open its details page. Click the Edit button in the upper-right corner. On the table editing page, you can modify the Lifecycle and Table Description, or add, modify, or delete fields.

   Important

   • Before you modify a field name, delete a field, or modify a field type, run the following command in a MaxCompute SQL node to enable schema evolution: setproject odps.schema.evolution.enable=true.

   • To delete a field, select the corresponding row, right-click, and select Delete.

   • After you update the table information, you must click Publish again. Otherwise, the operation does not take effect.

3. Delete a foreign table.

   Right-click the foreign table you want to delete and select Delete.

1. View an object table.

   After you create an OBJECT TABLE in a MaxCompute project, you can click the icon to the left of the project and then click Object Table to view basic information about the object table. You can also click a specific table name to view details, such as its Fields and DDL.

2. Operate on an object table.

   Right-click the object table that you want to manage. You can select Generate SQL Query Statement, Request Permissions, map MaxCompute metadata to a Hologres foreign table, or sync MaxCompute data to a Hologres internal table.

3. Delete an object table.

   Right-click the object table you want to delete and select Delete.

   Important

   This operation cannot be undone. Proceed with caution.

1. To the right of the schema, click > Create View to open the Create View page.

2. Modify the command as described in Create or update a view. Change view_name to the name of the view that you want to create. After the AS keyword, add the query statement for the view's data source.

3. Click Publish at the top of the page to create the view.

1. View the details of a view

   After you create a view in a MaxCompute project, you can click the icon to the left of the project and then click View to display its basic information. You can also click a specific view name to view details such as its Fields and DDL.

2. Operations view

   Double-click the view that you want to edit to open its details page. Click the Edit button in the upper-right corner. On the view editing page, you can modify the view by changing the data source query statement after the AS keyword.

3. Delete a view

   Right-click the view you want to delete and select Delete.

1. To the right of the schema, click > Create Materialized View to open the Create Materialized View page.

2. Modify the command as described in Create a materialized view. Replace view_name with the name of the materialized view that you want to create. After the AS statement, add the query statement for the data source of the materialized view.

3. Click Publish at the top of the page to create the materialized view.

1. View a materialized view.

   After a materialized view is created in a MaxCompute project, you can view its basic information. Click the icon to the left of the project, and then click Materialized View. You can also click a specific materialized view name to view more details, such as its Fields and DDL.

2. Delete a materialized view.

   Right-click the materialized view you want to delete and select Delete.

1. To the right of the schema, click > Create Delta Live Vaterialized View to open the Create Delta Live Vaterialized View page. The key configuration parameters are as follows:

   Parameter	Description
   Data Generation SQL	Fill in the Data Generation SQL based on the command in Dynamic materialized view. Click the Precompile button to check the syntax of the input SQL. Important A non-partitioned table must have a primary key.
   Field Details	The details of the parsed fields after a successful precompilation.
   Primary Key Field Information	After the data generation SQL is precompiled, the system reads the fields that can be used as primary keys and displays them in the primary key field drop-down list. You can select multiple primary key fields.
   Partition Field Information	After the data generation SQL is precompiled, the partition field reads the fields that can be partitioned and displays them in the partition field drop-down list. You can select partition fields to decide whether to partition the dynamic materialized view. You can select multiple partition fields.
   Tuning Parameters	You can configure multiple parameter key-value pairs. These key-value pairs need to be generated into the TBLPROPERTIES table property of the DDL.
   Data Refresh Policy	Non-partitioned Delta Live Materialized View Refresh Configuration: A non-partitioned dynamic materialized view is created if you do not specify a partition field. For dynamic materialized views in MaxCompute, the refresh mode is set to incremental refresh by default. The table property is TBLPROPERTIES("refresh_mode"="incremental"). The supported refresh scheduling methods are MaxCompute Auto Refresh and DataWorks Scheduled Refresh. Partitioned Delta Live Materialized View Refresh Configuration: A partitioned dynamic materialized view is created if you specify a partition field. Partition refresh: Configure settings such as the refresh scheduling method, whether to auto-refresh data, the scheduling epoch, and the refresh duration. Refresh historical partition data: Configure whether to refresh data in partitions that have completed their refresh cycle.

2. Click Publish at the top of the page to create the dynamic materialized view.

1. Publish to a development or production environment: In a standard mode workspace, you can publish an object to a different environment.

2. View a dynamic materialized view.

   After you create a dynamic materialized view in a MaxCompute project, you can click the icon to the left of the project name and then click Delta Live Materialized View to view its basic information. You can also click a specific materialized view name to view details such as its Fields and DDL statement.

3. Delete a dynamic materialized view.

   Right-click the dynamic materialized view that you want to delete and select Delete.

After you create a resource in DataWorks Resource Management, you can click the icon to the left of the MaxCompute project and then click Resource to view basic information about the resource. You can also click a specific resource name to view more information, such as its storage size.

After you create a function in DataWorks Resource Management, you can click the icon to the left of the MaxCompute project and then click Function to view basic information about the function. You can also click a specific function name to view more information, such as its storage size.