DataWorks:Schema synchronization from MaxCompute

Prerequisites

• A MaxCompute computing resource is bound.

• A Hologres computing resource is bound.

Limitations

This feature supports accelerated queries only for data in MaxCompute internal tables. It does not support accelerated queries for data in MaxCompute external tables or views.

Access the schema synchronization page

1. Go to the Data Studio page.

   Log on to the DataWorks console. In the target region, click Data Integration > Data Integration in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Integration.

2. Create a Workflow.

   If you have an existing Workflow, skip this step.

   1. Hover over the icon and select Create Workflow.

   2. In the Create Workflow dialog box, enter a Workflow Name.

   3. Click Create.

3. Create a Schema Synchronization from MaxCompute node.

   1. Hover over the icon and choose Create Node > Hologres > Schema Synchronization from MaxCompute.

      Alternatively, find the target workflow, right-click the workflow, and choose Create Node > Hologres > Schema Synchronization from MaxCompute.

   2. In the Create Node dialog box, enter a Name and select an Compute Engine Instance, Node Type, and Path.

   3. Click OK to go to the node configuration page.

Configure the schema synchronization node

1. Configure the node.

   On the node configuration page, configure the connection information for the destination Hologres instance, the source MaxCompute tables, and policies for handling conflicts during table creation. The node configuration page consists of three parts: ① Destination information: Set destination connection, destination database, and schema. You can click table management to go to the management page. ② Source (Batch create tables based on the following data): Set Type to MaxCompute, configure the server list and source project, and select either All Tables in Database or Selected Tables as the acceleration method. ③ Advanced options: Configure policies for handling table name conflicts and unsupported data types.

   1. Configure destination information.

      Specify the Hologres instance in which to create the Hologres external tables.

      Parameter	Description
      Destination Name	The name of the Hologres instance.
      Destination Database	The name of the database in the Hologres instance.
      Schema	The name of the schema in the Hologres database. The default value is public.

   2. Configure source information.

      Specify the source MaxCompute tables whose schemas you want to replicate.

      Parameter	Description
      Type	Currently, only MaxCompute source tables are supported.
      Servers	The server where the source tables are located. You can directly call the odps_server foreign server that is already created in the underlying layer of Hologres. For more information about the principles, see postgres_fdw.
      Source Project	The name of the project where the source tables are located.
      Select Tables for Query Acceleration	Select the source tables from which to create external tables. All Tables in Database: Creates external tables for all tables in the selected database. Selected Tables: Select specific tables from the database. You can search for tables by name. Note The search finds all tables whose names contain the keyword.

   3. Configure conflict policies.

      Specify the policies for handling conflicts that may occur when you create the external tables.

      Parameter	Description
      Action for Table Name Conflicts	This policy determines what to do if a table with the same name already exists in Hologres. The following options are available: Ignore, continue creating other tables. Update, modify the table with the same name. Report error, do not create duplicate table.
      Data type not supported	This policy determines how to handle data types in the source table that are not supported by Hologres. The following options are available: Report error, import fails: This option causes table creation to fail. Ignore, skip unsupported fields: The table is created without the unsupported fields.

2. Save and run the node.

   1. In the top menu bar of the node configuration page, click the icon to save the node configuration.

   2. In the top menu bar of the node configuration page, click the icon to batch-create the external tables.