You can use the one-click schema import feature to bulk-create Hologres external tables whose schemas match your MaxCompute tables, and then configure a MaxCompute schema sync node to keep the table schemas synchronized.
Background
DataWorks wraps the IMPORT FOREIGN SCHEMA statement in a visual interface so you can synchronize MaxCompute table schemas to Hologres in one click.
Hologres is a one-stop real-time data warehouse engine developed by Alibaba Cloud. It integrates with MaxCompute, letting you create external tables to accelerate queries on MaxCompute data.
Limitations
This feature accelerates queries on MaxCompute internal tables only, not external tables or views.
Procedure
-
On the editor page for the maxcompute schema sync node, follow these steps.
Node settings
Configure the following parameters on the node editor page:
Parameter
Description
Destination Name
The name of the Hologres instance.
NoteThe Destination Management section provides links to the Holo console (instance monitoring), Slow Query, Active connection management, DB authorization, and User Management pages.
Destination Database
The name of the database in the Hologres instance.
Schema
The name of the schema in the Hologres database. The default value is
public.Type
The source table type. Only MaxCompute tables are supported.
Servers
The server where the source tables are located.
You can use the odps_server external table server, which is created by default in Hologres. For more information about the working principle, see postgres_fdw.
Source Project
The name of the project where the source tables are located.
Select Tables for Query Acceleration
The source tables that you want to use to create external tables.
-
All Tables in Database: Creates external tables for all tables in the selected database.
-
Selected Tables: Creates external tables for only the selected tables in the database. You can search for tables by name.
NoteThe search feature supports fuzzy matching. Entering a keyword returns all tables whose names contain it.
Action for Table Name Conflicts
The action to take when a table to be created has the same name as an existing table in Hologres.
-
Ignore Conflicts and Continue Creating Tables.
-
Update and Change Names of Tables with Same Names.
-
Report Error and Create No Table.
Data type not supported
The action to take when a source table contains data types that Hologres does not support.
-
Report Error and Import Failed: The table creation fails.
-
Ignore and skip unsupported fields: Creates the table but omits the unsupported fields.
Run the sync task
-
In the Run Configuration section, select a Resource Group.
Note-
You can also configure the number of CUs for Scheduling based on the resource requirements of the task. The default value is
0.25. -
To access a data source over the public network or in a VPC, you must use a scheduling Resource Group that can connect to the data source. For more information, see Network connectivity solutions.
-
-
On the toolbar, click Save to save the task, and then click Run to execute it.
-
-
To run the node on a schedule, configure its scheduling configuration as needed. For more information, see Scheduling configuration.
-
After you configure the node, you must deploy it. For more information, see Node and workflow deployment.
-
After the task is deployed, you can monitor its scheduled runs in Operation Center. For more information, see Get started with Operation Center.
Next steps
After the external tables are created, synchronize data to the Hologres tables and use Hologres commands to accelerate queries on MaxCompute data. For more information, see Accelerate queries on MaxCompute data by using a foreign table.