The one-click MaxCompute data synchronization node moves data from a MaxCompute source table directly into a Hologres internal table — no SQL writing required. DataWorks generates the import SQL automatically based on your configuration, so you can set up and run a data pipeline in minutes.
To import data with more control over the SQL, see Import data from MaxCompute using SQL. This method typically provides better performance.
Prerequisites
Before you begin, ensure that you have:
-
A MaxCompute project. See Create a MaxCompute project.
-
A Hologres instance. See Create a Hologres instance.
-
Bound the MaxCompute project and the Hologres instance as DataWorks compute resources and passed the connectivity test. See Bind compute resources.
Create a one-click MaxCompute data synchronization node
Configure the synchronization node
Complete the following steps on the node configuration page. Work through each section in order: select a source table, configure the destination table, then set advanced options.
Step 1: Select a MaxCompute source table
| Parameter | Description |
|---|---|
| Project | The name of your MaxCompute project. |
| Schema | The schema of your MaxCompute project. |
| Table name | The name of the MaxCompute source table to synchronize. |
| Filter | A filter condition that limits which rows are synchronized. DataWorks generates this automatically for partitioned tables. Modify it as needed — it maps to the WHERE clause in the generated SQL statement. |
Step 2: Set the Hologres destination table
| Parameter | Description |
|---|---|
| Instance | The destination Hologres instance. In the Select Data Source section, select your bound Hologres data source — DataWorks identifies the instance automatically. After selecting a data source, click Destination Management to open HoloWeb pages, including HoloWeb Console (Instance Monitoring), Slow Queries, Active Connection Management, Database Authorization, and User Management. |
| Database | The database in the destination Hologres instance. |
| Schema | The schema in the destination Hologres instance. |
| Table name | The name of the Hologres internal table. If a table with the same name already exists, DataWorks handles it based on the table type: Non-partitioned table — DataWorks deletes the existing table and data, then re-creates the table. Partitioned table — The existing table and data are preserved. DataWorks creates a new partition subtable based on the partition value and inserts data into it. If the schema of the new table differs from the existing table, the system reports an error. |
| Fields to synchronize | The table fields to synchronize. |
| Partition settings | The partitions of the MaxCompute table to synchronize. Hologres supports only single-level partitioning. If the MaxCompute source table has multiple partition levels, DataWorks converts them into a single level and maps the extra partition keys to regular fields in the Hologres table. |
| Index settings | An index on the Hologres internal table for faster queries. For details on creating indexes, see CREATE TABLE. |
Step 3: Configure advanced options
| Parameter | Description |
|---|---|
| GUC parameters | Grand Unified Configuration (GUC) parameters to set before the import. For supported parameters, see GUC parameters. Other SQL statements are not supported. |
| Foreign server | The foreign server used for the import. Default: odps_server. |
| SQL script | The SQL statement DataWorks generates based on your configuration. This field is read-only. If you update the node configuration, refresh the script to regenerate the SQL. To run the import manually, copy this statement and run it in the Hologres SQL editor. For details, see Import data from MaxCompute using SQL. |
Debug the synchronization node
-
On the right side of the node configuration page, configure the Run Configuration and Resource group.
Parameter Description Compute resource Select your bound Hologres compute resource. Resource group Select the resource group that passed the connectivity test when you bound the Hologres compute resource. CUs for calculation The node uses the default number of CUs. No change needed. Script parameter If you defined variables in the format ${ParameterName}in the Filter field, specify the parameter name and value here. DataWorks replaces variables with their actual values at runtime. For details, see Node scheduling configuration. -
Click Save, then click Run.
What's next
-
Schedule the node: To run the node periodically, configure its scheduling policy in the Scheduling panel on the right. See Node scheduling configuration.
-
Deploy to production: To run the node in the production environment, click the
icon to start deployment. Deployed nodes run on the schedule you configured. -
Query the synchronized data: After the sync completes, use HoloWeb to query the data in the Hologres table. See HoloWeb.
Troubleshooting
Error: get table columns occurs Invalid name:xxx
The project name configured for the source contains spaces or invalid characters. Check the Project field and make sure the name is correct.