DataStudio lets you synchronize MaxCompute table data to Hologres through a guided UI — no SQL required. Once the sync node runs, you can query MaxCompute data in Hologres at accelerated speed.
The steps in this topic use the China (Shanghai) region as an example. The same steps apply to other regions.
Choose a sync method
Two methods are available for loading MaxCompute data into Hologres:
| Method | How | Best for |
|---|---|---|
| One-click sync node (this topic) | Configure a sync node in the DataStudio UI; DataWorks generates the SQL automatically | Fast setup with minimal SQL knowledge |
| SQL import | Write and run INSERT INTO ... SELECT statements directly in Hologres |
Higher throughput; more control |
SQL import generally delivers higher performance than the one-click sync approach. See Import data from MaxCompute using SQL for details.
How it works
The one-click sync node moves data through a three-step chain:
| Component | Role |
|---|---|
| MaxCompute source table | The table whose data you want to sync |
| Hologres external table | A mapping layer with the same schema as the MaxCompute source table; bridges MaxCompute and Hologres |
| Hologres internal table | The destination table in Hologres where the synced data is stored and queryable |
The external table must exist in Hologres before the sync runs. You can either select an existing external table or let DataStudio create one during configuration.
For more information about how to create an external table to synchronize MaxCompute data, see Create a node to synchronize schemas of MaxCompute tables.
Prerequisites
Before you begin, make sure you have:
-
A MaxCompute computing resource associated with DataStudio — see Associate a MaxCompute computing resource
-
A Hologres computing resource associated with DataStudio — see Associate a Hologres computing resource
Create a MaxCompute-to-Hologres sync node
Step 1: Go to DataStudio
Log on to the DataWorks console. In the top navigation bar, select your region. In the left-side navigation pane, choose Data Development and O&M > Data Development. Select your workspace from the drop-down list and click Go to Data Development.
Step 2: Create a workflow
Skip this step if you already have a workflow.
-
Hover over the
icon and select Create Workflow. -
In the Create Workflow dialog box, enter a Workflow Name.
-
Click Create.
Step 3: Add the sync node
-
Hover over the
icon and choose Create Node > Hologres > Data Synchronization from MaxCompute. Alternatively, right-click the workflow name and choose Create Node > Hologres > Data Synchronization from MaxCompute. -
In the Create Node dialog box, configure Name, Engine Instance, Node Type, and Path.
-
Click Confirm. The node configuration tab opens.
Step 4: Configure the node
Source table (MaxCompute)
Specify the Hologres instance and database that hold the external table mapping to your MaxCompute source data.
| Parameter | Description |
|---|---|
| Data source | The Hologres instance where the external table resides |
| Database | The database within that Hologres instance |
Then choose how to provide the external table:
| Option | When to use | What to configure |
|---|---|---|
| Existing external table | An external table already maps to your MaxCompute source table | Select the schema and name of the external table |
| Create external table | No external table exists yet | Specify the server, the MaxCompute project name, and the source table name |
Use the built-in odps_server server when creating an external table. This server is pre-created in the Hologres underlying layer using postgres_fdw.
Destination table (Hologres)
Configure the Hologres internal table where synced data will land.
| Parameter | Description |
|---|---|
| Schema | The schema to which the internal table belongs |
| Table name | The name of the internal table. If a table with this name already exists: for non-partitioned tables, Hologres drops the existing table and recreates it; for partitioned tables, Hologres adds new partitions without dropping the table (an error occurs if the existing table has a different schema) |
| Table description | A description of the internal table |
Synchronization settings
| Setting | What it controls | Notes |
|---|---|---|
| Synchronization field | Which fields from the MaxCompute source table to include | Select the fields in the source MaxCompute table from which you want to synchronize data |
| Partition configurations | Which partitions to sync | Hologres supports only level-1 partitions. If the source table has multiple partition levels, map the level-1 partition field to the destination; all other partition fields become regular columns |
| Index configuration | Indexes on the Hologres internal table | Indexes affect query performance. See CREATE TABLE for supported index types |
Review the generated SQL script
DataWorks parses your configuration and generates the SQL statement that the sync node will run. The script is read-only — if you change any sync settings, click Refresh to regenerate it.
To run the node in SQL mode instead, open the Hologres code editor. See Import data from MaxCompute using SQL.
Step 5: Configure scheduling
In the right-side navigation pane, click Properties and configure the following:
-
Basic properties — see Configure basic properties
-
Time cycle, rerun properties, and dependencies — see Configure time properties and Configure same-cycle scheduling dependencies
-
Resource properties — see Configure the resource property. To access a MySQL data source over the Internet or a VPC, select a scheduling resource group connected to the MySQL data source. See Network connectivity solutions
Configure Rerun Properties and Dependent Upstream Nodes before submitting the node.
Step 6: Save and run the node
-
Click the
icon in the top navigation bar to save the node. -
Click the
icon to run the node.
If your workspace runs in standard mode, commit the node first and then click Deploy in the top navigation bar to push it to the production environment. See Publish tasks.
Step 7: View the scheduled task
-
Click Operation Center in the upper-right corner to switch to the production environment.
-
Find and monitor your scheduled task. See Manage auto triggered tasks.
For an overview of all scheduled tasks, click Operation Center in the top navigation bar. See Overview.
What's next
After the sync completes, go to the tab management page to inspect the data. See Manage tables. You can also log on to the Hologres console and query the synced data using HoloWeb.