Manage full and incremental synchronization tasks - DataWorks

Operations overview

The table below lists all available operations, the target databases they support, and whether each works with sharded database configurations.

Operation	Supported targets	Available for sharded databases
Start	All	Yes
Edit	All	Yes
Stop	All	Yes
Rerun	Hologres, MaxCompute	No
Full Data Backfill	MaxCompute	No

Start a task

In the Operation column, click Commit and Run.

Edit a task

As business requirements change, add or remove tables from a synchronization task without recreating it.

In the Operation column, click More > Edit.
Add or remove tables on the configuration page.
Click Commit and Run.

When you commit, the system compares the current table list against the last successful run and applies the diff. New tables are added to the synchronization process automatically.

For more information, see Add or remove source tables to or from a synchronization solution that is running.

Adding tables with one-click real-time synchronization

If your task uses one-click real-time synchronization, new tables require a Full Initialization before they join the real-time pipeline:

The system runs Full Initialization for the new table.
After initialization completes, the Real-time Data Synchronization task stops, resets its start position to when the Full Initialization began, and resumes appending incremental data from that point.

This ensures eventual data consistency for all tables — including data that changed during initialization.

Example: A synchronization task starts at 8:00 and is running at 9:00 when you add a new table. Full Initialization runs from 9:00 to 10:00. The real-time task then resets to 9:00 and re-syncs all incremental changes between 9:00 and 10:00 to the Hologres target.

To reinitialize all tables instead, use Rerun.

Stop a task

In the Operation column, click Stop.

Rerun a task

Use Rerun when target table data is corrupted or unrecoverable and you need to re-migrate the entire database from source to target.

Limitations:

Available only for Hologres and MaxCompute targets.
Not available for tasks that use database and table sharding.

When to use Rerun

Scenario	Details
Binary log purged	The real-time task failed long enough for the binary log to be purged, making incremental data recovery impossible.
Missing columns	New columns from the source table are missing from the target table.
Missing or incorrect data	Target table data is missing or incorrect for any reason.

Run Rerun

Before you start, check for Incremental Merge task instances scheduled on the same Business Date. Concurrent runs on the same Business Date may overwrite partition or table data. Check Incremental Merge instance status on the View auto triggered instances page in the Operation Center. If a conflict exists, choose one of the following:
- Pause the Rerun, wait for the Incremental Merge instance to complete, then retry.
- Freeze the upcoming Incremental Merge instance, then resume it after Rerun succeeds.
In the Operation column, click More > Rerun. The system re-migrates source data to the target tables. If any source columns are missing from the target table, they are added automatically.

After Rerun completes

If the next day's data is not generated or the Incremental Merge task does not resume automatically, manually check and resume the instance:

Real-time task delayed: Resolve the latency first. See Solutions for real-time sync task latency.
Previous cycle instance did not run or failed: Remove its dependency on the previous cycle. See View auto triggered instances.

Backfill data

Use Full Data Backfill when data in a MaxCompute target table is missing or incorrect and you need to resynchronize the full dataset for a specific date.

Limitations:

Available only for full and incremental tasks that synchronize an entire database.
Not available for tasks that use database and table sharding.
One Business Date per operation. To backfill multiple days, run the operation separately for each date.

Run Full Data Backfill

In the Operation column, click Backfill all data.
Select the Business Date for the backfill. For partitioned tables, the full data is synchronized to the date partition that corresponds to the selected Business Date.
Check for Incremental Merge task instances scheduled on the same Business Date before proceeding. Concurrent runs on the same Business Date may overwrite partition or table data. Check Incremental Merge instance status on the View auto triggered instances page in the Operation Center. If a conflict exists, choose one of the following:
- Pause the Full Data Backfill, wait for the Incremental Merge instance to complete, then retry.
- Freeze the upcoming Incremental Merge instance, then resume it after Full Data Backfill completes.
In the selection box on the left, select the target tables, then click the move icon to add them to the right.
Click OK. The process copies all columns that are common between source and target tables, plus any additional columns defined in the synchronization task.

View task execution details

Click a task name on the Sync Task page to view its execution details. The detail page shows basic information and the current run status across four stages:

Stage	Description
Schema Migration	Migrates the source schema to the target.
Full Initialization	Loads the full dataset from source to target.
Real-time Data Synchronization	Continuously syncs incremental changes in real time.
Incremental Merge	Merges incremental changes into the target table (Scheduled Task).

Use the progress bar to see which stage is currently active and the execution details of subtasks within each stage.

The Incremental Merge task is a Scheduled Task. Click its name to navigate to the Operation Center and view the corresponding Periodic Instance.