Modify additional column naming rules to resolve name conflicts in data synchronization - Data Transmission Service - Alibaba Cloud - Data Transmission Service

When synchronizing data to DataHub or MaxCompute, Data Transmission Service (DTS) adds additional columns to the destination table for metadata management, sorting, and deduplication. If an additional column name conflicts with an existing column name, the synchronization task fails. You can modify the naming rule for additional columns to prevent such conflicts.

Background information

Before you modify the naming rule, check whether the additional column names conflict with existing columns in the destination table. The following table describes the additional column names and definitions.

Important When the legacy naming rule is used, DTS also adds the dts_ prefix to the business columns from the source database. When the new naming rule is used, DTS does not add a prefix to these columns.

Table 1. Additional column names and definitions
Destination instance	Legacy name	New name	Definition
DataHub	dts_record_id dts_instance_id dts_db_name dts_table_name dts_operation_flag dts_utc_timestamp dts_before_flag dts_after_flag	new_dts_sync_dts_record_id new_dts_sync_dts_instance_id new_dts_sync_dts_db_name new_dts_sync_dts_table_name new_dts_sync_dts_operation_flag new_dts_sync_dts_utc_timestamp new_dts_sync_dts_before_flag new_dts_sync_dts_after_flag	See Topic schema.
MaxCompute	record_id operation_flag utc_timestamp before_flag after_flag modifytime_year modifytime_month modifytime_day modifytime_hour modifytime_minute	new_dts_sync_record_id new_dts_sync_operation_flag new_dts_sync_utc_timestamp new_dts_sync_before_flag new_dts_sync_after_flag new_dts_sync_modifytime_year new_dts_sync_modifytime_month new_dts_sync_modifytime_day new_dts_sync_modifytime_hour new_dts_sync_modifytime_minute	See Incremental log table schema.

Procedure

Start configuring a data synchronization task. For more information, see Steps 1 to 7 in Configure a data synchronization task (old console).

On the Select Objects to Synchronize page, scroll to the bottom and select an option for Use new naming rule for additional columns.

Option Description

Yes

Uses the new naming rule.

Note If the legacy additional column names conflict with existing columns in the destination table, select Yes to avoid this issue.

DTS adds the new_dts_sync_ prefix to the legacy additional column names to create the new names.

For example, if a legacy additional column is named record_id, its new name is new_dts_sync_record_id.

No Uses the legacy naming rule for additional columns.

Note For more information about the additional columns, see Additional column names and definitions.

Follow the on-screen instructions to complete the configuration of the data synchronization task.

Data Transmission Service:Modify naming rules for additional columns

Background information

Procedure

Related documents