Creates a new-version synchronization task.
Operation description
- This API operation is available for all DataWorks editions.
- You can call this API operation to create a synchronization task. When you call this API operation, you must configure parameters such as SourceDataSourceSettings, DestinationDataSourceSettings, MigrationType, TransformationRules, TableMappings, and JobSettings. The SourceDataSourceSettings parameter defines the settings related to the source. The DestinationDataSourceSettings parameter defines the settings related to the destination. The MigrationType parameter defines the synchronization task type. The TransformationRules parameter defines the transformation rules for objects involved in the synchronization task. The TableMappings parameter defines the mappings between rules used to select synchronization objects in the source and transformation rules applied to the selected synchronization objects. The JobSettings parameter defines the settings for the dimension of the synchronization task, including policies for data type mappings between source fields and destination fields and settings for periodic scheduling.
Debugging
Authorization information
Request parameters
| Parameter | Type | Required | Description | Example |
|---|---|---|---|---|
| DestinationDataSourceType | string | Yes | The destination type. Valid values: Hologres, OSS-HDFS, OSS, MaxCompute, LogHub, StarRocks, DataHub, AnalyticDB for MySQL, Kafka, and Hive. | Hologres |
| Description | string | No | The task description. | The description of the synchronization task. |
| SourceDataSourceType | string | Yes | The source type. Valid values: PolarDB, MySQL, Kafka, LogHub, Hologres, Oracle, OceanBase, MongoDB, Redshift, Hive, SQL Server, Doris, and ClickHouse. | MySQL |
| ProjectId | long | No | The DataWorks workspace ID. You can log on to the DataWorks console and go to the Workspace page to obtain the ID. You must configure this parameter to specify the DataWorks workspace to which the API operation is applied. | 10000 |
| Name | string | No | The name of the synchronization task. | mysql_to_holo_sync_8772 |
| MigrationType | string | Yes | The synchronization type. Valid values:
| FullAndRealtimeIncremental |
| JobType | string | No | The type of the synchronization task. Valid values:
| DatabaseRealtimeMigration |
| SourceDataSourceSettings | array<object> | Yes | The list of source data source settings. | |
| object | Yes | A single source data source setting. | ||
| DataSourceName | string | No | The data source name. | mysql_datasource_1 |
| DataSourceProperties | object | No | The data source properties. | |
| Encoding | string | No | The database encoding. | UTF-8 |
| Timezone | string | No | The time zone. | GMT+8 |
| DestinationDataSourceSettings | array<object> | Yes | The list of destination data source settings. | |
| object | Yes | A single destination data source setting. | ||
| DataSourceName | string | No | The data source name. | holo_datasource_1 |
| ResourceSettings | object | Yes | The resource settings. | |
| OfflineResourceSettings | object | No | The batch synchronization resources. | |
| RequestedCu | double | No | The CU of the Data Integration resource group used for batch synchronization. | 2.0 |
| ResourceGroupIdentifier | string | No | The name of the Data Integration resource group used for batch synchronization. | S_res_group_111_222 |
| RealtimeResourceSettings | object | No | The real-time synchronization resources. | |
| RequestedCu | double | No | The CU of the Data Integration resource group used for real-time synchronization. | 2.0 |
| ResourceGroupIdentifier | string | No | The name of the Data Integration resource group used for real-time synchronization. | S_res_group_111_222 |
| ScheduleResourceSettings | object | No | The scheduling resources. | |
| RequestedCu | double | No | The CU of the scheduling resource group for batch synchronization tasks. | 2.0 |
| ResourceGroupIdentifier | string | No | The name of the scheduling resource group for batch synchronization tasks. | S_res_group_235454102432001_1579085295030 |
| TransformationRules | array<object> | No | The list of synchronization object transformation rule definitions. Note
[ { "RuleName":"my_database_rename_rule", "RuleActionType":"Rename", "RuleTargetType":"Schema", "RuleExpression":"{"expression":"${srcDatasoureName}_${srcDatabaseName}"}" } ]
| |
| object | No | A single synchronization object transformation rule definition. | ||
| RuleActionType | string | No | The action type. Valid values:
| Rename |
| RuleExpression | string | No | The rule expression in JSON string format.
| {"expression":"${srcDatasoureName}_${srcDatabaseName}"} |
| RuleName | string | No | The rule name. When the action type and target type are the same, the rule name must be unique. The name cannot exceed 50 characters. | rename_rule_1 |
| RuleTargetType | string | No | The target type for the action. Valid values:
| Table |
| TableMappings | array<object> | Yes | The list of synchronization object transformation mappings. Each element describes a set of source object selection rules and the transformation rules applied to those objects. Note
[ { "SourceObjectSelectionRules":[ { "ObjectType":"Database", "Action":"Include", "ExpressionType":"Exact", "Expression":"biz_db" }, { "ObjectType":"Schema", "Action":"Include", "ExpressionType":"Exact", "Expression":"s1" }, { "ObjectType":"Table", "Action":"Include", "ExpressionType":"Exact", "Expression":"table1" } ], "TransformationRuleNames":[ { "RuleName":"my_database_rename_rule", "RuleActionType":"Rename", "RuleTargetType":"Schema" } ] } ]
| |
| object | Yes | Each rule represents the selection of a table to synchronize. | ||
| SourceObjectSelectionRules | array<object> | No | Each rule can select a set of source objects to synchronize. Multiple rules together select a table. | |
| object | No | Each rule can select different object types from the source, such as source databases and source tables. | ||
| Action | string | No | The selection action. Valid values: Include and Exclude. | Include |
| Expression | string | No | The expression. | mysql_table_1 |
| ExpressionType | string | No | The expression type. Valid values: Exact and Regex. | Exact |
| ObjectType | string | No | The object type. Valid values:
| Table |
| TransformationRules | array<object> | No | The list of synchronization object transformation rule definitions. Each element represents a single transformation rule definition. | |
| object | No | The transformation rules applied to source objects. | ||
| RuleName | string | No | The rule name. The rule name must be unique for a given combination of action type and target type. The name cannot exceed 50 characters. | rename_rule_1 |
| RuleActionType | string | No | The action type. Valid values:
| Rename |
| RuleTargetType | string | No | The target type for the action. Valid values:
| Table |
| JobSettings | object | No | The task-level settings, including DDL handling policies, column data type mapping between source and destination, and runtime parameters. | |
| ChannelSettings | string | No | The channel-specific settings. You can configure special settings for specific channels. Currently supported: Holo2Holo (Hologres to Hologres) and Holo2Kafka (Hologres to Kafka).
| {"structInfo":"MANAGED","storageType":"TEXTFILE","writeMode":"APPEND","partitionColumns":[{"columnName":"pt","columnType":"STRING","comment":""}],"fieldDelimiter":""} |
| ColumnDataTypeSettings | array<object> | No | The array of column type mappings. Note
["ColumnDataTypeSettings":[ { "SourceDataType":"Bigint", "DestinationDataType":"Text" } ]
| |
| object | No | A single column type mapping. | ||
| DestinationDataType | string | No | The destination type, such as bigint, boolean, string, text, datetime, timestamp, decimal, or binary. Different data sources may have different types. | text |
| SourceDataType | string | No | The source type, such as bigint, boolean, string, text, datetime, timestamp, decimal, or binary. Different data sources may have different types. | bigint |
| CycleScheduleSettings | object | No | The scheduled task settings. | |
| CycleMigrationType | string | No | The synchronization type that requires scheduling. Valid values:
| Full |
| ScheduleParameters | string | No | The scheduling parameters. | bizdate=$bizdate |
| DdlHandlingSettings | array<object> | No | The array of DDL handling settings. Note
["DDLHandlingSettings":[ { "Type":"Insert", "Action":"Normal" } ]
| |
| object | No | A single DDL handling setting. | ||
| Action | string | No | Valid values:
| Critical |
| Type | string | No | The DDL type. Valid values:
| AddColumn |
| RuntimeSettings | array<object> | No | The runtime settings. | |
| object | No | |||
| Name | string | No | The setting name. Valid values:
| runtime.offline.concurrent |
| Value | string | No | The setting value. | 1 |
JobNamedeprecated | string | No | This parameter is deprecated and is replaced by the Name parameter. | mysql_to_holo_sync_8772 |
| Owner | string | No | The task owner. | 3726346 |
Response parameters
Examples
Sample success responses
JSONformat
{
"Id": 11792,
"RequestId": "4F6AB6B3-41FB-5EBB-AFB2-0C98D49DA2BB",
"DIJobId": 11792
}Error codes
For a list of error codes, visit the Service error codes.
Change history
| Change time | Summary of changes | Operation |
|---|---|---|
| 2025-03-27 | The request parameters of the API has changed | View Change Details |
| 2025-01-06 | The request parameters of the API has changed. The response structure of the API has changed | View Change Details |
