Objects of DTS tasks - Data Transmission Service - Alibaba Cloud Documentation Center

You can call the API operations of Data Transmission Service (DTS) to configure or query the objects of a data migration, data synchronization, or change tracking task. This topic describes the related API operations and provides object definitions and configuration examples.

Related API operations and parameters


Operation	Description
ConfigureDtsJob ConfigureSubscription	You can use the request parameter `DbList` to specify the objects of a data migration, data synchronization, or change tracking task.
DescribeDtsJobDetail DescribeDtsJobs	You can query the objects of a data migration, data synchronization, or change tracking task and view the query results in the value of the response parameter `DbObject`.

Object definitions

The value of an object-related parameter is a JSON string. This section describes object-related parameters.

Note You can configure the italic content in the following sample code based on your business requirements.

If the objects consist of multiple databases, refer to the following parameter description to define these objects:

{
    "The name of Source Database 1.": {
        "name": "The database name that you want to use in the destination instance for Source Database 1.",
        "all": A value of true specifies that the entire database is used as the source object.
    },
    "The name of Source Database 2.": {
        "name": "The database name that you want to use in the destination instance for Source Database 2.",
        "all": A value of false specifies that the source object is not the entire database.,
        "Table": {
            "The name of Source Table A.": {
                "name": "The table name that you want to use in the destination instance for Source Table A.",
                "all": A value of true specifies that the entire table is used as the source object.,
                "dml_op": "The DML operations that you want to migrate or synchronize during incremental data migration or synchronization.",
                "ddl_op": "The DDL operations that you want to migrate or synchronize during incremental data migration or synchronization."
            }
        }
    },
    "The name of Source Database 3.": {
        "name": "The database name that you want to use in the destination instance for Source Database 3.",
        "all": A value of true specifies that the entire database is used as the source object.,
        "dml_op": "The DML operations that you want to migrate or synchronize during incremental data migration or synchronization.",
        "ddl_op": "The DDL operations that you want to migrate or synchronize during incremental data migration or synchronization."
    }
}

If the objects are specific columns or include filter conditions, refer to the following parameter description to define these objects:

{
    "The name of the source database for the data migration, data synchronization, or change tracking task.": {
        "name": "The database name that you want to use in the destination instance for the source database.",
        "all": A value of false specifies that the source object is not the entire database.,
        "Table": {
            "The name of Source Table A.": {
                "name": "The table name that you want to use in the destination instance for Source Table A.",
                "all": A value of false specifies that the source object is not the entire table.,
                "filter": "id>10"
                "column": {
                    "id": {
                        "key": "PRI",
                        "name": "id",
                        "type": "int(11)",
                        "sharedKey": false,
                        "state": "checked"
                    }
                },
                "shard": 12
            }
        }
    }
}

If you migrate or synchronize objects to an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance, refer to the following parameter description to define these objects:

{
    "The name of the source database that you want to migrate or synchronize.": {
        "name": "The database name that you want to use in the destination instance for the source database.",
        "all": Set the value to false. If the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance, you must set the value to false regardless of whether the entire database or specific tables are used as the source objects. In this case, you must also specify the partition key that corresponds to each table.
        "Table": {
            "The name of Source Table A.": {
                "all": A value of true specifies that the entire table is used as the source object.,
                "name": "The table name that you want to use in the destination instance for Source Table A.",
                "primary_key": "The primary key. Example: id.",
                "type": "The type of the table. Example: dimension.",
            }
            "The name of Source Table B.": {
                "all": A value of true specifies that the entire table is used as the source object.,
                "name": "The table name that you want to use in the destination instance for Source Table B.",
                "part_key": "The partition key. Example: id.",
                "primary_key": "The primary key. Example: id.",
                "type": "The type of the table. Example: partition.",
                "tagColumnValue": "The value of the tag column."
            }
        }
    }
}


Parameter	Description
`name`	The database name, table name, or column name that you want to use in the destination instance. For example, if you want to migrate data from a database named dtssource to a database named dtstarget, you must set the name parameter to dtstarget.
`all`	Specifies whether to select all tables or columns. Valid values: `true`: yes. Note If you select all tables or columns, you do not need to configure the information about specific tables or columns. `false`: no.
`Table`	The information about the source table.
`filter`	The condition used to filter the source data. The filter condition takes effect only at the table level. For example, you can enter `id>10` to migrate or synchronize only entries whose value in the id column is greater than 10. For more information about the format of the filter condition, see Use SQL conditions to filter data. Note Filter conditions are unavailable for change tracking tasks.
`column`	The information about the source column.
`key`	Specifies whether to use the column as the primary key. Valid values: `PRI`: yes. Empty string: no.
`sharedKey`	Specifies whether to use the column as the shard key. Valid values: `true`: yes. `false`: no. Note This parameter is required only if the source database is a Kafka instance.
`type`	The data type of the column.
`state`	The state of the column. For example, the `checked` state indicates that the column is selected.
`shard`	The number of shards in the source table. Note This parameter is required only if the source database is a Kafka instance.
`dml_op`	The DML operations that you want to migrate or synchronize during incremental data migration or synchronization. Valid values: `i`: INSERT. `u`: UPDATE. `d`: DELETE. If the value is empty, the task migrates or synchronizes all DML operations supported by DTS. `none`: The task does not migrate or synchronize DML operations during incremental data migration or synchronization. Note For more information about the DML operations supported by different data migration or synchronization tasks, see the topics listed in Overview of data migration scenarios or Overview of data synchronization scenarios.
`ddl_op`	The DDL operations that you want to migrate or synchronize during incremental data migration or synchronization. Valid values: `ct`: CREATE TABLE. `at`: ALTER TABLE. `dt`: DROP TABLE. `rt`: RENAME TABLE. `tt`: TRUNCATE TABLE. If the value is empty, the task migrates or synchronizes all DDL operations supported by DTS. `none`: The task does not migrate or synchronize DDL operations during incremental data migration or synchronization. Note For more information about the DDL operations supported by different data migration or synchronization tasks, see the topics listed in Overview of data migration scenarios or Overview of data synchronization scenarios.
`primary_key`	The primary key. This parameter is available and required only if the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.
`part_key`	The partition key. This parameter is available and required only if the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.
`type` Important This `type` parameter is not the same as the `type` parameter that specifies the data type of a column.	The type of the source table. This parameter is required for each source table if the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance. Valid values: dimension: dimension table. partition: partitioned table.
`tagColumnValue`	The value of the __dts_data_source custom tag column. This parameter is available and required only if the destination instance is an AnalyticDB for MySQL instance.

Configuration examples

Example 1: Replicate data from all tables in the dtstestdata database.
```
{"dtstestdata": {   "name": "dtstestdata",   "all": true }}
```
Example 2: Replicate data from all tables in the dtstestdata database and change the database name to dtstestdata_new.
```
{"dtstestdata": {   "name": "dtstestdata_new",   "all": true }}
```

Example 3: Replicate data from a table named customer in the dtstestdata database.

{"dtstestdata": {
   "name": "dtstestdata",
   "all": false,
   "Table": {
     "customer": {
       "name": "customer",
       "all": true, 
       "column": { 
         "id": {
           "key": "PRI",
           "name": "id",
           "type": "int(11)",
           "sharedKey": false,
           "state": "checked"  
         },
         "gmt_create": {
           "key": "",
           "name": "gmt_create",
           "type": "datetime",
           "sharedKey": false,
           "state": "checked"
         },
         "gmt_modify": {
           "key": "",
           "name": "gmt_modify",
           "type": "datetime",
           "sharedKey": false,
           "state": "checked"
         },
         "valid_time": {
           "key": "",
           "name": "valid_time",
           "type": "datetime",
           "sharedKey": false,
           "state": "checked"
         },
         "creator": {
           "key": "",
           "name": "creator",
           "type": "varchar(200)",
           "sharedKey": false,
           "state": "checked"
         }
       },
       "shard": 12
     }
   }
 }
}

Example 4: Replicate data from specific columns of the customer and order tables in the dtstestdata database.

{"dtstestdata": {
   "name": "dtstestdata",
   "all": false,
   "Table": {
     "customer": {
       "name": "customer",
       "all": false, 
       "column": { 
         "id": {
           "key": "PRI",
           "name": "id",
           "type": "int(11)",
           "sharedKey": false,
           "state": "checked"  
         },
         "level": {
           "key": "",
           "name": "level",
           "type": "varchar(5000)",
           "sharedKey": false,
           "state": "checked"
         },
         "name": {
           "key": "",
           "name": "name",
           "type": "varchar(500)",
           "sharedKey": false,
           "state": "checked"
         },
       },
       "shard": 12
     },
     "order": {
       "name": "order",
       "all": false,
      "column": {
         "id": {
           "key": "PRI",
           "name": "id",
           "type": "int(11)",
           "sharedKey": false,
           "state": "checked"
         }
       },
       "shard": 12
     }
   }
 }
}

Example 5: Replicate data from the customer, order, and commodity tables in the dtstestdata database to the destination AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.

{
    "dtstestdata": {
        "name": "dtstestdatanew",
        "all": false,
        "Table": {
            "order": {
                "name": "ordernew",
                "all": true,
                "part_key": "id",
                "primary_key": "id",
                "type": "partition"
            },
            "customer": {
                "name": "customernew",
                "all": true,
                "primary_key": "id",
                "type": "dimension"
            },
            "commodity": {
                "name": "commoditynew",
                "all": false,
                "filter": "id>10",
                "column": {
                    "id": {
                        "key": "PRI",
                        "name": "id",
                        "type": "int(11)"
                    }
                },
                "part_key": "id",
                "primary_key": "id",
                "type": "partition"
            }
        }
    }
}