When you call the API operations of Data Transmission Service (DTS), you can specify or query the objects of data migration, data synchronization, or change tracking tasks. This topic describes the related API operations and provides the object definitions and configuration examples.

Related API operations and parameters

API operation Description
You can use the request parameter DbList to specify the objects of data migration, data synchronization, or change tracking tasks.
You can use the response parameter DbObject to query the objects of data migration, data synchronization, or change tracking tasks.

Object definitions

The value of an object-related parameter is a JSON string.

Note You can configure the italic content based on your needs.
  • If the objects contain multiple tables, you can define the objects as shown in the following example:
    {
        "The name of source database 1": {
            "name": "The database name that you want to use in the destination instance",
            "all": true indicates that the entire database is used as the source object
        },
        "The name of source database 2": {
            "name": "The database name that you want to use in the destination instance",
            "all": false indicates that the source object is not the entire database,
            "Table": {
                "The name of source table A": {
                    "name": "The table name that you want to use in the destination instance",
                    "all": true indicates that the entire table is used as the source object,
                    "dml_op": "The DML operations that you want to migrate or synchronize ",
                    "ddl_op": "The DDL operations that you want to migrate or synchronize"
                }
            }
        },
        "The name of source database 3": {
            "name": "The database name that you want to use in the destination instance",
            "all": true indicates that the entire database is used as the source object,
            "dml_op": "The DML operations that you want to migrate or synchronize ",
            "ddl_op": "The DDL operations that you want to migrate or synchronize"
        }
    }
  • If the objects are specific columns or include filter conditions, you can define the objects as shown in the following example:
    {
        "The name of the source database for the data migration, data synchronization, or change tracking task": {
            "name": "The database name that you want to use in the destination instance",
            "all": false indicates that the source object is not the entire database,
            "Table": {
                "The name of source table A": {
                    "name": "The table name that you want to use in the destination instance",
                    "all": false indicates that the source object is not the entire table,
                    "filter": "id>10"
                    "column": {
                        "id": {
                            "key": "PRI",
                            "name": "id",
                            "type": "int(11)",
                            "sharedKey": false,
                            "state": "checked"
                        }
                    },
                    "shard": 12
                }
            },
        }
    }
  • If you migrate or synchronize objects to an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance, you can define the objects as shown in the following example:
    {
        "The name of the source database that you want to migrate or synchronize ": {
            "name": "The database name that you want to use in the destination instance",
            "all": Valid value: false. If the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance, you must set the value to false regardless of whether the entire database or specific tables are used as the source objects. In this case, you must also specify the partition key corresponding to each table.
            "Table": {
                "The name of source table A": {
                    "all": true indicates that the entire table is used as the source object,
                    "name": "The table name that you want to use in the destination instance",
                    "primary_key": "The primary key, for example, id",
                    "type": "The type of the table, for example, dimension",
                }
                "The name of source table B": {
                    "all": true indicates that the entire table is used as the source object,
                    "name": "The table name that you want to use in the destination instance",
                    "part_key": "The partition key, for example, id",
                    "primary_key": "The primary key, for example, id",
                    "type": "The type of the table, for example, partition",
                }
            }
        }
    }
Parameter Description
name The database name, table name, or column name that you want to use in the destination instance. For example, if you want to migrate a database named dtssource to a database named dtstarget, you must set the name parameter to dtstarget.
all Specifies whether to select all tables or columns. Valid values:
  • true: yes
    Note If you select all tables or columns, you do not need to configure the information of specific tables or columns.
  • false: no
Table The information of the source table.
filter The condition used to filter the data to be migrated or synchronized. The filter condition takes effect only at the table level.
For example, you can enter id>10 to migrate or synchronize only the data records whose id column value is greater than 10. For more information about the format of the filter condition, see Use SQL conditions to filter data.
Note You cannot set filter conditions for change tracking tasks.
column The information of the source column.
key Specifies whether to use the column as the primary key. Valid values:
  • PRI: yes
  • Empty string: no
sharedKey Specifies whether to use the column as the shard key. Valid values:
  • true: yes
  • false: no
Note This parameter is required only if the source database is Kafka.
type The data type of the field.
state The checked state indicates that the column is selected.
shard The number of shards in the source table.
Note This parameter is required only if the source database is Kafka.
dml_op The DML operations that you want to migrate or synchronize. Valid values:
  • i: INSERT
  • u: UPDATE
  • d: DELETE
  • If the value is empty, the task migrates or synchronizes all DML operations supported by DTS.
  • none: The task does not migrate or synchronize DML operations.
Note For more information about the DML operations supported by different data migration or synchronization tasks, see the topics listed in Overview of data migration scenarios or Overview of data synchronization scenarios.
ddl_op The DDL operations that you want to migrate or synchronize. Valid values:
  • ct: CREATE TABLE
  • at: ALTER TABLE
  • dt: DROP TABLE
  • rt: RENAME TABLE
  • tt: TRUNCATE TABLE
  • If the value is empty, the task migrates or synchronizes all DDL operations supported by DTS.
  • none: The task does not migrate or synchronize DDL operations.
Note For more information about the DDL operations supported by different data migration or synchronization tasks, see the topics listed in Overview of data migration scenarios or Overview of data synchronization scenarios.
primary_key The primary key. This parameter is available and must be specified only if the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.
part_key The partition key. This parameter is available and must be specified only if the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.
type
Notice The type parameter here is not the same as the type parameter that represents the data type of a field.
If the destination instance is an AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance, you must specify the type of each source table. Valid values:
  • dimension: dimension table
  • partition: partitioned table

Configuration examples

  • Example 1: Replicate data from all tables in the dtstestdata database.
    {"dtstestdata": {   "name": "dtstestdata",   "all": true }}
  • Example 2: Replicate data from all tables in the dtstestdata database and change the database name to dtstestdata_new.
    {"dtstestdata": {   "name": "dtstestdata_new",   "all": true }}
  • Example 3: Replicate data from a table named customer in the dtstestdata database.
    {"dtstestdata": {
       "name": "dtstestdata",
       "all": false,
       "Table": {
         "customer": {
           "name": "customer",
           "all": true, 
           "column": { 
             "id": {
               "key": "PRI",
               "name": "id",
               "type": "int(11)",
               "sharedKey": false,
               "state": "checked"  
             },
             "gmt_create": {
               "key": "",
               "name": "gmt_create",
               "type": "datetime",
               "sharedKey": false,
               "state": "checked"
             },
             "gmt_modify": {
               "key": "",
               "name": "gmt_modify",
               "type": "datetime",
               "sharedKey": false,
               "state": "checked"
             },
             "valid_time": {
               "key": "",
               "name": "valid_time",
               "type": "datetime",
               "sharedKey": false,
               "state": "checked"
             },
             "creator": {
               "key": "",
               "name": "creator",
               "type": "varchar(200)",
               "sharedKey": false,
               "state": "checked"
             }
           },
           "shard": 12
         }
       },
     }
    }
  • Example 4: Replicate data from specific columns of the customer and order tables in the dtstestdata database.
    {"dtstestdata": {
       "name": "dtstestdata",
       "all": false,
       "Table": {
         "customer": {
           "name": "customer",
           "all": false, 
           "column": { 
             "id": {
               "key": "PRI",
               "name": "id",
               "type": "int(11)",
               "sharedKey": false,
               "state": "checked"  
             },
             "level": {
               "key": "",
               "name": "level",
               "type": "varchar(5000)",
               "sharedKey": false,
               "state": "checked"
             },
             "name": {
               "key": "",
               "name": "name",
               "type": "varchar(500)",
               "sharedKey": false,
               "state": "checked"
             },
           },
           "shard": 12
         },
         "order": {
           "name": "order",
           "all": false,
          "column": {
             "id": {
               "key": "PRI",
               "name": "id",
               "type": "int(11)",
               "sharedKey": false,
               "state": "checked"
             }
           },
           "shard": 12
         }
       },
     }
    }
  • Example 5: Replicate data from the customer, order, and commodity tables in the dtstestdata database to the destination AnalyticDB for MySQL or AnalyticDB for PostgreSQL instance.
    {
        "dtstestdata": {
            "name": "dtstestdatanew",
            "all": false,
            "Table": {
                "order": {
                    "name": "ordernew",
                    "all": true,
                    "part_key": "id",
                    "primary_key": "id",
                    "type": "partition"
                },
                "customer": {
                    "name": "customernew",
                    "all": true,
                    "primary_key": "id",
                    "type": "dimension"
                },
                "commodity": {
                    "name": "commoditynew",
                    "all": false,
                    "filter": "id>10",
                    "column": {
                        "id": {
                            "key": "PRI",
                            "name": "id",
                            "type": "int(11)"
                        }
                    },
                    "part_key": "id",
                    "primary_key": "id",
                    "type": "partition"
                }
            }
        }
    }

Supported regular expressions

Symbol Description
Period (.) Matches any single character except '\r\n'.
Asterisk (*) Matches zero or more repetitions of the preceding sub-expression. For example, h.*llo matches strings such as hllo and heeeello.
Question mark (?) Matches zero or one repetition of the preceding sub-expression. For example, h.?llo matches hllo and hello, but not haello.
Character set [characters] Matches any single character included in the brackets. For example, h[ae]llo matches hallo and hello.
Negative character set [^characters] Matches any single character except the characters included in the brackets. For example, h[ae]llo matches hcllo or hdllo, but not hallo or hello.
Character range [character1-character2] Matches any characters within the range from character1 to character2, for example, [0-9] and [a-z].