DTS-related questions - Data Transmission Service - Alibaba Cloud Documentation Center

Q: What happens if I change the password of the source or destination database while a DTS instance is running?
A: The DTS instance reports an error and stops. You can click the instance ID to go to the instance details page and change the source or destination account password in the Basic Information tab. Then, go to the Task Management tab, locate the failed module in the instance progress section, and restart the module in Basic Information.
Q: Why do some source or destination databases not have a public IP access method?
A: It depends on the access method, task type, and database type of the source or destination database. For example, for a MySQL source, data migration and change tracking tasks support public IP access, but data synchronization tasks do not.
Q: Can the source and destination databases be the same database instance?
A: Yes. If your source and destination databases are the same database instance, we recommend that you use the object name mapping feature to isolate and distinguish the data. Otherwise, the DTS instance may fail or data may be lost. For more information, see Object name mapping.
Q: Why does a task with Redis as the destination report the error "OOM command not allowed when used memory > 'maxmemory'"?
A: The destination Redis instance may have insufficient storage space. If the destination Redis instance is of the cluster edition, one of the shards may have reached its memory limit. You must upgrade the specifications of the destination instance.
Q: What is the AliyunDTSRolePolicy permission policy and what is it used for?
A: The AliyunDTSRolePolicy policy is used to access cloud resources such as ApsaraDB RDS and ECS in the current Alibaba Cloud account or across accounts. It allows DTS to call relevant cloud resource information when you configure data migration, data synchronization, or change tracking tasks. For more information, see Grant DTS permissions to access cloud resources.
Q: How do I authorize a RAM role?
A: When you log on to the console for the first time, DTS prompts you to authorize the AliyunDTSDefaultRole. Follow the on-screen instructions to go to the RAM authorization page and complete the authorization. For more information, see Grant DTS permissions to access cloud resources.
Important
You must log on to the console with your Alibaba Cloud account (root account) to perform this operation.
Q: Can I modify the database account password entered for a DTS task?
A: Yes, you can modify the database account password entered for a DTS task. Click the instance ID to go to the instance details, and in the Basic Information tab, click Change Password to modify the source or destination account password.
Important
The system account password of the DTS task cannot be modified.
Q: Why do MaxCompute tables have the _base suffix?
1. Initial schema synchronization.
  
  DTS synchronizes the schema definitions of the tables to be synchronized from the source database to MaxCompute. During initialization, DTS adds the `_base` suffix to the table names. For example, if the source table is `customer`, the table in MaxCompute becomes `customer_base`.
2. Initial full data synchronization.
  
  DTS synchronizes all historical data from the tables in the source database to the `_base` tables in MaxCompute. For example, data is synchronized from the `customer` table in the source database to the `customer_base` table in MaxCompute. This data serves as the baseline for subsequent incremental data synchronization.
  
  Note
  This table is also known as the full baseline table.
3. Incremental data synchronization.
  
  DTS creates an incremental log table in MaxCompute. The table name is the name of the destination table with the `_log` suffix, such as `customer_log`. DTS then synchronizes incremental data from the source database to this table in real time.
  
  Note
  For more information about the structure of the incremental log table, see Incremental log table schema.
Q: The Kafka topic cannot be retrieved. How do I resolve this?
The Kafka broker that you configured may not contain topic information. Use the following command to check the broker distribution of the topic:
```
./bin/kafka-topics.sh --describe --zookeeper zk01:2181/kafka --topic topic_name
```
Q: Can I set up a local MySQL instance as a replica of an ApsaraDB RDS for MySQL instance?
A: Yes. You can use the data migration feature of Data Transmission Service (DTS) to configure real-time synchronization from an ApsaraDB RDS instance to a local self-managed MySQL instance to implement a primary-replica architecture.
Q: How do I copy data from an ApsaraDB RDS instance to a newly created ApsaraDB RDS instance?
A: You can use the data migration feature of DTS. Set the migration types to schema migration, full data migration, and incremental data migration. For configuration details, see Migrate data between ApsaraDB RDS instances.
Q: Does DTS support replicating a database within an ApsaraDB RDS instance that is identical except for the database name?
A: Yes. The object name mapping feature of DTS allows you to replicate a database within an ApsaraDB RDS instance that is identical except for the database name.
Q: A DTS instance keeps showing latency. How do I resolve this?
The possible causes are as follows:
- Multiple DTS tasks were created on the source database instance by using different accounts, which increased the load on the instance. Use the same account to create the tasks.
- The destination database instance has insufficient memory. Arrange your business appropriately and then restart the destination database instance. If the issue persists, upgrade the specifications of the destination instance or perform a primary-secondary switchover.
  Note
  A primary-secondary switchover may cause a brief network disconnection. Make sure that your application has an automatic reconnection mechanism.
Q: After I synchronize or migrate data to the destination database in the previous console, all columns are in lowercase. How do I resolve this?
A: Use the new console to configure the task and enable the destination object name case policy. For more information, see Capitalization policy for object names in the destination database.
Q: Can a DTS task be resumed after it is suspended?
A: In general, a DTS task that has been suspended for no more than 24 hours can be resumed. If the data volume is small, a task suspended for no more than 7 days can be resumed. We recommend that the suspension does not exceed 6 hours.
Q: Why does the progress start from 0 after a task is suspended and restarted?
A: After the task is restarted, DTS re-queries the completed data and then continues to process the remaining data. During this process, the task progress may differ from the actual progress due to latency.
Q: What is the principle of lock-free DDL changes?
A: For the main principle of lock-free DDL changes, see Principle of lock-free schema change.
Q: Does DTS support suspending data synchronization or data migration of a specific table?
A: No.
Q: Do I need to purchase a new instance if the task fails?
A: No. You can reconfigure the original task.
Q: What happens if multiple tasks write data to the same destination?
A: Data inconsistency may occur.
Q: Why is the instance still in the locked state after renewal?
A: After you renew a locked DTS instance, it takes some time for the instance to be unlocked. Please wait patiently.
Q: Can I modify the resource group of a DTS instance?
A: Yes. Go to the Basic Information page of the instance, and in the Basic Information section, click Modification next to Resource Group Name to change the resource group.
Q: Does DTS provide a binlog analysis tool?
A: No. DTS does not provide a binlog analysis tool.
Q: An incremental task keeps showing 95%. Is this normal?
A: Yes. Incremental tasks run continuously and never complete. The progress does not reach 100%.
Q: Why is a DTS task not released after 7 days?
A: Occasionally, frozen tasks are retained for more than 7 days.
Q: Can I modify the port of a task that has already been created?
A: No.
Q: In a DTS task, can I downgrade the ApsaraDB RDS for MySQL instance attached to PolarDB-X 1.0?
A: We do not recommend downgrading. Downgrading triggers a primary-secondary switchover, which may cause data loss.
Q: Can I upgrade or downgrade the source or destination instance while a DTS task is running?
A: Upgrading or downgrading the source or destination instance while a DTS task is running may cause task latency or data loss. We do not recommend that you change the configuration of the source or destination instance.
Q: What is the impact of a DTS task on the source and destination instances?
A: Full data initialization consumes some read/write resources of the source and destination databases, which may increase database load. We recommend that you run full data tasks during off-peak hours.
Q: What is the typical latency of a DTS task?
A: The latency of a DTS task cannot be estimated because it is affected by multiple factors, including the load on the source instance, the bandwidth of the transmission network, network latency, and the write performance of the destination instance.
Q: If the Data Transmission console automatically redirects to the Data Management Service (DMS) console, how do I return to the previous Data Transmission console?
A: In the lower-right corner of the DMS console, click and then click to return to the previous Data Transmission console.
Q: Does DTS support data encryption?
A: DTS supports secure access to the source or destination database over SSL-encrypted connections to read data from the source or write data to the destination. However, DTS does not support data transmission encryption during data transfer.
Q: Does DTS support ClickHouse as a source or destination?
A: No.
Q: Does DTS support 2.0 as a source or destination?
A: AnalyticDB for MySQL 2.0 is supported only as a destination. The solution that uses AnalyticDB for MySQL 2.0 as the destination is not yet available in the new console and can only be configured in the previous console.
Q: Why can't I see a newly created task in the console?
You may have selected the wrong task list, or you may have filtered the tasks. Select the correct filters in the corresponding task list, such as the correct region and resource group.
Verify that the All Resources resource scope drop-down list and the Region drop-down list (for example, China (Hangzhou)) in the top navigation bar of the console are set correctly. A mismatch in the resource scope or region can cause a newly created task to be invisible.
Q: Can I modify the grayed-out configuration items of a task that has already been created?
A: No.
Q: How do I configure latency alerts and thresholds?
A: DTS provides monitoring and alerting features. You can set alert rules for key monitoring metrics in the console to keep track of the running status of your tasks. For configuration details, see Configure monitoring and alerting.
Q: Can I view the failure cause of a task that has been failing for a long time?
A: No. If a task has been failing for a long time (for example, more than 7 days), the related logs are cleared, so the failure cause cannot be viewed.
Q: Can a task that has been failing for a long time be recovered?
A: No. If a task has been failing for a long time (for example, more than 7 days), the related logs are cleared and the task cannot be recovered. You must reconfigure the task.
Q: What is the rdsdt_dtsacct account?
A: If you did not create the rdsdt_dtsacct account, it may have been created by DTS. DTS creates the built-in rdsdt_dtsacct account in some database instances to connect to the source and destination database instances.

Q: How do I retrieve information about heap tables, tables without primary keys, compressed tables, tables with computed columns, and tables with sparse columns in SQL Server?

Run the following SQL statements to check whether the source database contains such tables:

Check for heap tables in the source database:

SELECT s.name AS schema_name, t.name AS table_name FROM sys.schemas s INNER JOIN sys.tables t ON s.schema_id = t.schema_id AND t.type = 'U' AND s.name NOT IN ('cdc', 'sys') AND t.name NOT IN ('systranschemas') AND t.object_id IN (SELECT object_id FROM sys.indexes WHERE index_id = 0);

Check for tables without a primary key:

SELECT s.name AS schema_name, t.name AS table_name FROM sys.schemas s INNER JOIN sys.tables t ON s.schema_id = t.schema_id AND t.type = 'U' AND s.name NOT IN ('cdc', 'sys') AND t.name NOT IN ('systranschemas') AND t.object_id NOT IN (SELECT parent_object_id FROM sys.objects WHERE type = 'PK');

Check for primary key columns that are not included in the clustered index:

SELECT s.name schema_name, t.name table_name FROM sys.schemas s INNER JOIN sys.tables t ON s.schema_id = t.schema_id WHERE t.type = 'U' AND s.name NOT IN('cdc', 'sys') AND t.name NOT IN('systranschemas') AND t.object_id IN ( SELECT pk_colums_counter.object_id AS object_id FROM (select pk_colums.object_id, sum(pk_colums.column_id) column_id_counter from (select sic.object_id object_id, sic.column_id FROM sys.index_columns sic, sys.indexes sis WHERE sic.object_id = sis.object_id AND sic.index_id = sis.index_id AND sis.is_primary_key = 'true') pk_colums group by object_id) pk_colums_counter inner JOIN ( select cluster_colums.object_id, sum(cluster_colums.column_id) column_id_counter from (SELECT sic.object_id object_id, sic.column_id FROM sys.index_columns sic, sys.indexes sis WHERE sic.object_id = sis.object_id AND sic.index_id = sis.index_id AND sis.index_id = 1) cluster_colums group by object_id ) cluster_colums_counter ON pk_colums_counter.object_id = cluster_colums_counter.object_id and pk_colums_counter.column_id_counter != cluster_colums_counter.column_id_counter);

Check for compressed tables in the source database:

SELECT s.name AS schema_name, t.name AS table_name FROM sys.objects t, sys.schemas s, sys.partitions p WHERE s.schema_id = t.schema_id AND t.type = 'U' AND s.name NOT IN ('cdc', 'sys') AND t.name NOT IN ('systranschemas') AND t.object_id = p.object_id AND p.data_compression != 0;

Check for tables that contain computed columns:

SELECT s.name AS schema_name, t.name AS table_name FROM sys.schemas s INNER JOIN sys.tables t ON s.schema_id = t.schema_id AND t.type = 'U' AND s.name NOT IN ('cdc', 'sys') AND t.name NOT IN ('systranschemas') AND t.object_id IN (SELECT object_id FROM sys.columns WHERE is_computed = 1);

Check for tables that contain sparse columns:

SELECT s.name AS schema_name, t.name AS table_name FROM sys.schemas s INNER JOIN sys.tables t ON s.schema_id = t.schema_id AND t.type = 'U' AND s.name NOT IN ('cdc', 'sys') AND t.name NOT IN ('systranschemas') AND t.object_id IN (SELECT object_id FROM sys.columns WHERE is_sparse = 1);

Q: How do I resolve inconsistent schemas between the source and destination?
You can use the object name mapping feature to establish column-level mappings between the source and destination. For more information, see Object name mapping.
Note
Modifying column data types is not supported.
Q: Does database, table, and column mapping support modifying column data types?
A: No.
Q: Does DTS support limiting the read speed of the source database?
A: No. You must evaluate the performance of the source database (for example, whether the IOPS and network bandwidth meet the requirements) before running the task. We also recommend that you run the task during off-peak hours.
Q: How do I set the number of instances displayed per page in the task list in the console?
Note
This procedure uses data synchronization instances as an example.
1. Go to the data synchronization task list page in the destination region. You can do this in one of two ways.
  DTS console
  Log on to the DTS console.
  In the navigation pane on the left, click Data Synchronization.
  In the upper-left corner of the page, select the region where the synchronization instance is located.
  DMS console
  Note
  The actual steps may vary depending on the mode and layout of the DMS console. For more information, see Simple mode console and Customize DMS console layout and style.
  Log on to the DMS console.
  In the top menu bar, choose Data + AI > DTS (DTS) > Data Synchronization.
  To the right of Data Synchronization Tasks, select the region of the synchronization instance.
2. On the right side of the page, drag the scroll bar to the bottom of the page.
3. In the lower-right corner of the page, select Items per Page.
  Note
  Items per Page can be set only to 10, 20, or 50.
Q: How do I resolve a ZooKeeper connection timeout on a DTS instance?
Try restarting the instance to check whether it recovers. For information about how to restart an instance, see Start a DTS instance.
Q: Why is the DTS CIDR block automatically added back after I delete it in Cloud Enterprise Network (CEN)?
A: This may be because you used the Basic Edition transit router of Cloud Enterprise Network (CEN) to connect the database to DTS. If you use this database to create a DTS instance, DTS automatically adds the server IP address range to the corresponding router, even if you delete the DTS CIDR block in CEN.
Q: Can DTS tasks be exported?
A: No.
Q: How do I call OpenAPI in Java?
A: The method of calling OpenAPI in Java is similar to the method in Python. For more information, see Python SDK example. Go to the Data Transmission Service (DTS) SDK page, select the target programming language from All Languages, and view the sample code.
Q: Does DTS support Azure SQL Database?
A: Yes. If the source database is Azure SQL Database, set the SQL Server Incremental Synchronization Mode parameter to Polling and querying CDC instances for incremental synchronization.
Q: Do I need to configure RAM authorization when I use the Internet to configure a cross-account DTS task?
No, that is not necessary. When configuring the DTS task, you can select the public IP address as the connection method for the database instance and then proceed with the corresponding configuration.
Note
Data synchronization tasks do not support connecting to database instances via public IP addresses.
Q: Does DTS support automatic scaling?
A: DTS Serverless instances can automatically adjust instance resources to implement automatic scaling. For more information, see What is a DTS Serverless Instance.
Q: Does the public cloud support TDDL?
A: No. If the data that you want to synchronize or migrate has TDDL rules configured, the task may fail.

DTS console

DMS console