Data Transmission Service:What is Data Transmission Service (DTS)?

Core capabilities

DTS provides core capabilities for data flow, covering various scenarios from data migration and real-time synchronization to incremental change tracking. These capabilities can be categorized as follows:

Product advantages

DTS simplifies complex data interaction tasks with more features, better performance, greater ease of use, and more enhanced security and reliability than traditional tools.

• High compatibility

  • Heterogeneous data source support: Supports data transmission between various data sources, including relational databases (such as MySQL, PostgreSQL, SQL Server, Oracle), NoSQL databases (such as MongoDB, Redis), and big data warehouses (such as AnalyticDB, MaxCompute). For more information, seeSupported databases.

  • Seamless cross-environment connectivity: Enables data flow between cloud services (such as ApsaraDB RDS, cloud-native database PolarDB), on-premises self-managed databases, and databases from other cloud vendors to connect hybrid and multi-cloud architectures.

  • Cross-account collaboration: To meet complex enterprise collaboration needs, DTS lets you configure tasks across Alibaba Cloud accounts. You can simply configure RAM authorization to securely share data between different accounts, enabling unified data management and flow within a corporate group or across departments.

  • Flexible network topologies: Supports network access methods, including public networks, private connections, VPN Gateways, and Smart Access Gateways, to adapt to complex enterprise network environments.

• Rich functionality

  • Provides multiple transmission methods, including data migration, data synchronization, and Change Tracking. For heterogeneous data source migrations, DTS supports the transformation of database and table schemas.

  • The real-time synchronization feature supports unidirectional or bidirectional synchronization, applicable to various scenarios such as geo-disaster recovery, active geo-redundancy, and real-time data warehouses.

• Easy to use

  DTS provides a visual management interface and a guided task creation workflow, allowing you to create and manage transmission tasks with ease. The console displays the task's transmission status, progress, performance, and other relevant information. DTS also includes breakpoint resumption and automatic recovery and restart capabilities to effectively handle network or system exceptions.

• High performance

  DTS uses high-specification servers to ensure transmission performance. For data migration, multiple underlying performance optimizations enable peak full data migration performance of up to 70 MB/s. For real-time synchronization, DTS can narrow the concurrency granularity to the transaction level, achieving a peak performance of 30,000 RPS. DTS also supports multi-threaded compressed transmission to reduce bandwidth consumption.

• Secure and reliable

  The underlying architecture of DTS uses a service cluster. A single node failure triggers a quick switchover, ensuring high task stability. The service provides 24/7 data accuracy verification and uses secure transport protocols, secure token authentication, and SSL encryption to guarantee the reliability and security of data transmission. In cross-border and cross-region synchronization scenarios, DTS uses private connection networks for isolation, further enhancing data security.

• Zero-downtime migration

  You can migrate data while the source database remains operational, reducing application downtime to minutes.

Use cases

• Zero-downtime database migration to the cloud

  When migrating on-premises databases (such as MySQL, Oracle) or self-managed databases on ECS to Alibaba Cloud databases like ApsaraDB RDS or PolarDB, you can use data migration to avoid prolonged business interruption. This feature uses a full migration plus incremental synchronization approach to complete the data migration while the business continues to handle write operations, allowing you to switch your application to the new cloud database.

• Build active geo-redundancy and disaster recovery architectures

  To achieve high availability and geo-disaster recovery for core business systems, you can establish real-time bidirectional synchronization between database instances in different regions. Data synchronizationis recommended for this purpose. It ensures that data changes on either end are synchronized to the other in real time. If a failure occurs in one region, business traffic can be quickly switched to the other data center, ensuring business continuity.

• Real-time data warehouse ETL and cache updates

  When business data changes and needs to be updated in a downstream analytical warehouse (such as AnalyticDB and ClickHouse) or Redis in real time, you can use change tracking. DTS captures incremental logs (such as Binlog and WAL) from the source database and converts them into a structured JSON data stream. Applications can consume this data stream to write to the warehouse or directly update the cache, achieving real-time decoupling of the data link.

• Real-time data cleansing and formatting

  When synchronizing data from a production database to an analytical or testing environment, data processing is often required. For example, you might need to perform data desensitization on sensitive information like customer phone numbers or IDs, or merge the first_name and last_name fields from the source into a single full_name field in the target. In such cases, you can use ETL to perform these transformations in real time during data transmission, eliminating the need for secondary development on the target side, simplifying the data link, and ensuring data security.

• Data consistency verification

  After completing a critical database migration or during a long-term geo-disaster recovery synchronization process, you can use data verification to ensure data consistency. It automatically compares all data or a specified range of data and generates a detailed verification report, clearly indicating which tables and rows have data discrepancies. This provides data-level confidence for business cutover after migration and offers a monitoring tool for the health of long-term synchronization tasks, preventing data inconsistencies.

Get started with DTS

1. Identify the DTS feature that are applicable for your scenario.

2. Check if your database is supported for data migration, synchronization, or change tracking with the selected feature.

3. Create a DTS task. You can create tasks through the visual Data Transmission Service console or using the API or SDK.

   • Manage a data synchronization task

   • Manage a data migration task

   • Manage a change tracking task

More information

FAQ