Data Migration
|
Feature Set |
Feature |
Description |
References |
|
Homogeneous migration |
Logical migration |
Supports data migration between homogeneous databases. |
Migrate data from a self-managed MySQL database to an ApsaraDB RDS for MySQL instance |
|
Physical migration |
Migrates self-managed databases to the cloud by using a physical gateway. |
||
|
Heterogeneous migration |
Logical migration |
Supports data migration between heterogeneous databases. |
|
|
Traffic cutover |
Database cutover to the cloud |
Database cutover helps you smoothly migrate your business to a cloud database after you complete database assessment and application assessment and transformation. |
- |
|
Periodic tasks |
Scheduled full migration |
Periodically migrates the schema and existing data from the source database to the destination database by using the scheduling policy of the data integration feature. |
Configure a data integration task between ApsaraDB RDS for MySQL instances |
|
Account migration |
Full account migration |
When you configure a synchronization or migration task, you can enable the account migration feature to conveniently migrate the accounts (including passwords and permissions) of the source database to the destination database. |
Migrate database accounts |
Data Synchronization
|
Feature Set |
Feature |
Description |
References |
|
Synchronization instance management |
One-click reverse disaster recovery switchover |
Creates a reverse incremental synchronization instance with one click for a running synchronization instance. |
Create a reverse instance |
|
Disaster recovery & multi-active redundancy |
Two-way synchronization |
Supports creating real-time two-way data synchronization between two databases (such as ApsaraDB RDS for MySQL and a self-managed MySQL database). This feature applies to various scenarios such as geo-redundancy based on a cellular architecture deployments and geo-disaster recovery. |
|
|
One-click reversal of synchronization direction |
Supports reversing the direction of a two-way synchronization instance. When your business requires a primary/secondary database switchover or a dual-cloud switchover, you can easily adjust the direction of an existing two-way synchronization instance without reconfiguring it. |
Reverse the direction of a two-way synchronization instance | |
|
Global Active Database (GAD) cluster |
Supports building a Global Active Database cluster by using ApsaraDB RDS and DTS, which provides database disaster recovery protection while delivering a nearby-access experience for users. |
What is Global Active Database? | |
|
Synchronization topology management |
Supports upgrading the synchronization topology of a task from one-way to two-way to meet evolving business requirements. |
Upgrade a one-way synchronization task to a two-way synchronization task | |
|
Conflict management |
Conflict detection |
Supports detecting conflicts such as uniqueness conflicts caused by INSERT operations, incomplete matches of records updated by UPDATE operations, and missing records corresponding to DELETE operations. |
- |
|
Conflict handling |
Supports three conflict resolution policies: TaskFailed (the task reports an error and exits when a conflict occurs), Ignore (the conflicting record in the destination instance is used when a conflict occurs), and Overwrite (the conflicting record in the destination instance is overwritten when a conflict occurs). |
- |
|
|
Heterogeneous synchronization |
Real-time data warehouse synchronization |
Supports synchronizing data to a real-time data warehouse, helping you achieve high-throughput offline processing and high-performance online analysis. |
Synchronize data from an ApsaraDB RDS for MySQL instance to an AnalyticDB for MySQL V3.0 cluster |
|
Non-database synchronization |
Supports synchronizing data to a specified function in Function Compute (FC). You can write function code to further process the data. |
||
|
Heterogeneous database synchronization |
Supports data synchronization between heterogeneous databases. |
||
|
Data shipping |
Data shipping channel |
Establishes a data shipping channel by creating a data shipping instance, so that you can subsequently use an SDK to ship data sources to DTS. |
Create a data shipping instance |
|
Data shipping SDK |
Ships various types of data sources to DTS through an SDK, and then DTS synchronizes the data to the destination database, thereby supporting more types of data sources. |
Use the SDK to ship data to DTS | |
|
Homogeneous synchronization |
Logical real-time homogeneous synchronization |
Supports data synchronization between homogeneous databases. |
Synchronize data from a self-managed MySQL database to an ApsaraDB RDS for MySQL instance |
Change Tracking
|
Feature Set |
Feature |
Description |
References |
|
Change tracking |
Change tracking channel |
Creates a change tracking instance to obtain real-time incremental data changes of a database. |
Subscription plan overview |
|
Change tracking SDK |
Supports printing tracked data by using SDK client sample code, flink-dts-connector, or Kafka client sample code, so that the SDK client, Flink client, or Kafka client can consume the data in the change tracking channel. |
Consume tracked data | |
|
Change tracking traffic management |
When the source database of a change tracking instance is MySQL, you can choose whether to charge data traffic fees by configuring the billing type. |
Task Management
|
Feature Set |
Feature |
Description |
References |
|
Task management |
Task creation and configuration |
Supports creating data transmission tasks between various data sources, and provides two task creation methods: configure before purchase and purchase before configure. |
|
|
Task scheduling |
By default, DTS schedules tasks to a shared cluster. You can select a dedicated cluster to schedule the task during the advanced configuration stage. |
- |
|
|
Task monitoring |
Obtains the link status information and various performance metrics of a task through the DTS instance in the console, providing a reference for you to manage tasks. |
Monitor task performance | |
|
Task diagnostics |
Provides diagnostic results and suggestions by checking the performance of the source database, destination database, network, and DTS during incremental migration. |
||
|
Task modification |
Supports managing the environment tags of an instance, adding or removing synchronization objects while a data synchronization instance is running, and modifying the ETL configuration of a synchronization or migration instance. |
||
|
Task deletion |
For a DTS instance whose data transmission task is completed and whose configuration no longer needs to be retained, you can manually release a pay-as-you-go instance or unsubscribe from a subscription instance to avoid additional fees. |
Release DTS instances | |
|
Cross-account access |
Supports configuring tasks in which the source or destination database spans Alibaba Cloud accounts, which applies to various scenarios such as resource migration or merging between Alibaba Cloud accounts and business architecture adjustments. |
Configure a DTS task across Alibaba Cloud accounts | |
|
Hybrid cloud and cross-cloud access |
Connects to a database by using a public IP address to enable data migration between accounts with different attributes (such as between a public cloud account and a Financial Cloud account). |
- |
|
|
Operation audit |
Queries the operation logs of an instance to obtain information about the operations performed on the instance, the operation results, and the operators. |
Query the operation logs of an instance | |
|
Data connection management |
Supports entering database information. When you configure a task, you can directly select a registered database, and DTS automatically fills in the database information without manual input. |
Data connection management | |
|
OpenAPI |
POP API |
Supports calling and debugging APIs in the OpenAPI portal. |
Use OpenAPI |
|
SDK |
Supports calling APIs by using an SDK. |
Use OpenAPI Sample Code | |
|
Terraform |
Supports the open-source tool Terraform. |
Resources | |
|
Network management |
Cross-region connectivity |
Supports data transmission in which the source database region differs from the destination database region. |
Cross-border and cross-region tasks |
|
Transmission compression |
DTS links support multi-concurrent compressed transmission, reducing the bandwidth usage of the transmission link. |
Benefits | |
|
Private endpoint access |
When you connect to the source or destination database through a virtual private cloud (VPC), you can use a private endpoint to connect to the database. |
Synchronize a self-managed MySQL database connected over an Express Connect circuit, a VPN gateway, or a Smart Access Gateway to an ApsaraDB RDS for MySQL instance | |
|
Database Gateway (DG) |
Supports connecting a database to DTS by using Database Gateway (DG). |
Connect an on-premises database or a database hosted on a third-party cloud to Alibaba Cloud by using Database Gateway | |
|
Internet traffic connection |
Supports connecting a database to DTS by using a public IP address. |
Add the CIDR blocks of DTS servers | |
|
Cross-border transmission |
By default, only non-cross-border data synchronization tasks are supported. If you need to create a cross-border data synchronization task, you must apply for the permission to enable cross-border synchronization. |
||
|
Event Center |
Event notification |
Configures customized alert notifications for important events through event subscription on the CloudMonitor platform, so that you can promptly understand the occurrence and progress of events, grasp event dynamics in real time, and quickly analyze and locate problems when a business failure occurs. |
Subscribe to event notifications |
|
Proactive O&M platform |
Supports the O&M event alert feature. When the system detects a risk that may prevent a DTS instance from running properly, it triggers the corresponding O&M event and notifies you through the console, email, or internal messages. |
Manage O&M events | |
|
Serverless |
Serverless instance management |
Supports operations on Serverless instances such as pausing an instance, viewing metric data, and modifying the upper and lower DU limits. |
Manage a Serverless instance |
|
Dedicated cluster |
DU management |
Supports viewing the creation and usage information of dedicated cluster DUs, and modifying the number of DUs for a dedicated cluster task to adjust the task specifications. |
|
|
Disk space modification |
When the disk usage of a dedicated cluster is too high to meet current business needs, you can expand the disk of the dedicated cluster to ensure that its disk space matches your business needs. |
Increase the storage space of nodes in a DTS dedicated cluster | |
|
Migration between shared and dedicated clusters |
Supports modifying the cluster on which an instance runs, that is, migrating an instance from a dedicated cluster to a public cluster, or from a public cluster to a dedicated cluster. |
Migrate an instance between a dedicated cluster and a public cluster | |
|
Dedicated cluster instance management |
Supports operations on a dedicated cluster such as manual renewal, changing node configurations, and setting the overcommit ratio. |
Manage a DTS dedicated cluster | |
|
Security |
Data encryption |
Supports connecting to a database by using an SSL secure connection. |
Synchronize data between ApsaraDB RDS for MySQL instances |
|
Operation isolation |
Supports using a RAM identity with the least privileges (that is, a RAM user or RAM role) to access DTS, which improves data security and reduces security risks caused by privilege abuse. |
Identity management | |
|
Account permission control |
Accesses cloud resources such as ApsaraDB RDS and ECS under the current Alibaba Cloud account by authorizing system permission policies, and completes the corresponding data transmission tasks by using database accounts with sufficient permissions. |
||
|
Reliability |
Cluster high availability (HA) |
Uses high-specification servers to ensure that every migration or synchronization link has good transmission performance. |
Benefits |
|
Resumable transmission |
Features an automatic resumable transmission mechanism that effectively ensures the reliability of data transmission. |
Benefits | |
|
Data source disaster recovery |
When the source or destination database cannot be connected or encounters other problems, DTS supports immediate and continuous retry operations. |
- |
Data Integration
|
Feature Set |
Feature |
Description |
References |
|
Read/write splitting and traffic distribution |
Real-time caching for transaction processing |
Supports migrating a MySQL database (such as a self-managed MySQL database or ApsaraDB RDS for MySQL) to a Redis instance, reducing the load on the backend relational database and improving user experience. |
|
|
Metadata filtering and mapping |
Database, table, and column filtering |
When you configure task objects, you can select task objects at the database, table, or column granularity. |
- |
|
DDL and DML filtering |
When you configure synchronization or migration task objects, you can use SQL operations to filter the data that needs incremental synchronization or migration. |
- |
|
|
Database, table, and column name mapping |
When you configure synchronization or migration task objects, you can set the names of the task objects (databases, tables, and columns) in the destination database instance. You can use this feature to synchronize or migrate data to specified objects in the destination database instance, or to create an object in the destination database instance that has the same structure as the task object but a different name. |
Map database, table, and column names | |
|
Topology mapping |
The data synchronization feature supports multiple synchronization topologies. You can plan your synchronization instance based on your business needs. |
Introduction to data synchronization topologies | |
|
Data filtering, processing, and mapping |
WHERE clause filtering |
When you configure synchronization or migration task objects, you can set filter conditions to synchronize or migrate the data that meets your requirements from the source database to the destination database. |
Set filter conditions |
|
Data type mapping |
When you synchronize or migrate data between heterogeneous databases, data type mapping is performed during the schema synchronization or migration stage, that is, converting the data types in the source database to the data types supported by the destination database. |
Data Verification
|
Feature Set |
Feature |
Description |
References |
|
Homogeneous verification |
Metadata verification |
Supports verifying the schema of homogeneous data. |
What is data verification? |
|
Full data verification |
Supports verifying the existing data of homogeneous databases. |
What is data verification? | |
|
Incremental data verification |
Supports verifying the incremental synchronization or migration data between homogeneous databases. |
What is data verification? | |
|
Heterogeneous verification |
Metadata equivalence verification |
Supports verifying the schema of heterogeneous data. |
What is data verification? |
|
Full data verification |
Supports verifying the existing data of heterogeneous databases. |
What is data verification? | |
|
Incremental data verification |
Supports verifying the incremental synchronization or migration data between heterogeneous databases. |
What is data verification? | |
|
Correction |
Metadata correction |
When data schema inconsistencies are detected, you can correct them based on the verification results. |
|
|
Full data correction |
When full data inconsistencies are detected, you can correct them based on the verification results by downloading the correction SQL. |
View data verification details | |
|
Incremental data correction |
When incremental data inconsistencies are detected, you can correct them based on the verification results. |
View data verification details |
Scenario-based Solutions
|
Feature Set |
Feature |
Description |
References |
|
ZeroETL |
Synchronize data from PolarDB for PostgreSQL to AnalyticDB for PostgreSQL |
- |
Seamless integration (Zero-ETL) |
|
Synchronize data from PolarDB for MySQL to AnalyticDB for MySQL V3.0 |
Federated analytics uses the free high-speed data pipeline service (APS) of AnalyticDB for MySQL to synchronize data from PolarDB for MySQL to an AnalyticDB for MySQL Lakehouse Edition (V3.0) cluster in real time, helping you complete data synchronization and management in one stop. |
Zero-ETL seamless integration synchronization of PolarDB for MySQL data to AnalyticDB for MySQL |