Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Cloud Migration without Stopping Services: Alibaba Cloud DTS

Cloud Migration without Stopping Services: Alibaba Cloud DTS

Alibaba Cloud Data Transmission Service (DTS) lets you perform data migration without stopping services, ensuring data reliability while maintaining business continuity.

By the Database Business Unit

More and more enterprise customers are choosing to migrate to the cloud to fully benefit from its convenience and high reliability. However, for many customers, the first issue encountered when migrating to the cloud is to smoothly complete the cloud migration without affecting the business. Many traditional migration solutions require businesses to stop services during migration, which affects business continuity.

Alibaba Cloud Data Transmission Service (DTS) has the ability to perform data migration without stopping services, meaning the cloud migration is completed while the source database provides the normal services of the business. In this article, we will discuss the principles and process of cloud migration, as well as explain the benefits of using Alibaba Cloud's DTS.

Migration after Stopping Services - Traditional Database Migration Solution

A variety of database engines have emerged in the market to meet different user scenarios. To meet user requirements for importing data, database vendors typically have their own data migration tools. For example: mysqldump for MySQL and exp/imp for Oracle.

However, traditional migration tools realize the migration of existing data through data export and import. To ensure the integrity of the migrated data, the source database service is usually stopped and the amount of time for which the service remains stopped depends on how long it takes to migrate the existing data. This can be several hours, and results in an extremely high migration cost for users.

Migration without Stopping Services - Alibaba Cloud DTS

Alibaba Cloud provides a migration service called Data Transmission Service (DTS), to reduce the migration cost to users. DTS supports data transmission between data sources such as relational databases, NoSQL, and Big Data (OLAP). It is a data transmission service that integrates data migration, data log real-time subscription, and real-time data synchronization.

DTS provides data migration without stopping services, enabling the smooth completion of the migration of database data without stopping the services related to the source database. Users can apply data migration in various scenarios:

  1. Migrate a self-built database from an offline IDC room to the self-built database on Alibaba Cloud RDS or ECS.
  2. Migrate a self-built database from other cloud vendors to the self-built database on Alibaba Cloud RDS or ECS.
  3. Migrate a database on Alibaba Cloud across regions and accounts.
  4. Migrate a self-built database on Alibaba Cloud RDS or ECS to the offline IDC test environment.

In addition to data migration, DTS helps complete various business scenarios such as data disaster backup, multi-idc-live, real-time synchronization of business data with the data warehouse, asynchronous message notification, and cache update.

DTS supports dozens of data sources including Oracle, MySQL, SQLServer, PostgreSQL, DB2, Redis, and MongoDB. Data transmission services have been provided to tens of thousands of enterprise customers.

DTS Migration Process

1

The general process of using DTS to perform database migration without stopping services is divided into the following phases:

  1. Preparation, including that of the target database, the source database, and the migration account of the target database.
  2. Schema migration: DTS migrates the schema objects of the source database to the target database.
  3. Full migration. DTS migrates historical data from the source database to the target database.
  4. Incremental migration. DTS migrates incremental data from the source database to the target database. Incremental migration is divided into two steps: incremental data fetching and incremental data synchronization. To ensure the integrity of the migrated data, DTS starts incremental data fetching at the same time as the schema migration, and captures the incremental data generated by the source instance during the process of schema migration and full migration in real time.
  5. Data verification. Users verify the consistency of the migrated data between the source database and the target database.
  6. Business switching. Selects the low peak period for the business, and points the business to the target database when DTS incremental migration will have no delay.

Throughout the entire data migration period, the source database normally provides services, but points the business to the target database during the business switching phase.

Basic Principles of DTS Migration

In this section, we introduce the basic technical principles of each migration phase.

1. Schema Migration

2

During the schema migration phase, DTS generates a schema on the target instance based on the schema definition (e.g., tables and views) of the source instance. If the source instance and the target instance are heterogeneous data sources, DTS needs to convert data types and SQL syntax according to the schema syntax of the target instance. For objects that are incompatible with the target data sources, you can manually correct them in DTS Console – Migration Details before synchronizing them with the target instance.

2. Full Migration

3

During the full migration phase, DTS migrates historical data from the source database to the target instance. The table fragmentation technology allows simultaneous execution of fragmentation in the table, maximizing the migration speed.

DTS also supports resumable upload. When the migration is interrupted due to a network issue – source instance or target instance – DTS can resume data transmission from the breakpoint after the issue has been resolved, which greatly reduces the migration workload.

3. Incremental Migration

4

During the incremental migration phase, DTS captures data changes on the source database in real time and synchronizes it with the target database in real time.

With the deadlock algorithm, DTS can implement a transaction level-based concurrent write strategy to effectively improve the synchronization performance while ensuring the transaction sequentiality.

Migration Process - 3 Steps to Complete Database Migration

With DTS, users can completely configure the entire task in just 3 steps. We demonstrate below the use of DTS with a case involving migrating data from another cloud vendor's RDS to Alibaba Cloud RDS.

Preparations

Before configuring DTS tasks, you need to enable the external network address of the other cloud vendor's RDS instances.

DTS supports access to another cloud vendor's RDS instances through hardware devices such as private lines and VPN gateways, in addition to access through public networks.

Performing Migration

Open the Alibaba Cloud DTS console and click Create Migration Task in the upper-right corner to start task configuration.

Step 1: Configure the connection information for the source MySQL instance and the target RDS for the MySQL instance.

This step involves setting the migration task name and configuring the connection information for the source and target instances involved in the migration. Details are as follows:

  1. Name of the task: By default, DTS generates a name for every task automatically. The task name does not need to be unique, and can be modified. A name indicating the specific business of the task is recommended to facilitate task identification.

You will need to provide source instance connection information, including:

  1. Instance type: Select a self-built instance with a public network IP.
  2. Database type: Select MySQL.
  3. Host name or IP address: Configure the external network address for RDS MySQL that is on the other cloud vendor.
  4. Port: Configure instance listening ports for RDS MySQL that is on the other cloud vendor.
  5. Account: Configure the instance access account for MySQL that is on the other cloud vendor.
  6. Password: Configure the password corresponding to the access account for MySQL that is on the other cloud vendor.

You will also need to configure target RDS for MySQL instance connection information. The target instance configures the instance connection information for Alibaba Cloud RDS. The specific configuration information is as follows:

  1. Instance type: Select a RDS instance
  2. RDS instance ID: Instance ID of the target RDS instance to which you want to migrate data. DTS supports RDS for MySQL instances using classic networks and VPC networks
  3. Database account: the connection account for the RDS for MySQL instance
  4. Database password: the database password corresponding to the database account specified above

5

Step 2: Configure the migration object and migration type

  1. Migration type

    DTS supports schema migration, full migration, and incremental migration. To migrate without stopping services, select: Schema Migration + Full Migration + Incremental Migration.

  2. Migration object

    Select the object to be migrated. The migration object can be a database, a table, or a column. After the migration object and migration type are configured, the pre-check for the migration task is started.

Step 3: Migration pre-check

A pre-check is required before migration tasks can start. The pre-check needs to be passed successfully in order for the migration to start.

After passing the pre-check, you can start the migration task. After the task starts, you can view the specific migration status and progress in the task list.

This completes the configuration of the task for migrating data from a MySQL database instance on another cloud vendor to an Alibaba Cloud RDS for MySQL instance.

Conclusion

We introduced the basic principles of Alibaba Cloud Data Transmission Service (DTS) and the database migration process. You have learned that DTS is a convenient, safe, and efficient tool for database migration, and you can easily get started with Migration to Cloud based on the ability of DTS to perform migration without stopping services.

In addition to the function of data migration, DTS also supports real-time subscription of database logs and real-time synchronization of data. With these functions, users can realize business scenarios such as data disaster backup, multi-idc-live, real-time synchronization of business data to the data warehouse, asynchronous message notification, and cache update. Due to space limitations, no detailed introduction is given here.

Database dts Migration Data Transmission Service Data Migration
1 1 1
Share on

Read previous post:

Core Technologies of Alibaba Cloud Voice Recognition Model

Read next post:

Alibaba Cloud Blockchain-as-a-Service (BaaS) + Multimedia AI: Building a Trusted Digital Content Services Ecosystem

Alibaba Cloud Product Launch

10 posts | 2 followers

You may also like

Comments

Raja_KT March 9, 2019 at 3:43 am

Good one. Hope you can explain more on " deadlock algorithm"

Alibaba Cloud Product Launch

10 posts | 2 followers

Related Products

More Posts by Alibaba Cloud Product Launch

See All