Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Data Transmission Service – What and How | Alibaba Cloud

Data Transmission Service – What and How | Alibaba Cloud

This article explains what the Data Transmission service is and how you can use it to the fullest.

By Shantanu Kaushik

Data Transmission Service (DTS) is a fully managed data replication service that supports different modes for different operations. It supports migrations between relational databases, data warehouses, and NoSQL databases. The Data Transmission Service (DTS) supports incremental data transfers from on-premises to the cloud and from cloud to cloud.

1

Data Transmission Service (DTS) supports multiple replication modes to replicate data:

  • Data Migration: This mode is used to reduce any downtime since it is mostly used for one-time migration scenarios. Incremental data migration can be used to replicate changes in real-time.
  • Data Integration: This mode is used for larger migration jobs. In this mode, DTS schedules the migration tasks and migrates the data at regular intervals.
  • Data Synchronization: As the name suggests, this mode is used to synchronize data in real-time between two data stores. This is recommended most for a distributed system.
  • Change Tracking: This mode is used to capture changes made to the data store.

Advantages

The Data Transmission Service (DTS) provides massive advantages over traditional migration tools:

  • Completely managed data migration service that requires less management and maintenance
  • Multiple replication modes for different use-case scenarios
  • Highly stable data transmission
  • Supports multiple engines, architectures, and a variety of database engines for migration
  • Supports resumable transmission in case of failures
  • Helps create a distributed data system
  • Integrates Resource Access Management (RAM) to define fine-grained authorization policies for managing DTS tasks
  • Supports scheduled and incremental data transmission

Architecture

The Alibaba Cloud Data Transmission Service (DTS) has an architectural flow that supports dual redundancy features. All of the functions of DTS are deployed on multiple servers that have dual redundancy. Let's take a look at each module to understand how it functions:

  • HA Manager performs continuous health checks on each server. If any issue is detected, the workload shifts to the redundant service without any delay. Similarly, if there is an endpoint change detected, the HA manager reconfigures the data sources for proper functioning.
  • Monitoring Server: The HA manager and the monitoring server are a part of the same module and work in sync with each other. The main difference is that the Monitoring server sends data back and forth to the DTS console and the scheduler.

Let's take a look at the architectural flow:

2

Migration Mode

Data migration follows a framework that enables a more enhanced and feature-rich migration scenario. Consisting of multiple phases called Schema Migration, full data migration, and Incremental Migration, the migration scenario is considered complete after all of these phases are finished.

Let's take a look at an architectural diagram of Migration mode:

3

  • Before Migration – The Data Transmission Service (DTS) creates a schema migration scenario to define all of the data objects and types in the target database.
  • During the Full Migration Phase – Data from the source database is replicated to the target database while keeping the source database operational and continuing with updates.
  • Incremental Migration – The Data Transmission Service (DTS) utilizes incremental data migration to read and capture data in real-time.

Synchronization Mode

In this mode, the Data Transmission Service (DTS) synchronizes the changes made in data between two data sources. Usually, this is used with Online Transaction Processing to Online Analytical Processing (OLTP to OLAP.) This process is completed in two phases:

  • Initial Data Load
  • Ongoing Replication

After loading the already present data from the source to the target data source, the Data Transmission Service starts to update and sync both data sources in real-time.

Let's take a look at the architectural flow of this scenario:

4

Tracking Mode

In this replication mode, DTS operates to capture data and highlight any changes as a publisher-subscriber stream. ApsaraDB for RDS and the log processor communicate to read the transaction log. After tracking the changes in data, they are processed, kept, and reported.

Let's take a look at the architectural flow of this mode:

5

Benefits

No Downtime

DTS allows incremental data transfers. This ensures virtually no downtime. During migration, any changes made to the source database are replicated in real-time to the target system.

As a user, after a successful migration, you can continue synchronizing both the source and target databases for as long as you like. This will enable you to switch over applications at your convenience.

6

Remote Disaster Recovery | Geo Redundancy

The Data Transmission Service (DTS) supports remote data disaster recovery by performing data replication between two instances in different regions. This service has a primary and slave (secondary) instance.

The secondary deployment sits in a different region to increase the availability of your application. DTS replicates data in real-time and keeps the two deployments synchronized to account for zero downtime in case the primary deployment suffers a catastrophe. In this case, the remote disaster recovery is the slave instance. As soon as some issue occurs, the applications can switch to the slave instance. This will enable high-availability and continuity.

Let's take a look at how this works through a visual representation:

7

Integration with Cloud BI

Alibaba Cloud has a complete solution for Business Intelligence (BI) requirements. QuickBI is a well-known product that comes from the Alibaba Cloud foundry.

Data Transmission Services (DTS) allows you to leverage the BI capabilities by replicating data in real-time from the user databases to Alibaba Cloud MaxCompute. All you need to do is integrate your applications and the Alibaba Cloud BI Storage.

Let's take a look at a diagram of how this works:

8

Real-Time Big Data Analytics

The Data Transmission Service (DTS) ensures an optimized and high-performance data delivery from the Relational Database System (RDS) to Analytics DB. The primary features for real-time big data analytics are:

  • Coupling and betterment of low-latency data before it is processed to analytical tools
  • Ad-hoc discovery

9

Decrease Remote Access

The Data Transmission Service (DTS) utilizes this architecture to increase user experience by allowing a centralized approach for processing requests. User requests from multiple regions are routed back to the center. Data in the center unit is synchronized with units in different regions to allow for a more synced and stable user experience. This allows for more enhanced performance.

Let's take a look at the architecture closely:

10

Conclusion

The Data Transmission Service is a fully managed service that enables the replication of data using different architectures to support different use-case scenarios. It does that by defining instances. These instances deliver a different capacity of replication needs to suit different industry requirements.

DTS provides a seamless data replication service and proves to be an industry-leading tool.

Next in Line

  1. Using Alibaba Cloud and Jenkins for CI
  2. Mapping | Management | Control Using RAM – Alibaba Cloud
NoSQL RDS Database Migration Data Warehouse Data Transmission Service Data Migration
0 0 0
Share on

Read previous post:

Simplify and Optimize Your Network with SD-WAN

Read next post:

Easy Deployment with Simple Application Server

Alibaba Clouder

2,605 posts | 747 followers

You may also like

Comments

Alibaba Clouder

2,605 posts | 747 followers

Related Products

More Posts by Alibaba Clouder

See All