All Products
Search
Document Center

Introduction to LTS (formerly known as BDS)

Last Updated: Oct 14, 2021

Lindorm Tunnel Service (LTS) is a deeply customized data ecosystem service that is catered to ApsaraDB for HBase business scenarios. This service is formerly known as Big DataHub Service (BDS). LTS supports easy-to-use capabilities, including data exchange, processing, and subscription. This allows you to migrate data, subscribe to real-time data updates, dump data to data lakes, and configure backflow to data warehouses. This also allows you to apply multi-active redundancy based on units and implement backup and restoration. This way, LTS provides an all-in-one data ecosystem service for ApsaraDB for HBase.

Core capabilities

  • Cloud-native distributed system: LTS is a distributed system that is deployed based on Elastic Compute Service (ECS). It features excellent horizontal scalability and allows you to configure resources based on your business needs.

  • Ease of use: LTS allows you to configure data migration, import, subscription, and archiving tasks. For example, to create a data migration task, you need only to specify the migration source, the migration destination, and the columns to be synchronized. Based on these settings, LTS automatically replicates schemas, full data, and incremental data.

  • Security and reliability: LTS minimizes the impact on online source and destination systems and minimizes the impact of potential compatibility failures. Before a task is started, LTS prechecks the network connectivity and security. During the running process, LTS monitors the synchronization latency and the storage usage of the destination cluster in real time. LTS also implements rate limiting and reports alerts based on the monitoring data. After the task is completed, LTS verifies data.

  • Low cost: LTS is an optimized service based on open source systems, such as HBase, Phoenix, and Cassandra. It allows you to process data at the physical file level. This is 10 times more efficient than traditional data replication. LTS also optimizes the CPU, cache, memory, and network I/O capabilities. This enables LTS to provide cost-effective data channels and reduce your costs of data transfer and processing.

Features

Feature

Scenario

Reference

HBase <-> HBase

Seamless data migration between previous clusters and new clusters, cluster upgrades, online and offline workload decoupling, active/standby disaster recovery, and active geo-redundancy.

Synchronize full and incremental data

RDS -> HBase or Phoenix

Online and offline workload decoupling and historical data storage

Synchronize full and incremental data from ApsaraDB RDS

Export data from HBase to MaxCompute (ODPS)

Allows you to export historical data and incremental data.

Export full data to MaxCompute

Archive incremental data to MaxCompute

SLS-> HBase

Supports subscription to real-time data from Log Service to synchronize it to HBase.

Import incremental data from Log Service

Subscribe to HBase incremental data

Supports subscription to real-time data in ApsaraDB for HBase Performance-enhanced Edition.

Streams

Typical scenarios

  • Migration without downtime (HBase1.x, HBase2.x, HBase Performance-enhanced Edition, Phoenix4.x, and Phoenix5.x)

    • Migration without downtime. LTS allows you to migrate historical data and synchronize real-time incremental data in one task.

    • LTS does not need to interact with the source HBase cluster during the data migration process. It only reads data from the HDFS of the source cluster. This minimizes the negative impact on the workloads that run in the source cluster.

    • In most cases, compared with data migration at the API layer, data replication at the file layer can help you reduce more than 50% of the data usage.

    • High efficiency. Each node can migrate data at a rate of up to 100 MB/s. You can add nodes for horizontal scaling to migrate terabytes or even petabytes of data.

    • High stability. LTS supports a robust mechanism of rerunning failed tasks. LTS monitors the synchronization rates and the progress of tasks in real time, and reports alerts when tasks fail.

    • Data accuracy. LTS verifies the synchronized data.

    • Automatic schema synchronization. This ensures consistent partitions.

  • Online and offline workload decoupling

    • LTS allows you to synchronize online business data in real time to the HDFS or OSS storage system. LTS also works with components of big data services, such as Spark and MapReduce, to analyze data. This ensures that online business queries are not affected.

  • Active/standby disaster recovery

    • LTS supports two-way data synchronization between an active cluster and a standby cluster. When the active cluster fails, you can switch to the standby cluster to reduce the impact on your workloads. After the active cluster recovers, you can use LTS to synchronize the incremental data from the standby cluster to the active cluster.

  • Historical data storage in ApsaraDB RDS databases

    • In scenarios where historical data, such as transaction orders, is stored, performance bottlenecks may occur in ApsaraDB RDS databases due to the ever-increasing data size. Periodic data archiving or sharding is complicated and causes high costs. LTS allows you to synchronize ApsaraDB RDS data to ApsaraDB for HBase in real time. This can separate hot data from cold data. ApsaraDB for HBase supports automatic horizontal scaling, high-concurrency queries, multi-dimensional indexing, and lightweight analysis. Streams allows you to subscribe to data updates in order. LTS also allows you to synchronize data from ApsaraDB for HBase to other analytics systems for complex data analysis.