All Products
Search

This Product

    Lindorm:HBase data synchronization and migration

    Document Center

    Lindorm:HBase data synchronization and migration

    Last Updated:Jul 16, 2026

    If you want to migrate or synchronize data from a self-managed HBase cluster or an ApsaraDB for HBase cluster to the Lindorm wide table engine, you can use Lindorm Tunnel Service (LTS) to complete historical data migration and real-time data synchronization. This topic describes the use scenarios, features, migration advantages, usage limits, and migration tasks of HBase data synchronization and migration.

    Use scenarios

    • Migrate self-managed HBase to the cloud: Migrate HBase data to Lindorm.

    • Cross-region migration: For example, migrate from the China (Qingdao) region to the China (Beijing) region.

    • Business split: Migrate a portion of business to a new cluster.

    Features and advantages

    Features

    • Supports zero-downtime migration from HBase 1.x and 2.x to Lindorm.

    • Supports table schema migration, real-time data synchronization, and historical data migration.

    • Supports database-level, namespace-level, and table-level migration.

    • Supports table renaming during migration.

    • Supports specifying time ranges, rowkey ranges, and specific columns for migration.

    • Provides OpenAPI for creating migration tasks.

    Migration advantages

    • Zero-downtime data migration that completes both historical data migration and real-time incremental data synchronization.

    • During migration, the self-managed HBase does not interact with the source HBase cluster. It only reads HDFS from the source cluster, which effectively reduces the impact on online business of the source cluster.

    • File-layer data copying is more efficient than API-layer data migration, typically saving more than 50% of traffic.

    • A single node can achieve a migration speed of 150 MB/s. The number of nodes supports horizontal scaling to meet TB- or PB-level data migration requirements.

    • Provides comprehensive error retry mechanisms, real-time monitoring of task speed and progress, and supports task failure alerts.

    • Automatically synchronizes schemas to ensure partition consistency.

    Limits

    • Migration to self-managed HBase is not supported.

    • Clusters with Kerberos enabled are not supported.

    • Single-node ApsaraDB for HBase instances are not supported.

    • ApsaraDB for HBase instances in classic networks are not supported due to network limitations.

    • Migration or synchronization to Lindorm standalone instances is not supported.

    • Incremental data synchronization is implemented based on HBase WAL for asynchronous data synchronization. Data loaded by bulkload and data not written to WAL will not be synchronized.

    • Only LTS 3.8.10 and later support migration of search indexes in tables.

    Precautions

    • Before migration, confirm the HDFS capacity of the destination cluster to prevent the capacity from being full during migration.

    • If the source cluster is a self-managed HBase or ApsaraDB for HBase cluster, before submitting incremental synchronization, we recommend that you set the log retention time of the source cluster to more than 12 hours to reserve sufficient processing and recovery time for exceptions that may occur during synchronization. Modification method: Modify the value of the hbase.master.logcleaner.ttl parameter in the hbase-site.xml file and restart HMaster after modification. The unit of the hbase.master.logcleaner.ttl parameter is milliseconds (ms). Pay attention to unit conversion when setting. For example, hbase.master.logcleaner.ttl=43200000 indicates that the log retention time is set to 12 hours.

      Note

      If the source cluster is a Lindorm cluster, you do not need to modify the log retention time.

    • You do not need to create tables in the destination cluster. The LTS synchronization service automatically creates tables that are the same as those in the source cluster, including partition information. Customer-created destination tables may have partitions that are very inconsistent with the source tables, which may cause frequent splits and compactions in the destination table after migration. If the data volume of the table is very large, this process may be very time-consuming.

    • If the source table has a coprocessor, you need to ensure that the destination cluster contains the JAR package corresponding to the coprocessor when creating the destination table.

    • After incremental synchronization is enabled, if data is not consumed, logs are retained for 48 hours by default. After the timeout, the subscription relationship is automatically canceled and the retained data is automatically deleted.

    Prerequisites

    • Network connectivity between the source cluster, destination cluster, and LTS has been verified.

    • HBase and Lindorm data sources have been added. For more information, see Add an HBase data source and Add a LindormTable data source.

    • Enable underlying file access for Lindorm in the console.

    Create a task

    1. Sign in to LTS.

    2. In the left-side navigation pane, choose Lindorm/HBase Migration > Quick Migration.

    3. Click Create a task.

    4. In the Task Name (Optional) text box, enter a task name. The task name only supports English characters and digits. This parameter is optional. If not specified, the task ID is used as the task name by default.

    5. Set the Source Cluster and Destination Cluster as prompted.

    6. Select the required Operation.

      • Table Schema Migration: Creates tables in the destination cluster (with consistent schema and partition information). If the table already exists in the destination cluster, it will be skipped.

      • Real-time Data Replication: Synchronizes real-time incremental data from the source cluster.

      • Historical Data Migration: File-level full physical migration of files.

    7. In Tables to Synchronize/Migrate, enter the primary tables to be migrated.

    8. Select operations on index tables.

    9. (Optional) Configure Advanced Configurations.

    10. Click Create.

    View tasks and task details

    1. In the left-side navigation pane, choose Lindorm/HBase Migration > Quick Migration > to view tasks.

    2. Click the task name to view the execution status of the corresponding task.

    Switchover

    1. Wait for the full migration task to complete. The latency of incremental migration is relatively small (a few seconds or hundreds of milliseconds).

    2. Enable LTS data sampling verification. For large tables, the sampling ratio should not be too large to avoid impacting online business.

    3. Perform business verification.

    4. Perform business switchover.

    FAQ

    What scenarios cause data not to be consumed?

    Releasing the LTS cluster directly without terminating the task, suspending the synchronization task, or task exception blocking.

    What should I do if a migration task fails?

    There are many possible reasons for migration task failure, such as network instability and service conflicts. LTS has a built-in retry mechanism and will automatically retry after task failure. If it still fails after retry, contact Lindorm technical support (DingTalk ID: s0s3eg3).