All Products
Search

This Product

    Lindorm:HBase data synchronization and migration

    Document Center

    Lindorm:HBase data synchronization and migration

    Last Updated:Mar 30, 2026

    Use Lindorm Tunnel Service (LTS) to migrate historical data and synchronize incremental data from a self-managed HBase cluster or an ApsaraDB for HBase cluster to LindormTable — with zero downtime.

    Use cases

    • Migrate a self-managed HBase cluster to Lindorm.

    • Perform cross-region migration, for example from the China (Qingdao) region to the China (Beijing) region.

    • Split business workloads by migrating a portion to a new cluster.

    How LTS works

    LTS reads data directly from the Hadoop Distributed File System (HDFS) of the source cluster rather than interacting with the HBase service. This file-layer approach typically reduces data transfer volume by over 50% compared to API-layer replication and minimizes impact on your online business during migration.

    A migration runs in up to three phases, which you can combine into a single task:

    Phase Operation Description
    1 Table schema migration LTS creates tables in the destination cluster with the same schema and region information as the source. Region consistency is maintained automatically.
    2 Historical data migration LTS performs a physical, file-level migration of all existing data at up to 150 MB/s per node. Scale horizontally to handle terabyte- or petabyte-scale datasets.
    3 Incremental data synchronization LTS replicates write-ahead log (WAL) entries from the source cluster in real time, keeping the destination in sync while your business continues to run.

    Supported sources and features

    LTS supports migration from HBase 1.x and 2.x clusters, including self-managed clusters and ApsaraDB for HBase clusters. Within a single task, you can:

    • Migrate at the database, namespace, or table level.

    • Rename tables during migration.

    • Filter data by time range, row key range, or specific columns.

    • Create migration tasks via the LTS API.

    LTS includes a built-in retry mechanism and provides real-time monitoring of task speed and progress, with alerts for task failures.

    Limitations

    • Migration to a self-managed HBase cluster is not supported. LindormTable is the only supported destination.

    • HBase clusters with Kerberos enabled are not supported.

    • ApsaraDB for HBase single-node instances are not supported.

    • ApsaraDB for HBase instances in the classic network are not supported.

    • Lindorm standalone instances are not supported as a destination.

    • Incremental synchronization relies on WAL entries. Data imported via bulk loading or data not written to the WAL is not synchronized.

    • Search index data migration is not supported.

    Before you begin

    Prepare your environment

    Before starting migration, complete the following:

    1. Verify network connectivity between the source cluster, the destination cluster, and LTS.

    2. Add the HBase data source. For more information, see Add an HBase data source.

    3. Add the Lindorm wide table data source. For more information, see Add a Lindorm wide table data source.

    4. Verify that the destination cluster has enough HDFS capacity to prevent storage issues during migration.

    Configure log retention (self-managed HBase or ApsaraDB for HBase source only)

    Set the log retention period to more than 12 hours before starting incremental synchronization. This gives LTS enough time to recover from synchronization errors.

    To change the retention period, update the hbase.master.logcleaner.ttl parameter in hbase-site.xml and restart HMaster. The unit is milliseconds (ms). For example, to set a 12-hour retention period:

    hbase.master.logcleaner.ttl=43200000
    If the source cluster is a Lindorm cluster, skip this step.

    Review usage notes

    • Do not manually create tables in the destination cluster. LTS creates them automatically with the same schema and region information as the source. Manually created tables may have inconsistent region configurations, leading to excessive splits and compactions after migration — especially costly for large tables.

    • If a source table has a coprocessor, make sure the destination cluster has the corresponding coprocessor JAR file before LTS creates the destination table.

    • After enabling incremental synchronization, LTS retains unconsumed data logs for 48 hours by default. After this period, the subscription is automatically canceled and the retained data is deleted.

    Create a migration task

    1. Log on to LTS. For more information, see Log on to LTS.

    2. In the left-side navigation pane, choose Migration > Quick Migration.

    3. Click Create Task.

    4. In the task name (optional) field, enter a name. The name can contain only letters and digits. If left blank, the task ID is used as the task name.

    5. Configure the source cluster and destination cluster.

    6. Select the operations to include:

      Operation Description
      table schema migration Creates tables in the destination cluster with the same schema and region information as the source. Existing tables are skipped.
      real-time data replication Synchronizes incremental WAL data from the source cluster in real time.
      historical data migration Performs a file-level physical migration of all data.

    7. Configure table mapping and, if needed, advanced configuration.

    8. Click Create.

    Monitor the task

    In the left-side navigation pane, choose Migration > Quick Migration to view all tasks. Click a task name to see its execution status, including migration speed and progress.

    The task is ready for switchover when:

    • Historical data migration shows as complete.

    • Incremental synchronization latency is low — typically a few seconds or milliseconds.

    Perform a switchover

    1. Confirm that historical data migration is complete and incremental latency is low.

    2. In LTS, enable data sampling and validation. For large tables, use a small sampling ratio to avoid impacting your online business.

    3. Validate your business on the destination cluster.

    4. Perform the business switchover.

    FAQ

    Why did data consumption stop?

    Data consumption stops when the LTS cluster is released before the task is terminated, the synchronization task is suspended, or the task is blocked by an error.

    What should I do if a migration task fails?

    LTS automatically retries failed tasks. If the task still fails after multiple retries, contact Lindorm technical support on DingTalk (ID: s0s3eg3).