This feature is no longer available for Lindorm Tunnel Service (LTS) instances purchased after June 16, 2023. If your LTS instance was purchased before June 16, 2023, you can still use this feature.
Use LTS to export a full snapshot of your HBase data to MaxCompute for archival or large-scale analytics. LTS supports two output modes — KV mode and wide table mode — so you can choose the schema that fits your downstream workload.
Supported versions
The following HBase sources are supported:
-
Self-managed HBase V1.x and V2.x
-
E-MapReduce HBase
-
ApsaraDB for HBase Standard Edition
-
ApsaraDB for HBase Performance-enhanced Edition (cluster mode)
-
ApsaraDB for Lindorm (Lindorm)
Prerequisites
Before you begin, make sure you have:
-
Purchased LTS, configured a username and password for the LTS web UI, and logged in. For details, see Activate LTS and log on to the LTS web UI.
-
Added an HBase data source
-
Added a MaxCompute data source
Choose an export mode
LTS supports two modes for exporting HBase data to MaxCompute. Review the comparison below before creating a job.
| KV mode | Wide table mode | |
|---|---|---|
| Output schema | One row per key-value (KV) pair — includes rowkey, version, op type, column family, qualifier, and value | One row per HBase row — each specified column becomes a MaxCompute column |
| Column selection | Not required | Required — specify columns explicitly |
| Data type control | Values exported as HexString | Specify per-column types (string, int, long, short, decimal, double, float, boolean); defaults to HexString |
| Best for | Preserving full mutation history, CDC-style use cases | Structured analytics where each column maps to a known type |
Create an export job
-
In the LTS web UI, go to Data Export > Archive Full Data to MaxCompute in the left-side navigation pane.
-
On the Lindorm/HBase archive to MaxCompute page, click create new job.
-
Select the source cluster and the destination cluster, enter the table to migrate, and click Create.
-
Click the job name to monitor migration progress.
-
After the job completes, log in to the MaxCompute console to query the exported data.
Configure export parameters
The job accepts parameters as a JSON object after the table mapping. The basic syntax is:
<hbaseTable>[/<odpsTable>] <json-parameters>-
<hbaseTable>: the name of the source HBase table. -
<odpsTable>(optional): the name of the target MaxCompute table. If omitted, defaults to the HBase table name with hyphens (-) and periods (.) replaced by underscores (_).
KV mode
tablexxx {"tableMode": "KVTable"}Each exported row represents one KV pair in HBase. The MaxCompute table contains the following columns:
| Column | Description |
|---|---|
rowkey |
The rowkey of the KV pair |
version |
The version of the KV pair |
op |
The operation type. Valid values: Put, DeleteFamily, DeleteColumn |
family |
The column family |
qualifier |
The column name |
value |
The value, converted to HexString format |
Wide table mode
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["cf1:string|string", "cf1:int|int", "cf1:double|double"]}Specify the columns to export in the cols array. Each entry uses the format <family>:<qualifier> or <family>:<qualifier>|<type>. If you omit the type, values are exported as HexString by default.
Supported types: string, int, long, short, decimal, double, float, boolean
Examples:
Export columns with default HexString encoding:
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a", "f1:b", "f1:c"]}Export columns with explicit type conversion:
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"]}The response schema:
Filter by rowkey range or time range
Add startKey, endKey, startTs, or endTs to any job parameters to limit the data exported. Both KV mode and wide table mode support these filters.
table1 {"startKey": "xxx", "endKey": "zzz", "startTs": "20191001153000", "endTs": "20191001233000"}| Parameter | Description | Format |
|---|---|---|
startKey |
Start of the rowkey range | String |
endKey |
End of the rowkey range | String |
startTs |
Start of the time range | yyyyMMddHHmmss |
endTs |
End of the time range | yyyyMMddHHmmss |
Full example with all filters applied in wide table mode:
hbaseTable/odpsTable {"tableMode": "wideTable", "cols":["f1:a|string", "f1:b|boolean"], "startTs": "20190101000000", "endTs": "20200101000000", "startKey": "a", "endKey": "z"}Export to a differently named MaxCompute table
By default, the MaxCompute table name matches the HBase table name, with hyphens (-) and periods (.) replaced by underscores (_). To use a different name, specify odpsTable explicitly:
hbaseTable/odpsTable {"tableMode": "KVTable"}Replace hbaseTable with the source HBase table name and odpsTable with the target MaxCompute table name.