Lindorm:Separately store hot data and cold data based on timestamps

Use Apache HBase Shell

Step 1: Create a table with a boundary

HBase(main):002:0> create 'chsTable', {NAME=>'f', COLD_BOUNDARY=>'86400'}

Step 2 (optional): Change the boundary

HBase(main):005:0> alter 'chsTable', {NAME=>'f', COLD_BOUNDARY=>'42300'}

Step 3 (optional): Disable hot and cold data separation

HBase(main):004:0> alter 'chsTable', {NAME=>'f', COLD_BOUNDARY=>""}

Use ApsaraDB for HBase API for Java

Step 1: Create a table with a boundary

Admin admin = connection.getAdmin();
TableName tableName = TableName.valueOf("chsTable");
HTableDescriptor descriptor = new HTableDescriptor(tableName);
HColumnDescriptor cf = new HColumnDescriptor("f");
cf.setValue(AliHBaseConstants.COLD_BOUNDARY, "86400");
descriptor.addFamily(cf);
admin.createTable(descriptor);

Step 2 (optional): Change the boundary

HTableDescriptor descriptor = admin
    .getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
cf.setValue(AliHBaseConstants.COLD_BOUNDARY, "86400");
admin.modifyTable(tableName, descriptor);

Step 3 (optional): Disable hot and cold data separation

HTableDescriptor descriptor = admin
    .getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
cf.setValue(AliHBaseConstants.COLD_BOUNDARY, null);
admin.modifyTable(tableName, descriptor);

Use lindorm-cli (Lindorm SQL)

Step 1: Create a table with a boundary

Use the CREATE TABLE statement:

CREATE TABLE dt (
  p1 integer, p2 integer, c1 varchar, c2 bigint,
  constraint pk primary key(p1 desc))
WITH (COMPRESSION = 'ZSTD', CHS = '86400', CHS_L2 = 'storagetype=COLD');

Or enable hot and cold data separation on an existing table using ALTER TABLE:

-- Enable hot and cold data separation on an existing table.
-- The table was created without CHS:
-- CREATE TABLE dt (p1 integer, p2 integer, c1 varchar, c2 bigint,
--   constraint pk primary key(p1 desc)) WITH (COMPRESSION = 'ZSTD');

ALTER TABLE dt SET 'CHS' ='86400', 'CHS_L2' = 'storagetype=COLD';

Step 2 (optional): Change the boundary

ALTER TABLE dt SET 'CHS'='1000';

Step 3 (optional): Disable hot and cold data separation

ALTER TABLE dt SET 'CHS'='', 'CHS_L2' = '';

GET queries

Apache HBase Shell

-- Query all data (may include cold storage)
HBase(main):013:0> get 'chsTable', 'row1'

-- Query only hot storage
HBase(main):015:0> get 'chsTable', 'row1', {HOT_ONLY=>true}

-- Query by time range (UNIX timestamp in milliseconds)
HBase(main):016:0> get 'chsTable', 'row1', {TIMERANGE => [0, 1568203111265]}

The TimeRange parameter accepts UNIX timestamps in milliseconds since January 1, 1970, 00:00:00 UTC. Lindorm compares TimeRange against COLD_BOUNDARY to determine which storage tiers to read.

ApsaraDB for HBase API for Java

// Query all data (may include cold storage)
Get get = new Get("row1".getBytes());
System.out.println("result: " + table.get(get));

// Query only hot storage
get = new Get("row1".getBytes());
get.setAttribute(AliHBaseConstants.HOT_ONLY, Bytes.toBytes(true));

// Query by time range
get = new Get("row1".getBytes());
get.setTimeRange(0, 1568203111265);

Lindorm SQL

SELECT /*+ _l_hot_only_ */ * FROM dt WHERE pk IN (1, 2, 3);

SCAN queries

Apache HBase Shell

-- Scan all data (may include cold storage)
Lindorm(main):017:0> scan 'chsTable', {STARTROW =>'row1', STOPROW=>'row9'}

-- Scan only hot storage
Lindorm(main):018:0> scan 'chsTable', {STARTROW =>'row1', STOPROW=>'row9', HOT_ONLY=>true}

-- Scan by time range
Lindorm(main):019:0> scan 'chsTable', {STARTROW =>'row1', STOPROW=>'row9', TIMERANGE => [0, 1568203111265]}

ApsaraDB for HBase API for Java

TableName tableName = TableName.valueOf("chsTable");
Table table = connection.getTable(tableName);

// Scan all data (may include cold storage)
Scan scan = new Scan();
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner) {
    System.out.println("scan result:" + result);
}

// Scan only hot storage
scan = new Scan();
scan.setAttribute(AliLindormConstants.HOT_ONLY, Bytes.toBytes(true));

// Scan by time range
scan = new Scan();
scan.setTimeRange(0, 1568203111265);

Usage notes

• Row key order is not guaranteed. When COLD_HOT_MERGE is enabled, hot rows are returned before cold rows. The result set contains hot rows and cold rows sorted separately by row key — the overall result is not globally sorted by row key.

  // Normal scan (lexicographic order)
  HBase(main):001:0> scan 'chsTable'
  ROW                 COLUMN+CELL
   coldRow            column=f:value, timestamp=1560578400000, value=cold_value
   hotRow             column=f:value, timestamp=1565848800000, value=hot_value
  2 row(s)
  
  // With COLD_HOT_MERGE=true (hot rows first)
  HBase(main):002:0> scan 'chsTable', {COLD_HOT_MERGE=>true}
  ROW                 COLUMN+CELL
   hotRow             column=f:value, timestamp=1565848800000, value=hot_value
   coldRow            column=f:value, timestamp=1560578400000, value=cold_value
  2 row(s)

  To maintain a meaningful order in your application, design row keys that encode ordering information — for example, a composite key of customer ID and order creation time.

• Rows with updated fields appear twice. If a row in cold storage has been partially updated, the row spans both hot and cold storage. With COLD_HOT_MERGE enabled, results include two entries for the same row key — one from hot storage, one from cold storage.

Custom timestamp behavior

The timestamp you set on a row determines which storage tier it lands in — not the time of the write. Examples:

• A row written with a timestamp three days in the future is archived to cold storage three days later than a row written with the current timestamp.

• A row written with a timestamp three days in the past, when the boundary is set to three days, is archived to cold storage asynchronously shortly after the write — not after three more days.

Use custom timestamps carefully. Rows written with past timestamps may move to cold storage immediately after the next compaction, which can be unexpected if your query patterns assume recently written data is always in hot storage.

Frequently accessed cold data

If queries consistently hit cold storage, check whether the COLD_BOUNDARY (or CHS) value matches your actual access patterns. Cold storage throughput is throttled by the cold storage specification, so frequently accessed cold data degrades query performance.