ApsaraDB for HBase:Cold storage

When to use cold storage

Cold storage is suitable for:

• Data archiving: Retain historical records at low cost without deleting them.

• Infrequent data access: Store data that is queried occasionally but must remain accessible.

Cold storage is not suitable for:

• High-concurrency reads: Processing a large number of concurrent read requests may cause errors.

• Latency-sensitive reads: Cold storage has a low read Input/Output Operations Per Second (IOPS) — up to 25 per second per node — and works only for infrequent, non-time-critical queries.

Write throughput of cold storage equals that of hot storage backed by ultra disks, so write-heavy workloads are not affected.

To increase the read IOPS for large cold storage capacity, submit a ticket.

Prerequisites

Before you begin, make sure you have:

• A cluster running ApsaraDB for HBase Performance-enhanced Edition V2.1.8 or later. If the cluster version is earlier than V2.1.8, it is automatically upgraded to the latest version when you activate cold storage.

• AliHBase-Connector version later than V1.0.7 (for the 1.x line) or later than V2.0.7 (for the 2.x line)

• HBase Shell version later than alihbase-2.0.7-bin.tar.gz

Enable cold storage

Choose one of the following methods to enable cold storage.

Method 1: Enable when purchasing a cluster

Select the cold storage option and configure the capacity on the cluster purchase page. For more information, see Purchase a cluster.

Method 2: Enable from the console

1. Log on to the ApsaraDB for HBase console.

2. On the Clusters page, click the ID of the target cluster.

3. In the left-side navigation pane, click Cold Storage.

4. Click Activate Now.

Verify that cold storage is enabled

After activation, click Cold Storage Scaling on the Cold Storage page to confirm the storage status and current capacity.

Configure tables to use cold storage

Cold storage is configured at the column family level. Set the storage policy on a column family to route all data in that family to cold storage. The data does not occupy Hadoop Distributed File System (HDFS) space on the cluster.

You can use HBase Shell or the Java API. Before using the Java API, set up the software development kit (SDK) — see Use the ApsaraDB for HBase Java API to access an ApsaraDB for HBase Performance-enhanced Edition instance. Before using HBase Shell, see Use HBase Shell to access an ApsaraDB for HBase Performance-enhanced Edition instance.

In the examples below, replace coldTable with your table name and f with your column family name.

hbase(main):001:0> create 'coldTable', {NAME => 'f', STORAGE_POLICY => 'COLD'}

Admin admin = connection.getAdmin();
HTableDescriptor descriptor = new HTableDescriptor(TableName.valueOf("coldTable"));
HColumnDescriptor cf = new HColumnDescriptor("f");
// Route this column family to cold storage
cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_COLD);
descriptor.addFamily(cf);
admin.createTable(descriptor);

hbase(main):011:0> alter 'coldTable', {NAME=>'f', STORAGE_POLICY => 'COLD'}

Admin admin = connection.getAdmin();
TableName tableName = TableName.valueOf("coldTable");
HTableDescriptor descriptor = admin.getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
// Set the storage type of this column family to cold storage
cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_COLD);
admin.modifyTable(tableName, descriptor);

hbase(main):014:0> alter 'coldTable', {NAME=>'f', STORAGE_POLICY => 'DEFAULT'}

Admin admin = connection.getAdmin();
TableName tableName = TableName.valueOf("coldTable");
HTableDescriptor descriptor = admin.getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
// Revert to the default (hot) storage type
cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_DEFAULT);
admin.modifyTable(tableName, descriptor);

Monitor cold storage capacity

On the Cold Storage page in the ApsaraDB for HBase console, click Cold Storage Scaling to view the current usage and expand capacity.

To view the sizes of cold and hot data per table, go to the User tables tab in the cluster management system.

Limits

What's next

To automatically separate hot and cold data within the same table, use the hot/cold data separation feature in ApsaraDB for HBase Performance-enhanced Edition.