ApsaraDB for HBase:Use cold storage

Background

You can select cold storage as an additional storage option when you purchase an ApsaraDB for HBase cluster. You can use CREATE TABLE statements to direct cold data to the cold storage medium. ApsaraDB for HBase Performance-enhanced Edition also provides a data tiering feature that uses cold storage. This feature automatically places frequently accessed hot data in high-speed hot storage and moves infrequently accessed cold data to cold storage, reducing your storage costs.

Usage notes

• Cold storage has low read IOPS (Input/Output Operations Per Second), with a maximum of 25 per node. This makes it suitable only for scenarios with infrequent queries.

• The write throughput of cold storage is comparable to that of hot storage on Ultra Disk. You can write data with confidence.

• Cold storage is not suitable for high-concurrency read requests, as this can lead to request failures.

• If you have a large cold storage capacity and require a higher read IOPS limit, you can request an adjustment by submitting a ticket.

• We recommend that each core node manage no more than 30 TB of cold data on average. If you need a single core node to manage a larger volume of cold data, you can submit a ticket to request optimization suggestions.

Prerequisites

Cold storage is supported only in ApsaraDB for HBase Performance-enhanced Edition V2.1.8 or later. If your cluster runs an earlier version, the cluster is automatically upgraded to the latest version when you activate the feature. Ensure that you use AliHBase-Connector V1.0.7/V2.0.7 or later and HBase Shell alihbase-2.0.7-bin.tar.gz or later.

Use cases

Cold storage is ideal for scenarios such as data archiving and storing infrequently accessed historical data.

Enable cold storage

Method 1: When you create an ApsaraDB for HBase Performance-enhanced Edition cluster, you can purchase cold storage and specify its capacity on the purchase page. For more information, see Purchase a cluster.

Use cold storage

ApsaraDB for HBase Performance-enhanced Edition allows you to set storage properties at the column family level. You can set the storage policy for one or all column families of a table to cold storage. After the policy is set, all data in the specified column families is stored in cold storage and does not use the Hadoop Distributed File System (HDFS) space of your cluster. You can configure this setting when you create a table or by modifying the properties of an existing table.

You can create tables and modify table properties using the Java API or HBase Shell. For setup instructions, see Use the Java API to access an ApsaraDB for HBase Performance-enhanced Edition cluster for the Java API and Use HBase Shell to access an ApsaraDB for HBase Performance-enhanced Edition cluster for HBase Shell.

Create a table with cold storage

hbase(main):001:0> create 'coldTable', {NAME => 'f', STORAGE_POLICY => 'COLD'}

 Admin admin = connection.getAdmin();
 HTableDescriptor descriptor = new HTableDescriptor(TableName.valueOf("coldTable"));
 HColumnDescriptor cf = new HColumnDescriptor("f");
 cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_COLD);
 descriptor.addFamily(cf);
 admin.createTable(descriptor);

Modify table properties to use cold storage

If a table already exists, you can modify the properties of its column families to use cold storage. If a column family already contains data, a major compaction must run to move that data to cold storage.

hbase(main):011:0> alter 'coldTable', {NAME=>'f', STORAGE_POLICY => 'COLD'}

Admin admin = connection.getAdmin();
TableName tableName = TableName.valueOf("coldTable");
HTableDescriptor descriptor = admin.getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
// Set the storage policy of the column family to cold.
cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_COLD);
admin.modifyTable(tableName, descriptor);

Change table properties to use hot storage

If a column family is currently using cold storage, you can change its storage type back to hot storage by modifying its properties. If the column family contains data, a major compaction must run to move the data back to hot storage.

hbase(main):014:0> alter 'coldTable', {NAME=>'f', STORAGE_POLICY => 'DEFAULT'}

// Create a connection.
Admin admin = connection.getAdmin();
TableName tableName = TableName.valueOf("coldTable");
HTableDescriptor descriptor = admin.getTableDescriptor(tableName);
HColumnDescriptor cf = descriptor.getFamily("f".getBytes());
// Set the storage policy to the default value, which is hot storage.
cf.setValue("STORAGE_POLICY", AliHBaseConstants.STORAGETYPE_DEFAULT);
admin.modifyTable(tableName, descriptor);

View cold storage usage

On the Cold Storage page of the ApsaraDB for HBase console, you can view the overall cold storage usage and click Cold Storage Scaling to increase its capacity.

On the Tables tab of the Cluster Management System, you can view the amount of cold and hot storage that a specific table uses.