Principles and best practices for rowkey design - ApsaraDB for HBase

Poor rowkey design is a common cause of hotspotting in ApsaraDB for HBase. Hotspotting concentrates reads and writes on a single region, overloading its RegionServer and degrading performance across the entire node — in severe cases, the region becomes unavailable. This page describes the techniques for distributing load evenly across regions, along with the trade-offs of each approach.

How hotspotting occurs

HBase sorts rows in lexicographic order by rowkey. This ordering makes range scans efficient and keeps related rows physically adjacent. The downside is that sequential or time-based rowkeys cause all writes to land on the same region until that region splits.

When data is written to an ApsaraDB for HBase cluster, the write process is locked during each operation. All clients must wait for the region to become available before proceeding, after which the cycle starts again. This locking behavior is especially problematic with monotonically increasing rowkeys, because it concentrates all writes on a single region.

When a single region absorbs a disproportionate share of traffic, its RegionServer becomes a bottleneck. Other regions on the same RegionServer are also affected, because the server cannot keep up with the overall load.

Choose a distribution strategy

Before designing a rowkey, answer three questions about your workload:

Data size: How much data is written per second, and how large is each row?
Data shape: What query patterns does the application need to support — point lookups, range scans, or both?
Data velocity: Are writes sequential (time series, auto-increment IDs) or naturally scattered?

The answers determine which distribution strategy fits your use case.

Strategy	Write distribution	Read predictability	Use when
Salting	Excellent — random prefix spreads load evenly	Low — must query all salt buckets and merge results	Write throughput is the priority; range scans are rare
Hashing	Good — deterministic prefix distributes load	High — recalculate the prefix on the client to do a point Get	Both write distribution and targeted reads are needed
Key reversal	Good — randomizes monotonic rowkeys	Medium — row order is lost; range scans are less useful	Rowkeys are fixed-length or numeric and you need minimal implementation complexity

Salting

Salting prepends a random prefix to each rowkey. The number of possible prefix values matches the number of regions across which you want to spread data.

Without salting, rowkeys that share the same prefix all land in one region:

foo0001
foo0002
foo0003
foo0004

With four salt values (a, b, c, d), the same rows are distributed across four regions simultaneously, quadrupling write throughput:

a-foo0003
b-foo0001
c-foo0004
d-foo0002

As more rows arrive and share salt values, data continues to spread:

a-foo0003
b-foo0001
c-foo0003
c-foo0004
d-foo0002

Trade-off: Salting improves write throughput but breaks row order. To read all rows with a given original key, query each salt bucket independently and merge the results on the client — an operation that scales linearly with the number of salt buckets.

Hashing

Hashing replaces the random prefix with a one-way hash of the rowkey. Because the same input always produces the same hash, the prefix is deterministic.

For example, applying a one-way hash to foo0003 consistently yields prefix a. On the client, recalculate the hash to reconstruct the full rowkey, then issue a single Get — no scatter-gather required.

Hashing also makes it possible to co-locate specific rowkey pairs in the same region by choosing a hash function that maps them to the same prefix.

Trade-off: Hashing provides predictable reads while still distributing writes. The client must implement the same hash function used at write time.

Key reversal

Reversing a fixed-length or numeric rowkey moves the most frequently changing part (the least significant digit) to the front. This naturally randomizes rowkeys without requiring any prefix calculation.

Trade-off: Simple to implement — no client-side hash function required. Row order is lost, which limits range scan efficiency.

Avoid monotonically increasing rowkeys

Timestamps, auto-increment sequences (1, 2, 3…), and other monotonically increasing values concentrate all writes on a single region. Avoid using them directly as rowkeys.

Handle time-series data with composite rowkeys

If your application ingests time-ordered data such as logs or metrics, use a composite rowkey that places a high-cardinality field before the timestamp. OpenTSDB demonstrates this pattern: its rowkey format is [metric_type][event_timestamp]. With hundreds of distinct metric_type values, Put operations spread across many regions even though the data stream is continuous.

Minimize row and column sizes

HBase stores values as cells. Locating a cell requires three coordinates: the rowkey, column qualifier, and timestamp. These coordinates appear in every StoreFile index entry. When coordinates are large, the index consumes significantly more memory and can exhaust it.

To keep index size manageable:

Use the shortest rowkey that still supports your access pattern.
Keep column family names to a single character (for example, f).
Use short column qualifiers (for example, via instead of myVeryImportantAttribute).
Enable compression if coordinate size cannot be reduced further.

In small datasets, oversized keys have little impact. At billions of rows, the difference is significant — column families and column qualifiers are repeated on every cell.

Use binary encoding to reduce rowkey size

Storing numbers as binary rather than strings reduces their size substantially. A long value requires 8 bytes; the same value stored as a string requires roughly 3x as many bytes. The following code illustrates the difference:

// long stored as binary: 8 bytes
long l = 1234567890L;
byte[] lb = Bytes.toBytes(l);
System.out.println("long bytes length: " + lb.length);   // 8

// long stored as string: 10 bytes
String s = String.valueOf(l);
byte[] sb = Bytes.toBytes(s);
System.out.println("long as string length: " + sb.length);    // 10

// MD5 digest as binary: 16 bytes
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] digest = md.digest(Bytes.toBytes(s));
System.out.println("md5 digest bytes length: " + digest.length);    // 16

// MD5 digest as string: 26 bytes
String sDigest = new String(digest);
byte[] sbDigest = Bytes.toBytes(sDigest);
System.out.println("md5 digest as string length: " + sbDigest.length);    // 26

Binary rowkeys are compact but hard to read in tools like the HBase shell, which displays binary values as hex escape sequences:

hbase(main):001:0> incr 't', 'r', 'f:q', 1
COUNTER VALUE = 1

hbase(main):002:0> get 't', 'r'
COLUMN                                        CELL
 f:q                                          timestamp=1369163040570, value=\x00\x00\x00\x00\x00\x00\x00\x01
1 row(s) in 0.0310 seconds

Use binary encoding when storage efficiency matters and human-readable rowkeys in operational tooling are not required.

Use reverse timestamps to fetch the latest version

To efficiently retrieve the most recent version of a row, append Long.MAX_VALUE - timestamp to the rowkey:

[key][Long.MAX_VALUE - timestamp]

Because HBase sorts rowkeys in lexicographic order, the smallest reverse timestamp (corresponding to the largest actual timestamp) sorts first. Scanning from [key] returns the latest version as the first result.

This technique also lets you retrieve older versions using the same Scan pattern, and serves as an alternative to HBase version numbers for long-term version retention.

Rowkeys and column families

A rowkey is scoped to a column family. The same rowkey can exist in each column family of the same table without conflict.

Rowkeys cannot be changed

Once a rowkey is written, it cannot be updated in place. To change a rowkey, delete the existing row and insert a new one. Design your rowkey schema before loading data at scale — retrofitting a rowkey design across billions of rows is expensive.

Pre-split tables for known keyspaces

Pre-splitting a table allocates regions and their keyspace boundaries before any data is written. Without pre-splitting, all data initially lands in a single region until HBase automatically splits it — a process that temporarily creates a hotspot.

When designing split points, make sure each region covers a portion of the keyspace that will actually receive data. A common mistake is using default split logic against a keyspace that only uses a subset of possible values.

Example: Suppose rowkeys are 16-character hex strings in the range 0000000000000000 to ffffffffffffffff. Splitting this range into 10 regions using Bytes.split produces boundaries based on the full byte range (0–255), not the hex character range (0–9, a–f). The hex character set occupies only positions 48–57 and 97–102 in the ASCII table, so the middle regions (covering positions 58–96) never receive any data.

To pre-split correctly for a hex keyspace, calculate split points within the actual character range:

public static boolean createTable(Admin admin, HTableDescriptor table, byte[][] splits)
    throws IOException {
  try {
    admin.createTable(table, splits);
    return true;
  } catch (TableExistsException e) {
    logger.info("table " + table.getNameAsString() + " already exists");
    return false;
  }
}

public static byte[][] getHexSplits(String startKey, String endKey, int numRegions) {
  byte[][] splits = new byte[numRegions - 1][];
  BigInteger lowestKey = new BigInteger(startKey, 16);
  BigInteger highestKey = new BigInteger(endKey, 16);
  BigInteger range = highestKey.subtract(lowestKey);
  BigInteger regionIncrement = range.divide(BigInteger.valueOf(numRegions));
  lowestKey = lowestKey.add(regionIncrement);
  for (int i = 0; i < numRegions - 1; i++) {
    BigInteger key = lowestKey.add(regionIncrement.multiply(BigInteger.valueOf(i)));
    byte[] b = String.format("%016x", key).getBytes();
    splits[i] = b;
  }
  return splits;
}

The same principle applies to any keyspace, not just hex strings. Define split points that align with where your data actually falls in the keyspace.