This topic describes how to use JindoFS as the storage backend of HBase.

Background information

HBase is a real-time database in the Hadoop ecosystem and has high write performance. In E-MapReduce (EMR), JindoFS or Object Storage Service (OSS) can be used as the storage backend of HBase. JindoFS and OSS are more flexible than HDFS.
Note We recommend that you use EMR clusters of V3.36.0 or later.

JindoFS configuration

In this topic, a namespace named emr-jfs is created for an EMR V3.36.0 cluster, and an OSS bucket named oss-bucket is configured for the namespace:
  • jfs.namespaces=emr-jfs
  • jfs.namespaces.emr-jfs.oss.uri=oss://<oss-bucket>/oss-dir
  • jfs.namespaces.emr-jfs.mode=block

Specify a storage path for HBase

To specify a storage path for HBase, you must set the hbase.rootdir parameter in the hbase-site configuration file to a JindoFS or OSS path and set the hbase.wal.dir parameter in the file to a local HDFS path. This way, the HDFS service of your EMR cluster is used to store write-ahead logging (WAL) files. To release the cluster, you must disable the tables in this cluster and make sure that the updates in WAL files are written to HFile.

Parameter Description
hbase.rootdir The root directory of HBase in JindoFS or OSS.
Set this parameter to jfs://emr-jfs/hbase-root-dir.
Note emr-jfs is the namespace that you have configured.
hbase.wal.dir The local HDFS path in which the WAL files of HBase are stored.
Value values:
  • For a high-availability (HA) cluster, set this parameter to hdfs://emr-cluster/hbase.
  • For a non-HA cluster, set this parameter to hdfs://emr-header-1:9000/hbase.

Create a cluster

When you create a cluster, turn on Custom Software Settings and add custom software configurations. For more information about how to create a cluster, see Create a cluster. Smartdata-3-6
For example, to use JindoFS as the storage backend of HBase, add the following custom configurations. You must replace oss_bucket with the actual OSS bucket name and replace the OSS path:
[
  {
       "ServiceName":"SMARTDATA",
       "FileName":"namespace",
       "ConfigKey":"jfs.namespaces",
       "ConfigValue":"emr-jfs"
  },
  {
       "ServiceName":"SMARTDATA",
       "FileName":"namespace",
       "ConfigKey":"jfs.namespaces.emr-jfs.oss.uri",
       "ConfigValue":"oss://oss-bucket/jindoFS"
  },
  {
       "ServiceName":"SMARTDATA",
       "FileName":"namespace",
       "ConfigKey":"jfs.namespaces.emr-jfs.mode",
       "ConfigValue":"block"
  },
  {
       "ServiceName":"HBASE",
       "FileName":"hbase-site",
       "ConfigKey":"hbase.rootdir",
       "ConfigValue":"jfs://emr-jfs/hbase-root-dir"
  },
  {
       "ServiceName":"HBASE",
       "FileName":"hbase-site",
       "ConfigKey":"hbase.wal.dir",
       "ConfigValue":"hdfs://emr-cluster/hbase"
  }
]