Configure and use HiveServer2 load balancing - E-MapReduce

When an E-MapReduce (EMR) cluster runs multiple HiveServer2 services, you can use ZooKeeper and Server Load Balancer (SLB) to implement load balancing and distribute the load across multiple nodes. This topic describes how to configure and use HiveServer2 load balancing for both standard and Kerberos-enabled clusters.

Limitations

This topic applies only to clusters with High Service Availability enabled.

You must first create a high-availability cluster. For more information, see Create a cluster. The High Service Availability feature must be enabled. After this feature is enabled, E-MapReduce distributes master nodes across different underlying hardware to reduce the risk of failures.

Standard E-MapReduce clusters

The following methods apply to clusters for which Kerberos Authentication is disabled.

ZooKeeper load balancing

ZooKeeper is installed by default on high-availability clusters. You can use the following method to have ZooKeeper select a HiveServer2 instance for your connection, which provides load balancing.

Connect to the cluster over SSH. For more information, see Log on to a cluster.

Run the following command. ZooKeeper selects an available HiveServer2 instance for the connection.

beeline -u 'jdbc:hive2://master-1-1:2181,master-1-2:2181,master-1-3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2'

Note

In the command, master-1-1:2181,master-1-2:2181,master-1-3:2181 is the connection string for ZooKeeper.

To allow Hue to connect to the load-balanced HiveServer2 instance, add the following three custom parameters on the hue tab in the Hue configuration.

Parameter	Description
zookeeper.clusters.default.hostports	The connection string for ZooKeeper. Replace the example with your actual value. Example: master-1-1:2181,master-1-2:2181,master-1-3:2181.
beeswax.hive_discovery_hs2	Set the value to `true`.
beeswax.hive_discovery_hiveserver2_znode	Set the value to `/hiveserver2`.

SLB load balancing

Create a Server Load Balancer (SLB) instance. For more information, see Create and manage a CLB instance.
Add the ECS instances that run the HiveServer2 service to the default server group. Configure the weight of each ECS instance as needed.

For more information, see Manage a default server group.
Configure a TCP listener. You can customize the frontend port. Set Backend Protocol/Port to TCP:10000.

Select a scheduling algorithm based on your requirements. For more information, see Add a TCP listener.

Connect to HiveServer2.

Connect to the cluster over SSH. For more information, see Log on to a cluster.
Run the following command to connect to the HiveServer2 services through the load balancer.
```
beeline -u 'jdbc:hive2://<slb_ip_or_host>:<slb_port>'
```
Replace the following parameters with your actual values:
1. <slb_ip_or_host>: The IP address of the SLB instance or its bound hostname.
2. <slb_port>: The frontend listener port of the SLB instance.

To configure Hue to communicate with HiveServer2 through this load balancer, modify the following parameters on the hue tab in the Hue configuration.

Parameter	Description
hive_server2_host	The IP address of the SLB instance or its bound hostname.
hive_server2_port	The frontend listener port of the SLB instance.

Kerberos-enabled E-MapReduce clusters

The following methods apply to clusters for which Kerberos Authentication is enabled.

ZooKeeper load balancing

Run the kinit command to authenticate with Kerberos and obtain a ticket-granting ticket. For more information, see Use Kerberos authentication.
Connect to HiveServer2.
1. Connect to the cluster over SSH. For more information, see Log on to a cluster.
2. Run the following command. ZooKeeper selects an available HiveServer2 instance for the connection.
```
beeline -u 'jdbc:hive2://master-1-1:2181,master-1-2:2181,master-1-3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2'
```
  Note
  In the command, master-1-1:2181,master-1-2:2181,master-1-3:2181 is the connection string for ZooKeeper.
  
  Hue does not support connecting to a load-balanced HiveServer2 through ZooKeeper on a Kerberos-enabled E-MapReduce cluster. Therefore, use the SLB method for connections from Hue.

SLB load balancing

You can create and configure the SLB instance as described in the Standard E-MapReduce clusters section. On a Kerberos-enabled cluster, you must also configure the Kerberos principal used by HiveServer2 to connect successfully.

This example assumes the SLB IP address is 121.40.. and the frontend listener port is 10000. Replace these with your actual values.

Run the kinit command to authenticate with Kerberos and obtain a ticket-granting ticket. For more information, see Use Kerberos authentication.
Create a Hive principal for the 121.40.**.** address and export the keytab.
1. Log on to the master-1-1 node of the cluster. For more information, see Log on to a cluster.
2. Run the following command to enter the Kerberos admin tool:
```
kadmin.local
```
3. Run the following commands to create the principal and export it to /tmp/slb.keytab:
```
addprinc -randkey hive/121.40..
xst -k /tmp/slb.keytab hive/121.40.**.**
exit
```
  If your E-MapReduce Kerberos cluster is integrated with an external MIT Kerberos instance, use kadmin.local or kadmin to connect to the KDC and run the addprinc and xst commands.
  
  Important
  Exporting a Kerberos keytab multiple times may invalidate previous keytab files. Ensure that slb.keytab contains only the most recently exported principal. If you need to export the principal again, you must delete the previously generated keytab file first.
Transfer the slb.keytab file to all nodes that run the HiveServer2 service. Then, on each of those nodes, use the ktutil tool to merge the content of slb.keytab into the /etc/taihao-apps/hive-conf/keytab/hive.keytab file by running the following commands:
```
ktutil
rkt /tmp/slb.keytab
wkt /etc/taihao-apps/hive-conf/keytab/hive.keytab
quit
```

Run the following command to view the contents of the hive.keytab file:

klist -kt /etc/taihao-apps/hive-conf/keytab/hive.keytab

[root@master-1-1(172.16.xxx) ~]# klist -kt /etc/taihao-apps/hive-conf/keytab/hive.keytab
Keytab name: FILE:/etc/taihao-apps/hive-conf/keytab/hive.keytab
KVNO Timestamp              Principal
---- ------------------- ------------------------------------------------------
   2 10/30/2024 14:12:05 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:12:05 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:12:06 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:12:06 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:12:06 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:12:06 hive/master-1-1.c-7d27ce0e7xxx.cn-hangzhou.emr.aliyuncs.com@EMR.C-7D27CE0E7xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM
   2 10/30/2024 14:26:56 hive/121.40.xxx @EMR.C-7D27CE0E77xxx.COM

Modify the Hive configuration for the cluster.

On the EMR console, go to the Configure tab of the Hive service. Search for the hive.server2.authentication.kerberos.principal parameter and change its value to hive/121.40..@EMR...COM, the principal you created in the previous step. Then, save and apply the configuration.
Restart the HiveServer2 service.
1. On the EMR console, go to the Status tab of the Hive service. In the row for HiveServer2, click Restart in the Actions column.
2. Enter an execution reason and click OK.
3. In the Confirm dialog box, click OK.

On the master-1-1 node, run the following command to connect to HiveServer2:

beeline -u 'jdbc:hive2://121.40.**.**:10000/default;principal=hive/121.40.**.**@EMR.**.COM'