E-MapReduce:Modify the memory parameters of the Hive service

Prerequisites

Before you begin, ensure that you have:

• An EMR cluster with the Hive service deployed

• Sufficient available memory on the host nodes to support the new heap size

If you selected DLF Unified Metadata for the Metadata parameter when creating the cluster, HiveMetaStore is not enabled. This procedure does not apply to HiveMetaStore in that case. For more information about metadatabase types, see Hive metadata.

When to increase memory

Increase the heap size of HiveMetaStore when all of the following symptoms occur at the same time:

• Queries or jobs take longer than usual due to metadata-related operations

• HiveMetaStore throws exceptions

• Garbage collection (GC) metrics for HiveMetaStore exceed their thresholds, indicating memory exhaustion

If the same symptoms occur on HiveServer2, increase the heap size of HiveServer2 instead.

Increase the heap size of HiveMetaStore

1. Go to the Configure tab of the Hive service page.

2. In the search box, enter hive_metastore_heapsize and click the search icon.

   

3. Set the parameter value based on the available memory of your nodes. The default value is 500 MiB. For example, you can change the value to 2048 (2 GiB).

4. Click Save. In the dialog box that appears, enter a value for Execution Reason, and then click Save.

5. On the Status tab, restart HiveMetaStore and verify that the service returns to a running state.

Increase the heap size of HiveServer2

1. Go to the Configure tab of the Hive service page.

2. In the search box, enter hive_server2_heapsize and click the search icon.

3. Set the parameter value based on the available memory of your nodes. For example, you can change the value to 2048 (2 GiB).

4. Click Save. In the dialog box that appears, enter a value for Execution Reason, and then click Save.

5. On the Status tab, restart HiveServer2 and verify that the service returns to a running state.