Configure a self-managed RDS - E-MapReduce - Alibaba Cloud Documentation Center

Prerequisites

An RDS for MySQL instance is required.

This topic uses an ApsaraDB RDS for MySQL instance as an example. For more information, see Create an ApsaraDB RDS for MySQL instance.

Usage notes

Ensure the EMR cluster that you create can connect to the RDS for MySQL instance.

If you use an ApsaraDB RDS for MySQL instance:

Same VPC:
- The EMR cluster and the RDS for MySQL instance can communicate over the private network by default. However, you must add the IPv4 CIDR block of the EMR cluster's VPC to the whitelist of the RDS for MySQL instance.
- This step enables a connection between the EMR cluster and the RDS for MySQL instance.
Different VPCs:
- If the EMR cluster and the RDS for MySQL instance are in different VPCs, you must establish a network connection between them, for example, by using a VPC peering connection. For more information, see Use a VPC peering connection to enable private communication between VPCs.
- After the network connection is established, you must also add the IPv4 CIDR block of the EMR cluster's VPC to the whitelist of the RDS for MySQL instance.

Procedure

Step 1: Prepare the metadatabase

Create a database. For more information, see Create a database.
Create a standard user account and grant it read and write permissions. For more information, see Create an account.

Note
Record the username and password. You will need them in Step 2: Create the cluster.
Obtain the internal endpoint of the database.
1. Add the IPv4 CIDR block of the EMR cluster's VPC to the whitelist of the RDS for MySQL instance. For more information, see Configure an IP whitelist.
2. On the instance details page, click Database Connection in the navigation pane on the left.
3. On the Database Connection page, click the internal endpoint to copy it.
  
  Note
  Record the internal endpoint. You will need it in Step 2: Create the cluster.

Step 2: Create the cluster

On the Software Configuration page of the cluster creation wizard, configure the following parameters. For information about other parameters, see Create a cluster.

DataLake and Custom parameters	Hadoop parameters	Description
Metadata		Select Self-managed RDS. Note The Metadata parameter is displayed only after you select the HDFS (OSS-HDFS), YARN, and Hive services for a DataLake or Custom cluster.
javax.jdo.option.ConnectionURL	RDS Endpoint	Enter the connection URL in the format `jdbc:mysql://rm-xxxxxx.mysql.rds.aliyuncs.com/<Database name>`. `rm-xxxxxx.mysql.rds.aliyuncs.com`: The internal endpoint from Step 1: Prepare the metadatabase. `<Database name>`: The name of the database from Step 1: Prepare the metadatabase.
javax.jdo.option.ConnectionUserName	RDS Username	Enter the username of the account from Step 1: Prepare the metadatabase.
javax.jdo.option.ConnectionPassword	RDS Password	Enter the password of the account from Step 1: Prepare the metadatabase.

(Optional) Step 3: Initialize the metastore

Important

You must initialize the metastore if you created a Hadoop cluster of an early version (EMR V3.38.x or earlier, EMR V4.9.x or earlier, or EMR V5.4.x or earlier), or if you want to switch the metastore of an existing cluster to an RDS for MySQL instance.

Before initialization, the Hive Metastore, HiveServer2, and Spark ThriftServer services may not function correctly. These services are restored automatically when the initialization is complete.
DataLake and Custom clusters automatically initialize the Hive metadatabase during creation using the provided database connection parameters. Therefore, you can skip this step.

Log on to the master node of the cluster using SSH. For more information, see Log on to a cluster.
Run the following command to switch to the hadoop user.
```
su - hadoop
```
Run the following command to initialize the metastore.
```
schematool -initSchema -dbType mysql
```
The RDS for MySQL instance is now configured as the Hive metadatabase.