When Spark jobs in a workspace need to access services inside a VPC — such as a Hive Metastore or a self-managed database — those services are typically reachable only by IP address. Domain name management lets you create custom IP-to-domain name mappings at the workspace level so that jobs resolve private hostnames correctly, without modifying external DNS.
Limits
| Limit | Value |
|---|---|
| Supported IP version | IPv4 only |
| Workspace state required to add mappings | Running |
| Maximum mappings per workspace | 200 |
After you add, modify, or delete a mapping, restart the affected jobs for the change to take effect. Jobs that are not restarted continue to use the previous mapping and may encounter service access errors or business interruptions.
Prerequisites
Before you begin, make sure that:
The target workspace is in the Running state (required only when adding a mapping)
You have valid IP addresses and domain names for the services you want to map
Add a domain name
Log on to the E-MapReduce console.
In the left navigation pane, click EMR Serverless > Spark.
In the Actions column for the target workspace, click
> Manage Domain Names.In the Manage Domain Names dialog box, enter a valid IP address and its corresponding domain name.
Click OK.
Restart the affected jobs for the new mapping to take effect.
Modify a domain name
Log on to the E-MapReduce console.
In the left navigation pane, click EMR Serverless > Spark.
In the Actions column for the target workspace, click
> Manage Domain Names.In the Manage Domain Names dialog box, update the IP address or domain name. Make sure both values are valid.
Click OK.
Restart the affected jobs for the updated mapping to take effect.
Delete a domain name
Log on to the E-MapReduce console.
In the left navigation pane, click EMR Serverless > Spark.
In the Actions column for the target workspace, click
> Manage Domain Names.In the Manage Domain Names dialog box, remove the IP address-to-domain name mapping you no longer need.
Click OK.
Restart the affected jobs for the deletion to take effect.
Example: connect to Hive Metastore in a VPC
The following example shows how to use a custom domain name to connect a Spark job to a Hive Metastore service running inside a VPC.
Add a domain name mapping for the Hive Metastore instance. In the Manage Domain Names dialog box, enter the IP address of your Hive Metastore instance and its domain name.
Register the Hive Metastore as an external catalog.
On the Catalog page, click Add Catalog.
In the Add Catalog dialog box, click External Hive Metastore, fill in the required fields, and click OK.
For more information, see Connect to an external Hive Metastore service.
After completing these steps, the catalog is available for use in Data Development.
What's next
Use the catalog you created in Data Development to run Spark jobs against the Hive Metastore.