E-MapReduce:Knox - E-MapReduce

Apache Knox is a gateway for securely accessing the web UIs of Hadoop ecosystem services in your E-MapReduce (EMR) cluster — including Hadoop Distributed File System (HDFS), YARN, Apache Spark, and Ganglia — over the internet. This topic explains how to open the Knox port, create a Knox user account, and access service web UIs.

Prerequisites

Before you begin, ensure that you have:

An EMR cluster with Knox installed. If Knox is not installed, add it before proceeding. See Add services.

For instructions on creating a cluster, see Create a cluster.

Step 1: Open Knox port 8443

Important

Opening a port to the internet creates a potential attack surface. Restrict access to your own IP address only — do not set Authorization Object to 0.0.0.0/0.

Find your current public IP address by visiting http://myip.ipip.net/.
On the Basic Information tab of your cluster, go to the Security section and click the link next to Cluster Security Group.
On the Security Group Details tab, click Add Rule on the Inbound sub-tab.
Set Authorization Object to your public IP address and Port Range to 8443/8443.
Click Save in the Actions column.

Note

If you did not enable Assign Public IP Address when creating the cluster, the master node has no public IP. Assign a public IP to the master node from the ECS console before proceeding.

Step 2: Create a Knox user account

Knox uses Lightweight Directory Access Protocol (LDAP) for authentication, backed by Apache Directory Server running inside the cluster. Choose one of the following methods to create a user account.

Method 1 — EMR console (recommended for most users)

On the EMR on ECS page, click the name of your cluster.
Click the Users tab, then click Add User.

For details on managing user accounts, see Manage user accounts.

Method 2 — SSH and LDAP import

Log on to the cluster over SSH. See Log on to a cluster.
Switch to the knox user and open the LDIF template:
```
su knox
vim /opt/apps/KNOX/knox-current/templates/users.ldif
```
In the file, replace emr-guest and EMR GUEST with your username (for example, Tom), and set userPassword to the desired password.

Import the user into LDAP:

cd /opt/apps/KNOX/knox-current/templates
sh ldap-sample-users.sh

Step 3: Access service web UIs

After completing the above steps, access the web UIs of open source components through the EMR console. See Access the web UIs of open source components in the EMR console.

Troubleshooting

Knox fails to start with "Gateway SSL Certificate is Expired"

If Knox stops unexpectedly and you see this error when starting it:

Failed to start gateway: org.apache.hadoop.gateway.services.ServiceLifecycleException: Gateway SSL Certificate is Expired

The SSL certificate in the keystore has expired. To resolve this:

Log on to the cluster over SSH. See Log on to a cluster.

Move the expired keystore file:

Note

You can also move the file to a different directory.

sudo mv /opt/apps/KNOX/knox-current/data/security/keystores/gateway.jks /opt/apps/KNOX/knox-current/data/security/keystores/bak_gateway.jks

Restart Knox from the console:
1. Go to the Services tab of your cluster, find Knox, and click the icon, then select Start.
2. Enter an Execution Reason and click OK.
3. In the Confirm dialog, click OK.