The ports over which you can access the web UIs of open source components are disabled for security purposes. The open source components include Hadoop, Spark, and Flink in an E-MapReduce (EMR) cluster. You can access these web UIs from the EMR console. You can also create an SSH tunnel on your local server and enable port forwarding to access these web UIs. Dynamic port forwarding and local port forwarding are supported. This topic describes how to create an SSH tunnel to access web UIs of open source components.

Prerequisites

  • A cluster is created. For more information, see Create a cluster.
  • Port 22 is enabled for the security group where your cluster resides. You can turn on Remote Logon during the creation of a cluster or manually add security group rules after a cluster is created. For more information, see Add security group rules.
    Note When you add inbound security group rules, you must set Authorization Type to IPv4 CIDR Block and Port Range to 22/22.
  • Your local server is connected to the master node of the cluster. You can turn on Assign Public IP Address during the creation of a cluster to associate an EIP address with your cluster. Alternatively, you can assign a fixed public IP address or an EIP address to the master node of your cluster in the ECS console after a cluster is created. For more information, see Bind an ENI.

Background information

To access the web UIs of open source components from the EMR console, see Access open-source components.

Obtain the public IP address of the master node

  1. Log on to the Alibaba Cloud EMR console.
  2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.
  3. Click the Cluster Management tab.
  4. On the Cluster Management page that appears, find the target cluster and click Details in the Actions column.
  5. In the Instance Info section of the Cluster Overview page, view the public IP address of the master node.
    IP

Obtain the name of the master node

  1. Log on to the Alibaba Cloud EMR console.
  2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.
  3. Click the Cluster Management tab.
  4. On the Cluster Management page, find your cluster and click Details in the Actions column. In the left-side navigation pane of the Cluster Overview page, click Instances.
  5. On the Instances page, view the node name that corresponds to the public IP address of the master node.
    For information about how to obtain the public IP address of the master node, see Obtain the public IP address of the master node.header

Enable dynamic port forwarding

Create an SSH tunnel to allow communication between a port of your local server and the master node of an EMR cluster. Run the local SOCKS proxy server that listens on the port. The port data is forwarded to the master node of the EMR cluster by using the SSH tunnel.

  1. Create an SSH tunnel.
    • Use a private key:
      ssh -i [Path for storing the private key file] -N -D 8157 root@[Public IP address of the master node]
    • Use a username and password:
      ssh -N -D 8157 root@[Public IP address of the master node]
    Parameter description:
    • 8157: Port 8157 is used in this example. You can replace this port number with an unoccupied port number on your local server in actual configuration.
    • -D: Dynamic port forwarding is enabled. Start the SOCKS proxy process to listen on the port.
    • Public IP address of the master node: For information about how to obtain the public IP address of the master node, see Obtain the public IP address of the master node.
  2. Configure the Google Chrome browser.
    Notice Keep your local server running after the tunnel is created. No responses are returned.

    You can use one of the following methods to configure the Google Chrome browser:

    • Use a CLI
      1. Open the CLI and go to the local installation directory of the Google Chrome browser client.

        The default installation directory of Google Chrome depends on the operating system.

        Operating system Google Chrome installation directory
        MacOS /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
        Linux /usr/bin/google-chrome
        Windows C:\Program Files (x86)\Google\Chrome\Application\
      2. Run the following command in the default installation directory of Google Chrome:
        chrome --proxy-server="socks5://localhost:8157" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" --user-data-dir=/tmp/
        Parameter description:
        • /tmp/: For Windows, an example of a file directory is /c:/tmppath/. For Linux and macOS, the format of a file directory is /tmp/.
        • 8157: Port 8157 is used in this example. You can replace this port number with an unoccupied port number on your local server in actual configuration.
      3. Enter http://Name of the master node:Port number in the address bar of the browser and press Enter to access a specific web UI.

        For information about how to obtain the name of the master node, see Obtain the name of the master node.

        For example, enter http://emr-header-1:8088 in the address bar of the browser and press Enter to access the web UI of YARN.

    • Use a Google Chrome extension

      Extensions allow you to easily manage and use proxies in your web browser. You can use an extension to browse web pages and access web UIs at the same time.

      1. Add the Google Chrome extension Proxy SwitchyOmega.
      2. Click this extension and choose Options from the shortcut menu.
      3. On the SwitchyOmega page, click New profile in the left-side navigation pane. In the New Profile dialog box, specify Profile name, for example, SSH tunnel, and select PAC Profile.
      4. Enter the following content in the PAC Script editor:
        function regExpMatch(url, pattern) {    
          try { return new RegExp(pattern).test(url); } catch(ex) { return false; }    
        }
        
        function FindProxyForURL(url, host) {
            // Important: replace 172.31 below with the proper prefix for your VPC subnet
        
            if (shExpMatch(url, "*localhost*")) return "SOCKS5 localhost:8157";
            if (shExpMatch(url, "*emr-header*")) return "SOCKS5 localhost:8157";
            if (shExpMatch(url, "*emr-worker*")) return "SOCKS5 localhost:8157";
        
            return 'DIRECT';
        }
      5. In the left-side navigation pane, click Apply changes to complete the configurations.
      6. Open Google Chrome. Click the SwitchyOmega extension. Then, select the created SSH tunnel.
      7. Enter http://Name of the master node:Port number in the address bar of the browser and press Enter

        to access a specific web UI. For information about port configurations of components, see Configure cluster ports. For information about how to obtain the name of the master node, see Obtain the name of the master node.

        For example, enter http://emr-header-1:8088 in the address bar of the browser and press Enter to access the web UI of YARN.

Enable local port forwarding

Notice If you use this method to access a web UI, you cannot go to the job details page.

You can use the local port forwarding method to forward data on a port of the master node to the local port and access the web application interface running on the master node. The SOCKS proxy is not required.

  1. Run the following command on your local server to create an SSH tunnel:
    • Use a private key:
      ssh -i [Path for storing the private key file] -N -L 8157:[Name of the master node]:8088 root@[Public IP address of the master node]
    • Use a username and password:
      ssh -N -L 8157:[Name of the master node]:8088 root@[Public IP address of the master node]
    Parameter description:
    • -L: Local port forwarding is enabled. You can specify a local port to forward data to the remote port that is hosted on the local web server of the master node.
    • 8088: the port that is used to access ResourceManager on the master node. You can replace this port number as required.

      For information about port configurations of components, see Configure cluster ports. For information about how to obtain the name of the master node, see Obtain the name of the master node.

    • 8157: Port 8157 is used in this example. You can replace this port number with an unoccupied port number on your local server in actual configuration.
  2. Keep your local machine running. Open a browser, enter http://localhost:8157/ in the address bar of the browser, and press Enter.