Configure and enable LDAP authentication for Kyuubi Gateway - E-MapReduce

You can enhance the security of Kyuubi by enabling LDAP authentication. LDAP ensures that only clients with the correct username and password can connect to the Kyuubi Server to start a Spark Session and execute SQL queries. This prevents unauthorized access to sensitive data and features.

Prerequisites

You have created a Kyuubi Gateway and Token. For more information, see Manage Kyuubi Gateway.
We recommend following engine versions:
- esr-4.x: esr-4.3.0 and later versions.
- esr-3.x: esr-3.3.0 and later versions.
- esr-2.x: esr-2.7.0 and later versions.
Optional: If you need to use the OpenLDAP service of an EMR on ECS cluster, firstly create a cluster with the OpenLDAP service and add users. For more information, see Create a cluster and OpenLDAP user management.

Procedure

Step 1: Check the network

Before starting the configuration, you need to check network connectivity between Serverless Spark and your Virtual Private Cloud (VPC), so that the Kyuubi Gateway can connect to the LDAP service for authentication. For more information, see Network connectivity between EMR Serverless Spark and other VPCs.

Step 2: Configure Kyuubi Gateway

For opening LDAP authentication for Kyuubi Gateway, you need to stop the session first. Then edit the Kyuubi Gateway, select the connection name from the Network Connectivity dropdown list, and add the following configuration items to Kyuubi Configuration. Restart the Kyuubi Gateway to ensure that the changes take effect.

kyuubi.authentication                       LDAP
kyuubi.authentication.ldap.baseDN           <ldap_base_dn>
kyuubi.authentication.ldap.userDNPattern    <ldap_user_pattern>
kyuubi.authentication.ldap.binddn           <ldap_bind_dn>
kyuubi.authentication.ldap.bindpw           <ldap_bind_pwd>
kyuubi.authentication.ldap.url              ldap://<ldap_url>:<ldap_port>/

The following describes the parameters. Replace them based on your situation.

Note

If you are connecting to the OpenLDAP service of an EMR on ECS cluster, you can refer to the example for filling in.

Parameter	Description	Example
`kyuubi.authentication`	Fixed as `LDAP`. LDAP authentication is enabled.	`LDAP`
`kyuubi.authentication.ldap.baseDN`	The base DN for LDAP service authentication.	`o=emr`
`kyuubi.authentication.ldap.userDNPattern`	The pattern for converting login users to LDAP DN.	`uid=%s,ou=people,o=emr` When user "test" logs in, it will be converted to DN = `uid=test,ou=people,o=emr` for query verification in the LDAP service.
`kyuubi.authentication.ldap.binddn`	DN bound with LDAP connection service.	`uid=admin,o=emr`
`kyuubi.authentication.ldap.bindpw`	Password of DN bound with LDAP connection service.	Find the parameter value of admin_pwd in the OpenLDAP service configuration.
`kyuubi.authentication.ldap.url`	The connection address of the LDAP service, in the format of `ldap://<ldap_url>:<ldap_port>`.	For `<ldap_url>`, enter the internal IP address or domain name of the master node. For `<ldap_port>`, the default value is 10389, for example, `ldap://master-1-1.c-xxxxxxxxxx.cn-hangzhou.emr.aliyuncs.com:10389`.

Step 3: Connect to Kyuubi Gateway with LDAP authentication enabled

This section provides two methods for connecting to the Kyuubi Gateway. Before connecting, replace the following information based on your situation:

<endpoint>: The Endpoint (public) or Endpoint (internal) information from the Overview tab.
If you use the Endpoint(internal), you only have access to resources of Kyuubi Gateway within the same VPC.
<token>: The Token information from the Token Management tab.
<port>: The port number. The port number is 443 when you access the server by using a public endpoint, and 80 when you access the server by using an internal same-region endpoint.
<username> and <password>: Account name and password for the LDAP service. If you are connecting to the OpenLDAP service of an EMR on ECS cluster, these are the username and password on the User Management page of EMR on ECS.

Method 1: Use the kyuubi-beeline command-line tool

Directly specify: If you do not mind displaying the password in the command line, you can directly provide user credentials through command lines.
```
kyuubi-beeline -u 'jdbc:hive2://<endpoint>:<port>/;transportMode=http;httpPath=cliservice/token/<token>' -n <username> -p <password>
```
Interactive input: For security reasons, you can omit the password after -p, and manually enter the password without displaying characters.
```
kyuubi-beeline -u 'jdbc:hive2://<endpoint>:<port>/;transportMode=http;httpPath=cliservice/token/<token>' -n <username> -p
```

Method 2: Use JDBC URL

You can use other application, such as a Java application, or build a complete Java Database Connectivity (JDBC) URL to connect to the Spark Thrift Server. The URL need to be in the following format:

jdbc:hive2://<endpoint>:<port>/;transportMode=http;httpPath=cliservice/token/<token>;user=<username>;password=<password>