E-MapReduce:Use Kyuubi tokens to manage permissions for DLF data in EMR Serverless Spark

Step 1: Prepare the environment and RAM user permissions

1. Prepare basic resources.

   • Create a workspace. For more information, see Manage workspaces.

   • Create and start a Kyuubi Gateway in the workspace. For more information, see Manage Kyuubi Gateways.

   • Create a data catalog in DLF and prepare the required database and tables.

2. Configure a RAM user and permissions. Prepare a RAM user and grant the necessary permissions to it.

   • EMR Serverless Spark permissions: Grant the RAM user the basic operation permissions required to access EMR Serverless Spark. For more information, see Grant permissions to a RAM user.

   • Workspace permissions: Add the RAM user to the workspace and assign a role to the user based on their function. For more information, see Manage users and roles.

Step 2: Grant table permissions to the RAM user in DLF

Grant the RAM user access permissions for specific tables in DLF. To ensure the Kyuubi Gateway can initialize a Spark session, you must also grant the RAM user the Describe permission for the default database in DLF.

1. Log on to the .

2. Navigate to the database and table in the target catalog.

3. Select the table for which you want to grant permissions and click the Permissions tab.

4. Click Grant Permissions.

   1. Principal: select DLF User/DLF Role.

   2. Select DLF User: select the target RAM user.

   3. Permissions: select the required permissions.

5. Click OK. The authorization is complete.

   Note

   EMR Serverless Spark enables caching for DLF metadata and data by default. If you re-grant permissions for a table, the changes take about 10 minutes to take effect.

   If you want the permission changes to take effect immediately, add the spark.sql.catalog.lakehouse.cache-enabled false configuration to the Spark configuration in the Kyuubi Gateway.

Step 3: Generate a Kyuubi token for the RAM user

1. On the Kyuubi Gateway page, find the target gateway, and in the Actions column, click Tokens.

2. Click Create Token. In the dialog box that appears, configure the following parameters and click OK.

   Parameter	Description
   Name	The name of the new token.
   Expired At	Set the expiration time for the token. The value must be 1 or greater. By default, this feature is enabled and the token expires after 365 days.
   Assigned To	From the drop-down list, select the target RAM user.

3. Copy the token information.

   Important

   After the token is created, you must copy the token information immediately. You cannot retrieve it later. If your token expires or is lost, you must create a new one or reset the existing one.

Step 4: Connect using Beeline and verify permissions

1. Construct the JDBC connection command.

   beeline -u "jdbc:hive2://<endpoint>:<port>/;transportMode=http;user=<UserName or RoleName>;httpPath=cliservice/token/<Token>"

   Parameter description:

   Parameter	Description
   <endpoint>	The Endpoint of the Kyuubi Gateway.
   <port>	The access port. The port for a public Endpoint is 443. The port for an internal network Endpoint is 80.
   <UserName or RoleName>	The RAM user or RAM role. You can use the short name or the full name. Examples: RAM user: agent or agent@xxxx05398154xxxx.onaliyun.com RAM role: AliyunServiceRoleForDataworksEngine
   <Token>	The token generated for the RAM user in Step 3.

   

2. Verify the access control effect.

   • Query an authorized table:

     SELECT * FROM <database_name>.<authorized_table_name> LIMIT 10;

     The query data is returned successfully.

   • Query an unauthorized table:

     SELECT * FROM <database_name>.<unauthorized_table_name> LIMIT 10;

     The query fails and a permission-related error is returned, such as emr_test doesn't have privilege SELECT on TABLE.

Q1: After I grant permissions, I can still query an unauthorized table and retrieve results. Have the permissions not taken effect?
This issue can occur for the following reasons:

• DLF metadata cache: The Spark engine caches table schema information, which can cause the authorization check to be bypassed.
  Solution: Add spark.sql.catalog.lakehouse.cache-enabled false to the Spark configuration to disable metadata caching.

• Permission latency: In rare cases, DLF permission synchronization may have a short delay, usually no more than 10 seconds.
  Suggestion: Wait a moment and retry, or confirm that the permissions are correctly granted in the DLF console.

Q2: I cannot view the token after it is created. What should I do if it is lost?
The plaintext token is displayed only once in the dialog box after it is created. The system does not store the original value. If the token is lost or leaked, you must take the following actions immediately:

• On the Token Management page, find the corresponding entry and click Reset Token.

• The old token is automatically invalidated and a new credential is generated.

• Update all client configurations with the new token.

Q3: Why do RAM users need the Describe permission for the default database to connect to a Kyuubi Gateway?
By default, when a Spark session is established, the Kyuubi Gateway attempts to load the default database as the initial context. If the current identity does not have permission to access this database, the session initialization fails and the connection is terminated. This check is mandatory, even if your business tables are located in other databases. Therefore, all RAM users who connect to the gateway must have the DescribeDatabase permission for the default database.