E-MapReduce:Use Kyuubi tokens to control access to Data Lake Formation data in EMR Serverless Spark

Step 1: Prepare environment and permissions

1. Prepare the basic resources.

   • A workspace is created. Manage workspaces.

   • A Kyuubi Gateway is created and started in the workspace. Manage Kyuubi Gateways.

   • A data catalog is created in DLF with the required database and tables.

2. Configure RAM user permissions.

   • EMR Serverless Spark permissions: Grant permissions to a RAM user.

   • Workspace permissions: Add the RAM user to the workspace and assign a role. Manage users and roles.

Step 2: Grant table permissions in DLF

Grant the RAM user access to the target table in DLF. You must also grant the Describe permission on the default database so that Kyuubi Gateway can initialize the Spark session.

1. Log on to the .

2. Navigate to the database and table in the target data catalog.

3. Select the table for which you want to grant permissions and click the Permission tab.

4. Click Add Permission.

   1. Principal: Select DLF User.

   2. Select DLF User: Select the target RAM user.

   3. Permission: Select the required permissions.

5. Click OK.

   Note

   EMR Serverless Spark caches DLF metadata and data by default. Permission changes may take about 10 minutes to take effect.

   To apply permission changes immediately, add spark.sql.catalog.lakehouse.cache-enabled false to the Kyuubi Gateway Spark configuration.

Step 3: Generate a Kyuubi token

1. On the Kyuubi Gateway page, find the target gateway, and in the Actions column, click Token Management.

2. Click Create Token. In the dialog box that appears, configure the following parameters and click OK.

   Parameter	Description
   Name	Name of the new token.
   Expired At	Expiration period in days. Minimum: 1. Default: 365 days.
   Assigned to	Select the target RAM user.

3. Copy the token information.

   Important

   Copy the token immediately — it cannot be viewed again. If the token expires or is lost, create a new one or reset the existing token.

Step 4: Connect with Beeline and verify permissions

1. Construct the JDBC connection command.

   beeline -u "jdbc:hive2://<endpoint>:<port>/;transportMode=http;user=<UserName or RoleName>;httpPath=cliservice/token/<Token>"

   Parameters:

   Parameter	Description
   <endpoint>	Endpoint of the Kyuubi Gateway.
   <port>	443 for public endpoints, 80 for internal endpoints.
   <UserName or RoleName>	RAM user or RAM role. Use either the short name or full name. Examples: RAM user: agent or agent@xxxx05398154xxxx.onaliyun.com RAM role: AliyunServiceRoleForDataworksEngine
   <Token>	The token that you generated for the RAM user in Step 3.

   [root@master-1-1(xxx) ~]# beeline -u "jdbc:hive2://kyuubi-cn-hangzhou-internal.spark.emr.aliyuncs.com:80;transportMode=http;user=emr_test;httpPath=cliservice/token/j2xxx"
       xxx
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in [jar:file:/opt/apps/HIVE/hive-3.1.3-hadoop3.1-1.0.9/lib/log4j-slf4j-impl-2.17.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in [jar:file:/opt/apps/HADOOP-COMMON/hadoop-3.2.1-1.3.5-alinux3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
   Connecting to jdbc:hive2://kyuubi-cn-hangzhou-internal.spark.emr.aliyuncs.com:80;transportMode=http;user=emr_test;httpPath=cliservice/token/j2xxx
   Connected to: Spark SQL (version 3.5.2-emr)
   Driver: Hive JDBC (version 3.1.3)
   Transaction isolation: TRANSACTION_REPEATABLE_READ
   Beeline version 3.1.3 by Apache Hive
   0: jdbc:hive2://kyuubi-cn-hangzhou-internal.s>

2. Verify the access control.

   • Query an authorized table:

     SELECT * FROM <database_name>.<authorized_table_name> LIMIT 10;

     Expected output:

     2025-10-24 16:55:45.008 INFO nioEventLoopGroup-7-1 org.
     nnelInactive!
     +-----+----------+
     | id  |   name   |
     +-----+----------+
     | 1   | Alice    |
     | 2   | Bob      |
     | 3   | Charlie  |
     +-----+----------+
     3 rows selected (5.079 seconds)

   • Query an unauthorized table:

     SELECT * FROM <database_name>.<unauthorized_table_name> LIMIT 10;

     The query fails with a permission error such as emr_test doesn't have privilege SELECT on TABLE. Example error output:

     Caused by: org.apache.paimon.rest.exceptions.ForbiddenException: Forbidden: User acs:ram::xxx:user/emr_test doesn't have privilege SELECT on TABLE default.paimon_tbl
     requestId:xxx
     	at org.apache.paimon.rest.DefaultErrorHandler.accept(DefaultErrorHandler.java:59)
     	at org.apache.paimon.rest.DefaultErrorHandler.accept(DefaultErrorHandler.java:35)
     	at org.apache.paimon.rest.HttpClient.exec(HttpClient.java:156)
     	at org.apache.paimon.rest.HttpClient.get(HttpClient.java:69)
     	at org.apache.paimon.rest.RESTApi.getTable(RESTApi.java:465)

Q1: Why can I still query an unauthorized table after granting permissions?
Possible causes:

• DLF metadata cache: The Spark engine caches table schema, which can bypass permission checks.
  Solution: Add spark.sql.catalog.lakehouse.cache-enabled false to your Spark configuration.
  
  

• Permission latency: DLF permission synchronization may have a short delay, typically under 10 seconds.
  Suggestion: Wait and retry, or verify permissions in the DLF console.
  
  

Q2: What should I do if a token is lost?
The plaintext token is shown only once at creation. The system does not store it. If the token is lost or leaked:

• On the Token Management page, find the corresponding entry and click Reset Token.

• The old token is automatically invalidated, and a new credential is generated.

• Update all client configurations with the new token.

Q3: Why must a RAM user be granted default database Describe permission when connecting to Kyuubi Gateway?
When Kyuubi Gateway establishes a Spark session, it loads the default database as the initial context. Without access to this database, session initialization fails and the connection is terminated. This applies even if your business tables are in other databases, so all RAM users connecting to the gateway must have the DescribeDatabase permission on the default database.

Q4: How can I use a Kyuubi token in DataWorks to control access to DLF data for EMR Serverless Spark?

When you submit a job through a DataWorks Serverless Kyuubi node, DataWorks automatically passes the default access identity of the compute resource to the JDBC connection. Bind EMR Serverless Spark computing resources. Serverless Kyuubi node.