E-MapReduce:Control DLF data access in EMR Serverless Spark

Step 1: Prepare environment and RAM permissions

1. Prepare basic resources.

   • You have created a workspace. For more information, see Manage workspaces.

   • You have created and started a Kyuubi Gateway in the workspace. For more information, see Manage Kyuubi gateways.

   • You have created a data catalog in Data Lake Formation (DLF) and prepared the required database and tables.

2. Configure a RAM user and permissions. Create a RAM user and grant it the necessary permissions.

   • EMR Serverless Spark permissions: Grant the RAM user the permissions required to operate EMR Serverless Spark. For more information, see Grant permissions to a RAM user.

   • Workspace permissions: Add the RAM user to the workspace and assign a role based on the user's function. For more information, see Manage users and roles.

Step 2: Grant table permissions in DLF

In Data Lake Formation (DLF), grant the RAM user permissions to access specific tables. To ensure that the Kyuubi Gateway can successfully initialize a Spark session, you must also grant the RAM user the DescribeDatabase permission on the default database in DLF.

1. Log on to the .

2. Navigate to the target database and table within the desired catalog.

3. Select the table to which you want to grant permissions and go to the Permission tab.

4. Click Add Permission.

   1. Principal: Select DLF User.

   2. Select DLF User: Select the target RAM user.

   3. Permission: Select the required permissions.

5. Click OK. The authorization is complete.

   Note

   By default, EMR Serverless Spark caches metadata and data from Data Lake Formation. If you modify permissions for a table, the changes can take up to 10 minutes to take effect.

   To apply permission changes immediately, add the spark.sql.catalog.lakehouse.cache-enabled false parameter to your Spark configuration in the Kyuubi Gateway.

Step 3: Generate a Kyuubi token

1. On the Kyuubi Gateway page, find the target gateway and click Token Management in the Actions column.

2. Click Create Token. In the dialog box that appears, configure the following parameters and click OK.

   Parameter	Description
   Name	The name of the new token.
   Expired At	The token's expiration time. The value must be an integer of 1 or greater. By default, tokens expire after 365 days.
   Assigned to	From the drop-down list, select the target RAM user.

3. Copy the token information.

   Important

   After the token is created, you must immediately copy the token information because it cannot be retrieved later. If your token expires or is lost, create a new one or reset the existing token by clicking Reset Token.

Step 4: Connect and verify permissions

1. Build the JDBC connection command.

   beeline -u "jdbc:hive2://<endpoint>:<port>/;transportMode=http;user=<user_name_or_role_name>;httpPath=cliservice/token/<token>"

   Parameters:

   Parameter	Description
   <endpoint>	The endpoint of the Kyuubi Gateway.
   <port>	The access port. Use port 443 for a public endpoint or port 80 for an internal network endpoint.
   <user_name_or_role_name>	The RAM user or RAM role. You can use the short name or the full name. Examples: RAM user: agent or agent@xxxx05398154xxxx.onaliyun.com RAM role: AliyunServiceRoleForDataworksEngine
   <token>	The token you generated for the RAM user in Step 3.

   [root@master-1-1(xxx) ~]# beeline -u "jdbc:hive2://kyuubi-cn-hangzhou-internal.spark.emr.aliyuncs.com:80;transportMode=http;user=emr_test;httpPath=cliservice/token/j2xxx"
       xxx
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in [jar:file:/opt/apps/HIVE/hive-3.1.3-hadoop3.1-1.0.9/lib/log4j-slf4j-impl-2.17.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in [jar:file:/opt/apps/HADOOP-COMMON/hadoop-3.2.1-1.3.5-alinux3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
   Connecting to jdbc:hive2://kyuubi-cn-hangzhou-internal.spark.emr.aliyuncs.com:80;transportMode=http;user=emr_test;httpPath=cliservice/token/j2xxx
   Connected to: Spark SQL (version 3.5.2-emr)
   Driver: Hive JDBC (version 3.1.3)
   Transaction isolation: TRANSACTION_REPEATABLE_READ
   Beeline version 3.1.3 by Apache Hive
   0: jdbc:hive2://kyuubi-cn-hangzhou-internal.s>

2. Verify the access control.

   • Query an authorized table:

     SELECT * FROM <database_name>.<authorized_table_name> LIMIT 10;

     The query succeeds and returns data similar to the following:

     2025-10-24 16:55:45.008 INFO nioEventLoopGroup-7-1 org.
     nnelInactive!
     +-----+----------+
     | id  |   name   |
     +-----+----------+
     | 1   | Alice    |
     | 2   | Bob      |
     | 3   | Charlie  |
     +-----+----------+
     3 rows selected (5.079 seconds)

   • Query an unauthorized table:

     SELECT * FROM <database_name>.<unauthorized_table_name> LIMIT 10;

     The query fails, and you receive a permission-related error message, such as emr_test doesn't have privilege SELECT on TABLE. When you run a query on an unauthorized table, the following permission exception message is returned:

     Caused by: org.apache.paimon.rest.exceptions.ForbiddenException: Forbidden: User acs:ram::xxx:user/emr_test doesn't have privilege SELECT on TABLE default.paimon_tbl
     requestId:xxx
     	at org.apache.paimon.rest.DefaultErrorHandler.accept(DefaultErrorHandler.java:59)
     	at org.apache.paimon.rest.DefaultErrorHandler.accept(DefaultErrorHandler.java:35)
     	at org.apache.paimon.rest.HttpClient.exec(HttpClient.java:156)
     	at org.apache.paimon.rest.HttpClient.get(HttpClient.java:69)
     	at org.apache.paimon.rest.RESTApi.getTable(RESTApi.java:465)

Q1: Why are permission changes not taking effect?

• Data Lake Formation metadata cache: The Spark engine caches table schema information, which bypasses the authorization check.
  Solution: Add spark.sql.catalog.lakehouse.cache-enabled false to the Spark configuration to disable the metadata cache.
  
  

• Permission propagation delay: In rare cases, there is a short delay (usually less than 10 seconds) for permissions to sync in Data Lake Formation.
  Recommendation: Wait a moment and retry, or verify in the Data Lake Formation console that the permissions are correctly assigned.
  
  

Q2: What to do if a token is lost?

• On the Token management page, find the corresponding token and click Reset Token.

• The old token is immediately invalidated, and a new one is generated.

• Update all client configurations with the new token.

Q3: Why must a RAM user be granted Describe permission for the default database when connecting to Kyuubi Gateway?
By default, when Kyuubi Gateway establishes a Spark session, it attempts to load the default database as the initial context. If the RAM user lacks permission to access this database, session initialization fails and the connection is terminated. This check cannot be skipped, even if your business tables are located in other databases. Therefore, all RAM users who connect to the gateway must be granted the default database's DescribeDatabase permission in advance.

Q4: How to use Kyuubi tokens with DataWorks?

When you submit a task using a Serverless Kyuubi node in DataWorks, DataWorks automatically passes the default access identity configured in the compute resource to the JDBC connection. For information about how to bind an EMR Serverless Spark compute resource in DataWorks, see Bind an EMR Serverless Spark compute resource. For more information about how to use Serverless Kyuubi nodes, see Serverless Kyuubi node.