To run Spark jobs on AnalyticDB for MySQL, a Resource Access Management (RAM) user needs three types of permissions: a RAM policy granting cluster access, read/write permissions on AnalyticDB for MySQL databases and tables, and the AliyunADBSparkProcessingDataRole service role that lets Spark access other cloud resources such as Object Storage Service (OSS) and Tablestore. This topic covers how to set up the service role for same-account and cross-account scenarios.
Prerequisites
Before you begin, ensure that you have:
An AnalyticDB for MySQL cluster (Enterprise Edition, Basic Edition, or Data Lakehouse Edition)
One of the following RAM policies granted to the RAM user:
AliyunADBFullAccessorAliyunADBDeveloperAccessRead and write permissions on AnalyticDB for MySQL databases and tables attached via a database account. For details, see Attach a database account to or detach a database account from a RAM user
Choose an authorization method
Two methods are available for same-account authorization. Review the trade-offs before starting.
| Method | Permissions granted | Security |
|---|---|---|
| RAM console (recommended) | Only the policies you specify | Least privilege |
| Quick authorization | AliyunADBSparkProcessingDataRolePolicy (OSS, Tablestore, Data Management (DMS), and more) | Broad by default — trim permissions afterward |
Same-account authorization
How it works
Create a RAM role with Trusted Entity Type set to Alibaba Cloud Service and Select Trusted Service set to Cloud-native Data Warehouse AnalyticDB For MySQL.
Attach only the policies required by your Spark jobs.
Reference the role ARN in the
spark.adb.roleArnparameter when submitting a job.
Grant permissions in the RAM console (recommended)
Step 1: Create a RAM role and grant permissions
Create a RAM role.
On the Create Role page, set Trusted Entity Type to Alibaba Cloud Service and Select Trusted Service to Cloud-native Data Warehouse AnalyticDB For MySQL. Click OK.
In the Create Role dialog box, enter a Role Name and click OK.
Skip this step if a RAM role already exists with Trusted Entity Type set to Alibaba Cloud Service and Select Trusted Service set to Cloud-native Data Warehouse AnalyticDB For MySQL.
ImportantRecord the role name. You will reference it in the
spark.adb.roleArnparameter in Step 2.
Grant permissions to the RAM role.
Click Precise Authorization.
In the Precise Authorization panel, set the policy type to System Policy or Custom Policy and enter a policy name. To restrict access to specific cloud resources, create a custom policy. For details, see Create a custom permission policy.
Click OK.
Step 2: Submit a Spark job
After creating the role, reference it in the spark.adb.roleArn parameter in your Spark job configuration:
{
"comments": [
"-- Here is just an example of using LakeCache. Modify the content and run your spark program."
],
"args": ["oss://testBucketName/data/readme.txt"],
"name": "spark-oss-test",
"file": "oss://testBucketName/data/example.py",
"conf": {
"spark.adb.lakecache.enabled": "true",
"spark.adb.roleArn": "acs:ram::testAccountID:role/adbtest"
}
}Quick authorization
Quick authorization grants the AliyunADBSparkProcessingDataRole role the AliyunADBSparkProcessingDataRolePolicy policy by default. This policy includes broad access to OSS, Tablestore, DMS, and other resources, which may exceed what your jobs actually need. After authorization, detach AliyunADBSparkProcessingDataRolePolicy and grant the role only the permissions your jobs require. See also Revoke permissions from a RAM role.
Click the quick authorization link.
Only users with RAM user management permissions can use quick authorization. At a minimum, you need permissions to create RAM roles, create RAM policies, query RAM roles, and query RAM policies.
In the lower-left corner, click Agree To Authorize.
AnalyticDB for MySQL automatically creates the AliyunADBSparkProcessingDataRole service role and uses it to access other cloud resources.
Cross-account authorization
Use this flow when your AnalyticDB for MySQL cluster (account A) needs to access cloud resources owned by a different Alibaba Cloud account (account B).
Example: Account A (UID: testAccountIDA) needs to read data from account B (UID: testAccountIDB).
How it works
Account B creates a RAM role and configures its trust policy to allow account A's Spark service to assume it.
Account A creates a RAM user and grants it the
ram:PassRolepermission.When submitting a Spark job, reference account B's role ARN in
spark.adb.roleArn.
Step 1: Create a RAM role for account B and grant permissions
Perform the following steps using account B or a RAM administrator of account B.
Create a RAM role and allow account A to assume it.
Log on to the RAM console using Alibaba Cloud account B or a RAM administrator of account B.
In the left-side navigation pane, choose Identity Management > Roles.
On the Roles page, click Create Role.
On the Create Role page, set Trusted Entity Type to Alibaba Cloud Account, specify the account, and click OK.
Configure the role:
Role Name: Enter a name, such as
admin-oss.Note (optional): Enter a description.
Other Alibaba Cloud Account: Select Other Alibaba Cloud Account and enter the UID of account A (
testAccountIDA).
Click Finish.
Skip this step if a RAM role already exists that allows account A to assume it. Record the role ARN — you will need it in Step 2.
ImportantRecord the role ARN shown on the role details page. The format is
acs:ram::<testAccountIDB>:role/<role-name>. You will reference this value in thespark.adb.roleArnparameter.Grant permissions to the RAM role.
Click Precise Authorization.
In the Precise Authorization panel, set the policy type to System Policy or Custom Policy and enter a policy name.
Access OSS
AliyunOSSReadOnlyAccess: Grants read-only permissions on OSS.AliyunOSSFullAccess: Grants full control over OSS. You must grant this permission to the RAM role to perform data update operations, such as INSERT, on OSS external tables.
Access VPC resources
To access resources in a specific VPC, you must create a custom policy. In the
Resourceparameter of the policy, specify the security group and vSwitch to which the resources belong. For more information, see Create a custom permission policy.This example grants permissions to access only an ApsaraDB RDS for MySQL instance in a specific VPC. The name of the custom policy is
eni_policy. The following code shows the policy document:NoteYou must add a security group to the ApsaraDB RDS for MySQL instance. The inbound and outbound rules of the security group must allow access on the port of the ApsaraDB RDS for MySQL instance.
{ "Version": "1", "Statement": [ { "Effect": "Allow", "Action": "ecs:*", "Resource": "acs:ecs:*:*:securitygroup/<The ID of the security group to which the RDS instance belongs>" }, { "Effect": "Allow", "Action": "vpc:*", "Resource": "acs:vpc:*:*:vswitch/<The ID of the vSwitch to which the RDS instance belongs>" } ] }Click OK.
Click Close.
Update the trust policy to allow account A's Spark service to assume the role.
In the left-side navigation pane, choose Identity Management > Roles.
On the Roles page, click the name of the role you created.
On the Trust Policy tab, click Edit Trust Policy.
Replace the policy document with the following and click OK:
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": [ "acs:ram::testAccountIDA:root" ] } }, { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "testAccountIDA@ads.aliyuncs.com" ] } } ], "Version": "1" }Replace
testAccountIDAwith the actual UID of account A.
Step 2: Create a RAM user for account A and grant permissions
Perform the following steps using account A or a RAM administrator of account A.
Create a RAM user.
In the User Account Information section, configure the following parameters:
Parameter Description Logon Name Up to 64 characters. Allowed characters: letters, digits, periods ( .), hyphens (-), and underscores (_).Display Name Up to 128 characters. Tag (optional) Click the edit icon and enter a tag key and value to help manage the user. NoteClick Add User to create multiple RAM users at once.
In the Access Mode section, select Console Password Logon and configure the logon password, password reset policy, and multi-factor authentication (MFA) settings as needed. For details, see Console access.
Click OK.

Create a permission policy that allows the RAM user to pass roles.
In the left-side navigation pane, choose Permission Management > Policies.
On the Policies page, click Create Policy.
On the Create Policy page, click the Script Editor tab.
Enter the following policy document and click OK:
{ "Version": "1", "Statement": [ { "Action": "ram:PassRole", "Resource": "*", "Effect": "Allow" } ] }In the Create Policy dialog box, enter a Policy Name and optional Note, then click OK.
Grant the policy to the RAM user.
In the left-side navigation pane, choose Identity Management > Users.
On the Users page, find the RAM user and click Add Permissions in the Actions column.
In the Add Permissions panel, select the policy you created in the previous step.
Click OK, then click Close.