If you want to use a RAM user to access other cloud resources, you can refer to this document to grant the corresponding permissions to the RAM user based on your scenario (same-account or cross-account). This topic describes the detailed steps for same-account authorization and cross-account authorization.
Prerequisites
The cluster must be of Enterprise Edition, Basic Edition, or Lakehouse Edition.
Spark Job Permissions
To submit a Spark job, you must have the following three types of permissions:
Permission | Description |
One of the following permissions:
|
specificActionsseeGrant permissions to a RAM user. |
Read and write permissions on AnalyticDB for MySQL databases and tables | AnalyticDB for MySQL By default, database accounts are used to manage database and table permissions.When you submit a Spark job as a RAM user, if the Spark job needs to read or write data, you must bind a standard database account to the RAM user.specificActionsseeBind or unbind a RAM user and a database account. |
| Used to authorizeAnalyticDB for MySQL Spark to access other cloud resources, such as OSS directories and OTS data.specificActionsseeSame-Account Authorization. |
Same-Account Authorization
Prerequisites
Procedure
Authorize through the RAM Console (Recommended)
Step 1: Create a RAM role and grant permissions
Create a RAM role.
NoteIf you have already created a RAM role whose Trusted Entity TypeforAlibaba Cloud Service, Trusted EntityforAnalyticDB for MySQL, you can skip this step and continue with the following steps.
Log on to the RAM console as a RAM administrator.
In the left-side navigation pane, choose .
On the Rolespage, ClickCreate Role.
On the Create Role page, set Trusted Entity TypeforAlibaba Cloud Service, and thenchooseTrusted EntityforAnalyticDB for MySQL, and then click OK.
On the Create Roledialog box, EnterRole Name, and then clickOK.
Grant permissions to the RAM role.
Click Grant Permission by Policy.
In the Grant Permission by Policy panel, set the policy type to System Policyor Custom Policy, and then enter the policy name.
If you want to access only specific cloud resources, you must create a custom policy. For more information, see Create a custom policy.
Click OK.
Step 2: Submit a Spark job
After you create a RAM role and grant permissions through the RAM console, when you submit a Spark job, you must configure the spark.adb.roleArn parameter in the Spark job configuration to ensure that the Spark job runs properly. Example:
{
"comments": [
"-- Here is just an example of using LakeCache. Modify the content and run your spark program."
],
"args": ["oss://testBucketName/data/readme.txt"],
"name": "spark-oss-test",
"file": "oss://testBucketName/data/example.py",
"conf": {
"spark.adb.lakecache.enabled": "true";
"spark.adb.roleArn": "acs:ram::testAccountID:role/adbtest";
}
}Quick Authorization
After same-account authorization, the AliyunADBSparkProcessingDataRole role is granted the AliyunADBSparkProcessingDataRolePolicy policy by default. This policy includes access permissions on cloud resources such as OSS, Table Store, and DMS, which may cause over-authorization. We recommend that you, after authorization, remove the AliyunADBSparkProcessingDataRolePolicy policy and AliyunADBSparkProcessingDataRoleRolesgrant only the permissions required by your business.
Click quick authorization link.
NoteOnly users that have RAM user management permissions (at least the permissions to create RAM roles, create RAM policies, query RAM roles, and query RAM policies) can perform quick authorization.
Click in the lower-left cornerConfirm Authorization.
After authorization, the service role AliyunADBSparkProcessingDataRole is automatically created for AnalyticDB for MySQL to use. AnalyticDB for MySQL uses this role to access other cloud resources.
Cross-Account Authorization
AnalyticDB for MySQL Spark supports access to all cloud resources of other Alibaba Cloud accounts.This topic uses Alibaba Cloud Account A (the primary account with ID testAccountIDA) to access data of Alibaba Cloud Account B (the primary account with ID testAccountIDB) as an example to describe the cross-account authorization procedure.
Step 1: Create a RAM role for Account B and grant permissions
Create a RAM role and allow Account A to assume this RAM role.
NoteIf you have already created a RAM role and allowed Account A to assume this RAM role, you can skip this step and continue with the following steps.
Use Alibaba Cloud Account B (the primary account) or a RAM administrator to log on to the RAMconsole.
In the left-side navigation pane, choose .
On the Rolespage, ClickCreate Role.
On the Create Rolepage, chooseTrusted Entity TypeforAlibaba Cloud Account, specify the Alibaba Cloud account, and then click OK.
Configure the role information.
Enter a Role Name.In this example, set the role name to
admin-oss.Optional: Enter a Description.
Set Trusted Alibaba Cloud Account to Other Alibaba Cloud Account, and then enter the UID of Account A (
testAccountIDA).
Click Complete.
for RAMRoles Grant Permission by Policy.
Click Grant Permission by Policy.
On the Grant Permission by Policypanel, set the policy type toSystem PolicyorCustom Policy, and then enter the policy name.
Access OSS
AliyunOSSReadOnlyAccess: The permission to read OSS resources.
AliyunOSSFullAccess: The permission to manage OSS. If you need to perform data update operations such as INSERT on OSS external tables, you must grant this permission to the RAM role.
Access resources in a specified VPC
If you want to access resources in a specified VPC, you must create a custom policy and specify the security group and vSwitch in the Resource parameter. For more information, see Create a custom policy.
In this example, only access to the ApsaraDB RDS for MySQL instance in the specified VPC is authorized. The custom policy name is
eni_policy, and the policy content is as follows:NoteYou mustRDS MySQLinstance and add a security group, and allow inbound and outbound access in the security group rules toRDS MySQLport access requests.
{ "Version": "1", "Statement": [ { "Effect": "Allow", "Action": "ecs:*", "Resource": "acs:ecs:*:*:securitygroup/<Security group ID of the ApsaraDB RDS instance>" }, { "Effect": "Allow", "Action": "vpc:*", "Resource": "acs:vpc:*:*:vswitch/<vSwitch ID of the ApsaraDB RDS instance>" } ] }Click OK.
Click Close.
Modify the trust policy to allow any RAM user under Account A to assume this RAM role.
In the left-side navigation pane, choose .
On the Roles page, click the name of the target RAM role.
On the Trust Policytab, ClickEdit Trust Policy.
In the editor, Modify the trust policy content, and then clickOK.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": [ "acs:ram::testAccountIDA:root" ] } }, { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "testAccountIDA@ads.aliyuncs.com" ] } } ], "Version": "1" }
Step 2: forAaccount. CreateRAMUsersand grantassumeRAMRoles permissions
Create a RAM user.
Use Alibaba Cloud Account A (the primary account) or a RAM administrator to log on to the RAMconsole.
In the left-side navigation pane, choose .
On the Userspage, ClickCreate User.
On the Create User page, configure the basic user information in the User Account Information section.
Logon Name(Required): can contain letters, digits, periods (.), hyphens (-), and underscores (_). Maximum length: 64 characters.
Display Name(Optional): maximum length: 128 characters.
Tag(Optional): Click
, and then enter tag keys and tag values for tag-based user management.
NoteClick Add User to create multiple RAM users at a time.
In Access Method section, choose Console Access, and configure the console logon password, password reset policy, and MFA policy as needed. For more information, see Console Access.
Click OK.
CreatePolicies, allowRAMUsersassumeany role.
In the left-side navigation pane, choose .
On the Policiespage, ClickCreatePolicies.
On the CreatePoliciespage, ClickScript Editortab.
EnterPoliciescontent, and then clickOK.
{ "Version": "1", "Statement": [ { "Action": "ram:PassRole", "Resource": "*", "Effect": "Allow" } ] }On the CreatePoliciesdialog box, EnterPolicy NameandDescription, and then clickOK.
Grant permissions to the RAM user.
In the left-side navigation pane, choose .
On the Userspage, ClicktargetRAMUsersActionsin theAdd Permissions.
In the Add Permissions panel, add the policy created in Step 2 to the RAM user.
Click Confirm.
Click Close.