E-MapReduce:Use a custom ECS application role to access cloud resources within the same account

Prerequisites

Before you begin, ensure that you have:

• A cluster created — see Create a cluster

• Roles assigned to your Alibaba Cloud account — see Assign roles to an Alibaba Cloud account

• An OSS bucket in the same region where you plan to create the EMR cluster — see Create buckets

Overview

The setup involves three steps:

1. Create a policy — define which resources the cluster can access and what operations it can perform.

2. Create a RAM role — create a role that ECS can assume and attach the policy from step 1.

3. Create a cluster and verify access — associate the RAM role when creating the cluster, then confirm access works.

Step 1: Create a policy

1. Log on to the RAM console with your Alibaba Cloud account.

2. In the left-side navigation pane, choose Permissions > Policies, then click Create Policy.

3. On the Create Policy page, click the JSON tab and enter a policy document in the editor. The following example grants read-only access to a specific OSS bucket:

   {
       "Version": "1",
       "Statement": [
           {
               "Action": [
                   "oss:GetObject",
                   "oss:ListObjects"
               ],
               "Resource": [
                   "acs:oss:*:*:examplebucket",
                   "acs:oss:*:*:examplebucket/*"
               ],
               "Effect": "Allow"
           }
       ]
   }

   Replace examplebucket with your actual OSS bucket name. The Resource field requires two entries: one for the bucket itself (examplebucket) and one for all objects within it (examplebucket/*). The Action field grants read-only access — oss:GetObject to download objects and oss:ListObjects to list them. For details on policy syntax and available actions, see Policy elements.

4. Click OK, then set the Name field in the Basic information section. In this example, the name is test-emr.

5. Click OK.

Step 2: Create a RAM role

1. In the left-side navigation pane of the RAM console, choose Identities > Roles, then click Create Role.

2. In the Select Role Type step, set Select Trusted Entity to Alibaba Cloud Service and click Next.

3. In the Configure Role step, set the following parameters: Click OK.

   Parameter	Value
   RAM Role Name	Enter a name for the role. In this example: test-emr
   Select Trusted Service	Select Elastic Compute Service

4. (Optional) Update the trust policy. For earlier versions, the trusted service must be changed from ECS to EMR. Without this change, cluster creation fails with a NoPermission error.

   Important

   Skip this step if you are creating a cluster running EMR V5.x or later, a minor version later than EMR V4.5, or a minor version later than EMR V3.32. These versions do not require this change.

   1. On the Roles page, click the name of the RAM role you created.

   2. Click the Trust Policy tab, then click Edit Trust Policy.

   3. Change ecs.aliyuncs.com to emr.aliyuncs.com.

   4. Click Save trust policy document.

5. Grant the required permissions to the role:

   1. On the Roles page, find the role you created and click Grant Permission in the Actions column.

   2. In the Grant Permission panel, select Custom Policy from the drop-down list in the Policy section, then select the policy you created in Step 1.

   3. Click Grant permissions, then click Close.

Step 3: Create a cluster and verify access

1. Log on to the EMR console. In the left-side navigation pane, click EMR on ECS.

2. In the top navigation bar, select the region where your cluster will reside and select a resource group.

3. Click Create Cluster. In the Advanced Settings section of the Basic Configuration step, enter the name of the RAM role you created in Step 2. For other cluster configuration options, see Create a cluster.

   

4. After the cluster is created, log on to the cluster using SSH. See Log on to a cluster.

5. Run the following command to verify that the cluster can access the OSS bucket specified in the policy:

   hdfs dfs -ls oss://<yourBucketName>/

   Replace <yourBucketName> with your actual OSS bucket name.

   • If the bucket contents are listed, access is working correctly.

   • If access is denied, the cluster does not have permission to access the bucket — see the FAQ below.

FAQ

Why does a NoPermission error appear when I create a cluster?

Check the following in order:

1. The RAM user creating the cluster must have AliyunEMRFullAccess. If the user has AliyunEMRDevelopAccess, change it to AliyunEMRFullAccess.

2. Check that the ECS application role name you specified is valid and matches the RAM role you created.

3. If you are using an EMR version that requires updating the trust policy (earlier than EMR V5.x, EMR V4.5, or EMR V3.32), confirm that ecs.aliyuncs.com has been changed to emr.aliyuncs.com in the trust policy.

Why can't I access an OSS bucket from Hadoop Distributed File System (HDFS)?

Check the following in order:

1. The OSS bucket must be in the same region as the EMR cluster. If they are in different regions, add the bucket's endpoint to the connection strings.

2. The bucket must be listed as a resource in the policy. If it is not, update the Resource field in the policy to include it.

3. If bucket-level permissions are configured directly in the OSS console, those settings may conflict with the policy. Remove the OSS console permissions and manage access exclusively through the Action field in the RAM policy.