Configure RAM policies to grant access to OSS and OSS-HDFS - E-MapReduce

Prerequisites

Before you begin, ensure that you have:

An Alibaba Cloud account with permission to create RAM policies
The name of the OSS bucket or OSS-HDFS namespace to grant access to

Grant access to OSS

EMR clusters access OSS data through RAM policies attached to RAM users or roles. Follow the principle of least privilege: grant only the permissions your workload requires.

Important

If you deploy Hive and Spark in your cluster, include oss:ListObjectVersions and oss:DeleteObjectVersion in every policy — even if versioning is not enabled on your bucket. EMR internally triggers versioning for temporary directories such as _temporary, .staging, .hive-staging, and __magic.

Choose a policy for your scenario

Select the policy that matches your use case and paste it into the RAM console when creating your policy.

Read/write access (recommended for most workloads)

Grants full read and write access, including multipart uploads and object deletion. Replace <yourBucketName> with your bucket name.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListObjects",
        "oss:GetBucketInfo",
        "oss:PutObject",
        "oss:GetObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListMultipartUploads",
        "oss:ListParts",
        "oss:RestoreObject",
        "oss:ListObjectVersions",
        "oss:DeleteObjectVersion"
      ],
      "Resource": [
        "acs:oss:*:*:<yourBucketName>",
        "acs:oss:*:*:<yourBucketName>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListBuckets"
      ],
      "Resource": "*"
    }
  ]
}

Read-only access

Use this policy when your EMR job only reads from OSS. oss:ListObjectVersions is still required if you run Hive or Spark.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListObjects",
        "oss:GetBucketInfo",
        "oss:GetObject",
        "oss:RestoreObject",
        "oss:ListObjectVersions"
      ],
      "Resource": [
        "acs:oss:*:*:<yourBucketName>",
        "acs:oss:*:*:<yourBucketName>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListBuckets"
      ],
      "Resource": "*"
    }
  ]
}

Versioning-enabled bucket access

Use this policy only if versioning is enabled on the target bucket. It extends read/write access with version-specific operations (oss:GetObjectVersion, oss:DeleteObjectVersion, oss:RestoreObjectVersion).

Important

Avoid enabling versioning on OSS buckets used by EMR. Versioned objects require EMR to track and clean up multiple versions of temporary files during job execution, which increases API call volume and degrades performance. Grant all required permissions to avoid AccessDenied errors.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListObjects",
        "oss:GetBucketInfo",
        "oss:PutObject",
        "oss:GetObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListMultipartUploads",
        "oss:ListParts",
        "oss:RestoreObject",
        "oss:ListObjectVersions",
        "oss:GetObjectVersion",
        "oss:DeleteObjectVersion",
        "oss:RestoreObjectVersion"
      ],
      "Resource": [
        "acs:oss:*:*:<yourBucketName>",
        "acs:oss:*:*:<yourBucketName>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "oss:ListBuckets"
      ],
      "Resource": "*"
    }
  ]
}

For more information about configuring RAM policies for OSS, see RAM policies.

Create the policy in the RAM console

Log on to the RAM console with your Alibaba Cloud account.
Log on to the RAM console by using your Alibaba Cloud account.
In the left-side navigation pane, choose Permissions > Policies.
On the Policies page, click Create Policy.
On the Create Policy page, click the JSON tab.
Paste your policy document into the editor and click OK.
Set the Policy Name and Description, then click OK.

Grant access to OSS-HDFS

OSS-HDFS uses a different access model from standard OSS. OSS-HDFS exposes a POSIX-compatible file system layer on top of OSS, so it requires two additional actions beyond standard OSS permissions:

oss:PostDataLakeStorageFileOperation — for file-level operations (read, write, delete) on the OSS-HDFS namespace
oss:PostDataLakeStorageAdminOperation — for administrative operations on the namespace

The data managed by OSS-HDFS is stored in the .dlsdata path within the bucket. The policy grants full OSS access to that path.

Sample policy:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "oss:ListObjects",
      "Resource": [
        "acs:oss:*:*:*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "oss:GetBucketInfo",
        "oss:PostDataLakeStorageFileOperation",
        "oss:PostDataLakeStorageAdminOperation"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "oss:*",
      "Resource": [
        "acs:oss:*:*:*/.dlsdata",
        "acs:oss:*:*:*/.dlsdata*"
      ]
    }
  ]
}

Create the policy in the RAM console

Log on to the RAM console with your Alibaba Cloud account.
In the left-side navigation pane, choose Permissions > Policies.
On the Policies page, click Create Policy.
On the Create Policy page, click the JSON tab.
Paste the policy document into the editor and click OK.
Set the Policy Name and Description, then click OK.