Sts mode authorization for external data access - MaxCompute

MaxCompute uses RAM (Resource Access Management) and STS (Security Token Service) to control access to external data sources. Before external tables can read or write data in OSS, Tablestore, Hologres, or DLF, you must grant MaxCompute permission to assume a RAM role that has access to the target storage service.

These steps require RAM administrator permissions. If you are not a RAM administrator, ask your cloud account admin to complete steps that involve creating roles and editing policies.

This topic covers authorization for the following external data sources:

Data source	Role type	Section
OSS	Regular role	OSS STS-based authorization for a regular role
Tablestore	Regular role	Tablestore STS-based authorization for a regular role
Hologres	Regular role	Hologres STS-based authorization for a regular role
DLF+OSS	Regular role	DLF+OSS STS-based authorization for a regular role
Hologres	Service-linked role	Hologres STS-based service-linked role authorization
Paimon_DLF	Service-linked role	Paimon_DLF STS-based service-linked role authorization

OSS STS-based authorization for a regular role

To let MaxCompute external tables access OSS data, grant the required OSS permissions to the MaxCompute account using one of the following methods.

One-click authorization (recommended)click here to complete one-click authorization

Available when MaxCompute and OSS belong to the same Alibaba Cloud account. Log on to your Alibaba Cloud account and complete one-click authorization.

The default AliyunODPSDefaultRole has broad permissions. Use it only as a reference. Configure appropriate access policies and trust policies in the RAM console based on the data your external tables or external data sources need to access.

Custom authorization

Works for both same-account and cross-account scenarios. The trust policy Service value depends on your account setup:

Same account: set Service to odps.aliyuncs.com
Different accounts: set Service to <Alibaba Cloud account ID of the MaxCompute owner>@odps.aliyuncs.com Find the MaxCompute owner account ID on the Account Overview page.

Follow these steps:

Create a RAM role in the RAM console On the RAM roles page, create a role. For example, create a role named oss-admin. For details, see Create a RAM role whose trusted entity is an Alibaba Cloud account.

Edit the trust policy of the RAM role

Click OK.

If MaxCompute and OSS belong to the same Alibaba Cloud account

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

If MaxCompute and OSS belong to different Alibaba Cloud accounts

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "<Alibaba Cloud account ID of the MaxCompute owner>@odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Create an access policy On the Policies page, create an access policy. For example, name it AliyunODPSRolePolicy. The following policy grants common OSS permissions. Customize the action list based on what your external tables need to access. For details, see Create a custom access policy.

Permission	Action
List buckets	`oss:ListBuckets`
Read objects	`oss:GetObject`, `oss:ListObjects`
Write objects	`oss:PutObject`
Delete objects	`oss:DeleteObject`, `oss:AbortMultipartUpload`, `oss:ListParts`

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "oss:ListBuckets",
        "oss:GetObject",
        "oss:ListObjects",
        "oss:PutObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListParts"
      ],
      "Resource": "*",
      "Effect": "Allow"
    }
  ]
}

Attach the access policy to the RAM role For details, see Manage permissions for a RAM role.
1. In the RAM console, choose Identities > Roles in the left navigation pane.
2. Click the target Role Name to open its details page.
3. On the Permissions tab, click Grant Permission.
4. In the Grant Permission panel, select the access policy and click OK.

Tablestore STS-based authorization for a regular role

To let MaxCompute external tables access Tablestore data, grant the required Tablestore permissions to the MaxCompute account using one of the following methods.

One-click authorization (recommended)

Available when MaxCompute and Tablestore belong to the same Alibaba Cloud account. Log on to your Alibaba Cloud account and complete one-click authorization.

Custom authorization

Use this method when MaxCompute and Tablestore belong to different Alibaba Cloud accounts.

Create a RAM role in the RAM console Log on to the RAM console using the Tablestore account, then create a role on the RAM roles page. For example, name it oss-admin-ots. For details, see Create a RAM role whose trusted entity is an Alibaba Cloud account.

Edit the trust policy for cross-account access

Click OK.

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "<Alibaba Cloud account ID of the MaxCompute owner>@odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Create an access policy On the Policies page, create an access policy. For example, name it AliyunODPSRolePolicy. The following policy grants common Tablestore permissions. Customize the action list as needed. For details, see Create a custom access policy.

Permission	Action
List and describe tables	`ots:ListTable`, `ots:DescribeTable`
Read rows	`ots:GetRow`, `ots:GetRange`, `ots:BatchGetRow`
Write rows	`ots:PutRow`, `ots:UpdateRow`, `ots:BatchWriteRow`
Delete rows	`ots:DeleteRow`
Compute splits	`ots:ComputeSplitPointsBySize`

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "ots:ListTable",
        "ots:DescribeTable",
        "ots:GetRow",
        "ots:PutRow",
        "ots:UpdateRow",
        "ots:DeleteRow",
        "ots:GetRange",
        "ots:BatchGetRow",
        "ots:BatchWriteRow",
        "ots:ComputeSplitPointsBySize"
      ],
      "Resource": "*",
      "Effect": "Allow"
    }
  ]
}

Attach the access policy to the RAM role For details, see Manage permissions for a RAM role.
1. In the RAM console, choose Identities > Roles in the left navigation pane.
2. Click the target Role Name to open its details page.
3. On the Permissions tab, click Grant Permission.
4. In the Grant Permission panel, select the AliyunODPSRolePolicy policy and click OK.

Hologres STS-based authorization for a regular role

To let MaxCompute external tables access Hologres data, complete three tasks: create a RAM role, configure its trust policy, then add the role to the Hologres instance and grant it database permissions.

Step 1: Create a RAM role in the RAM console

Create a RAM role to get its ARN. You will use the ARN when creating an external table to configure STS authentication.

Choose the trusted entity type based on your needs:

Alibaba Cloud account: A RAM user under an Alibaba Cloud account assumes the role to access cloud resources. For details, see Create a RAM role whose trusted entity is an Alibaba Cloud account.
Identity provider: Use single sign-on (SSO) to let users log on to the Alibaba Cloud Management Console from your enterprise identity system. For details, see Create a RAM role whose trusted entity is an identity provider.

To create the role:

In the RAM console, choose Identities > Roles in the left navigation pane.
On the Roles page, click Create Role.

Step 2: Edit the trust policy

On the Roles page, click the target Role Name to open its details page.
On the Trust Policy tab, click Edit Trust Policy, then click the JSON tab.

Replace <UID> with your Alibaba Cloud account ID. Find your account ID on the User Information page. Configure the trust policy based on your trusted entity type:

Trusted entity type: Alibaba Cloud account

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "RAM": [
          "acs:ram::<UID>:root"
        ]
      }
    },
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "<UID>@odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Trusted entity type: identity provider

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "saml:recipient": "https://signin.aliyun.com/saml-role/sso"
        }
      },
      "Effect": "Allow",
      "Principal": {
        "Federated": [
          "acs:ram::<UID>:saml-provider/IDP"
        ]
      }
    },
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "<UID>@odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Step 3: Add the RAM role to the Hologres instance and grant permissions

The RAM role needs development permissions on the Hologres instance to operate within its permission scope. By default, RAM roles cannot view or manage Hologres instances in the Hologres Management Console — an Alibaba Cloud account must grant these permissions first.

Choose one of the following authorization methods:

Authorize through the Hologres Management Console

Log on to the Hologres Management Console and select a region in the upper-left corner.the Hologres Management Console
In the left navigation pane, select Instances.
Click the target instance name.
On the instance details page, select Account Management in the left navigation pane.
On the User Management page, click Add User.
In the Add User dialog box, configure the following settings and click OK:
- Select the AliyunODPSDefaultRole role.
- Set the member role to Regular user.
In the left navigation pane, select Database Authorization.
Click Create Database.
In the Create Database dialog box, set the Permission policy to SPM (Simple Permission Model) and click OK.
On the Database Authorization page, click Authorize User in the Actions column of the target database.
Click Grant Permissions in the upper-right corner. In the Grant Permissions dialog box, configure the following:
- User: Select the RAM user to configure.
- User Group: Select Developer.

Authorize through the RAM console

Attach the AliyunODPSRolePolicy access policy to the RAM role:

In the RAM console, choose Identities > Users in the left navigation pane.
Click the User Logon Name/Display Name of the target user.
On the user details page, click the Permissions tab.
Select Individual and click Grant Permission.
In the Grant Permission panel, select the AliyunRAMReadOnlyAccess permission policy and click OK. For details, see Grant permissions to a RAM user.

Authorize using SQL

For SQL-based authorization, see Hologres permission model.

Hologres also supports dual-signature authentication. For details, see Hologres external tables.

DLF+OSS STS-based authorization for a regular role

When building a data lakehouse with MaxCompute, DLF (Data Lake Formation), and OSS, the MaxCompute project cannot access DLF or OSS until authorized. Grant access using one of the following methods.

One-click authorization (recommended)

Available when the Alibaba Cloud account that creates the MaxCompute project is the same as the account that deploys DLF and OSS. Authorize DLF and OSS to complete one-click authorization.

Custom authorization

Works for both same-account and cross-account scenarios. The trust policy Service value depends on your account setup:

Same account (MaxCompute project account = DLF deployment account): set Service to odps.aliyuncs.com
Different accounts: set Service to <Alibaba Cloud account ID of the MaxCompute owner>@odps.aliyuncs.com Find the MaxCompute owner account ID on the Account Overview page.

Follow these steps:

Create a RAM role in the RAM console Create a RAM role whose trusted entity is an Alibaba Cloud account in the RAM console.

Edit the trust policy of the RAM role Modify the trust policy of the RAM role using one of the following policies based on your account setup.

The Alibaba Cloud account that creates the MaxCompute project is the same as the account that deploys DLF

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

The Alibaba Cloud account that creates the MaxCompute project is different from the account that deploys DLF

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "<Alibaba Cloud account ID of the MaxCompute owner>@odps.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Create a custom access policy Create a custom policy with the following content, which grants both OSS and DLF permissions:

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "oss:ListBuckets",
        "oss:GetObject",
        "oss:ListObjects",
        "oss:PutObject",
        "oss:DeleteObject",
        "oss:AbortMultipartUpload",
        "oss:ListParts"
      ],
      "Resource": "*",
      "Effect": "Allow"
    },
    {
      "Action": [
        "dlf:CreateFunction",
        "dlf:BatchGetPartitions",
        "dlf:ListDatabases",
        "dlf:CreateLock",
        "dlf:UpdateFunction",
        "dlf:BatchUpdateTables",
        "dlf:DeleteTableVersion",
        "dlf:UpdatePartitionColumnStatistics",
        "dlf:ListPartitions",
        "dlf:DeletePartitionColumnStatistics",
        "dlf:BatchUpdatePartitions",
        "dlf:GetPartition",
        "dlf:BatchDeleteTableVersions",
        "dlf:ListFunctions",
        "dlf:DeleteTable",
        "dlf:GetTableVersion",
        "dlf:AbortLock",
        "dlf:GetTable",
        "dlf:BatchDeleteTables",
        "dlf:RenameTable",
        "dlf:RefreshLock",
        "dlf:DeletePartition",
        "dlf:UnLock",
        "dlf:GetLock",
        "dlf:GetDatabase",
        "dlf:GetFunction",
        "dlf:BatchCreatePartitions",
        "dlf:ListPartitionNames",
        "dlf:RenamePartition",
        "dlf:CreateTable",
        "dlf:BatchCreateTables",
        "dlf:UpdateTableColumnStatistics",
        "dlf:ListTableNames",
        "dlf:UpdateDatabase",
        "dlf:GetTableColumnStatistics",
        "dlf:ListFunctionNames",
        "dlf:ListPartitionsByFilter",
        "dlf:GetPartitionColumnStatistics",
        "dlf:CreatePartition",
        "dlf:CreateDatabase",
        "dlf:DeleteTableColumnStatistics",
        "dlf:ListTableVersions",
        "dlf:BatchDeletePartitions",
        "dlf:ListCatalogs",
        "dlf:UpdateTable",
        "dlf:ListTables",
        "dlf:DeleteDatabase",
        "dlf:BatchGetTables",
        "dlf:DeleteFunction"
      ],
      "Resource": "*",
      "Effect": "Allow"
    }
  ]
}

Attach the access policy to the RAM role For details, see Manage permissions for a RAM role.

Hologres STS-based service-linked role authorization

To let MaxCompute access Hologres data through external data sources and external projects, the MaxCompute service-linked role must have Hologres access permissions. This approach passes the task executor identity to Hologres, so both services restrict the current task executor to their respective permission scopes.

Create the MaxCompute service-linked role as follows:

In the RAM console, choose Identities > Roles in the left navigation pane.
On the Roles page, click Create Role.
In the upper-right corner of the Create Role page, click Create Service Linked Role.
On the Create Service Linked Role page, select aliyunserviceroleformaxcomputeidentitymgmt for Select Service and click Create Service Linked Role. If the system shows that the role already exists, skip this step.

Paimon_DLF STS-based service-linked role authorization

To let MaxCompute access Paimon_DLF data through external data sources and external projects, the MaxCompute service-linked role must have DLF access permissions. This approach passes the task executor identity to DLF, so both services restrict the current task executor to their respective permission scopes.

Create the MaxCompute service-linked role as follows:

In the RAM console, choose Identities > Roles in the left navigation pane.
On the Roles page, click Create Role.
In the upper-right corner of the Create Role page, click Create Service Linked Role.
On the Create Service Linked Role page, select AliyunServiceRoleForMaxComputeLakehouse for Select Service and click Create Service Linked Role. If the system shows that the role already exists, skip this step.