Access data by using a custom role - Log Service - Alibaba Cloud Documentation Center

You can assign a custom role to a data transformation task to read data from a source Logstore and write transformed data to one or more destination Logstores. This topic describes how to grant access permissions on Logstores to a custom role.

Prerequisites

A Resource Access Management (RAM) role is created. For more information, see Create a RAM role for a trusted Alibaba Cloud service.

Grant the RAM role the permissions to read data from a source Logstore

After you use an Alibaba Cloud account to authorize the RAM role, the RAM role has permissions to read data from the source Logstore. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation job.

Log on to the RAM console by using your Alibaba Cloud account.

Create a policy.

The policy is used to allow the RAM role to read data from a source Logstore.

In the left-side navigation pane, choose Permissions > Policies.
On the Policies page, click Create Policy.

On the Create Policy page, configure the following information and click OK.

Configuration item Description

Configuration mode Click JSON.

Policy document

Replace the content in the editor with the following script.

Policy that uses exact match

In this example, the source project name is log-project-prod, and the source Logstore name is access_log. Replace the project and Logstore names based on your business requirements.

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-prod/logstore/access_log",
        "acs:log:*:*:project/log-project-prod/logstore/access_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}

Policy that uses fuzzy match

In this example, the source project name can be log-project-dev-a, log-project-dev-b, or log-project-dev-c, and the source Logstore name can be app_a_log, app_b_log, or app_c_log. Replace the project and Logstore names based on your business requirements.

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
    "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}

For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.

Policy name Enter a name for the policy. In this example, enter log-etl-source-reader-1-policy.

Attach the policy to the RAM role.
1. In the left-side navigation pane, choose Identities > Roles.
2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
3. In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-source-reader-1-policy.
4. Confirm the authorization result and click Complete.
Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logsource.

Grant the RAM role the permissions to write data to destination Logstores within the same Alibaba Cloud account

If the source and destination Logstores belong to the same Alibaba Cloud account, you can use an Alibaba Cloud account to authorize the RAM role. Then, the RAM role has the permissions to write transformed data to the destination Logstores. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation job.

Log on to the RAM console by using your Alibaba Cloud account.

Create a policy.

In the left-side navigation pane, choose Permissions > Policies.
On the Policies page, click Create Policy.

On the Create Policy page, configure the following information and click OK.


Configuration item	Description
Configuration mode	Click JSON.
Policy document	Replace the content in the editor with the following script. Policy that uses exact match In this example, the destination project name is log-project-prod, and the destination Logstore name is access_log_output. Replace the project and Logstore names based on your business requirements. `{ "Version": "1", "Statement": [ { "Action": [ "log:Post", "log:BatchPost" ], "Resource": "acs:log:::project/log-project-prod/logstore/access_log_output", "Effect": "Allow" } ] }` Policy that uses fuzzy match In this example, the destination project name can be log-project-dev-a, log-project-dev-b, or log-project-dev-c, and the destination Logstore name can be app_a_log_output, app_b_log_output, or app_c_log_output. Replace the project and Logstore names based on your business requirements. `{ "Version": "1", "Statement": [ { "Action": [ "log:Post", "log:BatchPost" ], "Resource": "acs:log:::project/log-project-dev-/logstore/app__log_output", "Effect": "Allow" } ] }` For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
Policy name	Enter a name for the policy. In this example, enter log-etl-target-writer-1-policy.

Attach the policy to the RAM role.
1. In the left-side navigation pane, choose Identities > Roles.
2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
3. In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-target-writer-1-policy.
4. Confirm the authorization result and click Complete.
Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.

Grant the RAM role the permissions to write data to destination Logstores across Alibaba Cloud accounts

If the source and destination Logstores belong to different Alibaba Cloud accounts, perform the following steps to grant permissions to the RAM role. For example, a data transformation task is created to read data from a source Logstore that belongs to Alibaba Cloud Account A and write transformed data to a destination Logstore that belongs to Alibaba Cloud Account B.

Important Before you perform the following steps, you must use Alibaba Cloud Account B to authorize the RAM role to access destination Logstores within the same Alibaba Cloud account. For more information, see Grant the RAM role the permissions to write data to destination Logstores within the same Alibaba Cloud account.

Use Alibaba Cloud Account B to log on to the RAM console.
In the left-side navigation pane, choose Identities > Roles.
On the Roles page, click the name of the RAM role.
On the page that appears, click the Trust Policy Management tab. Then, click Edit Trust Policy.
Modify the policy.
Add ID of Alibaba Cloud Account A to which the source Logstore belongs to the Service element. Replace ID of Alibaba Cloud Account A to which the source Logstore belongs with the ID of your Alibaba Cloud account. You can view the ID of your Alibaba Cloud account in the Account Management console. The following policy allows Alibaba Cloud Account A to obtain a temporary token to manage the cloud resources of Alibaba Cloud Account B.
```
{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "ID of Alibaba Cloud Account A to which the source Logstore belongs@log.aliyuncs.com"
                ]
            }
        }
    ],
    "Version": "1"
}
```
Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.

What to do next

When you create a data transformation task, specify the ARN of the RAM role. For more information, see Create a data transformation job.

In Section 1, enter the ARN of the RAM role that has permissions to read data from a source Logstore. For more information, see Grant the RAM role the permissions to read data from a source Logstore.
In Section 2, enter the ARN of the RAM role that has permissions to write data to destination Logstores. For more information, see Grant the RAM role the permissions to write data to destination Logstores within the same Alibaba Cloud account or Grant the RAM role the permissions to write data to destination Logstores across Alibaba Cloud accounts.