You can assign a custom role to a data transformation task to read data from a source Logstore and write transformed data to one or more destination Logstores. This topic describes how to grant access permissions on Logstores to a custom role.

Prerequisites

A Resource Access Management (RAM) role is created. For more information, see Create a RAM role for a trusted Alibaba Cloud service.

Grant the RAM role the permissions to read data from a source Logstore

After you use an Alibaba Cloud account to authorize the RAM role, the RAM role has permissions to read data from the source Logstore. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation task.

  1. Log on to the RAM console by using an Alibaba Cloud account.
  2. Create a policy.
    The policy is used to allow the RAM role to read data from a source Logstore.
    1. In the left-side navigation pane, choose Permissions > Policies.
    2. On the Policies page, click Create Policy.
    3. On the Create Custom Policy page, configure the following parameters and click OK.
      Parameter Description
      Policy Name The name of the policy. In this example, enter log-etl-source-reader-1-policy.
      Configuration Mode Select Script.
      Policy Document The content of the policy. Replace the content in the editor with one of the following scripts based on your business requirements.
      • Policy that uses exact match
        The source project name is log-project-prod. The source Logstore name is access_log. Replace the project and Logstore names based on your business requirements. 
        {
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-prod/logstore/access_log",
        "acs:log:*:*:project/log-project-prod/logstore/access_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}
      • Policy that uses fuzzy match
        The source project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c. The source Logstore names can be app_a_log, app_b_log, or app_c_log. Replace the project and Logstore names based on your business requirements. 
        {
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
    "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}
        For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
  3. Attach the policy to the RAM role.
    1. In the left-side navigation pane, choose Identities > Roles.
    2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
    3. In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-source-reader-1-policy.
      Add permissions
    4. Confirm the authorization result and click Complete.
  4. Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role.
    In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logsource.

Grant the RAM role the permissions to write data to destination Logstores within the same Alibaba Cloud account

If the source and destination Logstores belong to the same Alibaba Cloud account, you can use an Alibaba Cloud account to authorize the RAM role. Then, the RAM role has the permissions to write transformed data to the destination Logstores. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation task.

  1. Log on to the RAM console by using an Alibaba Cloud account.
  2. Create a policy.
    1. In the left-side navigation pane, choose Permissions > Policies.
    2. On the Policies page, click Create Policy.
    3. On the Create Custom Policy page, configure the following parameters and click OK.
      Parameter Description
      Policy Name The name of the policy. In this example, enter log-etl-target-writer-1-policy.
      Configuration Mode Select Script.
      Policy Document The content of the policy. Replace the content in the editor with one of the following scripts based on your business requirements.
      • Policy that uses exact match
        The destination project name is log-project-prod. The destination Logstore name is access_log_output. Replace the project and Logstore names based on your business requirements. 
        {
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:Post*",
        "log:BatchPost*"
      ],
       "Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",
      "Effect": "Allow"
    }
  ]
}
      • Policy that uses fuzzy match
        The destination project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c. The destination Logstore names can be app_a_log_output, app_b_log_output, or app_c_log_output. Replace the project and Logstore names based on your business requirements. 
        {
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:Post*",
        "log:BatchPost*"
      ],
       "Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output",
      "Effect": "Allow"
    }
  ]
}
        For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
  3. Attach the policy to the RAM role.
    1. In the left-side navigation pane, choose Identities > Roles.
    2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
    3. In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-target-writer-1-policy.
      Write permissions
    4. Confirm the authorization result and click Complete.
  4. Obtain the ARN of the RAM role.
    In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.

Grant the RAM role the permissions to write data to destination Logstores across Alibaba Cloud accounts

If the source and destination Logstores belong to different Alibaba Cloud accounts, perform the following steps to grant permissions to the RAM role. For example, a data transformation task is created to read data from a source Logstore that belongs to Alibaba Cloud Account A and write transformed data to a destination Logstore that belongs to Alibaba Cloud Account B.

Notice Before you perform the following steps, you must use Alibaba Cloud Account B to authorize the RAM role to access destination Logstores within the same Alibaba Cloud account. For more information, see Grant the RAM role the permissions to write data to destination Logstores within the same Alibaba Cloud account.
  1. Use Alibaba Cloud Account B to log on to the RAM console.
  2. In the left-side navigation pane, choose Identities > Roles.
  3. On the Roles page, click the name of the RAM role.
  4. On the page that appears, click the Trust Policy Management tab. Then, click Edit Trust Policy.
  5. Modify the policy.

    Add ID of Alibaba Cloud Account A to which the source Logstore belongs to the Service element. Replace ID of Alibaba Cloud Account A to which the source Logstore belongs with the ID of your Alibaba Cloud account. You can view the ID of your Alibaba Cloud account in the Account Management console. The following policy allows Alibaba Cloud Account A to obtain a temporary token to manage the cloud resources of Alibaba Cloud Account B. 

    {
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "ID of Alibaba Cloud Account A to which the source Logstore belongs@log.aliyuncs.com"
                ]
            }
        }
    ],
    "Version": "1"
}
  6. Obtain the ARN of the RAM role.
    In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.

What to do next

When you create a data transformation task, specify the ARN of the RAM role. For more information, see Create a data transformation task. ARN of a RAM role