You can authorize a data transformation task to assume a custom role to read data from a source Logstore and write transformed data to one or more destination Logstores. This topic describes how to grant access permissions on Logstores to a custom role.

Prerequisites

A Resource Access Management (RAM) role is created. For more information, see Create a RAM role for a trusted Alibaba Cloud service.

Grant the RAM role the permissions to read from a source Logstore

After the RAM role is authorized by an Alibaba Cloud account, the RAM role has permissions to read from the source Logstore. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation task.

  1. Log on to the RAM console by using an Alibaba Cloud account.
  2. Create a policy.
    The policy is used to allow the RAM role to read data from a source Logstore.
    1. In the left-side navigation pane, choose Permissions > Policies.
    2. On the Policies page, click Create Policy.
    3. On the Create Custom Policy page, configure the following parameters and click OK.
      Parameter Description
      Policy Name The name of the policy. In this example, enter log-etl-source-reader-1-policy.
      Configuration Mode Select Script.
      Policy Document The content of the policy. Replace the content in the editor with one of the following scripts based on your business requirements.
      • Policy that uses exact match
        The source project name is log-project-prod. The source Logstore name is access_log. Replace the project and Logstore names based on your business requirements.
        {
          "Version": "1",
          "Statement": [
            {
              "Action": [
                "log:ListShards",
                "log:GetCursorOrData",
                "log:GetConsumerGroupCheckPoint",
                "log:UpdateConsumerGroup",
                "log:ConsumerGroupHeartBeat",
                "log:ConsumerGroupUpdateCheckPoint",
                "log:ListConsumerGroup",
                "log:CreateConsumerGroup"
              ],
              "Resource": [
                "acs:log:*:*:project/log-project-prod/logstore/access_log",
                "acs:log:*:*:project/log-project-prod/logstore/access_log/*"
              ],
              "Effect": "Allow"
            }
          ]
        }
      • Policy that uses fuzzy match
        The source project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c. The source Logstore names can be app_a_log, app_b_log, or app_c_log. Replace the project and Logstore names based on your business requirements.
        {
          "Version": "1",
          "Statement": [
            {
              "Action": [
                "log:ListShards",
                "log:GetCursorOrData",
                "log:GetConsumerGroupCheckPoint",
                "log:UpdateConsumerGroup",
                "log:ConsumerGroupHeartBeat",
                "log:ConsumerGroupUpdateCheckPoint",
                "log:ListConsumerGroup",
                "log:CreateConsumerGroup"
              ],
              "Resource": [
                "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
            "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
              ],
              "Effect": "Allow"
            }
          ]
        }
        For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
  3. Attach the policy to the RAM role.
    1. In the left-side navigation pane, click RAM Roles.
    2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
    3. In the Select Policy section, click the Custom Policy tab. From the list of custom policies, click the policy that you created in Step 2 and click OK. In this example, the policy is log-etl-source-reader-1-policy.
      Add permissions
    4. Confirm the authorization results. Then, click Complete.
  4. Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role.
    You can obtain the ARN in the Basic Information section on the details page of the role. Example: acs:ram::13234:role/logrole.

Grant the RAM role the permissions to write to destination Logstores within the same Alibaba Cloud account

If the source and destination Logstores belong to the same Alibaba Cloud account, you can follow the instructions provided in this section to grant the permissions. After the RAM role is authorized by an Alibaba Cloud account, the RAM role has permissions to write to the destination Logstores. When you create a data transformation task, you can use the RAM role. For more information, see Create a data transformation task.

  1. Log on to the RAM console by using an Alibaba Cloud account.
  2. Create a policy.
    1. In the left-side navigation pane, choose Permissions > Policies.
    2. On the Policies page, click Create Policy.
    3. On the Create Custom Policy page, configure the following parameters and click OK.
      Parameter Description
      Policy Name The name of the policy. In this example, enter log-etl-target-writer-1-policy.
      Configuration Mode Select Script.
      Policy Document The content of the policy. Replace the content in the editor with one of the following scripts based on your business requirements.
      • Policy that uses exact match
        The destination project name is log-project-prod. The destination Logstore name is access_log_output. Replace the project and Logstore names based on your business requirements.
        {
          "Version": "1",
          "Statement": [
            {
              "Action": [
                "log:Post*",
                "log:BatchPost*"
              ],
               "Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",
              "Effect": "Allow"
            }
          ]
        }
      • Policy that uses fuzzy match
        The destination project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c. The destination Logstore names can be app_a_log_output, app_b_log_output, or app_c_log_output. Replace the project and Logstore names based on your business requirements.
        {
          "Version": "1",
          "Statement": [
            {
              "Action": [
                "log:Post*",
                "log:BatchPost*"
              ],
               "Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output",
              "Effect": "Allow"
            }
          ]
        }
        For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
  3. Attach the policy to the RAM user.
    1. In the left-side navigation pane, click RAM Roles.
    2. On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
    3. In the Select Policy section, click the Custom Policy tab. From the list of custom policies, click the policy that you created in 2 and click OK. In this example, the policy is log-etl-target-writer-1-policy.
      Write permissions
    4. Confirm the authorization results. Then, click Complete.
  4. Obtain the ARN of the RAM role.
    You can obtain the ARN in the Basic Information section on the details page of the role. Example: acs:ram::13234:role/logrole.

Grant the RAM role the permissions to write to destination Logstores within a different Alibaba Cloud account

If the source and destination Logstores belong to different Alibaba Cloud accounts, you can follow the instructions provided in this section to grant the permissions. The following procedure describes how to authorize a data transformation task to read data from a source Logstore within an Alibaba Cloud account (such as, Account A) and write transformed data to a destination Logstore within a different Alibaba Cloud account (such as, Account B).

Notice Before you perform the following procedure, make sure that you have completed the operations in the preceding section for Account B. For more information, see Grant the RAM role the permissions to write to destination Logstores within the same Alibaba Cloud account.
  1. Log on to the RAM console by using Account B.
  2. In the left-side navigation pane, click RAM Roles.
  3. In the RAM Role Name column, find the RAM role to which the destination Logstore belongs and click the role.
  4. On the page that appears, click the Trust Policy Management tab. On this tab, click Edit Trust Policy.
  5. Modify the trust policy.

    Add ID of Account A to which the source Logstore belongs to the Service field. You can replace the ID of Account A to which the source Logstore belongs with the actual ID. The following policy indicates that Account A is authorized to manage the cloud resources of Account B by using a temporary token.

    You can go to the Security Settings page to view the ID of an Alibaba Cloud account.

    {
        "Statement": [
            {
                "Action": "sts:AssumeRole",
                "Effect": "Allow",
                "Principal": {
                    "Service": [
                        "ID of Account A to which the source Logstore belongs@log.aliyuncs.com"
                    ]
                }
            }
        ],
        "Version": "1"
    }
  6. Obtain the ARN of the RAM role.
    You can obtain the ARN in the Basic Information section on the details page of the role. Example: acs:ram::13234:role/logrole.

What to do next

You can enter the ARN of the RAM role in a data transformation task. For more information, see Create a data transformation task. The ARN of a RAM role