Scenario 3: Use custom roles to transfer data across different Alibaba Cloud accounts - Simple Log Service

When you use a Resource Access Management (RAM) user to create a data transformation job, you can specify custom roles to transfer data across different Alibaba Cloud accounts.

Prerequisites

A project is created. A source Logstore and a destination Logstore are created in the project. The names of the source and destination Logstores and the name of the project are obtained. For more information, see Manage a Logstore and Manage a project.
A RAM user is created, and the RAM user is granted the required permissions to perform data transformation operations. For more information, see Grant a RAM user the permissions to manage a data transformation job.

Background information

In a cross-account scenario, a RAM user is used to create a data transformation job. Role A must have the read permissions on the source Logstore, and Role B must have the write permissions on the destination Logstores. The data transformation job is run within Alibaba Cloud Account 1. Therefore, you must modify the trust policy of Role B within Alibaba Cloud Account 2 to allow Alibaba Cloud Account 1 to assume Role B.

Step 1: Use Alibaba Cloud Account 1 to create Role A

Log on to the RAM console by using Alibaba Cloud Account 1.

Create Role A.

For more information, see Create a RAM role for a trusted Alibaba Cloud service. The following table describes the key parameters.

Parameter	Description
Select Trusted Entity	Select Alibaba Cloud Service.
Role Type	Select Normal Service Role.
RAM Role Name	Enter a name for the role. In this example, enter role-A.
Select Trusted Service	Select Log Service.

Step 2: Use Alibaba Cloud Account 1 to grant the read permissions to Role A

Log on to the RAM console by using Alibaba Cloud Account 1.

Create a custom policy on the JSON tab. The policy grants the permissions to read data from the source Logstore. In this example, create a policy named ori_read.

For more information, see Create a custom policy. The following table describes the key parameters.

Parameter

Description

Name

Enter a name for the custom policy. In this example, enter ori_read.

Policy document

Replace the content in the code editor with the following script.

In this example, the source project is named log-project-prod, and the source Logstore is named access_log. Replace the project and Logstore names based on your business requirements.

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListShards",
        "log:GetCursorOrData",
        "log:GetConsumerGroupCheckPoint",
        "log:UpdateConsumerGroup",
        "log:ConsumerGroupHeartBeat",
        "log:ConsumerGroupUpdateCheckPoint",
        "log:ListConsumerGroup",
        "log:CreateConsumerGroup"
      ],
      "Resource": [
        "acs:log:*:*:project/log-project-prod/logstore/access_log",
        "acs:log:*:*:project/log-project-prod/logstore/access_log/*"
      ],
      "Effect": "Allow"
    }
  ]
}

Grant the read permissions on the source Logstore to Role A.

For more information, see Grant permissions to a RAM role. The following table describes the key parameters.

Key parameter	Description
Authorized Scope	Select Alibaba Cloud Account. The authorization takes effect within the current Alibaba Cloud account.
Principal	Select role-A. role-A is the name of Role A that is created in Step 1: Use Alibaba Cloud Account 1 to create Role A.
Custom Policy	Select ori_read.

Obtain the Alibaba Cloud Resource Name (ARN) of Role A.
In the Basic Information section on the details page of Role A, obtain the ARN of the role. Example: acs:ram::1379******44:role/role-a.

Step 3: Use Alibaba Cloud Account 2 to create Role B

Log on to the RAM console by using Alibaba Cloud Account 2.

Create Role B.

For more information, see Create a RAM role for a trusted Alibaba Cloud service. The following table describes the key parameters.

Parameter	Description
Select Trusted Entity	Select Alibaba Cloud Service.
Role Type	Select Normal Service Role.
RAM Role Name	Enter a name for the role. In this example, enter role-B.
Select Trusted Service	Select Log Service.

Step 4: Use Alibaba Cloud Account 2 to grant the write permissions to Role B

Log on to the RAM console by using Alibaba Cloud Account 2.

Create a custom policy on the JSON tab. The policy grants the permissions to write data to the destination Logstores. In this example, create a policy named write.

For more information, see Create a custom policy. The following table describes the key parameters.

Parameter	Description
Name	Enter a name for the custom policy. In this example, enter write.
Policy document	Replace the content in the code editor with the following script. In this example, the destination project is named log-project-prod, and the destination Logstore is named access_log_output. Replace the project and Logstore names based on your business requirements. `{ "Version": "1", "Statement": [ { "Action": [ "log:Post", "log:BatchPost" ], "Resource": "acs:log:::project/log-project-prod/logstore/access_log_output", "Effect": "Allow" } ] }`

Grant the write permissions on the destination Logstore to Role B.

For more information, see Grant permissions to a RAM role. The following table describes the key parameters.

Key parameter	Description
Authorized Scope	Select Alibaba Cloud Account. The authorization takes effect within the current Alibaba Cloud account.
Principal	Select role-B. role-B is the name of Role B that is created in Step 3: Use Alibaba Cloud Account 2 to create Role B.
Custom Policy	Select write.

Obtain the ARN of Role B.
In the Basic Information section on the details page of Role B, obtain the ARN of the role. Example: acs:ram::1440******32:role/role-b.

Step 5: Modify the trust policy of Role B within Alibaba Cloud Account 2 to allow Alibaba Cloud Account 1 to assume Role B

Log on to the RAM console by using Alibaba Cloud Account 2.
In the left-side navigation pane, choose Identities > Roles.
In the RAM role list, click the RAM role that you want to manage. In this example, click role-B.
On the Trust Policy Management tab, click Edit Trust Policy.
Modify the trust policy.
Add ID of Alibaba Cloud Account 1 to Service. Replace ID of Alibaba Cloud Account 1 based on your business requirements. You can view the ID of your Alibaba Cloud account in the Account Management console. The following policy allows Alibaba Cloud Account 1 to obtain a Security Token Service (STS) token by using Simple Log Service to manage the cloud resources of Alibaba Cloud Account 2:
```
{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "ID of Alibaba Cloud Account 1@log.aliyuncs.com"
                ]
            }
        }
    ],
    "Version": "1"
}
```

Step 6: Use the RAM user to create a data transformation job

Log on to the Log Service console by using the RAM user.
Go to the data transformation page.
1. In the Projects section, click the project that you want to manage.
2. On the Log Storage > Logstores tab, click the Logstore that you want to manage.
3. On the query and analysis page, click Data Transformation.
In the upper-right corner of the page, specify a time range for the log data that you want to transform.
Make sure that log data exists on the Raw Logs tab.
In the code editor, enter a data transformation statement.
For more information, see Data transformation syntax.
Preview transformation results.
1. Click Quick.
  You can select Quick or Advanced. For more information, see Preview mode overview.
2. Click Preview Data.
  View the preview results.
  - If data fails to be transformed because the specified statement is invalid or the permissions that are granted are invalid, follow the on-screen instructions to troubleshoot the failure.
  - If the transformed data is returned as expected, go to the next step.

Create a data transformation job.

Click Save as Transformation Rule.

In the Create Data Transformation Rule panel, configure the parameters and click OK.

For more information about the parameters, see Quick start. The following table describes the key parameters. custom role-accross account

Key parameter	Description
Authorization Method	Click Custom Role.
Role ARN	Enter the ARN of Role A. Example: `acs:ram::1379******44:role/role-a`.
Authorization Method in Storage Target	Click Custom Role.
Role ARN	Enter the ARN of Role B. Example: `acs:ram::1440******32:role/role-b`.

After the data transformation job is created and run, data can be transferred across the two Alibaba Cloud accounts. For more information, see Manage a data transformation job.