You can assign a custom role to a data transformation task to read data from a source
Logstore and write transformed data to one or more destination Logstores. This topic
describes how to grant access permissions on Logstores to a custom role.
Grant the RAM role the permissions to read data from a source Logstore
After you use an Alibaba Cloud account to authorize the RAM role, the RAM role has
permissions to read data from the source Logstore. When you create a data transformation
task, you can use the RAM role. For more information, see Create a data transformation task.
- Log on to the RAM console by using an Alibaba Cloud account.
- Create a policy.
The policy is used to allow the RAM role to read data from a source Logstore.
- In the left-side navigation pane, choose .
- On the Policies page, click Create Policy.
- On the Create Custom Policy page, configure the following parameters and click OK.
Parameter |
Description |
Policy Name |
The name of the policy. In this example, enter log-etl-source-reader-1-policy.
|
Configuration Mode |
Select Script.
|
Policy Document |
The content of the policy. Replace the content in the editor with one of the following
scripts based on your business requirements.
- Policy that uses exact match
The source project name is log-project-prod. The source Logstore name is access_log.
Replace the project and Logstore names based on your business requirements. {
"Version": "1",
"Statement": [
{
"Action": [
"log:ListShards",
"log:GetCursorOrData",
"log:GetConsumerGroupCheckPoint",
"log:UpdateConsumerGroup",
"log:ConsumerGroupHeartBeat",
"log:ConsumerGroupUpdateCheckPoint",
"log:ListConsumerGroup",
"log:CreateConsumerGroup"
],
"Resource": [
"acs:log:*:*:project/log-project-prod/logstore/access_log",
"acs:log:*:*:project/log-project-prod/logstore/access_log/*"
],
"Effect": "Allow"
}
]
}
- Policy that uses fuzzy match
The source project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c.
The source Logstore names can be app_a_log, app_b_log, or app_c_log. Replace the project
and Logstore names based on your business requirements. {
"Version": "1",
"Statement": [
{
"Action": [
"log:ListShards",
"log:GetCursorOrData",
"log:GetConsumerGroupCheckPoint",
"log:UpdateConsumerGroup",
"log:ConsumerGroupHeartBeat",
"log:ConsumerGroupUpdateCheckPoint",
"log:ListConsumerGroup",
"log:CreateConsumerGroup"
],
"Resource": [
"acs:log:*:*:project/log-project-dev-*/logstore/app_*_log",
"acs:log:*:*:project/log-project-dev-*/logstore/app_*_log/*"
],
"Effect": "Allow"
}
]
}
For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
|
- Attach the policy to the RAM role.
- In the left-side navigation pane, choose .
- On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
- In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-source-reader-1-policy.
- Confirm the authorization result and click Complete.
- Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logsource.
Grant the RAM role the permissions to write data to destination Logstores within the
same Alibaba Cloud account
If the source and destination Logstores belong to the same Alibaba Cloud account,
you can use an Alibaba Cloud account to authorize the RAM role. Then, the RAM role
has the permissions to write transformed data to the destination Logstores. When you
create a data transformation task, you can use the RAM role. For more information,
see Create a data transformation task.
- Log on to the RAM console by using an Alibaba Cloud account.
- Create a policy.
- In the left-side navigation pane, choose .
- On the Policies page, click Create Policy.
- On the Create Custom Policy page, configure the following parameters and click OK.
Parameter |
Description |
Policy Name |
The name of the policy. In this example, enter log-etl-target-writer-1-policy.
|
Configuration Mode |
Select Script.
|
Policy Document |
The content of the policy. Replace the content in the editor with one of the following
scripts based on your business requirements.
- Policy that uses exact match
The destination project name is log-project-prod. The destination Logstore name is
access_log_output. Replace the project and Logstore names based on your business requirements.
{
"Version": "1",
"Statement": [
{
"Action": [
"log:Post*",
"log:BatchPost*"
],
"Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",
"Effect": "Allow"
}
]
}
- Policy that uses fuzzy match
The destination project names can be log-project-dev-a, log-project-dev-b, or log-project-dev-c.
The destination Logstore names can be app_a_log_output, app_b_log_output, or app_c_log_output.
Replace the project and Logstore names based on your business requirements. {
"Version": "1",
"Statement": [
{
"Action": [
"log:Post*",
"log:BatchPost*"
],
"Resource": "acs:log:*:*:project/log-project-dev-*/logstore/app_*_log_output",
"Effect": "Allow"
}
]
}
For more information about authorization scenarios, see Use custom policies to grant permissions to a RAM user.
|
- Attach the policy to the RAM role.
- In the left-side navigation pane, choose .
- On the RAM Roles page, find the RAM role and click Add Permissions in the Actions column.
- In the Select Policy section, click Custom Policy, select the policy that you created in Step 2, and then click OK. In this example, the policy is log-etl-target-writer-1-policy.
- Confirm the authorization result and click Complete.
- Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.
Grant the RAM role the permissions to write data to destination Logstores across Alibaba
Cloud accounts
If the source and destination Logstores belong to different Alibaba Cloud accounts,
perform the following steps to grant permissions to the RAM role. For example, a data
transformation task is created to read data from a source Logstore that belongs to
Alibaba Cloud Account A and write transformed data to a destination Logstore that
belongs to Alibaba Cloud Account B.
- Use Alibaba Cloud Account B to log on to the RAM console.
- In the left-side navigation pane, choose .
- On the Roles page, click the name of the RAM role.
- On the page that appears, click the Trust Policy Management tab. Then, click Edit Trust Policy.
- Modify the policy.
Add ID of Alibaba Cloud Account A to which the source Logstore belongs to the Service element. Replace ID of Alibaba Cloud Account A to which the source Logstore belongs with the ID of your Alibaba Cloud account. You can view the ID of your Alibaba Cloud
account in the Account Management console. The following policy allows Alibaba Cloud Account A to obtain a temporary
token to manage the cloud resources of Alibaba Cloud Account B.
{
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": [
"ID of Alibaba Cloud Account A to which the source Logstore belongs@log.aliyuncs.com"
]
}
}
],
"Version": "1"
}
- Obtain the ARN of the RAM role.
In the Basic Information section of the RAM role, obtain the ARN. Example: acs:ram::13234:role/logtarget.
What to do next
When you create a data transformation task, specify the ARN of the RAM role. For more
information, see Create a data transformation task.