When you run an OSS-HDFS data shipping job, the job reads data from a logstore and ships the data to an Object Storage Service (OSS) bucket. You can authorize the job to assume a custom Resource Access Management (RAM) role, which can access the required data. If the logstore and OSS bucket belong to different Alibaba Cloud accounts, perform the steps described in this topic.
Step 1: Grant the RAM role role-a in Alibaba Cloud Account A the permissions to read data from the logstore
After doing so, the OSS-HDFS data shipping job can assume the role to access the logstore in Account A. The details of this step are as follows:
Log on to the RAM console using Alibaba Cloud Account A.
Create a custom policy that grants the permissions to read data from a Logstore.
You can use a policy document that uses exact match or fuzzy match for authorization.
Exact match for authorization
On the Create Policy page, click the JSON tab. Replace the existing contents in the editor with the following script. For more information, see Create a custom policy on the JSON tab.
ImportantReplace
Project nameandLogstore namein the policy document based on your business requirements.{ "Version":"1", "Statement":[ { "Action":[ "log:GetCursorOrData", "log:ListShards" ], "Resource":[ "acs:log:*:*:project/Project name/logstore/Logstore name" ], "Effect":"Allow" } ] }Fuzzy match for authorization
On the Create Policy page, click the JSON tab. Replace the existing contents in the editor with the following script. For more information, see Create a custom policy on the JSON tab.
ImportantIn this example, the names of the projects are log-project-dev-a, log-project-dev-b, and log-project-dev-c, and the names of the Logstores are website_a_log, website_b_log, and website_c_log.
Replace
log-project-dev-*andwebsite_*_log*in the policy document based on your business requirements.
{ "Version":"1", "Statement":[ { "Action":[ "log:GetCursorOrData", "log:ListShards" ], "Resource":[ "acs:log:*:*:project/log-project-dev-*/logstore/website_*_log*" ], "Effect":"Allow" } ] }Create the RAM role
role-arequired by the OSS-HDFS data shipping job. For more information, see Create a RAM role for a trusted Alibaba Cloud service.ImportantWhen you create a RAM role, you must set the Select Trusted Entity parameter to Alibaba Cloud Service and the Select Trusted Service parameter to Log Service.
Check the trust policy of the RAM role. Make sure that the
Serviceelement contains at least"log.aliyuncs.com".{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "log.aliyuncs.com" ] } } ], "Version": "1" }
Attach the created custom policy to the
role-aRAM role. For more information, see Grant permissions to a RAM role.
What to do next
Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. For more information, see View the information about a RAM role.
When you create an OSS-HDFS data shipping job using a custom RAM role, you must enter the ARN in Logstore Read RAM Role. For more information, see Create an OSS-HDFS data shipping job.
Step 2: Grant the RAM role role-b in Alibaba Cloud Account B the permissions to write data to the OSS bucket
After doing so, the OSS data shipping job can ship data from the logstore in Alibaba Cloud Account A to the OSS bucket in Alibaba Cloud Account B. The details of this step are as follows:
Log on to the RAM console using Alibaba Cloud Account B.
Create a policy that grants the permissions to write data to an OSS bucket.
On the Create Policy page, click the JSON tab. Replace the existing contents in the editor with the following script. For more information, see Create a custom policy on the JSON tab.
{ "Statement": [ { "Action": [ "oss:PutObject" ], "Resource": "*", "Effect": "Allow" }, { "Effect": "Allow", "Action": "oss:ListObjects", "Resource": [ "acs:oss:*:*:*" ] }, { "Effect": "Allow", "Action": [ "oss:GetBucketInfo", "oss:PostDataLakeStorageFileOperation", "oss:PostDataLakeStorageAdminOperation" ], "Resource": "*" }, { "Effect": "Allow", "Action": "oss:*", "Resource": [ "acs:oss:*:*:*/.dlsdata", "acs:oss:*:*:*/.dlsdata*" ] } ], "Version": "1" }Create the RAM role
role-brequired by the OSS-HDFS data shipping job. For more information, see Create a RAM role for a trusted Alibaba Cloud service.ImportantWhen you create a RAM role, you must set the Select Trusted Entity parameter to Alibaba Cloud Service and the Select Trusted Service parameter to Log Service.
Check the trust policy of the RAM role. Make sure that the
Serviceelement contains at least"log.aliyuncs.com".{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "log.aliyuncs.com" ] } } ], "Version": "1" }
Modify the trust policy of the
role-bRAM role. For more information, see Edit the trust policy of a RAM role.ImportantAdd ID of Alibaba Cloud Account A@log.aliyuncs.com to the Service element. Replace the ID of Alibaba Cloud Account A with the actual ID. You can view the ID of your Alibaba Cloud account in the Account Center console.
The following policy allows Alibaba Cloud Account A to obtain a Security Token Service (STS) token to manage the cloud resources of Alibaba Cloud Account B.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "log.aliyuncs.com", "ID of Alibaba Cloud Account A@log.aliyuncs.com" ] } } ], "Version": "1" }Attach the created custom policy to the
role-bRAM role. For more information, see Grant permissions to a RAM role.
What to do next
Obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. For more information, see View the information about a RAM role.
When you create an OSS-HDFS data shipping job with a custom RAM role, you must enter the ARN in OSS-HDFS Write RAM Role. For more information, see Create an OSS-HDFS data shipping job.