MaxCompute external table permissions
This topic covers how to access data in Object Storage Service (OSS) within the same account or across different accounts.
Scenarios
This topic shows you how to access OSS data in both same-account and cross-account scenarios, using three Alibaba Cloud accounts (A, B, and C).
Usage notes
-
Test permissions in a read-only environment to prevent data write errors caused by incorrect authorization.
-
When creating a custom role, carefully define the permission scope to prevent unintended permission leaks.
Sample data
|
Parameter |
Account A |
Account B |
Account C |
|
Account ID |
124 |
146 |
139 |
|
MaxCompute Project Name |
mc_policy_config_a |
mc_policy_config_b |
mc_policy_config_c |
|
Region |
China (Hangzhou) |
||
|
OSS Data Path |
|
Not applicable |
Not applicable |
|
Default Role Name (for OSS access) |
AliyunODPSDefaultRole |
Not applicable |
Not applicable |
|
Custom Role Name (for OSS access) |
role-a-to-a, role-a-to-b |
Not applicable |
Not applicable |
|
RAM User Name |
mc_user_a |
mc_user_b |
mc_user_c |
|
RAM Role Name |
ramrole-user-A |
ramrole-user-B |
ramrole-user-C |
|
Access Policy Name |
OdpsPolicy_A |
OdpsPolicy_B |
Not applicable |
Access OSS data in the same account
Account A can create a role or use the default role to create an external table to access its OSS data.
Prepare a role to access OSS (custom or default)
Default: AliyunODPSDefaultRole
When MaxCompute and OSS belong to the same Alibaba Cloud account, the AliyunODPSDefaultRole is automatically created to access OSS after you complete the steps in Method 1: STS Mode Authorization for OSS.
The default AliyunODPSDefaultRole has extensive permissions and is for reference only. You must configure appropriate access policies and trust policies in the RAM console based on the permissions required by the MaxCompute external table or external data source.
Custom role
If you need to access OSS using a custom role, use Alibaba Cloud account A to create a custom role and grant permissions to it. For detailed steps, see Create a custom role. The following table shows a configuration example.
|
Actions |
Parameter |
Example |
|
Create RAM role |
Trusted entity type |
Select Cloud Service. |
|
Trusted entity name |
Select MaxCompute. |
|
|
Role name |
role-a-to-a |
|
|
Modify RAM role trust policy |
Trust policy |
The following sample trust policy allows the MaxCompute service to assume this role.
|
|
Add access policy |
Access policy |
|
Account A accesses its own data
Alibaba Cloud account A uses the default role AliyunODPSDefaultRole or the custom role role-a-to-a to create an external table to access OSS data.
-
Log on to the MaxCompute client as Alibaba Cloud account A and create an OSS external table.
Create with default role
CREATE EXTERNAL TABLE IF NOT EXISTS external_defaultrole_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' -- If you use the default role and access OSS data under the same Alibaba Cloud account, the system automatically generates the following content. -- WITH serdeproperties ( -- 'odps.properties.rolearn'='acs:ram::124:role/aliyunodpsdefaultrole' -- ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a';Create with custom role
CREATE EXTERNAL TABLE IF NOT EXISTS mc_oss_csv_external_test_ramrole_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-a' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a'; -
Query data from the external table as Alibaba Cloud account A.
Query with default role
SELECT * FROM external_defaultrole_a;The query returns the following result:
+-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+Query with custom role
SELECT * FROM mc_oss_csv_external_test_ramrole_a;The query returns the following result:
+-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+
A RAM user of Account A accesses data
A RAM user of Account A uses the default role AliyunODPSDefaultRole or the custom role role-a-to-a to create an external table to access OSS data.
-
Use Alibaba Cloud account A to create a RAM user and grant the RAM user permissions on the MaxCompute project. The following table shows a configuration example.
Parameter
Example
RAM user
mc_user_a
MaxCompute project
mc_policy_config_a
-
Log on to the MaxCompute client as the RAM user of Account A and create an OSS external table.
Create with default role
CREATE EXTERNAL TABLE IF NOT EXISTS external_defaultrole_ramuser_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' -- If you use the default role and access OSS data under the same Alibaba Cloud account, the system automatically generates the following content. -- WITH serdeproperties ( -- 'odps.properties.rolearn'='acs:ram::124:role/aliyunodpsdefaultrole' -- ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a/';Create with custom role
CREATE EXTERNAL TABLE IF NOT EXISTS external_ramrole_ramuser_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-a' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a/'; -
Query data from the external table.
To run the queries shown in the following examples, log on to the MaxCompute client as the RAM user of Account A.
Query with default role
SELECT * FROM external_defaultrole_ramuser_a;The query returns the following result:
+-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+Query with custom role
SELECT * FROM external_ramrole_ramuser_a;The query returns the following result:
+-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +-----------+----------+-----------+-------+------------------+-------------------+----------------+-----------+
A RAM role of Account A accesses data
A RAM role of Account A uses the default role AliyunODPSDefaultRole or a custom role to create an external table to access OSS data.
-
Use Alibaba Cloud account A to create and log on with a RAM role. The following table shows a configuration example.
Parameter
Example
RAM role
ramrole-user-A
MaxCompute project
mc_policy_config_a
-
Use the RAM role of Account A to create an OSS external table.
Create with default role
CREATE EXTERNAL TABLE IF NOT EXISTS external_defaultrole_ramrole_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' -- If you use the default role and access OSS data under the same Alibaba Cloud account, the system automatically generates the following content. -- WITH serdeproperties ( -- 'odps.properties.rolearn'='acs:ram::124:role/aliyunodpsdefaultrole' -- ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a';Create with custom role
CREATE EXTERNAL TABLE IF NOT EXISTS external_role_ramrole_a ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-a' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a'; -
Query data from the OSS external table.
Query with default role
SELECT * FROM external_defaultrole_ramrole_a;The query returns the following result:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+Query with custom role
SELECT * FROM external_role_ramrole_a;The query returns the following result:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Cross-account access to OSS data
To set up this scenario, Account A must create a role and delegate permissions to Account B. Account B then uses this role to create an external table and access the OSS data in Account A.
Role to access OSS (custom roles only)
Use Account A to create a RAM role that trusts an Alibaba Cloud service, and then attach an access policy to the role. For the procedure, see Create a custom role. The following table shows a configuration example.
|
Actions |
Parameter |
Example |
|
Create a RAM role |
Trusted entity type |
Select Cloud Service. |
|
Trusted entity name |
Select MaxCompute. |
|
|
Role name |
role-a-to-b |
|
|
Modify RAM role trust policy |
Trust policy |
The following sample trust policy lets the MaxCompute service in Account B assume this role. In this example, 146 is the ID of Account B.
|
|
Add access policy |
Access policy |
|
Access data in Account A as Account B
Account B uses the custom role role-a-to-b to create an external table and access OSS data in Account A.
-
Log on to the MaxCompute client as Account B and use the custom role
role-a-to-bto create an OSS external table.CREATE EXTERNAL TABLE IF NOT EXISTS external_ramrole_b ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-b' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a'; -
Alibaba Cloud account B queries external table data by using the custom Role
role-a-to-b.SELECT * FROM external_ramrole_b;The following result is returned:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Access data as a RAM user
A RAM user of Account B uses the custom role role-a-to-b from the Preparing a role to access OSS (custom roles only) section to create an external table and access OSS data in Account A.
-
Use Account B to create a RAM user and grant permissions to the MaxCompute project. The following table shows a configuration example:
Parameter
Example
RAM user
mc_user_b
MaxCompute project
mc_policy_config_b
-
Log on to the MaxCompute client as the RAM user of Account B and use the custom role
role-a-to-bto create an OSS external table.CREATE EXTERNAL TABLE IF NOT EXISTS external_ramrole_ramuser_b ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-b' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a'; -
As the RAM user of Account B, query data from the external table.
SELECT * FROM external_ramrole_ramuser_b;The following result is returned:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Access data as a RAM role
A RAM role of Account B uses the custom role role-a-to-b from the Preparing a role to access OSS (custom roles only) section to create an external table and access OSS data in Account A.
-
Use Account B to create and log on with a RAM role. The following table shows a configuration example:
Parameter
Example
RAM role
ramrole-user-B
MaxCompute project
mc_policy_config_b
-
Use the RAM role of Account B to create an OSS external table.
CREATE EXTERNAL TABLE IF NOT EXISTS external_role_ramrole_b ( vehicleId INT, recordId INT, patientId INT, calls INT, locationLatitute DOUBLE, locationLongtitue DOUBLE, recordTime STRING, direction STRING ) STORED BY 'com.aliyun.odps.CsvStorageHandler' WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::124:role/role-a-to-b' ) LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/oss-mc-test/test-oss-a'; -
Query data from the OSS external table.
SELECT * FROM external_role_ramrole_b;The following result is returned:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Cross-account access to OSS data
Access by Alibaba Cloud account C
The general procedure is as follows:
-
Account B creates a package and shares the
external_ramrole_btable with Account C's project, granting read access. For more information, see Share resources across projects by using packages. -
Alibaba Cloud account C accesses the OSS data of Account A by querying the external table
external_ramrole_b, which is created by Account B as described in the Procedure.
-
Log on to the MaxCompute client as Alibaba Cloud account B, create a package, and add the external table
external_ramrole_bto the package.-- Create a package. CREATE PACKAGE test_ramrole; -- Add the external_ramrole_b external table created by Account B to the package. ADD TABLE external_ramrole_b TO PACKAGE test_ramrole; -
Log on to the MaxCompute client as Alibaba Cloud account B and run the following command to authorize Alibaba Cloud account C's project to install the package.
ALLOW PROJECT mc_policy_config_c TO INSTALL PACKAGE test_ramrole; -
Log on to the MaxCompute client as Alibaba Cloud account C and install the package created by Account B.
INSTALL PACKAGE mc_policy_config_b.test_ramrole; -
As Alibaba Cloud account C, read data from the external table in the package.
NoteIf the schema switch is enabled for Account C, you must use a three-part name to read the data.
SELECT * FROM mc_policy_config_b.default.external_ramrole_b;The command returns the following output:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Access by RAM user of Account C
-
Log on to the RAM console as Alibaba Cloud account C, create a RAM user, and create an AccessKey pair. You do not need to grant permissions to the RAM user in this step.
-
Log on to the MaxCompute client as Alibaba Cloud account C and grant the RAM user permissions to access the package. A sample configuration is provided below.
Parameter
Example
RAM user
mc_user_c
MaxCompute project
mc_policy_config_c
-- Add Account C's RAM user as a project member. ADD USER RAM$mc_user_c; -- Grant the CreateInstance permission on the project to Account C's RAM user. GRANT CreateInstance ON PROJECT mc_policy_config_c TO USER RAM$C_testcloud_com:mc_user_c; -- Grant the permission to access the package to Account C's RAM user. GRANT Read ON PACKAGE mc_policy_config_b.test_ramrole TO USER RAM$C_testcloud_com:mc_user_c; -
Log on to the MaxCompute client as the Account C's RAM user and read data from the external table in the package.
NoteIf the schema switch is enabled for Account C, you must use a three-part name to read the data.
SELECT * FROM mc_policy_config_b.default.external_ramrole_b;The command returns the following output:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
Access by RAM role of Account C
-
Use Alibaba Cloud account C to create and authorize a RAM role. For more information, see Create and log on with a RAM role. The following table shows an example.
Parameter
Example
RAM role
ramrole-user-C
MaxCompute project
mc_policy_config_c
-
As Alibaba Cloud account C, grant the RAM role permissions to access the package.
NoteIn the following statement,
xxx_testcloud_comis theaccount_namereturned by thewhoami;command.-- Grant the permission to access the package to Account C's RAM role. GRANT Read ON PACKAGE mc_policy_config_b.test_ramrole TO USER `RAM$xxx_testcloud_com:role/ramrole-user-C`; -
Log on to the MaxCompute client by using the RAM role created by Account C and read data from the external table in the package. For more information about how to log on, see Log on with a RAM role.
SELECT * FROM mc_policy_config_b.default.external_ramrole_b;The command returns the following output:
+------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | vehicleid | recordid | patientid | calls | locationlatitute | locationlongtitue | recordtime | direction | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+ | 1 | 1 | 51 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:00 | S | | 1 | 2 | 13 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:01 | NE | | 1 | 3 | 48 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:02 | NE | | 1 | 4 | 30 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:03 | W | | 1 | 5 | 47 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:04 | S | | 1 | 6 | 9 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:05 | S | | 1 | 7 | 53 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:06 | N | | 1 | 8 | 63 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:07 | SW | | 1 | 9 | 4 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:08 | NE | | 1 | 10 | 31 | 1 | 46.81006 | -92.08174 | 9/14/2014 0:09 | N | +------------+------------+------------+------------+------------------+-------------------+----------------+------------+
FAQ
When using a custom role, you may encounter the error You are not authorized to do this action. You should be authorized by RAM.
-
Cause
The custom role's trust policy trusts only the current Alibaba Cloud account. This prevents other Alibaba Cloud services from assuming the role, which results in a permission denial.
-
Solution
In the custom role's trust policy, modify the
Principalelement to allow the MaxCompute service to assume the role. For example:{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "odps.aliyuncs.com" ] } } ], "Version": "1" }
Appendix
Create a custom role
To access OSS using a custom role, use your Alibaba Cloud account to create the role and grant it permissions. Follow these steps:
-
Create a RAM role
-
Log in to the RAM console.
-
In the left navigation bar, select .
-
On the Roles page, click Create Role.
-
On the Create Role page, set Principal Type to Cloud Service.
-
On the Create Role page, set Principal Name to MaxCompute.
-
Click OK. In the Create Role dialog box, enter a custom Role Name (required) and add a Tag (optional).
For more information on creating a RAM role, see Create a RAM role.
-
-
Modify RAM role trust policy
-
On the Roles page, click the target Role Name to go to the role details page.
-
On the Trust Policy tab, click Edit Trust Policy, and on the Edit Trust Policy page, select the JSON Editor tab.
Example: The following trust policy for
AliyunServiceRoleForMaxComputeLakehouseallows the MaxCompute service to assume this role.{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "lakehouse.odps.aliyuncs.com" ] } } ], "Version": "1" } -
When you are finished, click OK.
-
-
Create a permission policy
-
Log in to the RAM console.
-
In the left navigation bar, select .
-
On the Policies page, click Create Policy.
-
On the Create Policy page, select JSON Editor. Enter the following policy:
This policy grants the default role for the MaxCompute service permissions to access OSS, Tablestore, and DLF.
{ "Version": "1", "Statement": [ { "Action": [ "oss:ListBuckets", "oss:GetObject", "oss:ListObjects", "oss:PutObject", "oss:DeleteObject", "oss:AbortMultipartUpload", "oss:ListParts", "oss:GetBucketInfo", "oss:PostDataLakeStorageFileOperation", "oss:PostDataLakeStorageAdminOperation" ], "Resource": "*", "Effect": "Allow" }, { "Effect": "Allow", "Action": "oss:*", "Resource": [ "acs:oss:*:*:*/.dlsdata", "acs:oss:*:*:*/.dlsdata*" ] }, { "Action": [ "ots:ListTable", "ots:DescribeTable", "ots:GetRow", "ots:PutRow", "ots:UpdateRow", "ots:DeleteRow", "ots:GetRange", "ots:BatchGetRow", "ots:BatchWriteRow", "ots:ComputeSplitPointsBySize" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "pvtz:DescribeRegions", "pvtz:DescribeZones", "pvtz:DescribeZoneInfo", "pvtz:DescribeVpcs", "pvtz:DescribeZoneRecords" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "dlf:CreateFunction", "dlf:BatchGetPartitions", "dlf:ListDatabases", "dlf:CreateLock", "dlf:UpdateFunction", "dlf:BatchUpdateTables", "dlf:DeleteTableVersion", "dlf:UpdatePartitionColumnStatistics", "dlf:ListPartitions", "dlf:DeletePartitionColumnStatistics", "dlf:BatchUpdatePartitions", "dlf:GetPartition", "dlf:BatchDeleteTableVersions", "dlf:ListFunctions", "dlf:DeleteTable", "dlf:GetTableVersion", "dlf:AbortLock", "dlf:GetTable", "dlf:BatchDeleteTables", "dlf:RenameTable", "dlf:RefreshLock", "dlf:DeletePartition", "dlf:UnLock", "dlf:GetLock", "dlf:GetDatabase", "dlf:GetFunction", "dlf:BatchCreatePartitions", "dlf:ListPartitionNames", "dlf:RenamePartition", "dlf:CreateTable", "dlf:BatchCreateTables", "dlf:UpdateTableColumnStatistics", "dlf:ListTableNames", "dlf:UpdateDatabase", "dlf:GetTableColumnStatistics", "dlf:ListFunctionNames", "dlf:ListPartitionsByFilter", "dlf:GetPartitionColumnStatistics", "dlf:CreatePartition", "dlf:CreateDatabase", "dlf:DeleteTableColumnStatistics", "dlf:ListTableVersions", "dlf:BatchDeletePartitions", "dlf:ListCatalogs", "dlf:UpdateTable", "dlf:ListTables", "dlf:DeleteDatabase", "dlf:BatchGetTables", "dlf:DeleteFunction" ], "Resource": "*", "Effect": "Allow" }, { "Action": "pai:AssumeUser", "Resource": "*", "Effect": "Allow" }, { "Action": "odps:ActOnBehalfOfAnotherUser", "Resource": "*", "Effect": "Allow" } ] }For more information, see Create a custom permission policy.
-
Click OK. In the Create Policy dialog box, enter a custom Policy Name. You can also add a description and tags.
-
-
Attach the permission policy to the RAM role
-
Log in to the RAM console.
-
In the left navigation bar, select .
-
On the Roles page, click the target Role Name to go to the role details page.
-
On the Manage Permissions tab, click Create Authorization. In the Create Authorization panel that appears, select the policies to grant to the role, and click OK.
For more information, see Manage permissions for a RAM role.
-
Create a RAM user
-
Log on to the RAM console with your Alibaba Cloud account to create a RAM user and an AccessKey pair. You do not need to grant permissions to the RAM user in this step.
-
Add the user and grant permissions to the MaxCompute project.
Log on to the MaxCompute client using an Alibaba Cloud account and perform the following operations:
NoteIn the following statements,
ramuser_nameis the name of the RAM user that you created, andproject_nameis the name of your MaxCompute project.-
Run the ADD USER command to add the RAM user to the MaxCompute project.
ADD USER RAM$<ramuser_name>;NoteYou can run the
LIST USERS;command to query the RAM users in the project. -
Grant the CreateTable and CreateInstance permissions on the project to the RAM user. For more information about authorization methods, see Grant operation permissions on a project to a user.
GRANT CreateTable,CreateInstance ON PROJECT <project_name> TO USER RAM$<ramuser_name>;
-
Create and use a RAM role
-
Create a RAM role to access OSS.
Log on to the RAM console with an Alibaba Cloud account and create a RAM role that trusts an Alibaba Cloud account. This role is used to access the resources.
Note-
After you create the RAM role, its trusted entity is set to an Alibaba Cloud account by default. You do not need to modify the trust policy.
-
You do not need to attach a permission policy to this role.
-
-
Add the RAM role created in step 1 as a project member.
Log on to the MaxCompute client using an Alibaba Cloud account, add the RAM role as a project member, and grant the CreateTable and CreateInstance permissions on the project to the RAM role. For more information about authorization, see Grant operation permissions on an object to a user.
-
Run the
whoami;command to obtain the account name,account_name. For example, if the value of the Name field in the result isALIYUN$xxx_testcloud_com, theaccount_nameisxxx_testcloud_com. -
Run the following commands to add the RAM role as a member and grant permissions.
-- You must first enable RAM support. ADD accountprovider ram; -- Add the RAM role to the project without granting any permissions. The backticks (`) are required and cannot be replaced with single quotation marks ('). ADD USER `RAM$<account_name>:role/<ramrole_name>`; -- Grant the CreateTable permission on the project. GRANT CreateTable ON project <project_name> TO USER `RAM$<account_name>:role/<ramrole_name>`; -- Grant the CreateInstance permission on the project. GRANT CreateInstance ON project <project_name> TO USER `RAM$<account_name>:role/<ramrole_name>`;The following table describes the parameters.
Parameter
Description
account_name
The
account_nameobtained in the previous step.ramrole_name
The name of the RAM role that you created.
project_name
The name of the MaxCompute project.
-
-
Use your Alibaba Cloud account to grant the
AliyunSTSAssumeRoleAccesspermission policy to the RAM user. This allows the RAM user to call the AssumeRole API.-
Log in to the RAM console.
-
In the left navigation bar, select .
-
On the Users page, click the target User Logon Name/Display Name to go to the user details page.
-
On the user details page, click the Manage Permissions tab.
-
On the Manage Permissions tab, select Individual, and click Grant Permission.
-
In the Grant Permission panel, select the policy to grant to the user, and click OK.
In this step, select the
AliyunSTSAssumeRoleAccesspermission policy.
-
-
Log on to the MaxCompute client by assuming the RAM role.
-
Log on to the Alibaba Cloud official website as a RAM user, call the AssumeRole API, and obtain the temporary identity credentials,
Credentials, for assuming the role. -
From a terminal command line, such as CMD on a Windows system, navigate to the
bindirectory of the MaxCompute client and log in with temporary identity credentials. The command is as follows.The
AccessKeyId,AccessKeySecret, andSecurityTokenparameters are the temporary identity credentials that you obtained in the previous step.odpscmd --account-provider sts --access-id <AccessKeyId> --access-key <AccessKeySecret> --sts-token <SecurityToken>
-