E-MapReduce supports MetaService. MetaService allows you to access Alibaba Cloud resources from E-MapReduce clusters without providing the AccessKey.
Default application role
{
"Version": "1",
"Statement": [
{
"Action": [
"oss:GetObject",
"oss:ListObjects",
"oss:PutObject",
"oss:DeleteObject",
"oss:ListBuckets",
"oss:AbortMultipartUpload"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
Custom application role
The default application role can meet most business requirements. You can directly use it or edit it as required. E-MapReduce also allows you to use a custom application role. That is, when creating a cluster, you can use the default application role or select a custom application role. For more information about how to create a role and grant it to a service, see RAM documentation.
Access MetaService
MetaService is a Hypertext Transfer Protocol (HTTP). You can access MetaService to obtain metadata information. For example, you can obtain the region where the current cluster resides by running the curl http://localhost:10011/cluster-region command.
- Region: /cluster-region
- Role name: /cluster-role-name
- AccessKey ID: /role-access-key-id
- AccessKey secret: /role-access-key-secret
- Security token: /role-security-token
- Network type: /cluster-network-type
Use MetaService
- Reduces the risk of an AccessKey leakage. The use of a RAM role can minimize the security risk. You can grant only the required permissions to a role. This minimizes the permissions that are granted.
- Improves user experience. MetaService is especially useful when you access OSS resources because it shortens the OSS path that you need to enter.
I. Hadoop
Previously, we used: hadoop fs -ls oss://ZaH******As1s:Ba23N**************sdaBj2@bucket.oss-cn-hangzhou-internal.aliyuncs.com/a/b/c
Now, we use: hadoop fs -ls oss://bucket/a/b/c
II. Hive
Previously, we used:
CREATE EXTERNAL TABLE test_table(id INT, name string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '/t'
LOCATION 'oss://ZaH******As1s:Ba23N**************sdaBj2@bucket.oss-cn-hangzhou-internal.aliyuncs.com/a/b/c';
Now, we use:
CREATE EXTERNAL TABLE test_table(id INT, name string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '/t'
LOCATION 'oss://bucket/a/b/c';
III. Spark
Previously, we used: val data = sc.textFile("oss://ZaH******As1s:Ba23N**************sdaBj2@bucket.oss-cn-hangzhou-internal.aliyuncs.com/a/b/c")
Now, we use: val data = sc.textFile("oss://bucket/a/b/c")