Object Storage Service:Enable OSS-HDFS and grant access permissions

Prerequisites

OSS-HDFS is available in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Ulanqab), China (Shenzhen), China (Guangzhou), China (Zhangjiakou), China (Hong Kong), Japan (Tokyo), Singapore, Germany (Frankfurt), US (Silicon Valley), US (Virginia), Indonesia (Jakarta), and Thailand (Bangkok).

Billing rules

• Metadata management fees

  You are charged the metadata management fees for objects when you use OSS-HDFS. However, you are not charged for this billable item.

• Data storage fees

  When you use OSS-HDFS, data blocks are stored in Objects Storage Service (OSS). Therefore, the billing method of OSS is applicable to data blocks in OSS-HDFS. For more information, see Billing overview.

Limits

• Only the bucket owner and RAM users that are granted the required permissions can enable and access OSS-HDFS. For more information about the permissions required by RAM users to enable and access OSS-HDFS, see the Access permissions section of this topic.

• You can access OSS-HDFS only from a virtual private cloud (VPC). The VPC used to access OSS-HDFS must be located in the same region as the bucket for which you want to enable OSS-HDFS.

• You cannot disable OSS-HDFS after you enable OSS-HDFS. Exercise caution when you enable OSS-HDFS.

• OSS-HDFS cannot be enabled for Archive, Cold Archive, and Deep Cold Archive buckets.

• You cannot use methods that are not supported by OSS-HDFS to perform write operations on the .dlsdata/ directory in which OSS-HDFS data is stored and on the objects in the directory. For example, you cannot use methods that are not supported by OSS-HDFS to rename or delete the directory or delete the objects in the directory. Otherwise, OSS-HDFS does not work as expected and data may be lost.

• Before you can delete a bucket for which OSS-HDFS is enabled, you must delete the metadata of objects that are stored in the bucket from OSS-HDFS and then delete the objects in the bucket. Otherwise, you cannot delete the bucket.

Procedure

• Enable OSS-HDFS for a bucket when you create the bucket, and grant RAM users the permissions to access OSS-HDFS

  For more information, see Create a bucket.

• Enable OSS-HDFS for an existing bucket and grant RAM users the permissions to access OSS-HDFS

  1. Grant RAM users the permissions to access OSS-HDFS.

     1. Log on to the OSS console.

     2. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the bucket for which you want to enable OSS-HDFS.

     3. In the left-side navigation tree, choose Data Lake > OSS-HDFS.

     4. On the OSS-HDFS tab, click Authorize. Then, follow the on-screen instructions to grant RAM users the permissions to access OSS-HDFS.

  2. Enable OSS-HDFS.

     1. On the OSS-HDFS page, click Enable OSS HDFS.

     2. In the message that appears, click OK.

Access permissions

To authorize a RAM user to enable and access OSS-HDFS, attach the following policy to the RAM user:

{
    "Statement": [
        {
          "Effect": "Allow",
          "Action": "oss:ListObjects",
          "Resource": [
            "acs:oss:*:*:*"
          ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "oss:GetBucketInfo",                
                "oss:PostDataLakeStorageFileOperation",
                "oss:PostDataLakeStorageAdminOperation"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "oss:*",
            "Resource": [
                "acs:oss:*:*:*/.dlsdata",
                "acs:oss:*:*:*/.dlsdata*"
            ]
        }
    ],
    "Version": "1"
}

For information about how to grant permissions to RAM users, see Grant permissions to a RAM user.