Access archived objects in the OSS-HDFS service in real time without restoration - Object Storage Service

ArchiveDirectRead lets you directly access files of the Archive storage class in the OSS-Hadoop Distributed File System (HDFS) service without restoration. This feature is suitable for scenarios that require real-time access to infrequently used data.

Prerequisites

An Elastic Compute Service (ECS) instance has been created. For more information, see Create an instance.
The OSS-HDFS service has been activated. For more information, see Activate the OSS-HDFS service.
By default, your Alibaba Cloud account has the required permissions to enable ArchiveDirectRead for a bucket. If you want to use a Resource Access Management (RAM) user to enable this feature, the RAM user must have the required permissions. For more information, see Authorize a RAM user to access the OSS-HDFS service from a non-EMR cluster.

Limits

ArchiveDirectRead applies only to files of the Archive storage class in OSS-HDFS buckets. It does not apply to files of other storage classes.

Billing

After you enable real-time access of Archive objects for a bucket, when you access Archive objects that are not restored in the bucket, you are charged Archive data retrieval fees based on the size of accessed Archive data (RetrievalDataArchiveDirect). When you access Archive objects that are restored in the bucket, you are not charged Archive data retrieval fees. For more information, see Data processing fees.
The volume of retrieved data depends on the data read range specified in the request header when the HTTP connection is established. Breaking the connection early does not affect the volume of retrieved data for the initiated request. For example, if the connection breaks after 1 byte of data is read, but the requested range is from 100 MB to 200 MB, the volume of retrieved data is calculated based on the 100 MB to 200 MB range.

Procedure

Connect to an ECS instance. For more information, see Connect to an ECS instance.
Download the Jindofs SDK.
Configure the AccessKey pair and environment variables.
1. Go to the bin directory of the installed Jindofs SDK package.
  The following example uses jindofs-sdk-x.x.x-linux. If you use a different version of the Jindofs SDK, replace the package name with the actual package name.
```
cd jindofs-sdk-x.x.x-linux/bin/
```
2. In the bin directory, create a configuration file named jindofs.cfg. Then, configure the AccessKey pair of your Alibaba Cloud account or a RAM user that has the required permissions.
```
[client]
fs.oss.accessKeyId = <key>              
fs.oss.accessKeySecret = <secret>
```
3. Set the environment variable.
  Note
  Set <JINDOSDK_CONF_DIR> to the absolute path of the jindofs.cfg configuration file.
```
export JINDOSDK_CONF_DIR=<JINDOSDK_CONF_DIR>
```
Enable ArchiveDirectRead for the bucket.
The following example shows how to enable ArchiveDirectRead for a bucket named examplebucket in the China (Shanghai) region. Replace the region and bucket name as needed.
```
./jindofs admin -putConfig -dlsUri oss://examplebucket.cn-shanghai.oss-dls.aliyuncs.com/ -conf namespace.archive.directread.enable=true
```

You can view the ArchiveDirectRead configuration of the bucket.

./jindofs admin -getConfig -dlsUri oss://examplebucket.cn-shanghai.oss-dls.aliyuncs.com/ -name namespace.archive.directread.enable

The following response shows that the bucket has ArchiveDirectRead enabled.

namespace.archive.directread.enable: true

What to do next

After you enable ArchiveDirectRead for a bucket, you can directly perform read operations, such as downloading files, viewing file information, and copying files, on archived objects in the bucket without restoring them.

References

If ArchiveDirectRead is not enabled, you must restore files of the Archive storage class before you can read them. For more information about how to restore archived objects, see Temporarily restore archived objects.