This topic describes the features supported by OSS-HDFS.
In the following tables, Not applicable in the Whether the feature is supported column indicates that the feature is supported only by Object Storage Service (OSS) objects instead of OSS-HDFS files.
Features for buckets
Operation | Description | Whether the feature is supported |
OSS provides the following storage classes for various data storage scenarios from hot data storage to cold data storage: Standard, Infrequent Access (IA), Archive, Cold Archive, and Deep Cold Archive. | You can activate OSS-HDFS only for Standard and IA buckets. | |
The resource usage of a bucket in the OSS console is provided only for reference. | You can view only basic statistics, ranking statistics, region and operator statistics, and API statistics. | |
Buckets are listed in alphabetical order. You can list all buckets, buckets whose names contain a specific prefix, or a specific number of buckets that belong to the current Alibaba Cloud account. | Supported. | |
You can delete a bucket that you no longer need. | Supported. | |
You can call the GetBucketLocation operation to query the region of a bucket. The region is the physical location in which the bucket is located. | Supported. | |
You can use resource groups to manage your resources. You can group your buckets based on your business requirements and configure different access permissions for different resource groups. This way, you can manage access to your buckets by group. | Supported. | |
If you enable pay-by-requester for a bucket, the requester is charged the request and traffic fees when the requester accesses objects in the bucket. The bucket owner is charged only the storage fees of the objects. You can enable pay-by-requester to share your data in OSS without additional fees. | Supported. | |
OSS allows you to configure bucket tags to classify and manage buckets. For example, you can list buckets that have specific tags and configure the access control list (ACL) for buckets that have specific tags. | Supported. | |
You can configure the ACL for a bucket when you create the bucket or change the ACL of an existing bucket based on your business requirements. Only the owner of a bucket can configure or change the ACL of the bucket. | Supported. | |
You can configure policies for a bucket to grant permissions to other users to access specific OSS resources. | Supported. | |
OSS allows you to enable versioning for a bucket to protect objects that are stored in the bucket. After you enable versioning for a bucket, existing objects in the bucket are stored as previous versions when they are overwritten or deleted. After you enable versioning for a bucket, you can restore objects in the bucket to a previous version to protect your data from being accidentally overwritten or deleted. | Not applicable. | |
You can configure hotlink protection for a bucket to prevent unauthorized access to resources in the bucket. | Not applicable. | |
Cross-origin resource sharing (CORS) is a standard cross-origin solution provided by HTML5 to allow web application servers to control cross-origin access. This ensures the security of data transmission across origins. | Not applicable. | |
OSS supports the Write Once Read Many (WORM) feature. The feature helps prevent objects from being deleted or overwritten within a specific period of time. Enterprises use this feature to comply with the regulations of the U.S. Securities and Exchange Commission (SEC) and Financial Industry Regulatory Authority (FINRA). | Not applicable. | |
OSS encrypts objects uploaded to a bucket for which server-side encryption is configured and stores the encrypted objects. When you call the GetObject operation to download an object, OSS decrypts and returns the object. The x-oss-server-side-encryption header is contained in the response to indicate that the object is encrypted on the server side. | Supported. | |
If client-side encryption is configured, objects are encrypted on the on-premises client before they are uploaded to OSS. | Not applicable. | |
OSS provides the following redundancy types: zone-redundant storage (ZRS) and locally redundant storage (LRS). You can use the multi-zone mechanism or single-zone mechanism to ensure data durability and availability. | LRS is supported. ZRS is not supported. | |
OSS DDoS protection is a proxy-based attack mitigation service that integrates OSS with Anti-DDoS Proxy. When a bucket for which OSS DDoS protection is enabled suffers a DDoS attack, OSS DDoS protection diverts incoming traffic to an Anti-DDoS Proxy instance for scrubbing and then redirects normal traffic to the bucket. This ensures the continuity of your business in the event of DDoS attacks. | Not applicable. | |
Cross-region replication (CRR) enables the automatic and asynchronous (near real-time) replication of objects across OSS buckets in different regions. Operations, such as the creation, overwriting, and deletion of objects, can be replicated from a source bucket to a destination bucket. | Not applicable. | |
Same-region replication (SRR) enables the automatic and asynchronous (near real-time) replication of objects across buckets that are located in the same region. Operations, such as the creation, overwriting, and deletion of objects, can be replicated from a source bucket to a destination bucket. | Not applicable. | |
The Replication Time Control (RTC) feature provided by OSS can meet your compliance requirements or business requirements for CRR. After you enable the RTC feature, OSS replicates most of the objects that you upload to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. In addition, the RTC feature provides near real-time monitoring of data replication. After you enable the RTC feature, you can view various metrics of replication tasks. | Not applicable. | |
You can use the scheduled backup feature provided by OSS to periodically back up objects in a bucket to Cloud Backup. If an object is lost, you can restore the object from Cloud Backup. | Not applicable. | |
You can configure lifecycle rules for a bucket based on the last modified time of the objects in the bucket. After you configure a lifecycle rule for a bucket, OSS periodically converts the storage class of objects to IA, Archive, Cold Archive, or Deep Cold Archive, or deletes expired objects and parts. This reduces storage costs. | Not applicable. We recommend that you enable the automatic storage tiering feature for OSS-HDFS. For more information, see Automatic storage tiering of OSS-HDFS. | |
You can configure lifecycle rules based on the last access time of objects in a bucket. After you configure such a lifecycle rule, OSS monitors the access patterns of the objects in the bucket, identifies cold data based on access patterns, and automatically moves cold data to the specified storage class for tiered data storage and reduced storage costs. | Not applicable. | |
You can use the bucket inventory feature to export information about specific objects in a bucket, such as the number, size, storage class, and encryption status of the objects. To list a large number of objects, we recommend that you use the bucket inventory feature instead of calling the GetBucket (ListObjects) operation. | Not applicable. We recommend that you export the metadata of objects in a bucket for which OSS-HDFS is activated to OSS. For more information, see Export object metadata. | |
Static websites are websites in which all web pages consist of only static content, including scripts such as JavaScript code that can be run on a client. You can use the static website hosting feature to host your static website on an OSS bucket and use the domain name of the bucket to access the website. | Not applicable. | |
After you configure mirroring-based back-to-origin rules for a bucket, if a requested object does not exist in the bucket, OSS retrieves the object from the origin specified by the back-to-origin rules. OSS returns the object retrieved from the origin to the requester and stores the object in the bucket. | Not applicable. | |
OSS uses data centers that are distributed around the globe to perform transfer acceleration. When a request to access your bucket is sent, the request is parsed and routed to the data center where the bucket is located over the optimal network path and protocol. The transfer acceleration feature provides an optimized end-to-end acceleration solution to access OSS over the Internet. | Not applicable. | |
Map a custom domain name to the default domain name of a bucket | After you upload objects to a bucket, OSS automatically generates URLs for the uploaded objects. You can use these URLs to access the objects. If you want to access the objects by using custom domain names, map the custom domain names to the buckets in which the objects are stored and add CNAME records for the custom domain names. | Not applicable. |
You can specify image processing (IMG) parameters in GetObject requests to process image objects in OSS. For example, you can add image watermarks to images or convert image formats. | Not applicable. | |
You can configure parameters to capture video snapshots. | Not applicable. | |
You can configure rules for a bucket to automatically decompress ZIP packages uploaded to the bucket. After you configure decompression rules for a bucket, all ZIP packages uploaded to the path specified in the rules are automatically decompressed. | Not applicable. | |
Use event notifications to monitor object changes in real time | You can configure event notification rules for objects that you want to monitor in the OSS console. If the events that you specified in the rules occur on these objects, you are immediately notified. | Not applicable. |
The monitoring service of OSS provides metrics to measure the running status and performance of the system. The monitoring service also provides a custom alert service that allows you to track requests, analyze usage, collect statistics on business trends, and identify and diagnose system issues at the earliest opportunity. | Supported. | |
A large number of logs are generated when OSS resources are accessed. After you enable and configure logging for a bucket, OSS generates logs on an hourly basis based on predefined naming conventions and then stores the logs in a specific bucket. You can use Simple Log Service or build a Spark cluster to analyze the logs. | Not applicable. | |
After you enable real-time log query for a bucket, you can track the requests that are sent to access the bucket. This feature allows you to collect access statistics, audit access to OSS, track exceptions, and troubleshoot issues. Real-time log query can improve your efficiency and help you make informed decisions based on real-time data. | Not applicable. |
Features for objects
Operation | Description | Whether the feature is supported |
Simple upload allows you to call the PutObject operation to upload a single object that is less than 5 GB in size. Simple upload is suitable for scenarios in which an object can be uploaded in a single HTTP request. | Simple upload by using JindoSDK is supported. | |
You can perform multipart upload to split an object that you want to upload into multiple parts and upload the parts separately. After the parts are uploaded, you can call the CompleteMultipartUpload operation to combine these parts into a complete object. | Not applicable. | |
When you use resumable upload to upload objects to OSS, you can specify a checkpoint file. The checkpoint file stores the progress of the resumable upload task. If an object fails to be uploaded due to a network exception or program error, the upload task is resumed based on the progress recorded in the checkpoint file to upload the remaining data. | Not applicable. | |
After an object is uploaded, OSS can send a callback request to the application server. To configure upload callbacks, you need to only add the required callback parameters to the upload request that is sent to OSS. | Not applicable. | |
A direct client upload allows you to upload data directly to OSS from clients. The direct client upload solution accelerates uploads and reduces the resource usage of the application server by eliminating the need to transfer objects to and from the application server. | Not applicable. | |
You can call the PostObject operation to upload an object whose size is up to 5 GB by using an HTML form. | Not applicable. | |
You can perform append upload to append content to an appendable object. | Append upload by using JindoSDK is supported. | |
You can ingest H.264-encoded video streams and Advanced Audio Coding (AAC)-encoded audio streams to OSS over Real-Time Messaging Protocol (RTMP). Audio and video data uploaded to OSS can be played on demand or be used for live streaming in latency-insensitive scenarios. | Not applicable. | |
Simple download allows you to call the GetObject operation to download an object. Simple download is suitable for scenarios in which an object can be downloaded in a single HTTP request. | Simple download by using JindoSDK is supported. | |
You can download data from a specific position of an object. When you download a large object, you can split the object into multiple parts and download the parts at different points in time. If a download task is interrupted, you can resume the download task from the position at which the task is interrupted. | Not applicable. | |
You can authorize third-party users to download an object by providing them with temporary access credentials or a signed URL without exposing the AccessKey pair of the object owner. | Not applicable. | |
By default, when you list objects in a bucket, the objects are returned in alphabetical order. You can list all objects, objects whose names contain a specific prefix, or a specific number of objects in a bucket. | Listing objects by using JindoSDK is supported. | |
You can copy an object from a source bucket to a destination bucket within the same region without modifying the content of the object. | Copying objects by using JindoSDK is supported. | |
You can rename an object in a bucket. | Renaming objects by using JindoSDK is supported. | |
After you upload objects to a bucket, you can share the URLs of the objects with third-party users for downloads or previews. | Not applicable. | |
If a large number of objects are stored in a bucket, you can specify a prefix to search for objects. | Not applicable. | |
You must restore a Cold Archive or Deep Cold Archive object before you can access it. | Not applicable. | |
You can use multiple methods to delete objects that you no longer need from your buckets in OSS. | Deleting objects by using the OSS console, the HDFS shell command, or the Jindo CLI command is supported. For more information, see How do I delete a bucket for which OSS-HDFS is enabled? | |
You can configure object tags to classify objects. Tags allow you to configure lifecycle rules and ACLs for objects that have the same tag. | Not applicable. | |
You can use symbolic links to access objects that are frequently accessed. A symbolic link points to an object and allows you to quickly access the object. Symbolic links are similar to shortcuts in Windows. | Not applicable. | |
Object metadata describes the attributes of an object. The metadata includes standard HTTP headers and user metadata. You can configure HTTP headers to create custom HTTP request policies, such as object cache policies and policies for forcible object downloads. You can also configure user metadata to identify the purposes or attributes of objects. | Not applicable. | |
When you access objects in OSS by using a client, especially a client on which traffic throttling is difficult to perform, a large amount of bandwidth may be consumed. This affects other applications that access OSS. To prevent this issue, you can use the single-connection bandwidth throttling feature to throttle bandwidth for operations such as object upload and download. This way, sufficient bandwidth can be reserved for other applications that access OSS. | Not applicable. | |
You can call the SelectObject operation to execute SQL statements on an object and obtain the execution results. | Not applicable. | |
When you upload an object by using multipart upload, the object is split into multiple small parts. After all parts are uploaded to the OSS server, you can call the CompleteMultipartUpload operation to combine them into a complete object. | You can delete parts in OSS-HDFS in the same manner that you delete parts in OSS. | |
Compared with traditional file systems that use a hierarchical structure, data in OSS is stored as objects in a flat structure. All objects in OSS are stored in buckets. You can use directories in OSS to classify objects and control access to your objects in a simplified manner. | Managing directories by using JindoSDK is supported. | |
You can use the data indexing feature to index the metadata of objects. You can specify the metadata of objects as index conditions to query objects. Data indexing helps you understand and manage data structures in a more efficient manner. Data indexing also facilitates object queries, statistics, and management. | Not applicable. |