As businesses such as AI, data warehousing, and big data analytics continue to grow, workloads running on OSS increasingly demand lower data access latency, higher QPS, and greater throughput. OSS Accelerator caches frequently accessed objects on NVMe SSD high-performance storage media in the same zone as your compute resources, delivering millisecond-level low-latency and high-QPS data access.
Benefits
Low latency
OSS Accelerator is a zone-level service that is deployed in the same zone as your compute resources, reducing network latency. Combined with NVMe SSD media that minimizes storage access latency, OSS Accelerator delivers end-to-end millisecond-level download latency. This is particularly effective for inference model downloads and data warehouse hot data queries.
High IOPS
High-performance NVMe SSD media delivers 100,000-level read IOPS for frequent reads of small files (hundreds of KB) or small data blocks.
High throughput density
OSS Accelerator provides extremely high throughput for small data volumes, meeting burst read demands for hot data.
High throughput capacity
Accelerator bandwidth scales linearly with capacity, providing burst throughput of up to hundreds of GB/s.
Elastic scaling
Compute tasks are typically periodic, and each task requires different resources. You can scale up or scale down the accelerator online based on your needs, effectively avoiding resource waste and reducing costs. The accelerator supports a minimum of 50 GB and a maximum of hundreds of TB of cache space. OSS Accelerator inherits the massive data storage advantages of OSS and supports directly caching multiple tables or partitions from a data warehouse.
Decoupled storage and computing
Unlike cache space on compute servers, OSS Accelerator operates independently of compute servers, allowing you to adjust space and performance online without affecting compute instances.
Data consistency
The accelerator provides strong data consistency with OSS that conventional caching solutions lack. When objects in OSS are updated, the accelerator automatically identifies the changes and caches the updated objects, ensuring that compute engines always read the latest data.
Multiple prefetch strategies
OSS Accelerator can automatically identify updated objects in OSS, ensuring that engines read the latest data. OSS Accelerator provides the following prefetch strategies.
Prefetch on read: When a read request to the accelerator results in a cache miss, the accelerator automatically fetches the data from standard OSS, returns it, and stores it in the accelerator cache.
Synchronous prefetch: When data is written to OSS, it is simultaneously cached in the accelerator space.
Asynchronous prefetch: Through configuration, data in OSS is batch-cached to the accelerator space.
NotePrefetch on read is enabled by default and cannot be configured.
Synchronous prefetch and asynchronous prefetch must be manually enabled. Both features can be used simultaneously.
How it works
After an accelerator is created, it is assigned a zone-specific internal acceleration endpoint that is accessible only over the internal network and does not support public network access. For example, the acceleration endpoint for Zone H in the China (Beijing) region is cn-beijing-h-internal.oss-data-acc.aliyuncs.com. When you are in the same VPC as the accelerator, you can access resources in the accelerator through the acceleration endpoint. The process is as follows.
Write requests
Prefetch on read: Write requests sent to the acceleration endpoint are forwarded directly to the OSS bucket. The process is the same as using the default OSS endpoint.
Synchronous prefetch: Write requests sent to the acceleration endpoint are forwarded directly to both the OSS bucket and the OSS Accelerator.
Asynchronous prefetch: Data that needs to be prefetched is written to the OSS Accelerator before access requests begin.
Synchronous + asynchronous prefetch: Requests are forwarded directly to both the OSS bucket and the OSS Accelerator. In addition, hot data can be written to the OSS Accelerator before access requests begin.
Read requests
NoteRead requests follow the same process regardless of the prefetch strategy.
Read requests sent to the acceleration endpoint are forwarded to the OSS Accelerator.
When the accelerator receives a read request, it searches for the target object in the cache space:
If the target object exists in the cache space, it is returned directly to the client.
If the target object does not exist in the cache space, the accelerator requests it from the bound OSS bucket. After OSS receives the request, it caches the target object in the accelerator, and the accelerator returns the object to the client.
When the accelerator cache is full, the accelerator replaces low-popularity objects with high-popularity objects based on cache file popularity.
Use cases
OSS Accelerator is suitable for scenarios that require high bandwidth and repeated data reads. Specific scenarios are as follows:
Low-latency data sharing
Background
A customer purchases goods from a vending machine by scanning the items with a mobile app that takes photos and uploads them. The application backend receives the images and stores them through OSS Accelerator. Backend subsystems then perform content security analysis and barcode recognition on the images. The barcode recognition results are fed back to the application backend for billing operations. Image downloads must be completed within milliseconds.
Solution
Use the synchronous prefetch (write-time prefetch) mode of OSS Accelerator. OSS Accelerator effectively reduces image loading latency for the analysis system and shortens the transaction chain. OSS Accelerator is well-suited for latency-sensitive workloads with repeated reads.
Model inference
Background
AIGC model inference requires pulling and loading model files. During inference debugging, new model files must be frequently switched for testing. As model files continue to grow in size, the time required for inference servers to pull model files increases.
Solution
Use the asynchronous prefetch or prefetch-on-read mode of OSS Accelerator. Asynchronous prefetch is suitable for scenarios where you know the range of hot model files in advance. Prefetch on read is suitable for scenarios where the model file range is uncertain. If you know the list of hot model files, you can configure the appropriate accelerator space and use the accelerator SDK to pre-load the specified OSS files into the accelerator space. You can also configure an accelerator space of a certain size based on experience. The accelerator will automatically cache files to the high-performance accelerator media during data reads for faster subsequent access. The accelerator space can be scaled up or down at any time based on acceleration results. If your inference program needs to access OSS through a local directory, you need to deploy ossfs.
ACR image loading
Background
Container image pulling is one of the time-consuming steps in the Pod startup process. In AI reinforcement learning and model evaluation scenarios, a single job often needs to schedule thousands or even tens of thousands of compute Pods simultaneously. These Pods pull the same training image, creating concentrated burst read traffic that can easily exceed the OSS egress bandwidth limit of the image repository, causing large numbers of GPU nodes to idle while waiting for images. In AI inference scenarios, traffic fluctuations drive frequent elastic scaling of Pods, and image pull latency for new Pods directly increases cold start time, affecting service SLAs. In general containerized scenarios, cluster scaling, rolling updates, and fault recovery also generate large numbers of concurrent image pull requests in a short time.
Solution
Use OSS Accelerator to cache hot image files from Container Registry (ACR), meeting the burst throughput demands when large numbers of compute Pods pull image files. This shortens Pod preparation time and significantly reduces compute cost waste caused by waiting.If newly uploaded image files are likely to be read within a short time, you can enable the prefetch-on-write mode. For ACR-side configuration, see Customize OSS domain name routing rules.
Big data analytics
Background
A company's business data is partitioned by day and archived to OSS for long-term storage. Analysts use compute engines such as Hive or Spark to analyze the data, but are unsure about the query range. Analysts require minimized query analysis time.
Solution
Use the prefetch-on-read mode of OSS Accelerator. This mode is suitable for offline query scenarios with large data volumes where the data query range is uncertain and accurate prefetching is not possible. For example, data queried by Analyst A is cached in the accelerator cluster. When Analyst B queries data that includes Analyst A's previously queried data, the analysis is accelerated.
Multi-level acceleration
Background
Client-side caching and server-side acceleration do not conflict, and you may want to achieve multi-level acceleration based on your business needs.
Solution
Use OSS Accelerator together with client-side caching. We recommend deploying the client-side cache alongside the compute cluster. When a read misses the client-side cache, data is fetched from the backend storage. For OSS Accelerator, prefetch on read is used, and data is prefetched on the first access. Due to the limited cache space on client hosts, each file and directory in the client-side cache has a TTL. When the TTL expires, the cache is evicted to save space. At this point, data in OSS Accelerator is not immediately evicted, and its cache space can hold hundreds of TB of data. When data that is no longer in the client-side cache is read again, it can be loaded directly from OSS Accelerator, achieving two-level acceleration.
Metrics
Metric | Description |
Capacity |
If your business scenario requires higher capacity, submit a ticket to request a capacity increase. |
Accelerator bandwidth | Based on the configured space size, the accelerator provides throughput bandwidth for cached data. Each TB of accelerator space provides up to 2.4 Gbps of maximum bandwidth. The throughput bandwidth provided by the accelerator is in addition to the standard OSS bandwidth and is not subject to the standard OSS bandwidth limits. For more information about OSS standard bandwidth limits, see Limits and performance metrics. For example, in the China (Shenzhen) region, OSS provides 100 Gbps of standard bandwidth. After you enable the accelerator and configure 10 TB of accelerator space, you can obtain an additional 24 Gbps of low-latency bandwidth through the accelerator endpoint. For batch offline computing applications, we recommend using the OSS internal endpoint with large-scale concurrent block reads to leverage the 100 Gbps standard bandwidth. For hot data query workloads, you can access data cached on NVMe SSD media through the OSS Accelerator endpoint to obtain an additional 24 Gbps of low-latency throughput. |
Read bandwidth (peak) | Formula: MAX[600, 300 × Capacity (TB)] MB/s
For example, if the accelerator capacity is configured as 2048 GB (i.e., 2 TB), the read bandwidth is (600) MB/s = 600 MB/s. |
Maximum read bandwidth | 40 GB/s (320Gbps) If your business scenario requires greater read bandwidth, submit a ticket to apply. |
Minimum read latency for single-stream 128 KB | <10 ms |
Scale-up/scale-down interval | Can be modified once per hour |
Scale-up/scale-down method | Manually scale up or scale down through the console |
Cache eviction policy | Uses the LRU (Least Recently Used) cache eviction policy. The LRU cache eviction policy ensures that frequently accessed data is retained and data that has not been accessed for a long time is removed first, achieving efficient utilization of cache space. |
Billing
When you use OSS Accelerator, OSS charges fees based on the configured accelerator capacity and usage duration. For more information, see OSS accelerator charges.
When you read or write OSS data through the accelerator endpoint, OSS request fees are incurred even if no origin fetch occurs.
What to do next
For information about how to create an OSS Accelerator instance and modify the OSS Accelerator capacity, see Create, modify, and delete an OSS accelerator.
For information about how to configure and use OSS Accelerator with common OSS tools and OSS SDKs, see Use OSS accelerator.
For information about the performance differences between using the OSS internal endpoint and using OSS Accelerator in specific business scenarios, see Performance metrics.