Object Storage Service (OSS) - Release of OSS Connector for AI/ML
Sep 02 2024
Object Storage Service (OSS)Content
Target customers: customers who use the PyTorch framework for large-scale model training. Feature: OSS Connector for AI/ML is a client used to efficiently access OSS data in PyTorch training jobs. OSS Connector for AI/ML works with PyTorch dataloader to allow you to build map-style and iterable-style datasets suitable for random and sequential streaming data access. You do not need to download data from OSS to the local file system and read data from the local file system. PyTorch can directly read data from the system memory to train models. The embedded read-ahead and concurrency optimization features of OSS Connector for AI/ML can effectively take advantage of the high throughput of OSS and prevent the performance bottleneck of traditional FUSE clients.
Help Document
https://www.alibabacloud.com/help/oss/developer-reference/oss-connector-overview
