AI training often requires repeated reads of large amounts of data. This creates significant network overhead and affects training efficiency. PAI provides a local cache acceleration feature for Lingjun AI Computing Service. The feature caches data on local compute nodes to reduce network overhead, increase training throughput, and improve data read performance. This process speeds up your AI training tasks.
Technical advantages
High-speed cache: Leverages the memory and local disks of compute nodes to build single-node and distributed read caches. This accelerates access to datasets and checkpoints and significantly reduces data access latency.
Horizontal scaling: The cache throughput scales linearly with the number of compute nodes. It supports scales from hundreds to thousands of nodes.
P2P model distribution: Supports high-concurrency loading and distribution of large-scale models through peer-to-peer (P2P) technology. It uses the high-speed network between GPU nodes to accelerate parallel reads of hot spot data.
Serverless and easy to use: Enable or disable with a single click. No code modification is required. The feature is non-intrusive to your programs and requires no operations and maintenance (O&M).
Limitations and notes
Storage support: Supports OSS and Lingjun CPFS.
Applicable resources: Currently, only Lingjun resources are supported. Note that enabling this feature will consume certain resources (CPU and memory) from the compute node.
Capacity and policy: The maximum cache capacity depends on the specifications of the Lingjun resources. The eviction policy is Least Recently Used (LRU).
Acceleration target: The core goal is to improve data read performance. Write operations are not supported.
Data high availability: High availability is not guaranteed. Data in the local cache can be lost. Back up important training data promptly.
How it works: During multi-epoch training, the first epoch reads data from a storage instance, such as OSS or Lingjun CPFS. The performance is the same as reading directly from the storage instance. In subsequent epochs, data is read from the local cache, which improves the read speed.
How to use
Enable local cache for a resource quota. In the navigation pane on the left, click Resource Quota > Lingjun Resources. Find the target quota and click its name to open its management page. Enable Local Cache and set the storage paths to be cached.
If you use nested resource quotas, make sure that local cache is enabled for the top-level resource quota.
Create a DLC job using the Lingjun resources from the target resource quota and enable Used Cache. When a mounted storage address matches a cache path that you specified in Step 1, acceleration is enabled by default. You can choose to disable it.
