Accelerating data reads with Lingjun local cache - Platform For AI

AI training often involves repeatedly reading large datasets, which can create significant network overhead and reduce training efficiency. To address this, PAI (Platform of AI) offers local cache acceleration for Lingjun AI Computing Service. This feature caches data on local compute nodes to reduce network overhead, boost training throughput, and improve data read performance, thereby accelerating your AI training jobs.

Benefits

High-speed cache: Creates single-node and distributed read caches from the memory and local disks of compute nodes. These caches accelerate access to your dataset and checkpoints, significantly reducing data access latency.
Horizontal scaling: Cache throughput scales linearly with the number of compute nodes, supporting environments with hundreds or thousands of nodes.
Peer-to-peer (P2P) model distribution: Supports high-concurrency loading and distribution of large-scale models through Peer-to-peer (P2P) technology, leveraging the high-speed network between GPU nodes for accelerated parallel reads of hot data.
Serverless and easy to use: You can enable or disable the feature with a single click. It requires no code modification and operates non-intrusively, making it maintenance-free.

Limitations and notes

Supported storage: Supports Object Storage Service (OSS) and Lingjun CPFS.
Applicable resources: This feature is only available for Lingjun AI Computing Service resources. When enabled, it consumes resources on each compute node (CPU: 4 cores, Memory: 14 GB).
Capacity and policy: The maximum cache capacity depends on the specifications of your Lingjun AI Computing Service resources. The eviction policy is Least Recently Used (LRU).
Acceleration target: The primary goal is to improve data read performance. Write operations are not supported.
Data high availability: The local cache does not provide high availability, and cached data can be lost. Back up important training data promptly.
How it works: During multi-epoch training, the first epoch reads data from the storage instance, such as Object Storage Service (OSS) or Lingjun CPFS. The performance is the same as reading directly from the storage instance. In subsequent epochs, data is read from the local cache, improving read speed.

Procedure

Enable local cache on the resource quota. In the left-side navigation pane, navigate to Resource Quota > Intelligent Computing LINGJUN Resources and click the target quota's name. Turn on the Enable Local Cache switch and specify the storage paths to cache.

If you use nested resource quotas, ensure that local caching is enabled on the top-level resource quota.

In the Advanced Information section, after you turn on the Local Cache switch, click View Configuration to see the configured storage services, including the URI for OSS and the file system for Lingjun CPFS.
Create a DLC job using Lingjun resources from the target resource quota, and enable Use Cache. When a mounted storage address matches a cached address that you specified in Step 1, acceleration is enabled by default. You can choose to disable it.

In the Environment Information step, within the Direct Mount section, configure the OSS mount URI and a mount path, such as /mnt/data/. The corresponding Use Cache switch is automatically enabled.

Configure security group inbound rules

If you use an Enterprise Security Group, you must add an inbound rule to allow traffic from your Virtual Private Cloud (VPC).

On the resource quota page, find the configured security group in the Network Information section. Note the security group name or ID to locate it later.
When you enable cache acceleration, the number of inbound ports you configure must equal the number of configured storage services. This example uses four services.

On the resource quota page, in the Advanced Information section, turn on the Cache Acceleration switch. In the Enable Local Cache dialog box that appears, the Storage Services list shows the number of configured OSS entries. In this example, there are four entries, which means you need to configure four inbound ports.
Navigate to your security group. If it is an Enterprise Security Group, you must add an inbound rule.

Rule parameters: Set Authorization Policy to Allow, Priority to 1, and Protocol to Custom TCP. Set Source to the IPv4 type and enter 10.0.0.0/8 (the CIDR block of this VPC). For Destination (this instance), select the Port type and enter 10080/10083. After you complete the configuration, click Submit. The Source is the CIDR block of the vSwitch that is used by the resource quota. For Destination, you must configure the ports. The number of ports must be equal to the number of storage services in the cache configuration. If the number of storage services is n, configure the port range as 10080/10080+n-1, where n<=10. (This indicates the port range is 10080-10080+n-1). For example, if four storage services are configured, the port range is 10080/10083.