AnalyticDB:Lake Cache (LakeCache)

Prerequisites

Before you begin, ensure that you have:

• An AnalyticDB for MySQL Enterprise Edition, Basic Edition, or Data Lakehouse Edition cluster

How it works

When a Spark job reads from OSS with lake cache enabled:

1. The lake cache client forwards the read request to the master node to fetch object metadata.

2. The master node returns the metadata to the client.

3. The client uses the metadata to request the actual objects from worker nodes.

   • If the objects are already cached on the worker nodes, they are returned immediately.

   • If not, the objects are fetched from OSS, returned to the client, and stored in the cache for future reads.

Data consistency is maintained automatically: when OSS objects are updated, lake cache detects the change and refreshes the cache. Queries always read the latest data.

Performance

Billing

After you enable lake cache, you are charged for the used cache space on a pay-as-you-go basis. For pricing details, see Pricing for Enterprise Edition and Basic Edition and Pricing for Data Lakehouse Edition.

Limitations

• Lake cache is available in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), Singapore, US (Virginia), and Indonesia (Jakarta). To use lake cache in other regions, submit a ticket.

• If a hardware fault occurs on the cache nodes, queries continue to run but may slow down while data is re-fetched from OSS. Performance recovers automatically after the cache is repopulated.

• When the cache reaches its size limit, infrequently accessed objects are replaced by more frequently accessed ones. Increase the cache size to prevent replacement.

Enable lake cache

1. Log on to the AnalyticDB for MySQL console. In the upper-left corner, select a region. In the left-side navigation pane, click Clusters.

2. On the Enterprise Edition, Basic Edition, or Data Lakehouse Edition tab, find the cluster and click the cluster ID.

3. On the Cluster Information page, go to the Configuration Information section and click Configure next to Lake Cache.

4. In the Lake Cache dialog box, turn on Lake Cache and specify a cache size. If an error occurs when specifying the cache size, submit a ticket.

5. Click OK.

After enabling lake cache, open the Lake Cache dialog box again to confirm the configured cache size.

Use lake cache in Spark jobs

After enabling lake cache, set the spark.adb.lakecache.enabled parameter to true in your Spark job to activate OSS read acceleration.

-- Enable lake cache for this session
SET spark.adb.lakecache.enabled=true;

-- Run your queries
SHOW databases;

{
  "comments": [
    "Enable lake cache for OSS read acceleration."
  ],
  "args": ["oss://testBucketName/data/readme.txt"],
  "name": "spark-oss-test",
  "file": "oss://testBucketName/data/example.py",
  "conf": {
    "spark.adb.lakecache.enabled": "true"
  }
}

Monitor lake cache

After enabling lake cache, check whether your Spark jobs are using the cache and review usage metrics in CloudMonitor.

1. Log on to the CloudMonitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Cloud Service Monitoring.

3. Hover over the AnalyticDB for MySQL card and click AnalyticDB for mysql 3.0 - Data Lakehouse Edition.

4. Find the cluster and click Monitoring Charts in the Actions column.

5. Click the LakeCache Metrics tab to view cache details.

The following metrics are available:

If the cache hit ratio is low despite repeated queries over the same data, consider increasing the cache size to retain more objects between reads.