Cache update policy

To accelerate service access and improve concurrent read performance, many users create a caching layer in the service architecture. The business read requests are routed to the caching layer and the business read performance is enhanced through the reading data cached in memory. To ensure data integrity, the update data of the business can be stored to the persistent storage RDS. The business needs to implement the cache update policy to ensure that the cached data can be synchronized and updated when the business data is updated.

As described in the article Did you know cache invalidation can be solved in this way? , a cache update policy with high reliability and low latency can be achieved through the data subscription feature of DTS.

This topic describes how to use DTS to refresh the cache when RDS is used for persistent storage and when MongoDB or Redis is used for caching.

Cache refresh procedure

To synchronize updated data from persistent RDS databases to MongoDB/Redis caches using the RDS MySQL increment subscription feature provided by DTS, follow these steps:

1. Create the subscription channel of the RDS instance in the DTS Console. For more information, see Create Subscription Channel.
2. Encapsulate the SDK provided by DTS, listen on the subscription channel and then retrieve RDS increments in real time. After you encapsulate the incremental data to data update statements of MongoDB/Redis based on business requirements, synchronize them to MongoDB/Redis caches for real-time synchronization of incremental data from RDS to MongoDB/Redis, so that the cache update can achieve high reliability and low latency.

To allow you to easily use the feature, DTS provides demos for encapsulating DTS SDKs to synchronize incremental data to MongoDB/Redis. Next, we illustrate the specific procedures for running these sample demos.

MongoDB cache update demo

Create the Access Key

When the cache update demo is running, you need to configure the Access Key/Access Secret of the Alibaba Cloud account to which the subscription channel belongs. Therefore, before you run the demo, create the Access Key first. If the Alibaba Cloud account has created Access Key, you need not create the Access Key again. Otherwise, follow these steps:

1. Log on to the Alibaba Cloud Console, click the Alibaba Cloud account in the upper-right corner to enter the Account Management Center.
2. Click AccessKeys to enter the Access Key management page.
3. Click Create Access Key in the upper-right corner of the console to create the Access Key/Access Secret.

Install the Java SDK.

The development environment provided by DTS and supported by Java SDK is J2SE Development Kit (JDK) 1.5 or later.

Next, we take an Eclipse project as an example to describe the steps for installing the SDK:

1. Log on to the SDK Download Page to download the latest data subscription SDK package dts-subscribe-3.0.jar.
2. Enter Eclipse and create a Maven project. Right-click Project > Properties > Java Build Path > Libraries > Add External JARs, and select the path where dts-subscribe-3.0.jar is stored, so that dts-subscribe-3.0.jar can be added to the project.
3. Select dts-subscribe-3.0.jar and click OK. After the preceding steps, you can use DTS Java SDK in the project.

Run the sample demo

Next, we take an Eclipse project as an example to describe how to run the MongoDB data update demo.

1. Download the MongoDB data update demo.

2. Create a new class DTSMySQL2Mongo in the newly created project.

3. Open the generated Java file DTSMySQL2Mongo, delete the code template in it, and then paste the sample code in the DTSMySQL2Mongo file.

4. Configure Maven

   <dependency> 
    <groupId>com.vividsolutions</groupId>
    <artifactId>jts</artifactId>
    <version>1.13</version> 
    </dependency>
    <dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongo-java-driver</artifactId>
    <version>3.2.0</version>
    </dependency>

   After you configure the Maven settings by editing the pom.xml file, right click the project, select Maven > Update Project to update the Maven dependencies.

5. Modify the AccessKeyID, AccessKeySecret, subscription channel ID, mongoUrl, mongoUserName and mongoUserPassword in the demo. Specifically:

   • • AccessKeyID is the AccessKey created above
   • • AccessSecret is the AccessSecret corresponding to the AccessKey
   • • Subscription_Instance_ID is the subscription channel ID
   • • mongoUrl is the connection address of the MongoDB to be synchronized. The connection string format is:
   • - MongoDB address: MongoDB listening port/default connecting database, for example: 120.26.122.72:27037/admin1
   • • mongoUserName is the account to access MongoDB
   • • mongoUserPassword is the password of the account to access MongoDB

   

6. In Eclipse, right-click the demo file > operation mode > JAVA application, and you can run the example demo.

Demo of Redis cache update

Create Access Key

When the cache update demo is running, you need to configure the Access Key/Access Secret of the Alibaba Cloud account to which the subscription channel belongs. Therefor, before you run the demo, create the Access Key first. If the Alibaba Cloud account has created Access Key, you do not need to create one again. Otherwise, follow the procedures as follows:

1. Log on to the Alibaba Cloud Console, click the Alibaba Cloud account in the upper-right corner to enter the account management center.
2. Click AccessKeys to enter the Access Key management page.
3. Click Create Access Key in the upper-right corner of the console to create the Access Key/Access Secret.

Install the Java SDK.

The development environment provided by DTS and supported by Java SDK is J2SE Development Kit (JDK) 1.5 or later.

Next, we take an Eclipse project as an example to introduce the steps for installing the SDK:

1. Log on to the SDK Download Page to download the latest data subscription SDK package dts-subscribe-3.0.jar.
2. Enter Eclipse and create a Maven project. Right click Project > Properties > Java Build Path > Libraries > Add External JARs, and select the path where dts-subscribe-3.0.jar is stored, so that the dts-subscribe-3.0.jar can be added to the project.
3. Select dts-subscribe-3.0.jar and click OK. After the preceding steps, you can use DTS Java SDK in the project.

Run the sample demo

Next, we take an Eclipse project as an example to describe how to run the MongoDB data update demo.

1. Download Redis Data Update Demo.

2. Create a new class DTSMySQL2Redis in the newly created project.

3. Open the generated Java file DTSMySQL2Redis, delete the code template in it and paste the sample code in the DTSMySQL2Redis file.

4. Configure Maven

   <dependency> 
    <groupId>com.vividsolutions</groupId>
    <artifactId>jts</artifactId>
    <version>1.13</version> 
    </dependency>
    <dependency> 
    <groupId>redis.clients</groupId> 
    <artifactId>jedis</artifactId>
    <version>2.7.2</version>
    </dependency>

   After you configure Maven settings by editing the pom.xml file, right-click the project, select Maven > Update Project to update Maven dependencies.

5. Modify the AccessKeyID, AccessKeySecret, subscription channel ID, redisUrl, redisPort and redisPassword in the demo. Specifically:

   • • AccessKeyID is the AccessKey created above
   • • AccessSecret is the AccessSecret corresponding to the AccessKey
   • • Subscription_Instance_ID is the subscription channel ID
   • • redisUrl is the connection address of the Redis to be synchronized, for example, 120.26.122.72 is a demo address.
   • • redisPort is the listening port of Redis
   • • redisPassword is the account to access Redis

   

6. In Eclipse, right click the demo file > operation mode > JAVA application, and you can run the example demo.

   The cache update policy with high-reliability and low-latency using DTS data subscription and MongoDB/Redis data update writer is complete.