This topic describes how to implement cache updates with high reliability and low latency by using the data tracking of DTS and MongoDB or Redis.

Prerequisites

A change tracking task is created. For more information, see Track data changes from ApsaraDB RDS for MySQL (previous).

Background information

To accelerate service access and improve concurrent read performance, a cache layer is introduced to the service architecture. The read requests of the business are routed to the caching layer, and the business read performance is improved through the reading mechanism of the cached memory; At the same time, to ensure the integrity of the data, the updated business data is persistently stored in the ApsaraDB RDS for MySQL. The business needs to implement the cache update policy to ensure that the cached data can be synchronized and updated when the business data is updated.

Preparations

Create an AccessKey and obtain the AccessKey ID and AccessKey Secret. For more information, see create an AccessKey pair.

Notice If you use a RAM user to subscribe to data, you must have the AliyunDTSFullAccess permission. For more information , see Authorize a RAM user to use DTS.

Procedure

This procedure uses IntelliJ IDEA (Community Edition 2018.1.4 Windows) as an example.

  1. Download the SDK package (dts-subscribe-3.0.jar).
  2. Add the dts-subscribe-3.0.jar to project dependencies.
    1. Open IntelliJ IDEA, click File > New > Project, and then follow the prompts to create a Maven project.
    2. Click File > Project Structure.
    3. Click Moduleson the left, and click Add > 1 JARs or directoriesOn the right side of the page.
    4. Follow the prompts to select the downloaded file: dts-subscribe-3.0.jar , and add it to the project dependency.
      Dependency addition result
  3. Perform the following steps to run the sample code of MongoDB or Redis Cache :
    • Run sample code of MongoDB cache
      1. Double-click to open the pom.xml file, add the following configuration to the file.
        <dependencies>
            <dependency>
                <groupId>com.vividsolutions
                <artifactId>jts</artifactId>
                <version>1.13</version>
            </dependency>
            <dependency>
                <groupId>org.mongodb
                <artifactId>mongo-java-driver</artifactId>
                <version>3.2.0</version>
            </dependency>
        </dependencies>
      2. Save the configurations and wait for the relevant dependency package to be loaded.
        Note We recommend that you enable the Maven dependency automatic update function of IntelliJ IDEA.
      3. Create a new Java Class file named DTSMySQL2Mongo in the project.
      4. Download the sample code of MongoDB cache.
      5. Copy and replace the content in the downloaded sample code to the DTSMySQL2Mongo file in the project.
      6. Set the parameters in the DTSMySQL2Mongo file of the project.Configure MongoDB cache
        Parameter Description
        accessKey The AccessKey ID and AccessKey Secret of your Alibaba cloud account. For more information, see preparation.
        accessSecret
        Subscription_Instance_ID The ID of the change tracking instance. You can log in DTS console, click Change Tracking Tasks in the left-side navigation and find the target instance ID.
        mongUrl The connection string and authentication database name of MongoDB database, the format is <connection address: <port>/<authentication database>, for example ds-bp********-pub.mongodb.rds.aliyuncs.com:3717/admin.
        mongUserName The account of the MongoDB database.
        Note The database account must have the read and write permissions on the destination database. For example, the subscribed table is named testdata. After the program obtains the incremental change, it writes the data into the testdata database of the MongoDB database. Therefore, the database account must have the read and write permissions on the testdata database.
        mongUserPassword The password of the database account.
      7. On the top of the IntelliJ IDEA page, click Run > Run to run the program.
    • Run the sample code for Redis Cache
      1. Double-click to open the pom.xml file, add the following configuration to the file.
        <dependencies>
            <dependency>
                <groupId>com.vividsolutions
                <artifactId>jts</artifactId>
                <version>1.13</version>
            </dependency>
            <dependency>
                <groupId>redis.clients
                <artifactId>jedis</artifactId>
                <version>2.7.2</version>
            </dependency>
        </dependencies>
      2. Save the configurations and wait for the relevant dependency package to be loaded.
        Note We recommend that you enable the Maven dependency automatic update function of IntelliJ IDEA.
      3. Create a new Java Class file named DTSMySQL2Redis in the project.
      4. Download the Redis Cache Sample code.
      5. Copy and replace the content in the downloaded sample code to the DTSMySQL2Redis file in the project.
      6. Set the parameters in the DTSMySQL2Redis file of the project.Sample code for configuring Redis Cache
        Parameter Description
        accessKey The AccessKey ID and AccessKey Secret of your Alibaba cloud account. For more information, see preparation.
        accessSecret
        Subscription_Instance_ID The ID of the change tracking instance. You can log in DTS console, click Change Tracking Tasks in the left-side navigation and find the target instance ID.
        redisPassword The password of the Redis database.
        Note If a password is set in the Redis database, you also need to modify the line 73 in the code formpool = new JedisPool(config, redisUrl, redisPort, 3000); to pool = new JedisPool(config, redisUrl, redisPort, 3000,redisPassword).
        redisPort The service port of the Redis Database. The default port number is 6379.
        redisUrl The endpoint (domain name or IP address) of the Redis database.
      7. On the top of the IntelliJ IDEA page, click Run > Run to run the program.
  4. When the program runs, it changes the incremental data in the source database and writes the incremental data to the MongoDB or Redis database.

Result

  1. Log on to the source database and insert several data records into the source table. You can see that the program has subscribed to the corresponding incremental data changes and then write the corresponding data to the target database.
    • Sample code of MongoDB cacheMongoDB cache code running results
    • Sample code of Redis CacheRedis Cache code running results
  2. Log on to the target database and query the corresponding data. You can see that the corresponding data has been successfully written.
    • MongoDB databaseThe query result in MongoDB.
    • Redis databaseThe query result in the apsaradb for Redis instance.