This topic describes how to set a cache update policy for a MongoDB or Redis database when you use the change tracking feature. The cache update policy allows you to achieve high reliability and low latency.

Prerequisites

A change tracking task (the previous version) is configured. For more information, see Track data changes from an ApsaraDB RDS for MySQL instance (previous).

Background information

To accelerate service access and improve concurrent read performance, you can build a cache layer in the service architecture. The read requests are routed to the cache layer. The memory reading mechanism of the cache layer allows you to improve the read performance of your business. To ensure data integrity, the updated business data is persistently stored in ApsaraDB RDS for MySQL. In this case, you can set a cache update policy to ensure that the cached data is updated when the business data is updated.

Preparations

Create an AccessKey and obtain the AccessKey ID and AccessKey Secret. For more information, see create an AccessKey pair.

Notice If you use a RAM user to subscribe to data, you must have the AliyunDTSFullAccess permission. For more information , see Authorize a RAM user to use DTS.

Procedure

IntelliJ IDEA (Community Edition 2018.1.4 Windows) is used in this example.

  1. Download the SDK package (dts-subscribe-3.0.jar).
  2. Add the dts-subscribe-3.0.jar package to the project dependencies.
    1. Open IntelliJ IDEA, choose File > New > Project, and then create a Maven project.
    2. In the menu bar, choose File > Project Structure.
    3. In the left-side navigation pane, click Modules. On the right side of the page, choose Add icon > 1 JARs or directories.
    4. Select the dts-subscribe-3.0.jar package and add it to the project dependencies.
      Add the SDK package to the project dependencies
  3. Perform the following steps to set a cache update policy for a MongoDB or Redis database:
    • Run the demo code for MongoDB
      1. Double-click the pom.xml file and add the following settings to the file:
        <dependencies>
            <dependency>
                <groupId>com.vividsolutions</groupId>
                <artifactId>jts</artifactId>
                <version>1.13</version>
            </dependency>
            <dependency>
                <groupId>org.mongodb</groupId>
                <artifactId>mongo-java-driver</artifactId>
                <version>3.2.0</version>
            </dependency>
        </dependencies>
      2. Save the settings and wait for the dependencies to be loaded.
        Note We recommend that you enable IntelliJ IDEA to update Maven dependencies.
      3. Create a Java class file named DTSMySQL2Mongo in the Maven project.
      4. Download the demo code for MongoDB.
      5. Copy the downloaded demo code and replace the existing content in the DTSMySQL2Mongo file with the demo code.
      6. Edit the code in the DTSMySQL2Mongo file and set the parameters.Set a cache update policy for a MongoDB database
        Parameter Description
        accessKey Enter the AccessKey ID and AccessKey secret of the Alibaba Cloud account. For more information, see Before you begin.
        accessSecret
        Subscription_Instance_ID Enter the ID of the change tracking instance. To obtain the ID of the change tracking instance, perform the following steps: Log on to the DTS console. In the left-side navigation pane, click Change Tracking.
        mongUrl Enter the connection string of the MongoDB database and the name of the authentication database. The format is <Connection string>:<Service port number>/<Authentication database name>, for example, ds-bp********-pub.mongodb.rds.aliyuncs.com:3717/admin.
        mongUserName Enter the database account of the MongoDB database.
        Note The database account must have the read/write permissions on the destination database. For example, if the tracked data is written to a MongoDB database named testdata, the database account must have the read/write permissions on the testdata database.
        mongUserPassword Enter the password of the database account.
      7. In the menu bar of IntelliJ IDEA, choose Run > Run to run the program.
    • Run the demo code for Redis
      1. Double-click the pom.xml file and add the following settings to the file:
        <dependencies>
            <dependency>
                <groupId>com.vividsolutions</groupId>
                <artifactId>jts</artifactId>
                <version>1.13</version>
            </dependency>
            <dependency>
                <groupId>redis.clients</groupId>
                <artifactId>jedis</artifactId>
                <version>2.7.2</version>
            </dependency>
        </dependencies>
      2. Save the settings and wait for the dependencies to be loaded.
        Note We recommend that you enable IntelliJ IDEA to update Maven dependencies.
      3. Create a Java class file named DTSMySQL2Redis in the Maven project.
      4. Download the demo code for Redis.
      5. Copy the downloaded demo code and replace the existing content in the DTSMySQL2Redis file with the demo code.
      6. Edit the code in the DTSMySQL2Redis file and set the parameters.Set a cache update policy for a Redis database
        Parameter Description
        accessKey Enter the AccessKey ID and AccessKey secret of the Alibaba Cloud account. For more information, see Before you begin.
        accessSecret
        Subscription_Instance_ID Enter the ID of the change tracking instance. To obtain the ID of the change tracking instance, perform the following steps: Log on to the DTS console. In the left-side navigation pane, click Change Tracking.
        redisPassword Enter the database password of the Redis database.
        Note If a password is set for the Redis database, you must change the code in line 73 from pool = new JedisPool(config, redisUrl, redisPort, 3000); to pool = new JedisPool(config, redisUrl, redisPort, 3000,redisPassword).
        redisPort Enter the service port number of the Redis database. The default port number is 6379.
        redisUrl Enter the endpoint (domain name or IP address) of the Redis database.
      7. In the menu bar of IntelliJ IDEA, choose Run > Run to run the program.
  4. When the program runs, it retrieves incremental data from the source database and writes the data to the MongoDB or Redis database.

Result

  1. Log on to the source database and insert several data records into the source table. Then, the program tracks the inserted data records and writes the data records to the destination database.
    • Running result of the demo code for MongoDBRunning result of the demo code for MongoDB
    • Running result of the demo code for RedisRunning result of the demo code for Redis
  2. Log on to the destination database and query the inserted data records. Then, the data records are written to the database.
    • Query result for the MongoDB databaseQuery result for the MongoDB database
    • Query result for the Redis databaseQuery result for the Redis database