ApsaraDB for Redis cluster instances and read/write splitting instances use proxy nodes to route commands, balance loads, and perform failovers. If you know how proxy nodes route commands and handle specific commands, you can understand how to design more effective business systems.

Overview of proxy nodes

Figure 1. Architectures of cluster instances and read/write splitting instances
Architectures of cluster instances and read/write splitting instances

A proxy node is a component that runs in the standalone architecture in an ApsaraDB for Redis instance. Proxy nodes do not occupy resources of data shards. An ApsaraDB for Redis instance uses multiple proxy nodes to balance loads and perform failovers.

Capability Description
Enable architecture changes Proxy nodes allow you to use a cluster instance the same manner as you would use a standard instance. If your business requirements grow beyond the capabilities that can be provided by a standard instance, you can migrate the data of the standard instance to a cluster instance that has proxy nodes without having to modify code to reduce costs.
Balance loads and route commands Proxy nodes establish persistent connections with backend data shards to balance loads and route commands. For more information, see Routing methods of proxy nodes.
Manage the traffic to read replicas Proxy nodes monitor the state of each read replica in real time. If a read replica is in one of the following states, proxy nodes stop routing traffic to the read replica:
  • Abnormal: If a read replica is abnormal, proxy nodes reduce the weight of traffic to the read replica. If the read replica fails to connect for the specified number of times consecutively, the proxy nodes stop routing traffic to the read replica until the read replica is normal again.
  • Full data synchronization: If proxy nodes detect that full data is being synchronized on a read replica, the proxy nodes stop routing traffic to the read replica until the synchronization is complete.
Cache hotkey data After you enable the proxy query cache feature, proxy nodes cache the request and response data of hotkeys. If a proxy node receives a duplicate request during the validity period, the proxy node directly returns a response to the client without interacting with backend data shards. This prevents skewed requests caused by hotkeys that send a large number of read requests. For more information, see Use proxy query cache to address issues caused by hotkeys.
Note This feature is supported only by performance-enhanced cluster instances of the ApsaraDB for Redis Enhanced Edition (Tair). For more information about performance-enhanced instances, see Performance-enhanced instances.
Support multiple databases In cluster mode, multiple databases are not supported by open source Redis and Redis cluster clients. In this case, only the default 0 database can be used and the SELECT command is not supported. You can use proxy nodes to access ApsaraDB for Redis cluster instances. In this case, multiple databases can be used and the SELECT command is supported. By default, 256 databases can be specified for a single cluster instance.
Note As proxy nodes develop, the capabilities of the proxy nodes in clusters are no longer based only on the number of proxy nodes. Alibaba Cloud ensures that the number of proxy nodes meets the requirements described in the cluster specifications.

Routing methods of proxy nodes

Note For information about commands, see Overview.
Architecture Routing method Description
Cluster Basic routing method
  • When proxy nodes receive a command that manages an individual key, the proxy nodes identify the hash slot to which the key belongs and route the command to the data shard where the hash slot resides.
  • When proxy nodes receive a command that manages two or more keys stored in different shards, the proxy nodes split the command into multiple commands and route the commands to the corresponding shards.
Routing method for specific commands
  • Pub/Sub commands

    When proxy nodes receive Pub/Sub commands such as PUBLISH and SUBSCRIBE, the proxy nodes hash the channel names and route the commands to the corresponding data shards.

    Note To view the Pub/Sub monitoring data in a data shard, you can select Data Node on the Performance Monitor page in the ApsaraDB for Redis console and set Customize Metrics to Pub/Sub Monitoring Group. By default, the Pub/Sub data of the first data shard is displayed. For more information, see Customize metrics (previous version).
  • Redis commands developed by Alibaba Cloud

    When proxy nodes receive commands that are developed by Alibaba Cloud, such as IINFO and ISCAN, and the data shard IDs are specified in the idx parameter, the proxy nodes route the commands to the specified data shards. For more information, see Redis commands developed by Alibaba Cloud.

Read/write splitting Basic routing method
  • Proxy nodes route write commands to the master node.
  • The system evenly distributes read requests among the master node and read replicas. You cannot change the weights of these nodes. For example, if you purchase an instance that has three read replicas, the weights of the master node and three read replicas are all 25%.
    Note SLOWLOG and DBSIZE are read commands.
Routing method for specific commands
  • SCAN commands

    Proxy nodes route the HSCAN, SSCAN, and ZSCAN commands to the first read replica.If exceptions occur on all read replicas, these commands are routed to the master node.

  • Redis commands developed by Alibaba Cloud

    When proxy nodes receive commands that are developed by Alibaba Cloud, such as RIINFO and RIMONITOR, the proxy nodes route the commands to the read replicas specified by the ro_slave_idx parameter and the data shards specified by the idx parameter. For more information, see Redis commands developed by Alibaba Cloud.

  • Other commands

    Proxy nodes route transaction commands (such as MULTI and EXEC), Lua script commands (such as EVAL and EVALSHA), SCAN commands, INFO commands, and Pub/Sub commands (such as PUBLISH and SUBSCRIBE), to the master node.

Number of connections

In most cases, proxy nodes establish persistent connections with backend data shards to process requests from users. If the requests contain the following special commands, the proxy nodes establish additional connections with the data shards to process subsequent requests. Run the following commands with caution and make sure that the number of connections to each data shard does not exceed the upper limit.

Note In proxy mode, Community Edition instances allow up to 10,000 connections to each data shard, whereas Enhanced Edition (Tair) instances allow up to 30,000 connections to each data shard.
  • Blocking commands: BRPOP, BRPOPLPUSH, BLPOP, BZPOPMAX, and BZPOPMIN.
  • Transaction commands: MULTI, EXEC, and WATCH.
  • Monitoring commands: MONITOR, IMONITOR, and RIMONITOR.
  • Sub commands: SUBSCRIBE, UNSUBSCRIBE, PSUBSCRIBE, and PUNSUBSCRIBE.
  • SCAN command: scans multiple databases.

FAQ

  • Q: What cross-slot commands that manage multiple keys can I run in proxy mode?

    A: You can run the following cross-slot commands that manage multiple keys in proxy mode: DEL, EXISTS, MGET, MSET, SDIFF, SDIFFSTORE, SINTER, SINTERSTORE, SUNION, SUNIONSTORE, and UNLINK.

  • Q: Can proxy nodes route a Lua script command that only reads data to a read replica?

    A: Yes, proxy nodes can route a Lua script command that only reads data to a read replica. For this purpose, you must set the readonly_lua_route_ronode_enable parameter to 1 for your ApsaraDB for Redis instance. A value of 1 indicates that Lua script commands that only read data are routed to read replicas. For more information, see Modify parameters of an instance.

  • Q: What is the difference between the proxy mode and the direct connection mode? Which mode is recommended?
    A: We recommend that you use the proxy mode. The following items describe the difference between the proxy mode and the direct connection mode:
    • By default, cluster instances and read/write splitting instances provide proxy endpoints. Client requests are routed to data shards by proxy nodes. Proxy nodes provide rich features such as load balancing, read/write splitting, failover, proxy query cache, and persistent connection. For more information about proxy query cache, see Use proxy query cache to address issues caused by hotkeys.
    • If you use a cluster instance, you can use a private endpoint to bypass proxy nodes. This way, you can directly access backend data shards. This is similar to a connection to a native Redis cluster. For more information about private endpoints, see Enable the direct connection mode. Compared with the proxy mode, the direct connection mode reduces the routing time and accelerates the response of ApsaraDB for Redis.
  • Q: How does an abnormal data shard affect data reads and writes?
    A: Each data shard runs in a high-availability master-replica architecture. If the master node fails, the system switches the workloads to the replica node to ensure high availability. The following table describes the impacts of abnormal data shards on data reads and writes in specific scenarios and provides optimization methods for each scenario.
    Scenario Impact and optimization
    Figure 2. Commands that manage multiple keys
    Commands that manage multiple keys
    • Impact:

      The client sends four requests over four connections. If Data Shard 2 is abnormal, timeout errors are returned for the requests that are routed to Data Shard 2. The queried data is returned only for Request 1 GET Key1.

    • Optimization methods:
      • Reduce the frequency of the commands that manage multiple keys such as MGET or reduce the number of keys that are managed by a request. This ensures that not all requests fail just because a single data shard is abnormal.
      • Reduce the frequency of transaction commands or reduce the transaction size. This way, when a subtransaction fails, it does not cause the entire transaction to fail.
    Figure 3. Single connection
    Single connection
    • Impact:

      The client sends two requests over the same connection. If Data Shard 2 is abnormal, timeout errors are returned for Request 1 GET Key1 and Request 2 GET Key2. In this example, Request 1 fails because it uses the same connection as Request 2.

    • Optimization methods:
      • Use pipelines as little as possible.
      • Do not use clients that support only a single connection such as Lettuce. We recommend that you use clients that support connection pools such as Jedis. If you use Jedis, you must configure a reasonable timeout period and connection pool size. For more information about Jedis, see Jedis client.