ApsaraDB for Redis:Usage specifications of Lua scripts and solutions to common errors

Precautions

Commands related to Lua scripts cannot be used in the Data Management (DMS) console. For more information about DMS, see Overview. You can use a client or redis-cli to connect to ApsaraDB for Redis instances and use Lua scripts.

Syntax

For more information about Redis commands, visit the Redis official website.

Optimize memory and network overheads

Issue:

EVAL "return redis.call('set', 'k1', 'v1')" 0
EVAL "return redis.call('set', 'k2', 'v2')" 0

Flush the Lua script cache

Issue:

-OOM command not allowed when used memory > 'maxmemory'.

Solution:

Do not write Lua scripts that may take up excessive amounts of memory. Moreover, do not write Lua scripts that involve large amounts of data. Otherwise, memory usage significantly increases and the OOM error may even occur. To reduce memory usage, we recommend that you enable data eviction by using the volatile-lru policy. By default, data eviction is enabled in ApsaraDB for Redis. For more information about data eviction, see How does ApsaraDB for Redis evict data by default? However, ApsaraDB for Redis does not evict the Lua script cache regardless of whether data eviction is enabled.

Handle the NOSCRIPT error

Issue:

(error) NOSCRIPT No matching script. Please use EVAL.

Solution:

Run the EVAL or SCRIPT LOAD command to cache the script in ApsaraDB for Redis and try again. In specific scenarios such as instance migrations and configuration changes, ApsaraDB for Redis still flushes the Lua script cache because ApsaraDB for Redis cannot ensure the persistence and replicability of Lua scripts. For this reason, your client must have the ability to handle this error. For more information, see Caching, persistence, and replication of scripts.

import redis
import hashlib

# strin indicates a string in Lua scripts. This function returns the sha1 value of strin in the string format. 
def calcSha1(strin):
    sha1_obj = hashlib.sha1()
    sha1_obj.update(strin.encode('utf-8'))
    sha1_val = sha1_obj.hexdigest()
    return sha1_val

class MyRedis(redis.Redis):

    def __init__(self, host="localhost", port=6379, password=None, decode_responses=False):
        redis.Redis.__init__(self, host=host, port=port, password=password, decode_responses=decode_responses)

    def prepend_inLua(self, key, value):
        script_content = """\
        local suffix = redis.call("get", KEYS[1])
        local prefix = ARGV[1]
        local new_value = prefix..suffix
        return redis.call("set", KEYS[1], new_value)
        """
        script_sha1 = calcSha1(script_content)
        if self.script_exists(script_sha1)[0] == True:      # Check whether ApsaraDB for Redis already caches the script. 
            return self.evalsha(script_sha1, 1, key, value) # If the script is already cached, the EVALSHA command is used to run the script.
        else:
            return self.eval(script_content, 1, key, value) # Otherwise, use the EVAL command to run the script. Note that the EVAL command can cache scripts in ApsaraDB for Redis. You can also use the SCRIPT LOAD and EVALSHA commands. 

r = MyRedis(host="r-******.redis.rds.aliyuncs.com", password="***:***", port=6379, decode_responses=True)

print(r.prepend_inLua("k", "v"))
print(r.get("k"))
            

Handle timeouts of Lua scripts

• Issue:

  Slow Lua requests may block Tair because a Lua script is atomically executed in Tair. One Lua script can block Tair for up to 5 seconds. After 5 seconds, Tair returns a BUSY error for other commands until the script finishes running.

  BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

  Solution:

  Run the SCRIPT KILL command to terminate the Lua script or wait until the Lua script finishes running.

  Note

  • During the first 5 seconds when a slow Lua script is running, the SCRIPT KILL command does not take effect because Tair is being blocked.

  • To prevent Tair from being blocked for an extended period of time, we recommend that you estimate the amount of time required to execute a Lua script when you write the Lua script, check for infinite loops, and split the Lua script if necessary.

• Issue:

  If a Lua script has already run write commands against the dataset, the SCRIPT KILL command does not take effect. Error example:

  (error) UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.

  Solution:

  On the Instances page of the Tair console, find the instance that you want to manage and click Restart in the Actions column.

Caching, persistence, and replication of scripts

Issue:

ApsaraDB for Redis keeps caching the Lua scripts that have been executed in an instance if the instance is not restarted or the SCRIPT FLUSH command is not run for the instance. However, ApsaraDB for Redis cannot ensure the persistence of Lua scripts or the synchronization of Lua scripts from the current node to other nodes in scenarios such as instance migrations, configuration changes, version upgrades, and instance switchovers.

Solution:

Store all Lua scripts in your on-premise device. Recache the Lua scripts in ApsaraDB for Redis by using the EVAL or SCRIPT LOAD command if necessary. This prevents the NOSCRIPT error from occurring when Lua scripts are cleared during an instance restart or a high availability (HA) switchover.

Limits on Lua scripts in cluster instances

Open source Redis clusters have limits on the use of Lua scripts. Redis cluster instances have the following additional limits on the use of Lua scripts:

• Instances that use specific minor versions do not support EVAL-related commands. If the ERR command eval not support for normal user error message is returned, update the minor version of your instance and try again. For more information, see Update the minor version of an instance.

• All keys must be in the same slot. Otherwise, the system returns the -ERR eval/evalsha command keys must be in same slot\r\n error.

  You can run the CLUSTER KEYSLOT command to obtain the hash slot of a key.

• When you run the SCRIPT LOAD command on one node, a Lua script may not be stored in other nodes.

• The following Pub/Sub commands are not supported: PSUBSCRIBE, PUBSUB, PUBLISH, PUNSUBSCRIBE, SUBSCRIBE, and UNSUBSCRIBE.

• The UNPACK function is not supported.

Additional checks on Lua scripts in proxy mode

You can specify the script_check_enable parameter to disable the following checks on Lua scripts. We recommend that you do not disable the checks.

• Lua scripts use the redis.call or redis.pcall function to run Redis commands. All keys must be specified by using the KEYS array, and cannot be replaced by Lua variables. If you do not use the KEYS array to specify the keys, the following error message is returned: -ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS array\r\n.

  Note

  This limit applies only to instances that run Redis 5.0 and whose minor version is earlier than 5.0.8 and instances that run Redis 4.0 or earlier.

  Examples of valid and invalid command usage:

  # To prepare for this example, run the following commands: 
  SET foo foo_value
  SET {foo}bar bar_value
  
  # Example of valid command usage:
  EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar
  
  # Example of invalid command usage:
  EVAL "return redis.call('mget', KEYS[1], '{foo}bar')" 1 foo                      # This command is invalid because the '{foo}bar' key must be specified by using the KEYS array. 
  EVAL "local i = 2 return redis.call('mget', KEYS[1], KEYS[i])" 2 foo {foo}bar    # This command is invalid because the index of keys consists of variables, which is not allowed for an instance in proxy mode. Instances in direct connection mode are not subject to this limit. 
  EVAL "return redis.call('mget', KEYS[1], ARGV[1])" 1 foo {foo}bar                # This command is invalid because ARGV[1] cannot be specified as a key. 

• You must include keys in all commands that you want to run. Otherwise, the following error message is returned: -ERR for redis cluster, eval/evalsha number of keys can't be negative or zero\r\n.

  Note

  This limit applies only to instances that run Redis 5.0 and whose minor version is earlier than 5.0.8 and instances that run Redis 4.0 or earlier.

  Examples of valid and invalid command usage:

  # Example of valid command usage
  EVAL "return redis.call('get', KEYS[1])" 1 foo
  
  # Example of invalid command usage
  EVAL "return redis.call('get', 'foo')" 0

• You cannot run the EVAL, EVALSHA, or SCRIPT command in the MULTI or EXEC transactions.

• Commands that involve multiple Redis nodes, such as KEYS and SCAN, are not supported in Lua scripts.

  To ensure that Lua scripts are atomically executed, proxy nodes send the scripts to a Redis node based on the KEYS command for execution and obtain the output. This results in different command outputs of the Redis node and the other nodes.