Tair (Redis® OSS-Compatible):Lua script syntax and solutions to common errors

Reduce memory and network overheads

When many scripts with duplicate functionality are cached in an instance, they consume a significant amount of memory and may even cause an out-of-memory (OOM) error. The following is an example of incorrect usage.

EVAL "return redis.call('set', 'k1', 'v1')" 0
EVAL "return redis.call('set', 'k2', 'v2')" 0

Flush the Lua script cache

-OOM command not allowed when used memory > 'maxmemory'.

Solution:

Do not write Lua scripts that may take up excessive amounts of memory. Moreover, do not write Lua scripts that involve large amounts of data. Otherwise, memory usage significantly increases and an OOM error may occur. To reduce memory usage, we recommend that you enable data eviction (enabled by default for instances, with volatile-lru mode). However, the instance does not evict the Lua script cache regardless of whether data eviction is enabled.

NOSCRIPT error

(error) NOSCRIPT No matching script. Please use EVAL.

Solution:

Run the EVAL or SCRIPT LOAD command to cache the script in the instance and try again. In specific scenarios, such as instance migrations and configuration changes, the instance flushes the Lua script cache because the instance cannot ensure the persistence and replicability of Lua scripts. In this case, your client must be able to handle the error. For more information, see Persistence and replication issues.

import redis
import hashlib

# strin is a string in Lua scripts. This function returns the sha1 value of strin in the string format.
def calcSha1(strin):
    sha1_obj = hashlib.sha1()
    sha1_obj.update(strin.encode('utf-8'))
    sha1_val = sha1_obj.hexdigest()
    return sha1_val

class MyRedis(redis.Redis):

    def __init__(self, host="localhost", port=6379, password=None, decode_responses=False):
        redis.Redis.__init__(self, host=host, port=port, password=password, decode_responses=decode_responses)

    def prepend_inLua(self, key, value):
        script_content = """\
        local suffix = redis.call("get", KEYS[1])
        local prefix = ARGV[1]
        local new_value = prefix..suffix
        return redis.call("set", KEYS[1], new_value)
        """
        script_sha1 = calcSha1(script_content)
        if self.script_exists(script_sha1)[0] == True:      # Check whether the script is already cached in Redis.
            return self.evalsha(script_sha1, 1, key, value) # If the script is already cached, the EVALSHA command is used to run the script.
        else:
            return self.eval(script_content, 1, key, value) # Otherwise, use the EVAL command to run the script. Note that the EVAL command can cache scripts in Redis. You can also use the SCRIPT LOAD and EVALSHA commands.

r = MyRedis(host="r-******.redis.rds.aliyuncs.com", password="***:***", port=6379, decode_responses=True)

print(r.prepend_inLua("k", "v"))
print(r.get("k"))
            

Lua script timeout error

• Slow Lua requests may block the instance because a Lua script is atomically executed in the instance. One Lua script can block the instance for up to 5 seconds. After 5 seconds, the instance returns a BUSY error for other commands until the script execution is complete.

  BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

  Solution:

  Run the SCRIPT KILL command to terminate the Lua script or wait until the Lua script execution is complete.

  Note

  • During the first 5 seconds when a slow Lua script is being executed, the SCRIPT KILL command does not take effect because the instance is being blocked.

  • To prevent the instance from being blocked for an extended period of time, we recommend that you estimate the amount of time required to execute a Lua script when you write the Lua script, check for infinite loop, and split the Lua script if necessary.

• If a Lua script has already run write commands against the dataset, the SCRIPT KILL command does not take effect. Error example:

  (error) UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.

  Solution:

  On the Instances page of the console, find the instance that you want to manage and click restart.

Persistence and replication issues

Tair (Redis OSS-compatible) keeps caching the Lua scripts that have been executed in an instance if the instance is not restarted or the SCRIPT FLUSH command is not run for the instance. However, Tair (Redis OSS-compatible) cannot ensure the persistence of Lua scripts or the synchronization of Lua scripts from the current node to other nodes in scenarios such as instance migrations, configuration changes, version upgrades, and instance switchovers.

Solution:

Store all Lua scripts in your on-premise device. Recache the Lua scripts in Tair (Redis OSS-compatible) by using the EVAL or SCRIPT LOAD command if necessary. This prevents a NOSCRIPT error from occurring when Lua scripts are cleared during an instance restart or a high availability (HA) switchover.

Constraints of cluster architecture

• To ensure execution atomicity, a Lua script cannot be split and can be executed only on one shard in a cluster instance. In most cases, a key is used to determine which shard Lua scripts are routed to. Therefore, you must specify at least one key when you run a Lua script in a cluster instance. If you want to read or write multiple keys, the keys in the one Lua script must belong to the same slot. Otherwise, an abnormal execution result is returned. Lua scripts that do not have keys (such as KEYS, SCAN, and FLUSHDB) can be executed normally. However, only the data of a single shard is returned. This limit is caused by the architecture of cluster instances.

• When you run the SCRIPT LOAD command on one node, a Lua script may not be stored on other nodes.

Error codes in proxy mode

The proxy performs syntax checks to identify the keys that belong to multiple slots and throw exceptions in advance to assist troubleshooting. The proxy uses a check method different from that of the Lua virtual machine. This places additional limits on the execution of Lua scripts in proxy mode. For example, the UNPACK command is not supported, and the EVAL, EVALSHA, and SCRIPT commands are not supported in MULTI and EXEC transactions.

You can also set the script_check_enable parameter to disable additional checks on Lua syntax in proxy mode.

If the readonly_lua_route_ronode_enable parameter is set to 1 for a read/write splitting instance, proxy nodes check whether Lua scripts contain only read-only commands and determine whether to forward them to read-only nodes. This check logic imposes limits on Lua syntax.

The following describes the error codes and causes.

Limits of Redis Cluster architecture

• Error code: -ERR for redis cluster, eval/evalsha number of keys can't be negative or zero\r\n

  Description: You must include a key when you execute a Lua script. Proxy nodes use the key to determine the shard to which the Lua script is forwarded.

  # Example of valid usage
  EVAL "return redis.call('get', KEYS[1])" 1 fooeval
  
  # Example of invalid usage
  EVAL "return redis.call('get', 'foo')" 0

• Error code: -ERR 'xxx' command keys must in same slot

  Description: Multiple keys in a Lua script must belong to the same slot.

  # Example of valid usage:
  EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar
  
  # Example of invalid usage:
  EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo foobar

Additional limits caused by Lua syntax checks in proxy mode

• Error code: -ERR bad lua script for redis cluster, nested redis.call/redis.pcall

  Description: Nested calls are not supported. You can call the Lua script by using local variables.

  # Example of valid usage
  EVAL "local value = redis.call('GET', KEYS[1]); redis.call('SET', KEYS[2], value)" 2 foo bar
  
  # Example of invalid usage
  EVAL "redis.call('SET', KEYS[1], redis.call('GET', KEYS[2]))" 2 foo bar

• Error code: -ERR bad lua script for redis cluster, first parameter of redis.call/pcall must be a single literal string

  Description: The command called in redis.call/pcall must be a string literal.

  # Example of valid usage
  eval "redis.call('GET', KEYS[1])" 1 foo
  
  # Example of invalid usage
  eval "local cmd = 'GET'; redis.call(cmd, KEYS[1])" 1 foo

Read/write permission issues

• Error code: -ERR Write commands are not allowed from read-only scripts

  Description: Lua scripts sent by using the EVAL_RO command cannot contain write commands.

• Error code: -ERR bad write command in no write privilege

  Description: Lua scripts sent by a read-only account cannot contain write commands.

Unsupported commands

• Error code: -ERR script debug not support

  Description: The SCRIPT DEBUG command is not supported in proxy mode.

• Error code: -ERR bad lua script for redis cluster, redis.call/pcall unkown redis command xxx

  Description: Lua scripts contain commands that are not supported in proxy mode. For more information, see Limits on commands for cluster architecture and read/write splitting instances.

Lua syntax errors

• Error code: -ERR bad lua script for redis cluster, redis.call/pcall expect '(' or -ERR bad lua script for redis cluster, redis.call/redis.pcall definition is not complete, expect ')'

  Description: Lua syntax error. The redis.call function must be followed by the complete set of ( and ).

• Error code: -ERR bad lua script for redis cluster, at least 1 input key is needed for ZUNIONSTORE/ZINTERSTORE

  Description: The numkeys parameter of the ZUNIONSTORE and ZINTERSTORE commands must be greater than 0.

• Error code: -ERR bad lua script for redis cluster, ZUNIONSTORE/ZINTERSTORE key count < numkeys

  Description: The number of keys in the ZUNIONSTORE and ZINTERSTORE commands is less than the numkeys value.

• Error code: -ERR bad lua script for redis cluster, xread/xreadgroup command syntax error

  Description: The XREAD or XREADGROUP command uses incorrect syntax. Check the number of parameters.

• Error code: -ERR bad lua script for redis cluster, xread/xreadgroup command syntax error, streams must be specified

  Description: The streams parameter is required in the XREAD and XREADGROUP commands.

• Error code: -ERR bad lua script for redis cluster, sort command syntax error

  Description: The SORT command uses incorrect syntax.