ApsaraDB for Redis instances support Lua commands. Lua scripts can be used to efficiently process check-and-set (CAS) commands. This improves the performance of ApsaraDB for Redis and simplifies the implementation of features that used to be difficult to implement. This topic describes the syntax and usage of Lua scripts in ApsaraDB for Redis.

Precautions

Commands related to Lua scripts cannot be used in the Data Management (DMS) console. For more information about DMS, see Overview. You can use a client or redis-cli to connect to ApsaraDB for Redis instances and use Lua scripts.

Syntax

Command Syntax Description
EVAL EVAL script numkeys [key [key ...]] [arg [arg ...]] Executes a specified script that takes parameters and returns the output.
Parameters:
  • script: the Lua script.
  • numkeys: the number of arguments in the KEYS array. The number is an non-negative integer.
  • KEYS[]: the Redis keys that you want to pass to the script as arguments.
  • ARGV[]: the additional arguments that you want to pass to the script. The indexes of the KEYS[] and ARGV[] parameters start from 1.
Note
  • The EVAL command loads a script into the script cache of ApsaraDB for Redis in a similar way as the SCRIPT LOAD command.
  • Mixed use or misuse of the KEYS[] and ARGV[] parameters may cause ApsaraDB for Redis instances (especially cluster instances) to run abnormally. For more information, see Limits on Lua scripts in cluster instances.
EVALSHA EVALSHA sha1 numkeys key [key ...] arg [arg ...] Evaluates a cached script by its SHA1 digest and runs the script.

If the script is not cached in ApsaraDB for Redis when you use the EVALSHA command, ApsaraDB for Redis returns the NOSCRIPT error. Cache the script in ApsaraDB for Redis by using the EVAL or SCRIPT LOAD command and try again. For more information, see Handle the NOSCRIPT error.

SCRIPT LOAD SCRIPT LOAD script Caches a specified script in ApsaraDB for Redis and returns the SHA1 digest of the script.
SCRIPT EXISTS SCRIPT EXISTS script [script ...] Returns information about the existence of one or more scripts in the script cache by using their corresponding SHA1 digests. If a specified script exists, a value of 1 is returned. Otherwise, a value of 0 is returned.
SCRIPT KILL SCRIPT KILL Terminates a Lua script in execution.
SCRIPT FLUSH SCRIPT FLUSH Removes all the Lua scripts from the script cache in the Redis server.

For more information about Redis commands, visit the Redis official website.

Specific Redis commands are demonstrated in the following examples. Before the following commands are run, the SET foo value_test command is run.

  • Sample EVAL command:
    EVAL "return redis.call('GET', KEYS[1])" 1 foo
    Sample output:
    "value_test"
  • Sample SCRIPT LOAD command:
    SCRIPT LOAD "return redis.call('GET', KEYS[1])"
    Sample output:
    "620cd258c2c9c88c9d10db67812ccf663d96bdc6"
  • Sample EVALSHA command:
    EVALSHA 620cd258c2c9c88c9d10db67812ccf663d96bdc6 1 foo
    Sample output:
    "value_test"
  • Sample SCRIPT EXISTS command:
    SCRIPT EXISTS 620cd258c2c9c88c9d10db67812ccf663d96bdc6 ffffffffffffffffffffffffffffffffffffffff
    Sample output:
    1) (integer) 1
    2) (integer) 0

Optimize memory and network overheads

Issue:

A large number of scripts that serve the same purposes are cached in ApsaraDB for Redis. These scripts take up large amounts of memory and may cause the out of memory (OOM) error. Example of invalid usage:
EVAL "return redis.call('set', 'k1', 'v1')" 0
EVAL "return redis.call('set', 'k2', 'v2')" 0
Solution:
  • Do not pass parameters to Lua scripts as constants to reduce memory usage.
    # The following commands serve the same purposes as the preceding sample commands but cache scripts only once. 
    EVAL "return redis.call('set', KEYS[1], ARGV[1])" 1 k1 v1
    EVAL "return redis.call('set', KEYS[1], ARGV[1])" 1 k2 v2
  • Use the following command syntax to reduce memory and network overheads:
    SCRIPT LOAD "return redis.call('set', KEYS[1], ARGV[1])"    # After this command is run, the following output is returned: "55b22c0d0cedf3866879ce7c854970626dcef0c3".
    EVALSHA 55b22c0d0cedf3866879ce7c854970626dcef0c3 1 k1 v1
    EVALSHA 55b22c0d0cedf3866879ce7c854970626dcef0c3 1 k2 v2

Flush the Lua script cache

Issue:

Used memory of an instance may be higher than expected because the Lua script cache takes up memory of the instance. When the used memory of the instance approaches or exceeds the upper limit, the OOM error is returned. Error example:
-OOM command not allowed when used memory > 'maxmemory'.

Solution:

Flush the Lua script cache by running the SCRIPT FLUSH command on the client. Different from the FLUSHALL command, the SCRIPT FLUSH command is synchronous. If ApsaraDB for Redis caches a large number of Lua scripts, the SCRIPT FLUSH command can block ApsaraDB for Redis for an extended period of time and the involved instance may become unavailable. Proceed with caution. We recommend that you perform this operation during off-peak hours.
Note If you click Clear Data in the ApsaraDB for Redis console, data can be cleared but the Lua script cache cannot be flushed.

Do not write Lua scripts that may take up excessive amounts of memory. Moreover, do not write Lua scripts that involve large amounts of data. Otherwise, memory usage significantly increases and the OOM error may even occur. To reduce memory usage, we recommend that you enable data eviction by using the volatile-lru policy. By default, data eviction is enabled in ApsaraDB for Redis. For more information about data eviction, see How does ApsaraDB for Redis evict data by default? However, ApsaraDB for Redis does not evict the Lua script cache regardless of whether data eviction is enabled.

Handle the NOSCRIPT error

Issue:

If the script is not cached in ApsaraDB for Redis when you use the EVALSHA command, ApsaraDB for Redis returns the NOSCRIPT error. Error example:
(error) NOSCRIPT No matching script. Please use EVAL.

Solution:

Run the EVAL or SCRIPT LOAD command to cache the script in ApsaraDB for Redis and try again. In specific scenarios such as instance migrations and configuration changes, ApsaraDB for Redis still flushes the Lua script cache because ApsaraDB for Redis cannot ensure the persistence and replicability of Lua scripts. For this reason, your client must have the ability to handle this error. For more information, see Caching, persistence, and replication of scripts.

The following sample Python code shows a method for handling the NOSCRIPT error. The sample code prepends strings by using Lua scripts.
Note You can also use redis-py to handle this error. redis-py provides the Script class that encapsulates the judgement logic for Lua scripts of ApsaraDB for Redis, such as a catch statement for the NOSCRIPT error.
import redis
import hashlib

# strin indicates a string in Lua scripts. This function returns the sha1 value of strin in the string format. 
def calcSha1(strin):
    sha1_obj = hashlib.sha1()
    sha1_obj.update(strin.encode('utf-8'))
    sha1_val = sha1_obj.hexdigest()
    return sha1_val

class MyRedis(redis.Redis):

    def __init__(self, host="localhost", port=6379, password=None, decode_responses=False):
        redis.Redis.__init__(self, host=host, port=port, password=password, decode_responses=decode_responses)

    def prepend_inLua(self, key, value):
        script_content = """\
        local suffix = redis.call("get", KEYS[1])
        local prefix = ARGV[1]
        local new_value = prefix..suffix
        return redis.call("set", KEYS[1], new_value)
        """
        script_sha1 = calcSha1(script_content)
        if self.script_exists(script_sha1)[0] == True:      # Check whether ApsaraDB for Redis already caches the script. 
            return self.evalsha(script_sha1, 1, key, value) # If the script is already cached, the EVALSHA command is used to run the script.
        else:
            return self.eval(script_content, 1, key, value) # Otherwise, use the EVAL command to run the script. Note that the EVAL command can cache scripts in ApsaraDB for Redis. You can also use the SCRIPT LOAD and EVALSHA commands. 

r = MyRedis(host="r-******.redis.rds.aliyuncs.com", password="***:***", port=6379, decode_responses=True)

print(r.prepend_inLua("k", "v"))
print(r.get("k"))
            

Handle timeouts of Lua scripts

  • Issue:

    Slow Lua requests may block ApsaraDB for Redis because Lua script execution is atomic in ApsaraDB for Redis. One Lua script can block ApsaraDB for Redis for up to 5 seconds. After 5 seconds, ApsaraDB for Redis returns the BUSY error for other commands until the script execution is complete.
    BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

    Solution:

    Run the SCRIPT KILL command to terminate the Lua script or wait until the Lua script execution is complete.
    Note
    • During the first 5 seconds when a slow Lua script is being executed, the SCRIPT KILL command does not take effect because ApsaraDB for Redis is being blocked.
    • To prevent ApsaraDB for Redis from being blocked for an extended period of time, we recommend that you estimate the amount of time required to execute a Lua script when you write the Lua script, check for infinite loop, and split the Lua script if necessary.
  • Issue:

    If a Lua script has already run write commands against the dataset, the SCRIPT KILL command does not take effect. Error example:
    (error) UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.

    Solution:

    On the Instances page of the ApsaraDB for Redis console, find the instance and click restart in the Actions column.

Caching, persistence, and replication of scripts

Issue:

ApsaraDB for Redis keeps caching the Lua scripts that have been executed in an instance if the instance is not restarted or the SCRIPT FLUSH command is not run for the instance. However, ApsaraDB for Redis cannot ensure the persistence of Lua scripts or the synchronization of Lua scripts from the current node to other nodes in scenarios such as instance migrations, configuration changes, version upgrades, and instance switchovers.

Solution:

Store all Lua scripts in your on-premise device. Recache the Lua scripts in ApsaraDB for Redis by using the EVAL or SCRIPT LOAD command if necessary. This prevents the NOSCRIPT error from occurring when Lua scripts are cleared during an instance restart or a high availability (HA) switchover.

Limits on Lua scripts in cluster instances

Redis clusters impose limits on the usage of Lua scripts. The following additional limits exist for ApsaraDB for Redis cluster instances:

Note If an error message indicating that the EVAL command fails to run is returned, such as ERR command eval not support for normal user, update the minor version of the ApsaraDB for Redis instance to the latest version. For more information about the procedure, see Update the minor version.
  • All keys that a script uses must be allocated to the same hash slot. Otherwise, the following error message is returned:
    -ERR eval/evalsha command keys must be in same slot\r\n
    Note You can run the CLUSTER KEYSLOT command to obtain the hash slot of a key.
  • A Lua script may not be stored in other nodes when you run the SCRIPT LOAD command on one node.
  • The following Pub/Sub commands are not supported: PSUBSCRIBE, PUBSUB, PUBLISH, PUNSUBSCRIBE, SUBSCRIBE, and UNSUBSCRIBE.
  • The UNPACK function is not supported.

If all the operations can be performed in the same hash slot and you want to break through the limits that the cluster architecture imposes on your Lua script, you can set the script_check_enable parameter to 0 in the ApsaraDB for Redis console. This way, the system does not check your Lua script at the backend. In this case, you must specify at least one key in the KEYS array so that proxy nodes can route commands in the Lua script. If you cannot make sure that all the operations are performed in the same hash clot, an error is returned. For more information, see Modify parameters of an instance.

Additional limits on the proxy mode

  • Lua scripts use the redis.call or redis.pcall function to run Redis commands. For Redis commands, all keys must be specified by using the KEYS array, which cannot be replaced by Lua variables. If you do not use the KEYS array to specify the keys, the following error message is returned:
    -ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS array\r\n
    Examples of valid and invalid usage:
    # The following two commands must be run in advance.
    SET foo foo_value
    SET {foo}bar bar_value
    
    # Example of valid usage
    EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar
    
    # Examples of invalid usage
    EVAL "return redis.call('mget', KEYS[1], '{foo}bar')" 1 foo
    EVAL "return redis.call('mget', KEYS[1], ARGV[1])" 1 foo {foo}bar
  • Keys must be included in all the commands that you want to run. Otherwise, the following error message is returned:
    -ERR for redis cluster, eval/evalsha number of keys can't be negative or zero\r\n
    Examples of valid and invalid usage:
    # Example of valid usage
    EVAL "return redis.call('get', KEYS[1])" 1 foo
    
    # Examples of invalid usage
    EVAL "return redis.call('get', 'foo')" 0
  • You cannot run the EVAL, EVALSHA, or SCRIPT command in the MULTI or EXEC transactions.
  • Commands such as KEYS or SCAN that involve multiple Redis nodes are not supported.

    To ensure that the execution of Lua scripts is atomic, proxy nodes send Lua scripts to a Redis node that is determined by the KEYS command for execution and obtain the output. This leads to inconsistency of command outputs for this node and all other nodes.

Note

If you want to use features that are unavailable for the proxy mode, you can enable the direction connection mode for your ApsaraDB for Redis cluster instance. However, migrations or configuration changes fail for cluster instances when Lua scripts that do not conform to the requirements of the proxy mode are executed in direct connection mode. This is because cluster instances rely on proxy nodes to migrate data during migrations and configuration changes.

To prevent subsequent migrations and configuration changes based on Lua scripts from failing, we recommend that you conform to the usage limits of Lua scripts in proxy mode when you use Lua scripts in direct connection mode.