Redis Writer is a writer that is developed based on the Data Integration framework. It can be used to import data from data stores such as data warehouses to Redis.

REmote DIctionary Server (Redis) is a network-enabled key-value storage system that is either in-memory or permanent. It supports logs and delivers high performance. It can be used as a database, cache, and message broker. Redis supports diverse data types for values, including STRING, LIST, SET, ZSET (sorted set), and HASH. For more information about Redis, see redis.io.

Redis Writer interacts with a Redis server by using Jedis. As a preferred Java client development kit that is provided by Redis, Jedis supports almost all the features of Redis.
Note
  • Only sync nodes that are run on exclusive resource groups for Data Integration can write data to Redis clusters.
  • You must configure a Redis connection before you configure Redis Writer. For more information, see Configure a Redis connection.
  • If you write values of the LIST type to Redis by using Redis Writer, the result of rerunning a sync node is not idempotent. If the data type of the values is LIST, you must manually delete the data from Redis when you rerun a sync node.

Parameters

Parameter Description Required Default value
redisMode The deployment mode of Redis. A value of ClusterMode indicates the cluster mode.
Note This parameter is applicable only when the sync node is run on an exclusive resource group for Data Integration.
No N/A
datasource The connection name. It must be the same as the name of the added connection. You can add connections in the code editor. Yes N/A
keyIndexes The columns that are used as the key. The index of the first column is 0. For example, if you want to set the first and second columns of the source data as the key, set the keyIndexes parameter to [0,1].
Note After you specify the keyIndexes parameter, Redis Writer specifies the remaining columns as the value. If you do not want to synchronize all the columns, filter columns when you configure the reader.
Yes N/A
keyFieldDelimiter The delimiter that is used to separate keys when data is written to Redis. Example: key=key1\u0001id. If multiple keys need to be concatenated, this parameter is required. If only one key exists, this parameter is not required. No \u0001
batchSize The number of data records to write at a time. Set this parameter to a proper value based on your actual needs. This can greatly reduce the interactions between Data Integration and Redis on the network, and increase the throughput. However, an excessively large value may lead to the out of memory (OOM) error during the data synchronization process. No 1,000
expireTime The expiration time of the values to be cached in Redis. Unit: seconds. The data is valid permanently if you do not specify this parameter.
  • seconds: the relative time in seconds starting from the current time point. It specifies the time range during which data is valid.
  • unixtime: the UNIX timestamp, which indicates that data expires at a specific time point in the future. The UNIX timestamp represents the number of seconds that have elapsed since 00:00:00 on January 1, 1970.
    Note If the specified expiration time is larger than 30 days, the server identifies the time as the UNIX timestamp.
No 0, which indicates that the values never expire
timeout The timeout period to connect to Redis when data is written to Redis. Unit: milliseconds. No 30,000
dateFormat The format in which the data of the DATE type is written to Redis. Set the value to yyyy-MM-dd HH:mm:ss. No N/A
writeMode The write mode. Redis supports diverse data types for values, including STRING, LIST, SET, ZSET (sorted set), and HASH. Redis Writer allows you to write values of the preceding types to Redis. The value of the writeMode parameter varies based on the specified data type of the values.
Note When you configure Redis Writer, you can choose only one of the five data types that are described in the following table. If you do not specify a data type, the data type is STRING by default.
No string
The following table describes the data types that Redis Writer supports.
Data type Parameter Description Required
STRING 
"writeMode":{
        "type": "string",
        "mode": "set",
        "valueFieldDelimiter": "\u0001"
        }
 type The data type of the values is STRING. Yes
mode The mode in which data of the STRING type is written to Redis. Yes. Valid value: set (overwrites the existing data).
valueFieldDelimiter This parameter is required if two or more columns are specified as the values. This parameter is not required if only one column is specified as the values.

The delimiter that is used to separate values if the data is of the STRING type. Example: value1\u0001value2\u0001value3.

No. Default value: \u0001.
LIST 
"writeMode":{
    "type": "list",
    "mode": "lpush|rpush",
    "valueFieldDelimiter": "\u0001"
}
 type The data type of the values is LIST. Yes
mode The mode in which data of the LIST type is written to Redis. Yes. Valid values: lpush (stores the data at the leftmost of the list) and rpush (stores the data at the rightmost of the list).
valueFieldDelimiter The delimiter that is used to separate values if the data is of the STRING type. Example: value1\u0001value2\u0001value3. No. Default value: \u0001.
SET 
"writeMode":{
        "type": "set",
        "mode": "sadd",
        "valueFieldDelimiter": "\u0001"
        }
 type The data type of the values is SET. Yes
mode The mode in which data of the SET type is written to Redis. Yes. Valid value: sadd (stores the data to a set, or overwrites the existing data).
valueFieldDelimiter The delimiter that is used to separate values if the data is of the STRING type. Example: value1\u0001value2\u0001value3. No. Default value: \u0001.
ZSET (sorted set) 
"writeMode":{
        "type": "zset",
        "mode": "zadd"
        }
 type The data type of the values is ZSET.
Note If the data type of the values is ZSET, each data record must follow the following standard: Except for the key, a data record can contain only one score and one value. The score must be placed before the value. This way, Redis Writer can identify which column is the score and which column is the value.
Yes
mode The mode in which data of the ZSET type is written to Redis. Yes. Valid value: zadd (stores data to a sorted set, or overwrites the existing data).
HASH 
"writeMode":{
        "type": "hash",
        "mode": "hset"
        }
 type The data type of the values is HASH.
Note If the data type of the values is HASH, each data record must follow the following standards: Except for the key, a data record can contain only one attribute and one value. The attribute must be placed before the value. This way, Redis Writer can identify which column is the attribute and which column is the value.
Yes
mode The mode in which data of the HASH type is written to Redis. Yes. Valid value: hset (stores data to a hash sorted set, or overwrites the existing data).

If you do not specify a data type, the data type is STRING by default.

Configure Redis Writer by using the codeless UI

The codeless user interface (UI) is not supported for Redis Writer.

Configure Redis Writer by using the code editor

For more information about how to configure Redis Writer by using the code editor, see Create a sync node by using the code editor.

The following example shows how to configure a sync node to write data to Redis. For more information about the parameters, see the preceding parameter description. 
{
    "type":"job",
    "version":"2.0", // The version number.
    "steps":[
        { 
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"redis",// The writer type.
            "parameter":{
                "expireTime":{// The expiration time of the values to be cached in Redis.
                    "seconds":"1000"
                },
                "keyFieldDelimiter":"u0001",// The delimiter that is used to separate keys when data is written to Redis.
                "dateFormat":"yyyy-MM-dd HH:mm:ss",// The format in which the data of the DATE type is written to Redis.
                "datasource":"",// The connection name.
                "writeMode":{// The write mode.
                    "mode":"",// The mode that is used to write data of a specified data type.
                    "valueFieldDelimiter":"",// The delimiter that is used to separate values.
                    "type":""// The data type of the values.
                },
                "keyIndexes": [// The columns that are used as the key.
                    0,
                    1
                ],
                "batchSize":"1000",// The number of data records to write at a time.
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{
            "record":"0"// The maximum number of dirty data records allowed.
        },
        "speed":{
            "throttle":false,// Specifies whether to enable bandwidth throttling. A value of false indicates that the bandwidth is not throttled. A value of true indicates that the bandwidth is throttled. The maximum transmission rate takes effect only if you set this parameter to true.
            "concurrent":1// The maximum number of concurrent threads.
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}