This topic describes how to use Spark Streaming SQL to perform data analysis and interactive development on the Redis data source.

CREATE TABLE syntax

CREATE TABLE tbName[(columnName dataType [,columnName dataType]*)]
USING redis
OPTIONS(propertyKey=propertyValue[, propertyKey=propertyValue]*);

Table schema

When you create a Redis data table, you must explicitly define the fields in the data table. Example: 
spark-sql> CREATE TABLE redis_test_table(`key0` STRING, `value0` STRING, `key1` STRING, `value1` STRING)
         > USING redis
         > OPTIONS(
         > table="test",
         > redis.save.mode="append",
         > model="hash",
         > filter.keys.by.type="false",
         > key.column="uuid",
         > max.pipeline.size="100",
         > host="localhost",
         > port="6379",
         > dbNum="0");

Parameters

Parameter Description Required
table The prefix of the keys that are used to write data into Redis. The format of a key is ${table}:${key.column}, in which ${table} indicates the prefix and ${key.column} indicates the configuration item. Yes
redis.save.mode The processing method if data to be written already exists in Redis. Valid values: append (append data to be written to the existing data), overwrite (overwrite the existing data), errorifexists (throw an exception), and ignore (discard the existing data). Default value: append. No
model

The storage format of data. Valid values: hash and binary. Default value: hash.

 No
filter.keys.by.type Specifies whether to filter out the data that does not conform to the data storage format. Default value: false. No
key.column The column of keys that are used to write data into Redis. If you do not specify this parameter, universally unique identifiers (UUIDs) are used as keys. No
ttl The time to live (TTL) of a key. If you do not specify this parameter, data is stored permanently. If you specify this parameter, the value indicates the expiration time. Unit: seconds. No
max.pipeline.size The maximum number of bulk data write operations that a pipeline can support. Default value: 100. No
host The IP address of the on-premises machine where the Redis instance is deployed. Default value: localhost. No
port The port of the Redis instance. Default value: 6379. No
dbNum The sequence number of the database to which data is stored. Default value: 0. No