This topic provides the DDL syntax that is used to create an ApsaraDB for Redis dimension table, describes the parameters in the WITH clause and cache parameters, and provides data type mappings and sample code.

Note Flink can read data from self-managed Redis databases.

What is ApsaraDB for Redis?

ApsaraDB for Redis is a database service that is compatible with the protocols of the open source Redis system. ApsaraDB for Redis supports a hybrid of memory and hard disks for storage. ApsaraDB for Redis provides a hot standby architecture to ensure high availability. ApsaraDB for Redis can meet the business requirements for high throughputs, low-latency operations, and flexible configuration changes based on the scalable cluster architecture.

Prerequisites

Limits

  • Only Realtime Compute for Apache Flink that uses Ververica Runtime (VVR) 2.0.0 or later supports the ApsaraDB for Redis connector.
  • ApsaraDB for Redis dimension tables can read only the data of the STRING data type in ApsaraDB for Redis databases.

DDL syntax

CREATE TABLE redis_dim (
  id STRING,
  name STRING,
  PRIMARY KEY (id) NOT ENFORCED
) WITH (
  'connector' = 'redis',
  'host' = '<yourHost>',
  'port' = '<yourPort>',
  'password' = '<yourPassword>',
  'dbNum' = '<yourDbNum>'
);
Note
  • You must declare only one primary key for each ApsaraDB for Redis dimension table.
  • You can declare only two fields for an ApsaraDB for Redis dimension table. The data type of the fields must be STRING.
  • When you join a dimension table with another table, the ON condition must contain equivalent conditions that include all primary keys.
  • You must set the cache parameter to None or LRU for an ApsaraDB for Redis dimension table.

Parameters in the WITH clause

Parameter Description Required Remarks
connector The type of the dimension table. Yes Set the value to redis.
host The address that is used to access the ApsaraDB for Redis database. No You must configure this parameter if the clusterMode parameter is set to false.
port The port number of the ApsaraDB for Redis database. No Default value: 6379.
dbNum The sequence number of the ApsaraDB for Redis database. No Default value: 0.
password The password that is used to access the ApsaraDB for Redis database. No By default, this parameter is empty. This indicates that permission verification is not required.
clusterMode Specifies whether the ApsaraDB for Redis cluster is in cluster mode. No Valid values:
  • true: The cluster is in cluster mode.
  • false: The cluster is not in cluster mode. This is the default value.
hostAndPorts The host and port number of the ApsaraDB for Redis database.
Note If the cluster mode is enabled and connections in high availability (HA) mode is not required, you can use the host and port parameters to configure only one of the hosts. You can also configure only this parameter. The priority of this parameter is higher than the priority of the host and port parameters.
No This parameter is left empty by default. If the ClusterMode parameter is set to true and HA is required for the connection from Jedis to the self-managed Redis database in cluster mode, you must configure this parameter. The value of this parameter is a string that is in the "host1:port1,host2:port2" format.
hashName The hash key name in hash mode. No This parameter is left empty by default. In typical cases, the data type in the ApsaraDB for Redis dimension table is STRING, which is represented as key-value pairs. If you configure the hashName parameter, the ApsaraDB for Redis dimension table stores data of the HASHMAP data type in the format of key-{field-value} pairs.
  • key is the value of the hashName parameter.
  • field is the value of the key parameter that you specify in the CREATE TABLE statement.
  • value is the value that is assigned to key. The value field returns the same results as value in key-value for the STRING data type.

Cache parameters

Parameter Description Required Remarks
cache The cache policy. No Valid values:
  • None: indicates that data is not cached. This is the default value.
  • LRU: indicates that only specific data in the dimension table is cached. Each time the system receives a data record, the system searches the cache. If the system does not find the record in the cache, the system searches for the data record in the physical dimension table.

    If this cache policy is used, you must configure the cacheSize and cacheTTLMs parameters.

cacheSize The maximum number of rows of data that can be cached. No This parameter is available only if you set the cache parameter to LRU. Default value: 10000.
cacheTTLMs The cache timeout period. Unit: milliseconds. No The cacheTTLMs parameter applies configurations based on the value of the cache parameter.
  • If you set the cache parameter to None, the cacheTTLMs parameter can be left empty. This indicates that cache entries do not expire.
  • If you set the cache parameter to LRU, the cacheTTLMs parameter specifies the cache timeout period. By default, cache entries do not expire.
cacheEmpty Specifies whether to cache empty results. No Default value: true.

Data type mappings

Data type of ApsaraDB for Redis Data type of Flink
STRING VARCHAR

Sample code

CREATE TEMPORARY TABLE datagen_source (
  id STRING, 
  data STRING,
  proctime as PROCTIME()
) WITH (
  'connector' = 'datagen'
);

CREATE TEMPORARY TABLE redis_dim (
  id STRING,
  name STRING,
  PRIMARY KEY (id) NOT ENFORCED -- The row key type in the ApsaraDB for Redis database. 
) WITH (
  'connector' = 'redis',
  'host' = '<yourHost>',
  'port' = '<yourPort>',
  'password' = '<yourPassword>'
);

CREATE TEMPORARY TABLE blackhole_sink (
  id STRING,
  data STRING,
  name STRING
) WITH (
  'connector' = 'blackhole'
);

INSERT INTO blackhole_sink
SELECT e.*, w.*
FROM datagen_source AS e
JOIN redis_dim FOR SYSTEM_TIME AS OF e.proctime AS w
ON e.id = w.id;