This topic describes how to create a Time Series Database (TSDB) result table in Realtime Compute for Apache Flink. It also describes the parameters in the WITH clause that is used when you create a TSDB result table.
- This topic applies only to Blink 2.0 and later.
- To reference a TSDB result table in Realtime Compute for Apache Flink, you must configure a whitelist that controls access to storage resources. For more information, see Configure a whitelist for accessing storage resources.
Introduction to TSDB
Alibaba Cloud TSDB is a database service that supports efficient reads and writes, compressed storage, and real-time computing for time series data. TSDB is widely used in Internet of Things (IoT) and Internet fields to implement real-time monitoring, forecasting, and alerting on devices and services.
DDL syntax
CREATE TABLE stream_test_hitsdb (
metric VARCHAR,
`timestamp` INTEGER,
`value` DOUBLE,
tagk1 VARCHAR,
tagk2 VARCHAR,
tagk3 VARCHAR
) WITH (
type='hitsdb',
host='<yourHostName>',
virtualDomainSwitch = 'false',
httpConnectionPool = '20',
batchPutSize = '1000'
);
- Column 0: metric (VARCHAR).
- Column 1: timestamp (INTEGER). Unit: seconds.
- Column 2: value (DOUBLE).
- Columns 3 to N: tag keys. The field names in the time series database are used as the tag keys.
- You can specify multiple tag columns.
- You must declare the following fields: metric, timestamp, and value. The names, sequence, and data types of the fields must be the same as those in TSDB.
- For more information about parameter settings,see Write data.
Parameters in the WITH clause
Parameter | Description | Remarks |
---|---|---|
type | The type of the result table. | Set the value to hitsdb .
|
host | The IP address or the domain name that is mapped to the virtual IP address (VIP) of the TSDB instance. | Enter the hostname of the TSDB instance. For more information, see Connect to the instance. |
port | The port number that is used to access the TSDB instance. | Default value: 8242. |
virtualDomainSwitch | Specifies whether to use VIPServer. | Default value: false. If you need to use VIPServer, set this parameter to true. |
httpConnectionPool | The maximum number of connections in an HTTP connection pool. | Default value: 10. |
httpCompress | Specifies whether to use GZIP compression. | Default value: false. This value indicates that GZIP compression is not used. |
httpConnectTimeout | The timeout period for an HTTP connection. | Default value: 0. |
ioThreadCount | The number of I/O threads. | Default value: 1. |
batchPutBufferSize | The buffer size. | Default value: 10000. |
batchPutRetryCount | The maximum number of retries for writing data to the result table. | Default value: 3. |
batchPutSize | The maximum number of data records that can be submitted at a time. | Default value: 500. |
batchPutTimeLimit | The maximum duration for which a data record can be stored in the buffer. | Default value: 200. Unit: milliseconds. |
batchPutConsumerThreadCount | The number of serialized threads. | Default value: 1. |
Models for writing data from Realtime Compute for Apache Flink to TSDB
- Write single-value data points where no tags are included. To use this model, use
the specified schema that consists of three fields. The names of these fields cannot
be changed. You must use the following schema format for this model:
metric,timestamp,value
- Write single-value data points where tags are included. To use this model, use the
specified schema. In the schema, the names of the following fields cannot be changed:
metric, timestamp, and value. You can specify the tag names based on your business
requirements. You must use the following schema format for this model:
metric,timestamp,value,tagKey1,....,tagKeyN
- Write single-value data points where the number of tags is unknown. To use this model,
use the specified schema that consists of four fields. The names of the four fields
cannot be changed. You must use the following schema format for this model:
metric,timestamp,value,tags
In the schema, specify the value of the tags parameter as a JSON string. The JSON string allows you to specify an unknown number of tags. If you do not use the JSON string, you must specify a fixed number of tags for the Blink table schema. In the schema, specify the JSON string in the following format:{"tagKey1":"tagValue1","tagKey2":"tagValue2",……,"tagKeyN":"tagValueN"}
- Write multi-value data points where tags are not included. You must use the following
schema format for this model:
metric,timestamp,field_name1,field_name2,……,field_nameN
The names of the metric and timestamp fields cannot be changed. For multi-value fields, the field_ prefix is used for each field to distinguish fields from tags and to support single-value data writes. For example, when thefield_name1
field is written to TSDB, the prefixfield_
is automatically removed. In the preceding schema, name1 and name2 are the names of the multi-value fields. For the preceding schema, the following format is used to write data to TSDB:metric,timestamp,name1,name2,……,nameN
- Write multi-value data points where tags are included. To use this model, use the
specified schema. In the schema, the names of the following fields cannot be changed:
metric and timestamp. You can specify the tag names based on your business requirements.
You must use the following schema format for this model:
metric,timestamp,tagKey1,....,tagKeyN,field_name1,field_name2,……,field_nameN
- Write single-value data points where the number of tags is unknown. You must use the
following schema format for this model:
metric,timestamp,tags,field_name1,field_name2,……,field_nameN
The content of tags is a JSON string shown in the following example. Therefore, the limit that the Blink table schema requires a fixed number of tags does not apply.{"tagKey1":"tagValue1","tagKey2":"tagValue2",……,"tagKeyN":"tagValueN"}
FAQ
Q: Why does the following error occur during a failover "The values of the LONG data type cannot be converted into the values of the INT data type"?
A: Blink versions earlier than 2.2.5 support only the INT data type. Blink 2.2.5 and later support the BIGINT data type.