Time Series Database:FAQ

Series and series cardinality

• How do I perform modeling? What are the precautions?

• Why is the series cardinality important?

Administration

• How do I maintain the system stability?

• How do I view the disk usage of a measurement?

• How do I delete data in an efficient and secure manner?

• Why does the modified retention policy fail to take effect?

• Why are services frequently interrupted?

• Why is the memory usage high? Do I need to upgrade the specifications of the instance?

• Can I scale in the storage capacity?

• Do services restart when I resize a disk?

• How long does it take to restart services?

• Is the pre-configured Grafana in InfluxDB supported?

• Why did an IP block error occur while connecting to an instance?

• How do I identify the version of TSDB for InfluxDB®?

• What is the relationship between a shard group duration and a retention policy?

• Why is no data lost after I modify a retention policy?

• Why is TSDB for InfluxDB® unable to parse the microsecond time unit?

Data query

• How do I troubleshoot slow queries?

• How do I specify the time intervals that are returned by GROUP BY time() queries?

• Why does a query response contain no data or contain partial data?

• Why does the response of a GROUP BY time() query exclude the data whose timestamps occur after the time that is returned by the now() function?

• Can I perform mathematical operations on timestamps?

• Can I identify the time granularities for data writes based on the returned timestamps?

• When do I use single quotation marks (') and double quotation marks (") in queries?

• Why is data lost after I create a default retention policy (DEFAULT)?

• Why does the response of a WHERE OR clause that uses the OR operator to specify multiple time ranges contain no data?

• Why does the fill(previous) function fail to return values?

• Why is data lost when I run INTO queries?

• How do I query data when tag keys and field keys share the same name?

• How do I query data across measurements?

• Does the sequence of timestamp filters in query statements have a significant effect on the response time?

• How do I specify a SELECT statement if I want to query the tags that have no values?

Data writing

• Why does write jitter occur periodically?

• Precautions for writing data by using the InfluxDB line protocol

• Why am I unable to view the data that is written to a table?

• Why is data still not written to the disk after I reduce the disk space usage?

• How do I write a field value of the INTEGER data type?

• How does TSDB for InfluxDB® process duplicate points?

• What is the line feed that is required in HTTP API requests?

• What characters and words are not recommended when I write data to TSDB for InfluxDB®?

• When do I use single quotation marks (') and double quotation marks (") to write data?

• Does the timestamp granularity have a significant impact on system performance?

CLI of TSDB for InfluxDB®

• Why am I unable to I connect to TSDB for InfluxDB® by using the Influx CLI?

• How do I enable the CLI of TSDB for InfluxDB® to return human readable timestamps?

• How do I execute the USE statement to specify a database if I am not an administrator?

• How do I use the CLI of TSDB for InfluxDB® to write data to a non-default retention policy?

Data types

• Why am I unable to query the field values of the BOOLEAN data type?

• How does TSDB for InfluxDB® handle differences in field data types across shards?

• What are the smallest and the largest integers that TSDB for InfluxDB® can store?

• What are the smallest and the largest timestamps that TSDB for InfluxDB® can store?

• How do I view the data types of fields?

• Can I convert the data types of fields?

InfluxQL functions

• How do I perform mathematical operations within functions?

• Why does a query response use epoch 0 as a timestamp?

• Which InfluxQL functions can be nested?

InfluxDB data migration

• How do I migrate a self-managed InfluxDB to the cloud?

• How do I migrate data between different InfluxDB instances in the cloud?

• How do I migrate data from InfluxDB to LindormTSDB?

Why is the series cardinality important?

TSDB for InfluxDB® maintains an in-memory index for each series in the system. The random-access memory (RAM) usage increases if the number of series increases. If the series cardinality is excessively high, the operating system terminates the TSDB for InfluxDB® process and throws an out-of-memory (OOM) exception. InfluxQL reference

How do I perform modeling? What are the precautions?

We recommend that you store no more than one million time series. For more information about scenarios that involve a large number of time series, see Introduction. Before you start modeling, see InfluxDB schema design and data layout. InfluxDB creates indexes based on tags to accelerate queries. Excessive tags slow down the read/write speed due to a large amount of time series data. Therefore, you need to control the number of time series and take note of the following items:

• Do not use values that are prone to infinite expansion as tags, such as IDs, hash values, and time.

• Do not store information in the names of measurements and tags. Save relevant information to tags or fields.

• Do not store multiple pieces of information in one tag. Break them down into multiple tags.

• If a field is frequently used in the Group By operation, this field can be modeled as a tag.

How do I identify the version of TSDB for InfluxDB®?

You can use the following methods to identify the version of your TSDB for InfluxDB®:

• Run the curl path/ping command.

  $ curl -i 'https://<Endpoint>:3242/ping?u=<Account name>&p=<Password>'
  HTTP/1.1204NoContent
  Content-Type: application/json
  X-Influxdb-Build: OSS
  X-Influxdb-Version:1.7.x

• Use the CLI of TSDB for InfluxDB®.

  $ influx -ssl -username <Username> -password <Password> -host <Endpoint> -port 3242
  
  Connected to https://<Endpoint>:3242 version 1.7.x

What is the relationship between a shard group duration and a retention policy?

TSDB for InfluxDB® stores data in shard groups. Each shard group covers a specified time interval. To view the specified time interval in TSDB for InfluxDB®, you can check the value of the DURATION element in the retention policy. The following table lists the default relationships between the time intervals of shard groups and the values of the DURATION elements in the retention policies.

To view the shard group durations of retention policies, execute the SHOW RETENTION POLICIES statement.

Why is no data lost after I modify a retention policy?

This may be due to the following reasons:

• By default, TSDB For InfluxDB® checks and enforces the retention policy every 30 minutes. TSDB for InfluxDB® may delete data upon the next check. This applies if the data is obtained within a time range that is excluded from the duration. In this case, the duration is specified by the new retention policy.

• The changes of the DURATION and SHARD DURATION values in the retention policy may cause unexpected data retention. TSDB for InfluxDB® stores data in shard groups. Each shard group includes a specific retention policy and a time interval. If TSDB for InfluxDB® enforces a retention policy, TSDB for InfluxDB® deletes all the points in the shard group. In this case, individual points are not deleted. TSDB for InfluxDB® cannot split shard groups. The system is forced to save all data to a previous shard group if the following two conditions are met: (1) the DURATION value of the new retention policy is less than the SHARD DURATION value of the previous retention policy, and (2) TSDB for InfluxDB® is writing data to the previous shard group that covers a longer time interval than that specified by the DURATION element. In this case, the system saves all data to the previous shard group even if some points in the shard group fall out of the time interval specified by the new DURATION value. If all the data points in the previous shard group fall out of the time interval specified by the new DURATION value, TSDB for InfluxDB® deletes the entire shard group. Then, the system starts to write data to a shard group that has a new and shorter time interval specified by the SHARD DURATION element. This allows you to prevent unexpected data retention.

Why is TSDB for InfluxDB® unable to parse the microsecond time unit?

The syntax used to specify the microsecond time unit varies based on the specific scenario. These scenarios include data writing, time granularity settings in the CLI of TSDB for InfluxDB®, and data queries. The following table lists the syntax that is supported for each scenario.

Examples

The french_bulldogs measurement includes the color tag and the name field.

> SELECT * FROM "french_bulldogs"
name: french_bulldogs
---------------------
time                  color  name
2016-05-25T00:05:00Z  peach  nugget
2016-05-25T00:05:00Z  grey   rumple
2016-05-25T00:10:00Z  black  prince

How do I write a field value of the INTEGER data type?

To write a field value of the integer data type, add i to the end of the field value. If you do not add i, TSDB for InfluxDB® processes the value as a floating-point number.

Write an integer: value=100i. Write a floating-point number: value=100.

How does TSDB for InfluxDB® process duplicate points?

A point is uniquely identified by a measurement name, a tag set, and a timestamp. If two points have the same measurement name, tag set, and timestamp, they are considered as duplicate points. If you submit a duplicate point that has a different field set from an existing point, the field set of the point becomes the sum of the previous and new field sets. If a conflict occurs, the new field set takes precedence. This is the expected result.

Examples:

Previous point: cpu_load,hostname=server02,az=us_west val_1=24.5,val_2=7 1234567890000000

New point: cpu_load,hostname=server02,az=us_west val_1=5.24 1234567890000000

After you submit a new point, TSDB for InfluxDB® uses the new val_1 value to overwrite the previous value val_1. The value val_2 is retained in the field set of the point.

> SELECT * FROM "cpu_load" WHERE time =1234567890000000
name: cpu_load
--------------
time                      az        hostname   val_1   val_2
1970-01-15T06:56:07.89Z   us_west   server02   5.247

To store the previous and new points, you can use the following methods:

• Introduce a new tag to ensure uniqueness.

  Previous point: cpu_load,hostname=server02,az=us_west,uniq=1 val_1=24.5,val_2=7 1234567890000000

  New point: cpu_load,hostname=server02,az=us_west,uniq=2 val_1=5.24 1234567890000000

  After you write the new point to TSDB for InfluxDB®, the following result appears:

  > SELECT * FROM "cpu_load" WHERE time =1234567890000000
  name: cpu_load
  --------------
  time                      az        hostname   uniq   val_1   val_2
  1970-01-15T06:56:07.89Z   us_west   server02   124.57
  1970-01-15T06:56:07.89Z   us_west   server02   25.24

• Add one nanosecond to the timestamp of the new point.

  Previous point: cpu_load,hostname=server02,az=us_west val_1=24.5,val_2=7 1234567890000000

  New point: cpu_load,hostname=server02,az=us_west val_1=5.24 1234567890000001

  After you write the new point to TSDB for InfluxDB®, the following result appears:

  > SELECT * FROM "cpu_load" WHERE time >=1234567890000000 and time <=1234567890000001
  name: cpu_load
  --------------
  time                             az        hostname   val_1   val_2
  1970-01-15T06:56:07.89Z          us_west   server02   24.57
  1970-01-15T06:56:07.890000001Z   us_west   server02   5.24

What is the line feed that is required in HTTP API requests?

In the line protocol of TSDB for InfluxDB®, the \n line feed is used to indicate the end of a line and the start of a new line. The ASCII code of the line feed is 0x0A. If other line feeds rather than \n are used in files or data, the following errors occur: bad timestamp and unable to parse.

Note: Windows uses the \r\n line feed or the carriage return to indicate the end of a line and the start of a new line.

What characters and words are not recommended when I write data to TSDB for InfluxDB®?

• InfluxQL keywords

  If you use InfluxQL keywords as identifiers, you must enclose the keywords in double quotation marks (") in your queries. Otherwise, errors may occur. Identifiers can be continuous query names, database names, field keys, measurement names, retention policy names, subscription names, tag keys, or usernames.

• Time

  You can use the time keyword as an identifier without the need to enclose the identifier in double quotation marks (") in query statements. You can use the time keyword as a continuous query name, a database name, a measurement name, a retention policy name, or a username. In these cases, you do not need to use double quotation marks (") to enclose time. However, you cannot use the time keyword as a field key or a tag key. TSDB for InfluxDB® rejects data writes where the time keyword is a field key or a tag key, and reports an error. Examples:

  • Use the time keyword as a measurement name to write data and query the measurement.

    > INSERT time value=1
    
    > SELECT * FROM time
    
    name: time
    time                            value
    ---------
    2017-02-07T18:28:27.349785384Z1

    In TSDB for InfluxDB®, the time keyword is a valid measurement name.

  • Use the time keyword as a field key to write data and attempt to query the field key.

    > INSERT mymeas time=1
    ERR:{"error":"partial write: invalid field name: input field \"time\" on measurement \"mymeas\" is invalid dropped=1"}

    In TSDB for InfluxDB®, the time keyword is an invalid field key. The system fails to write the point and returns the 400 error code.

  • Use the time keyword as a tag key to write data and attempt to query the tag key.

    > INSERT mymeas,time=1 value=1
    ERR:{"error":"partial write: invalid tag key: input tag \"time\" on measurement \"mymeas\" is invalid dropped=1"}

    In TSDB for InfluxDB®, the time keyword is an invalid tag key. The system fails to write the point and returns the 400 error code.

• Characters

  To keep the use of simple regular expressions and quotation marks, we recommend that you do not use the following characters in identifiers: backslashes (\), carets (^), dollar signs ($), single quotation marks ('), double quotation marks ("), equal signs (=), and commas (,).

When do I use single quotation marks (') and double quotation marks (") to write data?

• If you write data based on the line protocol, do not use single quotation marks (') or double quotation marks (") to enclose identifiers. In the following examples, queries are more complicated after quotation marks are used. Identifiers can be continuous query names, database names, field keys, measurement names, retention policy names, subscription names, tag keys, or usernames.

  • Write a measurement that is enclosed in double quotation marks ("): INSERT "bikes" bikes_available=3. Applicable query: SELECT * FROM "\"bikes\"".

  • Write a measurement that is enclosed in single quotation marks ('): INSERT 'bikes' bikes_available=3. Applicable query: SELECT * FROM "\'bikes\'".

  • Write a measurement that is not enclosed in quotation marks: INSERT bikes bikes_available=3. Applicable query: SELECT * FROM "bikes".

• You must use double quotation marks (") to enclose string field values.

  Write data: INSERT bikes happiness="level 2". Applicable query: SELECT * FROM "bikes" WHERE "happiness"='level 2'.

• Do not use quotation marks to enclose special characters. Use backslash (\) escape characters to format special characters.

  Write data: INSERT wacky va\"ue=4. Applicable query: SELECT "va\"ue" FROM "wacky".

请查看文档行协议参考获得更多相关信息。

Does the timestamp granularity have a significant impact on system performance?

Yes, the timestamp granularity has a significant effect on system performance. To maximize system performance, we recommend that you use a coarse-grained time granularity to write data to TSDB for InfluxDB®.

In the following two examples, the default time granularity (in nanoseconds) is specified for the first request. The time granularity (in seconds) is specified for the second request:

curl -i -XPOST "https://<Endpoint>:3242/write?db=weather&u=<Account name>&p=<Password>"--data-binary 'temperature,location=1 value=90 1472666050000000000'

curl -i -XPOST "https://<Endpoint>:3242/write?db=weather&precision=s&u=<Account name>&p=<Password>"--data-binary 'temperature,location=1 value=90 1472666050'

However, if you use a more coarse-grained time granularity to write data, duplicate points that have the same timestamp are prone to occur. In this case, some points may be overwritten.

Why am I unable to I connect to TSDB for InfluxDB® by using the Influx CLI?

Check whether you meet the following conditions:

• You must have executable permissions. You can run the chmod +x ./influx command to add permissions.

• You must specify the -ssl option in the connection statement.

• You only need to specify the connection string for the -host option, such as ts-bp17j28j2y7pm****.influxdata.rds.aliyuncs.com. You do not need to specify the protocol and port number.

How do I enable the CLI of TSDB for InfluxDB® to return human readable timestamps?

When you connect to the CLI for the first time, you must specify the time granularity based on RFC 3339.

$ influx -ssl -username <Account name>-password <Password>-host <Endpoint>-port 3242-precision rfc3339

You can also specify the time granularity after you connect to the CLI.

$ influx -ssl -username <Username> -password <Password> -host <Endpoint> -port 3242
Connected to https://<Endpoint>:3242 version 1.7.x
> precision rfc3339

For more information, see CLI of TSDB for InfluxDB®.

How do I execute the USE statement to specify a database if I am not an administrator?

If you are not an administrator, you can execute the USE <database_name> statement to specify a database. However, in this case, you must have the READ and WRITE permissions, or only READ permission, for the database. Otherwise, the system returns the following error:

ERR:Database<database_name> doesn't exist. Run SHOW DATABASES for a list of existing databases.

How do I use the CLI of TSDB for InfluxDB® to write data to a non-default retention policy?

Execute the INSERT INTO [<database>.]<retention_policy> <line_protocol> statement to write data to a non-default retention policy. This method can be used only in the CLI of TSDB for InfluxDB® to specify the database and the retention policy. To write data through HTTP, you must use the db and rp parameters to specify the database and the retention policy, respectively. The rp parameter is optional. Example:

> INSERT INTO one_day mortality bool=true
Using retention policy one_day
> SELECT * FROM "mydb"."one_day"."mortality"
name: mortality
---------------
time                             bool
2016-09-13T22:29:43.229530864Z   true

If you want to query data in a non-default retention policy, you must fully qualify a measurement. You can use the following syntax to fully qualify the measurement:

"<database>"."<retention_policy>"."<measurement>"

Why am I unable to query the field values of the BOOLEAN data type?

The syntax to write BOOLEAN values is different from the syntax to query BOOLEAN values.

For example, the SELECT * FROM "hamlet" WHERE "bool"=True statement returns all points where the value of bool is TRUE. However, the SELECT * FROM "hamlet" WHERE "bool"=T statement returns no result.

How does TSDB for InfluxDB® handle differences in field data types across shards?

The data type of field values can be FLOAT, INTEGER, STRING, or BOOLEAN. The data types of field values must be the same in each shard. However, the data types of field values can be different across shards.

• SELECT statement

  A SELECT statement returns all field values if all the field values have the same data type. If the data types of field values are different across shards, TSDB for InfluxDB® performs the required operations to convert data types (if applicable). Then, TSDB for InfluxDB® returns all the field values based on the following data type sequence: FLOAT, INTEGER, STRING, and BOOLEAN. If field values are found to have different data types in your data, use the <field_key>::<type> syntax to query different data types. Example:

  In the just_my_type measurement, the my_field field has four values in four shards. The data types of these values of the my_field field are different from each other. The data types are FLOAT, INTEGER, STRING, and BOOLEAN.

  The SELECT * statement returns only the values of the FLOAT and INTEGER data types. In the response to the statement, TSDB for InfluxDB® converts the values of the INTEGER data type to the values of the FLOAT data type.

  > SELECT * FROM just_my_type
  
  name: just_my_type
  ------------------
  time                      my_field
  2016-06-03T15:45:00Z9.87034
  2016-06-03T16:45:00Z7

  The SELECT <field_key>::<type> [...] statements returns the values of all data types. TSDB for InfluxDB® stores the output data of each type in a separate column, and the column names are incremented, for example, my_field, my_field_1, and my_field_2. TSDB for InfluxDB® can be used to convert the data type of a field value into another data type. However, this is based on specific requirements. In the following example, TSDB for InfluxDB® converts the 7 integer to the floating-point number that is included in the first column. TSDB for InfluxDB® converts the 9.879034 floating-point number into the integer that is included in the second column. TSDB for InfluxDB® cannot convert a floating-point number or an integer into a string or a Boolean value.

  > SELECT "my_field"::float,"my_field"::integer,"my_field"::string,"my_field"::boolean FROM just_my_type
  
  name: just_my_type
  ------------------
  time                   my_field  my_field_1  my_field_2  my_field_3
  2016-06-03T15:45:00Z9.870349
  2016-06-03T16:45:00Z77
  2016-06-03T17:45:00Z                         a string
  2016-06-03T18:45:00Z                                     true

• SHOW FIELD KEYS statement

  The SHOW FIELD KEYS statement returns all the data types in each shard that is associated with a specified field key. Example:

  In the just_my_type measurement, the my_field field has four values in four shards. The data types of these values of the my_field field are different from each other. The data types are FLOAT, INTEGER, STRING, and BOOLEAN.

  The SHOW FIELD KEYS statement returns all of the four data types.

  > SHOW FIELD KEYS
  
  name: just_my_type
  fieldKey   fieldType
  -----------------
  my_field   float
  my_field   string
  my_field   integer
  my_field   boolean

What are the smallest and the largest integers that TSDB for InfluxDB® can store?

TSDB for InfluxDB® stores integers as signed Int64 values. The smallest valid Int64 value is -9023372036854775808. The largest valid Int64 value is 9023372036854775807. For more information, see Go builtins.

If the stored values are close to the smallest or largest integer, unexpected results may occur. Some functions and operators may convert Int64 values into Float64 values during computation. However, this can cause overflow issues.

What are the smallest and the largest timestamps that TSDB for InfluxDB® can store?

The minimum timestamp is -9223372036854775806 or 1677-09-21T00:12:43.145224194Z. The maximum timestamp is 9223372036854775806 or 2262-04-11T23:47:16.854775806Z. If timestamps fall out of the valid range, parsing errors occur.

How do I view the data types of fields?

You can execute the SHOW FIELD KEYS statement to view the data types of fields.

> SHOW FIELD KEYS FROM all_the_types
name: all_the_types
-------------------
fieldKey  fieldType
blue      string
green     boolean
orange    integer
yellow    float

Can I convert the data types of fields?

Yes, you can change the data types of fields. However, you can convert only a limited number of data types into other data types through TSDB for InfluxDB®. You can use the <field_key>::<type> syntax to convert field values from integers into floating-point numbers. You can also convert field values from floating-point numbers into integers. For more information about data type conversion, see Data Exploration. You cannot convert floating-point numbers or integers into strings or Boolean values. In similar cases, you cannot convert strings or Boolean values into floating-point numbers or integers.

You can use the following alternative methods to change data types:

• Write the data to another field

  The simplest method is to write the data of the new data type to another field in the same series.

• Use the shard system

  The data types of field values must be the same in each shard. However, the data types of field values can be different across shards.

  If you want to change the data type of a field value, you can execute the SHOW SHARDS statement to query the end_time value of the current shard. TSDB for InfluxDB® can be used to write data of a different data type to an existing field. This applies if the timestamps of points occur after the time that is specified by end_time. For example, if the data timestamps occur before the end time of the current shard, you can write only integers to a field. However, if the data timestamps occur after the time specified by end_time, you can write floating-point numbers to the field.

  This process does not change the data types of the field values in the original shard.

How do I perform mathematical operations within functions?

You cannot use TSDB for InfluxDB® to perform mathematical operations within functions. We recommend that you run subqueries to achieve the same purpose. Example:

InfluxQL does not support the following syntax:

SELECT MEAN("dogs"-"cats")from"pet_daycare"

However, you can run the following subquery to achieve the same purpose:

> SELECT MEAN("difference") FROM (SELECT "dogs"-"cat" AS "difference" FROM "pet_daycare")

For more information about subqueries, see Data Exploration.

Why does a query response use epoch 0 as a timestamp?

In most cases, epoch 0 (1970-01-01T00:00:00Z) is used as a null timestamp in TSDB for InfluxDB®. If no timestamps can be returned for your queries, TSDB for InfluxDB® returns epoch 0 as a timestamp. For example, this rule applies when no time ranges are specified for aggregate functions.

Which InfluxQL functions can be nested?

The following InfluxQL functions can be nested:

• COUNT() nested in DISTINCT()

• CUMULATIVE_SUM()

• DERIVATIVE()

• DIFFERENCE()

• ELAPSED()

• MOVING_AVERAGE()

• NON_NEGATIVE_DERIVATIVE()

• HOLT_WINTERS() and HOLT_WINTERS_WITH_FIT()

For more information about how to use subqueries as substitutes for nested functions, see Data Exploration.

How do I migrate a self-managed InfluxDB to the cloud?

You can use the official influx_inspect tool provided by InfluxDB to export line protocol files from your self-managed InfluxDB server, and then use CLI of TSDB for InfluxDB® to import the files into TSDB for InfluxDB®.

How do I migrate data between different InfluxDB instances in the cloud?

You cannot migrate data between different InfluxDB instances. To do this, you need to query and export the data to be migrated, and then manually migrate data. During the query process, you need to avoid large queries that affect system stability. You can filter by time or time series to migrate multiple pieces of data at the same time.

How do I migrate data from InfluxDB to LindormTSDB?

LindormTSDB provides Lindorm Tunnel Service (LTS) to import historical data from InfluxDB. You can use LTS to migrate full data from InfluxDB to LindormTSDB.