Lindorm:Release notes of LindormTSDB

Release date

Category

Description

2023-12-26

Optimized features

• The time period that is required for LindormTSDB to restart is reduced when the databases in LindormTSDB contain a large number of partitions.

• The following issue is fixed: The start time or end time for a continuous query is incorrectly calculated.

• The following issue is fixed: Data that is written to LindormTSDB after a data delete operation cannot be queried.

• The read and write performance of LindormTSDB is optimized. The stability issues of LindormTSDB are fixed.

Release date

Category

Description

2023-11-02

Optimized features

The stability of LindormTSDB is improved.

Release date

Category

Description

2023-10-11

Optimized features

• The experience of memory usage of LindormTSDB is optimized.

• The query performance of LindormTSDB is optimized.

• The compatibility of PromQL is improved.

• The following issue is fixed: The performance of downsampling queries performed on a 24-hour basis is not stable.

Release date

Category

Description

2023-09-12

New features

The standard CREATE TABLE statement is supported to create tables.

Optimized features

• The following issue is fixed: A query fails when the aggregation results based on time series filters in the subquery is not specified in the query.

• The following issue is fixed: Memory resources used by queries are not properly released.

• The following issue is fixed: The metadata cannot be retained properly when the value of the TTL parameter is deleted for a shard.

• Health check for TSProxy is enabled for instances by default.

• Slow query management is enabled for instances by default.

• The read and write performance of LindormTSDB is optimized. The stability issues of LindormTSDB are fixed.

Release date

Category

Description

2023-08-09

Optimized features

• The following issue is fixed: Authentication cannot be performed based on databases by calling API operations.

• The compatibility issue of PromSQL is fixed.

• The following issue is fixed: Data exceptions may occur when new data is written to LindormTSDB by overwriting existing data.

• The read and write performance of LindormTSDB is optimized.

Release date

Category

Description

2023-06-30

Optimized features

• The default username and password are changed.

• Some issues that may result in query exceptions are fixed.

• The read and write performance of LindormTSDB is optimized.

Release date

Category

Description

2023-06-06

Optimized features

The read and write performance of LindormTSDB is optimized.

Release date

Category

Description

2023-05-16

New features

The window and offset attributes are supported for continuous queries.

Optimized features

• The read and write performance of LindormTSDB is optimized.

• The following issue is fixed: Continuous queries may be unexpectedly terminated during execution.

Release date

Category

Description

2023-04-06

New features

Lindorm ML supports downsampling and automatic interpolation to obtain training data in time series forecasting. Training data can be automatically used for model inference.

Optimized features

• The write performance of LindormTSDB is optimized.

• The following issue is fixed: Resources are not released when a single-value query times out.

• The following issue is fixed: The data in the specified table is not filtered based on time partitions when the latest function is used in a query.

• The following issue is fixed: The memory may be exhausted when SQL is used to write data.

Release date

Category

Description

2023-03-14

New features

• Data files can be sorted by time window.

• The downsampling operator irate is added.

• Lindorm ML:

  • Nested SQL functions can be used together with non-downsampling operators to detect anomalies in time series data.

  • Historical covariates are supported in time series prediction.

Optimized features

• The following issue is fixed: An error may occur when a field column of the BIGINT type is used as the filter condition in a PrepareStatement statement.

• The following issue is fixed: The line protocol is not compatible with backslashes (\).

• The following issue is fixed: The latest() function does not return the latest value.

• The query performance of LindormTSDB is optimized.

• Lindorm ML:

  • The check logic for input parameters in time series anomaly detection and time series forecasting is optimized. The optimized logic returns an error when an unknown input parameter is specified.

  • The following issue is fixed: No inference result is returned when the frequency of the input time series data does not match the specified frequency in time series data forecasting.

  • The following issue is fixed: Model training may fail in time series anomaly detection because multiple model files are concurrently saved during the training.

  • The following issue is fixed: An error occurs when the istl-esd algorithm automatically detects data frequency in time series anomaly detection.

  • The following issue is fixed: The istl-esd algorithm does not return results as expected in time series anomaly detection.

  • The following issue is fixed: An error occurs for the verbose parameter of the OneShot STL and Online STL algorithms in time series anomaly detection.

Release date

Category

Description

2023-02-14

New features

• Access logs of Lindorm instances can be queried on the Lindorm console.

• The following features are supported for time series anomaly detection provided by Lindorm ML:

  • Nested SQL functions can be used to detect anomalies in time series downsampling data.

  • The Literal variable can be specified as the query target when you use nested SQL statements to perform time series anomaly detection.

  • SQL subqueries can be used to query the verbose column in the result when you use the SELECT statement to query the anomaly detection result.

  • The istl-esd detection algorithm is supported. This algorithm is an OneShot STL (Incremental STL) algorithm developed by DAMO Academy.

Optimized features

• The performance of queries with regular conditions are optimized.

• The following issue is fixed: The performance of downsampling queries performed on a 24-hour basis is not stable.

• The time period that is required for LindormTSDB to restart is reduced.

• The following issue is fixed: The scheme may fail to be automatically created when the weak mode is used.

• The following issue is fixed: The NullPointerException (NPE) error occurs when the fill linear policy is used in downsampling queries.

• The following issue is fixed: An error may occur when you use DMS to query data.

• The following issue is fixed: Data cannot be written to LindormTSDB by using SQL statements when the data type is set to BigDecimal.

• The following issue is fixed: You may fail to access the LindormTSDB node when you scale up the node after data with a timestamp in distant future is written to the node.

• The issue that the following parameters for time series anomaly detection algorithms do not take effect is fixed:

  • The alpha and warmupCount parameters of the esd algorithm.

  • The direction and warmupCount parameters of the nsigma algorithm.

• The following issue is fixed: An error occurs when you switch time series during time series anomaly detection performed by using STL-based algorithms.

• The mechanism of time series anomaly detection is optimized. The status of a model does not change if data that has been detected is detected again.

Release date

Category

Description

2023-01-06

Optimized features

The stability of LindormTSDB is improved.

Release date

Category

Description

2023-01-03

New features

• The policies used to merge index files are optimized. The COMPACTION operation can be performed based on the index file size.

• The verbose parameter is added for the ANOMALY_DETECT function of Lindorm ML. You can configure this parameter to output required auxiliary information.

• New features are added for the ANOMALY_DETECT function of Lindorm ML to reset and export the status of models.

Optimized features

• The query performance on partitions is improved.

• The following issue is fixed: A large number of version conflicts occur when multiple schemas are concurrently updated.

• The following issue is fixed: The FORECAST function of Lindorm ML cannot obtain enough data for time series forecasting if a time range that is earlier or later than a specific point in time is specified as the forecast condition.

• The following issue is fixed: Errors occur when the ostl-ttest and ostl-esd algorithms are used to perform time series anomaly detection on large amounts of data.

• The time series anomaly detection performance of Lindorm ML is optimized.

Release date

Category

Description

2022-11-22

New features

• Single pre-defined values can be specified as filter conditions in downsampling queries.

• Time series can be queried by using SQL statements.

• Data can be subscribed by database in LTS.

• The raw function is added for Lindorm ML. This function can be used in time series anomaly detection to output the raw values of fields.

Optimized features

• APIs related to series are supported by PromQL for queries.

• The following issue is fixed: In downsampling queries, server resources are not released after the client is disconnected.

• The default maximum number of connections for a single client is changed to 4096.

• The following issue is fixed: A time series forecasting task may fail when the task is performed by using multiple concurrent threads.

• The write and query performance of LindormTSDB is optimized.

Release date

Category

Description

2022-09-27

New features

Regular queries are supported by PromQL.

Optimized features

The write performance when pre-downsampling is enabled is optimized.

Release date

Category

Description

2022-08-31

New features

• Lindorm machine learning (ML) is supported.

• The feature binning operator is supported by Lindorm ML.

• The extension syntax of Lindorm ML is supported by TSQL.

• Time series anomaly detection algorithms are supported.

• Time series anomaly detection functions are supported.

• The maximum number of columns that can be modified in a single table is limited to 200.

• The query performance for large numbers of time series is optimized.

Optimized features

• The following bug is fixed: An error is returned when you execute the SHOW PROCESSLIST statement after permission verification is enabled.

• The following bug is fixed: The latest values cannot be correctly queried when data is updated.

• The following bug is fixed: The type of the data returned by time series anomaly detection functions is incorrect in subqueries.

Release date

Category

Description

2022-07-29

New features

• You can configure rules to discard data in the past or future when you write data to LindormTSDB to avoid data disorder.

• The partition_interval attribute is set to 30 days by default when you create LindormTSDB databases.

• Time series anomaly detection algorithms are supported.

• Pre-downsampling is supported.

• The deletion of time series is supported.

Optimized features

• The following bug is fixed: An type mismatch error is returned when you end an SQL statement that is being executed.

• The following bug is fixed: Lowercase parameters are not supported when you use API operations to access schema_policy.

• The following bug is fixed: Not a Number (NaN) data points cannot be processed by PromQL.

• The following bug is fixed: Data is missing when you perform previous interpolation.

Release date

Category

Description

2022-07-06

New features

The deletion of time series is supported by TSQL.

Optimized features

The following bug is fixed: An exception may occur when the conditions in a TSQL query contain BOOLEAN values.

Release date

Category

Description

2022-06-17

New features

The maximum time period in which data is written can be configured. The maximum time period is calculated as the sum of the current time plus the offset time. The data that is written after the offset period elapses is discarded.

Optimized features

• The logic that is used to verify the earliest time of data is changed.

• The following bug is fixed: An error is returned when negative numbers represented in scientific notation are specified in SQL statements.

• The following bug is fixed: An error is returned when data represented in scientific notation is written by using the line protocol of InfluxDB.

• The following bug is fixed: Continuous queries (CQs) fail.

• The following bug is fixed: An error is returned for COUNT(1) in nested subqueries.

Version

Release date

Category

Description

3.4.16

2022-05-31

New features

• Downsampling queries support downsampling operations that are based on local time zones.

• The syntax of latest value queries supports multi-column queries.

• SQL statements that are being executed can be queried and managed.

• Metadata queries are supported when PromQL is used to access data.

3.4.15

2022-05-16

New features

Downsampling queries support the offset feature.

Optimized features

• If an SQL query does not contain a time condition, the start time is set to 0 by default.

• The following bug is fixed: When the COUNT function is used together with other operators in downsampling queries, an error is returned because the data type fails to be converted.

• The following bug is fixed: LIMIT clauses in PrepareStatement statements do not support the LONG data type.

• The following bug is fixed: Data cannot be manually deleted after storage is locked.

• The following bug is fixed: SQL statements are not executed as expected when the SQL API path api/v2/sql is used to continuously initiate queries over the same connection that is established for a long period of time.

• The following bug is fixed: The time to live (TTL) value is set to 0, but the configuration does not take effect.

3.4.14

2022-04-25

New features

• Read and write thread pools are differentiated for Transact-SQL (T-SQL) access.

• Execution logs can be recorded for CQ statements.

Optimized features

• The following bug is fixed: An error is returned when a UNION operation is performed for a latest value query.

• The following bug is fixed: The close() method is not called for a T-SQL statement after the connection established by using a JDBC or SQL API is closed.

• The following bug is fixed: Line protocol conflicts occur when schemas are concurrently updated.

3.14.13

2022-04-14

New features

• The SHOW CREATE TABLE statement is supported.

• The length of a time window can be specified for CQ statements.

• The data returned by DESCRIBE TABLE statements can be displayed in the order in which table columns are created.

Optimized features

• The following bug is fixed: The prepared statement does not support downsampling queries.

• The following constraint is added when the line protocol is used to write data: The tag column and the field column cannot have the same column name.

• The following bug is fixed: CQs cannot be run as expected after permission management is enabled.

3.14.12

2022-03-30

New features

• Different TTLs can be specified to support different pre-downsamplings.

• By default, data of the DOUBLE data type can be converted into data of the BIGINT type when data of the BIGINT type is written to columns of the DOUBLE data type.

Optimized features

• The following bug is fixed: An unexpected result is returned when the most recent values of the BOOLEAN type are queried.

• Queries can be canceled after SQL connections are closed.

3.4.11

2022-03-16

New features

• SQL statements support custom names for time columns.

• SQL statements can be executed to change the data types of columns in time series tables.

Optimized features

• The following bug is fixed: An error is returned for a DISTINCT statement in some query scenarios.

• The following bug is fixed: Multiple data entries may be returned for latest value queries after nodes are added.

• The following bug is fixed: Constant folding is not supported in downsampling queries.

• The following issue is fixed: The time condition in a downsampling query that uses a regular expression does not take effect.

3.4.10

2022-02-28

New features

• Tags in SQL query statements can be filtered by using regular expressions.

• The non_negative_rate and non_negative_delta time series functions are supported.

• Length limits for tables, databases, CQs, and columns are added.

Optimized features

• The performance of scanning a large number of data points for queries is optimized.

• The following bug is fixed: Two time series tables that contain columns whose data types are the same cannot be joined.

3.4.9

2022-02-14

New features

• The DESCRIBE TABLE statement can return a table schema that contains the PRIMARY KEY keyword.

• SQL statements are case-sensitive.

• SQL statements can be executed to write NULL values to tags and fields.

• SQL statements can be executed to manage pre-downsampling rules. Downsampling rules can be created for a single table.

• The rules based on which errors are returned for the execution of SQL statements are improved. For example, the returned error codes and corresponding descriptions are standardized.

Optimized features

• The following bug is fixed: The data that is not in the specified time range is returned for a latest value query.

• Backticks (`) are not required for the interval parameter when a CQ is created.

3.4.8

2022-02-07

Optimized features

The bug that occurs for the hash policy is fixed.

3.4.7

2022-01-26

New features

• The T-SQL statement that is executed to write data must contain at least one field column.

• T-SQL statements can be executed to create or delete pre-downsampling rules.

• T-SQL statements support SAMPLE BY 0.

• RATE and DELTA functions can be nested in T-SQL statements.

• PromQL supports basic authentication. The DB parameter can be configured in PromQL statements.

• Strong schema constraints are supported for data that is written by using the line protocol.

• The maximum length of a table is 128 bytes.

Optimized features

• The following bug is fixed: Some data may fail to be deleted if tables are repeatedly deleted.

• The following bug is fixed: Integers fail to be written to two columns.

• The following bug is fixed: An error occurs when data in a single column in a nested subquery is downsampled.

3.4.6

2022-01-11

New features

• The schema policy can be configured based on the InfluxDB line protocol. Schemas can be created based on your business requirements.

• Database names and table names are case-sensitive in T-SQL statements.

• T-SQL query performance is optimized.

• T-SQL statements support GROUP BY queries.

• T-SQL statements support pre-downsampling queries.

• T-SQL statements support the ALTER TABLE syntax.

• By default, the schema consistency is checked for T-SQL INSERT statements.

• The /api/v2/sql API path can be used to return data in batches based on the HTTP CHUNKED mode.

Optimized features

• The following bug is fixed: Data in the cache fails to be flushed due to a cache flush exception.

• The following bug is fixed: CQs fail to be run because the JDBC prefixes do not match.

• The following bug is fixed: The time strings in SQL statements for data queries and writes are processed by using different methods.

• The following bug is fixed: Exceptions are thrown when sample by fill and percentile are used.

• The following bug is fixed: A NullPointerException (NPE) occurs when the access-control list (ACL) is disabled and a user is created.

3.4.5

2021-12-24

New features

• MySQL supports authentication.

• T-SQL statements can be used to improve the performance of simple queries.

• T-SQL statements support quantile functions.

• API operations support field queries.

• The time-based partitioning feature is optimized. The time-based partitioning feature can be disabled.

• The line protocol and API operations support the schemaless mode for data writes. You can modify SQL statements to configure the schemaless mode. The default value is schemaless.

Optimized features

• The following bug is fixed: If data is written to a table after the table is deleted, the data is lost.

• The following bug is fixed: An NPE occurs when the format of an api/v2/write API request is invalid.

• The following bug is fixed: An error occurs when multi-value downsampled data is queried by using the /api/mquery API path.

• The following bug is fixed: Historical files cannot be migrated to cold storage when the storage space that is occupied by historical files does not reach the upper limit.

• The bug that causes stack overflow for queries is fixed.

• The following issue is fixed: Fields of the STRING data type cause a compaction exception.

3.4.4

2021-12-14

New features

• SQL statements support the IF EXISTS syntax.

• SQL statements support the SHOW PRIVILEGES syntax and SHOW PARAMETER syntax.

• The unit of the interval at which data is downsampled can be milliseconds.

• In the CREATE TABLE syntax, the TIME and FIELD columns are required and partitioning rules can be configured.

• The maximum number of FIELD columns is 1,024.

• The logic that the current time is used as the default end time is deleted.

• RATE and DELTA functions can be used in queries.

• The pipeline processing logic is optimized to improve query performance.

• DESCRIBE TABLE statements can be executed to return the data types of columns.

• The read and write compression algorithm for data of the STRING data type is optimized to improve write performance.

• By default, the pre-downsampling rule avg is implemented based on the count or sum operations.

Optimized features

• The following bug is fixed: Downsampled data of the STRING data type fails to be queried.

• The following bug is fixed: An error occurs when the commit method or rollback method is called.

• The following bug is fixed: The content of an error that occurs when the time column is not specified in INSERT statements is not friendly.

• The following bug is fixed: An error occurs when an SQL statement that contains a SQL JOIN clause is executed.

• The bug that causes a count error for pre-downsampled memory data is fixed.

3.4.3

2021-12-01

New features

• The Grafana plug-in supports database authentication and user authentication.

• An independent flush thread is used for pre-downsampling to improve the pre-downsampling performance.

• DESCRIBE DATABASE statements can return database properties.

• The limit between the TTL value and the hot data and cold data boundary is removed.

• By default, the time-based partitioning feature is disabled when you create an instance.

• The statistical information about the data that is written to a table is added. The amount can be queried by using Lindorm-cli.

• Strings are processed as byte arrays. The proxy can compress strings.

• The number of partitions can be specified when a database is created.

Optimized features

• The following bug is fixed: Data is blocked when the data is flushed to time partitions on the disk.

• The stability-related bug that occurs when a database is created or deleted is fixed.

• The NPE bug that occurs when a database is updated is fixed.

• The following bug is fixed: Incorrect error information is returned when the time range of the time column in an INSERT statement is invalid.

• The bug that causes duplicate data when you run SQL queries across partitions is fixed.

• The following bug is fixed: The end time is not included when API queries are performed.

• The following bug is fixed: Empty strings cannot be written.

• The following bug is fixed: An error occurs for a column that stores values of multiple data types in pre-downsampled data.

3.4.2

2021-11-15

New features

• Streaming pipelines are used for latest value queries and downsampling in T-SQL.

• Streaming pipelines support the BIGINT data type.

• The authentication syntax is supported for T-SQL statements. LindormTable SQL statements are supported.

• Databases support the skip_wal option.

• The implementation rule of the pre-downsampling algorithm is optimized.

• PromQL supports multi-value queries.

Optimized features

• The following bug is fixed: T-SQL statements do not support filter conditions that include Chinese characters.

• The following bug is fixed: An error occurs when the AVG function is used to query pre-downsampled data.

• The following bug is fixed: If you delete a table when you write data, query exceptions may occur.

• The bug that occurs during write-ahead logging (WAL) rollback or during file replacement is fixed.

• The NPE bug that occurs for latest value queries is fixed.

• The following bug is fixed: An error occurs due to schema update when a T-SQL statement is executed to write data.

• The following bug is fixed: The suggest feature returns duplicate metrics.

• The following bug is fixed: A database name cannot be less than three characters in length.

• The following bug is fixed: The connections created by executing T-SQL statements are unavailable for new databases that are created.

3.3.4

2021-10-28

Optimized features

The NPE bug that occurs for latest value queries is fixed.

3.4.1

2021-10-22

New features

• T-SQL queries are optimized and a method of streaming data queries is added.

• The compression ratio of TSFile files is improved.

• The logic based on which the schema of TSCore is queried is removed from T-SQL to improve the T-SQL query performance. You must create a table for the data that is written by the SDK before you execute T-SQL statements to query the data.

• The error messages that are returned for T-SQL DML statements can be displayed.

Optimized features

• The following bug is fixed: If the details parameter is used, the SDK cannot receive responses.

• The following bug is fixed: TSCore schema memory overflow may occur when a large number of timelines exist.

• The following bug is fixed: TSCore full garbage collection (GC) is caused by large queries when multiple partitions are queried.

• The following bug is fixed: Data may fail to be queried due to multiple TSCore components.

3.4.0

2021-10-13

New features

• T-SQL statements support the BIGINT data type.

• T-SQL INSERT statements can be executed to write NULL values.

Optimized features

The efficiency of parallel data queries is improved.

Version

Release date

Category

Description

3.3.4

2021-10-28

Optimized features

The NPE bug that occurs for latest value queries is fixed.

3.3.3

2021-09-08

New features

• The SDK supports queries for pre-downsampled data.

• The OpenTSDB protocol supports the GROUP BY feature.

Optimized features

• The following bug is fixed: Data writes are blocked because the flush thread of WAL logs exits due to an out of memory (OOM) error.

• The following bug is fixed: WAL logs are stacked when pre-downsampled data fails to be flushed to the disk.

3.3.2

2021-08-25

New features

By default, the time-based partitioning feature is enabled. The default validity period is 30 days.

Optimized features

• The following bug is fixed: The returned error is not accurate when single-value data is written by using the /api/mput API path.

• The following bug is fixed: The TTL feature cannot be implemented in the default partition.

• The following bug is fixed: In scenarios in which data is overwritten, data points at a single point in time are duplicate after data is flushed.

3.3.1

2021-08-12

New features

• General compression algorithms are supported. By default, general compression is not enabled.

• Line protocol data can be represented in scientific notation.

• The verification policy for special characters is updated.

Optimized features

• Compaction policies are optimized to improve efficiency.

• JVM parameters are optimized.

• The bug related to interpolation alignment is fixed.

• The remote procedure call (RPC) process is optimized in scenarios in which cross-cluster queries are interrupted.

• The following bug is fixed: Data cannot be written to a new table after a table whose name is the same as the name of the created table is deleted.

3.3.0

2021-07-28

New features

PromQL queries are supported.

Optimized features

• The following bug is fixed: The connection to the SDK is closed.

• The following bug is fixed: T-SQL statements cannot be executed to query single-value data.

• The following bug is fixed: T-SQL alias settings cannot take effect.