Lindorm Edge is a high-performance, low-cost, stable, and reliable online time series database service that can be embedded into edge devices.
Lindorm Edge provides efficient read/write, high compression ratio storage, time series data interpolation, and aggregate computing services. It is widely used in the Internet of Things (IoT) equipment monitoring system, enterprise energy management system (EMS), production safety monitoring system, power detection system, and other industry scenarios.
Lindorm Edge can write millions of time series data in seconds. It provides functions, such as high compression ratio and low-cost storage, pre-drop sampling, multidimensional aggregation calculation, and visualization of query results. It solves problems, such as high storage cost, poor writing performance, low query, and analysis efficiency due to the huge number of equipment collection points and high data collection frequency.
Lindorm Edge is a high-performance, low-cost, easy-to-use, and fast operations and maintenance (O&M) database that provides:
It provides efficient read and write capabilities. The read/write efficiency has improved several times over when comparing it with open-source OpenTSDB.
It compresses raw data effectively based on an efficient compression algorithm, saving up to 90% of the storage space.
The console provides a wide range of data management and O&M functions. The operation is simple and convenient, allowing you to complete daily data management and O&M easily.
Lindorm Edge provides professional monitoring and alarm systems with quick O&M capabilities. The R&D Team features world-class database experts that provide professional technical support.
The core of Lindorm Edge is the integration of inverted index, TSM data compression format, rich dimensional aggregate query, and the TICK (telegraf, Lindorm Edge, chronograf, and kapacitor) ecosystem, which supports the promql query. Inverted index accelerates multi-dimensional query. The TSM data file is used for compact compression, which reduces storage space costs and speeds up the query. The rich dimensional aggregation query helps users analyze data more conveniently. The Enterprise Edition of Edge is composed of three nodes that guarantee service reliability and data consistency through the raft conformance protocol. All three nodes can accept write and query requests; once one of the nodes fails, the remaining two nodes can still serve as normal.
Lindorm Edge provides features, such as efficient read/write of time series data, data management, and high compressed storage, to ensure the stability of your business.
Lindorm Edge can read and write time series data ultra-fast. The response time is less than five seconds when reading millions of pieces of data. It can write tens of millions of pieces of data per second.
Lindorm Edge supports data writing through the Rest protocol.
Lindorm Edge supports two methods of querying data, using the Rest protocol or from the client.
You can set the data retention policy through the console or API. After the data retention policy is set, the data will be fragmented according to the shard duration, and the expired shard data will be deleted automatically.
You can execute data cleaning based on Measurements on the console or execute more flexible data cleaning through APIs.
Lindorm Edge uses efficient data compression technology to reduce the average storage space used by every data point to 1-2 bytes. This can reduce storage space usage by 90% and speed up data writing simultaneously.
Lindorm Edge provides rich time series data calculation functions, such as Aggregation, Selector, Transformation, and Predictor. It supports down-sampling and multi-dimensional aggregation of data and can meet various complex business data query scenarios.
Lindorm TSDB provides a real-time O&M system that displays the instance status, performance metrics, and storage usage, which helps you identify resource bottlenecks through alarm settings.
Lindorm Edge provides the following solutions to ensure data and instance security:
IoTDB is a database designed for IoT time series data, providing data collection, storage, and analysis functions. IoTDB provides an integrated solution on the cloud with high-performance data reading and writing and multi-dimensional query capabilities. It also has an efficient directory organization structure dedicated to IoT scenarios and is compatible with big data systems, such as Apache Hadoop, Spark, and Flink. At the edge, IoTDB provides lightweight TsFile management capabilities, writing data generated by edge devices to the local TsFile and providing specific basic query capabilities. Users can sync the TsFile data to the cloud simultaneously.
A tree structure is implemented through the organization of IoTDB metadata. An instance contains multiple Storage Groups (similar to the concepts of Namespace and Database), and a Storage Group contains multiple Devices. Each Device contains multiple Measurements, and the time series data corresponding to each Measurement is eventually stored in TsFile Chunk. In addition, to manage data duration, the data of each Storage Group will be segmented and stored in different directories based on a time range. It uses one week as a unit by default.
TimescaleDB is a time series database extended based on PostgreSQL. The data is written in chunks automatically by time and space.
The data in TimescaleDB must be displayed as two-dimensional tables, which require users to design and organize their business data this way.
There are two paradigms in the TimescaleDB official documentation that describes how to design tables using time series data.
For Narrow Table, the metrics are recorded separately. One row represents one record, and each row only contains one metricvalue-timestamp. Examples:
For Wide Table, the data is stored according to the timestamp. In this manner, you can store multiple metrics associated with a device in the same row. In general, the attributes (metadata) of a device can even be stored in other tables as auxiliary data for a single record. These kinds of data should be queried with the JOIN statement when needed.
Lindorm Edge uses adaptive intelligent algorithms. Multiple encoding, different compress algorithms and five types of data are used including timestamp, floating, integer, Boolean, and string. In this section, XOR encoding algorithm will be introduced.
XOR algorithm for encoding is roughly as follows.
The following is an example for XOR compression encoding of 1,600,000 Point value sampling data in Lindorm.
These are the information we can get from the results.
In addition, the XOR encoding algorithm is particularly effective for integer Point values.
The edge of Lindorm Edge realizes batch synchronization with the cloud through the File upload API and transmits the encoded TSM files directly. This approach has a high compression ratio, which saves bandwidth, but has high (minute or hour level) latency. The internal data on the edge is synchronized in real-time with the Replication Manager through WAL. The file hard link is used, which does not increase the storage cost. By watching WAL, parsing logs, and uploading via the http protocol, a high level of real-time performance is achieved with only seconds (or even millisecond) delays, but it requires high bandwidth and consumes more resources.
Lindorm Edge is widely used in industries and scenarios, such as Internet of Things (IoT) equipment monitoring systems, enterprise energy management systems (EMS), production safety monitoring systems, and power detection systems.
IoT devices generate massive amounts of device status data and business message data all the time. These data are helpful for device monitoring, business analysis and prediction, and fault diagnosis. The device sends the raw data to the IoT suite through the MQTT protocol, forwards the data to the message service system through the IoT suite, and performs real-time calculation processing on the data through the streaming computing system. Then, it writes the data to Lindorm Edge or writes raw data directly to Lindorm Edge through the IoT suite. The frontend monitoring system and big data processing system will use Lindorm Edge's data query and calculation analysis capabilities for business monitoring and real-time display of analysis results.
Traditional power chemical and industrial manufacturing equipment requires real-time monitoring systems for equipment status detection, fault discovery, and business trend analysis.
The equipment connects its status and production business data to the industrial equipment gateway through the industrial interface protocol and sends it to the IoT suite through the MQTT protocol. Then, it transmits it to the message service system on the cloud and processes it with the stream computing system before being written to Lindorm Edge, thus completing the storage and analysis of time series data.
You can understand equipment status, resource utilization, and business trends in real-time and realize digital operations, automated development, and O&M by monitoring large-scale application clusters and equipment in the data center.
By collecting raw data and performing real-time calculations through logs and other methods, the results of the real-time calculations are stored in Lindorm Edge to realize monitoring, analysis, and display.
Alibaba Clouder - February 14, 2020
ApsaraDB - June 4, 2020
Alibaba Clouder - September 28, 2020
Alibaba Clouder - November 20, 2020
XianYu Tech - October 10, 2019
ApsaraDB - January 26, 2021
Provides secure and reliable communication between devices and the IoT Platform which allows you to manage a large number of devices on a single IoT Platform.Learn More
Link IoT Edge allows for the management of millions of edge nodes by extending the capabilities of the cloud, thus providing users with services at the nearest location.Learn More
A cloud solution for smart technology providers to quickly build stable, cost-efficient, and reliable ubiquitous platformsLearn More
Migrating to fully managed cloud databases brings a host of benefits including scalability, reliability, and cost efficiency.Learn More
More Posts by ApsaraDB