Lindorm provides the wide table engine, time series engine, search engine, file engine, compute engine, and stream computing engine. Lindorm is compatible with standard APIs of multiple open source software and services, such as Apache HBase, Apache Cassandra, Amazon Simple Storage Service (Amazon S3) OpenTSDB, Apache Solr, Hadoop Distributed File System (HDFS), and Apache Kafka. It also provides capabilities such as SQL queries, time series data processing, and text retrieval and analysis.

To meet the requirements of dynamic workloads, each engine can separately scale computing resources and storage resources based on your business requirements. The wide table engine and the time series engine provide high concurrency and high throughput.

Engines

Different engines are suitable for different scenarios. You can select one or more engines based on your business requirements. The following table describes the engines.

Engine Compatibility Scenario Description
Wide table engine Compatible with SQL, the HBase API, Cassandra Query Language (CQL), and the Amazon S3 API. Suitable for storing and processing metadata, orders, bills, user personas, social information, feeds, and logs. The wide table engine is used for the distributed storage of large amounts of semi-structured data and structured data. It supports global secondary indexes, multi-dimensional searches, dynamic columns, and the time-to-live (TTL) feature. The wide table engine can handle tens of millions of concurrent requests and store petabytes of data. It also provides the hot and cold data separation feature. Compared with the performance of open source Apache HBase, the read/write performance is increased by 2 to 6 times, the percentile 99% (P99) latency is decreased by 90%, the compression ratio is decreased by 100%, and the storage cost is decreased by 50%.
Time series engine Provides an HTTP API and is compatible with the OpenTSDB API. Suitable for storing and processing time series data such as measurement data and operational data of devices in scenarios such as IoT and monitoring. The time series engine is a distributed storage engine that is used to process large amounts of time series data. It supports SQL queries. The time series engine provides a dedicated compression algorithm for time series data. This helps improve the data compression ratio. The time series engine allows you to use multiple dimensions to query and aggregate large amounts of time series data by timeline. The engine also supports downsampling and elastic scaling.
Search engine Compatible with SQL and the Apache Solr API. Suitable for querying large amounts of data, such as logs, text, and documents. For example, you can use the search engine to search for logs, bills, and user personas. Lindorm provides a distributed search engine. The search engine uses an architecture in which storage is decoupled from computing. The search engine can be seamlessly used to store the indexes of the wide table engine and the time series engine to accelerate data retrieval. The search engine provides various capabilities, including full-text searches, aggregation, and complex multi-dimensional queries. It also supports an architecture that consists of one write replica and multiple read-only replicas and provides features such as horizontal scaling, cross-zone disaster recovery, and TTL to meet the requirements of efficient retrieval of large amounts of data.
File engine Compatible with the HDFS API. Suitable for scenarios in which enterprise-grade data lakes are used for storage, Apache Hadoop is used as a storage base, or historical data is archived and compressed. The file engine provides cloud native storage capabilities. It is compatible with HDFS communication protocols. You can directly connect to the file engine by using open source HDFS clients. You can call the HDFS API to use all the features of the file engine. You can also seamlessly connect the file engine to all open source HDFS ecosystems and cloud computing ecosystems. The file engine is developed and optimized based on HDFS. It can store exabytes of data at a low cost and perform automatic scale-up operations within a few minutes. The file engine also provides multiple features, such as horizontal bandwidth scaling. The file engine is suitable for building enterprise-grade low-cost data lakes based on HDFS. The decoupled storage and computing architecture of the file engine helps reduce the overall cost.
Compute engine Compatible with the Apache Spark API. Suitable for scenarios such as the production of large amounts of data, interactive analytics, computational learning, and graph computing. The compute engine provides distributed computing services based on a cloud native architecture. It supports Community Edition computing models and programming interfaces. The compute engine also integrates the features of the Lindorm storage engine and uses the underlying data storage features and indexing capabilities to efficiently complete distributed jobs.
Stream computing engine Compatible with SQL and the Apache Kafka API. Suitable for scenarios such as IoT data processing, application log processing, logistics aging analysis, and travel data processing. The Lindorm stream computing engine is used to store and process streaming data. It provides lightweight computing capabilities. You can use the stream computing engine to store streaming data to Lindorm to meet the requirements for the processing and application of streaming data.