The TimeSeries model is designed based on the characteristics of time series data. This model is suitable for scenarios such as IoT device monitoring and can be used to store data collected by devices and the monitoring data of machines. The TimeSeries model supports automatic indexing of time series metadata and time series retrieval based on composite conditions. The TimeSeries model uses time series tables to store time series data and provides high concurrent write and queries capabilities. You can use the TimeSeries model to store petabytes of time series data at low costs. You can execute SQL statements to query and analyze time series data.
Model overview
The TimeSeries model is designed based on the characteristics of time series data. This model is suitable for scenarios such as IoT device monitoring and can be used to store data collected by devices and the monitoring data of machines.
In the TimeSeries model of Tablestore, a two-dimensional time series table is used to store time series data. Each row represents the data at a point in time in a time series. The time series identifiers and timestamp are the primary key columns of the row, and the data points of the time series under the timestamp are the data columns of the row. A row can contain multiple data columns. You do not need to predefine the schemas of the primary key columns and data columns. You only need to specify the names of the specific data columns when you write data to the time series table.
A time series table can store time series data of different metric types. In the following figure, the time series table stores data of the following two metric types: temperature and humidity.
In the figure, time series identifiers consist of measurement, data source, and tags. You can call an API operation to update the properties in the metadata of a time series. The properties can be used to retrieve the time series.
After data is written to a time series table, the system automatically extracts the metadata of the time series and automatically creates a metadata index. You can retrieve a time series based on the combination of the metric name, data source, and tags.
The following items describe the benefits of the TimeSeries model:
Provides a unified-common modeling method for time series data, which eliminates the need to predefine table schemas.
Allows metadata indexes to be automatically created for time series and supports time series retrieval based on composite conditions.
Supports queries and aggregation by using SQL.
Supports automatic scale-out of service capabilities, high-concurrency writes and queries, and low-cost storage of petabytes of data.
Terms
Term | Description |
time series data | Time series data consists of multiple time series. Each time series is a set of data points that are arranged in chronological order. Metadata is required to identify a time series. Therefore, time series data consists of metadata and data.
|
time series metadata | Time series metadata contains the identifiers and properties of a time series. The identifiers are used to uniquely identify a time series. The properties can be modified and can be used to retrieve time series. |
time series identifier | Time series identifiers are used to uniquely identify a time series. In the TimeSeries model of Tablestore, time series identifiers consist of the following three parts: metric name, data source, and tag. |
metric name | The name of a physical quantity or metric for data in a time series, such as cpu or net, which indicates the CPU usage or network usage. |
data source | The identifier of the data source for the time series. This parameter can be empty. |
tag | The tag of the time series. You can specify multiple key-value pairs of the string type. |
property | Properties are part of time series metadata and can be used to record modifiable property information of a time series. However, the properties cannot be used as the identifier of a time series and cannot be used to uniquely identify a time series. The properties of a time series are multiple key-value pairs of the string type, which are similar to tags in the format. You can specify or update the properties of a time series to retrieve the time series by using the properties. |
data in a time series | A data point in a time series consists of the time when the data is generated and the corresponding data value. If only one value is generated at each moment in a time series, the single-value model is used. If multiple values are generated at each moment in a time series, the multi-value model is used. The TimeSeries model of Tablestore uses the multi-value model. You can specify multiple data values at one point in time. Each value corresponds to a column in the time series table, including the column name and column value. Column values support the following data types: Boolean, integer, floating-point, string, and binary. |
Features
Create and manage time series tables
You can use the Tablestore console, SDKs, or CLI to query all time series tables in an instance, create a time series table, query the configurations of a time series table, update the configurations of a time series table, and delete a time series table.
When you create a time series table or update the configurations of a time series table, you can specify the time to live (TTL) for the data in the time series table. After the TTL value is specified, the system automatically checks the difference between the current time and the timestamp of the time series data. If the difference exceeds the TTL value, the system automatically deletes the expired data.
Read and write time series data
You can use the Tablestore console, SDKs, or CLI to write multiple rows of time series data to a time series table at the same time. After data is written to the time series table, you can specify time series identifiers to query the data in the time series within the specified time range.
Retrieve time series
You can use the Tablestore console, SDKs, or CLI to retrieve time series in a time series table. You can use a composite condition that consists of multiple conditions to retrieve time series. For example, you can retrieve all time series in which the metric name is cpu, the tags contain a tag whose name is region and value is hangzhou, and the properties contain a property whose name is status and value is online. After the time series are retrieved, you can call an API operation to further query the data in the time series.
Implement SQL query and analytics
Time series tables support queries by using SQL. In SQL, you can specify the metadata condition to filter time series and aggregate data based on the aggregation operations in different dimensions. For example, you can query the average value of the sample data that is collected from a batch of devices and aggregate second-level data into minute-level data.
In addition, you can query only the metadata of time series in SQL. This way, you can manage the metadata of time series by using SQL.
Limits
For more information, see Limits on the TimeSeries model.
Usage notes
The TimeSeries model is supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Hong Kong), Germany (Frankfurt), US (Virginia), India (Mumbai), SAU (Riyadh), and Singapore.
API operations
Category | Operation | Description |
API operations on time series tables | Creates a time series table. | |
Queries the names of time series tables in an instance. | ||
Queries information about a time series table. | ||
Updates the configurations of a time series table. | ||
Deletes a time series table. | ||
API operations on time series | Writes time series data to a time series table. | |
Queries the data in a time series. | ||
Retrieves time series metadata. | ||
Updates time series metadata. | ||
Deletes time series metadata. |
Procedure
Step | Operation | Description |
1 | After you create a RAM user, configure minimal permissions for the RAM user to access Tablestore resources. You can use system default policies or custom policies to grant the RAM user the permissions to access Tablestore resources. If you want to use an Alibaba Cloud account or a RAM user that has the required permissions to access Tablestore resources, skip this step. Important By default, an Alibaba Cloud account has permissions on all cloud resources. To ensure the security of your resources, we recommend that you create RAM users for your Alibaba Cloud account and authorize them to access different resources. | |
2 | Before you use the features of Tablestore, you must activate Tablestore. You need to activate Tablestore only once. You are not charged when you activate Tablestore. If Tablestore is activated, skip this step. | |
3 | Important Before you create an instance for the TimeSeries model, you must determine the type of the instance based on your business characteristics and the requirements of your business on the read and write performance and the costs. For more information, see Instance. Create an instance for the TimeSeries model based on the instance type that you determine in the required region. If an existing instance meets your business requirements, skip this step. | |
4 | Create a time series table to store time series data. When you create a time series table, you can configure the TTL of the time series table based on whether you want to clear historical data. | |
5 | Write time series data to the time series in a batch. Time series data consists of metadata and data. If you do not create metadata before you write the time series data, the system automatically extracts the metadata from the written data. Note You can also use the kafka-connect-tablestore package to synchronize data from Apache Kafka to the time series table. For more information, see Data synchronization to time series tables. | |
6 | Important If you are unsure about information of the time series whose data you want to query, such as the metric name and data source, you can specify multiple conditions to retrieve a time series. For more information, see Manage time series metadata. After the time series whose data you want to query is retrieved, you can query the time series data that meets the specified conditions such as a time range in the time series. | |
7 | You can create three types of mapping tables for a time series table in SQL to query data.
|
Use the TimeSeries model
You can have a quick try of the TimeSeries model by using the Tablestore console, SDKs, or CLI.
Billing
The billable items of the TimeSeries model include the read throughput, write throughput, and storage usage of time series data and time series metadata, and the outbound traffic over the Internet. For more information, see Billing overview.
FAQ
References
To delete historical time series data in a time series table, you can configure the TTL for the time series table. For more information, see TTL of time series data.
To back up time series data in Tablestore in a cost-effective manner or export time series data as files to local devices, you can use the Data Integration feature of DataWorks to synchronize time series data from Tablestore to Object Storage Service (OSS) for storage or download. For more information, see Synchronize data from Tablestore to OSS.
To visualize time series data, you can connect Tablestore to Grafana. For more information, see Connect Tablestore to Grafana.
If you use Realtime Compute for Apache Flink to compute and analyze data, you can use Tablestore time series tables to store the results. For more information, see Tutorial (TimeSeries model).