Tablestore provides highly concurrent write performance and low storage cost and is suitable for storing IoT data, logs, and monitoring data. When you write data to a Tablestore data table, you can cleanse the data by using Function Compute and write the cleansed data to another data table in Tablestore. You can access raw data or cleansed data in Tablestore in real time.
Sample scenarios
You want to write log data that includes three fields to Tablestore. To efficiently query the logs, you must write the logs in which the value of the level field is greater than 1 to another data table named result. The following table describes the fields that are included in the logs.
Field | Type | Description |
---|---|---|
id | Integer | The ID of the log. |
level | Integer | The level of the log. A larger value indicates a higher level. |
message | String | The content of the log. |
Step 1: Enable the Stream feature for the data table
Before you create a trigger, you must enable the Stream feature for the data table in the Tablestore console to allow the function to process incremental data that is written to the table.
- Log on to the Tablestore console.
- On the Overview page, click the name of the instance that you want to manage or click Manage Instance in the Actions column of the instance that you want to manage.
- In the Tables section of the Instance details tab, click the name of the required data table and click the Tunnels tab. Alternatively, you can click the
icon and then click Tunnels.
- On the Tunnels tab, click Enabled in the Stream Information section.
- In the Enable Stream dialog box, configure the Log Expiration Time parameter and click Enabled.
The value of the Log Expiration Time parameter must be a non-zero integer and cannot be changed after it is specified. Unit: hours. Maximum value: 168.
Note Specify a value for the Log Expiration Time parameter based on your business requirements.
Step 2: Configure a Tablestore trigger
You can create a Tablestore trigger in the Function Compute console to process the real-time data stream in a Tablestore data table.
- Create a Function Compute service.
- Create a Function Compute function. Note You can select Create from Scratch with a Standard Runtime, Use a Custom Runtime to Migrate Web Server, or Use Container Images (for Kubernetes and Docker Users) as the method to create a function. You can also use a template to create a function. The following procedure describes how to create a function by selecting Create from Scratch with a Standard Runtime. For information about other methods that can be used to create a function, see Create a function and Use function templates to create functions.
Step 3: Verify data cleansing
After you create a trigger, you can write data to Tablestore and query the data to verify whether the data is cleansed as expected.
- Write code.
- Write data to the data table named source_data. Enter the values of the id, level, and message fields and query the cleansed data in the table named result.
- When you write a log in which the value of the level field is greater than 1 to the source_data table, the log is synchronized to the result table.
- When you write a log in which the value of the level field is less than or equal to 1 to the source_data table, the log is not synchronized to the result table.