How do I query and analyze logs? - Tablestore - Alibaba Cloud Documentation Center

When you perform operations on the resources in a Tablestore instance, the system automatically collects audit logs and creates indexes. On the Audit Log page, you can query and analyze the collected logs in real time.

Prerequisites

The audit log feature is enabled. For more information, see Enable the audit log feature.

Procedure

Go to the Audit Log page.
1. Log on to the Tablestore console.
2. In the top navigation bar, select a region.
3. In the left-side navigation pane, click Audit Log.

On the Audit Log page, enter a query statement in the query statement field.

A query statement consists of a search statement and an analytic statement and is in the Search statement|Analytic statement format. Example: * | SELECT TableName, count(*) AS OperationCount GROUP BY TableName. The search statement and analytic statement are separated by a vertical bar (|). The search statement uses the exclusive syntax of Simple Log Service.

A search statement can be executed alone. However, an analytic statement must be executed together with a search statement. You can use the log analysis feature to analyze the data in search results or all data in a Logstore.

Statement

Description

Search statement

A search statement specifies one or more search conditions. The logs that meet the specified conditions are returned. For more information, see Search syntax.

A search statement can be a keyword, a numeric value, a numeric value range, a space, or an asterisk (*). A space or an asterisk (*) indicates that no filter condition is specified.

Analytic statement

An analytic statement is used to aggregate or analyze the data in search results or all data in a Logstore. For more information, see Aggregate function.

Examples:

Example 1: Query the top 5 operations that are most frequently called.
Important
If you use the audit log feature in the Tablestore console, the system automatically generates logs for operations that check whether secondary indexes are created. Therefore, you need to filter out the check logs by specifying the TableName != '$$' condition when you query operation logs.
```
* | SELECT API,COUNT(API) as count where TableName != '$$' group by API order by count desc limit 5
```
Example 2: Query the user who deleted a table and the table that the user deleted.
```
* | SELECT TableName,InvokerUid,time where API = 'DeleteTable' 
```
Example 3: Query all users who performed operations on the exampletable table in the exampleinstance instance and the operations performed by the users.
```
* | SELECT API,InvokerUid,from_unixtime(time/1000000,'Asia/shanghai') as time where InstanceName = 'exampleinstance' and TableName = 'exampletable'
```

Click Last 15 Minutes in the upper-right corner to specify a time range within which you want to query logs.
You can also specify a closed interval in an analytic statement by using the __time__ field. Example: * | SELECT * FROM log WHERE __time__>1558013658 AND __time__< 1558013660.
The time range that you specify by using one of the preceding methods is accurate to the minute. The query and analysis results may contain logs that are generated 1 minute earlier or later than the specified time range. If you want to specify a time range that is accurate to the second, you must use the from_unixtime or to_unixtime function in your analytic statement to convert the time format.
The system displays the query results. The following figure shows an example.
Note
The returned query results vary based on the search statement. The displayed query results prevail.
You can view the information about the audit logs, such as the query result histogram, raw logs, charts, and log clustering information.
- View the query result histogram
  The query result histogram displays the distribution of queried logs based on points in time. Click the green rectangle to view a more fine-grained log distribution. You can also view the query results within the specified time range on the Raw Logs tab.
- Query raw logs
  On the Raw Logs tab, you can view query results. Click Table or Raw Data to view the log data. For more information, see Query raw logs.
- View charts
  After you execute a query statement, you can view the charts that display the query and analysis results on the Graph tab. For more information, see View charts.
- Use the LogReduce feature
  On the LogReduce tab, click Enable LogReduce to cluster log data with high similarity during log collection. For more information, see Use the LogReduce feature.

What to do next

Configure default query settings.
By default, the system enables the query feature when you go to the Audit Log page for the first time and queries logs that are generated within 15 minutes. You can modify the default query settings based on your business requirements.
1. On the Audit Log page, click the icon.
2. In the Query dialog box, turn off or turn on Enable queries the first time access a page and configure the Custom Query Time Range parameter.
3. Click OK.
Scan logs.
You can use the scan feature to scan log fields without the need to configure indexes to query and analyze related logs. For more information, see Scan logs.
1. On the Audit Log page, click the icon to enable the scan feature.
2. In the query statement field, enter a scan statement.
  Enter a scan statement in one of the following formats: Search statement | WHERE bool_expression and Search statement|Analytic statement (standard SQL syntax).
3. Click Last 15 Minutes in the upper-right corner to specify the time range within which you want to scan logs.
Create an alert monitoring rule for logs.
You can create an alert monitoring rule to monitor the query and analysis results of logs. For more information, see Create an alert monitoring rule for logs.