Log Service provides the log analysis feature. This feature works together with the log search feature and is implemented by using SQL syntax. This topic describes the syntax and limits of the analytic statements. This topic also provides SQL functions that you can call when you use the log analysis feature.
- If you want to analyze logs, you must store the logs in a Standard Logstore and turn on Enable Statistics for the corresponding fields when you configure indexes. For more information, see Logstore type comparison and Configure indexes.
- Log Service provides reserved fields. For more information about how to analyze reserved fields, see Reserved fields.
Syntax
- You do not need to specify the FROM or WHERE clause in an analytic statement. By default, all data of the current Logstore is analyzed.
- You do not need to add a semicolon (;) at the end of an analytic statement to end the statement.
- Analytic statements are case-insensitive.
- Syntax
Search statement|Analytic statement
Statement Description Search statement A search statement specifies one or more search conditions. A condition can be a keyword, a numeric value, a numeric value range, a space character, or an asterisk (*). If you specify a space character or an asterisk (*) as the search statement, no conditions are specified and all logs are returned. For more information, see Search syntax.
Analytic statement An analytic statement is used to aggregate or analyze search results or all data in a Logstore. - Example
* | SELECT status, count(*) AS PV GROUP BY status
Limits
Item | Standard SQL | Dedicated SQL |
---|---|---|
Number of concurrent analytic statements | Each project supports a maximum of 15 concurrent analytic statements.
For example, 15 users can concurrently execute analytic statements in all Logstores of a project. |
Each project supports a maximum of 100 concurrent analytic statements.
For example, 100 users can concurrently execute analytic statements in all Logstores of a project. |
Data volume | Each shard supports only 1 GB of data for a single analytic statement. | An analytic statement can scan a maximum of 200 billion rows of data at the same time. |
Method to enable | By default, Standard SQL is enabled. | A switch is provided for you to manually enable Dedicated SQL. For more information, see Enable Dedicated SQL. |
Resource usage fee | Free of charge. | You are charged based on the actual CPU time. |
Applicable scope | You can analyze only the data that is written to Log Service after the log analysis
feature is enabled.
If you want to analyze historical data, you must reindex the historical data. For more information, see Reindex logs for a Logstore. |
You can analyze only the data that is written to Log Service after the log analysis
feature is enabled.
If you want to analyze historical data, you must reindex the historical data. For more information, see Reindex logs for a Logstore. |
Returned result | By default, an analytic statement returns a maximum of 100 rows of data.
If you want to view more data, use a LIMIT clause. For more information, see LIMIT clause. |
By default, an analytic statement returns a maximum of 100 rows of data.
If you want to view more data, use a LIMIT clause. For more information, see LIMIT clause. |
Size of a field value | The maximum size of a field value is 16 KB. If the size of a field value exceeds 16 KB, the excess content is not analyzed. | The maximum size of a field value is 16 KB. If the size of a field value exceeds 16 KB, the excess content is not analyzed. |
Timeout period | The maximum timeout period for a single analytic statement is 55 seconds. | The maximum timeout period for a single analytic statement is 55 seconds. |
Number of decimal places in the value of a double-type field | The value of a double-type field can contain a maximum of 52 decimal places.
If the number of decimal places is greater than 52, the accuracy of the field value is compromised. |
The value of a double-type field can contain a maximum of 52 decimal places.
If the number of decimal places is greater than 52, the accuracy of the field value is compromised. |
Analytic functions and syntax
This section lists the analytic functions and syntax that are supported by Log Service.
- SQL functions
- Aggregate function
- Security check functions
- Map functions and operators
- Approximate functions
- Mathematical statistics functions
- Mathematical calculation functions
- String functions
- Date and time functions
- URL functions
- Regular expression functions
- JSON functions
- Data type conversion functions
- IP functions
- Array functions and operators
- Binary functions
- Bitwise functions
- Interval-valued comparison functions and periodicity-valued comparison functions
- Comparison operators
- Lambda expressions
- Logical operators
- Geospatial functions
- Geo functions
- Machine learning syntax and functions
- Window functions
- Machine learning functions
- Smooth functions
- Multi-period estimation functions
- Change point detection functions
- Maximum value detection functions
- Prediction and anomaly detection functions
- Sequence decomposition function
- Time series clustering functions
- Frequent pattern statistical function
- Differential pattern statistical function
- Request URL classification function
- Root cause analysis function
- Correlation analysis functions
- Kernal density estimation functions
- Time series padding function
- Anomaly comparison function
- SQL syntax
Sample analysis results
The following figure shows a sample dashboard that displays the analysis results.
