Log Service allows you to search and analyze large amounts of log data in real time. This topic describes the benefits of the search and analytics feature, the methods to use the feature, and the statement format that is used by the feature.
Benefits
- Real-time: You can analyze logs immediately after they are collected.
- Fast:
- Search: Billions of log entries can be processed and searched within one second (with five conditions).
- Analytics: Hundreds of millions of log entries can be aggregated and analyzed within one second (with aggregation based on five dimensions and with the GROUP BY condition).
- Flexible: Search and analytics statements can be changed based on your requirements and results are returned in real time.
- All-in-one: The search and analytics feature seamlessly integrates with Grafana, DataV, and Jaeger and supports RESTful APIs and JDBC. In addition, the feature also supports multiple sub-features in the console. For example, the search and analytics feature allows you to visualize search and analysis results in charts and dashboards and provides quick analysis based on predefined rules.
Index
You can use the index feature of Log Service to sort log entries based on one or more columns. This feature ensures quick location of log data. For more information, see Enable and configure the index feature for a Logstore.
- Full-text index: The full-text index stores the key and value of each field as text. This index allows you to search the keys and values for a keyword.
- Field index: If you add a field to the index, you can specify the key of the field and a keyword to search the field for the keyword. In Log Service, you can add the fields of the following data types to a field index: text, JSON, long, and double.
Methods
- Use the console
You can specify a time range and query statement on a search and analytics page in the Log Service console to query log data. For more information, see Query logs and Query syntax.
- Use the API
You can call the GetLogs or GetHistograms API operation to query log data.
Statement format
Search statement|Analytic statement
Statement | Required | Description |
---|---|---|
Search statement | No | A search statement contains one or more search conditions. A condition can be a keyword,
a pattern that contains wildcard characters, a value, or a value range.
If you leave the search statement unspecified or specify an asterisk (*) as the search statement, it indicates that no condition is specified and all log data is returned. For more information, see Query syntax. |
Analytic statement | No | An analytic statement is used to aggregate or analyze a search result.
If you leave the analytic statement unspecified, it indicates that the search result is returned without being analyzed. For more information, see Real-time analysis. |
Additional considerations
If you need to run a query statement to search a large number of log entries (for example, you need to search more than 10 billion log entries), Log Service may fail to return the entire result. Instead, Log Service returns a partial result and notifies you that the returned result is incomplete. At the same time, the Log Service server caches the partial query result for 15 minutes. If you run the query statement again and part of the query result hits the cache, Log Service continues to search for the matching log entries that are not cached. Log Service then merges and returns the query results to you. Therefore, you can repeatedly run the query statement to obtain the complete result.