Introduction to the query and analysis capabilities of a Storeview for logstore - Simple Log Service

Storeviews allow you to perform JOIN query and analysis on logs across multiple Logstores or Metricstores. This topic describes how to create a Storeview for Logstore and query and analyze logs in the Storeview for Logstore.

Create a Storeview for Logstore

Prerequisites

A project and a Standard Logstore are created, and logs are collected. For more information, see Create a project, Create a logstore, and Data collection overview.

Procedure

Log on to the Simple Log Service console. In the Projects section, click the project that you want to manage.
In the left-side navigation pane, choose Storeview > Storeview for Logstore. On the page that appears, click Create Now or the icon to create a Storeview for Logstore.
In the Create Storeview dialog box, configure the Storeview Name parameter, add the Logstores that you want to associate with the Storeview, and then click OK. Approximately 1 minute is required to create a Storeview.
Note
- You can create up to 10 Storeviews for Logstore for each Simple Log Service project.
- You can associate up to 50 Logstores with each Storeview for Logstore.
- All Logstores in a Storeview for Logstore must belong to the same Alibaba Cloud account.
Warning
If cross-border JOIN query is required, you must read and agree to the Compliance Warranty on Cross-border Data Transfer. Then, click OK.

Log query

A Storeview for Logstore supports cross-Logstore query. For example, two Logstores, such as logstore-1 and logstore-2, are associated with a Storeview for Logstore and contain Field a. If you specify the * | select a from log where a > 1 statement, the values from Field a that are greater than 1 in logstore-1 and logstore-2 are returned.

If you use a keyword to query logs in a Storeview for Logstore that contains two Logstores and the Logstores have the same log content, results from different Logstores are returned. You can use the tag field to identify the source Logstore of the log.

Log analysis

A Storeview for Logstore supports data aggregation and statistical analysis across Logstores. For example, two Logstores, such as logstore-1 and logstore-2, are associated with a Storeview for Logstore and contain Field a. If you specify the * | select a, count(1) group by a statement, the total number of values from Field a in logstore-1 and logstore-2 are returned.

In aggregate analysis, field keys must be consistent. If the log content is the same but the key names are different, you can use the predefinition feature to filter specific data and rename the fields. For example, if you want to aggregate Field a of logstore-1 and Field b of logstore-2, you must rename Field b to a.

Note

When you create a Storeview for Logstore, you can predefine the associated Logstores in the Storeview for Logstore based on the Simple Log Service Processing Language (SPL) syntax. SPL syntax supports the extend and where keywords when you use the predefinition feature. The predefinition feature can filter specific data and resolve the issue of inconsistent field names in aggregation analysis. In aggregation analysis, field keys must be consistent. If you want to aggregate Field a of logstore-1 and Field b of logstore-2, you must specify extend a = b in your SPL statement to ensure that the column names are consistent.

The following scenarios are used as examples.

Scenario 1
A Storeview for Logstore is associated with different Logstores from the same service. The Logstores have the same content. In this case, you must check whether the index configurations of the Logstores are the same. For example, the Logstores use full-text index or the index configurations of the required fields are the same. Then, you must add the Logstores to the Storeview for Logstore for association.
After you create a Storeview for Logstore, you can use SQL statements to analyze the Storeview for Logstore. The following figure shows the aggregation results of all the logs of the two Logstores based on the values of the status parameter in the previous 15 minutes.
Scenario 2
A Storeview for Logstore is associated with different Logstores from the same service, and the names of the log fields in the Logstores are different. In this case, you can use the predefinition feature and specify a SPL statement to use the same name for the fields. In the following figure, the *|extend request_method = method statement is specified to rename the method field to request_method. Click Preview Data to preview the predefined result data.
You can use the request_method field in subsequent query and analysis.

Examples of common query and analysis results

Scenario description	Pre-defined query during Storeview creation	Input query on the Storeview query and analysis page	Output
logstore-1 and logstore-2 contain the key column.	None	`key:123`	Rows that contain `key:123` are returned.
logstore-1 and logstore-2 contain columns a and b.	None	`*\| select a where b=1`	All the values from the `a` column that meet the condition `b =1` are returned.
logstore-1 and logstore-2 contain columns key and a.	`key:123`	`*\| select a`	All the values from the `a` column that meet the condition `key:123` are returned.
logstore-1 and logstore-2 contain columns key and a.	`key:123\|where a>1`	`*\| select a`	All the values from the `a` column that meet the conditions `key:123` and `a > 1` are returned.
logstore-1 and logstore-2 contain columns key and a.	`key:123\|where a>1 \|extend y=upper(a)`	`\| select `	All columns, including the y column, that meet the condition are returned.
logstore-1 contains columns a, b, and c, and logstore-2 contains columns b, c, and d.	None	`select a,b,c,d`	Columns a, b, c, and d are returned. The empty values are set to null.
logstore-1 has index fields a and b, and logstore-2 has index fields b and c.	None	`a:* and b:* \| select a, count(1) group by a`	logstore-2 does not contain the a column. Only data of logstore-1 is calculated.

Handling of inconsistencies

If you specify different field types for the two Logstores in the same query statement, the query results can be returned.
If you specify fields for specific stores in an analytic statement, only the results from the stores that contain the field you specified are returned.
If the key index configurations across multiple stores are inconsistent, an error is returned and no results are returned.

What to do next

Query a Storeview for Llogstore

In the left-side navigation pane, choose Storeview > Storeview for Logstore to view the existing Storeviews for logstore .

Modify a Storeview for Logstore

In the Storeview list, move the pointer over the Storeview that you want to manage, click , and then click Modify.
In the Update Storeview panel, modify the associated Logstores and click OK.

Delete a Storeview for Logstore

In the Storeview list, move the pointer over the Storeview that you want to manage, click , and then click Delete.

References

Overview of Storeviews

You can call the following API operations to manage Storeviews: