In business scenarios, data may be stored in different Logstores or Metricstores of a Simple Log Service project or in projects in different regions. You can use Storeviews to perform associated query for data across Logstores or Metricstores and across regions. This helps improve work efficiency. This topic describes how to use Storeviews for Logstore and Storeviews for Metricstore.
Introduction to Storeviews
Storeviews, Logstores, and Metricstores are important resources in Simple Log Service projects. Storeviews are not entity resources that are used to store data. Storeviews are virtual resources that are created based on Logstores and Metricstores and are used to manage and store the relationships among Logstores or Metricstores. You can use Storeviews to perform associated query for data in different Logstores or Metricstores. You cannot use Storeviews to modify data. Storeviews can be classified into Storeviews for Logstore and Storeviews for Metricstore. You can use Storeviews for Logstore to query and analyze data in different Logstores and predefine the associated Logstores based on the Simple Log Service Processing Language (SPL) syntax. You can use Storeviews for Metricstore only to query data in different Metricstores.
You can use Storeviews for Logstore or Storeviews for Metricstore to perform associated query for data in different Logstores or Metricstores. In this example, the age field is included in Logstore 1 and Logstore 2 and the two Logstores are associated with Storeview 1. If you query the value of the age field in Storeview 1, the values of the age field in Logstore 1 and Logstore 2 are returned.
Permissions
If you use Storeviews as a Resource Access Management (RAM) user or RAM role, the RAM user or RAM role must have the required permissions. For more information about how to use an Alibaba Cloud account to grant permissions to a RAM user or RAM role, see Create a RAM user and authorize the RAM user to access Simple Log Service, Create a RAM role whose trusted entity is an Alibaba Cloud service and grant the RAM role the permissions to access Simple Log Service, and Create a RAM role whose trusted entity is an Alibaba Cloud account and grant the RAM role the permissions to access Simple Log Service.
You can attach system policies or custom policies to a RAM user.
System policy: All system policies are created and updated by Alibaba Cloud. You can use system policies but cannot modify system policies.
Policy that grants RAM users the read-only permissions on Simple Log Service: AliyunLogReadOnlyAccess
Policy that grants RAM users the management permissions on Simple Log Service: AliyunLogFullAccess
Custom policy: You can create, modify, delete, and update custom policies based on your business requirements.
When you create a Storeview or query data in a Storeview as a RAM user or RAM role, the RAM user or RAM role must have the permissions to call the GetLogs operation in all associated Logstores or Metricstores in the Storeview. To grant the permissions, you must add the following policy document to your custom policy.
ImportantWhen you create a custom policy, you can specify a Logstore or a Metricstore after the Logstore keyword. You can also refer to the following policy to manage Metricstores.
The Storeview used in a custom policy can be a Storeview for Logstore or Storeview for Metricstore.
{ "Version": "1", "Statement": [ { "Action": "log:GetLogStoreLogs", "Resource":[ "acs:log:*:*:project/${projectName}/Logstore/*", "acs:log:*:*:project/${projectName}/storeview/*" ], "Effect": "Allow" } ] }
Limits
You can create up to 10 Storeviews for Logstore or Storeviews for Metricstore in a Simple Log Service project.
Up to 50 Logstores can be associated with a Storeview for Logstore. Up to 50 Metricstores can be associated with a Storeview for Metricstore.
All Logstores in a Storeview for Logstore or all Metricstores in a Storeview for Metricstore must belong to the same Alibaba Cloud account.
Predefinition
Concept
When you create a Storeview for Logstore, you can predefine the associated Logstores in the Storeview for Logstore based on the SPL syntax. You can use only the extend and where instructions. For more information, see extend and where. You can use the predefinition feature in scenarios such as data filtering and key mismatch in aggregation and analysis. In aggregation and analysis, the keys of the fields that you want to aggregate must be the same. For example, if you want to aggregate the a
field in Logstore 1 and the b
field in Logstore 2, you must specify extend a = b
in your SPL statement to use the same key.
Limits
Storeviews for Logstore support the predefinition feature. Storeviews for Metricstore do not support the predefinition feature.
Supported instructions
You can use the extend and where instructions when you use the predefinition feature in Storeviews for Logstore. For more information, see extend and where.
For more information about the functions that are supported in SQL and SPL, see Function overview.
Examples
For more information about how to create or modify a Storeview for Logstore, see Create a Storeview for Logstore.
For example, if you want to aggregate the a
field in Logstore 1 and the b
field in Logstore 2, you must specify extend a = b
in your SPL statement when you create or modify a Storeview to use the same key.
Example 1
The * | where a>1
SPL statement is specified for Logstore 1 in your Storeview to query logs in which the values of the a
field are greater than 1. The logs that are returned are used for subsequent analysis.
Example 2
The * | extend b = a
SPL statement is specified for Logstore 1 in your Storeview to add the b
field whose values are the same as the values of the a
field to Logstore 1 for aggregation.
Query
Storeview for Logstore
You can use Storeviews for Logstore to query data across Logstores. In this example, two Logstores named logstore-1 and logstore-2 are associated with a Storeview for Logstore and the request_method
field is included in the two Logstores. If you execute the (request_method:GET or request_method:POST) and status in [200 299]
query statement, logs in the two Logstores that match the conditions specified in the query statement are returned. For more information about how to query data by using Storeviews for Logstore, see Query syntax and functions.
In this example, two Logstores that contain the same logs are associated with a Storeview for Logstore. If you query logs by keyword, logs in the two Logstores that contain the keyword are returned. You can identify the source of a log based on the tag field.
Storeview for Metricstore
You can use Storeviews for Metricstore to query data across Metricstores. In this example, two Metricstores named metricstore1 and metricstore2 are associated with a Storeview for Metricstore. If you specify a PromQL statement in the search box, the backend computing engine automatically computes data across Metricstores. For more information about how to query data by using Storeviews for Metricstore, see Query and analysis syntax on metric data. For more information about how to create or modify a Storeview for Metricstore, see Query data across Metricstores.
Storeviews for Metricstore support PromQL query. Storeviews for Metricstore do not support the SQL query, analysis, and predefinition features.
Analysis
Concept
You can use Storeviews to aggregate and analyze data across Logstores or Metricstores. In this example, two Logstores named logstore-1 and logstore-2 are associated with a Storeview for Logstore and the a
field is included in the two Logstores. If you execute the * | select a, count(1) group by a
query statement, logs that contain the a field are returned.
In aggregation and analysis, the keys of the fields that you want to aggregate must be the same. If the values of the fields are the same but the keys are different, you can filter data and rename the keys. For more information, see Predefinition.
Limits
Storeviews for Logstore support the analysis feature. Storeviews for Metricstore do not support the analysis feature.
Supported functions
For more information about the functions that are supported in SQL and SPL, see Function overview.
Examples
Example 1
In this example, different Logstores of the same service are associated with a Storeview for Logstore and the Logstores contain the same logs. In this case, check whether the index settings of the Logstores are the same. If full-text indexes are configured for the Logstores or the same settings are configured for the specified field indexes, add the Logstores to the Storeview for Logstore.
After you create the Storeview for Logstore, you can specify SQL statements to analyze data. In the following figure, the results of the query statement are the values of the status
field in the two Logstores within the previous 15 minutes.
Example 2
In this example, different Logstores of the same service are associated with a Storeview for Logstore and the Logstores contain log fields with different names. In this case, you can use the predefinition feature and specify an SPL statement to use the same name for the fields. In the following figure, the *|extend request_method = method
SPL statement is executed to rename the method
field request_method
. You can click Preview Data to preview the results of the predefinition operation.
You can use the
request_method
field name for query and analysis.
Differences between Storeviews and stores
Feature | Logstore or Metricstore | Storeview |
Quick analysis | Supported | Supported |
Chart | Supported | Supported |
Dashboard | Supported | Supported |
Log download | Supported | Supported |
Saved search | Supported | Not supported |
Scheduled SQL | Supported | Not supported |
Alert | Supported | Not supported |
Log clustering | Supported | Not supported |
Field analysis | Supported | Not supported |