Microservices Engine:Report MSE traffic protection metrics to Simple Log Service

MSE dashboard vs. SLS export

Use the built-in dashboard for real-time monitoring. Export to SLS when you need historical analysis, cross-resource correlation, or automated alerting.

How it works

MSE traffic protection is powered by Sentinel, which writes metric logs to local files on each pod. The Logtail agent collects these log files and ships them to an SLS Logstore. Once in SLS, you can query the data with SQL-like statements and build dashboards.

Pod (Sentinel metric logs) --> Logtail agent --> SLS Logstore --> Query & dashboards

Before you begin

• Activate Simple Log Service. See "Step 1: Activate Simple Log Service" in Getting Started.

• Create an ACK cluster. Use either an ACK managed cluster or an ACK dedicated cluster.

• Enable MSE traffic protection. Turn on traffic protection for MSE Microservices Governance and configure traffic protection rules. See Overview.

Metric log format

Each line in the Sentinel metric log contains pipe-delimited fields. Understanding this format helps when you configure the log parser and write queries.

Sample log line:

1724811932000|2024-08-28 02:25:32|com.alibabacloud.mse.demo.c.service.HelloServiceCTwo|24|0|24|0|1|0|1|202|OUT

Field reference:

For details about log file paths and formats, see Important logs.

Step 1: Install Logtail

Install the Logtail component in your ACK managed cluster or ACK dedicated cluster:

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find your cluster and click its name. In the left-side navigation pane, choose Operations > Add-ons.

3. On the Logs and Monitoring tab, find the logtail-ds card and click Install.

Step 2: Configure the SLS data source

Set up SLS to collect, parse, and index the Sentinel metric logs from your pods.

1. Log on to the Simple Log Service console.

2. Click the Quick Data Import card. In the Import Data dialog box, search for Kubernetes and click the Kubernetes - File card.

3. Select Logstore -- Configure the following parameters, then click Next.

   Parameter	Description
   Project	Select an existing project or create one. Use the naming convention k8s-log-{your ACK cluster ID}.
   Logstore	Select an existing Logstore or create one. For details, see Create a Logstore.

4. Machine Group Configurations -- Select the Kubernetes-group-${your ACK cluster ID} machine group from the Source Machine Group list, add it to the Applied Server Groups list, and click Next.

   Note

   If you selected a different project in the previous step, create a machine group as prompted.

5. Logtail Configuration -- Configure the data source and parsing format, then click Next.

   Parameter	How to configure
   Configuration Name	Enter a descriptive name, such as mse-sentinel-metrics.
   File Path	Enter the path in the format ${user_home}/logs/csp/{app_name}-metrics.log.{date}. For example, if the home directory is /root, set the directory to /root/logs/csp and the file name to *metrics.log*.
   Log Sample	Paste a sample log line: 1724811932000|2024-08-28 02:25:32|com.alibabacloud.mse.demo.c.service.HelloServiceCTwo|24|0|24|0|1|0|1|202|OUT
   Processing Method	Select Processors, then add a processor of type Data Parsing (Delimiter Mode). Use the pipe character | as the delimiter.

   

6. Query and Analysis Configurations -- Wait for the Preview Data section to refresh automatically. Review and adjust the index fields as needed, then click Next.

   

For more information about data source configuration, see Configure data source and parsing format.

Step 3: Verify log delivery

Confirm that metric data is flowing into SLS before you build dashboards.

1. Log on to the Simple Log Service console.

2. In the Projects section, click your project, then select your Logstore.

3. Run the following query to check for recent records: If records are returned, the pipeline is working. The results contain the parsed fields from the metric log format described earlier: timestamp, resource name, pass count, block count, average response time, and traffic direction.

   * | SELECT * LIMIT 10

4. (Optional) Verify that blocked-request data is present by running:

   Note

   The field names in these queries (resource_name, blocked) depend on how the index fields were named during Step 2. Adjust them to match your configuration.

   * | SELECT resource_name, blocked
          WHERE blocked > 0
          LIMIT 10