All Products
Search
Document Center

Alibaba Cloud Service Mesh:Enable control-plane log collection and log-based alerting in an ASM instance of version 1.17.2.35 or later

Last Updated:Mar 04, 2024

Service Mesh (ASM) provides Log Center that allows you to view control-plane logs and data-plane logs in the ASM console. After you enable log collection, you can use Log Center to view control-plane logs, configure log-based alert rules, and view data-plane logs and related dashboards. This topic describes how to enable control-plane log collection and log-based alerting.

Prerequisites

Enable control-plane log collection

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab. Then, select New project or Use existed project, configure the parameters, and then click Enable Control-Plane Log Collection. In the Submit message, click OK.

    After logs are collected to Simple Log Service, you can perform the following operations:

Note

If you no longer need to use this feature, click Disable Control-Plane Log Collection in the upper-right corner of the Control-Plane Logs tab. In the Submit message, click OK.

View control-plane logs

After you enable control-plane log collection, logs of control-plane components of the ASM instance are collected to Simple Log Service. You can view or retrieve control-plane logs in the Simple Log Service console or on the log browsing interface that is integrated into the ASM console.

Note

The system takes some time to establish a log distribution task for Simple Log Service. Therefore, you may not be able to see logs in the consoles for a short period of time after you enable log collection.

Method 1: View logs in the ASM console

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab to view log information.

Method 2: View logs in the Simple Log Service console

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the name of the Simple Log Service project that is used for the cluster.

  3. On the Logstores page, click the desired Logstore to view the log information on the right of the page.

Configure log-based alerting for the control plane

After you enable control-plane log collection, you can configure log-based alerting for the control plane.

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab. In the upper-left corner, click Alert Setting.

  4. In the Control-Plane Alert Setting dialog box, find the desired alert policy, select an action policy based on your business requirements, and click Enable Alert. In the Note message, click OK.

    An action policy describes how alert notifications are sent when an alert is triggered. ASM Built-in Action Strategy and Custom action strategy are supported. By default, no notification channels are configured for a built-in action policy of ASM. You can click the 设置.png icon to go to the alert center of Simple Log Service to set a notification channel. If an alert is triggered, handle the alert by referring to Handle alerts.

Configure alert contacts

You can configure built-in action policies for gateways, alert contacts, and notification templates in Simple Log Service.

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the name of the desired project. In the left-side navigation pane, click Alerts.

  3. On the Alert Center page, choose Notification Objects > User Group Management.

  4. On the User Group Management tab, find sls.app.asm.builtin and click Edit in the Actions column.

  5. In the Edit User Group dialog box, select the members that you want to add, click the 添加 icon to add the members to the user group, and then click OK. 修改用户组

Verify the alert notification settings

Note

This topic does not verify the alert notification settings for all alerts. In the following example, incorrect configurations are set to trigger an alert. For more information about error messages, see Alerts caused by failures of configuration pushes from the control plane due to errors in the configurations.

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Gateways > Gateway. On the page that appears, click Create from YAML.

  3. On the Create page, select a namespace and a template, configure a YAML file, and then click Create.

    In this example, the default namespace is used to configure an Istio gateway. Sample YAML code:

    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name:  gateway-test
      namespace: default
    spec:
      selector:
        istio: ingressgateway
      servers:
        - hosts:
            - '*console.aliyun.com'
          port:
            name: https
            number: 443
            protocol: HTTPS
          tls:
            credentialName: not-existing-credential
            mode: SIMPLE
  4. View alert notifications.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

    3. On the Log Center page, click Control-Plane Logs and search for ACK ERROR to view the alert information.

      If you have configured email notifications for alerts, you can view alert information by checking your emails.

Handle alerts

Alerts caused by potential configuration risks

If ASM detects configurations in your cluster that may cause the execution results of ASM to fail to meet expectations, alerts are reported. You can view the alerts on the Mesh Diagnosis page and follow the instructions on the Mesh Diagnosis page to correct the configurations. For more information, see Diagnose ASM instances.

Alerts caused by incorrect configurations

If ASM detects incorrect configurations in your cluster that are likely to cause unexpected behaviors, alerts are reported. We recommend that you view such alerts on the Mesh Diagnosis page and follow the instructions on the Mesh Diagnosis page to correct the configurations as soon as possible. For more information, see Diagnose ASM instances.

Alerts caused by failures of configuration pushes from the control plane due to errors in the configurations

The following table describes common error messages that may appear when synchronization to the data plane fails and provides suggestions for troubleshooting the errors. If your error message is not included in the table, submit a ticket.

Error message

Solution

Internal:Error adding/updating listener(s) 0.0.0.0_443: Failed to load certificate chain from <inline>, only P-256 ECDSA certificates are supported

Indicates that clusters on the data plane do not support the certificate that you configured for the data plane. Configure the P-256 ECDSA certificate for the data plane. For more information about how to reconfigure a certificate, see Use an ingress gateway to enable HTTPS.

Internal:Error adding/updating listener(s) 0.0.0.0_443: Invalid path: ****

Indicates that the path of the certificate that you configured for the data plane is invalid or that the specified certificate does not exist. Check whether the mount path of the certificate is the same as the path specified in the configurations of the gateway. For more information about how to reconfigure a certificate, see Use an ingress gateway to enable HTTPS.

Internal:Error adding/updating listener(s) 0.0.0.0_xx: duplicate listener 0.0.0.0_xx found

Indicates that duplicate listening ports are configured for your gateway. To resolve this issue, check your gateway and delete any duplicate ports.

Internal:Error adding/updating listener(s) 192.168.33.189_15021: Didn't find a registered implementation for name: '***'

Indicates that *** referenced based on the 15021 listener patch by using EnvoyFilter cannot be found in sidecar proxies or ingress gateway services. You must delete the reference.

Internal:Error adding/updating listener(s) 0.0.0.0_80: V2 (and AUTO) xDS transport protocol versions are deprecated in grpc_service ***

Indicates that the xDS v2 protocol on the data plane will be deprecated soon. This is usually because the version of sidecar proxies on the data plane does not match that on the control plane. To resolve this issue, update sidecar proxies on the data plane. To do so, you must delete existing pods. Sidecar proxies of the latest version are automatically injected into the recreated pods.

Alert policy: Certificate Management has an expired certificate, which may cause the gateway to fail to process TLS traffic.

This alert indicates that a certificate in Certificate Management has expired and needs to be handled immediately.

Alert policy: Certificate Management has a certificate that is about to expire. Please update the certificate in time.

This alert indicates that a certificate in Certificate Management will expire soon and needs to be handled as soon as possible.

Related operations

Modify the settings of a control-plane log project

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Control-Plane Logs tab of the Log Center page, click Change Log Project. In the Change Log Project dialog box, modify the settings as needed and click Submit.