All Products
Search
Document Center

Alibaba Cloud Service Mesh:Use Log Service to collect access logs on the data plane

Last Updated:May 18, 2023

Container Service for Kubernetes (ACK) integrates with Log Service. You can use ACK to collect access logs of clusters on the data plane of an Alibaba Cloud Service Mesh (ASM) instance. This topic describes how to enable and configure log collection and view the collected logs.

Background information

After you add a Kubernetes cluster to an ASM instance, Envoy proxies that are deployed on the data plane of the ASM instance can print all access logs of the cluster. For more information, see Access logging. You can run the kubectl logs command to view the Envoy access logs. Log Service allows you to view logs in a convenient manner. In addition, you can collect and retrieve logs and create log dashboards.

Step 1: Install Logtail for the Kubernetes cluster

To create a Kubernetes cluster, perform the following steps:

  1. Log on to the ACK console.

  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, click Create Kubernetes Cluster in the upper-right corner. For more information, see Create a managed Kubernetes cluster.

  4. In the Component Configurations step, select Enable Log Service to install Logtail for the Kubernetes cluster to be created.

    • Click Select Project and select an existing project to manage collected logs.Enable Log Service 1

    • Click Create Project. By default, the system names the project in the format of k8s-log-{ClusterID}. ClusterID indicates the unique ID of the cluster to be created.Enable Log Service 2

  5. Set other parameters as required and click Create Cluster.

To install Logtail in a Kubernetes cluster, perform the following steps:

  1. Log on to the ACK console.

  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane, choose Operations > Add-ons.

  5. Find logtail-ds in the Logs and Monitoring section and click Install.

  6. In the message that appears, click OK.

To upgrade Logtail whose version is earlier than V0.16.24.0-1fa7551-aliyun for a Kubernetes cluster, perform the following steps:

  1. Log on to the ACK console.

  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane, choose Operations > Add-ons.

  5. Find logtail-ds in the Logs and Monitoring section and click Upgrade.

  6. In the message that appears, click OK.

Step 2: Configure log collection

To use Logtail to collect Envoy access logs, you must configure the log collection feature in the configuration file of the Kubernetes cluster.

If the Istio version of your ASM instance is V1.7.5.26-gd318a562-aliyun or later, perform the following steps:

  1. Log on to the ASM console.

  2. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.

  4. On the details page of the ASM instance, choose ASM Instance > Base Information in the left-side navigation pane. On the page that appears, click Settings in the upper-right corner.

  5. In the Settings Update panel, select Collect access logs to Alibaba Cloud Log Service and click OK.

If the Istio version of your ASM instance is earlier than V1.7.5.26-gd318a562-aliyun, perform the following steps:

  1. Create a YAML file by using the following template:

    apiVersion: log.alibabacloud.com/v1alpha1
    kind: AliyunLogConfig
    metadata:
      # your config name, must be unique in you k8s cluster
      name: mesh-access-log-config
      namespace: kube-system
    spec:
      # must use same project with k8s cluster
      project: k8s-log-${K8SClusterID}
      # logstore name to upload log
      logstore: mesh-access-log
      # product code always been mesh-access-log
      productCode: mesh-access-log
      # logtail config detail
      logtailConfig:
        # docker stdout's input type is 'plugin'
        inputType: plugin
        # logtail config name, should be same with [metadata.name]
        configName: mesh-access-log-config
        inputDetail:
          plugin:
            inputs:
              - type: service_docker_stdout
                detail:
                  # collect stdout and stderr
                  Stdout: true
                  Stderr: true
                  IncludeEnv:
                    ISTIO_META_POD_NAME: ""
                  IncludeLabel:
                    io.kubernetes.container.name: "istio-proxy"
            processors:
              - type: processor_json
                detail:
                  # By default, the key of the data that is collected from Dockers is content.
                  SourceKey: content
                  ExpandConnector: ""
                  KeepSource: true
                  NoKeyError: true
    Note
    • Replace ${K8SClusterID} in the YAML template with the ID of your Kubernetes cluster.

    • You must prepare a configuration file for each Kubernetes cluster.

    • Set the project field in the YAML file in the format of k8s-log-${K8SClusterID}. Do not modify the value of the logstore field. Otherwise, the control panel of the ASM instance may not be normally displayed.

  2. After you edit the YAML file, deploy the file for the Kubernetes cluster.

    kubectl apply -f [The path of the YAML file]

Step 3: View logs

After you configure the log collection feature, Logtail uses the specified project to collect Envoy access logs and stores the logs in the specified Logstore. To view the logs, perform the following steps:

  1. Log on to the ASM console.

  2. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  3. On the Mesh Management page, find the ASM instance that you want to view. Click the name of the ASM instance or click Manage in the Actions column.

  4. On the details page of the ASM instance, choose Cluster & Workload Management > Kubernetes Clusters in the left-side navigation pane.

  5. On the Kubernetes Clusters page, click Monitoring Center or Detailed Logs in the Access Logs column.

    • If you click Monitoring Center, you can view multi-dimensional statistics on access logs.

    • If you click Detailed Logs, you can view the original access logs in a sorted manner.