This topic describes how to use the console to configure Logtail and collect Kubernetes container logs in the Sidecar mode.

Overview

In the Sidecar mode, the Logtail container shares the log directory with application containers in a pod. Application containers write logs to the shared directory and the Logtail container monitors the changes of log files in the shared directory and collects logs. For more information, see Sidecar container with a logging agent and How Pods manage multiple Containers.

Step 1: Install Sidecar

The configuration template for log collection in the Sidecar mode is as follows:
apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-log-sidecar-demo
  namespace: default
spec:
  template:
    metadata:
      name: nginx-log-sidecar-demo
    spec:
      restartPolicy: Never
      containers:
      - name: nginx-log-demo
        image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
        command: ["/bin/mock_log"]
        args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### logtail sidecar container
      - name: logtail
        # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
        # This image is released for every region
        image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
        # when receive sigterm, logtail will delay 10 seconds and then stop
        command:
        - sh
        - -c
        - /usr/local/ilogtail/run_logtail.sh 10
        livenessProbe:
          exec:
            command:
            - /etc/init.d/ilogtaild
            - status
          initialDelaySeconds: 30
          periodSeconds: 30
        resources:
          limits:
            memory: 512Mi
          requests:
            cpu: 10m
            memory: 30Mi
        env:
          ##### base config
          # user id
          - name: "ALIYUN_LOGTAIL_USER_ID"
            value: "${your_aliyun_user_id}"
          # user defined id
          - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
            value: "${your_machine_group_user_defined_id}"
          # config file path in logtail's container
          - name: "ALIYUN_LOGTAIL_CONFIG"
            value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
          ##### env tags config
          - name: "ALIYUN_LOG_ENV_TAGS"
            value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
          - name: "_pod_name_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: "_pod_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: "_namespace_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: "_node_name_"
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: "_node_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### share this volume
      volumes:
      - name: nginx-log
        emptyDir: {}
  1. Log on to your Kubernetes cluster. For more information, see Access Kubernetes clusters by using SSH key pairs.
  2. Configure the following basic parameters for log collection.
    ##### base config
              # user id
              - name: "ALIYUN_LOGTAIL_USER_ID"
                value: "${your_aliyun_user_id}"
              # user defined id
              - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                value: "${your_machine_group_user_defined_id}"
              # config file path in logtail's container
              - name: "ALIYUN_LOGTAIL_CONFIG"
                value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
    Parameter Description
    ${your_region_config} The ID of the region and the type of the network where your project resides. For more information, see Table 1.
    • If your project resides in the public network, the parameter value is formatted in region-internet, for example, cn-hangzhou-Internet.
    • If your project resides in an internal network, the parameter value is formatted in region, for example, cn-hangzhou.
    ${your_aliyun_user_id} The unique ID of your Alibaba Cloud account. For more information, see Configure an account ID for a server.
    ${your_machine_group_user_defined_id} The custom identifier of your server group. The identifier must be unique in the region where your project resides. For more information, see Create a custom ID-based machine group.
  3. Configure the mount path of logs.
    volumeMounts:
    - name: nginx-log
    mountPath: /var/log/nginx
    • The Logtail container and application containers in a pod must share the same directory.
    • We recommend that you mount an emptyDir volume on the containers.
  4. Configure the latency for stopping log collection
    Generally, the latency configured for stopping log collection is 10 seconds. This value indicates that the Logtail container stops collecting logs 10 seconds after it is ordered to stop collection. This configuration prevents incomplete data collection.
    command:        
    - sh        
    - -c        
    - /usr/local/ilogtail/run_logtail.sh 10

Step 2: Configure log collection

  1. Log on to the Log Service console.
  2. On the page that appears, click RegEx - Text Log in the Import Data section.
    A regex-text file is taken as an example. For more information about how to collect other text files, see Overview.
  3. Select an existing project and Logstore.
    You can also click Create Now to create a project and then a Logstore. For more information, see Quick start.
  4. Create a server group.
    If a server group is available, click Using Existing Server Groups.
    1. Make sure that a server group is created. Then, click Complete Installation.
    2. In the Create Server Group step, configure related parameters, and then click Next.

      Select Custom ID for the Identifier field. In the Custom Identifier field, enter ALIYUN_LOGTAIL_USER_DEFINED_ID that you configured in Step 1: Install Sidecar.

      Select a server group
  5. Select the server group.
    Select a server group by moving the group from Source Server Groups to Applied Server Groups.Select server groups
  6. Create a Logtail configuration file.
    The available modes for Logtail log collection include Simple Mode, NGINX Configuration Mode, Delimiter Mode, JSON Mode, Full Regex Mode, and other modes. For more information, see Overview.
    Note You must turn off the Docker File switch.
    Specify a collection mode
  7. Configure search and analytics statements. Click Next.
    Indexes are created by default. You can modify the indexes based on your needs.