This topic describes how to install Sidecar. This topic also describes how to use the Log Service console to create a Logtail configuration that is used to collect container text logs in Sidecar mode.

Prerequisites

The Logtail component is installed. For more information, see Install the Logtail component.

Background information

In Sidecar mode, the Logtail container shares a log directory with an application container. The application container writes logs to the shared directory. Logtail monitors changes to log files in the shared directory and collects logs. For more information, see Sidecar container with a logging agent and How Pods manage multiple containers.

Step 1: Install Sidecar

  1. Log on to your Kubernetes cluster.
  2. Create a YAML file.

    In this example, sidecar.yaml is used as the file name. You can specify a file name based on your business requirements.

    vim sidecar.yaml
  3. Enter the following script in the YAML file and configure the parameters based on your business requirements.
    Notice Make sure that the time zone below env in the configuration file is correctly set. If the time zones are inconsistent between raw logs and processed logs in a Log Service project, the time recorded for the collected logs may be a point in time in the past or in the future. For example, if the Log Service project resides in greater China, you can set the time zone to Asia/Shanghai.
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: nginx-log-sidecar-demo
      namespace: default
    spec:
      template:
        metadata:
          name: nginx-log-sidecar-demo
        spec:
          restartPolicy: Never
          containers:
          - name: nginx-log-demo
            image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
            command: ["/bin/mock_log"]
            args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
            volumeMounts:
            - name: nginx-log
              mountPath: /var/log/nginx
          ##### logtail sidecar container
          - name: logtail
            # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
            # this images is released for every region
            image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
            # when recevie sigterm, logtail will delay 10 seconds and then stop
            command:
            - sh
            - -c
            - /usr/local/ilogtail/run_logtail.sh 10
            livenessProbe:
              exec:
                command:
                - /etc/init.d/ilogtaild
                - status
              initialDelaySeconds: 30
              periodSeconds: 30
            resources:
              limits:
                memory: 512Mi
              requests:
                cpu: 10m
                memory: 30Mi
            env:
              ##### base config
              # user id
              - name: "ALIYUN_LOGTAIL_USER_ID"
                value: "${your_aliyun_user_id}"
              # user defined id
              - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                value: "${your_machine_group_user_defined_id}"
              # config file path in logtail's container
              - name: "ALIYUN_LOGTAIL_CONFIG"
                value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
              ##### env tags config
              - name: "ALIYUN_LOG_ENV_TAGS"
                value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
              - name: "_pod_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: "_pod_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.podIP
              - name: "_namespace_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: "_node_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: "_node_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.hostIP
            volumeMounts:
            - name: nginx-log
              mountPath: /var/log/nginx
          ##### share this volume
          volumes:
          - name: nginx-log
            emptyDir: {}
    1. Configure the following basic variables in the configuration script.
      ##### base config
                # user id
                - name: "ALIYUN_LOGTAIL_USER_ID"
                  value: "${your_aliyun_user_id}"
                # user defined id
                - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                  value: "${your_machine_group_user_defined_id}"
                # config file path in logtail's container
                - name: "ALIYUN_LOGTAIL_CONFIG"
                  value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
      Variable Description
      ${your_aliyun_user_id} Enter the ID of your Alibaba Cloud account. For more information, see Step 1: Obtain the ID of the Alibaba Cloud account for which Log Service is activated.
      ${your_machine_group_user_defined_id} Enter the custom identifier of your machine group. The identifier must be unique in the region where your project resides. Example: nginx-log-sidecar. For more information, see Create a custom ID-based machine group.
      ${your_region_config} Specify a value based on the ID of the region where your project resides and the type of the network for your project. For more information about regions, see Table 1.
      • If logs are collected to your project over the Internet, specify the value in the region-internet format. For example, if your project resides in the China (Hangzhou) region, specify cn-hangzhou-internet.
      • If logs are collected to your project over an internal network of Alibaba Cloud, specify the value in the region format. For example, if your project resides in the China (Hangzhou) region, specify cn-hangzhou.
    2. Specify the mount path in the configuration script.
      Note We recommend that you mount a volume of the emptyDir type.
              volumeMounts:
              - name: nginx-log
                mountPath: /var/log/nginx
            ##### share this volume
            volumes:
            - name: nginx-log
              emptyDir: {}
      Parameter Description
      name The name of the volume. You can specify a name based on your business requirements.
      Notice The value of the name parameter in the volumeMounts node and the value of the name parameter in the volumes node must be the same. This ensures that the same volume is mounted for both the Logtail container and the application container.
      mountPath The mount path. You can enter the path of files in which container text logs are recorded.
    3. Specify a waiting period for the Logtail container in the configuration script.
      In most cases, the waiting period is 10 seconds. This value specifies that the Logtail container exits 10 seconds after the container receives a stop command. This setting helps prevent incomplete data collection.
      command:        
      - sh        
      - -c        
      - /usr/local/ilogtail/run_logtail.sh 10
  4. Run the following command to apply the configurations in the sidecar.yaml file.

    In this example, sidecar.yaml is used as the file name. You can specify a file name based on your business requirements.

    kubectl apply -f sidecar.yaml

Step 2: Create a machine group

  1. Log on to the Log Service console.
  2. In the Projects section, click the project that you use to install Logtail components.
  3. In the left navigation sidebar, choose Resources > Machine Groups.
  4. In the Machine Groups list, choose Machine groups > Create Machine Group.
  5. In the Create Machine Group panel, configure the following parameters and click OK.
    Parameter Description
    Name The name of the machine group.
    Notice After you create a machine group, you cannot change the name of the machine group. Proceed with caution.
    Identifier The identifier of the machine group. Select Custom ID.
    Topic The topic of the machine group. The topic is used to differentiate log data that is generated on different servers. For more information, see Log topics.
    Custom Identifier The custom identifier of the machine group. Set the value to the value of ${your_machine_group_user_defined_id} when you install Sidecar. Example: nginx-log-sidecar.

Step 3: Create a Logtail configuration

  1. Log on to the Log Service console.
  2. In the Import Data section, click RegEx - Text Log.
    This example shows how to create a Logtail configuration that is used to collect text logs in full regex mode. For more information about how to collect text logs in other modes, see Collect text logs.
  3. Select a project and a Logstore. Then, click Next.
    Select the project that you use to install Logtail components and the Logstore that you create.
  4. Click Use Existing Machine Groups.
  5. Select a machine group from Source Server Groups and move it to Applied Server Groups. Then, click Next.

    Select the machine group that you create in Step 2: Create a machine group.

    Notice If you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Log Service. To resolve this issue, you can click Automatic Retry. If the issue persists, see What do I do if no heartbeat connections are detected on Logtail?
  6. Create a Logtail configuration and click Next.
    You can collect logs in simple mode, NGINX mode, delimiter mode, JSON mode, and full regex mode. For more information, see Collect text logs.
    Notice In Sidecar mode, you must turn off Docker File.
    Specify a collection mode
  7. Preview data, configure indexes, and then click Next.
    By default, full-text indexing is enabled for Log Service. You can also configure field indexes based on collected logs in manual or automatic mode. For more information, see Configure indexes.
    Note
    • If you want to query and analyze logs, you must enable full-text indexing or field indexing. If you enable both full-text indexing and field indexing, the system uses only field indexes.
    • If a field for which indexing is enabled is of the long or double type, you cannot configure the Case Sensitive or Delimiter parameter for the field.