This topic describes how to collect trace data from Apache SkyWalking by using a Logtail that is deployed in an Alibaba Cloud Container Service for Kubernetes (ACK) cluster to Log Service. After the trace data is collected to Log Service, you can store, analyze, visualize the data in Log Service. You can also configure alerts and perform AIOps based on the data.

Prerequisites

  • A Logstore is created. For more information, see Create a Logstore.
  • A machine group that has a custom identifier is created. For more information, see Create a custom ID-based machine group.
    Notice Make sure that the custom identifier is unique in the region of the Log Service project to which the Logstore belongs.
  • A trace instance is created. For more information, see Create a trace instance.

Step 1: Deploy a data collection image

  1. Deploy the Logtail image to an ACK cluster by using a configuration file.
    The following sample code provides a configuration template. For more information, see Create an application by using a private image repository.
          containers:    
          - name: logtail
            # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
            image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:v0.16.68.0-7a79f4e-aliyun
            command:
            - sh
            - -c
            - /usr/local/ilogtail/run_logtail.sh 10
            livenessProbe:
              exec:
                command:
                - /etc/init.d/ilogtaild
                - status
              initialDelaySeconds: 30
              periodSeconds: 30
            resources:
              limits:
                memory: 512Mi
              requests:
                cpu: 10m
                memory: 30Mi
            env:
              - name: "ALIYUN_LOGTAIL_USER_ID"
                value: "${your_aliyun_user_id}"
              - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                value: "${your_machine_group_user_defined_id} "
              - name: "ALIYUN_LOGTAIL_CONFIG"
                value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
              - name: "ALIYUN_LOG_ENV_TAGS"
                value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
              - name: "_pod_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: "_pod_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.podIP
              - name: "_namespace_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: "_node_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: "_node_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.hostIP
    Parameter Description
    ${your_region_config} Specify a value based on the ID of the region where your Log Service project resides and the type of the network for your project. For more information about regions, see Table 1.
    • If your project is accessible over the Internet, specify the value in the region-internet format. For example, if your project resides in the China (Hangzhou) region, specify cn-hangzhou-Internet.
    • If your project is accessible over an internal network of Alibaba Cloud, specify the value in the region format. For example, if your project resides in the China (Hangzhou) region, specify cn-hangzhou.
    ${your_aliyun_user_id} Enter the ID of your Alibaba Cloud account. For more information, see Obtain the ID of an Alibaba Cloud account.
    ${your_machine_group_user_defined_id} Enter the custom identifier that you specify when you create the machine group.
    Notice Make sure that the custom identifier is unique in the region of the Log Service project to which the Logstore belongs.
  2. Redeploy the container where Logtail runs.

Step 2: Create a Logtail configuration

  1. Log on to the Log Service console.
  2. In the Import Data section, click SkyWalking.
  3. In the Specify Logstore step, select the project and Logstore. Then, click Next.
  4. In the Create Machine Group step, click Use Existing Machine Groups.
  5. In the Machine Group Settings step, select the machine group that you want to use in the Source Server Groups section and move the machine group to the Applied Server Groups section. Then, click Next.
  6. In the Specify Data Source step, configure the Logtail plug-ins and click Next.
    A configuration template is provided in the Plug-in Config field. You must replace ${instance} with the ID of your trace instance.
    {
          "inputs" : [
            {
                "detail" : {
                    "Address" : "0.0.0.0:11800"
                },
                "type" : "service_skywalking_agent_v3"
            }
        ],
        "aggregators" : [
            {
                "detail" : {
                    "MetricsLogstore" : "${instance}-metrics",
                    "TraceLogstore" : "${instance}-traces"
                },
                "type" : "aggregator_skywalking"
            }
        ],
        "global" : {
            "AlwaysOnline" : true,
            "DelayStopSec" : 300
        }
    }
    Click Next to complete the Logtail configuration. Then, Log Service starts to collect logs.
    Note
    • A Logtail configuration requires a maximum of 3 minutes to take effect.
    • If an error occurs when a Logtail configuration is used to collect logs, follow the instructions provided in Diagnose collection errors to troubleshoot the error.

FAQ

How do I check whether a Logtail configuration is in effect?

In the container where Logtail runs, run the cat /usr/local/ilogtail/user_log_config.json | grep skywalking command.
  • If the command output includes SkyWalking, the Logtail configuration is in effect.
  • If the command output does not include SkyWalking, the Logtail configuration is not in effect. In this case, check whether the custom identifier that you specify for the machine group when you deploy the data collection image is the same as the custom identifier that you specify when you create the machine group.

What to do next

After you complete the preceding configurations, you can query and analyze the trace data of Apache SkyWalking in the specified Logstore. For more information, see Query and analyze trace data.