:Set up non-intrusive service observation

Limitations

Kubernetes data plane monitoring requires an x86_64 host with Linux kernel version 4.19 or later, or kernel version 3.1.0 (for CentOS 7.6 to 7.9 only). You can run the uname -r command to check the kernel information.

Set up using the console

1. Log on to the Simple Log Service console.

2. In the Log Application section, click the Intelligent O&M tab. Then, click Full-stack Observability.

3. On the Simple Log Service Full-stack Observability page, click the instance you want.

4. In the left-side navigation pane, click Full-stack Monitoring.

   If this is the first time you are using Full-stack Monitoring for this instance, click Enable.

5. In the left-side navigation pane, click Data Collection. On the Data Import Configurations page, find Non-intrusive Service Observation in the Kubernetes Monitoring section.

   If this is the first time you create a data access configuration for the target monitoring item, turn on the switch to open the configuration page. If a configuration already exists, click the icon to open the configuration page.

6. Create a machine group:

   If you have already created a machine group, skip this step.

   • To create a machine group for an ACK cluster, see Create an IP address-based machine group.

   • To create a machine group for a self-managed Kubernetes cluster, see Create a custom identifier-based machine group.

Set up using the CLI

1. Download the Logtail CRD tool to generate collection configurations.

   Method	Description
   Install from outside the cluster	Requires a ~/.kube/config file on your local machine with permissions to run kubectl commands against the target cluster.
   Install from inside a container	Uses the permissions of the alibaba-log-controller component to create CRDs. Use this method if you do not have a ~/.kube/config file or if network issues prevent you from connecting to the cluster.

   Install from outside the cluster

   1. Log on to your cluster and download the template tool.

      • China

        curl https://logtail-release-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/kubernetes/crd-tool.tar.gz -o /tmp/crd-tool.tar.gz

      • Regions outside China

        curl https://logtail-release-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/kubernetes/crd-tool.tar.gz -o /tmp/crd-tool.tar.gz

   2. Install the template tool. A successful installation generates the sls-crd-tool executable in the current directory.

      tar -xvf /tmp/crd-tool.tar.gz -C /tmp &&chmod 755 /tmp/crd-tool/install.sh  && sh -x  /tmp/crd-tool/install.sh

   3. Run ./sls-crd-tool list to verify the installation. The installation is successful if the command returns any output.

   Install from inside a container

   1. Log on to the cluster and access the alibaba-log-controller container.

      kubectl get pods -n kube-system -o wide |grep alibaba-log-controller | awk -F ' ' '{print $1}'
      kubectl exec -it {pod} -n kube-system bash
      cd ~

   2. Download the template tool.

      • If the cluster can access public networks, run one of the following commands:

        • China

          curl https://logtail-release-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/kubernetes/crd-tool.tar.gz -o /tmp/crd-tool.tar.gz

        • Regions outside China

          curl https://logtail-release-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/kubernetes/crd-tool.tar.gz -o /tmp/crd-tool.tar.gz

      • If the cluster cannot access public networks, download the tool from outside the cluster and transfer it to the container by using a command such as kubectl cp <source> <destination> or other file transfer methods supported by ACK.

   3. Install the template tool. A successful installation generates the sls-crd-tool executable in the current directory.

      tar -xvf /tmp/crd-tool.tar.gz -C /tmp &&chmod 755 /tmp/crd-tool/install.sh  && sh -x  /tmp/crd-tool/install.sh

   4. Run ./sls-crd-tool list to verify the installation. The installation is successful if the command returns any output.

2. Use the Logtail CRD tool to generate a collection configuration.

   1. View the template definition.

      ./sls-crd-tool  get ebpfK8sPlugin

   2. Replace REQUIRED with the ID of your current instance and run the following command to preview the parameters.

      ./sls-crd-tool  apply -f template-ebpfK8sPlugin.yaml --create=false

   3. Confirm that the project parameter is set to the Project associated with your current instance. Then, run the following command to deploy the template collection file.

      ./sls-crd-tool  apply -f template-ebpfK8sPlugin.yaml

   4. Return to the Data Import Configurations page. A successful installation increments the Configurations count in the Resource Monitoring section by one. The count remains unchanged if the installation fails.

Configure data collection

1. Click Use Existing Machine Groups.

   After you install the monitoring component, Simple Log Service automatically creates a machine group named k8s-group-${your_k8s_cluster_id}. You can use this machine group directly.

2. Select the target machine group, such as k8s-group-${your_k8s_cluster_id}. Move the group from Source Machine Group to Applied Machine Group, and then click Next.

   Important

   If the machine group heartbeat status is FAIL, click Automatic Retry. If the issue persists, see What do I do if a Logtail machine group has no heartbeats? for troubleshooting.

3. In the Configure Data Source step, set the following parameters, and then click Complete.

   Parameter	Description
   General settings
   Configuration Name	The name of the Logtail collection configuration.
   Cluster	The name of the cluster. After you set this parameter, Simple Log Service adds a cluster=cluster_name label to the monitoring data collected through this configuration. Important Ensure that the cluster name is unique to avoid data conflicts.
   Monitor Application Layer Protocols	If you turn on the Monitor Application Layer Protocols switch, Logtail parses application-layer network protocols such as HTTP, MySQL, and Redis.
   Statistical Interval of Network Metrics	The interval, in seconds, at which Layer 4 network data is aggregated and reported. We recommend a value of 600 seconds or less.
   Statistical Interval of Protocol Metrics	The interval, in seconds, at which Layer 7 protocol network data is aggregated and reported. We recommend a value of 60 seconds or less.
   Protocol Sample Rate	The sampling rate for network data. This setting filters only Layer 7 network data and does not affect Layer 4 network statistics.
   Protocol allowlist	Select the application-layer protocols that you want to parse.
   Kubernetes Selector
   Namespace allowlist	Enter a regular expression to match the names of Namespaces from which to collect data.
   Namespace denylist	Enter a regular expression to match the names of Namespaces to exclude from data collection.
   Pod allowlist	Enter a regular expression to match the names of Pods from which to collect data.
   Pod denylist	Enter a regular expression to match the names of Pods to exclude from data collection.
   Container allowlist	Enter a regular expression to match the names of containers from which to collect data.
   Container denylist	Enter a regular expression to match the names of containers to exclude from data collection.
   Label allowlist	Specifies which containers to collect data from based on their labels. Set Label key to a specific name and Label value to a regular expression. For example, setting Label key to io.kubernetes.container.name and Label value to ^(nginx|cube)$ matches containers named nginx or cube. Multiple entries are combined using OR logic. A container is matched if it satisfies any of the specified conditions.
   Label denylist	Specifies which containers to exclude from data collection based on their labels. Set Label key to a specific name and Label value to a regular expression. For example, setting Label key to io.kubernetes.container.name and Label value to ^(nginx|cube)$ excludes containers named nginx or cube. Multiple entries are combined using OR logic. A container is excluded if it satisfies any of the specified conditions.
   Environment Variable Whitelist	Specifies which containers to collect data from based on their environment variables. Set Env key to a specific name and Env value to a regular expression. For example, setting Env key to NGINX_SERVICE_PORT and Env value to ^(80|6379)$ matches containers where the NGINX_SERVICE_PORT environment variable is set to 80 or 6379. Multiple entries are combined using OR logic. A container is matched if it satisfies any of the specified conditions.
   Environment Variable Blacklist	Specifies which containers to exclude from data collection based on their environment variables. Set Env key to a specific name and Env value to a regular expression. For example, setting Env key to NGINX_SERVICE_PORT and Env value to ^(80|6379)$ excludes containers where the NGINX_SERVICE_PORT environment variable is set to 80 or 6379. Multiple entries are combined using OR logic. A container is excluded if it satisfies any of the specified conditions.
   Advanced Settings
   Drop Local Packets	If you turn on the Drop Local Packets switch, Logtail discards network requests with a local destination address.
   Drop Unix Packets	If you turn on the Drop Unix Packets switch, Logtail discards network requests from the UNIX domain. The UNIX domain is typically used for local network interaction.
   Drop Unknown Packets	If you turn on the Drop Unknown Packets switch, Logtail discards network requests that are not from the INET or UNIX domain.
   Read Interval of Container Data	The interval, in seconds, for a full read of container metadata. We recommend a value of 60 seconds or less.
   Read Interval of Socket Data	The interval, in seconds, for reading socket metadata. We recommend a value of 30 seconds or less.
   Protocol Aggregation Window	The size of the process-level data aggregation window within the protocol metrics statistical interval. This parameter helps control resource consumption and prevent high Logtail memory usage when there are many distinct calls. The default value is 500 for clients and 5,000 for servers.

   After you complete the configuration, Simple Log Service automatically creates assets such as a Metricstore.