Collect Container Logs via SLS CRD in ACK Serverless - Container Service for Kubernetes

Simple Log Service integrates with ACK Serverless through a Kubernetes-native CustomResourceDefinition (CRD). You define log collection rules in a YAML file; the platform deploys Logtail automatically and starts collecting logs from newly created Elastic Container Instance (ECI) Pods. This guide shows you how to install the alibaba-log-controller component and configure AliyunLogConfig CRDs to collect text logs and stdout from your ECI Pods.

Prerequisites

Before you begin, ensure that you have:

An ACK Serverless cluster. See Create an ACK Serverless cluster.
ack-virtual-node 2.7.0 or later installed in the cluster. See ack-virtual-node.

Configure log collection

Step 1: Deploy alibaba-log-controller

Log on to the ACK console.
On the Clusters page, find your cluster and click its name.
In the left navigation pane, choose Operations > Add-ons.
Click the Logs and Monitoring tab, find the Alibaba-log-controller card, and click Install.
In the confirmation dialog, click OK.

After installation completes, Installed appears in the upper-right corner of the Alibaba-log-controller card.

Step 2: Create an AliyunLogConfig CRD

Before you create the CRD, note the following constraints:

configName must match metadata.name exactly.
CRD-based log collection applies only to ECI Pods created after the CRD is created. To collect logs from existing Pods, perform a rolling update to recreate them.
After you create a Logtail configuration through the CRD, manage it only through the CRD. Changes made in the Simple Log Service console are not synchronized back to the CRD. To update a configuration, modify the AliyunLogConfig CRD resource directly.

CRD template

apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  name: simple-stdout-example
spec:
  project: k8s-my-project
  logstore: k8s-stdout
  logstoreMode: standard
  shardCount: 2
  lifeCycle: 90
  logtailConfig:
    inputType: plugin
    configName: simple-stdout-example
    inputDetail:
      ...

Basic parameters

Parameter	Type	Required	Description
`project`	string	No	The Simple Log Service project name. Defaults to the project used to install Logtail. Auto-created if the project does not exist.
`logstore`	string	Yes	The Logstore name. Auto-created if the Logstore does not exist.
`logstoreMode`	string	No	The Logstore type. Valid values: `query` (query Logstore), `standard` (standard Logstore). Takes effect only on Logstore creation. See Manage a Logstore.
`shardCount`	int	No	Number of shards. Valid values: 1–10. Default: 2.
`lifeCycle`	int	No	Log retention period in days. Valid values: 1–3650. Default: 90. A value of 3650 stores logs permanently. Takes effect only on Logstore creation.
`machineGroups`	array	No	The machine group. Simple Log Service automatically creates `k8s-group-${your_k8s_cluster_id}` when Logtail is installed.
`logtailConfig`	object	Yes	Logtail configuration object. See the sub-parameters below.

logtailConfig sub-parameters:

Sub-parameter	Description
`configName`	The Logtail configuration name. Must match `metadata.name`.
`inputType`	The log source type. `plugin` collects stdout and plugin-based sources (such as MySQL binary logs). `file` collects text logs using fixed modes such as regular expression or delimiter mode.
`inputDetail`	Detailed input configuration.
`outputType`	The output type. Set to `LogService`. Logs can only be sent to Simple Log Service.
`outputDetail`	Detailed output configuration.
`logSample`	A sample log entry.

For the full parameter reference, see Logtail configurations.

Container filtering

The available filtering options depend on your Logtail version.

Logtail earlier than v1.0.34 — filter by environment variables and container labels only.

A Kubernetes namespace and container name map to these container labels:

Namespace: io.kubernetes.pod.namespace
Container name: io.kubernetes.container.name

Use these two labels to filter containers. If they don't meet your requirements, use environment variable whitelists or blacklists instead.

Logtail v1.0.34 and later — filter by Kubernetes-level metadata including Pod name, namespace, container name, and labels.

Configure filtering parameters in logtailConfig.inputDetail.advanced.k8s:

Parameter	Description
`IncludeK8sLabel`	Kubernetes label whitelist — containers to include
`ExcludeK8sLabel`	Kubernetes label blacklist — containers to exclude
`K8sNamespaceRegex`	Filter by namespace (regex)
`K8sPodRegex`	Filter by Pod name (regex)
`K8sContainerRegex`	Filter by container name (regex)

For more details, see Collect text logs from Kubernetes containers in DaemonSet mode and Collect stdout and stderr from Kubernetes containers in DaemonSet mode (old version).

Example: collect text logs

Create log-file.yaml with the following content:

apiVersion: log.alibabacloud.com/v1alpha1  # Use the default value. Do not modify.
kind: AliyunLogConfig                      # Use the default value. Do not modify.
metadata:
  name: test-file                  # Unique resource name in the Kubernetes cluster.
spec:
  project: k8s-log-c326bc86****    # Optional. Auto-created if the project does not exist.
  logstore: test-file              # Required. Auto-created if the Logstore does not exist.
  logtailConfig:
    inputType: file                # file = text logs; plugin = stdout logs.
    configName: test-file          # Must match metadata.name.
    inputDetail:
      logType: common_reg_log      # Simple mode.
      logPath: /log/               # Log file directory.
      filePattern: "*.log"         # Log file name pattern. Supports * and ?.
      dockerFile: true             # Set to true to collect from containers.
      advanced:
        k8s:
          K8sNamespaceRegex: ^(default)$
          K8sPodRegex: '^(eci-sls-demo.*)$'

Example: collect stdout logs

Create log-stdout.yaml with the following content:

apiVersion: log.alibabacloud.com/v1alpha1   # Use the default value. Do not modify.
kind: AliyunLogConfig                       # Use the default value. Do not modify.
metadata:
  name: test-stdout               # Unique resource name in the Kubernetes cluster.
spec:
  project: k8s-log-c326bc86****   # Optional. Auto-created if the project does not exist.
  logstore: test-stdout           # Required. Auto-created if the Logstore does not exist.
  shardCount: 2                   # Optional. Default: 2. Valid values: 1–10.
  lifeCycle: 90                   # Optional. Default: 90 days. Valid values: 1–3650. 3650 = permanent.
  logtailConfig:
    inputType: plugin             # plugin = stdout logs; file = text logs.
    configName: test-stdout       # Must match metadata.name.
    inputDetail:
      plugin:
        inputs:
          - type: service_docker_stdout
            detail:
              Stdout: true
              Stderr: true
#              IncludeEnv:
#                aliyun_logs_test-stdout: "stdout"

Apply the CRD configuration

kubectl apply -f log-file.yaml
kubectl apply -f log-stdout.yaml

After the configuration is applied, Logtail collects stdout or text logs from each container and sends them to Simple Log Service.

Important

After logs are collected, create indexes to enable querying and analysis in the Logstore. See Create indexes.

Test log collection

After creating the AliyunLogConfig CRD, Simple Log Service automatically collects logs from newly created Pods. Use the following steps to verify the setup.

Create a test application

Create test-sls-crd.yaml with the following Deployment. The container continuously writes to both stdout and a log file after startup.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: eci-sls-demo
  labels:
    app: sls
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sls
  template:
    metadata:
      name: sls-test
      labels:
        app: sls
        alibabacloud.com/eci: "true"
    spec:
      containers:
      - args:
        - -c
        - mkdir -p /log;while true; do echo hello world; date; echo hello sls >> /log/busy.log; sleep 1;done
        command:
        - /bin/sh
        image: registry-vpc.cn-beijing.aliyuncs.com/eci_open/busybox:1.30
        imagePullPolicy: Always
        name: busybox

kubectl create -f test-sls-crd.yaml

Verify the Pod status

kubectl get pod

Expected output:

NAME                            READY   STATUS    RESTARTS   AGE
eci-sls-demo-7bf8849b9f-cgpbn   1/1     Running   0          2m14s

View collected logs

Log on to the Simple Log Service console.
Click the project name.
Find the Logstore for your containers and click its name to view the logs.

Text log collection:

Stdout log collection:

Disable log collection for specific Pods

By default, the AliyunLogConfig CRD collects logs from all Pods that match the collection rules. To exclude specific Pods, add the k8s.aliyun.com/eci-sls-enable: "false" annotation to the Pod's metadata.

Important

Add annotations in spec.template.metadata when creating a Deployment, not in the top-level metadata.
Annotations take effect only when ECI-based Pods are created. Adding or modifying annotations during a Pod update has no effect.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: eci-sls-demo2
  labels:
    app: sls
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sls
  template:
    metadata:
      name: sls-test
      labels:
        app: sls
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-sls-enable: "false"    # Disables log collection.
    spec:
      containers:
      - args:
        - -c
        - mkdir -p /log;while true; do echo hello world; date; echo hello sls >> /log/busy.log; sleep 1;
          done
        command:
        - /bin/sh
        image: registry.cn-shanghai.aliyuncs.com/eci_open/busybox:1.30
        imagePullPolicy: Always
        name: busybox

Troubleshooting

If logs are not collected after you apply the CRD, use the following steps to diagnose the issue.

Check whether the Pod was created after the CRD

CRD-based log collection applies only to ECI Pods created after the CRD is created. If existing Pods are not producing logs, perform a rolling update to recreate them:

kubectl rollout restart deployment/<your-deployment-name>

Inspect Pod events

Check Pod events for errors related to log controller configuration:

kubectl describe pod <pod-name>

Review the Events section of the output for warnings or errors from the log controller.

Verify that alibaba-log-controller is running

kubectl get pods -n kube-system | grep alibaba-log-controller

If the Pod is not in Running state, reinstall the component from the ACK console under Operations > Add-ons > Logs and Monitoring.

Verify that the AliyunLogConfig CRD was applied

kubectl get aliyunlogconfig

Confirm that your CRD resource appears in the output and that configName matches metadata.name in your YAML.