This topic describes how to use CRDs to collect Kubernetes container logs in the Sidecar mode.
Prerequisites
Overview
In the Sidecar mode, the Logtail container shares the log directory with application containers in a pod. Application containers write logs to the shared directory and the Logtail container monitors the changes of log files in the shared directory and collects logs. For more information, see Sidecar container with a logging agent and How Pods manage multiple Containers.
Step 1: Install Sidecar
apiVersion: batch/v1
kind: Job
metadata:
name: nginx-log-sidecar-demo
namespace: default
spec:
template:
metadata:
name: nginx-log-sidecar-demo
spec:
restartPolicy: Never
containers:
- name: nginx-log-demo
image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
command: ["/bin/mock_log"]
args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
##### logtail sidecar container
- name: logtail
# more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
# This image is released for every region
image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
# when receive sigterm, logtail will delay 10 seconds and then stop
command:
- sh
- -c
- /usr/local/ilogtail/run_logtail.sh 10
livenessProbe:
exec:
command:
- /etc/init.d/ilogtaild
- status
initialDelaySeconds: 30
periodSeconds: 30
resources:
limits:
memory: 512Mi
requests:
cpu: 10m
memory: 30Mi
env:
##### base config
# user id
- name: "ALIYUN_LOGTAIL_USER_ID"
value: "${your_aliyun_user_id}"
# user defined id
- name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
value: "${your_machine_group_user_defined_id}"
# config file path in logtail's container
- name: "ALIYUN_LOGTAIL_CONFIG"
value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
##### env tags config
- name: "ALIYUN_LOG_ENV_TAGS"
value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
- name: "_pod_name_"
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: "_pod_ip_"
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: "_namespace_"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "_node_name_"
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: "_node_ip_"
valueFrom:
fieldRef:
fieldPath: status.hostIP
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
##### share this volume
volumes:
- name: nginx-log
emptyDir: {}Step 2: Configure log collection
apiVersion: log.alibabacloud.com/v1alpha1 ## The default setting, which you do not need to modify.
kind: AliyunLogConfig ## The default setting, which you do not need to modify.
metadata:
name: simple-stdout-example ## The resource name, which must be unique in the cluster.
spec:
project: k8s-my-project ## The name of a project. If you do not specify this parameter, logs are collected to the project configured when you installed Log Service components on the cluster.
logstore: k8s-stdout ## The Logstore name. A Logstore is automatically created if the specified Logstore does not exist.
machineGroups:- nginx-log-sidecar ## The name of the server group. If you do not specify this parameter, logs are collected from the server group where Logtail DaemonSets reside.
shardCount: 2 ## The number of Logstore shards. Valid values: 1 to 10. Default value: 2. This parameter is optional.
lifeCycle: 90 ## The retention days of log data in the Logstore. Valid values: 1 to 7300. Default value: 90. The value 7300 indicates that log data is permanently stored in the Logstore. This parameter is optional.
logtailConfig: ## The Logtail settings.
inputType: plugin ## The type of the data source to be collected. Only text logs can be collected.
configName: simple-stdout-example ## The name of the Logtail configuration file. This name must be the same as the resource name.
inputDetail: ## The detailed settings of Logtail. For more information, see the examples.
...In the Sidecar mode, only text logs can be collected. The value of the dockerFile parameter must be set to false.
For more information about the logtailConfig parameter, see Logtail configuration.
After the configuration file is created, the Logtail container automatically collects and uploads logs to Log Service. You can log on to the Log Service console to view the logs.
Example
The following example shows how to use a CRD to collect logs in the Sidecar mode from
a user-created Kubernetes cluster deployed in a local IDC. The project created in
Log Service for log collection resides in the China (Hangzhou) region. Logs are collected
over the Internet. The volume mounted on the /var/log/nginx directory of the nginx-log-demo and Logtail containers is an emptyDir volume named nginx-log.