This topic describes how to use custom resource definitions (CRDs) to collect Kubernetes container logs in the Sidecar mode.
Prerequisites
Overview
In the Sidecar mode, the Logtail container shares the log directory with application containers in a pod. The application containers write logs to the shared directory and the Logtail container monitors the changes of log files in the shared directory and collects logs. For more information, vist Sidecar container with a logging agent and How pods manage multiple containers.
Step 1: Install Sidecar
apiVersion: batch/v1
kind: Job
metadata:
name: nginx-log-sidecar-demo
namespace: default
spec:
template:
metadata:
name: nginx-log-sidecar-demo
spec:
restartPolicy: Never
containers:
- name: nginx-log-demo
image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
command: ["/bin/mock_log"]
args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
##### logtail sidecar container
- name: logtail
# more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
# this images is released for every region
image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
# when recevie sigterm, logtail will delay 10 seconds and then stop
command:
- sh
- -c
- /usr/local/ilogtail/run_logtail.sh 10
livenessProbe:
exec:
command:
- /etc/init.d/ilogtaild
- status
initialDelaySeconds: 30
periodSeconds: 30
resources:
limits:
memory: 512Mi
requests:
cpu: 10m
memory: 30Mi
env:
##### base config
# user id
- name: "ALIYUN_LOGTAIL_USER_ID"
value: "${your_aliyun_user_id}"
# user defined id
- name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
value: "${your_machine_group_user_defined_id}"
# config file path in logtail's container
- name: "ALIYUN_LOGTAIL_CONFIG"
value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
##### env tags config
- name: "ALIYUN_LOG_ENV_TAGS"
value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
- name: "_pod_name_"
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: "_pod_ip_"
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: "_namespace_"
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "_node_name_"
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: "_node_ip_"
valueFrom:
fieldRef:
fieldPath: status.hostIP
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
##### share this volume
volumes:
- name: nginx-log
emptyDir: {}Step 2: Configure log collection
apiVersion: log.alibabacloud.com/v1alpha1 ## The default setting, which you do not need to modify.
kind: AliyunLogConfig ## The default setting, which you do not need to modify.
metadata:
name: simple-stdout-example ## The resource name, which must be unique in the cluster.
spec:
project: k8s-my-project ## The name of a project. If you do not specify this parameter, logs are sent to the project that is configured when you installed Log Service components on the cluster.
logstore: k8s-stdout ## The Logstore name. A Logstore is automatically created if the specified Logstore does not exist.
machineGroups: ## The name of the machine group. Set the value to the ${your_machine_group_user_defined_id} value in the Sidecar configurations. The Sidecar and CRD are associated by using this configuration.
- nginx-log-sidecar
shardCount: 2 ## Optional. The number of Logstore shards. Valid values: 1 to 10. Default value: 2.
lifeCycle: 90 ## Optional. The retention period of log data in the Logstore. Valid values: 1 to 7300. Default value: 90. Unit: days. The value 7300 indicates that log data is permanently stored in the Logstore.
logtailConfig: ## The Logtail configurations.
inputType: plugin ## The type of the data source from which data is collected. Only text logs can be collected.
configName: simple-stdout-example ## The name of the Logtail configuration file. This name must be the same as the resource name that is specified by the metadata.name parameter.
inputDetail: ## The detailed Logtail configurations for log collection. For more information, see the examples.
...In the Sidecar mode, only text logs can be collected. The value of the dockerFile parameter must be set to false.
For more information about the logtailConfig parameter, see Logtail configuration.
After the configuration file is created, the Logtail container collects and uploads logs to Log Service. You can log on to the Log Service console to view the logs.
Example
The following example shows how to use a CRD to collect logs in the Sidecar mode from a user-created on-premises Kubernetes cluster. The project that is created in Log Service for log collection resides in the China (Hangzhou) region. Logs are collected over the Internet. The volume that is mounted on the /var/log/nginx directory of the nginx-log-demo and Logtail containers is an emptyDir volume named nginx-log.