ACK collects and analyzes logs and audit data from multiple Kubernetes components, including the API server, Ingress, control plane components, and key Kubernetes events, to support root cause analysis and traceability when security or cluster issues occur. provides audit logging and event monitoring for Kubernetes clusters to support security analysis and troubleshooting.
Use cluster auditing
The API server audit log records and traces user activities for security and operations. See Work with cluster auditing to collect audit logs with SLS, configure alert rules, or disable auditing.
The ACK audit policy is as follows:
Audit log fields are defined in audit.k8s.io/v1 Event.
apiVersion: audit.k8s.io/v1beta1 # This is required.
kind: Policy
# Don't log events for requests in the RequestReceived stage.
omitStages:
- "RequestReceived"
rules:
# The following requests are high-volume and low-risk, so they are not logged.
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # core
resources: ["endpoints", "services"]
- level: None
users: ["system:unsecured"]
namespaces: ["kube-system"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["configmaps"]
- level: None
users: ["kubelet"] # legacy kubelet identity
verbs: ["get"]
resources:
- group: "" # core
resources: ["nodes"]
- level: None
userGroups: ["system:nodes"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["nodes"]
- level: None
users:
- system:kube-controller-manager
- system:kube-scheduler
- system:serviceaccount:kube-system:endpoint-controller
verbs: ["get", "update"]
namespaces: ["kube-system"]
resources:
- group: "" # core
resources: ["endpoints"]
- level: None
users: ["system:apiserver"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["namespaces"]
# Don't log requests to these read-only URLs.
- level: None
nonResourceURLs:
- /healthz*
- /version
- /swagger*
# Don't log event requests.
- level: None
resources:
- group: "" # core
resources: ["events"]
# Secrets, ConfigMaps, and token reviews can contain sensitive and binary data,
# so only log them at the Metadata level.
- level: Metadata
resources:
- group: "" # core
resources: ["secrets", "configmaps"]
- group: authentication.k8s.io
resources: ["tokenreviews"]
- level: Request
verbs: ["get", "list", "watch"]
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"
# Default level for known APIs.
- level: RequestResponse
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"
- group: "autoscaling.alibabacloud.com"
# Default level for all other requests.
- level: MetadataCustomize cluster audit log rules
Add custom audit rules in the auditPolicyRules field on the kube-apiserver configuration page.
# The rules must be defined as a list in YAML format.
- level: RequestResponse
resources:
- group: "policy.alibabacloud.com"
resources: ["policies"]
- level: None
userGroups: ["system:nodes"]
verbs: ["update", "patch"]
resources:
- group: ""
resources: ["nodes/status", "pods/status"]These rules:
-
Enhanced auditing: Logs all operations on ACK policy templates (
policies.policy.alibabacloud.com) at theRequestResponselevel. -
Reduced noise: Ignores node and Pod status updates from kubelet (
system:nodesgroup), such as heartbeats.
Audit rule syntax is documented in Auditing.
When configuring custom rules:
-
Rule format: Enter a list of audit rules (
rules), not a completePolicyobject. -
Rule priority: You cannot change the log level for requests covered by ACK default audit rules.
-
Rule validation: ACK validates rules against the policies.audit.k8s.io specification. See Kubernetes Audit for valid rule syntax. Important: An incorrectly formatted rule can prevent the API server from starting.
Container exec activity auditing
Attackers may use exec to enter containers and launch lateral attacks, but the default API server audit log does not record in-container commands. To capture post-intrusion commands for root cause analysis and security mitigation, enable container internal activity auditing.
Use audit metadata
Two audit log annotations, authorization.k8s.io/decision and authorization.k8s.io/reason, indicate whether a request was authorized and why. Use them to trace API call authorization decisions.
Monitor cluster events with NPD and SLS
node-problem-detector (NPD) converts node anomalies, such as hung Docker engines, kernel hangs, outbound network issues, and file descriptor exhaustion, into events, and works with kube-eventer for closed-loop alerting. The Kubernetes Event Center in SLS aggregates all cluster events in real time, including Pod evictions, image pull failures, and suspicious exec access, for querying, analysis, visualization, and alerting. See Event monitoring.
Enable the Ingress dashboard
The Ingress component logs HTTP requests to standard output and integrates with SLS for dashboard-based analysis. The Ingress dashboard displays key metrics, including page views (PVs), unique visitors (UVs), traffic, latency, and top URLs, for real-time traffic monitoring and DoS attack detection. See Ingress Dashboard.
Enable CoreDNS logging
ACK clusters use CoreDNS as the internal DNS server. Analyze CoreDNS logs to troubleshoot DNS resolution issues or investigate high-risk domain queries. The SLS CoreDNS dashboard helps identify suspicious requests. See Analyze and monitor CoreDNS logs.