Enable Kubernetes Audit Logs to Strengthen Cluster Security - Container Service for Kubernetes

Container Service for Kubernetes (ACK) provides several logging and auditing capabilities to help you investigate security incidents, trace operational history, and comply with audit requirements. This page covers the following features:

API server audit logs — record all control-plane operations with configurable detail levels
Container exec auditing — capture commands run inside containers after exec access
Cluster event monitoring — aggregate Kubernetes events for alerting and analysis
Ingress access log monitoring — analyze HTTP traffic patterns and detect anomalies
CoreDNS log analysis — troubleshoot DNS issues and identify suspicious domain requests

API Server Audit Logs

ACK configures the kube-apiserver with a default audit policy that records control-plane operations performed by all users and system components. Logs are stored in Simple Log Service (SLS), where you can query, visualize, and set up alerts. For instructions on collecting and analyzing audit logs in SLS, configuring custom alert rules, and disabling the audit feature, see Use the Cluster API Server Audit Feature.

Understand the Default Audit Policy

ACK applies a tiered audit policy to all clusters. The policy omits the RequestReceived stage to avoid duplicate events and uses the following levels to balance coverage against log volume:

Audit Level	What It Covers	Log Volume
`None`	High-frequency, low-risk system calls (kubelet heartbeats, kube-proxy watches, internal endpoint management)	No logs generated
`Metadata`	Sensitive resources: Secrets, ConfigMaps, TokenReviews	Request metadata only (no body)
`Request`	Read operations (get/list/watch) on standard Kubernetes API groups	Request metadata + request body
`RequestResponse`	Write operations on standard API groups, plus Alibaba Cloud autoscaling resources	Request metadata + request body + response body
`Metadata` (default)	All other requests not matched by the above rules	Request metadata only

The full policy YAML is shown below for reference. This is the policy ACK applies by default — you cannot modify these built-in rules directly, but you can add custom rules on top of them (see Customize Audit Rules).

Note

For details on audit log fields, see the audit-k8s-io-v1-Event.

apiVersion: audit.k8s.io/v1beta1 # This is required.
kind: Policy
# Do not generate audit events for all requests in the RequestReceived stage.
omitStages:
- "RequestReceived"
rules:
# The following requests are high-frequency, low-risk system calls. Exclude these requests from auditing.
- level: None
  users: ["system:kube-proxy"]
  verbs: ["watch"]
  resources:
    - group: "" # core
      resources: ["endpoints", "services"]
- level: None
  users: ["system:unsecured"]
  namespaces: ["kube-system"]
  verbs: ["get"]
  resources:
    - group: "" # core
      resources: ["configmaps"]
- level: None
  users: ["kubelet"] # legacy kubelet identity
  verbs: ["get"]
  resources:
    - group: "" # core
      resources: ["nodes"]
- level: None
  userGroups: ["system:nodes"]
  verbs: ["get"]
  resources:
    - group: "" # core
      resources: ["nodes"]
- level: None
  users:
    - system:kube-controller-manager
    - system:kube-scheduler
    - system:serviceaccount:kube-system:endpoint-controller
  verbs: ["get", "update"]
  namespaces: ["kube-system"]
  resources:
    - group: "" # core
      resources: ["endpoints"]
- level: None
  users: ["system:apiserver"]
  verbs: ["get"]
  resources:
    - group: "" # core
      resources: ["namespaces"]
# Do not log these read-only URLs.
- level: None
  nonResourceURLs:
  - "/healthz*"
  - "/version"
  - "/swagger*"
# Do not log event requests.
- level: None
  resources:
    - group: "" # core
      resources: ["events"]
# Log sensitive resource access at Metadata level only (no request/response body).
- level: Metadata
  resources:
    - group: "" # core
      resources: ["secrets", "configmaps"]
    - group: "authentication.k8s.io"
      resources: ["tokenreviews"]
# Log read operations on standard API groups at Request level.
- level: Request
  verbs: ["get", "list", "watch"]
  resources:
    - group: "" # core
    - group: "admissionregistration.k8s.io"
    - group: "apps"
    - group: "authentication.k8s.io"
    - group: "authorization.k8s.io"
    - group: "autoscaling"
    - group: "batch"
    - group: "certificates.k8s.io"
    - group: "extensions"
    - group: "networking.k8s.io"
    - group: "policy"
    - group: "rbac.authorization.k8s.io"
    - group: "settings.k8s.io"
    - group: "storage.k8s.io"
# Default level for known APIs — log writes at RequestResponse level.
- level: RequestResponse
  resources:
    - group: "" # core
    - group: "admissionregistration.k8s.io"
    - group: "apps"
    - group: "authentication.k8s.io"
    - group: "authorization.k8s.io"
    - group: "autoscaling"
    - group: "batch"
    - group: "certificates.k8s.io"
    - group: "extensions"
    - group: "networking.k8s.io"
    - group: "policy"
    - group: "rbac.authorization.k8s.io"
    - group: "settings.k8s.io"
    - group: "storage.k8s.io"
    - group: "autoscaling.alibabacloud.com"
# Default level for all other requests.
- level: Metadata

Customize Audit Rules

ACK lets you extend the default audit policy by adding custom rules. Configure these rules in auditPolicyRules on the kube-apiserver configuration page.

The following example shows two custom rules — one to increase audit coverage for a specific resource, and one to reduce log volume by skipping internal status updates:

# Rules must be defined in YAML list format
- level: RequestResponse
  resources:
  - group: "policy.alibabacloud.com"
    resources: ["policies"]
- level: None
  userGroups: ["system:nodes"]
  verbs: ["update", "patch"]
  resources:
    - group: ""
      resources: ["nodes/status", "pods/status"]

These rules serve two purposes:

Increase auditing: Record all operations on Container Service Policy Templates (policies.policy.alibabacloud.com) at the RequestResponse level, capturing full request and response bodies.
Reduce auditing: Skip recording node and pod status updates — such as kubelet heartbeats — from components in the system:nodes group. This reduces unnecessary log volume.

When configuring custom rules, note the following:

Rule scope: The auditPolicyRules field accepts a list of audit rules (rules), not a complete audit policy (Policy) object.
Rule priority: You cannot modify the log level for requests already defined in the ACK default audit rules. Custom rules apply only to requests not matched by the built-in policy.
Rule validity: ACK validates custom rules against the rules field definition in policies.audit.k8s.io. Only add valid rules as described in the Kubernetes upstream documentation on Auditing.

Important

Incorrect rule formatting can prevent the kube-apiserver from starting properly. Validate your YAML before applying.

For the full audit policy specification, see the Kubernetes upstream documentation on Auditing.

Container exec Auditing

Attackers often use the exec command to enter running containers and launch lateral attacks from within the cluster.

Important

By default, the cluster API server audit logs do not capture commands executed after a user or attacker gains container access via exec. The API server records the exec call itself, but not the commands run inside the container session.

To capture in-container command activity, enable the container internal operation audit feature. This feature records the commands run inside container instances, giving security engineers audit evidence for incident investigation and remediation.

Interpret Authorization Decisions in Audit Logs

Kubernetes audit logs include two annotations that indicate whether an API request was authorized and the reason for that decision:

authorization.k8s.io/decision — indicates whether the request was allowed or denied
authorization.k8s.io/reason — explains why the decision was made

Use these fields to determine why a specific API call was permitted or rejected. For example, if a user reports unexpected access being denied, you can query the audit log for that user's requests and inspect authorization.k8s.io/reason to identify the applicable RBAC rule or policy.

Cluster Event Monitoring

Kubernetes distinguishes between two types of cluster activity:

Audit logs — record API server requests (who called what API, when, and with what result)
Kubernetes events — record state-change notifications from controllers (pod evictions, image pull failures, node conditions, and so on)

Both are useful for security and operational visibility, but they serve different purposes. Use audit logs for API-level traceability; use Kubernetes events for real-time cluster health and anomaly detection.

Node Problem Detector and kube-eventer

ACK maintains Node Problem Detector (NPD), a Kubernetes node diagnostics tool that detects node-level anomalies including Docker Engine hangs, Linux kernel hangs, network egress failures, and file descriptor exhaustion. NPD converts these anomalies into Kubernetes node events. When combined with kube-eventer, this enables end-to-end node event alerting through SLS.

Enable the SLS Kubernetes Event Center

The Kubernetes Event Center in Simple Log Service (SLS) aggregates all Kubernetes events in real time, including suspicious operations such as regular users accessing specified containers via exec. It provides storage, query, analysis, visualization, and alerting capabilities to help platform engineers and SecOps engineers quickly identify risks to cluster stability. For setup instructions, see Event Monitoring.

Ingress Access Log Monitoring

The Alibaba Cloud Ingress component logs all HTTP requests to standard output. ACK integrates this access log data with Simple Log Service (SLS), enabling you to build a monitoring dashboard for Ingress traffic. The Ingress monitoring dashboard displays key metrics — including PV, UV, traffic, response latency, and Top URL statistics — so you can monitor service traffic dynamics in real time and detect malicious access or denial-of-service (DoS) attacks. For setup instructions, see Ingress Dashboard Monitoring.

CoreDNS Log Analysis

ACK deploys CoreDNS as the cluster's DNS server. CoreDNS logs are useful for troubleshooting slow DNS resolution and identifying access to high-risk domain names. The SLS dashboard provides log analysis reports for CoreDNS, helping you identify suspicious domain name requests before they escalate. For setup instructions, see Analyze and Monitor CoreDNS Logs.