在Kubernetes集群中,API Server的审计日志可以帮助集群管理人员记录或追溯不同用户的日常操作,是集群安全运维中重要的环节。本文主要介绍如何在注册集群中使用集群审计功能。

前提条件

已创建注册集群,并将自建Kubernetes集群接入注册集群。具体操作,请参见创建注册集群并接入本地数据中心集群

步骤一:在Master节点上配置审计配置策略文件

依次登录所有Master节点,在审计配置策略文件的路径/etc/kubernetes/audit-policy.yaml,请根据以下内容修改审计配置策略:
apiVersion: audit.k8s.io/v1beta1 # This is required.
kind: Policy
# Don't generate audit events for all requests in RequestReceived stage.
omitStages:
  - "RequestReceived"
rules:
  # The following requests were manually identified as high-volume and low-risk,
  # so drop them.
  - level: None
    users: ["system:kube-proxy"]
    verbs: ["watch"]
    resources:
      - group: "" # core
        resources: ["endpoints", "services"]
  - level: None
    users: ["system:unsecured"]
    namespaces: ["kube-system"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["configmaps"]
  - level: None
    users: ["kubelet"] # legacy kubelet identity
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes"]
  - level: None
    userGroups: ["system:nodes"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes"]
  - level: None
    users:
      - system:kube-controller-manager
      - system:kube-scheduler
      - system:serviceaccount:kube-system:endpoint-controller
    verbs: ["get", "update"]
    namespaces: ["kube-system"]
    resources:
      - group: "" # core
        resources: ["endpoints"]
  - level: None
    users: ["system:apiserver"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["namespaces"]
  # Don't log these read-only URLs.
  - level: None
    nonResourceURLs:
      - /healthz*
      - /version
      - /swagger*
  # Don't log events requests.
  - level: None
    resources:
      - group: "" # core
        resources: ["events"]
  # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data,
  # so only log at the Metadata level.
  - level: Metadata
    resources:
      - group: "" # core
        resources: ["secrets", "configmaps"]
      - group: authentication.k8s.io
        resources: ["tokenreviews"]
  # Get repsonses can be large; skip them.
  - level: Request
    verbs: ["get", "list", "watch"]
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
  # Default level for known APIs
  - level: RequestResponse
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
  # Default level for all other requests.
  - level: Metadata

步骤二:在Master节点上配置Kube API Server文件

依次登录所有Master节点机器,在Kube API Server文件的路径/etc/kubernetes/manifests/kube-apiserver.yaml,请完成以下相关配置:
  • 根据以下示例添加command参数--audit-log-*
    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
  • 根据以下示例添加env参数aliyun_logs_audit-*
    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
        ...
        env:
        - name: aliyun_logs_audit-${cluster_id}
          value: /var/log/kubernetes/kubernetes.audit
        - name: aliyun_logs_audit-${cluster_id}_tags
          value: audit=apiserver
        - name: aliyun_logs_audit-${cluster_id}_product
          value: k8s-audit
        - name: aliyun_logs_audit-${cluster_id}_jsonfile
          value: "true"
        image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/kube-apiserver:v1.20.4-aliyun.1
    注意 需要将{cluster_id}替换为您集群的Cluster ID。关于集群ID的获取,请参见查看基本信息
  • 根据以下示例挂载/etc/kubernetes/audit-policy.yaml到API Server Pod。
    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
        ...
        env:
        - name: aliyun_logs_audit-${cluster_id}
          value: /var/log/kubernetes/kubernetes.audit
        - name: aliyun_logs_audit-${cluster_id}_tags
          value: audit=apiserver
        - name: aliyun_logs_audit-${cluster_id}_product
          value: k8s-audit
        - name: aliyun_logs_audit-${cluster_id}_jsonfile
          value: "true"
        image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/kube-apiserver:v1.20.4-aliyun.1
        ...
        ...
        volumeMounts:
        - mountPath: /var/log/kubernetes
          name: k8s-audit
        - mountPath: /etc/kubernetes/audit-policy.yaml
          name: audit-policy
          readOnly: true
        ...
        ...
      volumes:
      - hostPath:
          path: /var/log/kubernetes
          type: DirectoryOrCreate
        name: k8s-audit
      - hostPath:
          path: /etc/kubernetes/audit-policy.yaml
          type: FileOrCreate
        name: audit-policy
      ...

步骤三:安装logtail-ds日志组件

关于安装logtail-ds日志组件的具体操作,请参见步骤二:安装logtail-ds组件

后续步骤

关于如何使用和查看集群审计功能,请参见使用集群审计功能