Security Center:Register a self-managed K8s cluster

Configure traffic forwarding for hybrid cloud deployments

If your cluster is deployed on a hybrid cloud and is not accessible over the Internet, configure port forwarding on an Elastic Compute Service (ECS) instance to route traffic to the on-premises server running the cluster's API server. The cluster cannot communicate with Security Center if this forwarding is not configured.

The examples below forward traffic from Port A on ECS instance 10.0.XX.XX to Port B on the on-premises server 192.168.XX.XX.

CentOS 7 — firewall-cmd

firewall-cmd --permanent --add-forward-port=port=<Port A>:proto=tcp:toaddr=<192.168.XX.XX>:toport=<Port B>

CentOS 7 — iptables

# Enable IP forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward

# Add the forwarding rule
iptables -t nat -A PREROUTING -p tcp --dport <Port A> -j DNAT --to-destination <192.168.XX.XX>:<Port B>

Windows — netsh

netsh interface portproxy add v4tov4 listenport=<Port A> listenaddress=* connectaddress=<192.168.XX.XX> connectport=<Port B> protocol=tcp

Add Security Center IP addresses to the whitelist

If access control policies are in place on your cluster, add the Security Center IP addresses for your region to the whitelist. The cluster cannot communicate with Security Center if these addresses are blocked.

Add master nodes and tainted nodes

The generated command does not schedule DaemonSet pods on master nodes or tainted nodes by default. To include these nodes, add tolerations to the pod template in the YAML file before running kubectl apply.

For master nodes — add the following under spec > template > spec:

spec:
  template:
    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule

This toleration allows DaemonSet pods to be scheduled on nodes with the node-role.kubernetes.io/master:NoSchedule taint, which adds the master nodes to Security Center as part of the cluster.

For other tainted nodes — apply the same toleration pattern, matching the taint key and effect for each node type.

Step 1. Install Logtail

Follow the Install Logtail instructions in Install Logtail components in a self-managed Kubernetes cluster.

Step 2. Enable cluster auditing

The following steps are based on Enable cluster auditing for registered clusters.

1. Create an ACK One registered cluster and add the self-managed Kubernetes cluster to it. See Create ACK One registered clusters.

2. On each master node, update /etc/kubernetes/audit-policy.yaml with the following policy:

   For clusters running Kubernetes earlier than 1.24, set apiVersion to audit.k8s.io/v1beta1. For 1.24 and later, use audit.k8s.io/v1. See (Discontinued) Kubernetes 1.24.

   apiVersion: audit.k8s.io/v1beta1
   kind: Policy
   # Don't generate audit events for all requests in RequestReceived stage.
   omitStages:
     - "RequestReceived"
   rules:
     # The following requests were manually identified as high-volume and low-risk,
     # so drop them.
     - level: None
       users: ["system:kube-proxy"]
       verbs: ["watch"]
       resources:
         - group: "" # core
           resources: ["endpoints", "services"]
     - level: None
       users: ["system:unsecured"]
       namespaces: ["kube-system"]
       verbs: ["get"]
       resources:
         - group: "" # core
           resources: ["configmaps"]
     - level: None
       users: ["kubelet"] # legacy kubelet identity
       verbs: ["get"]
       resources:
         - group: "" # core
           resources: ["nodes"]
     - level: None
       userGroups: ["system:nodes"]
       verbs: ["get"]
       resources:
         - group: "" # core
           resources: ["nodes"]
     - level: None
       users:
         - system:kube-controller-manager
         - system:kube-scheduler
         - system:serviceaccount:kube-system:endpoint-controller
       verbs: ["get", "update"]
       namespaces: ["kube-system"]
       resources:
         - group: "" # core
           resources: ["endpoints"]
     - level: None
       users: ["system:apiserver"]
       verbs: ["get"]
       resources:
         - group: "" # core
           resources: ["namespaces"]
     # Don't log these read-only URLs.
     - level: None
       nonResourceURLs:
         - /healthz*
         - /version
         - /swagger*
     # Don't log events requests.
     - level: None
       resources:
         - group: "" # core
           resources: ["events"]
     # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data,
     # so only log at the Metadata level.
     - level: Metadata
       resources:
         - group: "" # core
           resources: ["secrets", "configmaps"]
         - group: authentication.k8s.io
           resources: ["tokenreviews"]
     # Get responses can be large; skip them.
     - level: Request
       verbs: ["get", "list", "watch"]
       resources:
         - group: "" # core
         - group: "admissionregistration.k8s.io"
         - group: "apps"
         - group: "authentication.k8s.io"
         - group: "authorization.k8s.io"
         - group: "autoscaling"
         - group: "batch"
         - group: "certificates.k8s.io"
         - group: "extensions"
         - group: "networking.k8s.io"
         - group: "policy"
         - group: "rbac.authorization.k8s.io"
         - group: "settings.k8s.io"
         - group: "storage.k8s.io"
     # Default level for known APIs
     - level: RequestResponse
       resources:
         - group: "" # core
         - group: "admissionregistration.k8s.io"
         - group: "apps"
         - group: "authentication.k8s.io"
         - group: "authorization.k8s.io"
         - group: "autoscaling"
         - group: "batch"
         - group: "certificates.k8s.io"
         - group: "extensions"
         - group: "networking.k8s.io"
         - group: "policy"
         - group: "rbac.authorization.k8s.io"
         - group: "settings.k8s.io"
         - group: "storage.k8s.io"
     # Default level for all other requests.
     - level: Metadata

3. On each master node, update /etc/kubernetes/manifests/kube-apiserver.yaml:

   1. Add the following --audit-log-* flags to the command section:

      spec:
        containers:
        - command:
          - kube-apiserver
          - --audit-log-maxbackup=10
          - --audit-log-maxsize=100
          - --audit-log-path=/var/log/kubernetes/kubernetes.audit
          - --audit-log-maxage=30
          - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
          ...

   2. Add the following environment variables to the env section. Replace {cluster_id} with your cluster ID — find it on the Cluster tab in the Security Center console.

                  env:
                  - name: aliyun_logs_audit-${cluster_id}
                    value: /var/log/kubernetes/kubernetes.audit
                  - name: aliyun_logs_audit-${cluster_id}_tags
                    value: audit=apiserver
                  - name: aliyun_logs_audit-${cluster_id}_product
                    value: k8s-audit
                  - name: aliyun_logs_audit-${cluster_id}_jsonfile
                    value: "true"

   3. Mount the audit log directory and policy file into the kube-apiserver pods:

                  volumeMounts:
                  - mountPath: /var/log/kubernetes
                    name: k8s-audit
                  - mountPath: /etc/kubernetes/audit-policy.yaml
                    name: audit-policy
                    readOnly: true
                volumes:
                - hostPath:
                    path: /var/log/kubernetes
                    type: DirectoryOrCreate
                  name: k8s-audit
                - hostPath:
                    path: /etc/kubernetes/audit-policy.yaml
                    type: FileOrCreate
                  name: audit-policy

Step 3. Verify log collection

1. Log on to the Simple Log Service console.Log on to the Security Center console.Log on to the Security Center console.

2. Click the name of the project created during Logtail installation.

3. Confirm that audit logs are flowing into the expected Logstore.

Step 4. Enable threat detection in Security Center

1. Log on to the Security Center consoleSecurity Center consoleSecurity Center console. In the top navigation bar, select China or Outside China.

2. In the left-side navigation pane, choose Assets > Container.

3. On the Cluster tab, click Self-built cluster access.

4. Find the cluster and click Edit in the Actions column.

5. On the Enable Log Collection tab, select Enable Kubernetes Log Reporting to Detect Threats, configure the following parameters, and click Save.

   Parameter	Description	Example
   Region of Log Audit Service	The region where you want to store logs	—
   Project of Log Audit Service	The Simple Log Service project created during Logtail installation	k8s-log-custom-sd89ehdq
   Logstore of Log Audit Service	The Logstore automatically created during Logtail installation	audit-027b007a7dd11967a9f7e2449d8dc497