Pod security policy is a commonly used method to verify the security of pod configurations before pods are deployed. This ensures that applications are running in secure pods. This topic describes how to create, manage, and associate pod security policies, and use pod security policies in Container Service for Kubernetes (ACK) clusters.

Usage notes

This topic describes how to manage pod security policies on the PSPs (earlier version) page in the ACK console. The PSPs (earlier version) page will soon be deprecated. For more information about how to manage pod security policies on the PSPs (new version) page, see Configure and enforce ACK pod security policies.

Background information

Kubernetes allows you to configure pod context settings to ensure that your applications run in secure pods. Security engines such as AppArmor and SELinux can verify the security of pod configurations before pods are deployed. Kubernetes allows you to verify the security of pods by using pod security policies in the same way as security engines.
  • Forced security verification: A pod security policy is a cluster-level resource of Kubernetes. Pod security policy control is implemented as an admission controller of the API server to verify the security of pod configurations before pods are deployed. If the pod configuration fails to meet the conditions that are defined in a specified pod security policy, the Kubernetes API server rejects the request to create the pod.
  • Role-based access control (RBAC) bindings: To create pod security policies, you must enable the admission controller for pod security policies. Then, you must create RBAC bindings to authorize service accounts to use the pod security policies that you created.
  • Pod security policy setting: By default, the pod security policy feature is automatically enabled for managed and dedicated Kubernetes clusters.
  • Principle of least privilege: To ensure the compatibility of ACK clusters, a default pod security policy named ack.privileged is automatically created and associated with each legitimate account. The policy enforces the principle of least privilege. For example, the default pod security policy may forbid specific users to deploy privileged pods, allow them to use only read-only root file systems, and allow them to mount host paths only within the specified range.

Prerequisites

  • A managed Kubernetes cluster or a dedicated Kubernetes cluster is created, and the cluster version is 1.14.8-aliyun.1 or later. For more information, see Create an ACK managed cluster or Create an ACK dedicated cluster.
  • Log on to the Container Service for Kubernetes (ACK) console console with your Alibaba Cloud account. You can also log on to the console as a RAM user after you grant administrator permissions to the RAM user. For more information, see Assign RBAC roles to RAM users.
  • If you want to manage pod security policies as a RAM user, make sure that the RAM user has the following permissions:
    • cs:DescribeClusterPSPState: queries the status of the pod security policy feature for the cluster.
    • cs:DescribeServiceAccountBindingPSP: queries the pod security policy that is associated with a specified service account.
    • cs:UpdateClusterPodSecurityPolicy: updates a pod security policy.
    • cs:CreatePSPBinding: associates a pod security policy with a specified service account.
    • cs:UnbindingServiceAccountPSP: disassociates a pod security policy from a specified service account.

    For more information about custom RAM policies, see Create a custom RAM policy.

Create a pod security policy

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the cluster page, choose Security > PSPs.
  5. On the PSPs page, click Create. In the Select PSP Template dialog box, click Use for the policy template that you want to use.
    In addition to the default pod security policy ack.privileged which enables all pods to run in privileged mode, ACK also provides the following policy templates:
    • privileged-restricted: limits the privileged mode. For example, this policy template can limit the host networks, ports, and namespaces used by privileged pods.
    • privileged-root-volumes-restricted: limits the privileged mode. For example, this policy template can limit the host networks, ports, namespaces, and volume types used by privileged pods and also limit escalations to root privileges.
  6. In the Create YAML panel, modify the parameters of the policy template based on the desired security level. For more information, see Pod security policies.
    Parameter Description
    privileged Specify whether the containers in the specified pod can run in privileged mode.
    hostPID,hostIPC Specify whether the containers can use the host namespace.
    hostNetwork,hostPorts Specify whether the containers can use the host network and ports.
    volumes Specify a list of volume types that are allowed.
    allowedHostPaths Specify a list of host paths that can be used by hostPath volumes. The allowed host paths start with a prefix specified by the pathPrefix field.
    allowedFlexVolumes Specify a list of FlexVolume drivers that are allowed.
    fsGroup Specify the ID of an FSGroup that owns the volumes of the pod.
    readOnlyRootFilesystem Require the use of a read-only root file system.
    runAsUser,runAsGroup,supplementalGroups Specify the IDs of the user, primary user group, and supplemental user group of the containers.
    allowPrivilegeEscalation,defaultAllowPrivilegeEscalation Limit escalations to root privileges (allowPrivilegeEscalation=true). These options limit the use of setuid binaries and prevent applications from enabling extra capabilities.
    defaultAddCapabilities,requiredDropCapabilities,allowedCapabilities Specify the Linux capabilities that are allowed.
    seLinux Specify the SELinux context that is allowed.
    allowedProcMountTypes Specify a list of ProcMountTypes that are allowed.
    annotations Specify the AppArmor or seccomp profile that is allowed.
    forbiddenSysctls,allowedUnsafeSysctls Specify the sysctl profile that is allowed.
  7. Click OK.
    After the pod security policy is created, you can find it on the PSPs tab.
    Note
    • To modify a pod security policy on the PSPs tab, click YAML in the Actions column of the pod security policy.
    • To delete a pod security policy on the PSPs tab, click Delete in the Actions column of the pod security policy.

Associate a pod security policy with service accounts

After you enable the pod security policy feature for a cluster, you must authorize service accounts that are disassociated from the default pod security policy to use other pod security policies. Otherwise, the following error occurs when the system attempts to deploy pods with these service accounts: 
Error from server (Forbidden): error when creating xxx: pods "xxx" is forbidden: unable to validate against any pod security policy: []
You can associate a pod security policy with a specified service account or all service accounts in a specified namespace.
  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the cluster page, choose Security > PSPs.
  5. Click the Association Rules tab and click Add Association Rule.
  6. In the Add Association Rule panel, specify Namespace, Account, and PSP.
  7. Click OK.

Disassociate the default pod security policy

To make a custom pod security policy that you have associated with service accounts take effect, you must disassociate the default pod security policy.
  • When the system disassociates the default pod security policy, the system authorizes all service accounts in the kube-system namespace and the system:nodes user group to use the default pod security policy. This ensures that the system components and containers run as normal.
  • All service accounts in a namespace are authorized to use the default pod security policy if no custom pod security policy is associated with the namespace. This ensures that the containers in the namespace run as normal.
Notice
  • For a namespace in which service accounts are associated with a custom pod security policy:

    To make the custom pod security policy take effect, after you disassociate the default pod security policy, the services accounts are not authorized to use the default pod security policy. Therefore, before you disassociate the default pod security policy, make sure that the service accounts in the namespace are associated with the desired custom pod security policy.

  • For a namespace that is newly created:

    The system does not automatically associate the default pod security policy with service accounts in the newly created namespace. Therefore, after you create a namespace, you must create a custom pod security policy and associate it with service accounts in the namespace at the earliest opportunity.

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the cluster page, choose Security > PSPs.
  5. In the PSP Management Guide section, click Disassociate Default PSP.
  6. In the Disassociate message, click OK.

Enable the admission controller for pod security policies

By default, ACK automatically enables the admission controller for newly created clusters. To make custom pod security policies take effect within a cluster, enable the admission controller for the API server of the cluster.

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the cluster page, choose Security > PSPs.
  5. Enable the admission controller.
    • In the upper-right corner of the PSPs page, Disabled indicates that the admission controller is disabled. To enable the admission controller, turn on PSP Status and click OK in the Change PSP State message.
    • In the upper-right corner of the PSPs page, Enabled indicates that the admission controller is enabled.
    Notice After you enable the admission controller, the API server is restarted. To minimize the impacts on your workloads, we recommend that you change the status of the admission controller during off-peak hours.

Default pod security policy

By default, pod security policy control is enabled for standard managed Kubernetes clusters of version 1.16.6 and standard dedicated Kubernetes clusters of version 1.16.6. A default pod security policy named ack.privileged is automatically created for these clusters. The default pod security policy accepts all types of pods. This provides the same effect as when pod security policy control is disabled for the cluster. 

kubectl get psp ack.privileged
NAME             PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
ack.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *
kubectl describe psp ack.privileged
Name:  ack.privileged

Settings:
  Allow Privileged:                       true
  Allow Privilege Escalation:             true
  Default Add Capabilities:               <none>
  Required Drop Capabilities:             <none>
  Allowed Capabilities:                   *
  Allowed Volume Types:                   *
  Allow Host Network:                     true
  Allow Host Ports:                       0-65535
  Allow Host PID:                         true
  Allow Host IPC:                         true
  Read Only Root Filesystem:              false
  SELinux Context Strategy: RunAsAny
    User:                                 <none>
    Role:                                 <none>
    Type:                                 <none>
    Level:                                <none>
  Run As User Strategy: RunAsAny
    Ranges:                               <none>
  FSGroup Strategy: RunAsAny
    Ranges:                               <none>
  Supplemental Groups Strategy: RunAsAny
    Ranges:                               <none>
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: ack.privileged
  annotations:
    kubernetes.io/description: 'privileged allows full unrestricted access to
      pod features, as if the PodSecurityPolicy controller was not enabled.'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ack:podsecuritypolicy:privileged
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
rules:
- apiGroups:
  - policy
  resourceNames:
  - ack.privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated

Delete the ClusterRoleBinding resource that is related to the default pod security policy

Warning Before you delete the ClusterRoleBinding resource, you must create a custom pod security policy and a related RBAC binding. Otherwise, all users, controllers, and service accounts are unable to create or update pods after you delete the ClusterRoleBinding resource.
After you create a custom pod security policy and a related RBAC binding, you can delete the ClusterRoleBinding resource of the default pod security policy ack.privileged to enable the custom pod security policy.
Notice Do not delete or modify the default pod security policy ack.privileged or the ClusterRoleBinding resource ack:podsecuritypolicy:privileged. These resources are required to ensure that the ACK cluster can run as normal. 
cat <<EOF | kubectl delete -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated
EOF

Configure or restore the default pod security policy

After you enable the custom pod security policy, you can run the following code to configure or restore the default pod security policy. 

cat <<EOF | kubectl apply -f -
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: ack.privileged
  annotations:
    kubernetes.io/description: 'privileged allows full unrestricted access to
      pod features, as if the PodSecurityPolicy controller was not enabled.'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ack:podsecuritypolicy:privileged
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
rules:
- apiGroups:
  - policy
  resourceNames:
  - ack.privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated
EOF

Verify the pod security policy

The following example shows how a pod security policy is created, associated with service accounts, and then enforced.

  1. Use kubectl to connect to the ACK cluster. For more information, see Connect to ACK clusters by using kubectl.
  2. Create custom pod security policies based on the privileged-restricted and privileged-root-volumes-restricted templates. For more information, see Create a pod security policy.
  3. Run the following commands to create two namespaces, create service accounts in the namespaces, and then create RBAC bindings for these service accounts. 
    kubectl create ns ack-all-restrictive
kubectl create ns ack-restrictive

kubectl create sa ack-dev -n ack-restrictive
kubectl create sa ack-tester -n ack-all-restrictive

kubectl -n ack-restrictive create rolebinding ack-dev-editor \
           --clusterrole=edit \
           --serviceaccount=ack-restrictive:ack-dev
kubectl -n ack-all-restrictive create rolebinding ack-tester-editor \
           --clusterrole=edit \
           --serviceaccount=ack-all-restrictive:ack-tester
  4. On the Association Rules tab, associate the custom pod security policies with the service accounts.
    1. Click Add Association Rule. In the Add Association Rule panel, set Namespace to ack-restrictive, Account to ack-dev, and PSP to privileged-restricted. Then, click OK.
    2. Click Add Association Rule. In the Add Association Rule panel, set Namespace to ack-restrictive, Account to ack-dev, and PSP to privileged-root-volumes-restricted. Then, click OK.
  5. Run the following commands to generate aliases for the service account ack-dev in the namespace ack-restrictive and the service account ack-tester in the namespace ack-all-restrictive: 
    alias kubectl-dev='kubectl --as=system:serviceaccount:ack-restrictive:ack-dev -n ack-restrictive'
alias kubectl-tester='kubectl --as=system:serviceaccount:ack-all-restrictive:ack-tester -n ack-all-restrictive'

kubectl-dev auth can-i use  podsecuritypolicy/privileged-restricted
Warning: resource 'podsecuritypolicies' is not namespace scoped in group 'policy'
yes
  6. Run the following command to create a privileged pod that uses the host network: 
    kubectl-dev apply -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: privileged
spec:
  hostNetwork: true
  containers:
    - name: busybox
      image: busybox
      command: [ "sh", "-c", "sleep 1h" ]
EOF
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "privileged", Namespace: "ack-restrictive"
Object: &{map["apiVersion":"v1" "kind":"Pod" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "name":"privileged" "namespace":"ack-restrictive"] "spec":map["containers":[map["command":["sh" "-c" "sleep 1h"] "image":"busybox" "name":"busybox"]] "hostNetwork":%!q(bool=true)]]}
from server for: "STDIN": pods "privileged" is forbidden: User "system:serviceaccount:ack-restrictive:ack-dev" cannot get resource "pods" in API group "" in the namespace "ack-restrictive"
    The system fails to create the privileged pod and returns an error.
  7. Run the following command to create an unprivileged pod: 
    kubectl-dev apply -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: non-privileged
spec:
  containers:
    - name: busybox
      image: busybox
      command: [ "sh", "-c", "sleep 1h" ]
EOF
pod/non-privileged created
    The output indicates that the unprivileged pod is created.
  8. Run the following commands to create a pod that has root privileges and a pod that does not have root privileges: 
    kubectl-tester apply -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: root-pod
spec:
  containers:
    - name: busybox
      image: busybox
      command: [ "sh", "-c", "sleep 1h" ]
EOF

kubectl-tester apply -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: non-root-pod
spec:
  containers:
    - name: busybox
      image: busybox
      command: [ "sh", "-c", "sleep 1h" ]
      securityContext:
        runAsUser: 1000
        runAsGroup: 3000
EOF
  9. Run the following command to query the status of the pods:
    The pod that has root privileges cannot be created. The associated custom pod security policy forbids the service account ack-tester to create pods that have root privileges. 
    kubectl-tester get pod

    Expected output:

    NAME           READY   STATUS                       RESTARTS   AGE
non-root-pod   1/1     Running                      0          115s
root-pod       0/1     CreateContainerConfigError   0          24s
  10. Run the following command to query the error details. 
    kubectl describe pod root-pod

    Expected output:

    Warning  Failed     28s (x5 over 2m1s)   kubelet, cn-shenzhen.192.168.1.52  Error: container has runAsNonRoot and image will run as root