Container Service for Kubernetes:Identity verification and access control

Authentication and authorization in ACK clusters

Kubernetes uses X.509 certifications, bearer tokens, authentication proxies, or the OpenID Connect (OIDC) protocol to authenticate API requests. For more information about the default methods that are used to access Container Service for Kubernetes (ACK) clusters, see client certificates and service account tokens.

You can obtain a kubeconfig file that contains a client certificate in the ACK console or by calling the ACK API. For more information, see Query the kubeconfig file of a cluster.

The authorization mechanism of ACK consists of Resource Access Management (RAM) authorization and role-based access control (RBAC) authorization. If you want to modify the configuration of a cluster, scale a cluster, add nodes to a cluster, or access a cluster as a RAM user, you must perform RAM authorization. RAM authorization allows you to attach system policies or custom policies to a RAM user. If you want to manage Kubernetes resources in a specified cluster by using a RAM user, you must go to the Authorizations page of the the ACK console console and grant the RAM user resource-level permissions, such as the permissions to view information about pods and nodes. For more information, see Authorization overview.

• Use a temporary kubeconfig file for authentication when you access the Kubernetes API server of a cluster

  The default kubeconfig file of an ACK cluster contains a certificate that is valid for three years. If the kubeconfig file is exposed, attackers can use it to gain access to the Kubernetes API server of the cluster. A default service account token is a static credential that is permanently valid. If a default service account token is exposed, attackers can exploit the token to gain the same permissions as the relevant service account until it is deleted.

  If a delivered kubeconfig file may be exposed, we recommend that you revoke the kubeconfig file at the earliest opportunity. For more information, see Revoke a KubeConfig credential. For more information about the kubeconfig file of an ACK cluster, see Generate a temporary kubeconfig file.

• Grant access to Alibaba Cloud resources in compliance with the principle of least privilege

  When you authorize a user to access ACK clusters, do not grant the permissions to access resources of other Alibaba Cloud services. You can use RAM authorization and RBAC authorization to grant a user the permissions to access ACK clusters.

• Create RoleBindings and ClusterRoleBindings in compliance with the principle of least privilege

  You must create RoleBindings and ClusterRoleBindings in compliance with the principle of least privilege. When you define a Role or ClusterRole, specify the required actions instead of ["*"] in the Verbs field. If you are not sure about the permissions to grant, you can use tools, such as audit2rbac, to help create roles and role bindings. audit2rbac can generate roles and role bindings that cover all API operations that are recorded in the Kubernetes audit log.

• Limit access to the endpoint of an ACK cluster

  By default, the endpoint of an ACK cluster can be accessed only from within the cluster and the virtual private cloud (VPC) in which the cluster is deployed. For more information about how to enable Internet access for Services and modify the control access to the endpoint, see Add annotations to the YAML file of a Service to configure CLB instances.

• Audit the permissions of users on a regular basis

  The permissions that a user have on a cluster may change by time. The security administrators and O&M engineers of a cluster must regularly check the RAM permissions and RBAC permissions of each RAM user and make sure that only the required permissions are granted. The security administrators and O&M engineers can use open source tools to check the RBAC permissions that are granted to a service account, user, or group, and remove permissions that are not required. For more information, see kubectl-who-can and rbac-lookup.

Pod authentication

In Kubernetes clusters, some applications and programs require the permissions to call the API operations of Kubernetes. In ACK clusters, the cloud controller manager (CCM) requires the permissions to add, delete, modify, and query nodes.

• Kubernetes Service Accounts

  A service account can be used to assign an RBAC role to a pod. Kubernetes creates a default service account for each namespace in the cluster. When you create a pod, if you do not specify a service account, the pod is automatically assigned the default service account in the same namespace, and a Secret that stores a JSON web token (JWT) is mounted to the /var/run/secrets/kubernetes.io/serviceaccount directory of the pod as a volume. If you decode the token, the following metadata is returned:

  {
    "iss": "kubernetes/serviceaccount",
    "kubernetes.io/serviceaccount/namespace": "default",
    "kubernetes.io/serviceaccount/secret.name": "default-token-vpc2x",
    "kubernetes.io/serviceaccount/service-account.name": "default",
    "kubernetes.io/serviceaccount/service-account.uid": "1d059c50-0818-4b15-905d-bbf05e1d****",
    "sub": "system:serviceaccount:default:default"
  }

  The default service account has the following permissions on the Kubernetes API:

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    annotations:
      rbac.authorization.kubernetes.io/autoupdate: "true"
    labels:
      kubernetes.io/bootstrapping: rbac-defaults
    name: system:discovery
  rules:
  - nonResourceURLs:
    - /api
    - /api/*
    - /apis
    - /apis/*
    - /healthz
    - /openapi
    - /openapi/*
    - /version
    - /version/
    verbs:
    - get

  The role can be used to grant unauthenticated and authenticated users the permissions to read API information. This role is deemed safe to be publicly accessible.

  When an application or program that runs in a pod calls the Kubernetes API, the pod must be assigned a service account that has the required permissions. The permissions defined in the Role or ClusterRole that is associated with the service account must be scoped to the API resources that the pod needs to access and the methods that the pod needs to use. This complies with the principle of least privilege. To use non-default service accounts, set the spec.serviceAccountName field in the pod configuration to the name of the service account that you want to use. For more information about how to create a service account, see ServiceAccount permissions.

• Enable service account token volume projection

  ACK provides service account token volume projection to reduce security risks when pods use service accounts to access the Kubernetes API server. This feature enables kubelet to request and store the token on behalf of a pod. This feature also allows you to configure token properties, such as the audience and validity period. If a token has existed for more than 24 hours or only 20% or fewer of its validity period remains, kubelet automatically rotates the token. For more information, see Enable service account token volume projection.

• Limit access to instance metadata

  Elastic Compute Service (ECS) instance metadata contains the information about ECS instances in Alibaba Cloud. You can view the metadata of running instances and configure or manage the instances based on their metadata. Instance metadata contains sensitive information, such as the used cloud resources and user data. If the metadata of an instance is exposed to pods that are deployed on the instance, the information may be exploited by attackers in multi-tenant scenarios. For pods that do not need to send outbound requests, you can use network policies to forbid the pods to access meta-server. The following code block shows a network policy that denies all outbound requests from pods with the labels that are specified in the podSelector. This way, the matching pods cannot access meta-server.

  apiVersion: networking.k8s.io/v1
  kind: NetworkPolicy
  metadata:
    name: allow-metadata-access
    namespace: example
  spec:
    podSelector:
      matchLabels:
        app: myapp
    policyTypes:
    - Egress
    egress: []

  For more information, see Use network policies.

• Disable automatic mounting of service account tokens for pods

  You can use one of the following methods to disable automatic mounting of service account tokens for pods:

  Method 1: Set automountServiceAccountToken in PodSpec to false by using the following YAML template:

  apiVersion: v1
  kind: Pod
  metadata:
    name: my-pod
  spec:
    serviceAccountName: build-robot
    automountServiceAccountToken: false
    ...

  Method 2: Run the following command to set the automountServiceAccountToken parameter of the default service account in a namespace to false by using a patch:

  kubectl patch serviceaccount default -p $'automountServiceAccountToken: false'

• Assign a separate service account to each application

  To enable fine-grained application permission management, you must assign a separate service account to each application. For more information, see Configure service accounts for pods.

• Run applications as non-root users

  By default, a container runs as a root user. This allows processes in a container to perform read and write operations on data in the container, which violates the best practices for security. To run a container as a non-root user, add the spec.securityContext.runAsUser parameter to PodSpec. Set runAsUser to a proper value based on your requirements.

  The following template specifies that all processes in the pod run with the user ID that is specified in the runAsUser field:

  apiVersion: v1
  kind: Pod
  metadata:
    name: security-test
  spec:
    securityContext:
      runAsUser: 1000
      runAsGroup: 3000
    containers:
    - name: sec-test
      image: busybox
      command: [ "sh", "-c", "sleep 1h" ]

  This way, processes in the container cannot access files that require root privileges. If you add fsgroup=65534 [Nobody] to the securityContext parameter, processes in the container are allowed to access these files.

  spec:
    securityContext:
      fsGroup: 65534

• Grant permissions on Alibaba Cloud resources in compliance with the principle of least privilege

  Do not grant unnecessary permissions to RAM users or RAM roles. You must configure RAM policies in compliance with the principle of least privilege and must not configure settings that may grant unnecessary permissions, such as specifying ["*"] in the policy content.