Container Service for Kubernetes:Multi-tenancy security

Soft multi-tenancy

Native Kubernetes features such as namespace, roles, role bindings, and network policies provide logical tenant isolation. RBAC prevents tenants from accessing each other's resources. quotas and limit ranges control per-tenant resource consumption, while network policies restrict cross-namespace communication.

These controls do not prevent pods from different tenants from sharing a node. Use nodeSelector, anti-affinity rules, taints, and tolerations to schedule tenant pods onto dedicated nodes. With many tenants, this approach becomes complex and costly.

Soft multi-tenancy with namespaces cannot provide tenants a filtered namespace list because namespaces are cluster-scoped resources. If a tenant can view one namespace, they can view all namespaces in the cluster.

By default, tenants can query CoreDNS for all cluster services. An attacker can exploit this by running dig SRV ..svc.cluster.local from any pod. To restrict DNS record access, use the firewall or policy plugin for CoreDNS. See kubernetes-metadata-multi-tenancy-policy.

• Internal enterprise environments

  All cluster users are from within the organization. Because identities are controlled, security risk is manageable. Each tenant typically aligns with an administrative unit, such as a department or team.

  Cluster administrators typically create namespaces and manage policies. A delegated model grants individuals supervisory permissions over a namespace, allowing CRUD operations on non-policy objects such as deployments, services, pods, and jobs.

  Docker isolation mechanisms are acceptable here. Add Pod Security Policy (PSP) for extra controls. For stricter isolation, also restrict cross-namespace service communication.

• Kubernetes as a Service (KaaS)

  Soft multi-tenancy applies when offering Kubernetes as a Service (KaaS). Applications are hosted in a shared cluster with controllers and CRDs providing PaaS services. Tenants interact directly with the Kubernetes API server, perform CRUD operations on non-policy objects, and can create and manage their own namespaces. Tenants are assumed to run untrusted code.

  Isolate tenants with strict network policies and pod sandboxing. See Sandboxed-Container.

• Software as a Service (SaaS)

  Each tenant is associated with a specific application instance in the cluster. Each instance has its own data and uses a separate access control mechanism, often independent of Kubernetes RBAC.

  Tenants do not interact directly with the Kubernetes API. The SaaS application interacts with the Kubernetes API to create objects for each tenant.

Native Kubernetes configurations

Kubernetes is a single-tenant orchestration platform—one control plane instance is shared among all tenants. Use namespaces and RBAC to logically isolate tenants, and resource quotas and limit ranges to control per-tenant resource consumption. However, a cluster is the only strong security boundary: an attacker with host access can retrieve all secrets, configmaps, and volumes on that host, impersonate the Kubelet, and move laterally. The following configurations mitigate these risks.

• Namespaces

  Namespaces are the foundation of soft multi-tenancy, dividing a cluster into logical layers. Resource quotas, network policies, service accounts, and other resources operate within namespace scope.

• Authentication, authorization, and admission

  ACK cluster authorization involves two steps: RAM authorization and RBAC authorization. RAM authorization controls the cluster management interface, including CRUD permissions such as cluster visibility, scaling, and adding nodes. RBAC authorization controls access to Kubernetes resources with fine-grained, namespace-level permissions. ACK provides predefined role templates, supports binding user-defined cluster roles, and batch user authorization. See Authorization overview.

• Network policies

  By default, all pods in a Kubernetes cluster are allowed to communicate with each other. Use network policies to change this default behavior.

• Network policies restrict pod communication using labels or IP address ranges. For strict tenant isolation, add two rules:

  • A default rule that denies communication between pods.

  • A rule that allows all pods to query the DNS server for name resolution.

• Resource quotas and limit ranges

  Resource quotas cap CPU, memory, and resource counts per cluster or namespace. Limit ranges set minimum, maximum, and default values for each limit.

  Overcommitting resources maximizes utilization, but unrestricted access can cause resource starvation, degrading performance and availability. If a pod's request is too low and actual usage exceeds node capacity, the node experiences CPU or memory pressure, causing pod restarts or evictions.

  Enforce resource quotas on namespaces to require tenants to specify requests and limits. This also mitigates denial-of-service risks by capping per-pod resource consumption.

  In KaaS scenarios, use resource quotas to allocate resources aligned with tenant needs.

• Pod priority and preemption

  Pod priority and preemption enable different QoS levels for customers. Higher-priority pods preempt lower-priority ones when capacity is insufficient. In SaaS environments, this serves customers willing to pay for better QoS.

Mitigation measures

As a security administrator in a multi-tenant environment, your primary goal is preventing attackers from accessing the underlying host. Apply the following controls to reduce this risk:

• Sandboxed-Container

  Sandboxed-Container runs applications in a lightweight virtual machine sandbox with an independent kernel, providing stronger isolation than the standard Docker runtime.

  Sandboxed-Container suits scenarios such as isolating untrusted applications, fault isolation, performance isolation, and multi-user workload isolation. It has minimal performance impact and offers the same experience as Docker containers for logging, monitoring, and elasticity. See Sandboxed-Container.

• Open Policy Agent (OPA) and Gatekeeper

  Open Policy Agent (OPA) provides fine-grained access policy control at the object model level when RBAC namespace-level isolation is insufficient. Gatekeeper is an admission controller that enforces OPA policies during deployment.

  OPA also supports Layer 7 network policies and cross-namespace access control based on labels and annotations, enhancing native Kubernetes network policies.

• Kyverno

  Kyverno is a Kubernetes policy engine that validates, mutates, and generates resource policies. It supports Kustomize-style validation and can clone resources across namespaces based on flexible triggers.

  Use Kyverno to isolate namespaces, implement pod security best practices, and generate default configurations such as network policies. See the policy repository.

Hard multi-tenancy

Future directions

The Kubernetes community recognizes the limitations of soft multi-tenancy and the challenges of hard multi-tenancy. The Multi-Tenancy Special Interest Group (SIG) is addressing these challenges through several incubation projects:

• The Virtual Cluster proposal creates a separate control plane instance (API server, controller manager, scheduler) for each tenant—also known as "Kubernetes on Kubernetes".

• The Hierarchical Namespace Controller (HNC) proposal enables parent-child namespace relationships through policy object inheritance and tenant-managed subnamespaces.

• The Multi-Tenancy Benchmarks proposal provides namespace isolation guidelines and a CLI tool, Kubectl-mtb, for verifying compliance.