Kubernetes is a single-tenant orchestration system: one control plane is shared across all tenants in a cluster. This means that an attacker who gains access to a host in the cluster can retrieve all Secrets, ConfigMaps, and volumes mounted to that host, and can exploit the kubelet to manipulate the attributes of the node or move laterally within the cluster. Kubernetes-native isolation mechanisms reduce this risk but do not eliminate it entirely.
This document explains how to choose between soft and hard multi-tenancy based on your security requirements, and how to implement isolation controls in a shared ACK cluster.
Multi-tenancy models
| Model | Trust level | Isolation mechanism | Typical use cases |
|---|---|---|---|
| Soft multi-tenancy | Trusted tenants | Namespaces, role-based access control (RBAC), network policies, quotas | Enterprise departments, internal teams |
| Hard multi-tenancy | Untrusted tenants | Separate cluster per tenant | Regulated industries, SaaS providers, external customers |
Best practice: For most enterprise workloads, soft multi-tenancy is sufficient and significantly reduces cost and operational overhead. Use hard multi-tenancy only when regulatory requirements or untrusted-tenant scenarios demand a strong security perimeter.
Use cases
The right isolation model depends on who your tenants are and what level of trust exists between them.
Enterprise setting
When all tenants belong to the same organization, soft multi-tenancy is sufficient. Each tenant maps to an administrative division such as a department or team. Cluster administrators create namespaces and manage policies.
Using a delegated administration model, specific tenants can be granted permissions within their namespaces to perform create, read, update, and delete (CRUD) operations on non-policy objects such as Deployments, Services, pods, and Jobs.
Best practice: Use Docker's built-in isolation for this scenario. If stricter isolation is required, apply network policies to restrict cross-namespace communication.
Kubernetes as a Service (KaaS)
In KaaS environments, your applications run in a shared cluster that includes controllers and CustomResourceDefinitions (CRDs) providing Platform as a Service (PaaS) capabilities. Tenants interact with the Kubernetes API server and can perform CRUD operations on non-policy objects. Tenants can also create and manage their own namespaces.
In this environment, tenants are considered untrusted. Apply network policies and pod sandboxing. See Sandboxed-Container for details.
Software as a Service (SaaS)
In SaaS environments, each tenant is associated with a specific application instance in the cluster. Tenants do not interact with the Kubernetes API server directly — the SaaS application manages all Kubernetes objects on their behalf. Each instance uses its own access control policies separate from Kubernetes RBAC.
Soft multi-tenancy
Use the following Kubernetes-native resources to implement soft multi-tenancy: namespaces, roles, role bindings, and network policies. These provide logical isolation among tenants.
Soft multi-tenancy addresses two isolation planes:
Control plane isolation: RBAC prevents tenants from accessing or modifying other tenants' Kubernetes resources.
Data plane isolation: network policies and pod scheduling constraints limit how workloads interact at the network and compute layer.
Control plane isolation
Namespaces
Namespaces are the foundation of soft multi-tenancy. Use them to divide a cluster into logical partitions. Quotas, network policies, service accounts, and other objects required for multi-tenancy must be scoped to a namespace.
Namespaces are globally scoped. A tenant with access to any namespace in a cluster can list all namespaces in the cluster. Filtered namespace visibility is not supported natively.
Authorization
ACK clusters use two authorization layers:
Resource Access Management (RAM) authorization: controls cluster-level access, including cluster visibility, scaling, and node management.
RBAC authorization: controls access to Kubernetes resources within a namespace. ACK provides predefined role templates and supports custom cluster roles and multi-user permission grants. See Authorization overview for details.
Data plane isolation
Network policies
By default, all pods in a Kubernetes cluster can communicate with each other. Use network policies to restrict this.
For multi-tenant environments, add the following rules:
A default deny rule that blocks all pod-to-pod communication.
An allow rule that permits pods to send DNS queries to the DNS server.
Network policies limit communication based on labels or CIDR blocks.
Quotas and limit ranges
Quotas and limit ranges prevent any single tenant from exhausting cluster resources.
| Resource control | Description |
|---|---|
| Quotas | Cap the total CPU and memory resources available to a cluster or namespace |
| Limit ranges | Set minimum, maximum, and default resource values per pod or container |
Without quotas, a tenant can exhaust CPU or memory on a node, causing other pods to be evicted. Add quotas to every namespace in a multi-tenant environment. This forces tenants to specify resource requests and limits when scheduling pods, reducing the risk of service unavailability.
In KaaS scenarios, use quotas to allocate cluster resources to each tenant's namespace.
Pod priority and preemption
Use pod priority and preemption to provide differentiated quality of service (QoS) levels. Pods with higher priority values preempt lower-priority pods when resources are insufficient. This is useful in SaaS environments where premium customers require guaranteed resource availability.
Soft multi-tenancy limitations
| Limitation | Details | Mitigation |
|---|---|---|
| Shared nodes | RBAC, quotas, and network policies cannot prevent pods from different tenants from sharing a node | Use node selectors, anti-affinity rules, taints, and tolerations to schedule tenant pods to dedicated sole-tenant nodes. This increases cost and operational complexity at scale. |
| DNS information disclosure | By default, tenants can query CoreDNS for all services in the cluster by running dig SRV ..svc.cluster.local from a pod | Use a firewall or apply CoreDNS policy plugins. See kubernetes-metadata-multi-tenancy-policy for details. |
Mitigation methods
The primary concern in multi-tenant environments is preventing a compromised pod from accessing the underlying host. The following tools provide additional isolation beyond Kubernetes-native controls.
Sandboxed-Container
Sandboxed-Container is an alternative container runtime that runs applications in a sandboxed lightweight VM with a dedicated kernel. This improves resource isolation and security compared to the standard Docker runtime.
Sandboxed-Container is suitable for:
Untrusted application isolation
Fault isolation
Performance isolation
Load isolation across multiple users
Sandboxed-Container has minimal impact on application performance and provides the same operational experience as Docker for logging, monitoring, and elastic scaling. See Overview of Sandboxed-Container for details.
Open Policy Agent (OPA) and Gatekeeper
Open Policy Agent (OPA) is a policy engine that supports decoupled policy decisions. When RBAC-based namespace isolation does not meet your security requirements, use OPA to enforce access policies at the object level.
OPA also supports Layer 7 network policies and cross-namespace access control based on labels and annotations.
Gatekeeper is a Kubernetes admission controller that enforces OPA policies during workload deployment. See Gatekeeper for details.
Kyverno
Kyverno is a Kubernetes-native policy engine that validates, modifies, and generates configurations for Kubernetes resources. It uses Kustomize-style overlays for validation, supports strategic merge patch for mutation, and can clone resources across namespaces based on configurable triggers.
Use Kyverno to:
Isolate namespaces
Enforce pod security and other best practices
Generate default configurations such as network policies
See Kyverno and the policy repository for details.
Hard multi-tenancy
Create a separate cluster for each tenant to implement hard multi-tenancy. This provides strong isolation among tenants.
Disadvantages:
| Disadvantage | Details |
|---|---|
| Cost | A control plane fee is charged for each cluster. Computing resources cannot be shared across clusters, which leads to fragmentation when some clusters are underutilized. |
| Operational complexity | Managing hundreds of clusters requires specialized tooling and long-term maintenance overhead. |
| Slower provisioning | Creating a cluster takes longer than creating a namespace. |
Hard multi-tenancy is appropriate for highly regulated industries and SaaS environments that require strong tenant isolation.
Future trends
The Kubernetes Multi-Tenancy Special Interest Group (SIG) is working on several proposals to address the limitations of both soft and hard multi-tenancy:
Virtual cluster: creates dedicated control plane instances per tenant, including the API server, controller manager, and scheduler. The tenants in the cluster refer to Kubernetes on Kubernetes. See Virtual Cluster for details.
Hierarchical Namespace Controller (HNC): introduces parent-child relationships between namespaces, with policy object inheritance. Tenant administrators can create subnamespaces within their assigned hierarchy. See HNC for details.
Multi-Tenancy Benchmarks: provides guidelines for isolating and segmenting clusters using namespaces, and a
kubectl-mtbCLI for verifying compliance with those guidelines. See Multi-Tenancy Benchmarks for details.