The access control of network traffic between services and traffic encryption during data transmission - Container Service for Kubernetes

Restrict pod traffic with network policies and encrypt data in transit to secure ACK clusters.

Network policies vs. security groups

Network policies and security groups operate at different levels. Use them together for defense in depth.

Mechanism	Scope	Best for
Network policies	Pod-to-pod traffic (east-west traffic) and traffic between pods and external services	Restricting communication between microservices, namespaces, or specific pods
Security groups	Node-level and Virtual Private Cloud (VPC)-level traffic between nodes, other VPC resources, and external IP addresses	Controlling access to and from the cluster at the infrastructure layer

Use security groups to restrict infrastructure-level access and network policies to enforce fine-grained pod-level isolation within the cluster.

Restrict pod-to-pod traffic with network policies

By default, all pods in a Kubernetes cluster can communicate freely, which poses security risks. Network policies restrict traffic between pods (east-west traffic) and between pods and external services.

Network policies use pod selectors and labels to identify source and destination pods. Each policy can filter by IP address, port, protocol, or any combination.

With the Terway network plug-in, use network policies to control traffic at the IP address or port level. Also see Kubernetes Network Policy Recipes.

Important

Only Terway clusters support Kubernetes network policies. For clusters with more than 100 nodes, the policy proxy may increase control plane load. See Improve the performance of the NetworkPolicy feature for a large ACK cluster in Terway mode.

Recommended approach: default deny, then allow

Apply least privilege: deny all traffic by default, then add rules to allow only what each workload needs.

Create a default deny policy -- Block all inbound and outbound traffic in the namespace.
Allow DNS queries -- Pods need DNS resolution to function.
Allow specific traffic -- Open only the paths each workload requires.

Step 1: Create a default deny policy

Create a network policy to deny all inbound and outbound traffic in a namespace, or use Calico for a global policy.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Step 2: Allow DNS queries

With a default deny policy, pods cannot resolve DNS names. Create a policy to allow DNS queries to CoreDNS.

Add a label to the kube-system namespace:

   kubectl label namespace kube-system name=kube-system

Create a network policy to allow egress to CoreDNS on UDP port 53:

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: allow-dns-access
     namespace: default
   spec:
     podSelector:
       matchLabels: {}
     policyTypes:
     - Egress
     egress:
     - to:
       - namespaceSelector:
           matchLabels:
             name: kube-system
       ports:
       - protocol: UDP
         port: 53

Important

See Network Policies.

Step 3: Allow traffic from specific pods

With deny and DNS policies in place, allow traffic only from authorized pods. Common use cases:

Allow only specific microservices to access an application.
Allow only specific applications to access a database.

This example creates a pod and restricts ingress to pods with the app: bookstore label.

Create a pod with the app=bookstore and role=api labels:

   kubectl run apiserver --image=nginx --labels="app=bookstore,role=api" --expose --port=80

Apply a network policy that allows ingress only from pods with the app: bookstore label:

   kind: NetworkPolicy
   apiVersion: networking.k8s.io/v1
   metadata:
     name: api-allow
   spec:
     podSelector:
       matchLabels:
         app: bookstore
         role: api
     ingress:
     - from:
         - podSelector:
             matchLabels:
               app: bookstore

Verify that pods without the app=bookstore label are denied access:

   kubectl run test-$RANDOM --rm -i -t --image=alpine -- sh

   / # wget -qO- --timeout=2 http://apiserver
   wget: download timed out

Verify that pods with the app=bookstore label can access the server:

   kubectl run test-$RANDOM --rm -i -t --image=alpine --labels="app=bookstore,role=frontend" -- sh

   / # wget -qO- --timeout=2 http://apiserver
   <!DOCTYPE html>
   <html><head>

Add custom rules between pods in a namespace

After allowing pod communication within a namespace, use Kubernetes Network Policy Recipes to add custom rules for finer-grained control.

Control node-level traffic with security groups

ACK uses security groups to manage traffic between master and worker nodes, and between worker nodes, other VPC resources, and external IP addresses.

ACK automatically creates a security group for node communication when you create a cluster. To enforce least privilege, add recommended inbound and outbound rules.

See Configure security groups in different scenarios and Configure a security group.

Monitor and analyze traffic with flow logs

VPC flow logs record inbound and outbound traffic of elastic network interfaces (ENIs). Use flow logs to:

Verify access control list (ACL) rules
Monitor network traffic
Troubleshoot network issues
Identify abnormal traffic between resources (including pods) in a VPC

See Overview of flow logs.

Encrypt data in transit

Service Mesh (ASM)

Service Mesh (ASM) encrypts data between services. ASM supports:
- Mutual Transport Layer Security (mTLS) authentication between services
- Envoy Secret Discovery Service (SDS) for HTTPS and dynamic certificate loading on service gateways
- Traffic management for applications in ASM instances, integrated with Application High Availability Service (AHAS)
- Distributed tracing integrated with Tracing Analysis for trace mapping, call counting, topology, and dependency analysis
TLS for Ingress

Enable HTTPS for Ingress-exposed services. Use a Secret to configure TLS.