Container Service for Kubernetes:Work with Terway

When you create an ACK cluster, you can choose one of the following network plug-ins:

• Terway: a network plug-in developed by Alibaba Cloud for ACK. Terway allows you to assign ENIs to containers and use standard Kubernetes network policies to regulate how containers communicate with each other. Terway also supports bandwidth throttling on individual containers. Terway uses flexible IP Address Management (IPAM) policies to allocate IP addresses to containers. This avoids IP address waste. If you do not want to use network policies, you can select Flannel as the network plug-in. Otherwise, we recommend that you select Terway.

• Flannel: an open source CNI plug-in, which is simple and stable. You can use Flannel with VPC. This allows your clusters and containers to run in high-performance and stable networks. However, Flannel provides only basic features. It does not support standard Kubernetes network policies. For more information, see Flannel.

The maximum number of pods supported by a node depends on the maximum number of ENIs supported by the instance type of the ECS instance on which that node is deployed. We recommend that you deploy nodes on ECS instances of new instance types and with high specifications. If you want to add an ECS instance to a cluster, the ENIs supported by the ECS instance must meet specific limits. For more information, see Overview of instance families. For more information about how to call API operations to query the maximum number of ENIs supported by an ECS instance type, visit OpenAPI Explorer. The following table describes the maximum number of pods that Terway supports in different scenarios.

You can refer to the following table to assign CIDR blocks for a cluster that uses Terway.

• IPv6 Dual-stack: Select Enable.

• VPC: Select the VPC that you created in Step 1: Plan CIDR blocks.

• vSwitch: Select the vSwitch that you created in Step 1: Plan CIDR blocks.

• Network Plug-in: Select Terway.

  • Specify whether to enable the Assign One ENI to Each Pod feature. To use the Assign One ENI to Each Pod feature, you need to log on to the Quota Center console and submit an application.

    • If you select the check box, a separate ENI is assigned to each pod.
      Note After you select Assign One ENI to Each Pod, the maximum number of pods supported by a node is reduced. Exercise caution before you enable this feature.
    • If you clear the check box, an ENI is shared among multiple pods. A secondary IP address that is provided by the ENI is assigned to each pod.
  • Specify whether to use IPVLAN.

    • This option is available only when you clear Assign One ENI to Each Pod.
    • If you select IPVLAN, IPVLAN and extended Berkeley Packet Filter (eBPF) are used for network virtualization when an ENI is shared among multiple pods. This improves network performance. Only the Alibaba Cloud Linux operating system is supported.
    • If you clear IPVLAN, policy-based routes are used for network virtualization when an ENI is shared among multiple pods. The CentOS 7 and Alibaba Cloud Linux operating systems are supported. This is the default setting.

    For more information about the IPVLAN feature in Terway mode, see Terway IPVLAN.

  • Select or clear Support for NetworkPolicy.

    • The NetworkPolicy feature is available only when you clear Assign One ENI to Each Pod. By default, Assign One ENI to Each Pod is unselected.
    • If you select Support for NetworkPolicy, you can use Kubernetes network policies to control the communication among pods.
    • If you clear Support for NetworkPolicy, you cannot use Kubernetes network policies to control the communication among pods. This prevents Kubernetes network policies from overloading the Kubernetes API server.
  • Select or clear Support for ENI Trunking. To use the Support for ENI Trunking feature, you need to log on to the Quota Center console and submit an application. The Terway Trunk elastic network interface (ENI) feature allows you to specify a static IP address, a separate vSwitch, and a separate security group for each pod. This allows you to manage and isolate user traffic, configure network policies, and manage IP addresses in a fine-grained manner. For more information, see Configure a static IP address, a separate vSwitch, and a separate security group for each pod.

• Pod vSwitch: Select the pod vSwitch that you created in Step 1: Plan CIDR blocks.

• Service CIDR: Use the default value.

• IPv6 Service CIDR: This parameter is available after you enable IPv4/IPv6 dual stack. Use the default value.

Compared with the default Terway mode, the Terway IPVLAN mode optimizes the performance of pod networks, Service networks, and network policies.

• Pod networks are directly implemented based on the sub-interfaces of ENIs in IPVLAN L2 mode. This significantly simplifies network forwarding on the host and reduces the latency by 30% compared with the traditional mode. The performance of pod networks is almost the same as that of the host network.

• Service networks are implemented based on the eBPF technology instead of the kube-proxy mode. Traffic forwarding no longer depends on the host iptables or IP Virtual Server (IPVS). This maintains almost the same performance in larger-scale clusters and offers better scalability. Compared with traffic forwarding based on IPVS and iptables, this new approach greatly reduces network latency in scenarios that involve a large number of new connections and port reuse.

• The network policies of pods are implemented based on the eBPF technology instead of iptables. This way, large numbers of iptables rules are no longer generated on the host and the impact of network policies on network performance is reduced.

Limits on the Terway IPVLAN mode

• Only the Alibaba Cloud Linux operating system is supported.

• The Sandboxed-Container runtime is not supported.

• The implementation of network policies is different from when the default Terway mode is used.

  • The CIDR selector has a lower priority than the pod selector. Additional pod selectors are required if the CIDR block of pods is within the CIDR range specified by the CIDR selector.

  • The except keyword of the CIDR selector is not fully supported. We recommend that you do not use the except keyword.

  • If you use a network policy of the Egress type, you cannot access pods in the host network or the IP addresses of nodes in the cluster.

• You may fail to access the Internet-facing SLB instance that is associated with a LoadBalancer Service from within the cluster due to loopback issues. For more information, see Why am I unable to access an SLB instance?

Scenarios

• Middleware and microservices

  Avoids performance degradation in large-scale deployments and reduces the network latency of microservices.

• Gaming and live streaming applications

  Significantly reduces network latency and resource competition among multiple instances.

• High-performance computing

  High-performance computing requires high-throughput networks. The Terway IPVLAN mode reduces CPU overhead and saves computing resources for core workloads.