All Products
Search

This Product

    Container Service for Kubernetes:Work with Terway

    Document Center

    Container Service for Kubernetes:Work with Terway

    Last Updated:Mar 26, 2026

    Terway is an open source Container Network Interface (CNI) plug-in developed by Alibaba Cloud. It integrates with Virtual Private Cloud (VPC) and assigns real VPC IP addresses directly to pods by managing Elastic Network Interfaces (ENIs) on each node. Terway supports Kubernetes NetworkPolicyQuota Center console for traffic control and offers network acceleration through DataPath V2.

    Before you start, review Networking overview and Comparison between Terway and Flannel to select the right plug-in. Plan your cluster CIDR blocks before creating the cluster. For details, see ACK managed cluster network planning.

    Billing

    Terway itself is free. Terway pods run on every node and consume a small amount of node resources. For charges related to Alibaba Cloud services that ACK uses, see Cloud resource fees.

    Choose a Terway mode

    Terway operates in two modes: Shared ENI mode and Exclusive ENI mode. Starting with Terway v1.11.0, you configure the mode per node pool rather than at cluster creation.

    Shared ENI mode Exclusive ENI mode
    Pod IP assignment Multiple pods share one ENI Each pod gets a dedicated ENI
    Pod density High — a single node can support hundreds of pods Low — typically single digits per node
    Network performance Standard; DataPath V2 acceleration available ENI-native performance, comparable to VM-level networking
    NetworkPolicy Supported Not supported
    Static IP, separate vSwitch, separate security group Supported when ENI trunking is enabled Supported (always available)
    DataPath V2 acceleration Supported Not supported
    Node-level network configuration Supported Supported
    Best for General-purpose Kubernetes workloads Applications requiring high throughput, low latency, or VM-like network isolation (for example, databases or stateful services that need fixed IPs and dedicated security groups)
    In Terway v1.11.0 and later, both shared ENI and exclusive ENI node pools can coexist in the same cluster. See Terway release notes.

    Feature compatibility

    Use the following table to check which features can be combined before you design your cluster.

    Feature Shared ENI mode Exclusive ENI mode Requires ENI trunking Requires DataPath V2
    NetworkPolicy Yes No No No (eBPF-based from Terway v1.9.2)
    DataPath V2 acceleration Yes No No —
    Static IP address Yes (with trunking) Yes Yes (shared ENI only) No
    Separate vSwitch per pod Yes (with trunking) Yes Yes (shared ENI only) No
    Separate security group per pod Yes (with trunking) Yes Yes (shared ENI only) No
    IPv6 dual-stack Yes No No No

    Install Terway when you create a cluster

    Select Terway as the network plug-in during cluster creation. You cannot change the plug-in after the cluster is created.

    1. Log on to the Container Service Management Console. In the left navigation pane, click Clusters.

    2. On the Clusters page, click Create Kubernetes Cluster.

    3. Configure the key network parameters listed below. For all other cluster parameters, see Create an ACK managed cluster.

    Parameter Description
    Network Plug-in Select Terway.
    DataPath V2 Enable to activate DataPath V2 acceleration. Terway uses a different traffic forwarding path from standard shared ENI mode, reducing latency by 30%. For details, see Network acceleration.
    Support for NetworkPolicy Enable to support Kubernetes-native NetworkPolicy. From Terway v1.9.2, NetworkPolicy is implemented using eBPF.
    Support for ENI trunking Enable to assign a static IP address, a separate vSwitch, and a separate security group to each pod. For new ACK managed clusters with Kubernetes 1.31 or later, ENI trunking is enabled by default.
    VPC The VPC for the cluster.
    vSwitch The vSwitch CIDR block for cluster nodes. Select at least three vSwitches across different zones for high availability.
    Pod vSwitch The vSwitch CIDR block for pods. This block can overlap with the node vSwitch CIDR block.
    Service CIDR The CIDR block for Services. Must not overlap with the node or pod CIDR blocks.
    IPv6 Dual-stack Select Enable to create a cluster that supports both IPv4 and IPv6. Requires Kubernetes 1.22 or later and cannot be used with eRDMA. When using shared ENI mode, the node instance type must support IPv6 and have equal numbers of assignable IPv4 and IPv6 addresses.
    IPv6 Service CIDR Available after enabling IPv6 dual-stack.

    Calculate the maximum number of pods per node

    The pod limit per node is determined by the ENI capacity of the ECS instance type, not by the maxPods kubelet parameter. (Modifying maxPods only affects pods running in hostNetwork mode.)

    A node must meet the minimum pod threshold to join the cluster:

    • Shared ENI mode: the calculated maximum must be greater than 11.

    • Exclusive ENI mode: the calculated maximum must be greater than 6.

    Shared ENI mode:

    Max pods = (EniQuantity - 1) × EniPrivateIpAddressQuantity

    Example: ecs.g7.4xlarge supports 8 ENIs, each with 30 private IP addresses → (8 - 1) × 30 = 210 pods.

    Shared ENI + Trunk ENI mode:

    Max Trunk pods = EniTotalQuantity - EniQuantity

    Exclusive ENI mode:

    • ECS instances: EniQuantity - 1

    • Lingjun instances: LeniQuota - 1 Example: ecs.g7.4xlarge supports 8 ENIs → 8 - 1 = 7 pods.

      Lingjun instances do not support static IP addresses, separate vSwitches, or separate security groups in exclusive ENI mode.

    Look up ENI capacity for an instance type

    Option 1: In the ACK console, when creating a node pool, check the Instance type section. The Terway compatibility (supported pods) field shows the pod limit.

    Option 2: Query the DescribeInstanceTypes API in OpenAPI Explorer. Set InstanceTypes to your instance type and click Initiate call. The response includes:

    Field Meaning
    EniQuantity Maximum number of ENIs the instance supports
    EniPrivateIpAddressQuantity Number of private IP addresses per ENI
    EniTotalQuantity Total number of network interfaces (ENIs + trunk)

    You can also check the Instance family overview for ENI counts by instance type.

    Network acceleration

    DataPath V2 is available for shared ENI mode node pools. It replaces kube-proxy and iptables-based forwarding with eBPF, achieving pod network performance close to host network performance.

    DataPath V2 is an upgraded version of the earlier IPVLAN + eBPF mode. Clusters created with Terway 1.8.0 or later support DataPath V2 only. For clusters created with Terway 1.7.0 or earlier, see IPVLAN + eBPF acceleration.

    What DataPath V2 does:

    • Pod-to-pod (cross-node): eBPF bypasses the network protocol stacks of both nodes.

    • Pod-to-pod (same node): Traffic is forwarded internally without passing through the host protocol stack.

    • Pod-to-Service: eBPF resolves the Service address to a backend pod address, bypassing iptables and IPVS on the host.

    • NetworkPolicy: Implemented in eBPF instead of iptables, avoiding large rule sets on the host.

    Performance impact: Pod network latency is reduced by 30% compared to standard shared ENI mode. Service networking scales better in large clusters because eBPF conntrack replaces iptables.

    For new clusters with Kubernetes 1.34 or later and DataPath V2 enabled, kube-proxy no longer runs on Terway nodes. This mode includes built-in portmap support, so you do not need to configure the portmap plug-in separately. See Configure a custom CNI chain.

    eBPF conntrack: In DataPath V2, connection tracking data for container networks is stored in eBPF maps using Least Recently Used (LRU) eviction. When a map reaches capacity, the oldest connections are evicted to make room for new ones. Size the conntrack map according to your workload scale. For details, see Optimize conntrack configurations in Terway.

    IPVLAN + eBPF acceleration

    This mode applies to clusters created with Terway 1.7.0 or earlier.

    What IPVLAN + eBPF does:

    • Pod-to-Service: eBPF resolves the Service address to a backend pod address within the pod's network namespace.

    • Pod-to-pod: IPVLAN bypasses the network protocol stacks of both nodes.

    To use this mode, set Network Plug-in to Terway and select Pod IPVLAN when creating the cluster.

    Configure access control

    NetworkPolicy

    Shared ENI mode node pools support Kubernetes-native NetworkPolicy. Exclusive ENI mode node pools do not.

    To enable NetworkPolicy, select Support for NetworkPolicy when creating the cluster. For usage details, see Use network policies in ACK clusters.

    Managing NetworkPolicy through the console is in public preview. Submit an application in the Quota Center console to access this feature.

    Static IP address, separate vSwitch, and separate security group

    Both modes support assigning a static IP address, a dedicated vSwitch, and a dedicated security group to each pod, enabling fine-grained traffic isolation and IP address management.

    • Exclusive ENI mode: These features are available without additional configuration.

    • Shared ENI mode: Requires ENI trunking. Select Support for ENI trunking when creating the cluster. After enabling, the terway-eniip and terway-controlplane components are installed.

    You can enable ENI trunking for an ACK managed cluster without submitting an application. For an ACK dedicated cluster, submit an application in the Quota Center console. For new ACK managed clusters with Kubernetes 1.31 or later, ENI trunking is enabled by default.

    For configuration steps, see Configure a fixed IP address, a separate vSwitch, and a separate security group for a pod.

    Limitations

    Scale limits

    Terway allocates node network interfaces and IP addresses by calling cloud product OpenAPIs. The following parallel allocation limits apply and cannot be adjusted.

    Mode Parallel allocation limit
    Shared ENI mode 500 nodes
    Exclusive ENI / Trunk ENI mode 100 pods

    DataPath V2 limitations

    • Kernel version: Linux kernel 5.10 or later is required. Alibaba Cloud Linux OS images are recommended.

    • Sandboxed containers: Not supported.

    • NetworkPolicy — CIDR selectors: Pod CIDR block control is not supported. Use pod label selectors to control pod access.

    • NetworkPolicy — `except` keyword: Not fully supported in CIDR selectors. Avoid using it.

    • NetworkPolicy — Egress: Egress-type NetworkPolicy prevents access to pods in hostNetwork mode and to node IP addresses within the cluster.

    • NodePort with `ExternalTrafficPolicy=Local`: Traffic may fail. Set ExternalTrafficPolicy=Cluster instead.

    • NodePort with `ExternalTrafficPolicy=Cluster`: SNAT is applied to the source address. Available port range: 32768–65535.

    • SLB loopback: Cluster-internal access to SLB instances associated with LoadBalancer Services may cause loopback and connectivity failures. See Why am I unable to access an SLB instance?

    • IPv6 hairpin: Not supported.

    • DataPath V2 resource consumption: The Terway policy container consumes an additional 0.5 cores and 512 MB per worker node. The default CPU limit is 1 core, with no memory limit. Resource consumption grows with cluster size.

    Data plane configuration requirements

    Terway depends on the precise order and integrity of kernel-level configurations. Uncoordinated changes to IP Rule, IP Route, or eBPF hooks by external components — such as priority overrides, rule conflicts, or program unloads — can cause pod network outages, policy failures, or traffic hijacking. Validate all third-party component integrations carefully before deploying them alongside Terway.

    TC filter rules

    Interface Direction Program Priority Function
    ethx toContainer VLAN Untag 20000 Remove VLAN tag
    ethx toContainer cil_from_netdev 25000 Cilium service/network policy
    veth toContainer cil_to_container 25000 Cilium service/network policy
    veth fromContainer cil_from_container 25000 Cilium service/network policy
    ethx fromContainer cil_to_netdev 25000 Cilium service/network policy
    ethx fromContainer VLAN Tag 50001 Add VLAN tag

    IP rule priorities

    Direction Priority Routing table
    toContainer 512 1000 + linkIndex (ENI index)
    fromContainer 512 1000 + linkIndex (ENI index)

    Usage notes

    The Terway configuration file eni-config contains system parameters. Do not modify or delete unsupported fields — doing so can cause network interruptions or pod creation failures. For the list of editable parameters, see Custom Terway configuration parameters.

    Terway uses Custom Resource Definitions (CRDs) to track resource status. Modifying system resources can cause network interruptions or pod creation failures.

    Resource name Resource type Manage the CRD? Manage the CR?
    podnetworkings.network.alibabacloud.com User resource No Yes
    podenis.network.alibabacloud.com System resource No No
    networkinterfaces.network.alibabacloud.com System resource No No
    nodes.network.alibabacloud.com System resource No No
    noderuntimes.network.alibabacloud.com System resource No No
    \*.cilium.io System resource No No
    \*.crd.projectcalico.org System resource No No
    Important

    The primary ENI on each node is allocated to the node OS. The remaining ENIs are managed by Terway. Do not manually configure these ENIs. For ENI management details, see Configure a filter for ENIs.

    FAQ

    How do I tell whether a node pool is in shared ENI mode or exclusive ENI mode?

    In Terway v1.11.0 and later, shared ENI mode is the default. To enable exclusive ENI mode for a node pool, see Configure the exclusive ENI network mode for a node pool.

    For clusters created with Terway v1.11.0 or earlier, the mode was selected at cluster creation. Check the DaemonSet name in the kube-system namespace:

    • terway-eni — exclusive ENI mode

    • terway-eniip — shared ENI mode

    Can I change the CNI plug-in for an existing ACK cluster?

    No. The network plug-in is set at cluster creation and cannot be changed afterward. To use a different CNI plug-in, create a new cluster and migrate your workloads.

    What's next