All Products
Search
Document Center

Container Service for Kubernetes:Terway

Last Updated:Nov 10, 2023

Terway is an open source Container Network Interface (CNI) plug-in that is developed by Alibaba Cloud. Terway works with Virtual Private Cloud (VPC) and allows you to use standard Kubernetes network policies to control how containers communicate with each other. You can use Terway to set up network connectivity within a Kubernetes cluster. The topic introduces Terway and describes the usage notes and release notes for Terway.

Introduction

Terway is a CNI plug-in developed by Container Service for Kubernetes (ACK). The plug-in builds networks based on elastic network interfaces (ENIs) of Alibaba Cloud to make full use of cloud resources. Terway supports the use of extended Berkeley Packet Filter (eBPF) to accelerate network traffic and reduce latency. Terway supports standard Kubernetes network policies that define how containers communicate with each other.

In a cluster that has Terway installed, each pod has a separate network stack and is assigned a separate IP address. Pods on the same Elastic Compute Service (ECS) instance communicate with each other by forwarding packets inside the ECS instance. Pods on different ECS instances communicate with each other through ENIs in the VPC in which the ECS instances are deployed. This improves communication efficiency because no tunneling technologies, such as Virtual Extensible Local Area Network (VXLAN), are required to encapsulate packets.

Usage notes

For more information about how to use Terway, see Work with Terway.

Release notes

October 2023

Version number

Image address

Release date

Description

Impact

v1.6.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.6.0

2023-10-10

  • migrate terway eip function to ack extend network controller

  • The Kind field can be set to StatefulSet for the terway-controlplane component.

No impact on workloads

August 2023

Version number

Image address

Release date

Description

Impact

v1.5.7

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.7

2023-08-24

  • The startup of Terway is accelerated.

  • The health check timeout issue that occasionally occurs is fixed.

No impact on workloads

July 2023

Version number

Image address

Release date

Description

Impact

v1.5.6

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.6

2023-07-30

Custom CNI chains can be configured.

No impact on workloads

v1.5.5

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.5

2023-07-06

ENI filters can be configured.

No impact on workloads

June 2023

Version

Image address

Release date

Description

Impact

v1.5.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.4

2023-06-01

The issue that network jitters may occur during Terway updates in IPVLAN mode is fixed.

No impact on workloads

v1.4.8

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.8

2023-06-01

The issue that network jitters may occur during updates when Terway runs in IPVLAN mode is fixed.

No impact on workloads

May 2023

Version

Image address

Release date

Description

Impact

v1.5.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.3

2023-05-16

The issue that you may fail to access an external IP address when Terway 1.4.4 is used is fixed.

No impact on workloads

v1.5.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.2

2023-05-10

  • The issue that nodes may enter the NotReady state when the Terway update fails is fixed.

  • The issue that additional zone spread constraints are injected by using webhooks in ENI trunking mode is fixed.

No impact on workloads

v1.4.6

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.6

2023-05-10

The issue that nodes may enter the NotReady state when the Terway update fails is fixed.

No impact on workloads

April 2023

Version

Image address

Release date

Description

Impact

v1.5.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.1

2023-04-28

  • Kubernetes 1.26 is supported. Kubernetes 1.22 and later are supported by Terway 1.5.1 and later.

  • Felix is updated to 3.24.5

No impact on workloads

March 2023

Version

Image address

Release date

Description

Impact

v1.4.5

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.5

2023-03-31

Role-based access control (RBAC) permissions of Terway are reduced.

No impact on workloads

v1.4.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.4

2023-03-14

  • vSwitch configurations can be modified on the Add-ons page of the ACK console.

  • The amount of memory required by Terway in IPVLAN mode is reduced.

  • Cilium is updated to 1.12.7.

  • Iptables is updated to 1.8.8.

In IPVLAN mode, if externalTrafficPolicy=Local is specified in the configuration of a LoadBalancer Service, the Service routes requests only to the backend pods that are deployed on the node on which the Service is deployed. To allow the Service to route requests to backend pods on other nodes, change the Service type to ClusterIP or specify externalTrafficPolicy=Cluster in the configuration of the Service.

December 2022

Version

Image address

Release date

Description

Impact

v1.4.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.3

2022-12-23

  • Cilium is updated to 1.12.4.

  • The issue that an IP address may be allocated to multiple pods is fixed.

  • The update policy is changed to RollingUpdate.

No impact on workloads

November 2022

Version

Image address

Release date

Description

Impact

v1.4.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.2

2022-11-21

  • The Terway and localdns compatibility issue that occurs when ENI Trunking is enabled is fixed.

  • The NetworkPolicies and PostStart hook compatibility issue that occurs when the IPVLAN mode is disabled is fixed.

No impact on workloads

September 2022

Version

Image address

Release date

Description

Impact

v1.4.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.1

2022-09-28

  • The parameters of the terway-eniip component can be configured on the Add-ons page in the ACK console.

  • The issue that the hairpin access method fails is fixed.

  • The issue that IP addresses fail to be allocated when an unexpected allocation order is used by the container runtime to use the CNI plug-in is fixed.

No impact on workloads

August 2022

Version

Image address

Release date

Description

Impact

v1.4.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.0

2022-08-29

  • Cilium is updated to 1.12.1.

  • The Kubernetes version must be 1.20 or later.

After you enable the IPVLAN mode for a cluster, CustomResourceDefinitions (CRDs) that are created for Cilium are automatically updated when you update the cluster. This may rapidly increase the load on kube-apiserver if the cluster size is large. In this case, we recommend that you pay attention to the load on kube-apiserver.

This update has no negative impact on workloads.

July 2022

Version

Image address

Release date

Description

Impact

v1.3.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.3.0

2022-07-22

Quality of service (QoS) management is supported for pods. For more information, see Configure QoS for pods.

No impact on workloads

June 2022

Version

Image address

Release date

Description

Impact

v1.2.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.4

2022-06-28

  • The issue that the portmap plug-in does not take effect is fixed.

  • The issue that elastic IP addresses (EIPs) cannot be deleted is fixed.

  • The issue that traffic from Services to backend pods cannot be balanced in IPVLAN mode is fixed.

No impact on workloads

May 2022

Version

Image address

Release date

Description

Impact

v1.2.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.3

2022-05-26

  • EIP bandwidth plans and BGP (Multi-ISP) Pro EIPs are supported.

  • The issue that containers fail to pass the health checks that are performed by the kubelet when Cilium network policies are used is fixed.

No impact on workloads

March 2022

Version

Image address

Release date

Description

Impact

v1.2.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.2

2022-03-31

The ARP probe issue in VLAN mode is fixed.

No impact on workloads

v1.2.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.1

2022-03-15

  • The issue that calico-felix changes to the legacy iptables mode when calico-felix runs on Alibaba Cloud Linux 3 is fixed. The issue that calico-felix changes to the legacy iptables mode when calico-felix runs on Alibaba Cloud Linux 3 is fixed.

  • The issue that IP addresses cannot be revoked when metadata errors occur is fixed.

No impact on workloads

January 2022

Version

Image address

Release date

Description

Impact

v1.2.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.0

2022-01-11

  • The efficiency of file locking when CNI is being executed is improved.

  • A Service can be accessed by using its backend pods in IPVLAN mode.

  • In IPVLAN mode, if you access an external IP address or a Server Load Balancer (SLB) instance from within a cluster, the traffic is routed to the backend Service. This feature is automatically enabled for newly created clusters. To enable load balancing within an existing cluster in Terway IPVLAN mode, see How do I enable load balancing within a cluster in Terway IPVLAN mode? .

  • The issue that false positive alerts are generated in Terway VPC mode is fixed.

  • The performance of calico-felix is improved.

No impact on workloads

December 2021

Version

Image address

Release date

Description

Impact

v1.1.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.1.1

2021-12-20

  • ENI reclamation is optimized to help resolve the issue that IPVLANs cannot be created due to netns leaks in containerd scenarios.

  • An ENI can be associated with multiple security groups. For more information, see Associate multiple security groups with an ENI.

No impact on workloads

November 2021

Version

Image address

Release date

Description

Impact

v1.1.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.1.0

2021-11-22

  • IPv4/IPv6 dual stack is supported.

  • Felix is updated to V3.20.2

  • CNI is updated from V0.3.0 to V0.3.1.

  • The deployment template of Terway is compatible with Kubernetes 1.22. Terway of this version and later versions are compatible with Kubernetes 1.18 and later versions.

  • The issue that ENI configuration errors occasionally occur in IPVLAN mode is fixed. For more information, see #261.

  • If Typha is deployed in your cluster to improve the performance of network policies, update Terway to this version. For more information, see Improve the performance of the NetworkPolicy feature for a large ACK cluster in Terway mode.

  • If Typha is not deployed in your cluster, your workloads are not affected.

  • After you disable the IPVLAN mode for a cluster, CRDs that are created for calico-felix are automatically updated when you update the cluster. This may rapidly increase the load on kube-apiserver if the cluster size is large. In this case, we recommend that you pay attention to the load on kube-apiserver.

September 2021

Version

Image address

Release date

Description

Impact

v1.0.10.443-gaa1bfcc-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.443-gaa1bfcc-aliyun

2021-09-14

Network latency is reduced.

This update applies only to the exclusive ENI mode. Other modes do not require this update.

No impact on workloads

August 2021

Version

Image address

Release date

Description

Impact

v1.0.10.398-g63d2e57-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.398-g63d2e57-aliyun

2021-08-04

  • The issue that network policies occasionally fail to take effect when Terway is used in inclusive ENI mode is fixed.

  • Cilium is updated to 1.10.

After you enable the IPVLAN mode for a cluster, CRDs that are created for Cilium are automatically updated when you update the cluster. This may rapidly increase the load on kube-apiserver if the cluster size is large. In this case, we recommend that you pay attention to the load on kube-apiserver.

This update has no negative impact on workloads.

July 2021

Version

Image address

Release date

Description

Impact

v1.0.10.390-g5f3c461-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.390-g5f3c461-aliyun

2021-07-02

  • The issue that pod annotations cannot be modified when EIPs are used in Terway mode is fixed.

  • The issue that Terway cannot be started when a dedicated ENI is assigned to each pod in Terway mode is fixed.

  • The network namespace path is automatically adjusted in containerd containers.

  • Alibaba Cloud Linux 3 is supported.

No impact on workloads

May 2021

Version

Image address

Release date

Description

Impact

v1.0.10.368-g2890967-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.368-g2890967-aliyun

2021-05-24

  • The issue that the ResourceInvalid alert occurs when EIPs are used is fixed.

  • Communication between pods and the node is supported in IPVLAN mode. Data can be transmitted within the node and does not need to be transmitted over the VPC in which the node is deployed.

  • The ip forwarding setting can be proactively checked and corrected.

No impact on workloads

April 2021

Version

Image address

Release date

Description

Impact

v1.0.10.333-gfd2b7b8-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.333-gfd2b7b8-aliyun

2021-04-26

  • The IP address conflicts that occur when stateful applications use EIPs are fixed.

  • Hubble can be enabled in IPVLAN mode. For more information about Hubble, see What is Hubble.

No impact on workloads

March 2021

Version

Image address

Release date

Description

Impact

v1.0.10.323-g778c128-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.323-g778c128-aliyun

2021-03-22

  • The number of API calls per minute is reduced to prevent API abuse.

  • Event alerts are optimized when security group configurations are inspected.

No impact on workloads

February 2021

Version

Image address

Release date

Description

Impact

v1.0.10.317-g0652857-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.317-g0652857-aliyun

2021-02-22

  • CiliumIdentity leaks are fixed.

  • The issue that occasionally occurs in StatefulSet pod IP management is fixed.

No impact on workloads

v1.0.10.309-g5314eee-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.309-g5314eee-aliyun

2021-02-05

  • Errors in the configurations of security group rules are no longer automatically fixed. The system generates alerts only when errors are detected. You can follow the suggestions to fix the errors.

  • The issue that the IP addresses of pods may be reclaimed and reassigned to the pods when the nodes are overloaded is fixed.

No impact on workloads

January 2021

Version

Image address

Release date

Description

Impact

v1.0.10.301-g0115576-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.301-g0115576-aliyun

2021-01-21

New features:

  • Custom maximum transmission units (MTUs) are supported.

  • Routing based on the host network stack in exclusive ENI mode is supported.

Fixed issues:

  • Packet loss due to traffic throttling of Terway.

  • Residual IP policies.

  • Incorrect counting of the number of IP addresses provided by ENIs.

No impact on workloads

December 2020

Version

Image address

Release date

Description

Impact

v1.0.10.280-gdc2cb6c-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.280-gdc2cb6c-aliyun

2020-12-25

  • Proactive checks for Terway are supported. The following items are checked:

    • ENIs and secondary IP addresses of ENIs.

    • The consistency of security groups.

    • The pod network configurations.

    • The host network configurations.

  • Routing based on the host network stack in IPVLAN mode is supported.

No impact on workloads

v1.0.10.263-gdbe50a9-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.263-gdbe50a9-aliyun

2020-12-03

The Terway error that occurs in exclusive ENI mode is fixed.

No impact on workloads

November 2020

Version

Image address

Release date

Description

Impact

v1.0.10.261-g8342155-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.261-g8342155-aliyun

2020-11-27

  • Synchronous calls to the ECS API are supported.

  • The issue that error messages are returned when the CNI plug-in is used is fixed.

No impact on workloads

v1.0.10.250-gb7bb10a-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.250-gb7bb10a-aliyun

2020-11-23

  • The pod network issue that occurs when the ENI driver fails to be loaded is fixed.

  • The status issue of the IP addresses that are allocated by ENIs when the ENI API is throttled is fixed.

No impact on workloads

October 2020

Version

Image address

Release date

Description

Impact

v1.0.10.247-g4cb77d0-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.247-g4cb77d0-aliyun

2020-10-26

ECS instances that are deployed on dedicated hosts are supported.

No impact on workloads

September 2020

Version

Image address

Release date

Description

Impact

v1.0.10.237-g6a0f948-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.237-g6a0f948-aliyun

2020-09-16

The time required to bind ENIs to pods is reduced.

No impact on workloads

August 2020

Version

Image address

Release date

Description

Impact

v1.0.10.221-g8d6386a-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.221-g8d6386a-aliyun

2020-08-11

IPvlan and the eBPF are supported for network virtualization.

No impact on workloads

v1.0.10.213-g27145cc-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.213-g27145cc-aliyun

2020-08-04

The pod network issue that occurs due to occasional ENI failures is fixed.

No impact on workloads

July 2020

Version

Image address

Release date

Description

Impact

v1.0.10.208-gf3144bf-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.208-gf3144bf-aliyun

2020-07-20

  • The issue that policy-based routes for nodes in inclusive ENI mode are exposed is fixed.

  • API calls over internal networks are supported.

  • The issue that pod IP addresses cannot be released when the number of IP addresses provided by the vSwitch reaches the upper limit is fixed.

  • The error report page that returns CNI errors is optimized.

No impact on workloads

v1.0.10.211-gef088a4-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.211-gef088a4-aliyun

2020-07-24

Cluster ID tags can be added to ENIs.

No impact on workloads

April 2020

Version

Image address

Release date

Description

Impact

v1.0.10.156-g8660a0f-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.156-g8660a0f-aliyun

2020-04-22

  • The caching efficiency is improved when ENIs are used.

  • The pre-installed Felix is updated to V3.5.8.

  • Reclaiming network resources from pods that are in the Completed or Failed state is supported.

No impact on workloads

February 2020

Version

Image address

Release date

Description

Impact

v1.0.10.139-g14a4f84-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.139-g14a4f84-aliyun

2020-02-12

The issue that pod creation requests occasionally time out is fixed.

No impact on workloads

January 2020

Version

Image address

Release date

Description

Impact

v1.0.10.133-g001396b-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.133-g001396b-aliyun

2020-01-10

  • The NetworkPolicy feature can be disabled.

  • IPVLAN is supported for network virtualization in inclusive ENI mode.

No impact on workloads

December 2019

Version

Image address

Release date

Description

Impact

v1.0.10.122-gd0be015-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.122-gd0be015-aliyun

2019-12-24

The efficiency of IP address allocation is optimized in inclusive ENI mode.

No impact on workloads

October 2019

Version

Image address

Release date

Description

Impact

v1.0.10.100-g92a3fa5-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.100-g92a3fa5-aliyun

2019-10-11

The issue that the host node enters the NotReady state when a large number of jobs concurrently request resources is fixed.

No impact on workloads

August 2019

Version

Image address

Release date

Description

Impact

v1.0.9.20-g35ae000-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.20-g35ae000-aliyun

2019-08-23

Kubernetes 1.14.6 is supported.

No impact on workloads

April 2019

Version

Image address

Release date

Description

Impact

v1.0.9.15-g3957085-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.15-g3957085-aliyun

2019-04-11

The issue that the Terway update occasionally fails is fixed.

No impact on workloads

March 2019

Version

Image address

Release date

Description

Impact

v1.0.9.14-ga0346bb-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.14-ga0346bb-aliyun

2019-03-28

  • The issue that Terway fails to obtain the ENI information when the meta server is throttled is fixed.

  • The issue that the failed to move veth to host netns: file exists error is returned when you create a container is fixed.

  • Periodic scanning is supported to check the status of ENIs. ENIs that are abnormally released are periodically reclaimed.

  • Health checks are supported. TCP port check is performed instead of HTTP path check.

No impact on workloads