Use annotations in a Service YAML file to configure Edge Load Balancer (ELB) instances, listeners, backend server groups, and edge elastic IP addresses (edge EIPs).
All annotation keys in this topic use the prefix service.beta.kubernetes.io/. Only the suffix is shown in the tables.
Usage notes
-
Annotation values are case-sensitive. Use lowercase letters.
-
Set
type: LoadBalanceron the Service. -
Set
loadBalancerClass: alibabacloud.com/elbto use ELB.
ELB instance configuration cannot be updated after the instance is created. To change any instance-level setting, delete the Service and recreate it with the new values.
Baseline Service configuration
All examples in this topic build on the following Service. It selects ENS node pools by label and enables Service topology so that traffic is forwarded only to Pods in the specified node pool.
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool # Enable Service topology.
service.openyurt.io/nodepool-labelselector: key1=val1 # Select ENS node pools.
spec:
ports:
- name: nginx-80
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
loadBalancerClass: alibabacloud.com/elb
The nodepool-labelselector annotation accepts two syntaxes:
| Syntax | Logic | Example |
|---|---|---|
key1=val1,key2=val2 |
AND — node pool must satisfy all labels | env=prod,region=cn-north |
key in (val1,val2) |
OR — node pool satisfies any value | region in (cn-north,cn-south) |
ELB instance configuration
Specify an instance type and billing method
For instance type details and pricing, see CreateLoadBalancer and Instance types and billing.
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-spec |
ELB instance type, for example, elb.s1.small. |
elb.s2.small |
alibaba-cloud-loadbalancer-pay-type |
Billing method. Only PostPaid (pay-as-you-go) is supported. |
PostPaid |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: elb.s1.small
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-pay-type: PostPaid
Set the Service exposure mode
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-address-type |
internet: exposes the Service to the internet (requires an internal ELB instance and an edge EIP). intranet: exposes the Service to the private network (requires only an internal ELB instance). |
internet |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: internet
Configure edge EIP specifications (internet mode only)
When address-type is internet, both an internal ELB instance and an edge EIP are required.
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-eip-bandwidth |
Maximum bandwidth of the edge EIP. Unit: Mbit/s. | 10 |
alibaba-cloud-loadbalancer-eip-isp |
ISP. Values: cmcc (China Mobile), unicom (China Unicom), elecom (China Telecom). |
None |
alibaba-cloud-loadbalancer-eip-instance-charge-type |
EIP billing method. Only PostPaid (pay-as-you-go) is supported. |
PostPaid |
alibaba-cloud-loadbalancer-eip-internet-charge-type |
Internet bandwidth billing method. Only 95BandwidthByMonth (pay-by-monthly-95th-percentile) is supported. |
95BandwidthByMonth |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool # Enable Service topology.
service.openyurt.io/nodepool-labelselector: key1=val1 # Select ENS node pools.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: internet
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-eip-bandwidth: "10"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-eip-isp: cmcc
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-eip-instance-charge-type: "PostPaid"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-eip-internet-charge-type: "95BandwidthByMonth"
Backend server groups
Set the external traffic policy
The external traffic policy controls which ENS nodes receive traffic:
-
Local mode (
externalTrafficPolicy: Local): traffic goes only to nodes that host application Pods. Cannot be used when multiple Services share one ELB instance. -
Cluster mode (default): traffic is distributed to all ENS nodes in the virtual private cloud (VPC). Required when multiple Services share one ELB instance.
Example (local mode):
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool # Enable Service topology.
service.openyurt.io/nodepool-labelselector: key1=val1 # Select ENS node pools.
spec:
ports:
- name: nginx-80
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
loadBalancerClass: alibabacloud.com/elb
externalTrafficPolicy: Local
Add backend servers by node label
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-backend-label |
Labels of nodes to add as backend servers. Multiple labels use AND logic and are separated by commas, for example, k1=v1,k2=v2. |
N/A |
To exclude specific edge nodes from the backend server pool, add the label node.kubernetes.io/exclude-from-external-load-balancers=true to those nodes.
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: key2=val2
Set backend server weights
When listeners use the weighted round-robin (WRR) algorithm, requests are distributed based on backend server weights.
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-backend-weight |
Weight assignments in base=<weight>,<nodeName>=<weight> format. base sets the default weight for all backend servers. Individual node overrides are separated by commas. |
base=100 |
Example — set the default weight to 50 and override node1 to 80 and node2 to 90:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1,key2=val2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-weight: base=50,node1=80,node2=90
Listeners
EdgeControllerManager (ECM) automatically creates listeners based on spec.ports. Backend ports are assigned by the cluster. To pin a specific backend port, set nodePort explicitly:
spec:
ports:
- name: nginx-80
port: 80
protocol: TCP
nodePort: 30080
targetPort: 80
Set the scheduling algorithm
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-scheduler |
Load balancing algorithm. | rr |
Supported algorithms:
| Value | Algorithm | Behavior |
|---|---|---|
rr |
Round robin | Distributes requests evenly across all backend servers. |
wrr |
Weighted round robin | Backend servers with higher weights receive more requests. |
wlc |
Weighted least connections | Distributes requests based on weight and current connection count. |
sch |
Source IP consistent hashing | Routes requests from the same source IP to the same backend server. |
qch |
QUIC ID consistent hashing | Routes requests with the same QUIC ID to the same backend server. |
iqch |
iQUIC CID consistent hashing | Routes requests with the same 2nd, 3rd, and 4th bytes of the iQUIC CID to the same backend server. |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: wrr
Configure session persistence (TCP listeners)
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-persistence-timeout |
Session persistence duration in seconds. Valid range: 0–3600. Set to 0 to disable. When an ELB instance has multiple TCP listeners, this setting applies to all of them. |
0 |
Configure the connection timeout (TCP listeners)
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-established-timeout |
TCP connection timeout in seconds. Valid range: 10–900. | 500 |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1,key2=val2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "900"
Configure health checks
| Annotation suffix | Description | Default |
|---|---|---|
alibaba-cloud-loadbalancer-healthy-threshold |
Number of consecutive health check passes required to mark a backend server as healthy. Valid range: 2–10. | 3 |
alibaba-cloud-loadbalancer-unhealthy-threshold |
Number of consecutive health check failures required to mark a backend server as unhealthy. Valid range: 2–10. | 3 |
alibaba-cloud-loadbalancer-check-interval |
Interval between health checks in seconds. Valid range: 2–10. | 2 |
alibaba-cloud-loadbalancer-health-check-connect-timeout |
Health check response timeout in seconds. If a backend server does not respond within this period, the check is marked as failed. Valid range: 1–300. | 5 |
Example:
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool
service.openyurt.io/nodepool-labelselector: key1=val1,key2=val2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "5"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "5"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-check-interval: "2"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "5"
Advanced configuration
Use an existing ELB instance
To attach an existing ELB instance to a Service, specify the network ID and ELB instance ID.
-
Apply the Service with the
managed-by-userannotation:apiVersion: v1 kind: Service metadata: name: nginx namespace: default annotations: openyurt.io/topologyKeys: openyurt.io/nodepool # Enable Service topology. service.openyurt.io/nodepool-labelselector: key1=val1 service.beta.kubernetes.io/alibaba-cloud-loadbalancer-managed-by-user: "true" spec: ports: - name: nginx-80 port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer loadBalancerClass: alibabacloud.com/elb -
Find the PoolService that corresponds to the target node pool (ENS VPC) and annotate it with the ELB instance ID:
kubectl annotate ps {<SERVICENAME>-NodePoolID} service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id=<lb_ID>Replace
<SERVICENAME>with the PoolService name of the node pool (ENS VPC), and replace<lb_ID>with the ELB instance ID.
Share one ELB instance across multiple Services
When multiple Services share one ELB instance, all of the following conditions must be met:
-
Set
externalTrafficPolicyto cluster mode (the default). -
Use a self-managed ELB instance and specify it with the
managed-by-userannotation. -
If internet access via EIP is needed, create and manage the EIP manually.
-
Force-overwrite listeners by setting
service.openyurt.io/elb-force-override-listeners: "true".
Example:
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
annotations:
openyurt.io/topologyKeys: openyurt.io/nodepool # Enable Service topology.
service.openyurt.io/nodepool-labelselector: key1=val1
service.openyurt.io/elb-force-override-listeners: "true"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-managed-by-user: "true"
spec:
ports:
- name: nginx-80
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
loadBalancerClass: alibabacloud.com/elb