Container Service for Kubernetes:Manage north-south traffic

Prerequisites

Before you begin, make sure you have:

• ALB activated

• Fleet management enabled — see Enable multi-cluster management

• An ACK One Fleet instance associated with two ACK clusters in the same virtual private cloud (VPC) — see Manage associated clusters

• The kubeconfig file of the Fleet instance downloaded from the ACK One console, with kubectl connected to the Fleet instance

• The latest version of Alibaba Cloud CLI installed and configured

How it works

Traffic flows from the internet through the ALB instance to pods in your associated clusters:

Internet → ALB instance → ACK One Fleet instance → Associated cluster 1 (pods)
                                                  → Associated cluster 2 (pods)

The Fleet instance acts as the control plane. An AlbConfig object on the Fleet instance provisions the ALB instance. Ingress rules on the Fleet instance control how ALB routes traffic to pods across all associated clusters.

Step 1: Create an ALB multi-cluster gateway

Create an AlbConfig object on the Fleet instance to provision the ALB instance and register the associated clusters as traffic targets.

1. Get the IDs of two vSwitches from the VPC where the Fleet instance resides.

2. Create gateway.yaml with the following content. Replace ${vsw-id1} and ${vsw-id2} with the vSwitch IDs, and replace ${cluster1} and ${cluster2} with the IDs of the associated clusters.

   For each associated cluster (${cluster1} and ${cluster2}), configure the inbound rules of their security groups to allow traffic from all IP addresses and ports in the vSwitch CIDR block.

   Parameter	Required	Description
   metadata.name	Yes	Name of the AlbConfig.
   metadata.annotations: alb.ingress.kubernetes.io/remote-clusters	Yes	Comma-separated list of cluster IDs to add to the ALB multi-cluster gateway. Each cluster ID must already be associated with the Fleet instance.
   spec.config.name	No	Name of the ALB instance.
   spec.config.addressType	No	Network type of the ALB instance. Internet (default): public-facing, requires an elastic IP address (EIP), charged for instance fees and bandwidth. Intranet: private, accessible only within the VPC.
   spec.config.zoneMappings	Yes	vSwitch IDs for the ALB instance. The vSwitches must be in zones supported by ALB and in the same VPC as the clusters. For high availability, use vSwitches in at least two zones if the region supports it. For supported regions and zones, see Regions and zones in which ALB is available.
   spec.listeners	No	Listener port and protocol. The example uses HTTP on port 8001. Retain this configuration — without a listener, you cannot use ALB Ingresses.

   apiVersion: alibabacloud.com/v1
   kind: AlbConfig
   metadata:
     name: ackone-gateway-demo
     annotations:
       # Add associated clusters that are used to handle traffic to the ALB multi-cluster instance.
       alb.ingress.kubernetes.io/remote-clusters: ${cluster1},${cluster2}
   spec:
     config:
       name: one-alb-demo
       addressType: Internet
       addressAllocatedMode: Fixed
       zoneMappings:
       - vSwitchId: ${vsw-id1}
       - vSwitchId: ${vsw-id2}
     listeners:
     - port: 8001
       protocol: HTTP
   ---
   apiVersion: networking.k8s.io/v1
   kind: IngressClass
   metadata:
     name: alb
   spec:
     controller: ingress.k8s.alibabacloud/alb
     parameters:
       apiGroup: alibabacloud.com
       kind: AlbConfig
       name: ackone-gateway-demo

3. Apply the manifest to create the ALB multi-cluster gateway and IngressClass:

   kubectl apply -f gateway.yaml

4. After 1 to 3 minutes, verify the gateway is created:

   kubectl get albconfig ackone-gateway-demo

   Expected output:

   NAME                  ALBID      DNSNAME                                    PORT&PROTOCOL   CERTID   AGE
   ackone-gateway-demo   alb-xxxx   alb-xxxx.<regionid>.alb.aliyuncs.com                              4d9h

5. Verify the associated clusters are connected to the gateway:

   kubectl get albconfig ackone-gateway-demo -ojsonpath='{.status.loadBalancer.subClusters}'

   The output is a list of cluster IDs.

Step 2: Route traffic with Ingress

Create an Ingress with ingressClassName: alb to use the ALB multi-cluster gateway. ALB Ingresses support commonly used Nginx Ingress annotations and provide enhanced features for ALB instances. For a full list of supported annotations, see ALB Ingress configuration with annotations.

The following table summarizes the four routing scenarios covered in this section:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTP": 8001}]
  name: alb-ingress
  namespace: demo
spec:
  ingressClassName: alb
  rules:
  - host: alb.ingress.alibaba.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTP": 8001}]
    alb.ingress.kubernetes.io/cluster-weight.c6XXXXXXXXXXXXXXXXXXXXXXXXX: "100"
  name: alb-ingress
  namespace: demo
spec:
  ingressClassName: alb
  rules:
  - host: alb.ingress.alibaba.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTP": 8001}]
    alb.ingress.kubernetes.io/conditions.service1: |
     [{
       "type": "Header",
       "headerConfig": {
          "key":"stage",
           "values": [
              "gray"
           ]
       }
      }]
    alb.ingress.kubernetes.io/cluster-weight.c6XXXXXXXXXXXXXXXXXXXXXXXXX: "100"
  name: alb-ingress
  namespace: demo
spec:
  ingressClassName: alb
  rules:
  - host: alb.ingress.alibaba.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTP": 8001}]
    alb.ingress.kubernetes.io/cluster-weight.c6XXXXXXXXXXXXXXXXXXXXXXXXX: "60"
    alb.ingress.kubernetes.io/cluster-weight.cdXXXXXXXXXXXXXXXXXXXXXXXXX: "40"
  name: alb-ingress
  namespace: demo
spec:
  ingressClassName: alb
  rules:
  - host: alb.ingress.alibaba.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

Troubleshooting

Gateway not created after 3 minutes

Inspect the AlbConfig status for error details:

kubectl describe albconfig ackone-gateway-demo

Common causes:

• The vSwitches are not in zones supported by ALB. Verify using Regions and zones in which ALB is available.

• The security group inbound rules of the associated clusters do not allow traffic from the vSwitch CIDR block.

Associated clusters not showing in subClusters

Verify the cluster IDs in the alb.ingress.kubernetes.io/remote-clusters annotation match the IDs of clusters associated with the Fleet instance:

kubectl get albconfig ackone-gateway-demo -ojsonpath='{.status.loadBalancer.subClusters}'

If the output is empty, confirm the cluster IDs in gateway.yaml are correct and the clusters are associated with the Fleet instance.

What's next

• ALB Ingress configuration with annotations — full reference for ALB Ingress annotations

• Manage associated clusters — add or remove clusters from a Fleet instance

• Pay-as-you-go — billing details for Internet-facing ALB instances