Container Service for Kubernetes:Access services using Gateway with Inference Extension

In an ACK Edge cluster, the deployment method for Envoy Gateway differs from that in a standard ACK cluster:

The current deployment is configured with a node pool distribution policy. You can adjust the number of replicas in component management to ensure that at least one control plane replica runs in each node pool.

Step 1: Create a test application

Save the following example as backend.yaml and run the kubectl apply -f backend.yaml command to create the test applications: backend and backend-2.

For ACK Edge scenarios, you must configure the service topology. Add the openyurt.io/topologyKeys: openyurt.io/nodepool annotation to the Service to ensure that traffic forwards only to pods within the same node pool. For more information, see Node pool service topology management.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: backend
---
apiVersion: v1
kind: Service
metadata:
  name: backend
  annotations:
    openyurt.io/topologyKeys: openyurt.io/nodepool # Configure the node pool topology
  labels:
    app: backend
    service: backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend
      version: v1
  template:
    metadata:
      labels:
        app: backend
        version: v1
    spec:
      serviceAccountName: backend
      containers:
        - image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/envoygateway-echo-basic:v20231214-v1.0.0-140-gf544a46e
          imagePullPolicy: IfNotPresent
          name: backend
          ports:
            - containerPort: 3000
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: backend-2
---
apiVersion: v1
kind: Service
metadata:
  name: backend-2
  annotations:
    openyurt.io/topologyKeys: openyurt.io/nodepool # Configure the node pool topology
  labels:
    app: backend-2
    service: backend-2
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: backend-2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend-2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend-2
      version: v1
  template:
    metadata:
      labels:
        app: backend-2
        version: v1
    spec:
      serviceAccountName: backend-2
      containers:
        - image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/envoygateway-echo-basic:v20231214-v1.0.0-140-gf544a46e
          imagePullPolicy: IfNotPresent
          name: backend-2
          ports:
            - containerPort: 3000
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace

Step 2: Confirm the GatewayClass

After you install the Gateway with Inference Extension component, a GatewayClass is created by default. Run the following command to confirm:

kubectl get gatewayclass

Expected output:

NAME          CONTROLLER                                      ACCEPTED   AGE
ack-gateway   gateway.envoyproxy.io/gatewayclass-controller   True       2m31s

If the GatewayClass resource is not found, you can create it manually. Save the following example as gatewayclass.yaml and run the kubectl apply -f gatewayclass.yaml command to create it.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: ack-gateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

Step 3: Create a custom EnvoyProxy configuration

In an ACK Edge cluster, you can create a custom EnvoyProxy configuration for each node pool to specify the scheduling node pool for data plane pods and the service exposure method.

Replace NPXXX in the following example with your actual node pool ID. Then, save the configuration as gateway-config.yaml and run the kubectl apply -f gateway-config.yaml command.

You can find the node pool ID on the Node Management > Node Pools page in the ACK console.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        pod:
          nodeSelector:
            alibabacloud.com/nodepool-id: NPXXX  # Replace with your actual node pool ID
      envoyService:
        annotations:
          openyurt.io/topologyKeys: openyurt.io/nodepool  # Configure the service topology
        type: NodePort  # ACK Edge uses NodePort to expose services

This configuration includes three key settings for ACK Edge scenarios:

Step 4: Create the Gateway resource

Save the following example as gateway.yaml and run the kubectl apply -f gateway.yaml command.

The component's control plane automatically creates an EnvoyProxy deployment and a NodePort service based on the Gateway resource. It also starts a listener on the specified port on the node.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: ack-gateway
spec:
  gatewayClassName: ack-gateway
  listeners:
    - name: http
      protocol: HTTP
      port: 80
  infrastructure:
    parametersRef:
      group: gateway.envoyproxy.io
      kind: EnvoyProxy
      name: custom-proxy-config  # Associate with the custom configuration created in Step 3

Step 5: Get the gateway endpoint

The gateway service is exposed as a NodePort. The corresponding service endpoint is the IP address of any node in the node pool and its service port.

1. Query the service port.

   kubectl get service -n kube-system -l gateway.envoyproxy.io/owning-gateway-name=ack-gateway

   Expected output:

   NAME                                 TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
   envoy-default-ack-gateway-7452df7d   NodePort   192.168.86.174   <none>        80:30364/TCP   19m

2. You can set the gateway host environment variable. Replace NODEIP with the IP address of any node in the node pool and NODEPORT with the port number obtained in the previous step.

   export GATEWAY_HOST=NODEIP:NODEPORT