How to configure Gateway API to manage AI agent traffic in ACK? - Container Service for Kubernetes

To enable AI agent applications to serve external traffic, you can install the ack-agent-gateway extension, which is based on the Gateway API. This extension lets you precisely manage Agent2Agent (A2A) protocol traffic.

Applicability

An ACK managed cluster of version 1.32 or later.
Gateway API version 1.3.0 or later is installed in your cluster.

Install ack-agent-gateway

ack-agent-gateway is a gateway component that implements the Gateway API standard. It uses the standardized traffic management model of the Gateway API to configure and manage service traffic for AI agent scenarios.

On the ACK Clusters page, click the name of the target cluster. In the left navigation pane, choose Applications > Helm.
On the Helm page, click Create. In the Chart section, search for and select ack-agent-gateway. Keep the default settings and click Next.
By default, the component is installed in the ack-agent-gateway namespace. The application is published using the component name.
On the Parameters page, select the latest Chart version and click OK.
After the installation is complete, you can view the component on the Helm page. The status of the component is Deployed.

Step 1: Deploy a sample A2A service

In this step, you deploy a simple agent service that supports the A2A protocol. This service acts as a backend for testing gateway routing in a later step.

Create a file named a2a-server.yaml that contains the following content. This file defines a deployment and a service to expose the deployment within the cluster.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-a2a
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: demo-a2a
  template:
    metadata:
      labels:
        app.kubernetes.io/name: demo-a2a
    spec:
      containers:
        - image: registry-cn-hangzhou.ack.aliyuncs.com/dev/sample-a2a-agent-helloworld:v0.2.0
          imagePullPolicy: IfNotPresent
          name: agent
          ports:
            - containerPort: 9999
              name: server
              protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: demo-a2a-agent-server
spec:
  ports:
    - name: server
      port: 9999
      protocol: TCP
      targetPort: 9999
  selector:
    app.kubernetes.io/name: demo-a2a
  sessionAffinity: None
  type: ClusterIP

Deploy the sample A2A service.
```
kubectl apply -f a2a-server.yaml
```

Check the pod status of the service.

kubectl get pod -l app.kubernetes.io/name=demo-a2a

Expected output: The pod is in the Running state.

NAME                        READY   STATUS    RESTARTS   AGE
demo-a2a-77dd75ddcf-qn9jx   1/1     Running   0          13s

Step 2: Create a gateway and a routing rule

In this step, you create a gateway instance and define a routing rule. The rule directs external traffic to the A2A service that you deployed in the previous step.

Create a file named a2a-gateway.yaml that contains the following content.

---
# Custom Backend resource that defines the backend A2A service
apiVersion: agentgateway.alibabacloud.com/v1alpha1
kind: Backend
metadata:
  name: test-a2a
spec:
  type: A2A
  a2a:
    targets:
      - name: a2a-target
        static:
          # Points to the Service created in Step 1
          host: demo-a2a-agent-server
          port: 9999
---
# Gateway resource that defines the traffic entrypoint
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: test-a2a-gateway
spec:
  # Specifies ack-agent-gateway as the implementer of this Gateway
  gatewayClassName: ack-agent-gateway
  listeners:
    - name: http
      # The port exposed by the gateway
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          # Allows only HTTPRoutes in the same namespace to attach to this Listener
          from: Same
---
# HTTPRoute resource that defines the routing rule
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: route-for-a2a-backend
spec:
  parentRefs:
    # Attaches this HTTPRoute to the Gateway named test-a2a-gateway
    - name: test-a2a-gateway
  rules:
    - backendRefs:
        # References the custom Backend resource as the backend
        - group: agentgateway.alibabacloud.com
          kind: Backend
          name: test-a2a

Run the following command to create the gateway and related resources.
```
kubectl apply -f a2a-gateway.yaml
```
Run the following command to obtain the public access address of the gateway.
```
kubectl get gateway test-a2a-gateway
```
The ADDRESS field shows the public IP address of the Server Load Balancer (SLB) instance that is associated with the gateway. Record this IP address for later use.
```
NAME               CLASS               ADDRESS        PROGRAMMED   AGE
test-a2a-gateway   ack-agent-gateway   8.136.xx.xx    True         4m36s
```

Step 3: Test service connectivity

In this step, you use an A2A client tool to access the backend service through the gateway that you created in the previous step. This verifies that the routing configuration is effective.

Install the Git tool and download the official sample code for the A2A protocol.
In this example, you run git checkout on a specific commit to ensure that the sample code is compatible with this document. In a real-world project, you should check out a stable release tag.
```
git clone https://github.com/a2aproject/a2a-samples.git && \
cd a2a-samples && git checkout d4fa006438e52 && \
cd samples/python/hosts/cli
```
Ensure that a local Python environment is installed. Then, use the uv tool to install the dependencies for the client program.
```
uv sync
```

Access the A2A service through the gateway. Replace <GATEWAY_IP> in the command with the gateway address (ADDRESS) that you obtained in the previous step.

uv run . --agent http://<GATEWAY_IP>

Expected output: After a successful connection is established, the agent's information card (Agent Card) is displayed and the program waits for input.

Will use headers: {}
======= Agent Card ========
{"capabilities":{"streaming":true},"defaultInputModes":["text"],"defaultOutputModes":["text"],"description":"Just a hello world agent","name":"Hello World Agent",...}
=========  starting a new task ========

What do you want to send to the agent? (:q or quit to exit):

At the prompt, enter hello and press Enter. The client sends the message to the backend service. Follow the prompts to receive a Hello World reply.

What do you want to send to the agent? (:q or quit to exit): hello
Select a file path to attach? (press enter to skip):
stream event => {"kind":"message","messageId":"...","parts":[{"kind":"text","text":"Hello World"}],"role":"agent"}

{"kind":"message","messageId":"...","parts":[{"kind":"text","text":"Hello World"}],"role":"agent"}

Enter quit to exit the client program.

Step 4: Add API key authentication to the service (Optional)

This step shows how to add API key authentication to the agent service using a TrafficPolicy resource without modifying the application code.

TrafficPolicy (CRD): A CRD provided by ack-agent-gateway that attaches advanced features to routing rules. It is an implementation of Policy Attachment, which decouples policies, such as authentication and rate limiting, from the HTTPRoute.
Secret: Used to securely store API key credentials.

API key authentication is a simple authentication method. If credentials are leaked, they can be misused. You must manage and protect your credentials with care.

Create a file named a2a-api-key.yaml that contains the following content. This YAML file defines a Secret that contains two API keys and a TrafficPolicy that attaches the API key authentication policy to the HTTPRoute.

---
apiVersion: v1
kind: Secret
metadata:
  name: a2a-api-key
stringData:
  # The key (such as key1) is only an identifier. The value (such as key-value-foo) is the actual credential that the client must provide.
  key1: 'key-value-foo'
  key2: 'key-value-bar'
---
apiVersion: agentgateway.alibabacloud.com/v1alpha1
kind: TrafficPolicy
metadata:
  name: test-a2a-apikey-auth
spec:
  targetRefs:                              # The target for policy attachment
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: route-for-a2a-backend          # Attaches to the previously created HTTPRoute
  traffic:
    authentication:
      apiKeyAuth:
        secretRef:
          name: a2a-api-key                # References the Secret that stores the credentials

Apply the authentication policy.
```
kubectl apply -f a2a-api-key.yaml
```

Test the authentication.

First, try to access the service without providing any credentials. Replace <GATEWAY_IP> with your gateway address.
```
uv run . --agent http://<GATEWAY_IP>
```
Expected output: The gateway rejects the request because a valid API key is not provided. The program fails to start and returns a 401 Unauthorized error.
```
a2a.client.errors.A2AClientHTTPError: HTTP Error 401: Failed to fetch agent card from http://8.136.xx.xx/.well-known/agent-card.json: Client error '401 Unauthorized' ...
```

Next, access the service again. Use the --bearer-token parameter to provide a valid API key, such as key-value-foo.

uv run . --bearer-token 'key-value-foo' --agent http://<GATEWAY_IP>

Expected output: The request header contains Authorization: Bearer key-value-foo. The gateway authenticates the credentials, forwards the request to the backend service, and the program starts successfully.

Will use headers: {'Authorization': 'Bearer key-value-foo'}
======= Agent Card ========
{"capabilities":{"streaming":true},"defaultInputModes":["text"],"defaultOutputModes":["text"],"description":"Just a hello world agent","name":"Hello World Agent",...}
=========  starting a new task ========

What do you want to send to the agent? (:q or quit to exit):

Clean up resources

To avoid incurring unnecessary fees, delete all Kubernetes resources and cloud resources that you created in this tutorial after you complete the experiment.

Delete all Kubernetes resources that you created in this tutorial.

# If you performed the optional Step 4, first delete the authentication-related resources
kubectl delete -f a2a-api-key.yaml

# Delete the gateway and routing rule
kubectl delete -f a2a-gateway.yaml

# Delete the backend sample service
kubectl delete -f a2a-server.yaml

Confirm that the SLB instance is released. When you delete the Gateway resource, the associated SLB instance is automatically deleted. You can log on to the Server Load Balancer console to confirm that the instance has been deleted.