How to build an MCP service gateway in ACK? - Container Service for Kubernetes

To expose a Model Context Protocol (MCP) service in a Container Service for Kubernetes (ACK) cluster to external Large Language Models (LLMs), install the ack-agent-gateway extension. This extension is built on the Gateway API and lets you route MCP traffic quickly and securely.

Prerequisites

Your ACK managed cluster runs Kubernetes version 1.32 or later.
Gateway API version 1.3.0 or later is installed in your cluster.

Install ack-agent-gateway

ack-agent-gateway is a gateway component that implements the standard Gateway API to simplify configuring and managing traffic for your MCP services.

On the ACK Clusters page, click the name of the target cluster. In the left navigation pane, choose Applications > Helm.
On the Helm page, click Deploy. In the Chart section, search for and select ack-agent-gateway. Keep the default settings and click Next.
By default, the component is installed in the ack-agent-gateway namespace, and its Helm release shares the same name.
On the Parameters step, select the latest chart version and click OK.
After the installation finishes, you can view the component on the Helm page. Verify that its status is Deployed.

Step 1: Deploy a sample MCP service

In this step, deploy a sample MCP service that you will use later to validate the gateway.

Create a file named mcp-server.yaml with the following content. This YAML manifest defines a Deployment and a Service to deploy the sample service in the default namespace.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-mcp
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: demo-mcp
  template:
    metadata:
      labels:
        app.kubernetes.io/name: demo-mcp
    spec:
      containers:
        - image: registry-cn-hangzhou.ack.aliyuncs.com/dev/sample-mcp-server-fetch:v0.2.0
          imagePullPolicy: IfNotPresent
          name: mcp
          ports:
            - containerPort: 8000
              name: server
              protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: demo-mcp-server
spec:
  ports:
    - name: server
      port: 8000
      protocol: TCP
      targetPort: 8000
  selector:
    app.kubernetes.io/name: demo-mcp
  sessionAffinity: None
  type: ClusterIP

Deploy the sample MCP service.
```
kubectl apply -f mcp-server.yaml
```

Check the pod status.

kubectl get pod -l app.kubernetes.io/name=demo-mcp

Expected output: Ensure the pod is in the Running state.

NAME                        READY   STATUS    RESTARTS   AGE
demo-mcp-58fddf4cd9-8jvzn   1/1     Running   0          3s

Step 2: Create a Gateway and routing rules

In this step, you use a Gateway, an HTTPRoute, and the ack-agent-gateway CustomResourceDefinition (CRD) Backend to create a gateway entrypoint for the MCP service that you deployed in the previous step.

Backend (CRD): The standard backendRef in the Gateway API does not support the fine-grained attributes required by the MCP protocol, such as streaming HTTP. To resolve this, ack-agent-gateway introduces the Backend CRD. This CRD extends the Gateway API's capabilities, allowing you to precisely define the protocol type and access path of your backend service.
Gateway: Defines a traffic entrypoint, specifies the listener port and protocol, and associates ack-agent-gateway as its controller.
HTTPRoute: Defines traffic routing rules that direct traffic received by the Gateway to your custom Backend resource.

Create a file named mcp-gateway.yaml with the following content.

# Define the details of the MCP backend service
apiVersion: agentgateway.alibabacloud.com/v1alpha1
kind: Backend
metadata:
  name: test-mcp
spec:
  type: MCP # Define the backend type as MCP
  mcp:
    targets:
      - name: mcp-target # The unique identifier for the Target
        static:
          # The FQDN of the backend Kubernetes Service
          host: demo-mcp-server.default.svc.cluster.local 
          # The protocol path of the MCP service
          path: /mcp 
          port: 8000
          # Define the transport protocol for MCP, which is Streamable HTTP in this case
          protocol: StreamableHTTP 
---
# Define the gateway instance as the traffic entrypoint
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: test-mcp-gateway
spec:
  gatewayClassName: ack-agent-gateway # Specify ack-agent-gateway as the gateway controller
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Same # Allow only routing rules created in the same namespace to be associated with this gateway
---
# Define routing rules to direct traffic to the MCP backend
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: route-for-mcp-backend
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: test-mcp-gateway # Associate with the Gateway created above
  rules:
    - backendRefs:
        # Note: The backendRef here references our custom Backend CRD
        - group: agentgateway.alibabacloud.com
          kind: Backend
          name: test-mcp

Deploy the gateway and routing rules.
```
kubectl apply -f mcp-gateway.yaml
```

Get the public IP address of the gateway.

kubectl get gateway test-mcp-gateway

Record the gateway IP address from the ADDRESS column, which you will use for testing in the next step.

NAME               CLASS               ADDRESS        PROGRAMMED   AGE
test-mcp-gateway   ack-agent-gateway   114.55.xx.xx   True         13m

Step 3: Test service connectivity

Ensure that you have a Node.js environment installed locally. In this step, you use the MCP Inspector tool to simulate a client and test access to the MCP service through the gateway.

In your local terminal, run the following command to install and start MCP Inspector.
```
npx @modelcontextprotocol/inspector@v0.17.5
```
After the command runs, the terminal outputs a local access address, which is typically in the format http://localhost:xxxx. Open this address in your browser.
On the MCP Inspector page that opens, fill in the following information:
- Transport Type: Select Streamable HTTP.
- URL: Enter http://<GATEWAY_IP>:80/mcp. Replace <GATEWAY_IP> with the gateway IP address that you obtained in the previous step (ADDRESS).
Click Connect. If the connection is successful, the page displays a connected status.
(Optional) Click Tools > List Tools to view the list of tools from the MCP service and further verify its availability.

(Optional) Step 4: Add API key authentication

This step shows how to add API key authentication to the MCP service using a TrafficPolicy resource, without modifying any application code.

TrafficPolicy (CRD): A CRD provided by ack-agent-gateway to attach advanced features to routing rules. It implements the Policy Attachment model, which decouples policies, such as authentication and rate limiting, from the HTTPRoute.
Secret: Used to securely store API key credentials.

API key authentication provides a simple way to secure your service. However, leaked credentials can be misused. Always manage and protect your credentials carefully.

Create a file named mcp-api-key.yaml with the following content. This YAML manifest defines a Secret that contains two API keys, and a TrafficPolicy that attaches the API key authentication policy to the HTTPRoute.

apiVersion: v1
kind: Secret
metadata:
  name: mcp-api-key
stringData:
  # The key (such as key1) is only an identifier. The value (such as key-value-foo) is the actual credential the client must provide.
  key1: 'key-value-foo'
  key2: 'key-value-bar'
---
apiVersion: agentgateway.alibabacloud.com/v1alpha1
kind: TrafficPolicy
metadata:
  name: test-mcp-apikey-auth
spec:
  targetRefs: # The target for policy attachment
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: route-for-mcp-backend # Attach to the previously created HTTPRoute
  traffic:
    authentication:
      apiKeyAuth:
        secretRef:
          name: mcp-api-key # Reference the Secret that stores the credentials

Apply the API key authentication policy.
```
kubectl apply -f mcp-api-key.yaml
```
Return to the MCP Inspector page to verify the authentication.
1. Refresh the page, then click Connect again.
2. The connection now fails, and the page displays Connection Error. At the same time, the terminal running MCP Inspector also outputs an error log similar to api key authentication failure, which confirms that the authentication policy is active.
3. Click the Authentication button on the page, enter one of the credentials defined in the Secret (for example, key-value-foo), and click Connect again. A successful connection confirms that the API key authentication is working correctly.

Clean up resources

To avoid unnecessary charges, delete all Kubernetes and cloud resources created during this tutorial.

Delete all Kubernetes resources created in this tutorial.

# If you completed the optional Step 4, first delete the authentication-related resources
kubectl delete -f mcp-api-key.yaml

# Delete the gateway and routing rules
kubectl delete -f mcp-gateway.yaml

# Delete the backend sample service
kubectl delete -f mcp-server.yaml

Verify that the associated Server Load Balancer (SLB) instance was released. Deleting the Gateway resource automatically deletes the SLB instance, which you can confirm in the SLB console.