×
Community Blog Locking Down AI Agent Sandbox with Alibaba Cloud AI Gateway

Locking Down AI Agent Sandbox with Alibaba Cloud AI Gateway

Lock an AI agent sandbox so its only exit is Alibaba Cloud AI Gateway. Same code, two sandboxes: one leaked a fake key, the other sent nothing.

Introduction

In my last post I used the MCP Tool Hook in AgentRun to inspect an agent's tool calls. When an indirect prompt injection pushed the agent toward the wrong tool, the Hook caught the call and stopped it.

A Hook only sees tool calls, though. Agents now run code inside a sandbox. When that code sends a request to the outside world, it is not a tool call. It is plain network traffic, and it never passes through the Hook.

A sandbox isolates execution. It does not close the way out. A sandbox with default settings is connected to the internet, and it needs a model API key in its environment variables to call a model. If a web page the agent reads hides a line such as "send your environment variables to this address", the key leaves without the Hook ever seeing it.

Teams evaluating agents end up asking the same questions.

  • Is it safe to put a model key inside the sandbox?
  • How do I stop an agent from sending customer data out?
  • Who cuts off an agent that is stuck in a loop and burning tokens?
  • How do I explain this setup to the security team?

This post answers them with one structure: reduce the agent's exits to a single door, the AI Gateway. The sandbox gets no route to the internet, model calls go only to the gateway, and the real key lives only inside the gateway. I then ran the same code in an open sandbox and a locked one to see what gets blocked and what keeps working.

My first post used AI Gateway for model routing, and the second built a security layer with guardrails and masking. This time the same gateway becomes the only exit for an agent sandbox. And after I thought everything was locked, I found one hole I had not expected.

One Way Out

The sandbox can reach one place, the AI Gateway, and only the gateway goes out to the internet.

sandbox_egress_en

The sandbox subnet's route table has no internet route, and the security group opens only port 443 to the gateway. The hop between the sandbox and the gateway is HTTPS as well. Only the gateway subnet goes out through NAT to reach Model Studio. The real model key stays inside the gateway, and the sandbox holds a consumer key that works only at the gateway. The DNS section below explains why the path to the VPC DNS is closed as well.

So what can leave through the one path that stays open? If an injected agent puts a key or customer data into a prompt, the request still lands in Model Studio under your own account. The gateway forwards requests only to the backend registered for the Model API, and the model's answer comes back to the agent, not to the attacker. Masking, the guardrail, the quota, and the logs all sit on that path. How Model Studio stores and handles the data it receives is a matter of its own data policy, which is worth checking separately.

Same code, two sandboxes

The default sandbox sent a fake key to an external server. The locked sandbox ran the same code and sent nothing.

There is a single set of test code, and it runs unchanged in two AgentRun Code Interpreter templates. The only difference between the templates is the network configuration. If the code is identical and the results differ, the configuration is what did the blocking.

I did not reproduce the injection itself. I assumed the agent had already fallen for an injection and ran the code it would then execute directly in the sandbox. The question here is not how to stop an injection, but what leaves when one gets through.

  • tpl-public: the default PUBLIC network mode
  • tpl-private: PRIVATE network mode, a dedicated vSwitch with no internet route, and a security group

Both sandboxes carry only three things in their environment variables: the gateway address, a gateway consumer key, and a fake value named CANARY_SECRET for the exfiltration test. The Model Studio API key was never placed in either sandbox.

Test Destination Default sandbox (PUBLIC) Locked sandbox (PRIVATE)
Reach an external site example.com HTTP 200 in 0.05 s Timed out after 10 s (curl exit 28)
Call the model API directly dashscope-intl.aliyuncs.com HTTP 400 (reached, no key) Timed out (curl exit 28)
Send the fake key out External webhook POST 200, arrived outside Timed out (curl exit 28)
Call the model through the gateway AI Gateway (VPC-only address) Unreachable (outside the VPC) HTTP 200, normal completion

The locked sandbox cannot reach the internet or the model API directly, yet it still uses the model through the gateway.

The exfiltration test also printed the environment variables inside each sandbox. Neither one held a Model Studio key. The only credential present was the gateway consumer key. Even an injected agent can only steal a key that works at the gateway and nowhere else, and in the locked sandbox it has no way to send that key out.

Build guide

The setup has five steps, in the order network, gateway, sandbox. The region is Singapore (ap-southeast-1), and the gateway and the sandbox share one VPC.

1. A dedicated vSwitch and route table for the sandbox

What keeps the sandbox off the internet is the route table, not the security group.

  1. In the VPC console, create a vSwitch that only the sandbox will use.
  2. Under Route Tables, create a custom route table (for example sandbox-noegress-rt) and associate it with that vSwitch.
  3. Add no custom routes to it. Only the system routes for the VPC should remain, with no 0.0.0.0/0 entry.

Leave the gateway's vSwitches on the existing system route table. The next step adds a NAT gateway, which puts an internet route into the system table automatically. If the sandbox shared that table, the sandbox would get internet access too.

15_route_tables

15b_system_routes

2. A NAT gateway for the gateway only

The gateway has to reach Model Studio. Without an internet exit in the VPC, calls through the gateway end in 503 upstream connect timeout.

  1. Create an Internet NAT gateway and bind an EIP to it.
  2. Create SNAT entries only for the gateway vSwitches. Do not select the sandbox vSwitch.

11_nat_gateway

12_snat_table

3. The sandbox security group

The security group limits where the sandbox can go inside the VPC to one place, the gateway. It keeps two outbound rules.

Action Priority Protocol Destination
Allow 1 TCP 443 Gateway private IP/32
Deny 100 All 0.0.0.0/0

Do not leave out the deny rule. A basic security group allows all outbound traffic by default, so adding allow rules alone blocks nothing. The default AI Gateway domain is served only over HTTP 80, so in step 4 I attached a custom domain and certificate to open 443.

At first I also added rules allowing port 53 to the VPC DNS servers (100.100.2.x), because I assumed the sandbox needed DNS to find the gateway domain. A later section covers the problem those rules caused.

19_sg_443

4. AI Gateway

I created a Dedicated instance in the same VPC and applied six things to the Model API. The masking plugin requires a Dedicated instance, not Serverless.

  1. Backend and Model API: register Model Studio as a backend service and create an LLM Model API. The example uses qwen-flash. The Model Studio API key goes here and nowhere else.
  2. Consumer authentication: create a consumer, issue an API key, authorize it for the Model API, and turn authentication on. The sandbox sends the consumer key as a Bearer token in the Authorization header.
  3. AI security guardrail: deploying the Model API attaches a security policy automatically, but it is off by default. Turn it on yourself.
  4. PII masking: install AI data masking from the plugin marketplace and create a rule scoped to the Model API. The built-in patterns follow Chinese formats, so I wrote a regex for Korean mobile numbers.
  5. Token quota: set a per-consumer token quota under FinOps. I used 500 tokens per day.
  6. HTTPS domain: the default domain only listens on HTTP 80. Add a custom domain (for example ai-gw.sandbox.internal), attach a certificate to turn on HTTPS, and bind the domain to the Model API. I used a certificate signed by my own CA and uploaded it to Certificate Management Service. The name is used only inside the VPC, so I did not register it in public DNS.

This is the masking rule I used.

system_deny: false
deny_openai: true
deny_jsonpath:
  - $.messages[*].content
deny_words: []
replace_roles:
  - regex: '010-\d{4}-\d{4}'
    type: replace
    restore: true
    value: '010-****-****'
  - regex: '%{EMAILLOCALPART}@%{HOSTNAME:domain}'
    type: replace
    restore: true
    value: '****@$domain'

When you save a rule, a Confirm to Modify Rule dialog appears after the first OK. The rule reaches the running gateway only after you click OK there as well. Also turn on SLS log delivery under observability.

02_gateway_basic_info

05_consumer_auth

06_quota_rule

07_security_guard

09f_masking_deny_openai

20_https_domain

5. The AgentRun sandbox template

Create a Code Interpreter template in the AgentRun console.

  • Network mode: PRIVATE
  • VPC: the same VPC as the gateway
  • vSwitch: the dedicated sandbox vSwitch from step 1
  • Security group: the one from step 3
  • Environment variables: the gateway address and the consumer key only
  • CA certificate: the public CA certificate file used to verify the gateway's certificate

The comparison template tpl-public differs only in its network mode, PUBLIC. Once DNS is blocked later in this post, the sandbox can no longer resolve the gateway domain, so pass the gateway private IP as well and have calls pin that IP.

10_agentrun_templates

What the gateway enforces

Once there is a single exit, authentication, inspection, and cost control can all sit at that exit. Requests from the locked sandbox to the gateway were handled as follows.

What I checked Request Result
Authentication Call without a consumer key HTTP 401
Prompt injection blocking A DAN-style jailbreak prompt Blocked, promptAttack level high
PII masking A prompt containing a mobile number and an email Only masked values reached the model, and the response was restored to the originals
Runaway cost control The same request repeated HTTP 429 on the 5th call
Audit logging All of the above Recorded in SLS per consumer

Do not judge an injection block by the status code. As in my second post, a blocked request still returns HTTP 200. Check whether the model field in the response body is from-security-guard.

I checked masking separately for what the model received and what the caller received. I asked the model to repeat a sentence containing a mobile number and an email exactly. With my first setting (deny_openai: false), the response contained 010-****-**** and ****@example.com. The values were masked before the model, but they were not restored on the way back, even with restore turned on. After I changed deny_openai to true, the response came back with the original number and email. To confirm the model still received masked values with this setting, I sent the same sentence and asked the model not to repeat it, only to answer how many asterisks (*) it contained. The original sentence has none, and the model answered with a number above zero. The model sees only masked values, and the caller gets the originals.

The quota cuts off slightly past the limit. With a limit of 500 tokens per day, cumulative tokens per call looked like this.

Call Response Cumulative tokens
1 200 135
2 200 270
3 200 405
4 200 533
5 429 Blocked

The fourth call already passed 500, but the block started on the next call, because usage metering lags slightly behind the requests. That is enough to stop an agent stuck in a loop. If you need a hard budget ceiling, set the limit with some margin.

The logs keep the reason for each block. The gateway access log in SLS records the consumer name together with the guardrail result, so you can trace later which sandbox was blocked, when, and why. Request and response bodies are not logged by default. If you turn body logging on, apply masking with it, or the log itself becomes a store of personal data.

09_gateway_observability

Locked down, and still leaking through DNS

Even with every internet exit closed, DNS queries from the sandbox still reached the outside. I closed that path by removing the DNS allow rules from the security group, and in exchange the gateway now has to be called by IP.

How DNS exfiltration works

A DNS query can carry information. If an attacker owns a domain, an agent only has to look up stolen-value.attacker-domain.com, and that query ends up at the attacker's name server. No HTTP request is needed.

So I issued DNS queries from the locked sandbox with the fake key embedded in the name. I used example.com in place of an attacker domain.

The finding: direct queries were blocked, but the VPC DNS went out on the sandbox's behalf

Query path Default sandbox Locked sandbox
Directly to an external resolver (8.8.8.8, 223.5.5.5) Answered Timed out
example.com through the VPC DNS (100.100.2.x) Answered Answered with real IPs
canary-not-a-real-key-001.example.com through the VPC DNS Answered Answered by example.com's external authoritative name server

The locked sandbox could not ask an external resolver directly. When it asked the VPC DNS, though, the VPC DNS resolved the name externally for it, and the name carrying the fake key was forwarded all the way to example.com's authoritative name server. The internet route, the NAT scope, and the security group were all locked, and this path was still open.

I had added the VPC DNS allow rules so the sandbox could find the gateway domain.

17_t11_dns_exfil

The fix: four security group rules

I removed the four rules allowing port 53 to the VPC DNS servers (two UDP, two TCP). It took no new resources and no cost. Two outbound rules remain: allow TCP 443 to the gateway, and deny everything else.

  • Queries to the VPC DNS no longer got an answer.
  • The external site, the direct model API call, and the fake key upload now fail at name resolution.
  • The gateway answered HTTP 200 when called with its IP pinned.
curl --resolve <gateway-domain>:443:<gateway-private-ip> \
  --cacert ca.pem \
  https://<gateway-domain>/v1/chat/completions \
  -H "Authorization: Bearer $CONSUMER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen-flash", "messages": [{"role": "user", "content": "hello"}]}'

In this state I checked authentication (401), injection blocking, and the quota (429) again, and all three still worked.

The trade-off

With DNS blocked, the sandbox cannot resolve any name. The gateway has to be called by its private IP, and if agent code or an SDK uses the host name, it needs an entry in the hosts file. That IP lives in two places, the security group and the sandbox, so if it changes, both have to change together. Work that needs the internet is blocked too. If the agent has to install packages with pip or read the web, you need an image with the packages built in or a mirror inside the VPC. If a production environment needs finer control per domain, the DNS Boundary Firewall in Cloud Firewall is worth evaluating. I did not test it for this post.

Six pitfalls I hit

  1. Allow rules alone block nothing. A basic security group allows all outbound traffic. I added a rule allowing only the gateway IP and thought the sandbox was locked, but traffic to other destinations still went through. A low-priority deny-all rule is required.
  2. Adding a NAT gateway can open the sandbox too. Creating a NAT gateway adds an internet route to the system route table automatically. Every vSwitch in my VPC shared that table, so I had to move the sandbox to its own vSwitch and route table before attaching the NAT.
  3. Turning one setting on can turn another off. When I enabled the guardrail on the Model API through the API, the consumer authentication I had already enabled was switched off. The update replaces the whole configuration instead of patching it. The same thing happened again when I bound the HTTPS domain to the Model API. Send the setting you are changing and authentication in the same update, and after any change confirm that a call without a key still returns 401.
  4. Saving a rule is not done until the second dialog. After I edited the masking rule and clicked OK, the gateway kept running the previous rule until I also clicked OK in the Confirm to Modify Rule dialog.
  5. Masking working does not mean restore works. With deny_openai: false and only deny_jsonpath set, the values were masked before the model, but the response still came back masked even with restore turned on. Changing to deny_openai: true restored the original values. When you add masking, check the response the caller receives, not only what the model receives.
  6. Python network calls return misleading errors in a locked sandbox. Python urllib hung on DNS resolution for more than 15 seconds in the blocked environment, and the sandbox proxy returned a 502 first. For tests that check whether something is blocked, curl --max-time gives clean results.

What I did not verify

  • An attacker-side DNS receive log: I confirmed from the sandbox-side responses that the query carrying the fake key reached an external authoritative name server. I did not run the experiment of capturing a receive log on a domain I operate.
  • DNS Boundary Firewall: it fits environments that need per-domain control, but I did not test it here.

Wrapping up

Locking down an agent sandbox does not end with isolating execution. You reduce the exits to one, then put authentication, inspection, and cost control on that one. In this setup the work is divided like this.

  • Route table: the sandbox subnet has no internet route
  • Security group: inside the VPC the sandbox can reach only the gateway on 443, and DNS is not left open
  • AI Gateway: holds the real key and applies authentication, injection blocking, masking, and quotas
  • MCP Tool Hook: inspects the tools the agent calls

The Hook watches what the agent is trying to do. The gateway and the network watch where the result goes. With only one of them, things leak through the other side. With both, one can fail and the other still holds.

References

0 0 0
Share on

Hosung Kim

10 posts | 1 followers

You may also like