API Gateway:Connect to Vertex AI via AI Gateway

1. Prepare a GCP Service Account

Before you start the configuration, you must create a Service Account in the Google Cloud Console and download its JSON key file.

1. Log on to the Google Cloud Console.

2. In the navigation pane on the left, choose IAM & Admin > Service Accounts.

3. Click Create service account. Enter a name and description.

4. Assign the Vertex AI User role or a higher-level role to the service account.

5. Click KEY and select Create to download the key JSON file.

Note The key file contains sensitive information. You must store it securely. The following code block shows a sample JSON format:

{
  "type": "service_account",
  "project_id": "your-project-id",
  "private_key_id": "your-private-key-id",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "your-sa@your-project.iam.gserviceaccount.com",
  "token_uri": "https://oauth2.googleapis.com/token"
}

For more information about how to manage GCP Service Accounts, see the Google documentation.

2. Create an AI service

1. Log on to the AI Gateway console.

2. In the navigation pane on the left, choose Instances. Then, select a region from the top navigation bar.

3. On the Instances page, click the ID of the target Instance.

4. In the navigation pane on the left, click Services, and then click the Services tab.

5. Click Create service. In the Create service panel, configure the AI service with the following parameters:

OpenAI protocol compatibility modes:

Important:

• If the JSON key format is incorrect or lacks required fields, the system displays an error message.

• The vertexProjectId field is read-only. The frontend automatically parses the project_id field from the JSON key.

• When you use the Vertex AI native compatible endpoint mode, you must add the provider prefix google/ to the model parameter in the request body. For example, you must change gemini-3-flash-preview to google/gemini-3-flash-preview.

For more information about the supported parameters and models in the Vertex AI native compatible endpoint mode, see the Google documentation.

3. Create a Model API

1. Log on to the AI Gateway console.

2. In the navigation pane on the left, choose Instances. Then, select a region from the top navigation bar.

3. On the Instances page, click the ID of the target Instance.

4. In the navigation pane on the left, click Model API, and then click Create Model API.

5. In the Create Model API panel, configure the following basic settings:

   • Domain name: Configure a custom domain name. If you use the default environment domain, rate limiting may be triggered.

   • Base Path: The base path for the API, such as /gemini.

   • Model type: Select Text generation.

   • Protocol: Select OpenAI-compatible.

   • Routing configuration: Select the /v1/chat/completions route.

   • AI request monitoring: Enable this feature.

   • Service model: Select Single-model service.

   • Service list:

     • Service name: Select the Vertex AI service that you configured in the previous step.

     • Model name: Select pass-through or specify a model, such as gemini-2.0-flash.

6. Click OK.

4. Debug Model API

1. In the Actions column of the target Model API, click Test.

2. In the Test panel, enter the name of a Gemini-series model, such as gemini-3-flash-preview, in the Model name field. Then, you can use the Model response tab to chat with the large language model.

Important The Model response tab uses the /v1/chat/completions chat interface. To test other interfaces, you can use the cURL command or Raw output options with cURL or an SDK.

cURL example:

curl -X POST "http://your-gateway-domain/gemini/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3-flash-preview",
    "messages": [
      {"role": "user", "content": "Hello. Please introduce yourself."}
    ],
    "stream": false
  }'

1. Get a Vertex AI Express Mode API key

For more information, see the Vertex AI in express mode documentation.

Note Express Mode API keys may have usage limits. For more information, see the Google documentation.

2. Create an AI service

1. Log on to the AI Gateway console.

2. In the navigation pane on the left, choose Instances. Then, select a region from the top navigation bar.

3. On the Instances page, click the ID of the target Instance.

4. In the navigation pane on the left, click Services, and then click the Services tab.

5. Click Create service. In the Create service panel, configure the AI service with the following parameters:

Note The configuration for Express Mode is similar to that for other large language model providers, such as OpenAI and Claude. You only need to enter the API key.

3. Create a Model API

1. In the navigation pane on the left, click Model API, and then click Create Model API.

2. In the Create Model API panel, configure the following basic settings:

   • Domain name: Configure a custom domain name. If you use the default environment domain, rate limiting may be triggered.

   • Base Path: The base path for the API, such as /gemini.

   • Model type: Select Text generation.

   • Protocol: Select OpenAI-compatible.

   • Routing configuration: Select the /v1/chat/completions route.

   • AI request monitoring: Enable this feature.

   • Service model: Select Single-model service.

   • Service list:

     • Service name: Select the Vertex AI Express service that you configured in the previous step.

     • Model name: Select pass-through.

3. Click OK.

4. Test the Model API

1. In the Actions column of the target Model API, click Test.

2. In the Test panel, select a Gemini model to test the chat function.

Use cases

• Use image generation models, such as Imagen 4.

• Use video generation models, such as Veo 3.

• Use third-party models that are hosted in the Vertex AI Model Garden.

• Call the native REST API of Vertex AI directly.

1. Create an AI service

The procedure for creating an AI service is similar to that in Scenario 1 or Scenario 2, but with the following key differences:

1. Log on to the AI Gateway console.

2. In the navigation pane on the left, choose Instances. Then, select a region from the top navigation bar.

3. On the Instances page, click the ID of the target Instance.

4. In the navigation pane on the left, click Services, and then click the Services tab.

5. Click Create service. In the Create service panel, configure the AI service with the following parameters:

Important:

• You must select Native Protocol. This means that you will use the native REST API format of Vertex AI.

• After you select Native Protocol, the OpenAI protocol compatibility option does not appear because no protocol conversion occurs.

2. Create a Model API

1. In the navigation pane on the left, click Model API, and then click Create Model API.

2. In the Create Model API panel, configure the following basic settings:

   • Domain name: Configure a custom domain name. If you use the default environment domain, rate limiting may be triggered.

   • Base Path: The base path for the API, such as /vertex-api.

   • Use case: Select a use case as needed. This setting is used only for categorization in the console. Mixing categories has no effect.

     • Text generation: For text models, such as Llama.

     • Image generation: For image generation models, such as Imagen 4.

     • Video generation: For video generation models, such as Veo 3.

   • Protocol: Select VertexAI.

   • Routing configuration: After you select the VertexAI protocol, the gateway automatically configures the following two built-in routes:

     Route path	Purpose
     /{api-version}/publishers/{publisher}/models/{model}:{action}	Path format used by Express Mode.
     /{api-version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:{action}	Path format used by Service Account mode.

   • Service Model: Single Model Service.

   • Service list:

     • Service name: Select the AI service that you created in the previous step, such as vertex-rest-api.

     • Model name: Select pass-through.

3. Click OK.

3. Call examples

After the configuration is complete, you can make calls using the native REST API format of Vertex AI.

Generate an image using Imagen 4 (Express Mode):

curl -X POST "https://your-gateway-domain/vertex-api/v1/publishers/google/models/imagen-4.0-generate-preview-06-06:predict" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "instances": [
      {
        "prompt": "A cute orange cat napping in the sunshine."
      }
    ],
    "parameters": {
      "sampleCount": 1,
      "aspectRatio": "1:1"
    }
  }'

Generate a video using Veo 3 (Express Mode):

curl -X POST "https://your-gateway-domain/vertex-api/v1/publishers/google/models/veo-3.0-generate-preview:predictLongRunning" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "instances": [
      {
        "prompt": "Slow-motion video of an orange cat running on grass."
      }
    ],
    "parameters": {
      "aspectRatio": "16:9",
      "durationSeconds": 8
    }
  }'

Supported models

Note For a complete list of supported models, see the Google documentation.

1. Create a DNS service

Because the SDK handles authentication, you must create a DNS-type service that points directly to the Vertex AI backend.

1. Log on to the AI Gateway console.

2. In the navigation pane on the left, choose Instances. Then, select a region from the top navigation bar.

3. On the Instances page, click the ID of the target Instance.

4. In the navigation pane on the left, click Services, and then click the Services tab.

5. Click Create service. In the Create service panel, configure the DNS service with the following parameters:

Note

• Use the global endpoint aiplatform.googleapis.com:443 for Express Mode.

• Use the regional endpoint {location}-aiplatform.googleapis.com:443 for Service Account mode. Fill in the region where your GCP project resides.

2. Create a Model API

1. In the navigation pane on the left, click Model API, and then click Create Model API.

2. In the Create Model API panel, configure the following basic settings:

   • Domain name: Configure a custom domain name. If you use the default environment domain, rate limiting may be triggered.

   • Base Path: The base path for the API, such as /vertex. This parameter is optional.

   • Model type: Select Text generation or Image generation.

   • Protocol: Select VertexAI.

   • Routing configuration: After you select the VertexAI protocol, the gateway automatically configures the following two built-in routes:

     | Route path | Purpose |

     |------------------------------------------------------------------------------------------------|---------|

     | /{api-version}/publishers/{publisher}/models/{model}:{action} | Path format used by Express Mode |

     | /{api-version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:{action} | Path format used by Service Account mode |

   • Service Model: Single model.

   • Service list:

     • Service name: Select the DNS service that you created in the previous step, such as vertex-dns.

     • Model name: Select pass-through.

3. Click OK.

3. Configure the GenAI SDK

In your Python code, set the base_url of the GenAI SDK to the AI Gateway address:

from google import genai

# Configure API key (Express Mode) or use default credentials (Service Account)
API_KEY = "your-api-key"

# Gateway address = domain name + Base Path (if configured)
# Example: If domain name is your-gateway-domain.com and Base Path is /vertex
base_url = "https://your-gateway-domain.com/vertex"

# Create a client to proxy requests to AI Gateway
client = genai.Client(
    vertexai=True,  # Currently supports only vertexai=True
    api_key=API_KEY,
    http_options={
        'base_url': base_url,
    },
)

# Call the model
response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="Hello. Please introduce yourself."
)

print(response.text)

Configuration notes:

Using Service Account mode:

If you authenticate using a Service Account, you can configure credentials using environment variables:

# Set the path to the Service Account key file
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"

from google import genai

# No api_key needed in Service Account mode
base_url = "https://your-gateway-domain.com/vertex"

client = genai.Client(
    vertexai=True,
    http_options={
        'base_url': base_url,
    },
)

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="Hello. Please introduce yourself."
)

print(response.text)

4. Verify the connection

After the configuration is complete, you can view request metrics in the AI Gateway console:

1. In the navigation pane on the left, click Observability & Analysis > Log Center.

2. Review the requests that are sent through the GenAI SDK, including the following information:

   • Request path

   • Response status code

   • Response time

   • Token usage (if included in the response)

Note Because authentication occurs on the client side, the gateway cannot proxy authentication. For full AI Gateway functionality, we recommend that you use Scenario 1, Scenario 2, or Scenario 3.

Key configuration points

• AI service: Select Vertex AI as the large language model provider. You can use either a GCP Service Account or Express Mode.

• Model API: Select Text generation as the use case. Select OpenAI-compatible as the protocol. Select the /v1/chat/completions route.

Image understanding

Method 1: Use an image URL

from openai import OpenAI

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1"
)

completion = client.chat.completions.create(
    model="gemini-3-flash-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "How do I solve this problem?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://img.alicdn.com/imgextra/i1/O1CN01gDEY8M1W114Hi3XcN_!!6000000002727-0-tps-1024-406.jpg"
                    },
                },
            ],
        }
    ],
)

print(completion.choices[0].message.content)

Method 2: Use Base64-encoded images

import base64
from openai import OpenAI

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1"
)

# Encode a local image as Base64
def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")

# Specify the image path
image_path = "<your-image-path>"
base64_image = encode_image(image_path)

completion = client.chat.completions.create(
    model="gemini-3-flash-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{base64_image}",
                    },
                },
            ],
        }
    ],
)

print(completion.choices[0].message.content)

Supported image formats: image/png, image/jpeg, image/webp, image/heic, and image/heif

For more information about model-specific capabilities, see the Google documentation.

Video understanding

Note: Although the OpenAI API specification defines only the image_url type, you can still send video content. AI Gateway handles compatibility conversion and correctly passes video content to Vertex AI.

Method 1: Use a video URL

from openai import OpenAI

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1"
)

completion = client.chat.completions.create(
    model="gemini-3-flash-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Briefly describe the video content."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4"
                    },
                },
            ],
        }
    ],
)

print(completion.choices[0].message.content)

Method 2: Use Base64-encoded videos

import base64
from openai import OpenAI

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1"
)

# Encode a local video as Base64
def encode_video(video_path):
    with open(video_path, "rb") as video_file:
        return base64.b64encode(video_file.read()).decode("utf-8")

# Specify the video path
video_path = "example.mp4"
base64_video = encode_video(video_path)

completion = client.chat.completions.create(
    model="gemini-3-flash-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Briefly describe the video content."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:video/mp4;base64,{base64_video}",
                    },
                },
            ],
        }
    ],
)

print(completion.choices[0].message.content)

Supported video formats: video/x-flv, video/quicktime, video/mpeg, video/mpegs, video/mpg, video/mp4, video/webm, video/wmv, and video/3gpp

For more information about model-specific capabilities, see the Google documentation.

Code Description

Key configuration points

The configuration is similar to that in Scenario 1 or Scenario 2, but with the following key differences:

Important: When you generate, edit, or create variations of images using the GCP Service Account mode, you must select AI Gateway conversion for the OpenAI protocol compatibility mode. The native compatible endpoints of Vertex AI support only the Chat Completions interface. The protocol conversion of AI Gateway supports image-related interfaces.

Interface support matrix

Image generation (/v1/images/generations)

After the configuration is complete, you can call the image generation interface using the OpenAI Python SDK:

from openai import OpenAI
from PIL import Image
from io import BytesIO
import base64

# Create a client pointing to the AI Gateway address
client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1"
)

# Call the image generation interface
response = client.images.generate(
    model="gemini-3-pro-image-preview",  # Gemini Nano Banana Pro model
    prompt="A cute orange cat napping in the sunshine.",
    size="1024x1024",
    n=1
)

# Get the generated image (Base64 encoded)
image_data = response.data[0].b64_json
if image_data:
    # Decode and save the image locally
    image = Image.open(BytesIO(base64.b64decode(image_data)))
    image.save("output_folder/example-image-cat.png")
    print("Image saved to output_folder/example-image-cat.png")

Code notes:

Size parameter mapping:

The OpenAI API uses resolution, such as 1024x1024, while Vertex AI uses aspect ratio (aspectRatio) and image size level (imageSize). Because of this difference, AI Gateway automatically performs parameter conversion.

Aspect ratios supported by Vertex AI: 1:1, 3:2, 2:3, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, and 21:9

Resolution levels supported by Vertex AI: 1k, 2k, and 4k

cURL example:

curl -X POST "https://your-gateway-domain/gemini-image/v1/images/generations" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3-pro-image-preview",
    "prompt": "A cute orange cat napping in the sunshine.",
    "size": "1024x1024",
    "n": 1
  }'

Image editing (/v1/images/edits)

After the configuration is complete, you can use the OpenAI Python SDK to upload the original image and edit it using multipart:

from openai import OpenAI
from PIL import Image
from io import BytesIO
import base64

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1",
    api_key="YOUR_API_KEY"
)

# Upload the original image using the file upload method (recommended by OpenAI SDK)
with open("assets/image-cat.png", "rb") as image_file:
    response = client.images.edit(
        model="gemini-3-pro-image-preview",
        prompt="Change the cat breed to Ragdoll while keeping the same pose.",
        image=image_file,
        size="1024x1024",
        n=1
    )

image_data = response.data[0].b64_json
if image_data:
    image = Image.open(BytesIO(base64.b64decode(image_data)))
    image.save("output_folder/edited-image-cat.png")
    print("Image editing completed. Saved to output_folder/edited-image-cat.png")

# 1) Convert a local image to Base64 (remove line breaks)
BASE64_IMAGE=$(base64 < assets/image-cat.png | tr -d '\n')

# 2) Call the edits interface using image_url (data URL)
curl -X POST "https://your-gateway-domain/{model-api-base-path}/v1/images/edits" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d "{
    \"model\": \"gemini-3-pro-image-preview\",
    \"prompt\": \"Change the cat breed to Ragdoll while keeping the same pose.\",
    \"image_url\": {
      \"url\": \"data:image/png;base64,${BASE64_IMAGE}\"
    },
    \"size\": \"1024x1024\",
    \"n\": 1
  }"

Image variations (/v1/images/variations)

After the configuration is complete, you can use the OpenAI Python SDK to generate image variations using multipart:

from openai import OpenAI
from PIL import Image
from io import BytesIO
import base64

client = OpenAI(
    base_url="http(s)://{your-domain}/{model-api-base-path}/v1",
    api_key="YOUR_API_KEY"
)

with open("assets/image-cat.png", "rb") as image_file:
    response = client.images.create_variation(
        model="gemini-3-pro-image-preview",
        image=image_file,
        size="1792x1024",
        n=1
    )

image_data = response.data[0].b64_json
if image_data:
    image = Image.open(BytesIO(base64.b64decode(image_data)))
    image.save("output_folder/variation-image-cat.png")
    print("Image variation completed. Saved to output_folder/variation-image-cat.png")

If you prefer to use image_url (URL or data URL) instead of file uploads for variations, you can use HTTP JSON requests directly:

# 1) Convert a local image to Base64 (remove line breaks)
BASE64_IMAGE=$(base64 < assets/image-cat.png | tr -d '\n')

# 2) Call the variations interface using image_url (data URL)
curl -X POST "https://your-gateway-domain/{model-api-base-path}/v1/images/variations" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d "{
    \"model\": \"gemini-3-pro-image-preview\",
    \"image_url\": {
      \"url\": \"data:image/png;base64,${BASE64_IMAGE}\"
    },
    \"size\": \"1792x1024\",
    \"n\": 1
  }"

Supported image-generation models

Note For a complete list of supported image generation models, see the Google documentation.

Multi-provider high availability

To improve availability, you can configure Vertex AI and other providers in a fallback relationship:

1. Create multiple AI services, for example, Vertex AI and Alibaba Cloud Model Studio.

2. When you create a Model API, select Multi-model service (by model name).

3. Configure routing rules:

   • Service name: vertex-ai. Model name pattern: gemini*

   • Service name: bailian. Model name pattern: qwen*

4. Enable Fallback and select a backup service.

Security best practices

1. Credential management: Store Service Account keys or API keys in Alibaba Cloud KMS instead of in plain text to reduce the risk of credential exposure.

2. Access control: Configure IP address whitelists or other access control policies to restrict the sources of API access.

3. Rate limiting: Set reasonable rate limiting rules to prevent unexpectedly high costs.

401 or 403 authentication failure

• Check the integrity of the Service Account JSON key. Make sure that it includes required fields, such as private_key, client_email, and token_uri.

• Confirm that the Service Account has the Vertex AI User role or a higher-level role.

• For Express Mode, confirm that the API key is valid and has not expired.

404 model not found

• Confirm that the model name is correct, such as gemini-3-flash-preview or gemini-3-pro-preview.

• Check whether the model is supported in the selected region.

JSON parsing failure

• Make sure that you pasted the complete content of the Service Account JSON file. Do not omit the opening { or closing }.

• You can verify the JSON format using an online JSON validator.

Request timeout

• Check whether the VPC is configured with a public NAT Gateway.

• Confirm the network connectivity to the target region.