Access Model Studio APIs over a private network by using an endpoint

How it works

After you create an interface endpoint in a VPC, PrivateLink establishes a private endpoint connection between your VPC and Model Studio. This connection is unidirectional: it allows resources in your VPC to access Model Studio, but Model Studio cannot use this connection to access resources in your VPC.

When compute resources in your VPC access the endpoint, traffic is routed to the Model Studio service through PrivateLink and bypasses the public internet.

To access the service privately from a VPC in a different region, see Cross-region private access.

Model Studio is available in the following regions:

• Public cloud: Singapore and China (Beijing).

  Private network access is not currently supported in the US (Virginia) region.

Accessing APIs via an endpoint

# Replace the original domain name with the endpoint service domain name obtained in the previous step.
curl -X POST http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen-flash",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user", 
            "content": "Who are you?"
        }
    ]
}'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace the original domain name with the endpoint service domain name obtained in the previous step.
    base_url="http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
    model="qwen-flash",
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Who are you?'}],
)
print(completion.model_dump_json())

import os
from http import HTTPStatus
# We recommend that you use DashScope SDK v1.14.0 or later.
import dashscope
from dashscope import Generation

# Replace the original domain name with the endpoint service domain name obtained in the previous step.
dashscope.base_http_api_url = "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1"
dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
messages = [{
    'role': 'user', 'content': 'Who are you?'
}]
response = Generation.call(
    model="qwen-flash",
    messages=messages,
    result_format='message'
)
if response.status_code == HTTPStatus.OK:
    print(response)
else:
    print('Request id: %s, Status code: %s, error code: %s, error message: %s' % (
        response.request_id, response.status_code,
        response.code, response.message
    ))

// We recommend that you use DashScope SDK v2.12.0 or later.
import java.util.Arrays;

import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.utils.JsonUtils;

public class Main {
    public static GenerationResult callWithMessage() throws ApiException, NoApiKeyException, InputRequiredException {
        // Replace the original domain name with the endpoint service domain name obtained in the previous step.
        Generation gen = new Generation(Protocol.HTTP.getValue(), "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1");
        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .content("You are a helpful assistant.")
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content("Who are you?")
                .build();
        GenerationParam param = GenerationParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-flash")
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }
    public static void main(String[] args) {
        try {
            GenerationResult result = callWithMessage();
            System.out.println(JsonUtils.toJson(result));
        } catch (ApiException | NoApiKeyException | InputRequiredException e) {
            // Print the error message.
            System.err.println("An error occurred while calling the generation service: " + e.getMessage());
        }
    }
}

Cross-region private access

Model Studio is deployed in Singapore and China (Beijing). To access Model Studio APIs over a private network from a VPC in another region, choose one of the following methods based on your connection range:

• Cross-region within the Chinese mainland or outside the Chinese mainland (for example, a VPC in Japan (Tokyo) accessing Model Studio in Singapore): Use Method 1: Enable a cross-region endpoint (recommended).

• Cross-border across regions (between the Chinese mainland and other regions, for example, a VPC in Singapore accessing Model Studio in China (Beijing)): Use Method 2: Cross-region VPC connection through CEN.

Method 1: Enable a cross-region endpoint (recommended)

For the full procedure, see Access a cross-region service via an interface endpoint. Key configurations for the Model Studio scenario:

• Region: Select the region of your source VPC.

• Type: Select Alibaba Cloud Service.

• Service Region: Select the Enable cross-region endpoint checkbox and select Singapore or China (Beijing).

• Endpoint Service: Select com.aliyuncs.dashscope from the available services list.

• Cross-region configuration: Cross-region traffic fees are settled and billed by CDT. The default bandwidth depends on the connection range: 1,000 Mbit/s for connections between regions in the Chinese mainland and 100 Mbit/s for connections between regions outside the Chinese mainland.

Configure the remaining parameters as for in-region access. After creating the endpoint, add an inbound rule in the associated security group to allow traffic from the source VPC on ports 80 and 443.

Method 1 does not support connections between the Chinese mainland and other regions. For cross-border access, use Method 2.

After the configuration is complete, when you access the default service domain name of the endpoint from the source VPC, PrivateLink routes traffic directly to the region of the Model Studio service, enabling private cross-region access.

Method 2: Cross-region VPC connection through Cloud Enterprise Network (CEN) — for cross-border scenarios

Use this method for cross-border cross-region access between the Chinese mainland and other regions. The endpoint must be in the same region as the Model Studio service, and CEN connects the source VPC to the endpoint VPC across regions:

1. Complete the configurations described in Accessing APIs via an endpoint.

2. Use Cloud Enterprise Network (CEN) to configure a cross-region VPC connection. Note the following:

   • Select VPCs with different CIDR blocks to avoid connection failures from network conflicts.

   • To establish a cross-region VPC connection between the Chinese mainland and other regions by using CEN, your account must complete enterprise identity verification.

3. In the security group associated with the endpoint, add an inbound rule to allow traffic from the source VPC on ports 80 and 443.

After you complete the configuration, when you access the default service domain name of the endpoint from the source VPC, a Transit Router routes traffic to the endpoint in the region of the Model Studio service. This enables private cross-region access to Model Studio APIs.

By default, the default service domain name of an endpoint is accessible from an interconnected VPC in another region. However, a custom service domain name is valid only within the endpoint's VPC. To access Model Studio APIs from the source VPC by using a custom domain name over a private network, see Quick start with Private DNS. You can create a private domain name that is the same as the custom service domain name and resolve it to the default service domain name of the endpoint by creating a CNAME record.

1. Add a private authoritative domain name that is the same as the custom service domain name, such as vpc-ap-southeast-1.dashscope.aliyuncs.com, and set its scope to the source VPC.

2. Add a DNS record: set Record Type to CNAME, Host to @, and Record Value to the default service domain name of the target endpoint, such as ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   Note: When you configure Private DNS, do not use underscores (_) in the host record or full domain name. Otherwise, API calls may fail. We recommend that domain names contain only letters, numbers, and hyphens (-). For example, use test-for-dns.dashscope.aliyuncs.com instead of test_for_dns.dashscope.aliyuncs.com.

After the configuration is complete, you can access Model Studio APIs from the source VPC by using the custom service domain name. If you use a private domain name that is different from your custom service domain name for the configuration, see Configure Private DNS.

Billing

Using PrivateLink and Private DNS (Private Hosted Zone) incurs additional costs. Cross-border scenarios incur additional CEN cross-region fees. See the following billing documentation to estimate your costs:

• Billing for PrivateLink

• Billing for CEN (cross-border scenarios only)

• Billing for Private DNS

FAQ

1. Why can't my ECS instance access Model Studio APIs over a private network?

   Check the following items:

   1. Confirm that the resources are in the same VPC.

      If the ECS instance and the endpoint are in different VPCs, you cannot access Model Studio APIs over the private network. In this case, you must first configure VPC connectivity.

   2. Check the security group associated with the endpoint. Confirm that an inbound rule allows traffic from the CIDR block of the source ECS instance on port 80 (HTTP) or 443 (HTTPS).

   3. Confirm the endpoint service domain name.

      When you access the Model Studio platform over a private network by using the default service domain name, only HTTP is supported.

2. Can an endpoint be accessed from the public internet?

   No. PrivateLink only establishes private connections within the Alibaba Cloud internal network. Endpoints cannot be accessed from the public internet, and you cannot associate an elastic IP address with an endpoint ENI.

3. When I use Private DNS, why do I receive an error when I call a model by using my custom domain name?

   This issue usually occurs when the host record or full domain name for private domain resolution contains invalid characters, such as an underscore (_). The domain name should consist of only letters, digits, and hyphens (-).

   Configure the DNS record by following these steps:

   1. Authoritative Domain: In Private DNS, add a DNS record for the dashscope.aliyuncs.com authoritative domain.

   2. Host Record: Select CNAME for the record type and enter your custom domain prefix, such as test-for-dns-right. Note: The host record cannot contain underscores (_).

      Correct example	Incorrect example

   3. Record Value: Enter the default service domain name of the Model Studio endpoint. Example: ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   After the configuration is complete, you can call the model at https://test-for-dns-right.dashscope.aliyuncs.com/api/v1 (the endpoint for the OpenAI-compatible mode is https://test-for-dns-right.dashscope.aliyuncs.com/compatible-mode/v1/chat/completions).

   Using a domain name that contains an underscore, such as https://test_for_dns_wrong.dashscope.aliyuncs.com/api/v1, will cause the API call to fail.