Alibaba Cloud Model Studio:Access APIs over VPC

How it works

After you create an interface endpoint in a VPC, PrivateLink establishes a private, unidirectional connection between your VPC and Model Studio. This connection only allows resources in your VPC to access Model Studio. Model Studio cannot access resources in your VPC.

When computing resources in the VPC access the endpoint, traffic is forwarded to Model Studio through PrivateLink and does not pass through the internet.

The endpoint must be in the same region as the Model Studio service. To use a VPC in another region, see Cross-region private access.

Region:

• Public cloud: Singapore, China (Beijing).

  Private network access is not currently supported in the US (Virginia) region.

Access APIs using an endpoint

# Replace the original domain name with the endpoint service domain name from the previous step.
curl -X POST http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen-flash",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user", 
            "content": "Who are you?"
        }
    ]
}'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace the original domain name with the endpoint service domain name from the previous step.
    base_url="http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
    model="qwen-flash",
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Who are you?'}],
)
print(completion.model_dump_json())

import os
from http import HTTPStatus
# Use DashScope SDK v1.14.0 or later.
import dashscope
from dashscope import Generation

# Replace the original domain name with the endpoint service domain name from the previous step.
dashscope.base_http_api_url = "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1"
dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
messages = [{
    'role': 'user', 'content': 'Who are you?'
}]
response = Generation.call(
    model="qwen-flash",
    messages=messages,
    result_format='message'
)
if response.status_code == HTTPStatus.OK:
    print(response)
else:
    print('Request id: %s, Status code: %s, error code: %s, error message: %s' % (
        response.request_id, response.status_code,
        response.code, response.message
    ))

// Use DashScope SDK v2.12.0 or later.
import java.util.Arrays;

import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.utils.JsonUtils;

public class Main {
    public static GenerationResult callWithMessage() throws ApiException, NoApiKeyException, InputRequiredException {
        // Replace the original domain name with the endpoint service domain name from the previous step.
        Generation gen = new Generation(Protocol.HTTP.getValue(), "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1");
        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .content("You are a helpful assistant.")
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content("Who are you?")
                .build();
        GenerationParam param = GenerationParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-flash")
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }
    public static void main(String[] args) {
        try {
            GenerationResult result = callWithMessage();
            System.out.println(JsonUtils.toJson(result));
        } catch (ApiException | NoApiKeyException | InputRequiredException e) {
            // Print the error message.
            System.err.println("An error occurred while calling the generation service: " + e.getMessage());
        }
    }
}

Cross-region private access

Endpoints must reside in the same region as the Model Studio service. To access the service privately from a VPC in another region, complete the following steps:

1. Follow the instructions in the previous section to configure access to Model Studio APIs using an endpoint.

2. Use Cloud Enterprise Network (CEN) to connect VPCs in different regions. Note:

   • Select VPCs with non-overlapping CIDR blocks at both ends to avoid conflicts that could cause connection failures.

   • To establish cross-region VPC connectivity between the Chinese mainland and other regions using CEN, your account must complete enterprise identity verification.

3. In the security group associated with the endpoint, add an inbound rule to allow traffic on ports 80 and 443 from resources in the requester VPC.

After the configuration is complete, accessing the endpoint’s default service domain name from the requester VPC routes traffic through the Transit Router (TR) to the endpoint in the region where Model Studio is deployed.

By default, the endpoint’s default service domain name is accessible directly from the interconnected cross-region VPC. However, the custom service domain name is valid only within the VPC where the endpoint is located. To access Model Studio APIs from the requester VPC using a custom domain name, see Quickly use private domain name resolution. Create a private domain name that matches the custom service domain name, and resolve it to the endpoint’s default service domain name using a CNAME record:

1. Add an authoritative zone that matches the custom domain name, such as vpc-ap-southeast-1.dashscope.aliyuncs.com. For Effective Scope, choose the requester VPC.

2. Add a DNS record: Set the record type to CNAME, enter @ for the hostname, and enter the endpoint’s default domain name for the record value, such as ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   Note: When configuring private domain name resolution, do not use underscores (_) in the host record or the full domain name. A domain name can contain only letters, numbers, and hyphens (-), such as test-for-dns.dashscope.aliyuncs.com, not test_for_dns.dashscope.aliyuncs.com.

After the configuration is complete, you can access Model Studio APIs from the requester VPC using the custom service domain name. If you use a private domain name that differs from the custom service domain name, see Configure private domain name resolution.

Billing

Using PrivateLink, CEN, and Private Zone may incur additional fees:

• Billing of PrivateLink

• Billing of CEN

• Billing for Private Zone

FAQ

1. Why can't my ECS instance access Model Studio APIs through a private connection?

   Follow these steps to troubleshoot:

   1. Confirm that the resources are in the same VPC.

      If the ECS instance and the endpoint are in different VPCs, you must first configure VPC peering.

   2. Check the security group associated with the endpoint. Confirm that an inbound rule has been added to allow access on port 80 (HTTP) or port 443 (HTTPS) from the CIDR block of the requester ECS instance.

   3. Confirm the endpoint service domain name.

      Private access to the Model Studio through the default service domain name supports only HTTP.

2. Can an endpoint be accessed from the internet?

   No. PrivateLink establishes private connections within the Alibaba Cloud internal network only. Endpoints do not have internet access, and endpoint ENIs cannot be associated with elastic IP addresses (EIPs).

3. Why do I receive an error when I call a model using my custom domain name with internal domain name resolution?

   This issue usually occurs because the host record (or full domain name) used for internal domain name resolution contains invalid characters, such as an underscore (_). Domain names should only contain letters, numbers, and hyphens (-).

   Configure the DNS record as follows:

   1. Authoritative zone: In Private Hosted Zone, add a DNS record for the dashscope.aliyuncs.com authoritative domain name.

   2. Host record: Set Record Type to CNAME, and enter your custom domain prefix, such as test-for-dns-right. Note: Host records cannot contain underscores (_).

      Correct example	Incorrect example

   3. Record value: Enter the default service domain name of the Model Studio endpoint, such as ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   After the configuration is complete, call the model using https://test-for-dns-right.dashscope.aliyuncs.com/api/v1 (or https://test-for-dns-right.dashscope.aliyuncs.com/compatible-mode/v1/chat/completions for OpenAI compatible mode).

   Using a domain name with an underscore, such as https://test_for_dns_wrong.dashscope.aliyuncs.com/api/v1, will result in a call error.