Deploy EAS services using a dedicated gateway - Platform For AI

Shared Gateways use shared bandwidth and fixed access policies, which often fail to provide the isolation and elasticity that high-concurrency services require. To address this, Elastic Algorithm Service (EAS) offers dedicated gateways. A dedicated gateway offers flexible access control for public or internal networks, supports custom domain names, and provides dedicated bandwidth to improve service stability and reliability.

Overview and selection

EAS offers two types of dedicated gateways:

Application Load Balancer (ALB) dedicated gateway: This gateway is based on Application Load Balancer (ALB) to provide Layer 7 traffic management for HTTP and HTTPS protocols. It features auto scaling, high security and reliability, and intelligent routing.
Important
We recommend using an ALB dedicated gateway in production environments for better performance, stability, and scalability.
Fully-managed dedicated gateway: This is the original dedicated gateway offered by EAS. You can configure network settings, custom domain names, and other gateway requirements directly in the Platform for AI (PAI) console. The network architecture is shown in the following figure:

Important

When a service uses a dedicated gateway, Auto Scaling cannot scale up from zero Instances. You must set the minimum number of Instances to a value greater than 0.

Billing

ALB dedicated gateway: You are billed for the associated ALB Instance. For more information, see ALB billing rules.
Fully-managed dedicated gateway:
- The gateway itself supports both pay-as-you-go and subscription billing methods. For more information, see Elastic Algorithm Service (EAS) billing.
- Accessing the service through an internal network incurs additional PrivateLink costs, including instance fees and data processing fees. For more information about billing, see PrivateLink billing.
- If you access the service through the public network, public network traffic is billed through Cloud Data Transfer (CDT). For more information, see the CDT console.

1. Create and configure a dedicated gateway

[Recommended] ALB dedicated gateway

1.1 Create an ALB gateway

First, create a logical configuration for the gateway. This step does not create actual cloud resources or incur any costs.

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Gateway tab, click Create Dedicated Gateway and select Application Load Balancer.
The system checks for the required service-linked role permissions. If the role is not activated, follow the prompts to grant authorization.
Enter a gateway name and click Submit.

1.2 Create and associate an ALB instance to enable network access

After creating the gateway, you need to enable internal or public network access for it. This action automatically creates and associates an ALB Instance under your account, and billing begins.

Important

To enable both public and internal network access for an ALB dedicated gateway, you must select the same VPC. Additionally, any EAS service deployed through this gateway must use the same VPC.

On the Inference Gateway tab, click the name of the ALB gateway you just created to open its details page.
In the Gateway Access Control section, you will see the VPC and Internet tabs.
Enable VPC (internal network) access
1. On the VPC tab, click Add VPC.
2. In the configuration panel that appears, select the appropriate VPC and vSwitches for your service. To ensure high availability (HA), select at least two vSwitches in different zones. You can select only one vSwitch per zone.
  Important
  If public network access is already enabled for this gateway, the VPC you select here must be the same.
3. Click OK. The system will begin creating the ALB Instance.
Enable public network access
1. Switch to the Internet tab and click Enable Public Network.
2. In the configuration panel that appears, select the appropriate VPC and vSwitches for your service. To ensure HA, select at least two vSwitches in different zones. You can select only one vSwitch per zone.
  Important
  If internal network access is already enabled for this gateway, the VPC you select here must be the same.
3. Click OK. The system will begin creating the ALB Instance.

Fully-managed dedicated gateway

A fully-managed dedicated gateway supports the following features:

Access control: Control public and internal network access through a whitelist.
Custom domain name access: Configure custom domain names and certificates to provide external services.
Cross-account VPC access: Allow servers within a VPC in a different account but the same region to access EAS services through an internal network address.
Authoritative DNS resolution: Use authoritative resolution for the gateway's domain name when calling EAS services from other clouds or on-premises data centers. This requires a network connection to be established with Alibaba Cloud.

1.1 Create a fully-managed dedicated gateway

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Gateway tab, click Create Dedicated Gateway and select Fully-managed Dedicated Gateway.
On the EAS dedicated gateway purchase page, configure the parameters. Refer to Appendix: Dedicated gateway capacity planning to select the gateway specification to ensure service stability.
After configuring the parameters, click Buy Now. Follow the on-screen instructions to confirm your order and complete the payment.
You can view your purchased fully-managed dedicated gateway in the Inference Gateway list. You can start using it when its Status is Running.

Note

You can update the gateway specification and the number of gateway nodes after a fully-managed dedicated gateway is created. The changes take effect in about 3 to 5 minutes.

1.2 Configure access control

On the Inference Gateway tab, click the name of the target fully-managed dedicated gateway to go to its details page. Configure the settings in the Gateway Access Control section.

Public network access control

On the Internet tab, turn on the Access Entry switch. When the status is Enabled, the public access channel for the fully-managed dedicated gateway is open.
By default, the fully-managed dedicated gateway is not accessible from the public network. You must click Add To Whitelist and enter the public IP address ranges that are allowed access (for example, 192.0.2.0/24).
- Separate entries with a comma (,) or a line break.
- To allow access from any public IP address, add the 0.0.0.0/0 address range. You can add up to 15 address ranges.
Verify the public network connectivity of the fully-managed dedicated gateway. For example, add the public IP address of your local machine to the whitelist.
1. On the Internet tab, find the Domain Name Address.
2. In your local terminal, access the domain name address. If the output is similar to the following, it indicates that the whitelisted address can access the dedicated gateway over the public network.
Disable public network access for the fully-managed dedicated gateway.
1. On the Internet tab, turn off the Access Entry switch to disable public network access for the gateway.
2. In your local terminal, access the domain name address. If the output is similar to the following, it confirms that public network access to the gateway is disabled.

Internal network access control

On the VPC tab, click Add VPC and select the VPC and vSwitches you want to connect.
- You can add a VPC from a different account in the same region. After you add the VPC of Account B, servers in that VPC can access the EAS service that uses this dedicated gateway through the VPC address.
  Note
  This is a whitelist feature. To use this feature, you must submit a ticket.
- Authoritative DNS resolution is supported. This is used when calling EAS services from other clouds or on-premises data centers and requires a pre-established network connection with Alibaba Cloud. Currently, authoritative domain resolution can only be used in one VPC configuration.
When you add a VPC, the system adds a default whitelist entry of 0.0.0.0/0 for that VPC, allowing access from all IP addresses within it. You can Modify Whitelist as needed.
Verify the internal network connectivity of the dedicated gateway.
1. On the VPC tab, find the Domain Name Address.
2. On a terminal machine within the VPC, access the domain name address. If the output is similar to the following, it indicates that whitelisted addresses can access the dedicated gateway over the internal network.
  Note
  Within a VPC, any zone can access the dedicated gateway if it is on the whitelist, not just the zones of the vSwitches added to the gateway.
Close the VPC access channel for the dedicated gateway.
1. In the VPC list, click Delete in the VSwitch Operation column to disable VPC access to the dedicated gateway.
2. On a terminal machine within the VPC, access the domain name address. If the output is similar to the following, it confirms that internal network access to the gateway is disabled.

1.3 Configure a custom domain name

(Optional) Manage digital certificates. If you plan to use HTTPS, you must first upload or purchase an SSL Certificate for your custom domain in the Certificate Management Service. You can then configure this certificate in the dedicated gateway.
1. Log on to the Certificate Management Service console and choose SSL Certificate Management.
2. If your domain does not have a certificate, you can select Purchase Certificate or upload an existing one. For more information, see Purchase an SSL Certificate and Upload an SSL Certificate.
Configure public and internal custom domain names.
Configure a public custom domain name
1. On the dedicated gateway details page, switch to the Domain Names tab, click Create Domain Name, and configure the parameters as shown in the following figure.
  If a service is already deployed using this dedicated gateway, you must wait a short period (up to 5 minutes) for the public custom domain name to take effect after it is configured. Check the service invocation information. If the domain in the Public Endpoint is the public custom domain name you configured, the setting has taken effect.
2. Configure public domain name resolution. Add a CNAME record for the public custom domain name to point it to the gateway's public domain name.
  1. On the Gateway Details tab of the dedicated gateway, find the gateway's public domain name address.
  2. Take Alibaba Cloud DNS as an example. The process is similar for other cloud providers. Log on to the Alibaba Cloud DNS console. On the Authoritative Domain Names tab, find your custom domain name. If the domain name is not registered with Alibaba Cloud, you must add it manually. Click the domain name to go to the DNS Settings page and click Add Record. Set Record Type to CNAME. Set Host to your custom domain name. Set Value to the public domain name of the dedicated gateway from Step a. For more information, see Add a domain name and Add a DNS record.
Configure an internal custom domain name
1. On the dedicated gateway details page, switch to the Domain Names tab and click Create Domain Name. Refer to the following configuration.
2. If a service is already deployed using this dedicated gateway, you must wait a short period (up to 5 minutes) after the private custom domain name is configured. Check the service invocation information. If the domain in the VPC Endpoint is the private domain name you configured, the setting has taken effect.

Note

You can set a dedicated gateway as the default gateway. The system will automatically select it for subsequent service deployments.

2. Attach a dedicated gateway to a service

This section shows how to bind a dedicated gateway when deploying a new service. For existing services, you can update the service to modify its bound gateway.

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Service tab, click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.
In the Network information section, select Dedicated Gateway and choose the gateway that you created from the drop-down list.
Important
If you use an ALB dedicated gateway, the service and the gateway must be in the same VPC.

3. Test service invocation

On the Inference Services tab, find the target service and click Invocation Method in the Service Type column. On the Dedicated Gateway tab, you can find the Public Endpoint, VPC Endpoint, and Token.

You can use a curl command to send a request and verify that the response data is correct.

Public invocation: Run the command on your on-premises device.
Internal invocation: Run the command on a terminal within the VPC.

curl <Endpoint_URL> -H'Authorization:<token>'

The following figure shows a test of a GET request with no parameters. The expected return value is True.

Public invocation

Advanced management and monitoring

ALB dedicated gateway: For advanced network configuration and monitoring, go to the Application Load Balancer (ALB) console to centrally manage the gateway with maximum flexibility.
Fully-managed dedicated gateway: Logging and monitoring are disabled by default. To use these features, go to the gateway details page. On the Log or Monitoring tab, click Enable Now. If the underlying service (such as Log Service) is not active, you will be prompted to activate it before you can enable gateway monitoring or logging.

FAQ

Error when adding a VPC: Vswitch vsw-2zeqwh8hv0gb96zcd**** in zone cn-beijing-g is not supported, supported zones: [cn-beijing-i cn-beijing-l cn-beijing-k]
This error occurs because the selected vSwitch is in an unsupported zone. You must select a vSwitch that is in a supported zone.

References

For more information about service access methods, see Overview of invocation methods.
Properly configuring the timeout for a dedicated gateway prevents resource exhaustion and request stacking, and improves the user experience. For configuration details, see Configure the timeout for a dedicated gateway.