Shared gateways have shared bandwidth and fixed access policies, which makes it difficult for them to meet high demands for concurrency, isolation, and elasticity. To solve this issue, Elastic Algorithm Service (EAS) offers dedicated gateways. A dedicated gateway provides flexible public or private network access control, supports custom domain names, and uses dedicated bandwidth to ensure your service is stable and reliable.
Overview and selection
EAS provides two types of dedicated gateways:
Application Load Balancer (ALB) dedicated gateway: This gateway is based on Application Load Balancer (ALB) and provides Layer 7 traffic management. It supports the HTTP and HTTPS protocols and features automatic scaling, high security, high reliability, and smart routing.
ImportantUse an ALB dedicated gateway in a production environment for better performance, stability, and extensibility.
Fully-managed dedicated gateway: This is the original EAS dedicated gateway. You can configure network settings, custom domain names, and other gateway requirements directly in the PAI console.
When a service uses a dedicated gateway, Auto Scaling does not support scaling out from zero instances. The minimum number of instances must be greater than 0.
Billing
ALB dedicated gateway: Charges are incurred by the associated ALB instance. For more information, see ALB billing rules.
Fully-managed dedicated gateway:
The gateway itself supports the pay-as-you-go and subscription billing methods. For more information, see Elastic Algorithm Service (EAS) billing.
If you access the service over an internal network, additional PrivateLink fees apply. These fees include instance fees and data processing fees. For more information about billing, see PrivateLink billing.
If you access the service over the internet, the generated Internet traffic is billed through Cloud Data Transfer (CDT). For more information, see the CDT console.
1. Create and configure a dedicated gateway
[Recommended] ALB dedicated gateway
1.1 Create an ALB gateway
First, create a logical configuration for the gateway. This step does not create actual cloud resources or incur costs.
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Gateway tab, click Create Dedicated Gateway and select Application Load Balancer.
The system checks your permissions for the service-linked role. If the role is not activated, follow the on-screen instructions to grant the required permissions.
Enter a name for the gateway and click Submit.
1.2 Create and associate an ALB instance to enable network access
After you create the gateway, you must enable internal or public network access for it. This action automatically creates and associates an ALB instance with your account, and billing starts.
For an ALB dedicated gateway, you must select the same VPC for both public and internal network access. The EAS service that is deployed using this gateway must also be configured with the same VPC.
In the list on the Inference Gateway tab, click the name of the ALB gateway that you created to go to its details page.
In the Gateway Access Control section, you can view the VPC and Internet tabs.
Enable VPC (internal network) access
On the VPC tab, click Add VPC.
In the configuration panel that appears, select a VPC and vSwitches that meet your business requirements. To ensure high availability, select at least two vSwitches in different zones. You can select only one vSwitch per zone.
ImportantIf public network access is already enabled for the gateway, the VPC that you select here must be the same.
Click OK. The system starts to create the ALB instance.
Enable public network access
Switch to the Internet tab and click Enable Public Network.
In the configuration panel that appears, select a VPC and vSwitches that meet your business requirements. To ensure high availability, select at least two vSwitches in different zones. You can select only one vSwitch per zone.
ImportantIf VPC access is already enabled for the gateway, the VPC that you select here must be the same.
Click OK. The system starts to create the ALB instance.
Fully-managed dedicated gateway
A fully-managed dedicated gateway supports the following features:
Access control: You can control public and internal network access using a whitelist.
Custom domain name access: You can configure custom domain names and certificates to provide external services.
Cross-account VPC access: You can allow servers in a VPC that belongs to a different account in the same region to access the EAS service through an internal network address.
Authoritative DNS resolution: You can use authoritative DNS resolution for the gateway domain name when you invoke the EAS service from other clouds or on-premises data centers. This requires a connection to the Alibaba Cloud network.
1.1 Create a fully-managed dedicated gateway
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Gateway tab, click Create Dedicated Gateway and select Fully-managed Dedicated Gateway.
On the EAS Dedicated Gateway purchase page, configure the parameters. To ensure service stability, select a gateway specification. For more information, see Appendix: Dedicated gateway capacity planning.
After you configure the parameters, click Buy Now. Follow the on-screen instructions to confirm the order and complete the payment.
You can view the purchased fully-managed dedicated gateway in the inference gateway list. When its Status is Running, the gateway is ready to use.
After a fully-managed dedicated gateway is created, you can update its specification and the number of nodes. The changes take about 3 to 5 minutes to take effect.
1.2 Configure access control
On the Inference Gateway tab, click the name of the target fully-managed dedicated gateway to go to its details page. Configure the settings in the Gateway Access Control section.
Public network access control
On the Internet tab, turn on the Access Entry switch. When the status is Enabled, the public access channel for the fully-managed dedicated gateway is open.
By default, the fully-managed dedicated gateway is not accessible from the internet. Click Add To Whitelist and enter the public IP address ranges that are allowed access, such as 192.0.2.0/24.
Separate entries with a comma (,) or a line feed.
To allow access from any public IP address, add the 0.0.0.0/0 address range. You can add up to 15 address ranges.
Verify the public network connectivity of the fully-managed dedicated gateway. For example, you can add the public IP address of your on-premises device to the whitelist.
On the Internet tab, find the Domain Name Address.
On your on-premises device, access the domain name address. If the output is similar to the following, the whitelisted address range can access the dedicated gateway over the internet.
Close the public access channel for the fully-managed dedicated gateway.
On the Internet tab, turn off the Access Entry switch to disable public access to the gateway.
On your on-premises device, access the domain name address. If the output is similar to the following, the public access channel for the gateway is closed.
Internal network access control
On the VPC tab, click Add VPC and select the VPC and vSwitches that you want to connect.
You can add a VPC from a different account in the same region. After you add the VPC of Account B, servers in that VPC can access the EAS service that uses this dedicated gateway through the VPC address.
NoteThis is a whitelist feature. To use this feature, you must submit a ticket.
Authoritative DNS resolution for the gateway domain name is supported. You can use this feature when you invoke the EAS service from other clouds or on-premises data centers. You must first establish a connection to the Alibaba Cloud network. Currently, you can use authoritative DNS resolution in the configuration of only one VPC.
When you add a VPC, the system configures a default whitelist of 0.0.0.0/0 for that VPC. This allows access from all IP addresses within the VPC. You can click Modify Whitelist as needed.
Verify the internal network connectivity of the dedicated gateway.
On the VPC tab, find the Domain Name Address.
On a terminal within the VPC, access the domain name address. If the output is similar to the following, the whitelisted address range can access the dedicated gateway over the internal network.
NoteWithin the VPC, you can configure a whitelist to allow access to the dedicated gateway from any zone, not just the zones of the vSwitches that are added to the gateway.
Close the VPC access channel for the dedicated gateway.
In the VPC list, click Delete in the VSwitch Operation column to disable VPC access to the dedicated gateway.
On a terminal within the VPC, access the domain name address. If the output is similar to the following, the internal access channel for the dedicated gateway is closed.
1.3 Configure a custom domain name
(Optional) Manage digital certificates. If you use the HTTPS protocol to access the service, you must first manage the SSL Certificate for your custom domain name in the Certificate Management Service console before you can configure it in the dedicated gateway.
Log on to the Certificate Management Service console and choose SSL Certificate Management.
If your domain name does not have a certificate, you can click Purchase Certificate or upload an existing one. For more information, see Purchase an SSL Certificate and Upload an SSL Certificate.
Configure public and internal custom domain names.
Configure a public custom domain name
On the dedicated gateway details page, switch to the Domain Names tab, click Create Domain Name, and configure the parameters as shown in the following figure.
If a service is already deployed using this dedicated gateway, the public custom domain name takes effect after a short wait (less than 5 minutes). Check the service invocation information. If the domain name in the public endpoint is the public custom domain name that is configured for the gateway, the setting has taken effect.
Configure public domain name resolution. Add a CNAME record for the public custom domain name to point it to the public domain name of the dedicated gateway.
On the Gateway Details tab of the dedicated gateway, view the public domain name address of the gateway.
Take Alibaba Cloud DNS as an example. The process is similar for other cloud providers. Log on to the Alibaba Cloud DNS console. On the Authoritative Domain Names tab, find your custom domain name. If the domain name is not registered with Alibaba Cloud, you must add it manually. Click the domain name to go to the DNS Settings page and click Add Record. Set Record Type to CNAME. Set Host to your custom domain name. Set Value to the public domain name of the dedicated gateway from Step a. For more information, see Add a domain name and Add a DNS record.
Configure an internal custom domain name
On the dedicated gateway details page, switch to the Domain Names tab and click Create Domain Name. Refer to the following configuration.
If a service is already deployed using this dedicated gateway, the internal custom domain name takes effect after a short wait (less than 5 minutes). Check the service invocation information. If the domain name in the VPC endpoint is the private domain name that is configured for the gateway, the setting has taken effect.
If you set a dedicated gateway as the default gateway, the system automatically selects it when you deploy services.
2. Attach a dedicated gateway to a service
The following steps describe how to deploy a new service with a custom image from the console. For an existing service, you can update the service to change the attached gateway.
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
On the Inference Service tab, click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.
In the Network information section, select Dedicated Gateway and choose the gateway that you created from the drop-down list.
ImportantIf you use an ALB dedicated gateway, the service and the gateway must be in the same VPC.
3. Test service invocation
On the Inference Services tab, find the target service and click Invocation Method in the Service Type column. On the Dedicated Gateway tab, you can find the Public Endpoint, VPC Endpoint, and Token.
You can use a curl command to send a request and verify that the response data is correct.
Public invocation: Run the command on your on-premises device.
Internal invocation: Run the command on a terminal within the VPC.
curl <Endpoint_URL> -H'Authorization:<token>'The following figure shows a test of a GET request with no parameters. The expected return value is True.
Advanced management and monitoring
ALB dedicated gateway: For advanced network configuration and monitoring, go to the Application Load Balancer (ALB) console to centrally manage the gateway with maximum flexibility.
Fully-managed dedicated gateway: Logging and monitoring are disabled by default. To use these features, go to the gateway details page. On the Log or Monitoring tab, click Enable Now. If a required service is not activated, first activate the service, and then click Enable in the lower-right corner.
FAQ
Error when adding a VPC:
Vswitch vsw-2zeqwh8hv0gb96zcd**** in zone cn-beijing-g is not supported, supported zones: [cn-beijing-i cn-beijing-l cn-beijing-k]This error occurs because the selected vSwitch is in an unsupported zone. You must select a vSwitch that is in a supported zone.
References
For more information about service access methods, see Overview of invocation methods.
Properly configuring the timeout for a dedicated gateway prevents resource exhaustion and request stacking, and improves the user experience. For configuration details, see Configure the timeout for a dedicated gateway.