Call a service by using NLB - Platform For AI - Alibaba Cloud Documentation Center

After enabling VPC direct connection, you can associate your Elastic Algorithm Service (EAS) service with a Network Load Balancer (NLB) instance created on Alibaba Cloud. This integration allows for efficient management and scheduling of your EAS service with NLB. It can also leverage the access control capabilities of NLB, ideal for high-traffic services that require load balancing. This topic describes how to associate an NLB instance.

Prerequisites

An NLB instance has been created. For more information, see Create and manage an NLB instance.
A virtual private cloud (VPC), a vSwitch, and a security group have been established to enable VPC direct connection. Make sure that the security group is not occupied by other services and that the vSwitch has a sufficient number of available IPs for the instances. For more information, see Create and manage a VPC and Manage security groups.

Billing

NLB usage incurs charges, see NLB billing rules.

How it works

Create an NLB instance within the designated VPC and vSwitch. Make sure the network environment matches that used for the VPC direct connection.
During service deployment or updates, activate VPC direct connection and associate the NLB instance with the listener port. EAS will attach the elastic network interface (ENI) under the vSwitch to the EAS pod and add a TCP listener to the NLB instance, enabling access to the EAS service using NLB domain:listener port.

Mount NLB

To associate an NLB instance, include the following key parameters in the JSON configuration file when deploying or updating a service. For information about how to deploy or update EAS services, see Deploy a model service in the PAI console and Run commands to use the EASCMD client.

"cloud": {
        "networking": {
            "vpc_id": "vpc-bp15jr4ggze4rvow39****",
            "vswitch_id": "vsw-bp11dobkizy0jjsgn****",
            "security_group_id": "sg-bp1dz9lad9s0615r****"
        }
    },
"networking": {
        "nlb": [
            {
               "id": "nlb-5q4sp7u6oorkha****",
                "listener_port": 9090
            }
        ]
    }

Important

As shown in the above code, the configuration for the NLB instance (networking) should be at the same level as cloud.

Parameter			Description
cloud	networking	vpc_id	Enable VPC direct connection by configuring the VPC, vSwitch, and security group. Important Use the same VPC as the NLB instance. Make sure that the security group is not used by other cloud products and that the number of available IPs in the vSwitch matches the number of instances. The EAS instance IP and its 8080 port are within the authorized inbound IP and port range of the security group.
		vswitch_id
		security_group_id
networking	nlb	id	The ID of the created NLB instance.
		listener_port	A custom port. When deploying a service, EAS will automatically create it as the listener port of the NLB instance. After the service is successfully deployed, you can access the EAS service through `NLB domain name:listener port`. Note Because the EAS engine listens on the 8080 port, the port must not be 8080. Use a port not listened to by NLB to avoid conflicts.

Verify network connectivity

After successful service deployment, confirm the network connectivity of NLB domain name:listener port:

To view the NLB domain name, log on to the NLB console and check the Domain Name column for the desired instance.
Use the command telnet domain name listener port to test the connectivity of the NLB domain. The listener port is the listener_port you specified during service deployment, and the domain name is checked in the previous step.
- For internet network NLB, run the command on your local terminal.
- For intranet network NLB, run the command on a terminal within the VPC.
A sample result is shown below:

You can then access the EAS service using the NLB domain name and port. Sample command:

# Sample command  
curl http://<NLB domain>:<listener port>/api/predict/<service_name> -H 'Authorization:<service_token>' -d '[{"What is the capital city of Zhejiang?"}]'
# Sample result
{"response":"The capital city of Zhejiang is Hangzhou.","history":[["[{\"What is the capital city of Zhejiang?\"}]","The capital city of Zhejiang is Hangzhou."]]}

Where:

The service access address is http://<NLB domain name>:<listener port>/api/predict/<service_name>. To view it:
1. Click the desired service name on the Elastic Algorithm Service (EAS) page
2. Click View Endpoint Information in the Basic Information section.
3. Substitute the shared gateway domain address in the endpoint with <NLB domain name>:<listener port>.
  - <NLB domain name>: The domain name checked earlier.
  - <listener port>: Use the listener_port specified during the service deployment.
<service_token>: Take steps a and b to view it.