Platform For AI:EAS FAQ

Q: The service has been stuck in Waiting state for a long time. How do I fix this?

After you deploy a service, it enters the Waiting state while resources are scheduled and instances start. When all instances start successfully, the service transitions to Running state. The following scenarios may cause a prolonged Waiting state:

[2025-07-18 18:50:46] WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
[2025-07-18 18:50:50]  * Serving Flask app 'service'
[2025-07-18 18:50:50]  * Debug mode: off
[2025-07-18 18:50:50] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
[2025-07-18 18:50:50]  * Running on all addresses (0.0.0.0)
[2025-07-18 18:50:50]  * Running on http://127.0.0.1:7000
[2025-07-18 18:50:50]  * Running on http://xxx:7000
[2025-07-18 18:50:50] Press CTRL+C to quit

Q: The service is in Failed state. How do I fix this?

A service enters the Failed state in two situations:

• During deployment: If a resource specified at deployment time (such as a model path) doesn't exist, the service status shows the error reason. The error message usually indicates why the deployment failed.

• During startup: If the service fails after deployment succeeds and scheduling begins, the status shows: Instance <network-test-5ff76448fd-h9dsn> not healthy: Instance crashed, please inspect instance log.

  This message means one or more instances failed to start. Check the Service instances list on the service Overview page to identify the specific cause. Common failure causes include:

  • The instance ran out of memory during startup and the system killed it (OOMKilled). Redeploy the service with a higher memory allocation. In the instance list, the Last status column shows OOMKilled(247), indicating the container was terminated for exceeding the memory limit. Check the Restart count and Last exit reason columns for further confirmation.

  • A code error caused a crash during startup. In this case, Last status shows Error(error code). Click Log in the Actions column of the instance to review the service logs and identify the startup failure.

  • The service image failed to pull. See What do I do if image pull fails (ImagePullBackOff)?

Q: What do I do if image pull fails (ImagePullBackOff)?

If the Last exit reason column in the service instance list shows ImagePullBackOff, the image pull has failed. If an icon appears in the Status column, click it to view the specific reason.

Common image pull failure causes are listed below:

Based on the regions where the image registry and EAS service are located, use the following guidance:

• Same region: Pull images using the private image address.

• Cross-region: ACR Personal Edition supports public image addresses only. For ACR Enterprise Edition, choose based on your requirements:

  • For higher security and reliability, use a private image address. This requires connecting the VPCs via CEN. See Access ACR Enterprise Edition instances cross-region or from IDC.

  • For simpler scenarios or when private network connectivity isn't available yet, use a public image address as a temporary option. Public network downloads are slower.

Additional notes for ACR Enterprise Edition:

• Configure access control for both VPC and public network access as needed.

• If the repository doesn't allow anonymous pulls, cross-region pulls over the public address require configuring the image registry username and password in the EAS service.

Q: The EAS service restarts automatically after being stopped. What's happening?

Problem description: An EAS service restarts on its own after being stopped for a period.

Root cause:

The service has auto-scaling configured with a minimum instance count of 0. When traffic stops for a period of time, the instance count automatically scales down to 0. If a request arrives when no instances are available due to scale-down, EAS triggers a scale-up automatically — this doesn't require the configured scale-up threshold to be reached.

Check the deployment events for auto scaling messages to confirm whether auto scale-up occurred. A Service is now auto scaling entry in the information column indicates the service was triggered to scale up automatically, after which the event status transitions from Waiting through Scaling to Running, and the available instance count changes from 0 to 1.

Solution:

• If the service is no longer needed, delete it.

• To keep the service but prevent automatic restart, stop it through the console or by calling the StopService API. A manually stopped service won't scale up when traffic arrives.

• To prevent auto-scaling from stopping the service, don't set the minimum instance count to 0.

• Disable auto-scaling entirely if unexpected traffic triggering a scale-up is a concern.

Q: PAI-EAS startup fails with IoError(Os { code: 28, kind: StorageFull, message: "No space left on device" })

Problem description:

[2024-10-21 20:59:33] serialize_file(_flatten(tensors), filename, metadata=metadata)

[2024-10-21 20:59:33] safetensors_rust.SafetensorError: Error while serializing: IoError(Os { code: 28, kind: StorageFull, message: "No space left on device" })

[2024-10-21 20:59:35] time="2024-10-21T12:59:35Z" level=info msg="program stopped with status:exit status 1" program=/bin/sh

Root cause: The EAS instance's system disk is full because of large model files. This prevents the service from starting.

Solutions:

Option 1: Expand the system disk for the EAS instance.

Option 2: If the model files are too large, store them in external storage (OSS, NAS, or similar) and read them using Storage mounts.

Q: Deployment fails with fail to start program with error: fork/exec /bin/sh: exec format error

The exec format error means the operating system can't run the target binary. The most common cause is a CPU architecture mismatch between the executable or container image and the host machine.

Try a different resource specification.

Q: Error: Invalid GPU count 6, only supported: [0 1 2 4 8 16]

The number of GPUs per service must be a power of 2 to maximize inter-GPU communication efficiency.

Supported GPU counts per service: 0, 1, 2, 4, 8, or 16.

Call errors

Troubleshoot based on the returned HTTP status code. For details, see Service status codes and errors.

HTTPS and custom domains

Q: Does EAS support HTTPS calls?

Yes. Replace http:// with https:// in the service endpoint URL to enable encrypted transport. If the client (such as the Python requests library) reports an SSL certificate verification error, this is a client-side configuration issue, not an EAS issue.

Q: How do I force HTTPS-only access?

• Shared gateway: HTTPS redirect is not supported.

• Dedicated gateway: Supported. Enable HTTPS redirect in the dedicated gateway configuration. Once enabled, all HTTP requests are automatically redirected to HTTPS.

Q: Can I call a service using a custom domain?

Yes. Create and use a fully managed dedicated gateway, then configure your custom domain in the gateway settings. For details, see Use a dedicated gateway.

Token management

Q: Does the service token expire or change?

No. The Token generated when a service is deployed is long-lived. Restarting, updating (unless you manually change the authentication method), or scaling the service doesn't change the Token. The Token is invalidated only when you manually reset it or delete the service.

Q: Can I create multiple tokens for one service?

No. Each EAS service instance supports a single authentication Token. For multi-user permission management or separate usage tracking, use Alibaba Cloud RAM-based authentication or a similar access control solution.

Other call issues

Q: How do I enable streaming responses for an LLM service?

EAS doesn't have a global streaming toggle. Specify streaming in each API request body individually. For example, when calling an OpenAI-compatible LLM service, include "stream": true in the JSON request body.

Q: What's the difference between VPC address calls and VPC direct connect?

• VPC address calls: Requests go through an internal SLB plus a gateway (public address calls go through a public SLB plus a gateway). This is the standard request path. Each request passes through layer-4 forwarding at the SLB and layer-7 forwarding at the gateway before reaching the service instance. Under high concurrency and heavy traffic, this forwarding adds latency overhead. The gateway also has a bandwidth ceiling (1 Gbps by default).

• VPC direct connect: EAS provides a high-speed direct connect mode that addresses both performance and scalability without additional cost. Enabling VPC direct connect establishes a network path between your VPC and the EAS service VPC. Your requests use EAS service discovery to locate the service, then perform client-side software load balancing. This requires the EAS SDK and setting endpoint_type to DIRECT.

  For example, in the Python SDK scenario, add the following line to your code to switch from gateway calls to direct connect:

  • client = PredictClient('http://pai-eas-vpc.cn-hangzhou.aliyuncs.com', 'mnist_saved_model_example')
    client.set_token('M2FhNjJlZDBmMzBmMzE4NjFiNzZhMmUxY2IxZjkyMDczNzAzYjFi****')
    client.set_endpoint_type(ENDPOINT_TYPE_DIRECT) # Direct link
    client.init()

Q: Why can't a RAM user automatically create or delete the EAS service-linked role?

Only users with specific permissions can create or delete AliyunServiceRoleForPaiEas automatically. If a RAM user can't perform this action, grant them the required policy as follows:

1. Create the following policy as a custom policy using the script-based configuration method. For details, see Create a custom policy.

   Policy for creating or deleting a service-linked role

   {
     "Statement": [
       {
         "Action": "ram:CreateServiceLinkedRole",
         "Resource": "*",
         "Effect": "Allow",
         "Condition": {
           "StringEquals": {
             "ram:ServiceName": "eas.pai.aliyuncs.com"
           }
         }
       }
     ],
     "Version": "1"
   }

2. Attach the custom policy created in the previous step to the target RAM user. For details, see Manage RAM user permissions.

Q: How does an EAS service access the internet?

EAS services don't have internet access by default. To enable internet access, configure a VPC with internet connectivity for the EAS service. See EAS access to public and private resources.

Q: Does EAS support SSH access to instances?

No. EAS is a managed service and doesn't provide SSH access into containers. To run specific commands when the container starts, specify them in the Run Command field of the service configuration.

Q: What service statuses does EAS have?

EAS services have the following statuses. You can also view the status in the status column on the Model Serving (EAS) page.

• Creating

• Waiting — waiting for instances to start

• Stopped

• Failed

• Updating — instances are being updated

• Stopping

• HotUpdate — updating without replacing instances

• Starting

• DeleteFailed

• Running

• Scaling — instances are scaling

• Pending — waiting for processing

• Deleting

• Completed

• Preparing

Q: How do I find out which RAM user created a service?

Query events in the ActionTrail console. Set the event name filter to CreateService. For details, see Query events in the ActionTrail console.

Q: Can I download PAI official images from the internet?

No. PAI official images are platform-internal images and can only be used within the PAI platform. They can't be downloaded outside of platform containers.

Q: What are the uniqueness rules for PAI-EAS service names and service IDs?

• Service name uniqueness: EAS service names are globally unique within a single region. If you see a "service name already exists" error when creating a service but the name doesn't appear in the Model Serving (EAS) list, another user in the same region has taken that name. Choose a different name and try again.

• Service ID uniqueness: PAI-EAS service IDs are globally unique and assigned automatically by the system. You can't specify them manually.

Q: Do I need to repurchase EAS after disabling the module? What if I can't find my service instances?

• No repurchase needed: Once activated, PAI-EAS remains available indefinitely. To use it again, go to the PAI console, select or create a workspace, and create an EAS instance in that workspace. No additional purchase or reactivation is required.

• If you can't find your instances:

  1. Log in to the PAI console and verify that the Workspace shown at the top of the page is the one you're looking for.

  2. If the workspace was deleted or switched, look for the Model Serving (EAS) list from the Overview page or within the target workspace.

  3. If the list shows no data and no instances, no billable EAS instances have been created in that workspace. Activating the module alone doesn't incur charges and requires no additional cleanup.

Q: Why can't I select an OSS bucket when deploying an EAS service?

When deploying an EAS service, you can mount models and code by specifying storage locations. Make sure the OSS bucket and NAS file system are in the same region as the EAS service. Cross-region storage can't be selected.

Q: TensorFlow issues

For details, see TensorFlow FAQ.