All Products
Search

This Product

    Platform For AI:Deploy a high-performance service with a decoupled frontend and backend

    Document Center

    Platform For AI:Deploy a high-performance service with a decoupled frontend and backend

    Last Updated:Sep 09, 2026

    Compared to the WebUI version, the decoupled frontend/backend version offers higher performance. The backend instance handles 80% of the full-stack computing workload. By using lossless acceleration technology, a single backend instance can manage traffic from up to eight frontend instances, which increases throughput and reduces latency by 25%. This topic shows you how to deploy the decoupled frontend/backend version of CosyVoice2 on Alibaba Cloud PAI-EAS (Elastic Algorithm Service).

    Overview

    This solution deploys the high-concurrency version of CosyVoice2 and offers the following benefits:

    • Enterprise-grade deployment: It uses a decoupled architecture to separate Hift/Flow from Qwen2LM. This allows you to deploy the most time-consuming component, Qwen2LM, independently.

    • High throughput: A high-performance inference engine increases throughput. A single Qwen2LM instance can support multiple Hift/Flow instances, which saves GPU resources and reduces costs.

    • Low latency: Compared to the open-source implementation, this solution reduces end-to-end latency for a single request by 25%.

    image

    If you only want to try the features, you can deploy the CosyVoice2.0 WebUI version directly. For more information, see Quickly Deploy a WebUI Service.

    Limitations

    The WebUI only supports a single frontend instance. To access the service from multiple frontend instances simultaneously, you must use API calls.

    Deployment

    Scenario-based

    1. On the Inference Service tab, click Deploy Service. In the Scenario-based Model Deployment area, click Deploy CosyVoice for AI Speech Generation.

    2. In the Basic Information section, configure the Service Name. For Version selection, choose High performance. Then, configure the parameters for the backend and frontend instances.

      Frontend instance

      Parameter

      Description

      Environment Information

      Deployment version

      Select an image based on the resource type. This topic uses cosyvoice-frontend:0.2.0-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04.

      Note

      Image versions are updated frequently. The versions available on the console take precedence. If multiple versions are listed, select the latest one.

      Storage Mount

      To deploy multiple frontend instances, you must mount external storage to store uploaded audio files or fine-tuned model files. The following example uses Standard NAS:

      • Select a file system: Select an existing file system. If none is available, click Create NAS File System.

      • Mount Target: Select a mount target. If none is available, click Create a mount point.

      • File System Path: Enter a subpath within the file system, such as /.

      • Mount Path: Specify the mount path inside the service instance, such as /mnt/data/.

      Note

      If high stability is not required, such as in a test environment, you can skip mounting external storage to reduce costs.

      Command to Run, Port

      After you select an image, the system automatically configures the command and port number. Do not change them.

      Note that in the Command to Run, --data_dir specifies the mount directory, which must be consistent with the mount path in the Storage Mount.

      Resource Information

      Resource Type

      This solution uses Public Resources.

      Number of Replicas

      Configure based on the invocation method:

      • For WebUI calls: Set to 1.

      • For API calls: A single backend instance supports up to eight frontend instances. If you have one backend instance, the number of frontend instances can range from 1 to 8.

      Deployment

      You must select a GPU instance type with at least 16 GB of GPU memory. Examples include ecs.gn8is.4xlarge and ml.gu8is.c16m128.1-gu60.

      Configure a system disk

      We recommend a 100 GiB system disk. This size helps prevent deployment failures caused by insufficient storage for the large image file. If you do not set this parameter, PAI-EAS allocates 100 GiB by default for this scenario.

      VPC configuration

      If you have configured a NAS file system, the VPC is configured automatically. You only need to select a security group.

      Backend instance

      Parameter

      Description

      Environment Information

      Deployment version

      Select an image based on the resource type. This solution uses cosyvoice-backend:0.2.0-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04.

      Note

      Image versions are updated frequently. The versions available on the console take precedence. If multiple versions are listed, select the latest one.

      Command to Run, Port

      After you select an image, the system automatically configures the command and port number. Do not change them.

      Resource Information

      Resource Type

      This solution uses Public Resources.

      Number of Replicas

      Configure as needed. One backend instance can handle up to eight frontend instances. This solution sets the value to 1.

      Deployment

      You must select a GPU instance type with at least 16 GB of GPU memory. Examples include ecs.gn8is.4xlarge and ml.gu8is.c16m128.1-gu60.

      Configure a system disk

      We recommend a 100 GiB system disk. This size helps prevent deployment failures caused by insufficient storage for the large image file. If you do not set this parameter, PAI-EAS allocates 100 GiB by default for this scenario.

      VPC configuration

      Select a VPC, vSwitch, and security group. The frontend and backend instances must be in the same VPC. If no VPC is available, see Create and manage VPCs and Manage security groups.

    3. After you configure the parameters, click Deploy. When the Service Status changes to Running, the deployment is successful.

    Custom

    For a custom deployment, you must deploy the backend instance first, and then deploy the frontend instance.

    Step 1: CosyVoice-Backend

    1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).

    2. Click Deploy Service In the Inference Service section, click Custom Deployment in the Custom Model Deployment section.

    3. On the Custom Deployment page, configure the following key parameters. After you configure the parameters, click Deploy.

      • Deployment Method: Select Image-based Deployment.

      • Image Configuration: In the Alibaba Cloud Image list, select cosyvoice-backend > cosyvoice-backend:0.2.0-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04.

        Note

        Image versions are updated frequently. The versions available on the console take precedence. If multiple versions are listed, select the latest one.

      • Resource Type: This solution uses Public Resources.

      • Number of Replicas: Configure as needed. One backend instance can handle up to eight frontend instances. This solution sets the value to 1.

      • Command to Run, Port Number: After you select an image, the system automatically configures the command and port number. Do not change them.

      • Deployment: You must select a GPU instance type with at least 16 GB of GPU memory. Examples include ecs.gn8is.4xlarge and ml.gu8is.c16m128.1-gu60.

      • Configure a system disk: We recommend a 100 GiB system disk. This size helps prevent deployment failures caused by insufficient storage for the large image file. If you do not set this parameter, PAI-EAS allocates 100 GiB by default for this scenario.

      • VPC configuration: Select a VPC, vSwitch, and security group. Ensure that the frontend instance you create later is in the same VPC. For more information, see Create and manage VPCs and Manage security groups.

    4. After the deployment is complete (about 3 minutes), obtain the VPC endpoint and Token. You will need them to connect the frontend instance.

      1. Click the target service name. In the Basic Information section, click View Endpoint Information.

      2. In the Invocation Information panel, go to the Shared Gateway tab. Record the VPC endpoint and Token. You can click the eye icon to view the token and the copy icon to copy it.

        Note

        When connecting a frontend instance to a backend instance, do not use the public endpoint. Using the public endpoint is slower and incurs additional charges.

    Step 2: CosyVoice-Frontend

    1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).

    2. Click Deploy Service In the Inference Service section, click Custom Deployment in the Custom Model Deployment section.

    3. On the Custom Deployment page, configure the following key parameters. After you configure the parameters, click Deploy.

      • Deployment Method: Select Image-based Deployment and select the Enable Web App checkbox.

      • Image Configuration: In the Alibaba Cloud Image list, select cosyvoice-frontend > cosyvoice-frontend:0.2.0-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04 .

        Note

        Image versions are updated frequently. The versions available on the console take precedence. If multiple versions are listed, select the latest one.

      • Storage Mount: To deploy multiple frontend instances, you must mount external storage to store uploaded audio files or fine-tuned model files. The following example uses Standard NAS:

        • File System: Select an existing file system. If none is available, click Create NAS File System.

        • Mount Target: Select a mount target. If none is available, click Create a mount point.

        • File System Path: Enter a subpath within the file system, such as /.

        • Mount Path: Specify the mount path inside the service instance, such as /mnt/data/.

        Note

        If high stability is not required, such as in a test environment, you can skip mounting external storage to reduce costs.

      • Command to Run: After you select an image, the system automatically provides a command. Update the following parameters in the command: /bin/bash /tmp/entry.sh --action=start_frontend --port=9000 --data_dir=/mnt/data/ --model_dir=/nasmnt/models/pretrained_models/CosyVoice2-0.5B/ --ttsfrd_dir=/nasmnt/models/pretrained_models/CosyVoice-ttsfrd/ --llm_api_key=token --llm_base_url=http://cosyvoice-server-ip:8000/v1

        • --workers: Sets the number of workers for the frontend service. If not specified, the system allocates workers based on the selected resource specifications. To access the WebUI page from a browser, you must explicitly set --workers to 1.

        • --port: The service port number. This must match the port number configured for the service.

        • --data_dir: The mount directory. This must match the Mount path in the Storage Mount settings.

        • --model_dir: The model loading path. If you use external storage for fine-tuned model files, adjust this path according to your mount path.

        • --llm_api_key: Set this to the Token of the CosyVoice-Backend service. For example, Yjk4YjdlNjM1YW*****GIxZDRmZmNhMjRjZmQwMz*****.

        • --llm_base_url: Set this to the VPC endpoint of the CosyVoice-Backend service, and append /v1 to the end. For example, http://11577032709*****.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/cosyvoice_backend1/v1.

      • Resource Type: This solution uses Public Resources.

      • Deployment: You must select a GPU instance type with at least 16 GB of GPU memory. Examples include ecs.gn8is.4xlarge and ml.gu8is.c16m128.1-gu60.

      • Number of Replicas: Configure based on the invocation method:

        • For WebUI calls: Set to 1.

        • For API calls: A single backend instance supports up to eight frontend instances. If you have one backend instance, the number of frontend instances can range from 1 to 8.

      • Configure a system disk: We recommend a 100 GiB system disk. This size helps prevent deployment failures caused by insufficient storage for the large image file. If you do not set this parameter, PAI-EAS allocates 100 GiB by default for this scenario.

      • VPC configuration: If you have configured a NAS file system, the VPC is configured automatically. You only need to select a security group. Ensure that it is in the same VPC as the backend instance.

    Generate audio

    Important

    Scenario-based deployment aggregates the two services from a custom deployment into a single service that appears on the Inference Service tab. Note the following differences:

    • Custom deployment: Access the WebUI and make API calls through the frontend instance service.

    • Scenario-based deployment: The WebUI is not supported. You must make API calls by using the invocation information of the aggregated service.

    Use the API

    Make API calls using the invocation information from the frontend instance service (for custom deployments) or the aggregated service (for scenario-based deployments). For details, see API Reference.

    FAQ

    Q: Why a 404 {"detail":"Not Found"} error on API calls?

    A 404 error typically indicates an incorrect request path. When you deploy a service with a decoupled frontend and backend, you must direct API calls to the frontend service. This error occurs if you use the backend service endpoint. Check that your request path points to the frontend service.