CosyVoice 2.0 is a high-fidelity speech synthesis model developed by Alibaba DAMO Academy. It supports voice cloning, including cross-lingual cloning, with as little as 30 seconds of prompt audio. The model is well suited for scenarios such as customer service conversations, audiobook narration, and short video voice-overs. Alibaba Cloud PAI-EAS provides the model through a WebUI so you can quickly deploy a cloud-based speech inference service.
Background
CosyVoice 2.0 is designed to produce natural, warm, and emotionally expressive AI voices. Trained on large-scale speech corpora with refined prosody modeling, it delivers vocal expressiveness comparable to professional human presenters. Whether for a thoughtful greeting in customer service or an expressive reading for an audiobook, CosyVoice 2.0 generates warm, natural speech. It moves beyond the cold, mechanical feel of synthetic audio to deliver a more engaging listening experience.
CosyVoice 2.0 provides the following features:
-
Natural and warm voice: Moves beyond the robotic feel by emulating the rhythm, emotion, and prosody of human speech.
-
Multi-scenario support: Supports customer service conversations, audiobook narration, short video voice-overs, e-commerce voice recommendations, and more.
-
High efficiency and low latency: Lightweight cloud deployment for fast, fluent speech generation.
-
High controllability: Supports tone adjustment, emotion control, and voice character customization to create a distinctive brand voice.
The CosyVoice 2.0 WebUI service described in this topic is for trial use only. You can also use the high-concurrency version of CosyVoice 2.0 for high-performance inference. For more information, see Deploy a high-performance service with a decoupled frontend and backend.
Limits
The pretrained voice inference mode is not supported in this release.
Billing
When you deploy a CosyVoice 2.0 service, you are charged only for resource usage and the system disk. If you no longer need the service, click Stop in the Actions column of the target service to avoid unnecessary costs. For more billing details, see Elastic Algorithm Service (EAS) billing.
Deploy a CosyVoice 2.0 service
Method 1: Scenario-based deployment (recommended)
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).
-
On the Inference Service tab, click Deploy Service. In the Scenario-based Model Deployment area, click Deploy CosyVoice for AI Speech Generation.
-
Configure the following key parameters:
Parameter
Description
Basic Information
Version selection
Select Standard Edition.
Environment Information
Image Version
Select an image that matches your resource type. This topic uses
cosyvoice-webui:3.0.6-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04.NoteImage versions are updated frequently. Select the latest version when you deploy.
Command
After you select an image version, the system automatically configures the following run command:
/bin/bash /tmp/entry.sh --action=start_webui --port=9000 --data_dir=/mnt/data/ --model_dir=/nasmnt/models/pretrained_models/Fun-CosyVoice3-0.5B/ --ttsfrd_dir=/nasmnt/models/pretrained_models/CosyVoice-ttsfrd/ --workers 1. The parameters are described as follows:-
--port: The service port. This must match the port configured for the EAS service.
-
--data_dir: The mount directory used to store reference audio and models. The default is
/mnt/data. If you configure a storage mount, this path must match the mount path specified in Storage Mount. -
--model_dir: The directory from which the model is loaded.
The following parameter is also supported:
-
--gpu_memory_utilization: Sets the GPU memory utilization limit.
Port Number
After you select an image version, the system automatically configures the port
9000. You do not need to change it.Resource Information
Resource Type
This solution uses Public Resources. You can also select other resource types as needed.
Instances
Set this value to 1.
Deployment Resources
For the resource specification, you must select a GPU instance type, such as
ecs.gn8is.4xlargeorml.gu8is.c16m128.1-gu60.Configure a system disk
Because the image file is large, we recommend that you set the system disk to 100 GiB to prevent deployment failures caused by insufficient storage.
If you do not specify a value, the EAS backend defaults to 100 GiB of storage for CosyVoice 2.0.
Network Information
VPC
Configure a VPC if you need to access the service through a VPC direct connection or configure public network access for the service. In the drop-down lists, select a VPC, a vSwitch, and a security group. For information about creating these resources, see Create and manage a VPC and Manage security groups.
-
-
After you configure the parameters, click Deploy.
Image pulling and deployment take approximately 5 to 10 minutes. When the Service Status changes to Running, the service is deployed successfully.
Method 2: Custom deployment
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).
Click Deploy Service In the Inference Service section, click Custom Deployment in the Custom Model Deployment section.
-
On the Custom Deployment page, configure the following key parameters. For information about other parameters, see Custom deployment.
Parameter
Description
Environment Information
Deployment Method
Select Image-based Deployment and select Enable Web App.
Image Configuration
From the Alibaba Cloud Image list, select
cosyvoice-webui:3.0.6-pytorch2.3.1-gpu-py310-cu128-ubuntu22.04.NoteImage versions are updated frequently. Select the latest version when you deploy.
Command
After you select an image, the system automatically configures the following run command:
/bin/bash /tmp/entry.sh --action=start_webui --port=9000 --data_dir=/mnt/data/ --model_dir=/nasmnt/models/pretrained_models/Fun-CosyVoice3-0.5B/ --ttsfrd_dir=/nasmnt/models/pretrained_models/CosyVoice-ttsfrd/ --workers=1. The parameters are described as follows:-
--port: The service port. This must match the port configured for the EAS service.
-
--data_dir: The mount directory used to store reference audio and models. The default is
/mnt/data. If you configure a storage mount, this path must match the mount path specified in Storage Mount. -
--model_dir: The directory from which the model is loaded.
-
--workers: Sets the number of workers for the built-in frontend service. If not specified, the system automatically configures the value based on the resource specification. To access the WebUI from a browser, you must set
--workers 1.
The following parameter is also supported:
-
--gpu_memory_utilization: Sets the GPU memory utilization limit.
Port Number
After you select an image, the system automatically configures the port
9000. You do not need to change it.Resource Information
Resource Type
This solution uses Public Resources. You can also select other resource types as needed.
Instances
Set this value to 1.
Deployment Resources
For the resource specification, you must select a GPU instance type, such as
ecs.gn8is.4xlargeorml.gu8is.c16m128.1-gu60.Configure a system disk
Because the image file is large, we recommend that you set the system disk to 100 GiB to prevent deployment failures caused by insufficient storage.
If you do not specify a value, the EAS backend defaults to 100 GiB of storage for CosyVoice 2.0.
Network Information
VPC
Configure a VPC if you need to access the service through a VPC direct connection or configure public network access for the service. In the drop-down lists, select a VPC, a vSwitch, and a security group. For information about creating these resources, see Create and manage a VPC and Manage security groups.
-
-
After you configure the parameters, click Deploy.
Image pulling and deployment take approximately 5 to 10 minutes. When the Service Status changes to Running, the service is deployed successfully.
Generate audio by using the inference service
Call the service through the API
To generate audio by calling the service through the API, see CosyVoice 2.0 API.