Alibaba Cloud Speech Synthesis API reference - Intelligent Speech Interaction

The speech synthesis service converts input text to binary audio data over a WebSocket connection.

Features

Supports PCM, WAV, and MP3 output formats.
Supports configuring the speaking rate, pitch, and volume.
Supports synthesizing up to 300 characters per request. A Chinese character, English letter, punctuation mark, or interword space counts as one character. Content beyond 300 characters is truncated.
Input text must be UTF-8 encoded.

Note

Timestamp support: The service can return the position of each Chinese character or English word in the audio. You can use the timestamps for digital-human lip synchronization and video subtitles. For more information, see Introduction to the speech synthesis timestamp feature.

Endpoint

Access type	Description	URL
Internet access	All servers can access the service over the Internet. The SDK uses this URL by default.	`wss://nls-gateway-ap-southeast-1.aliyuncs.com/ws/v1`

Interaction flow

Note

The preceding figure shows the WebSocket interaction flow. For the RESTful API interaction flow, see RESTful API.
In addition to the audio stream, each server response contains the task_id parameter in the header. The parameter uniquely identifies the request.
To play the audio stream in real time, use a player that supports streaming playback, such as ffmpeg, pyaudio for Python, AudioFormat for Java, or MediaSource for JavaScript.

Authenticate
The client uses a token for authentication when it establishes a WebSocket connection to the server.

Start synthesis

The client sends a speech synthesis request and configures the request parameters. The following table describes the parameters.

Parameter	Type	Required	Description
appkey	String	Yes	consoleThe appkey of the project created in the console.
text	String	Yes	The text to synthesize. The text must be UTF-8 encoded and cannot exceed 300 characters. Separate English words with spaces. Note Only voices that support multiple emotions can use the <emotion> tag. Otherwise, the request returns the error Illegal ssml text.
voice	String	No	The voice. Default: `xiaoyun`.
format	String	No	The audio format. Valid values: .pcm, .wav, and .mp3. Default: `pcm`.
sample_rate	Integer	No	The audio sample rate. Default: 16000 Hz.
volume	Integer	No	The volume. Valid values: 0 to 100. Default: 50.
speech_rate	Integer	No	The speaking rate. Valid values: -500 to 500. Default: 0. The values [-500, 0, 500] correspond to the speed multipliers [0.5, 1.0, 2.0]. -500 specifies 0.5 times the default speaking rate. 0 specifies the default speaking rate. The default rate varies by voice and is approximately four Chinese characters per second. 500 specifies twice the default speaking rate. Use the following formulas: 0.8× speed: (1-1/0.8)/0.002 = -125 1.2× speed: (1-1/1.2)/0.001 = 166 Note For a speed multiplier below 1, use the coefficient 0.002. For a speed multiplier above 1, use the coefficient 0.001. The algorithm rounds the result to an approximate value.
pitch_rate	Integer	No	The pitch. Valid values: -500 to 500. Default: 0.
enable_subtitle	Boolean	No	Specifies whether to enable character-level timestamps. For more information, see Introduction to the speech synthesis timestamp feature.

Receive synthesized audio
The server returns synthesized audio as a binary stream. Use the SDK to receive and process it.

Complete synthesis

After synthesis is complete, the server sends a SynthesisCompleted event, as shown in the following example.

{
    "header": {
        "message_id": "05450bf69c53413f8d88aed1ee60****",
        "task_id": "640bc797bb684bd6960185651307****",
        "namespace": "SpeechSynthesizer",
        "name": "SynthesisCompleted",
        "status": 20000000,
        "status_message": "GATEWAY|SUCCESS|Success."
    }
}

Note

The sample saves the synthesized audio to a file. For lower playback latency, use streaming playback to play the audio as it arrives.

Handle synthesis failures

If synthesis fails because of invalid parameters or another error, the client receives a TaskFailed event, as shown in the following example. The underlying connection is closed after the event is received.

{
   "header":{
      "namespace":"Default",
      "name":"TaskFailed",
      "status":41020001,
      "message_id":"62c126f7d9b340deb82b5b7eaca0****",
      "task_id":"4552df26d1f547aab9a2c4a94678****",
      "status_text":"TTS:TtsClientError:[tts]Engine return error code: 418"
   }
}

Voices

Name	Value of the voice parameter	Type	Scenario	Supported language	Supported sampling rate (Hz)	Phoneme boundary detection for each character or word	Remarks
Xiaoyun	Xiaoyun	Standard female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	No	None
Xiaogang	Xiaogang	Standard male voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	No	None
Ruoxi	Ruoxi	Gentle female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K/24K	No	None
Siqi	Siqi	Gentle female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K/24K	Yes	None
Sijia	Sijia	Standard female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K/24K	No	None
Sicheng	Sicheng	Standard male voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K/24K	Yes	None
Aiqi	Aiqi	Gentle female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aijia	Aijia	Standard female voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aicheng	Aicheng	Standard male voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aida	Aida	Standard male voice	All scenarios	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Ning'er	Ninger	Standard female voice	All scenarios	Chinese only	8K/16K/24K	No	None
Ruilin	Ruilin	Standard female voice	All scenarios	Chinese only	8K/16K/24K	No	None
Siyue	Siyue	Gentle female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K/24K	Yes	None
Aiya	Aiya	Harsh female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aixia	Aixia	Amiable female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aimei	Aimei	Sweet female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aiyu	Aiyu	Natural female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aiyue	Aiyue	Gentle female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Aijing	Aijing	Harsh female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	None
Xiaomei	Xiaomei	Sweet female voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K/24K	No	None
Aina	Aina	Female voice with Zhejiang accent	Customer service	Chinese only	8K/16K	Yes	None
Yina	Yina	Female voice with Zhejiang accent	Customer service	Chinese only	8K/16K/24K	No	None
Sijing	Sijing	Harsh female voice	Customer service	Chinese only	8K/16K/24K	Yes	None
Sitong	Sitong	Child voice	Scenarios in which child voices are required	Chinese only	8K/16K/24K	No	None
Xiaobei	Xiaobei	Little girl voice	Scenarios in which child voices are required	Chinese only	8K/16K/24K	Yes	None
Aitong	Aitong	Child voice	Scenarios in which child voices are required	Chinese only	8K/16K	Yes	None
Aiwei	Aiwei	Little girl voice	Scenarios in which child voices are required	Chinese only	8K/16K	Yes	None
Aibao	Aibao	Little girl voice	Scenarios in which child voices are required	Chinese only	8K/16K	Yes	None
Harry	Harry	Male voice with British accent	English only	English only	8K/16K	No	None
Abby	Abby	Female voice with American accent	English only	English only	8K/16K	No	None
Andy	Andy	Male voice with American accent	English only	English only	8K/16K	No	None
Eric	Eric	Male voice with British accent	English only	English only	8K/16K	No	None
Emily	Emily	Female voice with British accent	English only	English only	8K/16K	No	None
Luna	Luna	Female voice with British accent	English only	English only	8K/16K	No	None
Luca	Luca	Male voice with British accent	English only	English only	8K/16K	No	None
Wendy	Wendy	Female voice with British accent	English only	English only	8K/16K/24K	No	None
William	William	Male voice with British accent	English only	English only	8K/16K/24K	No	None
Olivia	Olivia	Female voice with British accent	English only	English only	8K/16K/24K	No	None
Shanshan	Shanshan	Voice of a female that speaks Cantonese	Scenarios in which dialects are used	Cantonese (simplified) and bilingual (Cantonese and English)	8K/16K/24K	No	None
Xiaoyue	Xiaoyue	Female voice with Sichuan accent	Scenarios in which dialects are used	Chinese or bilingual (Chinese and English)	8K/16K	No	In public preview
Lydia	Lydia	Female voice with bilingual (Chinese and English)	English only	English only	8K/16K	No	In public preview
Aishuo	Aishuo	Natural male voice	Customer service	Chinese or bilingual (Chinese and English)	8K/16K	Yes	In public preview
Qingqing	Qingqing	Voice of a female that speaks Taiwanese	Scenarios in which dialects are used	Chinese only	8K/16K	No	In public preview
Cuijie	Cuijie	Voice of a female that speaks Northeastern Mandarin	Scenarios in which dialects are used	Chinese only	8K/16K	No	In public preview
Xiaoze	Xiaoze	Male voice with strong Hunan accent	Scenarios in which dialects are used	Chinese only	8K/16K	Yes	In public preview

Status codes

Each response contains a status field that indicates the service status code. The following tables describe the status codes.

Common errors

Status code	Cause	Solution
40000001	The client failed to pass the authentication.	Check whether the token that is used by the client is valid or expired.
40000002	The request is invalid.	Check whether the request that is sent by the client meets the requirements.
403	The token is expired or the request contains invalid parameters.	Check whether the token that is used by the client is expired. Then, check whether the parameter values are valid.
40000004	The client timed out.	Check whether the client did not send data to the server for a long period of time, such as 10 seconds.
40000005	The number of requests exceeds the upper limit.	Check whether the number of concurrent connections or queries per second (QPS) value exceeds the upper limit. If the number of concurrent connections exceeds the upper limit, we recommend that you upgrade Intelligent Speech Interaction from Trial Edition to Commercial Edition. If you use Commercial Edition, we recommend that you purchase more resources to increase the concurrency.
40000000	A client error occurred. This is the default status code for client errors.	Resolve the error based on the error message or submit a ticket.
50000000	A server error occurred. This is the default status code for server errors.	If the status code is occasionally returned, ignore it. If the status code is returned multiple times, submit a ticket.
50000001	An internal call error occurred.	If the status code is occasionally returned, ignore it. If the status code is returned multiple times, submit a ticket.

Gateway errors

Status code	Cause	Solution
40010001	The method is not supported.	If you use the SDK, submit a ticket.
40010002	The instruction is not supported.	If you use the SDK, submit a ticket.
40010003	The instruction format is invalid.	If you use the SDK, submit a ticket.
40010004	The client unexpectedly disconnected.	Check whether the client disconnected before the server completed the requested task.
40010005	The task status is invalid.	Check whether the instruction is supported when the task is in the current state.

Configuration errors

Status code	Cause	Solution
40020105	The application does not exist.	Check whether the appkey is correct and belongs to the same Alibaba Cloud account as the token.

Text-to-speech (TTS) service errors

Status code	Cause	Solution
41020001	One or more parameters are invalid.	Check whether the specified parameter values are valid.
51020001	A TTS server error occurred.	If the status code is occasionally returned, ignore it. If the status code is returned multiple times, submit a ticket.