All Products
Search
Document Center

Overview

Last Updated: Sep 21, 2020

The speech synthesis service provides the Natural User Interaction (NUI) SDK for mobile clients to convert text to binary speech data.

Description

Compared with common SDKs, the NUI SDK is smaller in size and supports more comprehensive status management. The NUI SDK provides comprehensive speech processing capabilities and can also serve as an atomic SDK, meeting diverse user requirements. In addition, the NUI SDK uses a unified API.

The NUI SDK has the following features:

  • Supports generating audio files in the pulse-code modulation (PCM) and MP3 formats.

  • Allows you to set the speed, intonation, and volume of the generated speeches.

  • Allows you to set the speaker type of the generated speeches. The following table describes the supported speaker type.

Name

Value of the voice parameter

Category

Scenario

Supported language

Supported sampling rate (Hz)

Support phoneme boundary detection for each word

Remarks

XiaoyunXiaoyunStandard female voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000NoN/A
XiaogangXiaogangStandard male voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000NoN/A
RuoxiRuoxiGentle female voiceCommon scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000NoN/A
SiqiSiqiGentle female voiceCommon scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000YesN/A
SijiaSijiaStandard female voiceCommon scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000NoN/A
SichengSichengStandard male voiceCommon scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000YesN/A
AiqiAiqiGentle female voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AijiaAijiaStandard female voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AichengAichengStandard male voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AidaAidaStandard male voiceCommon scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
NingerNingerStandard female voiceCommon scenarioSimplified Chinese8,000, 16,000, and 24,000NoN/A
RuilinRuilinStandard female voiceCommon scenarioSimplified Chinese8,000, 16,000, and 24,000NoN/A
SiyueSiyueGentle female voiceCustomer service scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000NoN/A
AiyaAiyaStrict female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AixiaAixiaAmiable female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AimeiAimeiSweet female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AiyuAiyuNatural female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AiyueAiyueGentle female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
AijingAijingStrict female voiceCustomer service scenarioChinese and mixed Chinese and English8,000 and 16,000YesN/A
XiaomeiXiaomeiSweet female voiceCustomer service scenarioChinese and mixed Chinese and English8,000, 16,000, and 24,000NoN/A
AinaAinaFemale voice with Zhejiang accentCustomer service scenarioSimplified Chinese8,000 and 16,000YesN/A
YinaYinaFemale voice with Zhejiang accentCustomer service scenarioSimplified Chinese8,000, 16,000, and 24,000NoN/A
SijingSijingStrict female voiceCustomer service scenarioSimplified Chinese8,000, 16,000, and 24,000YesN/A
SitongSitongChild voiceChild voice scenarioSimplified Chinese8,000, 16,000, and 24,000NoN/A
XiaobeiXiaobeiLolita female voiceChild voice scenarioSimplified Chinese8,000, 16,000, and 24,000YesN/A
AitongAitongChild voiceChild voice scenarioSimplified Chinese8,000 and 16,000YesN/A
AiweiAiweiLolita female voiceChild voice scenarioSimplified Chinese8,000 and 16,000YesN/A
AibaoAibaoLolita female voiceChild voice scenarioSimplified Chinese8,000 and 16,000YesN/A
HarryHarryMale voice with British accentEnglish scenarioEnglish8,000 and 16,000NoN/A
AbbyAbbyFemale voice with American accentEnglish scenarioEnglish8,000 and 16,000NoN/A
AndyAndyMale voice with American accentEnglish scenarioEnglish8,000 and 16,000NoN/A
EricEricMale voice with British accentEnglish scenarioEnglish8,000 and 16,000NoN/A
EmilyEmilyFemale voice with British accentEnglish scenarioEnglish8,000 and 16,000NoN/A
LunaLunaFemale voice with British accentEnglish scenarioEnglish8,000 and 16,000NoN/A
LucaLucaMale voice with British accentEnglish scenarioEnglish8,000 and 16,000NoN/A
WendyWendyFemale voice with British accentEnglish scenarioEnglish8,000, 16,000, and 24,000NoN/A
WilliamWilliamMale voice with British accentEnglish scenarioEnglish8,000, 16,000, and 24,000NoN/A
OliviaOliviaFemale voice with British accentEnglish scenarioEnglish8,000, 16,000, and 24,000NoN/A
ShanshanShanshanCantonese female voiceDialect scenarioCantonese (simplified) and mixed Cantonese and English8,000, 16,000, and 24,000NoN/A
XiaoyueXiaoyueFemale voice with Sichuan accentDialect scenarioChinese and mixed Chinese and English8,000 and 16,000NoAvailable in public preview of Intelligent Speech Interaction
LydiaLydiaFemale voice of mixed Chinese and EnglishEnglish scenarioEnglish8,000 and 16,000NoAvailable in public preview of Intelligent Speech Interaction
AishuoAishuoNatural male voiceCustomer service scenarioEnglish8,000 and 16,000YesAvailable in public preview of Intelligent Speech Interaction
QingqingQingqingFemale voice with Formosan accentDialect scenarioSimplified Chinese8,000 and 16,000NoAvailable in public preview of Intelligent Speech Interaction
CuijieCuijieFemale voice of Northeastern MandarinDialect scenarioSimplified Chinese8,000 and 16,000NoAvailable in public preview of Intelligent Speech Interaction
XiaozeXiaozeMale voice with strong Hunan accentDialect scenarioSimplified Chinese8,000 and 16,000YesAvailable in public preview of Intelligent Speech Interaction

Limits

  • The entered text must be UTF-8 encoded.

  • The entered text can contain a maximum of 300 characters. If the text contains more than 300 characters, the excessive characters are deleted, and only the first 300 characters are synthesized.

Endpoints

Access type

Description

URL

External access from the Internet

This endpoint allows you to access the speech synthesis service from any host over the Internet. By default, the Internet access URL is built in the SDK.

wss://nls-gateway.cn-shanghai.aliyuncs.com/ws/v1

Internal access from an Elastic Compute Service (ECS) instance located in the China (Shanghai) region

This endpoint allows you to access the speech synthesis service from an ECS instance located in the China (Shanghai) region over an internal network. You cannot access an AnyTunnel virtual IP address (VIP) from a classic network-connected ECS instance. This means that you cannot use such an ECS instance to access the speech synthesis service over an internal network. To access this service by using an AnyTunnel VIP, you must create a virtual private network (VPC) and access the service from the VPC.

Note

  • Access from an ECS instance over the internal network does not consume Internet access traffic.

  • For more information about the network types of ECS instances, see Network types.

ws://nls-gateway.cn-shanghai-internal.aliyuncs.com:80/ws/v1

Interaction process

Interaction process

Note

In addition to audio streams returned in the response, the server adds the task_id parameter to the response header for all responses to indicate the ID of the synthesis task. You can record the value of this parameter. If an error occurs, you can submit a ticket to report the task ID and error message.

1. Authenticate the client

To establish a WebSocket connection with the server, the client must use a token for authentication. For more information about how to obtain the token, see Obtain a Token.

The following table describes the parameters used for authentication and initialization.

Parameter

Type

Required

Description

workspaceStringYesThe working directory from which the SDK reads the configuration file.
app_keyStringYesThe appkey of your project created in the Intelligent Speech Interaction console.
tokenStringYesThe token provided as the credential for you to use Intelligent Speech Interaction. Make sure that the token is valid. You can set the token when you initialize the SDK and update the token when you set the request parameters.
device_idStringYesThe unique identifier of the device, for example, the media access control (MAC) address, serial number, or pseudo unique ID of the device.

2. Send a request to use the speech synthesis service

You must set the request parameters for the client to send a service request. You can set the request parameters by calling the setparamTts method in the SDK. The following table describes the request parameters.

Parameter

Type

Required

Description

appkeyStringYesThe appkey of your project created in the Intelligent Speech Interaction console.
tokenStringNoThe token provided as the credential for you to use Intelligent Speech Interaction. You can update the token as required by setting this parameter.
direct_hostStringNoThe IP address that is resolved from the Domain Name System (DNS) domain name. The client completes the resolution and uses the obtained IP address to access the service.
font_nameStringNoThe speaker type. Default value: xiaoyun.
encode_typeStringNoThe audio encoding format. Default value: PCM. Valid values: PCM, WAV, and MP3.
sample_rateStringNoThe audio sampling rate. Unit: Hz. Default value: 16000.
volumeStringNoThe volume of the speaker. Valid values: 0 to 2. Default value: 1.0.
speed_levelStringNoThe speed of the speaker. Valid values: 0.5 to 2. Default value: 1.0. A greater value indicates a higher speed.
pitch_levelStringNoThe intonation of the speaker. Valid values: -500 to 500. Default value: 0. A greater value indicates a sharper voice.

3. Receive the synthesized speech data

The server returns the synthesized speech data in the binary format, and the SDK receives and processes the binary data.

4. Complete the synthesis task

After the synthesis task is completed, the server sends an notification message.

Error codes

If an error occurs during speech synthesis, the SDK reports an TTS_EVENT_ERROR event to the server and returns an error message to the client. The following table describes the error messages that may be returned.

Error code

Error message

Description

0TTS_SUCCESSThe task is successful.
140000TTS_CREATE_FAILEDThe error message returned because the engine failed to be initialized.
140001TTS_ENGINE_INVALIDThe error message returned because the engine is not initialized.
140002TTS_TEXT_ERRORThe error message returned because the entered text is invalid. For example, it is left empty.
140003TTS_MALLOC_FAILEDThe error message returned because the memory that you have applied for failed to be allocated.
140004TTS_TEXT_QUEUE_FULLThe error message returned because the task queue is full.
140005TTS_ASSETPATH_INVALIDThe error message returned because the specified resource path is invalid.
140006TTS_HANLDE_INVALIDThe error message returned because the processing thread does not exist.
140007TTS_CREATE_HANLDE_FAILEDThe error message returned because the processing thread failed to be created.
140008TTS_AUTH_FAILEDThe error message returned because the authentication failed. You cannot use the SDK before you complete the authentication.
140009TTS_TEXT_QUEUE_EMPTYThe error message returned because the queue of synthesis tasks is empty.
140010TTS_MODE_INVALIDThe error message returned because the synthesis mode is invalid.
140012TTS_OPEN_FILE_FAILEDThe error message returned because the file failed to be opened.
140013TTS_STATE_INVALIDThe error message returned because the state of the state machine is invalid.
140014TTS_SYNTHESIZER_INIT_ERRORThe error message returned because the synthesizer failed to be initialized.
140015TTS_SYNTHESIZER_RELEASE_ERRORThe error message returned because the synthesizer failed to be released.
140016TTS_SYNTHESIZER_FAILEDThe error message returned because the speech synthesis failed.
140017TTS_WAIT_TIMEOUTThe error message returned because the request timed out.
140018TTS_CLOSEDThe error message returned because the code used for speech synthesis is not provided.
140100TTS_PARAM_INVALIDThe error message returned because a specified parameter is invalid.
140101TTS_PARAM_VALUE_INVALIDThe error message returned because a specified parameter value is invalid.
140102TTS_CFG_OPEN_FAILEDThe error message returned because the configuration file failed to be opened.
140103TTS_CFG_WRONG_FORMATThe error message returned because the configuration file is in an invalid format.
140150TTS_LOG_OPEN_FAILEDThe error message returned because the log file failed to be created.
140200TTS_AM_CREATE_FAILEDThe error message returned because the player failed to be created.
140201TTS_AM_OPEN_FAILEDThe error message returned because the player failed to be opened.
140210TTS_DECODER_INIT_FAILEDThe error message returned because the audio decoder failed to be initialized.
140211TTS_DECODER_MALLOC_FAILEDThe error message returned because the memory that you have applied for failed to be allocated to the audio decoder.
140212TTS_DECODER_INPUT_TOO_MANYThe error message returned because the size of the text entered for a single time exceeds the upper limit. The excessive data is deleted.
140213TTS_DECODER_OUTPUT_TOO_MANYThe error message returned because the size of the generated speech data exceeds the cache size. The excessive data is deleted.
140220TTS_AP_INIT_FAILEDThe error message returned because the audio processing unit failed to be opened.
140221TTS_AP_START_FAILEDThe error message returned because the audio processing unit failed to be started.
140222TTS_AP_MALLOC_FAILEDThe error message returned because the memory that you have applied for failed to be allocated to the audio processing unit.
140230TTS_BGM_START_FAILEDThe error message returned because the background music (BGM) failed to be played.
140231TTS_BGM_DECODE_INVALIDThe error message returned because the BGM decoder failed to be initialized.
140232TTS_BGM_ADD_FAILEDThe error message returned because the BGM failed to be added to the sentence.
140233TTS_BGM_MALLOC_FAILEDThe error message returned because the memory that you have applied for failed to be allocated to the BGM.
140234TTS_BGM_OPEN_FILE_FAILEDThe error message returned because the BGM file failed to be opened.
140235TTS_BGM_FILE_FORMAT_ERRORThe error message returned because the BGM file is in an invalid format.
140300TTS_CACHE_INIT_FAILEDThe error message returned because the cache failed to be initialized.
140301TTS_CACHE_MGR_INVALIDThe error message returned because the cache manager is not initialized.
140302TTS_CACHE_CMD_ERRORThe error message returned because the issued cache instruction is invalid.
140303TTS_CACHE_CALLBACK_INVALIDThe error message returned because the callback method is not initialized.
140304TTS_CACHE_START_READ_FAILEDThe error message returned because the cache file failed to be opened.
140305TTS_CACHE_READ_FAILEDThe error message returned because the cached data failed to be read.
140306TTS_CACHE_MALLOC_FAILEDThe error message returned because the memory that you have applied for failed to be allocated to the cache.
140307TTS_CACHE_DELETE_FAILEDThe error message returned because the cache file failed to be deleted.
140308TTS_CACHE_PATH_INVALIDThe error message returned because the directory for storing cache files failed to be created.
140309TTS_CACHE_LIST_CREATE_FAILEDThe error message returned because the cache file list failed to be created.
140310TTS_CACHE_FAILEDThe error message returned because the caching failed.
140311TTS_CACHE_TOO_MANYThe error message returned because the size of cached data exceeds the upper limit.
140312TTS_CACHE_PARAM_INVALIDThe error message returned because a specified cache parameter is invalid.
140313TTS_CACHE_RECORDING_OPEN_FAILEDThe error message returned because the local file failed to be opened.
140350TTS_FONT_INIT_FAILEDThe error message returned because the font manager failed to be initialized.
140351TTS_FONT_INITLIST_FAILEDThe error message returned because the fontlist manager failed to be initialized.
140352TTS_FONT_INITLIST_INVALIDThe error message returned because the fontlist manager is not initialized.
140353TTS_FONT_CMD_INVALIDThe error message returned because the instruction is in an invalid format.
140354TTS_FONT_RESPONSE_ERRORThe error message returned because the response from the server is in an invalid format.
140355TTS_FONT_RESPONSELIST_ERRORThe error message returned because the response to a fontlist request is in an invalid format.
140356TTS_FONT_GET_FONTLIST_FAILEDThe error message returned because the fontlist failed to be queried.
140357TTS_FONT_REQUEST_CMD_ERRORThe error message returned because the instruction used to create a request is invalid.
140358TTS_FONT_LOCALMSG_ERRORThe error message returned because the local list file failed to be parsed.
140359TTS_FONT_LOCALFILE_ERRORThe error message returned because the local list file failed to be saved.
140360TTS_FONT_CLOUDMSG_ERRORThe error message returned because the list on the server failed to be parsed.
140900TTS_LOCAL_CRE_ENGINE_ERRORThe error message returned because the client engine failed to be initialized.
140901TTS_LOCAL_ENGINE_INVALIDThe error message returned because the client engine is not initialized.
140902TTS_LOCAL_ASSET_ERRORThe error message returned because the local resource verification failed.
140903TTS_LOCAL_CRE_TASK_ERRORThe error message returned because the synthesis task failed to be created on the client.
140904TTS_LOCAL_TASK_INVALIDThe error message returned because the synthesis task created on the client is invalid.
140905TTS_LOCAL_START_FAILEDThe error message returned because the synthesis task created on the client failed to be started.
141000TTS_CLOUD_CREATE_FAILEDThe error message returned because the server engine failed to be initialized.
141001TTS_CLOUD_ENGINE_INVALIDThe error message returned because the server engine is not initialized.
141002TTS_CLOUD_TASK_FAILEDThe error message returned because the synthesis task failed to be created on the server.
141003TTS_CLOUD_TASK_INVALIDThe error message returned because the synthesis task created on the server is invalid.
141004TTS_CLOUD_START_FAILEDThe error message returned because the synthesis task created on the server failed to be started.
141005TTS_CLOUD_CANCEL_FAILEDThe error message returned because the synthesis task created on the server failed to be canceled.
141006TTS_CLOUD_NETWORK_BROKENThe error message returned because the network connection is unstable.
144001TTS_CLOUD_AUTH_FAILEDThe error message returned because the authentication failed.
144002TTS_CLOUD_INVALID_MESSAGEThe error message returned because the returned message is invalid.
144003TTS_CLOUD_INVALID_TOKENThe error message returned because the token expires or is invalid.
144004TTS_CLOUD_WAIT_TIMEOUTThe error message returned because the idle connection timed out.
144005TTS_CLOUD_EXCEED_CONCURRENCYThe error message returned because the number of requests exceeds the upper limit.
144100TTS_CLOUD_INVALID_INTERFACEThe error message returned because the method is not supported.
144101TTS_CLOUD_UNSUPPORTED_ORDERThe error message returned because the instruction is not supported.
144102TTS_CLOUD_INVALID_ORDERThe error message returned because the instruction is invalid.
144103TTS_CLOUD_CLIENT_DISCONNECTThe error message returned because the client is disconnected.
144200TTS_CLOUD_INVALID_APPKEYThe error message returned because the specified appkey is invalid.
144300TTS_CLOUD_INVALID_PARAMThe error message returned because a specified cloud service parameter is invalid.
144400TTS_CLOUD_SERVER_ERRORThe error message returned because a server error has occurred.