Short sentence recognition API reference - Intelligent Speech Interaction

Short sentence recognition converts audio clips of up to 60 seconds into text for chat conversations, voice commands, voice input, and voice search. Clients send audio streams over WebSocket and receive recognition events and results.

Usage notes

The audio encoding must match the request parameters. A mismatch can cause recognition to fail or return an empty result.

Channels and bit depth: mono, 16-bit.
Audio formats: PCM, PCM-encoded WAV, OPUS in an OGG container, SPEEX in an OGG container, and AMR.
Sample rate: 8000 Hz or 16000 Hz. The project model must support the audio sample rate and language.
Audio duration: up to 60 seconds.
Audio size: up to 2 MB.

Request parameters control features such as intermediate results, punctuation, and inverse text normalization (ITN).

Select a recognition model

Language and dialect models cannot be specified in request parameters. On the All Projects page of the Intelligent Speech Interaction console, find the project and click Configure Project Features. Select a model that matches the audio language and sample rate. For configuration instructions, see Manage projects.

Available models are listed on the project feature configuration page. Select a sample rate, and then select a language model.

Endpoints

Use the following WebSocket endpoint over the public network: wss://nls-gateway-ap-southeast-1.aliyuncs.com/ws/v1.

Interaction process

The following process applies to WebSocket and SDKs that use this protocol. For RESTful requests, see RESTful API.

Every server event includes the recognition task's task_id in its header. Use this ID to correlate requests and responses and troubleshoot issues.

1. Authenticate

The client uses an NLS token to authenticate when establishing a WebSocket connection.

For instructions, see Obtain an access token.

2. Start recognition

The client sends a StartRecognition directive with the recognition parameters. The server validates the request and returns a RecognitionStarted event. Wait for this event before sending audio data.

3. Send data

The client sends binary audio in chunks while receiving server events.

If enable_intermediate_result is true, the server can return multiple RecognitionResultChanged events with intermediate results.
If enable_intermediate_result is false, the server does not return intermediate results. The client must still handle events such as RecognitionCompleted and TaskFailed.

Important
The last intermediate result can differ from the final result. Use the result in RecognitionCompleted as the final result.

4. Stop recognition

After sending the audio, the client sends a StopRecognition directive. The server returns RecognitionCompleted with the final result on success, or TaskFailed on failure. Wait for the server's completion or failure event before closing the connection.

If voice activity detection is enabled, the server can complete recognition when trailing silence exceeds max_end_silence. Audio sent after recognition completes is not recognized.

Request parameters

In an SDK, use the methods of the SpeechRecognizer object to set these parameters. For direct WebSocket requests, set appkey in the request header and the other recognition parameters in the StartRecognition payload.

Parameter	Type	Required	Description
appkey	String	Yes	The Appkey of the project created in the console.
format	String	No	The audio format: `pcm`, `wav`, `opus`, `speex`, or `amr`. WAV must use PCM encoding. OPUS and SPEEX must use OGG containers.
sample_rate	Integer	No	The sample rate in Hz. Default: `16000`. Valid values: `8000` and `16000`. The value must match the audio and project model.
enable_intermediate_result	Boolean	No	Whether to return intermediate recognition results. Default: `false`.
enable_punctuation_prediction	Boolean	No	Whether to add punctuation during post-processing. Default: `false`.
enable_inverse_text_normalization	Boolean	No	Whether to enable inverse text normalization (ITN). Setting this parameter to `true` converts Chinese numerals to Arabic numerals in the output. Default: `false`.
disfluency	Boolean	No	Whether to filter filler words (output smoothing). Default: `false`.
customization_id	String	No	The ID of the custom language model. For configuration instructions, see Customize language models.
vocabulary_id	String	No	The ID of the custom hotword list. For configuration instructions, see Customize hotwords.
enable_voice_detection	Boolean	No	Whether to enable voice activity detection (VAD). VAD detects the start and end of speech and excludes noise. Default: `false`.
max_start_silence	Integer	No	Takes effect only when `enable_voice_detection` is `true`. The maximum duration of leading silence, in milliseconds. Recommended range: (0, 60000]. If no speech is detected within this period, the server returns `TaskFailed` and ends recognition.
max_end_silence	Integer	No	Takes effect only when `enable_voice_detection` is `true`. The maximum duration of trailing silence, in milliseconds. Valid range: 200–6000. When trailing silence exceeds this value, the server returns `RecognitionCompleted` and ends recognition. Subsequent audio is not recognized.

Response events

The header contains the following common fields.

Parameter	Type	Description
namespace	String	The namespace: `SpeechRecognizer`.
name	String	The event name. See the event descriptions below.
status	Integer	The status code. `20000000` indicates success. For other values, see Status codes.
status_text	String	The status message.
task_id	String	The globally unique task ID, which corresponds to the task ID in the client request. Record this value for troubleshooting.
message_id	String	The ID of this server response message.

RecognitionStarted

The server has accepted the start request, and the client can send audio data. This event does not contain recognition results.

RecognitionResultChanged

The payload.result field contains the intermediate recognition result as a string. This event is returned only when enable_intermediate_result is true.

{
  "header": {
    "namespace": "SpeechRecognizer",
    "name": "RecognitionResultChanged",
    "status": 20000000,
    "message_id": "8b756a91809843619c920e176d26****",
    "task_id": "af41104b6806410e9d3f192532aa****",
    "status_text": "Gateway:SUCCESS:Success."
  },
  "payload": {
    "result": "hello today is a beautiful day i would like to buy for two apples thank you"
  }
}

RecognitionCompleted

Recognition has completed successfully. The payload.result field contains the final recognition result as a string.

{
  "header": {
    "namespace": "SpeechRecognizer",
    "name": "RecognitionCompleted",
    "status": 20000000,
    "message_id": "25227a2a43a74259a45685fc1633****",
    "task_id": "af41104b6806410e9d3f192532aa****",
    "status_text": "Gateway:SUCCESS:Success."
  },
  "payload": {
    "result": "hello today is a beautiful day i would like to buy for two apples thank you"
  }
}

TaskFailed

The recognition task failed. Check header.status and header.status_text for the cause. For example, stopping a request without sending audio returns 40000000 and Gateway:CLIENT_ERROR:Empty audio data!.

Status codes

Check both status and status_text when troubleshooting. A status code can have multiple causes, so use the specific status message to identify the issue.

Common error codes

Status code	Status message	Cause	Solution
40000000	The default client error code. This code corresponds to multiple error messages.	Invalid parameters or call sequence.	Check the request parameters, directive order, and specific status message.
40000001	The token 'xxx' has expired. The token 'xxx' is invalid	The token has expired or is invalid.	Obtain a valid NLS token and retry.
40000002	Gateway:MESSAGE_INVALID:Can't process message in state'FAILED'!	The message is invalid or is not accepted in the current task state.	Check the message structure and directive order. Start a new recognition task after a task fails.
40000003	PARAMETER_INVALID Failed to decode url params	Invalid parameters.	Check the parameter names, types, and values against the API requirements.
40000005	Gateway:TOO_MANY_REQUESTS:Too many requests!	Too many concurrent requests.	Reduce concurrent requests and check the available concurrency quota.
40000009	Invalid wav header!	The message header is invalid.	If you send a WAV audio file and set the `format` parameter to `wav`, check whether the WAV header of the audio file is correct. If the header is incorrect, the server may reject the request.
40000009	Too large wav header!	The WAV header of the transmitted audio is invalid.	You can send the audio stream in a format such as PCM or OPUS. If you use the WAV format, make sure that the WAV header of the audio file contains the correct data length.
40000010	Gateway:FREE_TRIAL_EXPIRED:The free trial has expired!	The trial has expired without a commercial upgrade, or the account has overdue payments.	Check the activation status of short sentence recognition and the account balance in the console.
40010001	Gateway:NAMESPACE_NOT_FOUND:RESTful url path illegal	An unsupported API or parameter was used.	Check the endpoint, namespace, and parameters against the API requirements.
40010003	Gateway:DIRECTIVE_INVALID:[xxx]	An invalid parameter or directive was sent.	Use the specific status message to check the parameters and directive.
40010004	Gateway:CLIENT_DISCONNECT:Client disconnected before task finished!	The client disconnected before the task completed.	Wait for the server's completion or failure event before closing the connection.
40010005	Gateway:TASK_STATE_ERROR:Got stop directive while task is stopping!	The directive is not supported in the current task state.	Check the directive order and avoid sending duplicate stop directives.
40020105	Meta:APPKEY_NOT_EXIST:Appkey not exist!	The Appkey does not exist.	Check the Appkey in the project configuration in the console.
40020106	Meta:APPKEY_UID_MISMATCH:Appkey and user mismatch!	The Appkey and token belong to different accounts.	Use a project Appkey and NLS token from the same account.
403	Forbidden	The token does not exist, has expired, or is invalid.	Use a valid NLS token and obtain a new token before it expires.
41000003	MetaInfo doesn't have end point info	Routing information for the Appkey could not be obtained.	Check the Appkey and make sure it belongs to the same account as the token.
41010101	UNSUPPORTED_SAMPLE_RATE	The sample rate is not supported by the current configuration.	Use 8000 Hz or 16000 Hz audio and make sure the audio, request parameter, and project model match.
50000000	GRPC_ERROR:Grpc error!	A server-side call failed, possibly due to load or network conditions.	Retry the request. If the issue persists, contact technical support and provide the task_id.
50000001	GRPC_ERROR:Grpc error!	A server-side call failed, possibly due to load or network conditions.	Retry the request. If the issue persists, contact technical support and provide the task_id.
52010001	GRPC_ERROR:Grpc error!	A server-side call failed, possibly due to load or network conditions.	Retry the request. If the issue persists, contact technical support and provide the task_id.

Short sentence recognition error codes

Status code	Status message	Cause	Solution
40000000	Gateway:CLIENT_ERROR:Empty audio data!	No audio data was sent.	Send non-empty binary audio data before sending the stop directive.
40000004	Gateway:IDLE_TIMEOUT:Websocket session is idle for too long time	No data was sent after the WebSocket connection was established, and the connection was idle for more than 10 seconds.	Send recognition directives and audio promptly after connecting. Send the stop directive when all audio has been sent.
40010002	Gateway:DIRECTIVE_NOT_SUPPORTED:Directive'SpeechRecognizer.EnhanceRecognition'isnotsupported!	The server does not support the directive.	Check the directive name and use a directive supported by short sentence recognition.
40010003	Gateway:DIRECTIVE_INVALID:Too many items for ‘vocabulary'!(173)	Too many hotwords were specified.	Reduce the hotword count to the limit for the hotword configuration method in use.
40270002	NO_VALID_AUDIO_ERROR	The audio is invalid and no valid text was recognized.	Check that the audio contains clear speech and that its encoding, sample rate, and model match.
41010104	TOO_LONG_SPEECH	The audio exceeds the duration limit for short sentence recognition.	Use audio of up to 60 seconds. For longer audio, use real-time speech recognition.
41010105	SILENT_SPEECH	No speech was detected in silent or noisy audio.	Check the audio. If VAD is enabled, also check the leading silence setting.