Alibaba Cloud Model Studio:Qwen-Audio-ASR-Streaming Real-Time Speech Recognition iOS SDK

This guide shows you how to use the Qwen-Audio-ASR-Streaming real-time speech recognition iOS SDK to transcribe speech into text.

Quick start

1. Get an API key: Obtain an API key

2. Download the SDK and run the sample code:
   • Download the latest SDK package.
   • Extract the ZIP package and add the included nuisdk.xcframework to your project.
   • In Build Phases → Link Binary With Libraries, add nuisdk.xcframework.
   • In General → Frameworks, Libraries, and Embedded Content, set nuisdk.xcframework to Embed & Sign.
   • Open the sample project in Xcode. The sample code is in DashFunAsrSpeechTranscriberViewController.m. Replace the API key with your own to try out the feature.

Call sequence

1. Initialize the SDK.
2. Set parameters for your use case: use the nui_initialize API to set Connection and control parameters, and use the nui_set_params API to set Recognition quality parameters.
3. Call nui_dialog_start to start the recognition process.
4. In the onNuiAudioStateChanged callback, open the recording device based on the audio state.
5. In the onNuiNeedAudioData callback, continuously supply recording data, or call nui_update_audio_data to actively push recording data.
6. In the onNuiEventCallback callback, listen for events and retrieve speech recognition results.
7. Call nui_dialog_cancel to stop recognition, and confirm that recognition has finished by listening for the EVENT_TRANSCRIBER_COMPLETE event.
8. When you no longer need the recognition feature, call nui_release to release the SDK resources.

Request parameters

Connection and control parameters

Configure these parameters by passing a JSON string in the parameters argument of the nui_initialize API.

Example parameters: The following JSON string is an example and does not list all parameters. Add the parameters you need when you write your code:

{
    "url": "wss://dashscope.aliyuncs.com/api-ws/v1/inference",
    "apikey": "st-****",
    "device_id": "my_device_id",
    "service_mode": "1"
}

Recognition quality parameters

Configure these parameters by passing a JSON string in the params argument of the nui_set_params API.

Example parameters: The following JSON string is an example and does not list all parameters. Add the parameters you need when you write your code:

{
    "service_type": 4,
    "nls_config": {
        "model": "qwen-audio-3.0-asr-flash-streaming",
        "sr_format": "pcm",
        "sample_rate": "16000"
    }
}

Key APIs

NeoNui

nui_initialize

Initializes the speech recognition SDK instance. The SDK is a singleton. Do not initialize it again before you call nui_release.

-(NuiResultCode) nui_initialize:(const char *)parameters
                       logLevel:(NuiSdkLogLevel)level
                        saveLog:(BOOL)save_log;

nui_set_params

Sets the Recognition quality parameters in JSON format. Call this method before nui_dialog_start.

-(NuiResultCode) nui_set_params:(const char *)params;

nui_dialog_start

Starts recognition.

-(NuiResultCode) nui_dialog_start:(NuiVadMode)vad_mode
                      dialogParam:(const char *)dialog_params;

{
  "apikey": "st-****",
  "input_context": [
    {
      "role": "user",
      "content": [
        {
          "text": "xxxxx",
          "type": "input_text"
        }
      ]
    }
  ]
}

nui_dialog_cancel

Ends recognition or cancels the current interaction immediately.

-(NuiResultCode) nui_dialog_cancel:(BOOL)force;

nui_dialog_action

Sends a dialog action during an interaction to update the recognition context or other runtime behavior.

-(NuiResultCode) nui_dialog_action:(const char *)action_params;

{
  "context": [
    {
      "role": "user",
      "content": [
        {
          "text": "xxx",
          "type": "input_text"
        }
      ]
    }
  ]
}

nui_update_audio_data

When audio_update_manually is set to "true", recording data is no longer supplied through onNuiNeedAudioData. Use this method to actively push it instead.

-(NuiResultCode) nui_update_audio_data:(const char *)data
                                    Len:(int)length
                              FirstPack:(BOOL)first_pack;

nui_push_reference_data

When audio_update_manually is set to "true" and on-device acoustic echo cancellation (AEC) is enabled, use this method to push the audio played by the player as the reference signal.

-(NuiResultCode) nui_push_reference_data:(const char *)data
                                     Len:(int)length
                               FirstPack:(BOOL)first_pack;

nui_release

Releases all internal SDK resources and forcibly terminates all running tasks. After you call this method, the SDK instance becomes unavailable. To use it again, you must call nui_initialize to initialize it again.

-(NuiResultCode) nui_release;

nui_get_version

Gets the current SDK version. This method returns a value only after nui_initialize is called.

-(const char*) nui_get_version;

The current SDK version.

nui_get_all_response

Gets the complete information for the current event callback.

-(const char*) nui_get_all_response;

The complete event information as a JSON string.

NeoNuiSdkDelegate: callback listeners

onNuiEventCallback: listen for events and speech recognition results

-(void) onNuiEventCallback:(NuiCallbackEvent)nuiEvent
                    dialog:(long)dialog
                 kwsResult:(const char *)wuw
                 asrResult:(const char *)asr_result
                  ifFinish:(BOOL)finish
                   retCode:(int)code;

onNuiAudioStateChanged: listen for the audio state

The SDK uses this callback to notify you when to start or stop recording.

-(void) onNuiAudioStateChanged:(NuiAudioState)state;

onNuiNeedAudioData: supply audio data for recognition

After recognition starts, this callback is triggered continuously. Supply the audio data to be recognized in it.

-(int) onNuiNeedAudioData:(char *)audioData length:(int)len;

onNuiAssistEventCallback: receive auxiliary data and information

Receives auxiliary events and related data from the SDK.

-(void) onNuiAssistEventCallback:(NuiCallbackEvent)nuiEvent
                            info:(char*)info
                         infoLen:(int)info_len
                          buffer:(char*)buffer
                             len:(int)len;

onNuiLogTrackCallback: listen for tracking logs

This callback receives the SDK's detailed internal logs to help you locate and debug issues.

-(void) onNuiLogTrackCallback:(NuiSdkLogLevel)level
                   logMessage:(const char *)log;

NuiCallbackEvent: event types