NUI SDK for iOS - Intelligent Speech Interaction

The speech synthesis service provides a Natural User Interaction (NUI) SDK for iOS. This topic describes how to download the NUI SDK for iOS, lists the key methods in the SDK, and provides sample code for you to use the SDK.

Prerequisites

You understand how the SDK works. For more information, see Overview.
The appkey of your project is obtained. For more information, see Create a project.
A token used to access the service is obtained. For more information, see Obtain a token.

Download and install the SDK

Download the NUI SDK for iOS and sample code.
Decompress the downloaded package to obtain the demo project and use the nuisdk.framework framework to integrate the demo project with the iOS system.
Note
The demo project is written in the Objective-C and C++ programming languages. You must use the files with the .mm file extension.
Use Xcode to open the demo project.
The sample code for the speech synthesis service is stored in the TTSViewController.mm file.

Key methods

nui_tts_initialize: initializes the SDK.

/**
     * Initialize the SDK. The SDK uses a singleton pattern. To initialize the SDK again, you must first release the SDK. Do not call the SDK on the user interface (UI) thread. Otherwise, the process may be blocked.
     * @param parameters: the parameters used in the initialization. For more information, see Overview.
     * @param listener: the event listener callback. For more information, see the following callback methods.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @param level: the log level to use. The smaller the parameter value is, the more logs are recorded.
     * @param save_log: specifies whether to store logs in files. The debug_path parameter specifies the directory where log files are stored.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_initialize:(const char *)parameters
              ttsListener:(const NuiTtsSdkListener *)listener
            asyncCallback:(const NuiAsyncCallback *)async_listener
                 logLevel:(NuiSdkLogLevel)level
                  saveLog:(bool)save_log;

The following table lists the parameters in the NuiTtsSdkListener method.

Name	Type	Description
tts_event_callback	FuncTtsListenerOnEvent	Reports the occurred event to the server.
tts_user_data_callback	FuncTtsUserProvideData	Provides the synthesized speech data.
tts_user_volume_callback	FuncTtsUserProvideVolume	Sets the volume of the synthesized speech.
user_data	void *	Obtains the user data, which corresponds to the first parameter in the callback.

FuncTtsListenerOnEvent: reports the occurred event to the server.

/**
     * Report the occurred event to the server.
     * @param user_data: Reserved.
     * @param event: the event to be reported by the client. You can view possible events in the following table.
     * @param taskid: the ID of the speech synthesis task.
     * @param code: the returned error code. This parameter is valid for the TTS_EVENT_ERROR event.
     */
    typedef void (*FuncTtsListenerOnEvent) (void *user_data, NuiSdkTtsEvent event, char *taskid, int code);

The following table lists the possible events in the SDK.

Name	Description
TTS_EVENT_START	Starts the speech synthesis task and starts the playback of the synthesized speech.
TTS_EVENT_END	Ends the playback of the synthesized speech.
TTS_EVENT_CANCEL	Cancels the speech synthesis task.
TTS_EVENT_PAUSE	Pauses the speech synthesis task.
TTS_EVENT_RESUME	Resumes the speech synthesis task.
TTS_EVENT_ERROR	Returns an error during speech synthesis.

FuncTtsUserProvideData: provides the synthesized speech data.

/**
     * After the speech synthesis task is started, this callback is used to provide the synthesized speech data to the client for playback.
     * @param user_data: Reserved.
     * @param text: (reserved) the text to be processed. This parameter is valid only for local speech synthesis tasks.
     * @param word_idx: (reserved) the position of the word in the text to be synthesized. This parameter is valid only for local speech synthesis tasks.
     * @param buffer: the synthesized speech data.
     * @param len: the duration of the synthesized speech.
     * @param taskid: the ID of the speech synthesis task.
     */
    typedef void (*FuncTtsUserProvideData) (void *user_data, char *text, int word_idx, char *buffer, int len, char *taskid);

FuncTtsUserProvideVolume: sets the volume of the synthesized speech.

/**
     * Set the volume of the synthesized speech.
     * @param user_data: Reserved.
     * @param volume: the volume of the synthesized speech. The parameter value is dynamic.
     * @param taskid: the ID of the speech synthesis task.
     */
    typedef void (*FuncTtsUserProvideVolume) (void *user_data, int volume, char *taskid);

nui_tts_play: starts the speech synthesis task.

/**
     * Start the speech synthesis task.
     * @param priority: the priority of the speech synthesis task. Set this parameter to 1.
     * @param taskid: the ID of the speech synthesis task. You can set this parameter to a 32-byte universally unique identifier (UUID), or leave this parameter empty. If this parameter is left empty, the SDK generates a task ID.
     * @param text: the source text to be processed.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_play:(const char *)priority
             taskId:(const char *)taskid
               text:(const char *)text
      asyncCallback:(const NuiAsyncCallback *)listener;

nui_tts_cancel: cancels the speech synthesis task.

/**
     * Cancel the speech synthesis task.
     * @param taskid: the ID of the task that you want to cancel. If this parameter is left empty, all the speech synthesis tasks are canceled.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_cancel:(const char *)taskid
        asyncCallback:(const NuiAsyncCallback *)listener;

nui_tts_pause: pauses the speech synthesis task.

/**
     * Pause the speech synthesis task.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_pause:(const NuiAsyncCallback *)listener;

nui_tts_resume: resumes the speech synthesis task.

/**
     * Resume the speech synthesis task.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_resume:(const NuiAsyncCallback *)listener;

nui_tts_set_param: sets the request parameters.

/**
     * Set the request parameters in the format of key-value pairs.
     * @param param: the parameter name. For more information, see Overview.
     * @param value: the parameter value. For more information, see Overview.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_set_param:(const char *)param
                   value:(const char *)value
           asyncCallback:(const NuiAsyncCallback *)listener;

nui_tts_get_param: obtains the parameter configuration.

/**
     * Obtain the parameter configuration.
     * @param param: the parameter name. For more information, see Overview.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned parameter value.
     */
    -(const char *) nui_tts_get_param:(const char *)param
                    asyncCallback:(const NuiAsyncCallback *)listener;

nui_tts_release: releases the SDK.

/**
     * Release the SDK.
     * @param async_listener: the asynchronous callback mode. A value of nullptr specifies that the callback is executed synchronously.
     * @return: the returned error code. For more information, see Error codes.
     */
    -(int) nui_tts_release:(const NuiAsyncCallback *)async_listener;

Procedure

Initialize the SDK and playback components.
Set request parameters based on your business requirements.
Call the nui_tts_play method to start the speech synthesis task.
Write the synthesized speech data that is returned in the callback to the player and start the playback. We recommend that you use stream playback.
The client receives the callback of the completion of the speech synthesis task.

Sample code

Initialize the SDK

NSString * initParam = [self genInitParams];
    //nui listener
    NuiTtsSdkListener ttsListener;
    ttsListener.tts_event_callback = ttsListenerOnEvent;
    ttsListener.tts_font_progress_callback = nullptr;
    ttsListener.tts_user_volume_callback = ttsUserProvideVolume;
    ttsListener.tts_user_data_callback = ttsListenerUserdata;
    ttsListener.user_data = nullptr;
    [_nui nui_tts_initialize:[initParam UTF8String] ttsListener:&ttsListener asyncCallback:nullptr logLevel:LOG_LEVEL_VERBOSE saveLog:true];

The genInitParams method generates a JSON string that contains the information about the resource directory and user. The user information contains the following parameters:

[dictM setObject:id_string forKey:@"device_id"];
[dictM setObject:@"" forKey:@"url"];
[dictM setObject:@"" forKey:@"app_key"];
[dictM setObject:@"" forKey:@"token"];

Start the speech synthesis task

[self.nui nui_tts_play:"1" taskId:"" text:[content UTF8String] asyncCallback:nullptr];

Handle callbacks

Call the onNuiTtsEventCallback method to control the player based on the status of the speech synthesis task.

- (void)onNuiTtsEventCallback:(nuisdk::NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code {
    TLog(@"onNuiTtsEventCallback event[%d]", event);
    if (event == TTS_EVENT_START) {
        loop_in = TTS_EVENT_START;
        dispatch_async(dispatch_get_main_queue(), ^{
            [self->_voicePlayer play];
        });
    } else if (event == TTS_EVENT_END) {
        loop_in = TTS_EVENT_END;
        dispatch_async(dispatch_get_main_queue(), ^{
            [self->_voicePlayer drain];
        });
    }
}

Call the onNuiTtsUserdataCallback method to write the synthesized speech data to the player and start the playback.

- (void)onNuiTtsUserdataCallback:(char*)text wordIdx:(int)word_idx buffer:(char*)buffer len:(int)len taskId:(char*)task_id {
    TLog(@"onnuiTtsUserdataCallback played text %s index %d", text, word_idx);
    [_voicePlayer write:(char*)buffer Length:(unsigned int)len];
}

Complete the speech synthesis task

[self.nui nui_tts_cancel:nullptr asyncCallback:nullptr];