Enable Push-to-Talk Mode with AICallKit SDK - Intelligent Media Services - Alibaba Cloud - Intelligent Media Services

Usage notes

This example uses the API to implement this feature without UI integration.
You must first integrate the AICallKit SDK. For integration instructions, see Android Integration Overview, iOS Integration Overview, Web Integration Overview.

Preview

Enable push-to-talk mode

Switches the conversation mode from natural conversation mode to push-to-talk mode.

This option is in the Settings panel. The panel also provides a Voiceprint recognition (invitational test) switch, which, when enabled, ensures the AI uses only your voice as input, and a Select timbre feature with options such as Zhi Xiao Bai, Zhi Xiao Xia, and A Bin.

Start talking

Press and hold the talk button to start communicating with the agent.

When you press and hold the microphone button, a voice animation appears in the center of the screen with the prompt You talk, I'm listening and a call timer. The bottom action bar contains the Turn off speaker button, a microphone slider labeled Release to send, and a Hang up button.

Note

This feature and the user interface (UI) are built into the UI-integrated solution. For more information about the UI-integrated solution, see the UI-integrated solution.

Implement push-to-talk

In push-to-talk mode, each user input and agent response is an independent event, similar to a walkie-talkie.

API reference

API	Description
enablePushToTalk	Enables or disables push-to-talk mode. The final state is confirmed through the `onPushToTalk` callback, which is the source of truth.
startPushToTalk	Initiates a transmission when the user presses and holds the talk button. This method works only when push-to-talk mode is enabled. During the transmission, the agent does not perform endpointing, remains in the listening state, and provides real-time transcription of the user's speech. Note Call this API on the button-down event, not the button-release event.
finishPushToTalk	Completes the current speech input when the user releases the button. The agent then enters the thinking state and plays its response.
cancelPushToTalk	Cancels the current speech input when the user releases the button. The agent ignores the current input and waits for the user to start a new turn.

Code examples

Android

// By default, push-to-talk mode is disabled. Pass true to enable it.
mARTCAICallEngine.enablePushToTalk(enable: true)

// Press and hold to start talking.
mARTCAICallEngine.startPushToTalk()

// Release to finish the current turn.
mARTCAICallEngine.finishPushToTalk()

// Release to cancel the current input.
mARTCAICallEngine.cancelPushToTalk()

// Callback handling (only relevant callback operations are shown).
ARTCAICallEngine.IARTCAICallEngineCallback mCallEngineCallbackWrapper = new ARTCAICallEngine.IARTCAICallEngineCallback() {
    
    @Override
    public void onPushToTalk(boolean enable) {
        // Callback that reports whether push-to-talk mode is enabled for the current call.
        // The status from this callback is the source of truth.
    }
}

iOS

// By default, push-to-talk mode is disabled. Pass true to enable it.
_ = self.engine.enablePushToTalk(enable: true)

// Press and hold to start talking.
_ = self.engine.startPushToTalk()

// Release to finish the current turn.
_ = self.engine.finishPushToTalk()

// Release to cancel the current input.
_ = self.engine.cancelPushToTalk()

func onPushToTalk(enable: Bool) {
    // Callback that reports whether push-to-talk mode is enabled for the current call.
    // The status from this callback is the source of truth.
}

Web

// By default, push-to-talk mode is disabled. Pass true to enable it.
engine.enablePushToTalk(enable);

// Press and hold to start talking.
engine.startPushToTalk();

// Release to finish the current turn.
engine.finishPushToTalk();

// Release to cancel the current input.
engine.cancelPushToTalk();

engine.on('pushToTalkChanged', (enable) => {
  // Callback that reports whether push-to-talk mode is enabled for the current call.
  // The status from this callback is the source of truth.
  console.log('AICallPushToTalk', enable);
});