Intelligent Speech Interaction:Run examples

Choose an example

Configure access parameters

Set the following environment variables in a terminal or the IDE run configuration. The commands use Bash syntax. On Windows, set variables with the same names in the system or IDE. Replace the placeholders and ensure that the IDE process can read the variables.

export NLS_APP_KEY="<APPKEY>"
export NLS_TOKEN="<NLS_TOKEN>"

export NLS_HTTP_URL="https://nls-gateway-ap-southeast-1.aliyuncs.com"
export NLS_WS_URL="wss://nls-gateway-ap-southeast-1.aliyuncs.com/ws/v1"

Short sentence recognition

Download the sample audio file. Save it as nls-sample-16k.wav and use a matching 16 kHz English recognition model. You can also use your own recording: a PCM-encoded WAV file no longer than 60 seconds, with a 16 kHz sample rate, 16-bit samples, one channel, and a language that matches the recognition model.

export NLS_AUDIO_FILE="./nls-sample-16k.wav"

curl -X POST "${NLS_HTTP_URL}/stream/v1/asr?appkey=${NLS_APP_KEY}&format=wav&sample_rate=16000" \
  -H "X-NLS-Token: ${NLS_TOKEN}" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@${NLS_AUDIO_FILE}"

A successful response looks like the following. result contains the recognized text:

{
  "task_id": "<TASK_ID>",
  "result": "weather in beijing",
  "status": 20000000,
  "message": "SUCCESS"
}

For all parameters and audio requirements, see the RESTful API for short sentence recognition.

Real-time speech recognition

This Java example reads a local WAV file as a PCM stream and sends it in chunks. The example uses SDK 2.2.1. Add the following dependency to the Maven project:

<dependency>
  <groupId>com.alibaba.nls</groupId>
  <artifactId>nls-sdk-transcriber</artifactId>
  <version>2.2.1</version>
</dependency>

Download the sample audio file and set NLS_AUDIO_FILE to its local path. Reuse NLS_APP_KEY, NLS_TOKEN, and NLS_WS_URL. The code simulates a live audio stream using a local PCM-encoded WAV file with a 16 kHz sample rate, 16-bit samples, and one channel.

import com.alibaba.nls.client.protocol.InputFormatEnum;
import com.alibaba.nls.client.protocol.NlsClient;
import com.alibaba.nls.client.protocol.SampleRateEnum;
import com.alibaba.nls.client.protocol.asr.SpeechTranscriber;
import com.alibaba.nls.client.protocol.asr.SpeechTranscriberListener;
import com.alibaba.nls.client.protocol.asr.SpeechTranscriberResponse;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.AudioFormat;
import java.io.File;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;

public class RealtimeExample {
    private static String env(String name) {
        String value = System.getenv(name);
        if (value == null || value.isEmpty()) {
            throw new IllegalArgumentException("Set " + name);
        }
        return value;
    }

    public static void main(String[] args) throws Exception {
        String appKey = env("NLS_APP_KEY");
        String token = env("NLS_TOKEN");
        String url = env("NLS_WS_URL");
        String audioFile = env("NLS_AUDIO_FILE");
        CountDownLatch done = new CountDownLatch(1);
        AtomicReference<String> error = new AtomicReference<>();
        SpeechTranscriberListener listener = new SpeechTranscriberListener() {
            public void onTranscriberStart(SpeechTranscriberResponse r) {
                System.out.println("task_id=" + r.getTaskId());
            }
            public void onSentenceBegin(SpeechTranscriberResponse r) {}
            public void onTranscriptionResultChange(SpeechTranscriberResponse r) {}
            public void onSentenceEnd(SpeechTranscriberResponse r) {
                System.out.println(r.getTransSentenceText());
            }
            public void onTranscriptionComplete(SpeechTranscriberResponse r) {
                System.out.println(r.getName() + ": " + r.getStatus());
                done.countDown();
            }
            public void onFail(SpeechTranscriberResponse r) {
                error.set("task_id=" + r.getTaskId() + ", status="
                    + r.getStatus() + ", message=" + r.getStatusText());
                done.countDown();
            }
        };

        NlsClient client = new NlsClient(url, token);
        SpeechTranscriber transcriber = null;
        try (AudioInputStream audio = AudioSystem.getAudioInputStream(new File(audioFile))) {
            AudioFormat f = audio.getFormat();
            if (!f.getEncoding().equals(AudioFormat.Encoding.PCM_SIGNED)
                    || f.getSampleRate() != 16000 || f.getSampleSizeInBits() != 16
                    || f.getChannels() != 1 || f.isBigEndian()) {
                throw new IllegalArgumentException("Use a 16 kHz, 16-bit, mono PCM WAV file");
            }
            transcriber = new SpeechTranscriber(client, listener);
            transcriber.setAppKey(appKey);
            transcriber.setFormat(InputFormatEnum.PCM);
            transcriber.setSampleRate(SampleRateEnum.SAMPLE_RATE_16K);
            transcriber.setEnablePunctuation(true);
            transcriber.start();
            byte[] buffer = new byte[3200];
            int count;
            while ((count = audio.read(buffer)) != -1) {
                if (error.get() != null) throw new IllegalStateException(error.get());
                transcriber.send(buffer, count);
                Thread.sleep(Math.max(1, count / 32));
            }
            transcriber.stop();
            if (!done.await(30, TimeUnit.SECONDS)) {
                throw new IllegalStateException("Timed out waiting for transcription completion");
            }
            if (error.get() != null) throw new IllegalStateException(error.get());
        } finally {
            if (transcriber != null) transcriber.close();
            client.shutdown();
        }
    }
}

The output contains the recognized text and a completion event, for example:

task_id=<TASK_ID>
weather in beijing. 
TranscriptionCompleted: 20000000

For more Java examples, download nls-sdk-java-demo. For API details, see the real-time speech recognition API reference.

Audio file transcription

Use Python to submit an audio file transcription task. The service fetches the audio from a URL rather than receiving file contents in the request. Prepare an accessible audio URL. The file must not exceed 512 MB. For format and duration requirements, see the audio file transcription API reference.

Install the dependency. The example uses aliyun-python-sdk-core 2.15.1.

python3 -m pip install aliyun-python-sdk-core

The configuration below uses the sample audio file. Reuse NLS_APP_KEY configured above.

export ALIYUN_AK_ID="<ACCESS_KEY_ID>"
export ALIYUN_AK_SECRET="<ACCESS_KEY_SECRET>"
export NLS_FILE_URL="https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20230426/vsxg/nls-sample-16k-en.wav"

To use your own recording, upload it to OSS and replace NLS_FILE_URL. For private objects, see Use presigned URLs to download or preview files. Ensure that the URL remains valid while the service fetches the audio.

export NLS_REGION_ID="ap-southeast-1"
export NLS_FILETRANS_DOMAIN="filetrans.ap-southeast-1.aliyuncs.com"
export NLS_FILETRANS_API_VERSION="2019-08-23"

import json
import os
import time
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest


def main():
    client = AcsClient(os.environ["ALIYUN_AK_ID"],
                       os.environ["ALIYUN_AK_SECRET"],
                       os.environ["NLS_REGION_ID"])

    def call(action, method, params):
        request = CommonRequest()
        request.set_domain(os.environ["NLS_FILETRANS_DOMAIN"])
        request.set_version(os.environ["NLS_FILETRANS_API_VERSION"])
        request.set_product("nls-filetrans")
        request.set_protocol_type("https")
        request.set_accept_format("json")
        request.set_action_name(action)
        request.set_method(method)
        for key, value in params.items():
            if method == "POST":
                request.add_body_params(key, value)
            else:
                request.add_query_param(key, value)
        return json.loads(client.do_action_with_exception(request))

    task = {"appkey": os.environ["NLS_APP_KEY"],
            "file_link": os.environ["NLS_FILE_URL"],
            "version": "4.0", "enable_words": False}
    result = call("SubmitTask", "POST", {"Task": json.dumps(task)})
    if result.get("StatusText") != "SUCCESS" or not result.get("TaskId"):
        raise RuntimeError(result)
    task_id = result["TaskId"]
    print("TaskId:", task_id)

    for _ in range(60):
        result = call("GetTaskResult", "GET", {"TaskId": task_id})
        status = result.get("StatusText")
        if status == "SUCCESS":
            print(json.dumps(result.get("Result"), ensure_ascii=False, indent=2))
            return
        if status == "SUCCESS_WITH_NO_VALID_FRAGMENT":
            print("No valid speech fragment was recognized.")
            return
        if status not in ("RUNNING", "QUEUEING"):
            raise RuntimeError(result)
        time.sleep(10)
    raise TimeoutError("Task is still running. Query it later using TaskId: " + task_id)


if __name__ == "__main__":
    main()

The program submits a task and waits for transcription to complete. Example output:

TaskId: <TASK_ID>
{
  "Sentences": [
    {
      "EndTime": 1300,
      "SilenceDuration": 0,
      "SpeakerId": "1",
      "BeginTime": 0,
      "Text": "weather in beijing. ",
      "ChannelId": 0,
      "SpeechRate": 138,
      "EmotionValue": 7.1
    }
  ]
}

Speech synthesis

Use the Java SDK to synthesize text into a 16 kHz WAV file. The example uses SDK 2.2.1. Add the following dependency to the Maven project:

<dependency>
  <groupId>com.alibaba.nls</groupId>
  <artifactId>nls-sdk-tts</artifactId>
  <version>2.2.1</version>
</dependency>

Reuse NLS_APP_KEY, NLS_TOKEN, and NLS_WS_URL, and set the voice and text:

export NLS_VOICE="abby"
export NLS_TEXT="Hello. The weather is nice today."

import com.alibaba.nls.client.protocol.NlsClient;
import com.alibaba.nls.client.protocol.OutputFormatEnum;
import com.alibaba.nls.client.protocol.SampleRateEnum;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizer;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizerListener;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizerResponse;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;

public class SynthesisExample {
    private static String env(String name) {
        String value = System.getenv(name);
        if (value == null || value.isEmpty()) throw new IllegalArgumentException("Set " + name);
        return value;
    }

    public static void main(String[] args) throws Exception {
        String appKey = env("NLS_APP_KEY");
        String token = env("NLS_TOKEN");
        String url = env("NLS_WS_URL");
        String voice = env("NLS_VOICE");
        String text = env("NLS_TEXT");
        CountDownLatch done = new CountDownLatch(1);
        AtomicReference<String> error = new AtomicReference<>();
        NlsClient client = new NlsClient(url, token);
        SpeechSynthesizer synthesizer = null;
        try (FileOutputStream output = new FileOutputStream("speech.wav")) {
            SpeechSynthesizerListener listener = new SpeechSynthesizerListener() {
                public void onMessage(ByteBuffer message) {
                    byte[] bytes = new byte[message.remaining()];
                    message.get(bytes);
                    try { output.write(bytes); }
                    catch (IOException e) { error.set(e.getMessage()); done.countDown(); }
                }
                public void onComplete(SpeechSynthesizerResponse r) {
                    System.out.println(r.getName() + ": " + r.getStatus());
                    done.countDown();
                }
                public void onFail(SpeechSynthesizerResponse r) {
                    error.set("task_id=" + r.getTaskId() + ", status="
                        + r.getStatus() + ", message=" + r.getStatusText());
                    done.countDown();
                }
                public void onMetaInfo(SpeechSynthesizerResponse r) {}
            };
            synthesizer = new SpeechSynthesizer(client, listener);
            synthesizer.setAppKey(appKey);
            synthesizer.setFormat(OutputFormatEnum.WAV);
            synthesizer.setSampleRate(SampleRateEnum.SAMPLE_RATE_16K);
            synthesizer.setVoice(voice);
            synthesizer.setText(text);
            synthesizer.start();
            if (!done.await(60, TimeUnit.SECONDS)) {
                throw new IllegalStateException("Timed out waiting for synthesis completion");
            }
            if (error.get() != null) throw new IllegalStateException(error.get());
            System.out.println("Audio saved to speech.wav");
        } finally {
            if (synthesizer != null) synthesizer.close();
            client.shutdown();
        }
    }
}

After synthesis succeeds, play the saved speech.wav file. The output is:

SynthesisCompleted: 20000000
Audio saved to speech.wav

For more Java examples, download nls-sdk-java-demo. For other voices, languages, and parameters, see the speech synthesis API reference.