All Products
Search

This Product

    Intelligent Media Services:Create an audio/video call agent

    Document Center

    Intelligent Media Services:Create an audio/video call agent

    Last Updated:Jun 03, 2026

    Set up a real-time conversational AI agent by creating a workflow, binding it to an agent, and testing the result.

    Prerequisites

    Before you begin, ensure you meet the following requirements:

    • Real-time Conversational AI is enabled. To enable the feature, go to the buy page.

    Step 1: Create an audio/video workflow

    1. Log in to the Intelligent Media Service (IMS) console and click Create Workflow Template.

    2. Select a workflow type, such as Audio Call, Avatar Call, Vision Call, or Video Call, and configure the workflow nodes.

      Speech-to-text

      Converts speech to text with multi-language support.

      Model Version options: Preset ASR (recommended for mixed Chinese-English, higher accuracy), NLS-ASR (ultra-low latency), Qwen3-ASR-Realtime, and Fun-ASR-Realtime. Default Silent Time: 400 ms.

      • Preset: With preset models, you can select a language model, set a silent time, and configure custom hotwords.

        • Model: Select a language model for your scenario.

        • Silent Time: How long the agent waits for the user to speak.

        • Custom Hotword: Improves recognition accuracy for domain-specific vocabulary. Speech recognition hotwords.

        • Sensitive Words: Configured words are automatically redacted with asterisks (*) in client-side subtitles. Custom sensitive words.

      • Third-party Plug-in: Currently, you can select iFLYTEK. To get the required parameters, go to iFLYTEK Real-time Speech Dictation.

      LLM

      The LLM processes STT output to understand and generate natural language responses.

      Configure the System Persona (role, objectives, capabilities, response requirements, and constraints; up to 3,072 characters) and Conversation Memory Rounds (0–30). More rounds retain more context but may increase processing time.

      Supported LLM providers: Qwen (system preset), Alibaba Cloud Model Studio, Tongyi Xingchen, and OpenAI-compliant self-developed models.

      Alibaba Cloud Model Studio

      A platform for large model development and application building. Connect through the Model center or the Application center.

      Tongyi Xingchen

      Tongyi Xingchen enables deeply personalized agents with unique personas, combinable with avatar-based real-time voice interaction.

      • ModelId: Tongyi Xingchen offers five models: xingchen-lite, xingchen-base, xingchen-plus, xingchen-plus-v2, and xingchen-max.

      • API key: Go to the Tongyi Xingchen console to create and obtain an API key.

      Self-developed model

      Connect your self-developed large model using the OpenAI specification.

      Enter the following parameters:

      Parameter

      Description

      Example

      ModelId

      The model name. Maps to the model field in the OpenAI specification.

      abc

      API key

      The API authentication credential. Maps to the api_key field in the OpenAI specification.

      AUJH-pfnTNMPBm6iWXcJAcWsrscb5KYaLitQhHBLKrI

      Model URL (HTTPS)

      The service endpoint URL. Maps to the base_url field in the OpenAI specification.

      http://www.abc.com

      LLM Standard Interface.

      Text-to-speech

      Converts processed text to audio so the user hears the agent's response.

      In the TTS node, select the Version (Text-to-Speech 2.0 or Text-to-Speech 1.0 (Legacy)), set the Timbre (for example, Yunfeng), adjust the Volume (0–100), and preview audio by entering text and clicking Preview.

      • You can choose a text-to-speech model that suits your application scenario, including Preset Template, Self-developed Template, and Third-party Plug-in.

        • System Preset Template: Includes System Default TTS, CosyVoice, and Qwen3-TTS.

        • Self-developed Template: Integrate your own model via a standard protocol. TTS Standard Interface.

        • Third-party Plug-in: Only MiniMax Speech Model is supported. Use the latest version. MiniMax Speech Model.

      • The TTS node can also filter content received from the LLM.

        In Filter Settings, select bracket types to filter: Chinese parentheses（）, English parentheses (), Chinese square brackets【】, English square brackets [], and English curly braces {}.

      • Text Normalization: Converts numbers, symbols, and other elements in the text into a standardized format to improve the quality of synthesized speech. For example, "120" is converted to "one two zero."

      Virtual Avatars

      Generates a video stream of the avatar, synchronizing movements, expressions, and lip movements with the text and audio.

      Configure the Node Name (auto-generated). Input types: Text Stream and Audio Stream. Output type: Video Stream.

      The avatar node currently supports connecting to an Avatar Plug-in or the Lingjing Digital Avatar Platform:

      • Avatar Plug-in:

        • Faceunity: You must contact Faceunity customer service to activate their 3D avatar service and obtain the AppID, AppKey, and AvatarId.

      • Lingjing Digital Avatar Platform: To integrate with the Lingjing Digital Avatar Platform, you need to submit a ticket to enable the service.

      Video Frame Extraction

      Extracts image frames from a video stream.

      Configure the Node Name (auto-generated) and Frame Extraction Frequency (1–30 frames/second). Input: Video Stream. Output: Image (Base64 format).

      Video Content Recognition

      Detects specific behaviors in video content.

      In the Video Content Recognition Node configuration, set the Node Name and select Recognition Content options: Still Frame Detection, Invalid Frame Recognition, Person Count, Head Movement Recognition, Electronic Device Recognition, Gaze Deviation Recognition, and Custom Model Detection. For Custom Model Detection, enter a Detection Task ID (up to 100 characters: numbers, letters, hyphens, and underscores; hyphens and underscores cannot be at the start or end).

      Multi-modal LLM

      The MLLM processes images and text from upstream nodes to generate natural language responses. Select Multimodal Model or Text-to-Text Model as the input mode.

      Configure the System Persona (role, objectives, capabilities, constraints; up to 3,072 characters) and Conversation Memory Rounds (1–30). More rounds retain more context but may increase processing time.

      Alibaba Cloud Model Studio

      A platform for large model development and application building. Connect through the Model center or the Application center.

      Tongyi Xingchen

      Tongyi Xingchen enables deeply personalized agents with unique personas, combinable with avatar-based real-time voice interaction.

      • ModelId: Tongyi Xingchen offers five models: xingchen-lite, xingchen-base, xingchen-plus, xingchen-plus-v2, and xingchen-max.

      • API key: Go to the Tongyi Xingchen console to create and obtain an API key.

      Self-developed model

      Connect your self-developed large model using the OpenAI specification.

      OpenAI specification: Enter the following parameters:

      Parameter

      Type

      Required

      Description

      Example

      ModelId

      String

      Yes

      The model name. This parameter corresponds to the model field in the OpenAI specification.

      abc

      API-KEY

      String

      Yes

      The authentication information. This parameter corresponds to the api_key field in the OpenAPI specification.

      AUJH-pfnTNMPBm6iWXcJAcWsrscb5KYaLitQhHBLKrI

      Model URL (HTTPS)

      String

      Yes

      The service request URL. This parameter corresponds to the base_url field in the OpenAPI specification.

      http://www.abc.com

      Maximum Number of Images per Call

      Integer

      Yes

      Some multi-modal large models limit the number of image frames that can be received in a single request. You can set this parameter to adapt to different models. During a request, video frames are automatically sampled according to this value.

      15

      MLLM Standard Interface.

    3. Click Save to create the audio/video workflow.

    Step 2: Create an audio/video agent

    1. Log in to the Intelligent Media Service (IMS) console and click Create AI Agent.

    2. Configure the basic information and bind a real-time audio/video workflow.

      1. Bind an audio/video workflow that the agent will follow.

        Set the Workflow Type (Audio Call, Avatar Call, Vision Call, Message-based Conversation, or Video Call) and select the workflow from the Workflow ID dropdown.

      2. Select an existing ApsaraVideo Real-time Communication (ARTC) application under your account. If none exists, the system can create one automatically. Introduction to ApsaraVideo Real-time Communication.

        Note

        Real-time Conversational AI relies on an ARTC application as the communication bridge for conversations.

      3. For Audio Call workflows, you can upload a custom image displayed during calls.

        Enable Custom Agent Image, set Image Source to Specify Image URI or Upload Image, and provide the image. Supported formats: GIF, PNG, JPG (max 3 MB).

    3. Click Submit to create the audio/video agent.

    Step 3: Test the agent

    After creating the agent, test it by scanning the demo QR code.

    1. Generate a demo QR code in the console.

      In the left-side navigation pane, click AI Agents. Find the target agent and click Demo QR Code in the Actions column. Select an expiration time (1 hour, 7 hours, 24 hours, or 3 days) and click Generate.

    2. Scan the QR code with DingTalk, WeChat, or a browser, or copy the demo URL into your browser to use the H5 demo.

      In the Demo QR Code dialog, the APP QR code is on the left and the H5 QR code is on the right. The H5 Demo URL, Experience Token, and Expiration Time are displayed at the bottom.

    Integrate the agent

    Gather the following parameters to integrate the agent into your project. Integrate an audio/video agent.

    • Region ID: The region of your workflow and agent. Find your Region ID in the region selector at the top-left of the console.

      Region name

      Region ID

      China (Hangzhou)

      cn-hangzhou

      China (Shanghai)

      cn-shanghai

      China (Beijing)

      cn-beijing

      China (Shenzhen)

      cn-shenzhen

      Singapore

      ap-southeast-1

    • The AppId and AppKey of the ARTC application.

      On the AI Agents page, click Basic Information. In Workflow Configuration, find the RTC AppID in the ApsaraVideo Real-time Communication (ARTC) Application field.

      In the ApsaraVideo Live console, go to Live+ > ApsaraVideo Real-time Communication > Application Management. Click the target application and copy the values from Basic Information.

    • AccessKey ID and AccessKey secret: To obtain them, see Create an AccessKey pair.