Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Wan-Streamer: A Native-Streaming Model for Real-Time Audio-Visual Conversation

Wan-Streamer: A Native-Streaming Model for Real-Time Audio-Visual Conversation

Introducing our latest research: Wan-Streamer v0.2.

Real-time multimodal interaction is moving beyond text and speech toward fully audio-visual conversation.

Today, we’re introducing Wan-Streamer v0.2, the latest iteration of our research on end-to-end real-time audio-visual interaction. Building on the native-streaming architecture introduced in Wan-Streamer v0.1, this work explores how to significantly increase the resolution of the generated video stream while preserving the low latency required for natural, duplex conversation.

The result is a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model that raises its output resolution from 192×336 to 640×368 at 25 FPS, while maintaining approximately 200 ms model-side signal-to-signal latency. With a typical 350 ms bidirectional network budget, total remote interaction latency remains around 550 ms, allowing conversations to retain the responsiveness expected from live video communication.

Wan-Streamer v0.2:

Wan-Streamer v0.1:

Technical Approach

At its core, Wan-Streamer is a single autoregressive Transformer that jointly models omni-modal understanding and generation for real-time duplex interaction.

Native Streaming

Instead of treating speech recognition, language understanding, speech synthesis, and avatar animation as separate systems, Wan-Streamer unifies text, audio, and video into a single native-streaming interaction framework.

The model uses causal audio-video encoders and decoders, block-causal attention, full-history streaming inference, and low-latency multimodal token scheduling so that each short streaming unit can update perception, update interaction state, generate the next unit, and decode the previous unit.

1

Thinker–Performer Architecture

Real-time interaction requires balancing fast reasoning with computationally intensive generation.

Wan-Streamer addresses this through a Thinker–Performer architecture.

  • The Thinker is a low-latency path for streaming perception, language and state update, K/V construction, and final decoding.
  • The Performer handles the latent generation path, where most of the computational cost resides.

In v0.2, high-resolution video latent generation is handled by a multi-GPU Ulysses-style context-parallel performer, moving the added visual generation cost away from the latency-critical thinker path.

2

From Wan-Streamer v0.1 to v0.2

Wan-Streamer v0.1 established that end-to-end native-streaming audio-visual interaction is feasible. It demonstrated that a single model can continuously perceive, reason, and generate synchronized audio and video while maintaining low interaction latency.

Wan-Streamer v0.2 scales to higher resolution without altering the interaction experience. The key change is extending the Performer into a multi-GPU Ulysses-style context-parallel architecture.

It raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS.

3

Facial expressions and lip movements become clearer, while gaze direction, body posture, hand gestures, nearby objects, and local scene layout remain legible throughout a conversation.

This enables interactions that extend beyond close-up talking-head calls toward richer, scene-grounded mid-shot conversations without sacrificing responsiveness.

Looking Ahead

Building AI systems that can participate naturally in live conversation requires more than realistic speech or visually convincing avatars.

We see this work as another step toward real-time multimodal agents that integrate understanding and generation within a single interaction loop, making AI communication feel increasingly natural.

This is an ongoing line of research — we’ll continue releasing further iterations of Wan-Streamer. Stay tuned.

Source

AI Artificial Intelligence WAN Generative AI GenAI Multimodal AI Wan-Streamer Alibaba Wan Native-Streaming Model Real-time Audio-Visual Conversation Thinker-Performer Architecture
0 1 0
Share on

Read previous post:

Qwen-Coder-Qoder: Customizing a Fast-Evolving Frontier Model for Real Software

Read next post:

Alibaba Cloud Unveils Agent-Native Innovations at WAIC 2026

Alibaba Cloud Community

1,466 posts | 503 followers

You may also like

Comments

Alibaba Cloud Community

1,466 posts | 503 followers

Related Products

More Posts by Alibaba Cloud Community

See All