This topic describes how to solve the problem that the style and layout are changed when a co-streamer switches between video co-streaming and audio co-streaming.
Background information
In scenarios where a viewer co-streams with a streamer, the viewer (co-streamer) can choose audio co-streaming or video co-streaming. The display styles for audio co-streaming and video co-streaming are different. Conventionally, when the co-streamer switches between audio co-streaming and video co-streaming, the layout and style are changed based on the signaling message. This scheme has the following drawbacks:
Due to different transmission latencies of live streaming and signaling, the styles of app UI parts (such as the avatar and alias) may appear different from the style of the video stream.
If the co-streamer frequently switches between audio co-streaming and video co-streaming, the mixed-stream layout is also frequently updated, which may increase systemic risks.
Solution
In the video co-streaming mode, the co-streamer ingests a video stream that is used for co-streaming and stream mixing. When the co-streamer switches to the audio co-streaming mode, the client ingests a system image for stream mixing. At the same time, the display style of the live room is adjusted in real time based on the supplemental enhancement information (SEI).
The following examples show you the style handling methods when the co-streamer switches from video co-streaming to audio co-streaming.
Handling method for the styles of the streamer and existing viewers in the live room:
When you create an AlivcLivePushConfig object of the stream ingest engine, the pauseImg parameter passes in a default image. When the mode is switched from video co-streaming to audio co-streaming, this default image is displayed in the layout and used for stream mixing. This can prevent the mixed-stream layout from being frequently updated.
Co-streamer B calls muteLocalCamera to disable video ingest. On the client side of Co-streamer B, the avatar of Co-streamer B is displayed and audio co-streaming starts.
After the system notifies the app server of Co-streamer B's disabling video ingest, the app server preserves the status and broadcasts a message that indicates Co-streamer B's disabling video ingest to the streamer. After the streamer receives the message, the avatar of Co-streamer B is displayed on the client side of the streamer.
Co-streamer B calls sendMessage (the SEI method) to customize the video ingest disabling/enabling message body and send a message that indicates Co-streamer B's disabling video ingest to viewers.
After the viewers parse the custom SEI, the video of Co-streamer B is closed and the avatar of Co-streamer B is displayed in the mini-window on the viewer side.
The layout information of the mini-window for Co-streamer B is included in the periodic SEI. For more information, see Switch between layouts based on SEI.
Handling method for the style of new viewers who join the live room:
After entering the live room, a new viewer obtains the stream information such as the streaming URL from the app server and uses a CDN streaming URL to play the stream.
The new viewer obtains the live room information, the basic information about the streamer and co-streamer, the layout information, and the video ingest enabling/disabling status of the streamer and co-streamer from the app server.
If the co-streamer is in the video co-streaming mode, the user information about the co-streamer is overlaid in the video. If the co-streamer is in the audio co-streaming mode, the avatar of the co-streamer is displayed.
The layout information of the mini-window for Co-streamer B is included in the periodic SEI. For more information, see Switch between layouts based on SEI.