This topic describes how to reduce live streaming latency.
Causes that lead to live streaming latency
- A Group of Pictures (GOP) is a collection of keyframes in a video stream and is the basic unit to encode and decode the video stream. During live streaming, each frame of data is timestamped and transmitted over networks. A large number of keyframes in a video stream causes live streaming latency.
- Most third-party stream ingest software increases the encoding cache size to prevent playback stuttering. However, a large encoding cache size causes live streaming latency.
- The encoding hardware may not meet the requirements of the specified bitrate, frame rate, or encoding profile. This causes encoding latency and impedes smooth streaming.
Before the delivery of a live stream, the server caches a specific amount of streaming data so that the live stream can be instantly loaded and stuttering can be reduced. This ensures the smoothness of live streaming. However, caching data leads to live streaming latency at a specific level. During the delivery of a live stream, the data may not be transmitted to the client in real time due to network jitters. This also causes a latency of 2 to 3 seconds.
Most streaming clients that do not support fast forwarding decode and play a live stream only when the cache is full. The caching process causes live streaming latency.
How to reduce live streaming latency
You can reduce live streaming latency by using the following methods:
- Stream ingest client
- Set the GOP size to 1 to 2 seconds to reduce the time required by a player to load a GOP. This reduces live streaming latency. For more information about how to set the GOP size in the ApsaraVideo Live console, see Configure custom transcoding. For more information about how to set the GOP size by calling an API operation, see AddCustomLiveStreamTranscode.
- To reduce live streaming latency that is caused by a large encoding cache size, we recommend that you use the stream ingest SDK of ApsaraVideo Live.
- For iOS-based stream ingest clients, we recommend that you use hardware encoding because it is efficient and power-saving. The performance of Android-based stream ingest clients varies with the terminal type and CPU type. Hardware coding may cause compatibility issues. Therefore, we recommend that you use software encoding on Android-based clients.
Decrease the cache size to reduce live streaming latency. You can decrease the cache size by specifying lower live streaming latencies for different streaming protocols in the ApsaraVideo Live console. The following figure shows the page used to configure live streaming latencies for different streaming protocols.Note After the cache size is decreased, data may fail to be downloaded to keep pace with the playback if the network connection is unstable. In this case, stuttering occurs during the live streaming.
- Streaming protocol
ApsaraVideo Live supports the following streaming protocols: HTTP-FLV, HTTP Live Streaming (HLS), and Real-Time Messaging Protocol (RTMP). You can select the appropriate protocol based on your live streaming scenario.
Note If your streaming client uses HLS, a latency of 10 to 30 seconds is normal. To reduce the latency, you can change HLS to HTTP-FLV.
- HTTP-FLV and RTMP have low latency and are suitable for live streaming scenarios that require low latency. HLS has relatively high latency but is compatible with a wide range of terminals. It is suitable for live streaming scenarios in which latency is not sensitive and a large number of terminals are playing the live stream.
- HTTP-FLV and RTMP require Flash Player, whereas HLS video streams can be directly played in a browser.
- You must use HLS if users play your live stream in mobile browsers.
The following table describes the differences among HTTP-FLV, HLS, and RTMP.
Streaming protocol Description Transmission protocol Container format Applicable live streaming scenario HTTP-FLV Introduced by Adobe, HTTP-FLV encapsulates streaming data into the FLV format and transmits the data to clients by using the HTTP protocol. The latency is about 2 seconds. HTTP-FLV supports encrypted transmission by using HTTPS. HTTP-FLV is supported on both Android and iOS terminals. HTTP FLV and TAG Low-latency live streaming HLS Introduced by Apple, HLS is a media streaming protocol based on HTTP. HLS divides a live stream into consecutive TS segments. The length of each segment is more than 5 seconds. Typically, the playback starts after three to four segments are cached. Therefore, the total latency of HLS live streaming is about 10 to 30 seconds, but the playback is smooth. The HLS protocol applies to iOS terminals and is used to provide live streaming services and recording services. HTTP M3U8 and TS Live streaming across multiple terminals RTMP Introduced by Adobe, RTMP splits messages to small chunks during transmission. RTMP transmits the chunks to the receiver by using TCP. Then, the receiver decodes the chunks to obtain the streaming data. Splitting a large file is complex, which may lead to the instability. For iOS, a third-party decoder is required to play an RTMP stream. HTTP FLV and TAG Interactive live streaming
If live streaming latency is still unsatisfactory after you take the preceding measures, you can use Real Time Streaming (RTS). Compared with conventional live streaming that has a latency of 3 to 6 seconds, RTS supports the playback on tens of millions of concurrent streams at a latency of milliseconds. RTS is suitable for large-scale interactive live streaming events. You can use RTS to provide extreme live streaming experience that features low latency, stuttering-free, and instant loading.