Live streaming latency accumulates on the stream ingest client, on the server, and on the playback client. Locate the stage that introduces the latency, and then apply the configurations that reduce it.
Causes of live streaming latency
On the stream ingest client
A Group of Pictures (GOP) is the fundamental unit for video encoding and decoding, and it starts with a keyframe. Live streaming adds a timestamp to each frame and then transmits the frames over the network. A large GOP size delays playback startup, because the playback client must buffer more data before playback can start.
To resolve stuttering, most third-party stream ingest software increases the encoding buffer. However, an oversized encoding buffer adds latency.
Hardware limitations can prevent the encoder from meeting the required bitrate, frame rate, and encoding profile settings. The resulting encoding delay adds to the end-to-end latency.
To address these causes, see Configure the stream ingest client.
On the server side
Before playback starts, part of the live stream data is cached to ensure fast startup and reduce stuttering. The cache keeps the data flowing smoothly, but cached data adds latency. During playback, network jitter can prevent data from reaching the client, which adds 2 to 3 seconds.
To reduce the server-side cache, see Configure the server-side latency level.
On the playback client
Most playback clients that do not support fast-forwarding start decoding the live stream only after the receive buffer is full. The playback client must fill this receive buffer before playback can start.
Reduce live streaming latency
The following configurations reduce latency at the three stages where it accumulates: the stream ingest client, the streaming protocol, and the server side. If your playback client uses HLS, start with the streaming protocol, because HLS has higher latency than HTTP-FLV and RTMP. Otherwise, configure the stream ingest client first, and then reduce the server-side cache. For millisecond-level latency, see Real-time Streaming (RTS).
Configure the stream ingest client
GOP size — Set the GOP size (keyframe interval) to 1 to 2 seconds to shorten the time that the player spends loading GOP data and to lower latency. To set the GOP size in the console, see Custom transcoding. To configure the GOP size by using an API operation, see AddCustomLiveStreamTranscode.
Encoding buffer — If an oversized encoding buffer causes the latency, use the Alibaba Cloud Push SDK to reduce the latency.
Encoding mode — On iOS stream ingest devices, use hardware encoding, because it is efficient and saves power. On Android, the wide range of device models and CPU types means that hardware encoding may cause compatibility issues, so use software encoding for stream ingest on Android.
Select a streaming protocol
ApsaraVideo Live supports three streaming protocols: HTTP-FLV, HLS, and RTMP. Consider the following playback constraints before you select one:
Player dependency — HTTP-FLV and RTMP require a Flash player. HLS is natively supported by most modern web browsers.
Mobile browsers — To watch live streams in a mobile browser, you can use only the HLS protocol.
Latency and compatibility — HTTP-FLV and RTMP have lower latency and suit scenarios that require low-latency playback. HLS has higher latency but better compatibility, and suits scenarios that are not sensitive to latency but require playback on more devices.
The following table compares the HTTP-FLV, HLS, and RTMP protocols.
| Protocol | Description | Transport protocol | Container format | Use case |
| HTTP-FLV | Developed by Adobe. Encapsulates streaming data in the FLV format and transmits the data to clients over HTTP, with about 2 seconds of latency. Supports encrypted transmission over HTTPS and playback on Android mobile devices. | HTTP | FLV, TAG | Low-latency streaming |
| HLS | Developed by Apple. An HTTP-based streaming protocol. Splits streaming data into consecutive TS segments. Each segment is longer than 5 seconds, and the number of segments is typically 3 to 4, so the total latency is about 10 to 30 seconds, but playback is smooth. Mainly used on iOS devices, and provides services such as live audio and video streaming and recorded content. | HTTP | M3U8, TS | Cross-platform compatibility |
| RTMP | Developed by Adobe. During transmission, messages are split into smaller chunks, the chunks are transmitted over TCP, and the receiver reassembles the chunks into streaming data. This complexity can cause stream instability. On iOS, a third-party decoder is required for playback. | TCP | FLV, TAG | Interactive live streaming |
If your playback client uses the HLS protocol, the latency listed in the preceding table is expected. To reduce it, adjust the server-side settings. For lower latency, switch from the HLS protocol to the FLV protocol.
Configure the server-side latency level
Reduce the server-side cache to lower live streaming latency. In the ApsaraVideo Live console, you can set the latency for each streaming protocol. The lower the latency setting, the smaller the server-side cache.
Log on to the ApsaraVideo Live console.
In the left-side navigation pane, click Domain Names to go to the Domain Management page.
Find the streaming domain that you want to configure and click Domain Settings in the Actions column.
Click .
The page displays latency level settings for three protocols: RTMP Latency, FLV Latency, and HLS Latency. Click Modify Configuration to adjust the settings. The actual latency depends on both the latency level that you set and the GOP size of the ingest stream, so keep the GOP size at 1 to 2 seconds.
After you reduce the cache, data cannot be downloaded in time when the network is unstable, and the live stream stutters. If stuttering occurs, click Modify Configuration again and restore the previous latency level. For more information, see Latency Settings.
Real-Time Streaming (RTS)
If the preceding configurations do not bring latency below the 3 to 6 seconds that is typical of traditional live streaming, use Real-time Streaming (RTS). RTS provides millisecond-level live streaming latency in scenarios with tens of millions of concurrent viewers. It suits large interactive streaming scenarios and delivers low stuttering and fast startup.
RTS billing — RTS is billed differently from standard live streaming. For the billing rules, see Alibaba Cloud Pricing page.
Activate the RTS service — For information about how to activate the RTS service and how to use it, see RTS overview.