Causes of live streaming latency and how to reduce it - ApsaraVideo Live

This topic describes the causes of high live streaming latency and explains how to resolve the issue.

Causes of live streaming latency

Stream ingest client

A Group of Pictures (GOP) is a collection of frames that begins with a keyframe. A GOP is the basic unit for video encoders and decoders. During a live stream, each frame is timestamped and transmitted over the network. A large GOP size can cause live streaming latency.
Many third-party stream ingest applications increase the encoding buffer size to prevent stuttering. However, a large encoding buffer can cause live streaming latency.
Hardware limitations can prevent the encoder from meeting the requirements for bitrate, frame rate, and high-profile encoding settings. This causes encoding delays that affect the live stream.

Server-side

Before playback begins, some live stream data is cached to ensure a fast startup and reduce stuttering. Although this caching helps ensure smooth data transmission, it also introduces live streaming latency. During playback, network jitter can prevent data from reaching the client, which can cause a delay of 2 to 3 seconds.

Playback client

Most playback clients that do not support fast-forwarding start decoding a live stream only after the receive buffer is full. The time required to fill this buffer causes live streaming latency.

How to reduce live streaming latency

You can configure the following settings to reduce live streaming latency:

Configure the stream ingest client
- Set the GOP size to 1 or 2 seconds. This reduces the time the player needs to load a GOP, which reduces live streaming latency. To set the GOP size in the console, see Custom transcoding. To configure it using an API operation, see AddCustomLiveStreamTranscode.
- If a large encoding buffer is causing high latency, you can use the Alibaba Cloud Push SDK to reduce it.
- For stream ingest devices, use hardware encoding on iOS clients for higher efficiency and lower power consumption. For Android clients, use software encoding. Hardware encoding on Android may cause compatibility issues because of the variety of device models and CPU types.

Select an appropriate streaming protocol

ApsaraVideo Live provides three playback protocols: HTTP-FLV, HTTP Live Streaming (HLS), and RTMP. You can select a protocol based on your streaming scenario.

HTTP-FLV and RTMP have lower latency and are suitable for low-latency playback scenarios. HLS has higher latency but better compatibility. It is suitable for scenarios that are not latency-sensitive but require playback on a wider range of devices.
HTTP-FLV and RTMP require a Flash player. HLS can be played directly in a browser.
To watch live streams in a mobile browser, you must use the HLS protocol.

Note

If your playback client uses the HLS protocol, a latency of 10 to 30 seconds is normal. If the latency is too high, you can adjust the server configuration. For even lower latency, you can switch from the HLS protocol to the FLV protocol.

The following table compares the HTTP-FLV, HLS, and RTMP protocols.

Protocol	Description	Transport protocol	Video container format	Recommended scenario
HTTP-FLV	Developed by Adobe. It encapsulates streaming media data in FLV format and transmits it to the client over HTTP. The latency is about 2 seconds. It supports HTTPS encryption and Android mobile clients.	HTTP	FLV, TAG	Low-latency streaming
HLS	Developed by Apple. It is an HTTP-based streaming media protocol. It segments streaming media data into a series of TS shards. Each shard is 5 seconds or longer, and there are typically 3 to 4 shards. This results in a total latency of about 10 to 30 seconds but provides smooth playback. It is mainly used on iOS devices for live audio and video services and content recording.	HTTP	M3U8, TS	Cross-platform streaming
RTMP	Developed by Adobe. During transmission, messages are split into smaller units called chunks. These chunks are then transmitted over TCP. The receiver reassembles the chunks to restore the streaming media data. The process is complex and can sometimes be unstable. On iOS, a third-party decoder is required for playback.	HTTP	FLV, TAG	Interactive streaming

Configure the server
You can reduce the server-side cache size to decrease live streaming latency. You can configure live streaming latency in the console. The latency settings vary based on the streaming protocol. A lower latency setting corresponds to a smaller server-side cache.
1. Log on to the ApsaraVideo Live console.
2. In the navigation pane on the left, click Domain Names to go to the Domain Management page.
3. For the streaming domain that you want to configure, click Domain Settings in the Actions column.
4. Click Streaming Management > Latency Settings.
Note
If the cache size is decreased, data may not be downloaded in a timely manner if the network connection is unstable. This can cause stuttering during playback. For more information, see Delay Configuration.

Real-Time Streaming (RTS)

If the preceding methods do not resolve the high latency issue, you can use Real-Time Streaming (RTS). RTS provides millisecond-level latency for live streaming with tens of millions of concurrent viewers and is suitable for large-scale interactive streaming scenarios. RTS overcomes the 3- to 6-second latency of traditional live streaming and ensures an ultra-low latency, low stuttering, and fast startup experience.

RTS costs
The costs for RTS are different from those for standard live streaming. For more information about the billing rules, see the Alibaba Cloud Pricing page.
Activate the RTS service
For more information about how to activate and use the RTS service, see RTS overview.