Alibaba Cloud Push SDK is a software development kit (SDK) for ingesting streams from clients. This SDK is built on Alibaba Cloud's powerful content delivery network (CDN) and real-time audio and video communication technologies. It provides easy-to-use APIs, a smooth network-adaptive experience, low-latency optimization across multiple nodes, and powerful real-time retouching for audio and video live streaming. This topic introduces the Push SDK, its features, core advantages, scenarios, and workflow.
Product introduction
Alibaba Cloud Push SDK is a development tool for ingesting streams from clients. This SDK is based on Alibaba Cloud's powerful CDN and real-time audio and video communication technologies. It provides easy-to-use APIs, a smooth and network-adaptive playback experience, latency optimization across multiple nodes, powerful real-time retouching, and other audio and video live streaming services. The Push SDK is free of charge. It reduces architectural design complexity and maintenance costs, allowing you to focus on your business logic and user experience. For more information, see the Push SDK page.
Alibaba Cloud Push SDK integrates the smart retouching effects from Queen. It provides advanced retouching features that use facial recognition, such as face slimming, face resizing, eye enlarging, and skin whitening.
To integrate the smart retouching feature of Queen, you must apply for a license. For more information about the product and the application process, see Smart retouching.
Scenarios
Scenario 1: Education live streaming
Scenario description: Education live streaming focuses on the interaction between teachers and students. In a live classroom, you can use Alibaba Cloud's Interactive Messages to enable real-time text-based interaction. The Push SDK allows teachers to answer student questions in a timely and effective manner. The Push SDK also provides cloud-based recording and transcoding features. This allows students to replay class video recordings at any time to review content and enhance their learning.
Usage notes: Activate ApsaraVideo Live and enable the recording and transcoding features. Integrate the Push SDK and the Interactive Messages SDK to implement live classrooms or question-and-answer (Q&A) services. Use the ApsaraVideo Player SDK on the playback client to watch live streams or replay video courses in a low-latency, highly interactive education scenario.
Scenario 2: Entertainment live streaming
Description: Entertainment live streaming has become popular due to the widespread use of mobile phones. Retouching and filter effects are essential for streamers. Streamers interact with viewers through real-time chat, likes, and rewards to increase their popularity and engagement. However, the barrier to entry for mobile entertainment live streaming is low. The content must be strictly moderated for security reasons, such as for pornography and terrorism. You can use the pornography detection feature for live streaming to reduce moderation costs.
Usage notes: Activate ApsaraVideo Live and enable the recording and pornography detection features. Integrate the Push SDK and enable the retouching feature for stream ingest. Integrate Interactive Messages into your interactive chat scenarios. This allows users to send information, such as text and images, in the chat panel during a live stream. You can also use this feature to build a custom gift system by combining Instant Messaging (IM) with a payment service. On the playback client, use the ApsaraVideo Player SDK to watch live streams or replays.
Scenario 3: Game live streaming
Scenario description: Mobile game live streaming is implemented using screen recording technology to merge the game screen with the camera feed. The Push SDK is then used to ingest the merged stream, which requires the SDK to support the screen recording feature. The interaction between streamers and viewers is similar to that in entertainment live streaming. You can use the Alibaba Cloud Interactive Messages SDK to implement interactive features such as chatting, liking, and rewarding. To replay exciting clips from the game, you can use the live stream recording service.
Usage instructions: Activate ApsaraVideo Live and enable the live stream recording service. Integrate the Push SDK and use its live stream recording feature. Integrate Interactive Messages into your interactive chat scenarios. This allows users to send information, such as text and images, in the chat panel during live streams. You can also use this feature to build a custom gift system by combining Instant Messaging (IM) with a payment service. Integrate the ApsaraVideo Player SDK to enable instant playback and dynamic frame synchronization for watching live streams or highlight replays.
Interactive streams can use the ARTC + bypass push solution. For more information, see Introduction to co-streaming.
Core advantages
Simple and easy to integrate
The SDK provides unified APIs and error codes for Android and iOS. It also offers synchronous and asynchronous APIs to meet the integration needs of different development architectures. Comprehensive API documentation and demos are available for reference.
All-in-one solution
The SDK provides an all-in-one video live streaming solution that covers video capturing, rendering, stream ingest, transcoding, delivery, and playback. The adaptive bitrate stream ingest on the client, Narrowband HD transcoding in the cloud, and instant playback on the viewer's client work together seamlessly to provide a high-quality, end-to-end service.
High performance and low latency
The SDK offers industry-leading performance with a low stream ingest stuttering rate, low CPU and memory consumption, low power consumption, and minimal heat generation. More than 2,500 global live streaming nodes ensure low latency across all regions.
WebRTC stream ingest
The SDK provides Web Real-Time Communication (WebRTC) stream ingest based on the UDP protocol. The console supports self-service activation of WebRTC ingest domains. WebRTC stream ingest provides better resistance to stuttering when the upstream network quality is poor.
Basic concepts
Bitrate control: An optimized encoding algorithm used to control a video stream's bitrate. For the same video encoding format, a higher bitrate contains more information and results in a clearer image, and vice versa.
Frame skipping: When sending video frames, if the network quality is poor and causes a severe backlog of video frames, you can drop frames to reduce stream ingest latency.
In-ear monitoring: Allows a streamer to hear their own voice in real time through headphones. For example, when a streamer is singing with headphones, they need to control their pitch. In this case, they must enable in-ear monitoring. This is because the sound transmitted through the network to their ears is very different from the sound transmitted through the air. The streamer needs to hear the audio as the viewer hears it.
Audio mixing: Combines sounds from multiple sources into a single stereo or mono audio track. The Push SDK supports mixing music and human voices.
Features
Feature | Description |
RTMP stream ingest | Supports stream ingest over the Real-Time Messaging Protocol (RTMP). Supports stream pulling over RTMP, FLV, HLS, and ARTC. Supports resolutions from 180p to 720p. We recommend that you use 540p. Note The Alibaba Cloud Push SDK does not support stream ingest over the Real Time Streaming Protocol (RTSP). To implement RTSP stream ingest, use a third-party tool, such as FFmpeg, to convert the RTSP stream to the RTMP protocol. |
WebRTC stream ingest | Supports stream ingest over the ARTC protocol based on UDP. |
Screen recording live stream | iOS supports screen recording for live streaming using ReplayKit. Android supports stream mixing of camera and screen recording for live streaming. Supports Narrowband HD, which provides better image quality at the same bitrate. |
Insert SEI information | Supports inserting SEI messages into live streams. You can parse SEI messages on the player to implement various business features. |
Animated watermarks | Supports adding or removing watermarks with animation effects in real time during a live stream. |
External audio and video stream ingest | Supports ingesting external audio and video data streams for live streaming. |
Background image ingest | Supports ingesting a specified image when the app is switched to the background. Also supports replacing the video stream with an image when the network connection is poor. |
Audio and video encoding | Supports H.264 video encoding (software and hardware) and AAC audio encoding (software and hardware). |
Real-time retouching | Supports advanced retouching based on facial recognition, including skin smoothing, skin whitening, face slimming, face resizing, and eye enlarging. |
Adaptive bitrate | Supports automatic adjustment of the ingest bitrate based on network conditions. Supports multiple modes to make live streaming smoother. |
Adaptive resolution | Supports automatic adjustment of the ingest resolution based on network conditions. This feature is available only in definition and low definition modes. |
Background stream ingest | Supports continuous stream ingest after the app is switched to the background. The stream ingest continues when the app returns to the foreground. |
Stereo stream ingest | Supports stereo stream ingest. You can set the stream ingest to mono or stereo. |
Multiple watermarks | Supports adding multiple watermarks (up to three). You can set the position and size of the watermarks. |
Landscape stream ingest | Supports starting stream ingest in three orientations: portrait, landscape left, and landscape right. |
Collection parameters | Supports setting multiple collection parameters, such as resolution, frame rate, audio sampling rate, GOP, and bitrate, to meet the needs of different scenarios. |
Mirrored stream ingest | Supports separate settings for mirroring the camera preview and the ingested stream. The mirroring feature must be enabled by default for the front camera. |
Audio-only stream ingest | Supports collecting and ingesting only the audio stream. This saves bandwidth and traffic in audio-only scenarios. |
Muted stream ingest | Supports muting the microphone during stream ingest to push only the video. |
Autofocus | Supports enabling or disabling the autofocus feature. You can also use manual focus. |
Camera zoom | Supports zooming the camera preview up to the maximum zoom ratio supported by the camera. |
Camera switching and flash | Supports switching between the front and rear cameras and turning the flash on or off (rear camera only). |
Background music | Supports background music playback, including start, stop, pause, resume, and loop playback functions. |
Audio mixing | Supports mixing music and human voices. You can adjust the volume of the music and voice separately. |
In-ear monitoring | Supports in-ear monitoring. For example, when a streamer sings with headphones, they can hear their own voice in real time. This is suitable for karaoke scenarios. |
Noise reduction | Supports noise reduction for noise caused by ambient sound and mobile phone interference. |
SDK workflow
Workflow of the basic Push SDK
The client app requests an ingest URL from the AppServer.
The AppServer constructs the ingest URL based on predefined rules and returns it to the app.
The app passes the ingest URL to the Push SDK and uses the SDK to start the stream ingest.
The Push SDK pushes the live stream to the CDN.
Development support
If you have any questions or suggestions when using the Push SDK, you can find technical support in Self-service troubleshooting for Push SDK issues.