How does Alibaba ApsaraVideo manage to support the revelry of 40 million people?

Alibaba video cloud ApsaraVideo
In the Double 11 shopping carnival this year, the live broadcasting of the gala night became a highlight globally. Official data disclosed that a total of 42.57 million people watched the live broadcast, with the concurrent watchers numbering up to 5.29 million at the peak. The H.265 real-time transcoding and narrow-band high-definition technologies which feature high computing complexity are implemented on the cloud. In fact, apart from the Double 11 shopping carnival, live videos have become the most heated topic on the internet in 2016. Behind the mushroom content are the upgrading of computing, storage and bandwidth resources as well as cost reduction. The appearance of ApsaraVideo dramatically lowers the threshold of live video technologies, enabling enterprises to quickly launch their own video businesses. As an engineer participating in the Double 11 video cloud in person, I would like to share some experience on how Alibaba Cloud ApsaraVideo managed to quickly create a large scale live broadcasting scenario for the Double 11 shopping carnivals.
In traditional video businesses, we are usually faced with these technical pains:
1. Generally, you need to deploy the upload service, cache service, storage server, video encoding and transcoding services, and scheduling service.
2. In terms of the hardware, you need to prepare a series of hardware and facilities including the IDC server rooms and CDN nodes.
3. For startup teams or individuals, this obstacle generated by technology and hardware settings is hard to surmount. And the biggest pitfall is the time- and effort-consuming process and the difficulty to achieve elasticity.
During the Double 11 shopping carnival, veterans realized these advantages thanks to Alibaba Cloud ApsaraVideo:
1. Fast deployment: Based on the customer SDK and service-end openAPI provided by Alibaba Cloud video service, you can launch a large-scale video business in as little as one week.
2. Technical maturity and stability: You can output high-quality video encoding and decoding services and enjoy stable and reliable CDN distribution service leveraging the years of technical accumulation and experience of Alibaba Cloud video service.
3. Cost-saving: ApsaraVideo service carries forward the cloud computing features and is paid as you go. It saves a large amount of manpower and material resources compared with traditional self-built IDCs and CDN establishment.
How to establish a video service using Alibaba ApsaraVideo:
In general, a video business will be divided into three important parts. We will explain this taking the live broadcasting business as an example: Part 1, functions on the client, including capturing and encoding of live videos, video stream pushing and playing over the internet, as well as advanced features such as beautify, bullet screens, and mic interaction. Part 2, functions on the business service end, including live broadcast room management, live stream status display, live video recording to on-demand videos, and the Boss system. Part 3, CDN functions, including live broadcasting domain name management, domain name acceleration, bandwidth traffic expansion and resource monitoring.
While the Alibaba Cloud video service provides client SDKs for the three parts mentioned above, not only including the functions needed on the above live broadcasting business end, but also many stream pushing optimizations on weak networks and instant loading of the first screen among other features. Live broadcasting openAPI provides open APIs for managing live broadcasting rooms, observing live stream statuses and configuring live video recording and screenshots among other features. The CDN management provides features including adding/deleting live broadcasting domain names, configuring and optimizing distribution acceleration, and monitoring bandwidth traffic. So you can use the SDKs and openAPI provided by Alibaba cloud video service to set up your own businesses quickly and conveniently.
 
 
Figure 1. End-to-end solution of Alibaba Cloud live broadcasting
The Double 11 shopping carnival organized by Alibaba this year pooled a number of hot stars of the year. Besides, the gala was broadcast live on the mobile Taobao client, Tmall client, and Youku Tudou client, the three traffic entries with a lot of active users. The huge live video concurrency behind this is easy to imagine. In the live broadcasting of this gala, we adopted the following optimization schemes:
1. The live broadcasting stream pushing end used H.264 coded streams to push the live videos to the cloud side. The transcoding service on the service end then converted the H.264 into H.265.
2. Through CDN, the streams were distributed to various business parties. The client players should support H.265 video playing according to a certain mark.
3. We extended the standard protocol on the video cloud service end to support distinguishing H.264-coded streams and H.265-coded streams.
4. We modified the transcoding service module on the service end to support H.265 transcoding.
5. We adopted respective primary/standby solutions for both the network lines and live broadcasting centers and contingency measures to ensure smooth live broadcasting in case of various emergencies. The live broadcasting, based on the actual rehearsal, can save around 30% bandwidth costs without compromising the definition.
6. We provide a full set of solutions for content screening: Apart from the necessary transcoding, recording, screenshot and other standard configurations for live broadcasting, Alibaba live broadcasting cloud service also added the intelligent pornography identification, face service, voice recognition and other AI-related features.
7. The bandwidth resources can be expanded on demand at any time. If the peak bandwidth volumes vary greatly, the system can also be optimized by virtue of Alibaba Cloud’s years of experience accumulated in the video field.
The live broadcasting optimization solution of Double 11 shopping carnival is one of its kind for more than 100 live broadcasting activities over the past year. With the boom of live broadcasting this year, our customers may face a traffic peak at any moment. With the live broadcasting cloud service, you can not only dynamically expand your computing, storage and bandwidth resources, the experienced architects of Alibaba Cloud live broadcasting cloud are also making efforts to optimize the live broadcasting system architecture to help our customers weather live traffic peaks one after another.
As a video cloud service, apart from being convenient for access and easy to cope with network stress, the ApsaraVideo product is empowered by its in-depth research into the video encoding and decoding technology - the core technology. Alibaba Cloud's video technicians have made years of studies and accumulations to lower the bit rate and increase the video definition.
We have always been religious about the video watching experience and have made an exclusive launch of “image quality reviver” service integrating the research achievements of years of cooperation between Alibaba and various universities and institutions in the video industry. Alibaba Cloud customers are able to feel the qualitative leap in the video watching experience. Through the processing by the image quality reviver, nostalgic old movies decades old can also be played with a high definition and the 30-FPS videos captured by general cameras can also offer a smooth watching experience like a 60-FPS one. Both of the mosaic from multiple compressions and the image jitter from mobile shooting can be eliminated or remitted, greatly improving the watching experience. Below is the broadcast level video processing product we launched:
1. High-frame-rate video remaking
At present, most of the film and television programs that we watch refresh 24 to 30 frames per second, which frame rate can satisfy the basic video watching needs. But in a fast-moving scene, a 30-FPS video player may be stuck, which is perceivable. With the rising mainstream resolution in the video industry, the issue of videos of a general frame rate getting stuck during playback is becoming more and more prominent. On the present popular 4K television, a 60-FPS freshing rate has become the basic requirement for watching a film. However, restricted by the performance of the shooting equipment, the frame rates of most video programs are below 30 FPS, hard to meet the demand for watching HD videos, except very few high-invested films such as Hobbit which adopted 48-60 FPS. In view of this, Alibaba Cloud launched the high FPS video remaking service. No matter whether it is the classic serials shelved in your program library for years, or the latest-made programs, any video sources of general frame rates can be remade into 60-FPS or even 120-FPS videos through the frame inserting algorithm. With this technology, any video service provider is capable of providing smooth watching experience of high-frame-rate videos.
2. 2K-to-4K remaking
At present, 4K televisions have gradually become the mainstream of homes using televisions. They can indeed provide a tremendous visual impact to the audience. But because of the scarce video sources, users who have bought a 4K television still use it more often to watch 1080P videos or lower. So many 2K-to-4K services are on the rise in the industry to remake 1080P films into a 4K version using video processing algorithms. Most of the current 2K-to-4K features are dependent on the classic super-resolution algorithm, so pseudo-4K videos in the market share many similarities, with obvious traces of artificial processing such as oil painting effects. The 2K-to-4K remake service launched by Alibaba Cloud introduces the latest super-resolution algorithm in the academic world, and draws on Alibaba Cloud's research results in video quality assessment. After years of accumulation in the film and television industry, the remaking effect of Alibaba Cloud 2K-to-4K service is second to none in the industry.
 
3. Source repairing
Today, 1080P and 4K definition has gradually become the mainstream, but many nostalgic old movies still enjoy a large audience. However, their images are fuzzy and full of noise because of the low resolution or media aging. On the other hand, some UGC produced in recent years also has a lot of mosaic and glitch noise due to excessive compression on internet platforms, which is far from being satisfactory for the audience. Therefore, Alibaba Cloud launched the image quality repair service for these damaged video sources. Through the deep learning network, the service can remove the mosaic and noise, and restore the erased lines, so that the low-definition film sources captured on the internet can regain the collection value, and nostalgic old movies can glow with new vitality.
ApsaraVideo products carry forward all the characteristics of cloud computing products such as “pay as you go” and “shared and economical” so that the video expression means is no longer exclusive to the film industry. Instead, with the video cloud service, ordinary businesses and individuals can also easily produce videos as a more visualized expression means.