Analysis on Mogujie.com live broadcasting architecture (2) 

Difficulties and optimization practices of live broadcasting
Difficulties
The problems faced by Mogujie.com in live broadcasting mainly involve iteration, cloud services, code quality, stability and performance.
The major problem of iteration is because of insufficient requirements analysis in the early stage. From project approval to the product launch, there were only around three weeks. Most developers were actually unfamiliar with the live broadcasting technology at that time. Once a new requirement is proposed, the product personnel list a pile of requirements for the launch, and sort the requirements by priority. The resulting problem is that the live broadcasting team has to work overtime quite often to solve online issues and new requirements for iterations. Apart from this, the fast iteration also gives rise to new and old bugs, and some issues to third-party service access also become difficulties of the live broadcasting.
The causes of stability issues mainly involve the following: Memory leakage, unstable client SDKs and hardware incompatibility issues and sequence exceptions brought about by the complicated threads. The stability issues mainly lead to the failures for the audience to watch the live broadcasting normally and for the broadcaster to broadcast normally. A series of optimization measures will be taken in the future to solve the stability issues.
Figure 10 is the performance issue. In the early stage of live broadcasting when cloud services were just introduced, hardware decoding was not supported. This will cause the mobile end of the broadcaster to get stuck or the mobile phone to get hot. In addition, it may also cause too-frequent refreshes of the comments list, as well as thumb ups, gifts and bullet screens rendering and high concurrency issues among others.
Live broadcasting optimization practices
1. Stability optimization
Mogujie.com teams have made significant effort in improving the stability, including code reviews, code specifications, and introduction of testing tools for static analysis and memory leakage; at the same time, the logs are also optimized to log the key procedures and error-prone processes. The logs can be viewed locally and reported for analysis.
Figure 11 shows a case in stability improvement. This is a multi-layer block embedding module for code optimization to improve the stability. Multi layers of blocks are embedded on the big screen and context parameter and calling are comfortable, but with poor readability. For example, only the first half can be displayed on a lap top and is almost unreadable. We can see that its readability is really poor. It may also cause calling stream loss and loop reference from embedded blocks. Later, the code was further modified and the multi-layer block embedding was changed to “single role” method calling, increasing code readability and maintainability. At the same time, the calling loss and memory leakage are effectively reduced.
The stability optimization also includes content leakage improvements. Some instruments are used to check the memory leakage. On iOS clients, MLeaksFinder is used to check for memory leakage in debug mode for live broadcasting components so as to eliminate memory leakage in the development phase.
Figure 12 demonstrates the usage of MLeaksFinder and Figure 13 presents its principles. Simply put, the principle of MLeakerFinder is that when VC is POPed, it will ping all the views in three seconds. If the ping is successful, it indicates the view is not released within three seconds, and memory leakage may have occurred. Despite some false reports, when a new segment of code is added, it gives a prompt so that you can locate any issues in the new code, serving to assist in the development.
Process streamlining is also a very important part of stability optimization and features a high customization level to generate or store some files locally. Key procedures, as well as errors and SDK warnings can all be logged. App lifecyles and VC lifecyles can be logged, too. In this way, we can easily find clues through logs to solve online issues.
2. Performance optimization
1) Slow room loading
Slow room loading is the most important part in performance optimization. Figure 14 shows a serial room loading process, which was adopted first. Part of the process is supported in the serial manner and then the stream is pulled. This will take a long time as the time of every step is added up and the total time will definitely exceed one second. As a result, the instant video loading requirement cannot be fulfilled.

Figure 15 shows an optimization for slow room loading. A pre-login mechanism is configured for extractable censorship; synchronization and asynchronization processing is performed, saving 500 milliseconds for cloud service pre-login and 50 milliseconds for joining the chat room, as well as 200 milliseconds for getting the live broadcasting room details. In total, 550 milliseconds is saved.


2) Messages
Messages in the live broadcasting scenario include comments or other kinds of messages. Frequent problems for this part include too-frequent triggers, lack of message cache pools and absence of cache list high calculation.
Figure 17 shows a previous messaging system. First, the VC calls the message management class to send the message interface, the message management class holds the chat room instance for sending, this interface is called back asynchronously to the live broadcasting room VC for receiving and caching the message content, and then merges and forwards the messages. Such a handling method is concise and clear. But with the business growth, the live broadcasting room VC bears too much message processing while coupling related messaging operations. Because the live broadcasting is a broadcasting room and an audience room, relatively speaking, it integrates many functions with a huge amount of code. Now the messaging function is also coded into the live broadcasting room, making the live broadcasting room VC a “God-like class” that is hard to develop and maintain.
Figure 18 represents the four procedures for message optimization
The first is the message registration. Upon entry to the live broadcasting room, messages will be registered to the message dispatching management class by the message type. The message dispatching management class holds NSHashTable internally and has a weak reference to the message registration object.
The second is the message sending. Various business modules call the sending interface of the message management class separately. The message management class calls the instance it holds internally to send messages, dropping the mediator of the live broadcasting room VC.
The third is the message processing. After receiving the message callback, the message management class calls the message dispatching management class to dispatch the messages (one-to-many) according to message registration in the first step. The message related to the comment list needs to enter the message pool for on-demand callbacks.
The fourth is the comment list refreshing. At the beginning, the view is refreshed immediately after the messages are received. In high concurrency, this will lead to too-frequent refreshes and may even take up half of the CPU resources in the worst case, causing the live broadcasting room to get stuck. After the optimization, messages in the message pool are regularly polled and are retrieved from the pool only when there is a message. The view is refreshed after the corresponding business processing.
The four procedures of messages are relatively clear and reasonable, both reducing the coupling of live broadcasting VC, and enhancing the messaging performance.
3) Animation
Animation also takes a large part in performance optimization. As shown in Figure 19, the reusability of animation controls, serious off-screen rendering, sequence frame image buffering are all problems here.
For thumb ups (bullet screens), we mainly impose restrictions on the frequency of thumb ups (bullet screens). The frequency of 20 times/second is enough to hype up the atmosphere in the live broadcasting room. If every one of the 100 users in the live broadcasting room clicks once, there will be 100 thumb ups (bullet screens), which will definitely cause the broadcaster side of the live broadcasting room to get stuck. This phenomenon can be avoided by our imposing a ceiling for the highest frequency.
Off-screen rendering can eliminate a lot of rounded corners (such as some reusable rounded corners in the live broadcasting room list) and replace them with images. Because off-screen rendering affects the CPU performance a lot, it will cause the list or live broadcasting room to become apparently stuck.
Highly-reusable small images of the sequence frames can be cached. But if some large gift animations with large images are played all the time in the live broadcasting room, they will occupy memory. They should be released immediately after use.
4) Logging
Figure 20 presents the initial logging procedure. These master processes will be logged to the disk and the data is retrieved from the disk and sent to the server. But this will increase the frequency of IO operations on the disk and network.
To this end, the process optimization shown in Figure 21 is made. Logs will enjoy the priority to be stored into memory. In case of exceptions, the logging data will be retrieved from the memory and stored into the disk. Then a test on the logging data sending will be conducted. Data will be sent when it reaches a threshold value, reducing the network and disk IO operations.
Figure 22 is the performance after optimized logging. Previously 50 logs are recorded once per second, occupying 70% CPU resources. After the optimization, only 35% of CPU resources are used.
3. Componentized live broadcasting
Figure 23 shows some function modules of componentized live broadcasting, including room management, message channel, red packets, gifts, and comments function modules.
Componentization mainly aims to solve the high code coupling between live broadcasting rooms. It provides customized function modules to define the UI functions on your own. When selecting the overall design scheme, we made a selection according to the demand with regard to the entire design mode:
1) We need a clear interface definition and interface-oriented programming, and we can rewrite the specific implementations to achieve self-defined UI and functions;
2) Low coupling and customizable function modules.
Based on the two requirements above, we selected the MVP designing mode. MVP can decouple the code well between various modules of the live broadcasting room, and boasts a sound interface layer. The specific View and Presenter can implement the logic of their respective interface layer and by replacing the implementations of View and Presenter we can achieve customizable UI. In addition, the combined calls of the Presenter interface layer of various modules of the live broadcasting room can also well support the customization requirements of function modules. Compared with the heavyweight MVC mode and MVVM of data two-way binding, MVP is more suitable for the Mogujie.com business scenario.
Figure 24 shows a broadcaster information display component. The componentized working mode of MVP greatly facilitates the subsequent SDK-oriented and platform-based processing. The access of live broadcasting playback can be completed by assembling the business component in the live broadcasting room for the playback, with a great ease in operation.
4. SDK-oriented processing
SDK-oriented processing has four main objectives:
1. Reduce the access costs of other Apps in the group;
2. Uniform interfaces and documents;
3. Sound framework design and extendibility;
4. Configurable business functions and UI customization;
Figure 25 displays the overall SDK-oriented client framework of Mogujie.com.
At the bottom is the core SDK for live video broadcasting. It contains two modules: one is the live broadcasting audio and video module, and the other is the live broadcasting IM module; the audio and video module integrates the Mogujie.com-developed face recognition, open-source GPUImage, and the cloud service login SDK; IM module contains the message dispatching module and the IM SDK. The message dispatching module is the one aforementioned which splits the message step into four sub-steps.
The business module is the componentization just mentioned, including some function modules.
Figure 26 showcases the functions of the core SDK. Creating live video broadcasting, joining live video broadcasting, registering IM callback, sending IM, and quitting live video broadcasting are all basic functions.
Figure 27 demonstrates the SDK business layer. It depends on the core SDK of live video broadcasting, with a business of its own on it. It is then componentized and can achieve customization through accessing other Apps.
5. Platform-based
Platform-based processing, on the one hand, provides better access for the business party (similar to the common SDK in the market which provides UID); on the other hand, it provides a convenient and accurate data platform inside the platform for distinction from the business side. This task is dominated by the back-end, and the client provides support to the interface changes related to live broadcasting. Because the client has been componentized and SDK-oriented, two main supports for quick access of the business party, and UI functions can be customized.
Figure 28 outlines the platform-based structure. On the top is the live broadcasting source, with platform-based processing in the middle including live broadcasting SDK access, live broadcasting data storage, live broadcasting reports and the background systems. The interactive live broadcasting, on-demand videos and IM cloud service are integrated at the bottom.
Live broadcasting devices
Pulling streams for playing videos on mobile phones may cause limited resolution and stability issues, Mogujie.com adopts professional equipment to push the stream for playing videos.
Figure 29 shows an interface of the professional equipment pushing the stream. The portrait and landscape modes are supported and can be switched with ease.
Mogujie.com mainly adopts the following solutions for pushing streams from professional equipment. Video collection can be carried out by a computer camera or professional camera equipment. After the video capture completes, content is pushed to the computer (the computer needs to install OBS software). OBS receives the captured video stream and uploads the stream to the cloud through video stream pushing for dispatching. The playing side is comparatively simple, and on-demand video SDK can provide the support. Figure 30 is the complete procedure chart.
One of the differences between professional equipment and normal live broadcasting is that there is no broadcaster side on the mobile phone, and only the camera collects the video streams. At this point, some problems may occur, such as room number generation, group chat creation, and business information acquisition. This information is currently processed by a series of allocation operations in the management background (the pushing stream address is obtained by clicking the button in the operation background to call the cloud service to enable the pushing stream channel interface). In addition, in collection for pushing streams, the OBS should be available in the computer to push the stream and then conduct the CDN dispatching. At last, during the playing, the room status is manually set. The live broadcasting has three states. The subscribed state when the live broadcasting is not started. When the live broadcasting is ended, the page will jump to the live broadcasting ending page. You can only enter the live broadcasting room for pulling stream playback in normal conditions. The room status is maintained in the operation background and push stream, disconnect and re-connect features are added.