New Oriental's Road to Serverless Practice

New Oriental Education and Technology Group is positioned as a comprehensive education group with the comprehensive growth of students as the core and technology as the driving force. The cloud classroom system of the online education business of New Oriental supports all online education scenarios of New Oriental, such as live video, transcoding and on-demand. With the increase of business volume, the low resource utilization of the self-built computer room has become the core pain point of the business due to the obvious peak-trough characteristics of the live broadcast transcription and video transcoding task processing platform.

In order to improve the utilization of computing resources and further achieve the goal of reducing costs and improving efficiency, New Oriental embarked on the path of Serverless practice after several attempts. The following content was shared by Mo Jingguo, the technical director of the cloud classroom live broadcast platform of New Oriental Education and Technology Group, at the Yunqi Conference.

How to deal with unpredictable business volume?

In addition to using the cloud classroom platform for its own online education business, New Oriental has also opened its live broadcast capability to the public through Meike Cloud Live. The cloud classroom live broadcast platform mainly supports four business modes:

• Cloud classroom: live interactive classes online, emphasizing interactivity.

• Cloud on-demand: recorded class.

• Cloud live broadcast: large-scale live broadcast, mainly the anchor.

• Smart classroom: a combination of software and hardware, providing an educational model similar to that of double teachers.

Live+recorded broadcasting is the main delivery mode of courses promoted by New Oriental. The live course is highly interactive, which can achieve good classroom interaction and stimulate students' interest in learning; Teachers can directly evaluate each other with students based on their classroom feedback or interaction, so that teachers can adjust the teaching environment in time, so as to make the teaching process more targeted; Through interaction and classroom Q&A, the teaching process and effect can be more guaranteed. The live course is more suitable for students of young age.

For students of high school and above, recorded and broadcast classes are more suitable. The characteristics of recorded and broadcast classes are flexible learning time. Students can independently search the learning content and conduct selective learning, which is generally applicable to students of high school and above. The advantage of the recorded and broadcast course is that it can continuously polish and edit the teaching content, and produce high-quality courses. The demand is gradually increasing. At the beginning, the team adopted the recording technology scheme to record the teacher's live broadcast for the client to facilitate the students to watch it repeatedly. However, this method has high error rate and high CPU usage, and can not flexibly customize the recording UI layout. It can only see what is recorded. This method can only meet the needs of the curriculum at a young age. This year, New Oriental began to connect with the online education business of college students, putting forward higher requirements for the quality of recorded and broadcast courses. The team began to consider using server-side recording to solve the problem. The two core points of server-side recording are live recording and video standardization production. Our business model determines that it is difficult for us to accurately predict business volume. Therefore, the key technical task of New Oriental is to achieve computing flexibility.

Three options: function calculation stands out

Three options: function computing stands out. To solve the problem of server recording, there are three optional technical routes before the team:

• Directly use ECS self-built. The advantage of this solution is that it is relatively flexible, but the problem is that the computing is not flexible. Although the cloud manufacturer provides an API for elastic allocation of ECS resources, it needs a huge amount of development to realize the entire computing flexibility by itself. At the same time, the subsequent operation and maintenance is relatively complex, and the resource cost is high, which makes it difficult to achieve standardization.

• Cloud video SaaS solution. The advantage of this solution is that it has standardized services, less research and development investment, and less operation and maintenance work. However, the problem lies in poor flexibility, high resource cost, and difficulty in further performance optimization. We hope to find a mature SaaS manufacturer to provide services to quickly support business, but after trial, the maturity and technical indicators of these platforms can not meet our needs.

• Using Alibaba Cloud functional computing FC, we found that Alibaba Cloud functional computing products can perfectly meet the elastic needs of computing. We only need to focus on the specific needs and develop on the platform. The R&D investment is small, and the operation and maintenance are free. The development process is independent and controllable, with high flexibility. It can be used on demand, greatly reducing the cost of use, and it is relatively easy to achieve standardization. However, function calculation is a relatively new technology, and the team needs a period of time to become familiar with it.

After repeated comparison, the New Oriental team chose to use function calculation to solve the recording problem on the server side.

New Oriental's Serverless Practice

Recording and broadcasting transcoding, function calculation

We first made an attempt in the video and broadcast transcoding scenario. The core requirement of recording and broadcasting transcoding is to transcode the live stream in real time and save it as a standard video format for subsequent processing and use.

In this scenario, for the first time, we feel the flexibility advantages brought by functional computing FC. After the teacher enters the room and initiates a transcoding request, you can quickly start the function instance for transcoding. At the end of the class, the transcoding task is ended, and the function instance can be released immediately after the temporary audio and video results are uploaded to the cloud storage, without any waste of computing resources.

With the experience of applying function calculation in recording, broadcasting and transcoding projects, we have greater confidence in the function calculation scheme.

The function calculation of live broadcast combined transcoding scheme

After that, we launched the cloud recording project. Use Chrome browser to join the live room and take a screenshot of the browser interface. The key of this solution is to provide the browser instance flexibly.

Therefore, we use Alibaba Cloud function computing to start the Linux container, and run Chrome browser on the Linux container to provide flexible browser instances.

The whole recording process is as follows: after the teacher enters the classroom, he/she begins to perform audio and video streaming and whiteboard operation. At the same time, the recording platform initiates the recording request, starts the function processing, starts to receive the audio and video stream and whiteboard operation of the classroom, displays the entire classroom picture in the browser, and takes a screenshot at the same time. At the end of the course, the platform initiates a request to end the recording, and the function computing platform gracefully terminates the instance. Before termination, the instance will upload the temporary results to the cloud storage, and then the function instance will be destroyed. There is no waste of resources in the whole process.

Out-of-the-box observability

We believe that the observability is crucial to the function calculation platform. First of all, a large number of function instances need to be started at the peak of business. Therefore, complete metrics, log and trace are required to effectively monitor massive instances. Secondly, because the function calculation instance is created on demand and destroyed after completing the task, the platform must save a complete log so that the developer can troubleshoot the problem after finding it.

The problem we faced in the process of developing the recording service is that after starting the function instance, Chrome browser needs to access the live service. At this time, there is a problem with the network, resulting in the recording failure. Later, we used Alibaba Cloud SLS log platform to view logs and found that the Chrome browser kernel is too sensitive to network processing. After finding out the problem, apply the right medicine to the case, and add a retry mechanism to solve the problem.

Exceed expectations: function calculation brings more surprises

Before using the function computing technology, we expect that it can pull up tens of thousands of instances in a hundred milliseconds, complete solve the difficulties of cold start and help us carry the business peak of live transcoding and screen recording. Effectively deal with large-scale sudden online traffic, pay as you go, improve resource utilization, reduce resource costs by 20%, and greatly reduce operation and maintenance costs, so that we can only focus on business innovation.

In the process of actual use, we found that function calculation can not only perfectly meet our needs, but also bring surprises: our developers can easily use the platform by mastering a few new concepts and using a few APIs. Since the function computing scheme has been running for some time, the cost of cloud resources has been greatly reduced. In addition, function calculation allows us to make templates according to our own business scenarios and can be used by other business parties, which also brings us unexpected benefits.