How Serverless Helps Enterprises Expand Their Business Systems

1、 The pain of enterprise application and business system development

There are many pain points in the traditional enterprise application development process. For example, project development, deployment, construction and maintenance at the business logic level, such as resource evaluation, server configuration, server security control and server operation and maintenance at the server use level.

In addition, from the level of operation and maintenance and cost expenditure, traditional applications are also faced with dual challenges of efficiency and cost. For example, continuous operation will cause energy loss, and long-term standby status will also cause cost loss for enterprises.

Applications developed by enterprises often carry the business system of enterprises. In terms of architecture selection, there are many technical products and architecture concepts on the market, and appropriate new technologies need to be selected to achieve rapid business innovation; Secondly, the distributed system technology is complex, and it needs high learning costs and operation and maintenance costs to put into use, such as how to manage the flow and control the underlying elastic resources when integrating micro-services, which will also bring great challenges to the business; Finally, the degree of integration of technology portfolio is insufficient, and there is a lack of overall planning and integrated end-to-end solutions.

Buying a wide variety of resources will also affect the efficiency of business system construction. Resources, networks, traffic governance, architecture design, resource operation and maintenance, observability, run-time adaptation, multilingual or task scheduling, etc. will become obstacles to R&D efficiency.

Serverless can help enterprises from technology upgrading to cost reduction and efficiency improvement.

So, what is Serverless, what does it do, and what can it do?

There are three core concepts of Serverless: pay-as-you-go (cost level), serverless operation and maintenance (DevOps level), and extreme flexibility (infrastructure level).

In the early stage, Serverless mainly relied on extreme flexibility to lead the exploration of the industry. Now, Serverless should really be implemented to solve the problems of enterprise business systems.

Serverless advocates fragmented programming, only cares about business code, and the whole structure is relatively agile. The platform has built a good context, and developers can quickly get output by doing code development, which greatly improves the efficiency of developers.

Through Serverless, a medium-sized business system can be implemented at the minute level, and online POC can be completed quickly. With 100ms of extreme flexibility, it can help online businesses cope with all kinds of sudden traffic, and the resource delivery ability is very strong. Although the cold start problem of function calculation cannot be avoided, we are also trying to solve this problem through continuous iteration. In addition, the resource utilization rate can reach 100%, charging by request, 1ms computing granularity, and only need to pay for the actual consumption of resources.



In addition, building its own business system based on the function computing system still requires platform-based operation and maintenance at a large level, and the operation and maintenance exemption is only limited to the function computing level.

The cost reduction of Serverless is applicable to the scenario with obvious tidal flow, and the business scenario without obvious peak and trough needs a long time to get a good cost reduction effect.

2、 Atomization capability of the Serverless system out of the box

One of the characteristics of seckill or online video live broadcast activities is that the traffic is uncertain, so the bottom layer must have the ability to provide resources flexibly, which is two dimensions with the flexibility to start the business. Code startup is mainly reflected in the RT level, and resource flexibility is mainly reflected in the QPS level.

Running a piece of code often requires several stages, such as downloading code, starting container, runtime initialization, function initialization, function execution, etc. We optimized the function calculation and realized the startup of 100 milliseconds.

On the system side, in order to avoid the re-creation of resources and enable the reuse of resources under the same tenant or function, we realized the pooling of instance resources and the prediction of instance concurrency, and quickly provided universal templates through memory mirroring. In terms of network, in order to connect with the user's VPC, a plug-and-play VPC optimization scheme is provided to quickly plug the network capability into the running instance of the function. Finally, the function execution, profiling ability and acceleration ability will be fed back to the subsequent operation.

Only when the request processing of each business is effective can it be reflected in the efficiency improvement of the whole business. As an event-driven type, Serverless needs to help the system connect with cloud products of various data types, such as API gateways, Kagka, logs, etc. After getting through the data source, you can use the event-driven method to process the data deposited in the product, and finally use the data in the function calculation to achieve the business objectives of the system.

In addition to data products, customers who research and develop operations and maintenance may be concerned about whether the ECS is full, whether there is a problem with cloud monitoring, VPC creation and other operations and maintenance related events, and also hope to be able to use the operations and maintenance events to actively call the SDKs of other cloud products to create related resources.

AliCloud's cloud products can be connected to function computing through EventBridge, and operation and maintenance operations can be performed based on events.

The most common message middleware in enterprise business systems is system decoupling products. It is very easy to write the consumer end of the business message, and the purpose of using function calculation to consume is to enable the pull message to pass the serviceability of the platform and realize the resource elasticity of the back-end business message processing. When there is no message, the resources calculated by the function can be recycled without calculating the cost.

The whole function calculation realizes the stateless consumption message capability, which is similar to the reactive mode. The stateful part is provided by the trigger capability of the platform. The stateless, business-related or user-defined part is precipitated into the function, which ensures the business customization and the responsive form of resources.

Common application gateways usually need to buy servers, write code, and write thread pools. There are many problems to be solved in the above processes: how many machines to buy? What is the concurrency of the thread pool at runtime on each machine?

In the function calculation scenario, requests processed by asynchronous tasks can be sent to Async Service, through which requests can be concurrently scheduled. The function is similar to the thread pool model with unlimited capacity. It can make managed requests and perform requests reliably, which can ensure the execution of at least once. Flow control can be performed when applying for flexible configuration of functions. In addition, execution management is provided to view the execution of each phase or cancel the task during execution.

After execution, the service results are provided. No additional functions need to be written. The results can be delivered to the next function or message queue, so that unrelated loosely coupled products can further process the results.

Serverless provides the system's out-of-the-box observation capability, which can view the RT of request execution, CPU utilization, memory usage, and the time spent in each phase of the call chain.

3、 How do enterprises use Serverless to rapidly expand their business systems

Enterprises will face a problem in the process of expanding their business systems: whether they need to migrate their full business to Serverless. If it is a relatively streamlined business system, full migration is feasible; If it is a large business system that has accumulated for many years, it needs to be deconstructed to migrate some capabilities to Serverless. For the latter, how to offload the request to the function calculation through loose coupling?

First of all, you can access functions through HTTP Restful (event functions are currently not supported, and the subsequent plans support URL access). In addition, it can be delivered to OSS or message queue, and the loose coupling decoupling of DB triggers the downstream business system.

The business system also needs to have a running context. It can write the programming specification according to the spec of function calculation. It also provides the way of image or custom runtime to run the original image on the function calculation (container transformation is required). The running context remains the same, but the underlying computing resources are moved to function computing.

The following principles are recommended for system deconstruction.

First, split the task class to process the business logic and move it to function calculation. You can use SDK, HTTP, timing trigger or event-driven submission methods to make the task execute on function calculation, which can realize the convenience of paying on demand or development of function calculation, without too much intrusion into the original business system.

Second, split the business message processing logic of MQ and use event-driven business message processing.

Third, split the file business logic and use the OSS event-driven method for rapid processing. After production, store it on OSS to complete the whole process without writing additional systems.

Fourth, split the lightweight data processing business logic and use the Serverless ETL capability to deliver it to downstream systems such as OSS, ElastisSearch, RDS, etc.

4、 Analysis of the dilemma faced by enterprises using Serverless system

There are three main driving forces for enterprises to migrate and upgrade: business drive and architecture upgrade; Efficiency driven, technology upgrading; Cost reduction drive and technical transformation.

In the process of migration, enterprises will face the following difficulties:

First, it is difficult to migrate and transform, which requires the programming specification and runtime requirements of adaptive function calculation.

Second, there is a maximum time limit for function operation, which is not friendly to stock business migration.

Third, the function calculation request model is quite different from traditional applications, and the concept of concurrency is difficult to understand. Function calculation provides two methods, namely, one request for one instance and multiple concurrent instances for a single instance.

Fourth, the calculation instance is transparent to users, and it is difficult to handle errors and troubleshoot problems.

Fifth, the billing model is complex and difficult to understand, and I am worried that the cost is not controllable.

Sixth, cold start. The current link delay is 3-5 ms.

5、 Customer scenario case sharing

Case 1: Focus Media Advertising Effect Identification

Focus has developed a picture recognition processing system. After changing the poster, the staff can take photos and upload them to the background server through APP. Weekend is the peak of change, and the peak flow is more than 10 times higher than normal.

Case 2: IOT monitoring video return scheme

The monitoring video of IoT in the consumer electronics industry is sent back to the OSS. The OSS needs to process quickly through function calculation, intercept file content, etc., and then connect the processing results with the business system. The average daily call volume is about 8w+QPS.

Case 3: Microblog massive image processing

The massive image processing of Weibo mainly uses function calculation to process images quickly, and uses millisecond elastic scaling to process images through event-driven model.

Case 4: Content review of live streaming

Live streaming in the interactive entertainment industry needs to achieve real-time and segmented security audit or authentication of online video and video interaction, which can be achieved through function calculation.

Case 5: Analysis of client data stream processing

After the data of the game industry is generated, the message queue reported to kafka needs to be processed with kafka's ETL capability.

Case 6: Card game battle settlement

In the card game scenario, calculation occurs only after each battle, which is also very suitable for using Serverless.

Case 7: Packaging scheme of game distribution channel

The scenario of game distribution channel packaging is implemented through event-driven.

Case 8: Internet of Vehicles platform

After collecting data from the end, the Internet of Vehicles platform transmits it to Kafka or RabbitMQ for processing.

Q&A

Q: What is the approximate cold start time of GPU container?

A: The starting time of the container is related to the reference model size in the GPU. For example, it is a customized image of 3G, and the cache acceleration is about 2 seconds; If there are more than ten megabytes of files in it, it may take 10-30 seconds.

Q: Is it feasible to migrate services to function computing and want to check the operation with Arthas tools?

A: Currently, Arthas is not supported. Java has relatively strong dynamic injection capability, so it is easy to implement Arthsa support. The function calculation is multilingual, so it is difficult to provide special support for observation or diagnosis for Java or micro-service scenarios. Arthas support may be implemented for subsequent web scene adaptation.

Q: Is it appropriate to move the microservice scenario directly to function calculation?

A: Not suitable. In the lambda architecture, the gateway plus method role is usually more suitable. We provide HTTP functions for fast web, as well as URL or custom domain name, which is relatively long tail, so it is also suitable for function calculation. If it is a high-frequency request, it is not suitable for direct handling. It is necessary to configure the gateway's ability first, and at the same time, cooperate with the call of method (method call is unbalanced, some methods are high frequency, some are low frequency) to provide the complete ability of service.

In terms of concurrency, single instance and multiple requests can be used to reduce the average startup time of cold start, because most requests are on one instance.