Cloud Native Architecture | Serverless

1: Technical features

With the cloud native technology represented by Kubernetes becoming the container interface of cloud computing, Kubernetes has become a new generation operating system of cloud computing . Back-end cloud services (BaaS) for specific fields are service APIs on this operating system. A large number of products and technologies in the fields of storage, database, middleware, big data, AI, etc. have begun to provide fully managed cloud services. More and more users have become accustomed to using cloud services instead of building storage systems and deploying database software by themselves.

As these BaaS cloud services become more and more perfect, Serverless has gradually become one of the mainstream cloud native technologies because it shields the various operational and maintenance complexities of servers , allowing developers to focus more on business logic design and implementation. Serverless computing includes the following characteristics:
Fully managed computing service, customers only need to write code to build applications, and do not need to pay attention to the development, operation and maintenance, security, high availability and other work of homogeneous and heavy-duty server-based infrastructure;
Versatility, combined with the capabilities of cloud BaaS APIs, can support all important types of applications on the cloud;
Automatic elastic scaling, so that users do not need to plan capacity in advance for resource usage;
Billing by volume allows enterprises to effectively reduce usage costs without paying for idle resources.

Function as a Service is the most representative product form in Serverless. It divides the application logic into multiple functions, and each function is triggered and executed in an event-driven manner. For example, when an event such as uploading/deleting an object is generated in the object storage (OSS), it can automatically and reliably trigger the processing of FaaS functions. And each link is elastic and highly available, and customers can quickly realize real-time parallel processing of large-scale data. Similarly, through the integration of message middleware and function computing, customers can quickly realize real-time processing of large-scale messages.

At present, there are still some difficulties in popularizing the serverless form of function computing, for example:

Functional programming is executed in an event-driven manner, which will have major changes in application architecture, development habits, and R&D delivery processes;
The ecosystem of functional programming is still immature, and application developers and internal R&D processes within enterprises need to be re-adapted;
The fine-grained function operation also raises new technical challenges, such as cold start, which can cause application response delays, and high cost of establishing database connections on demand.

In response to these situations, more other forms of services have emerged in serverless computing, typically integrating innovation with container technology. With good portability, containerized applications can run indiscriminately on development machines, self In computer rooms and public cloud environments ; the container-based tool chain can speed up the delivery of serverless solutions. Cloud vendors such as Alibaba Cloud provide Elastic Container Instance (ECI) and upper-level Serverless Application Engine (SAE), and Google provides CloudRun service, all of which help users focus on building containerized applications without worrying about infrastructure management costs . In addition, Google has also open sourced Knative, a Kubernetes-based serverless application framework .

Compared with the programming mode of function computing, this type of serverless application service supports container images as a carrier, and can be deployed in a serverless environment without modification . and other advantages. The following is a comparison of traditional elastic computing services, container-based serverless application services and function computing:

2: Common scenarios

In the past two years, serverless has shown an accelerated development trend in recent years, and users have achieved significant benefits in terms of reliability, cost, and R&D, operation and maintenance efficiency by using the serverless architecture.

Mini Program/Web/Mobile/API backend service

In scenarios such as applets, Web/ Moible applications, and API services, the business logic is complex and changeable, and the iterative online speed is required to be high. Moreover, the resource utilization rate of such online applications is usually less than 30%, especially for long-tail applications such as applets. Resource utilization is even lower than 10%. Serverless is free of O&M and pay-as-you-go. It is very suitable for building applet/Web/Mobile/API back-end systems. By reserving computing resources + real-time automatic scaling, developers can quickly build stable latency and high-frequency access systems. Online application. Inside Alibaba, the use of Serverless to build back-end services is the most common scenario, including Serverless For Frontends in the front-end full- stack field, machine learning algorithm services, and small program platform implementation.

Large-scale batch tasks

When building a typical task batch system, such as a large-scale audio and video file transcoding service, it needs to include a series of functions such as computing resource management, task priority scheduling, task scheduling, reliable task execution, and task data visualization. If building from the machine or container layer, users usually use message queues to persist task information and allocate computing resources, use container orchestration systems such as Kubernetes to achieve resource scaling and fault tolerance, and build or integrate monitoring and alarm systems by themselves. With the serverless computing platform, users only need to focus on the processing of task processing logic, and the extreme flexibility of serverless computing can well meet the computing power requirements under sudden tasks.

By integrating object storage and serverless computing platform, it can respond to operations such as object creation and deletion in real time, and realize large-scale data processing centered on object storage. Users can either incrementally process new data on object storage, or create a large number of function instances to process existing data in parallel.

Online application and offline data processing based on event-driven architecture

A typical serverless computing service can be widely integrated with various types of services in the cloud through an event-driven approach. Users can easily build loosely coupled, distributed-based services without managing infrastructure such as servers and writing "glue" code that integrates multiple services. Application of event-driven architecture.

Through the integration with the event bus, whether it is a BaaS cloud service of one party, a SaaS service of a third party, or a user-built system, all events can be quickly and easily processed by function computing. For example, by integrating with API Gateway, external requests can be converted into events that trigger backend function processing. Through event integration with message middleware, users can quickly process massive messages.

DevOps Automation

With timing triggers, users can quickly implement timing tasks in the form of functions without managing the underlying server that executes the tasks. By integrating the timing trigger with the time trigger of the monitoring system, users can receive operation and maintenance events of IaaS layer services such as machine restart, downtime , and capacity expansion in a timely manner, and automatically trigger function execution and processing.

3: Technical focus

Flexible scheduling of computing resources

In order to achieve accurate and real-time instance scaling and placement, the characteristics of the application load must be used as the basis for resource scheduling, and the "white box" scheduling strategy must be used, and the serverless platform is responsible for managing the computing resources required by the application. The platform must be able to identify application characteristics, expand computing resources in a timely manner when the load increases rapidly, and ensure stable application performance; when the load drops, timely recycle computing resources, speed up the flow of resources between different tenant functions, and improve data center utilization. Therefore, more real-time, more active, and more intelligent elastic scaling capabilities are the key to a good user experience for function computing services. Through flexible scheduling of computing resources, it helps users complete the closed loop of indicator collection, online decision-making, offline analysis, and decision-making optimization.

When creating a new instance, the system needs to determine how to place the application instance on the underlying computing node. The placement algorithm should meet multiple goals:
Fault tolerance: When there are multiple instances, distribute them on different computing nodes and availability zones to improve application availability.
Resource utilization: On the premise of not losing performance, schedule computing-intensive and I/O-intensive applications to the same computing node, and make full use of the computing, storage, and network resources of the node as much as possible. Dynamically migrate fragmented instances on different nodes and perform "defragmentation" to improve resource utilization.
Performance: For example, reuse nodes that have started the same application instance or function, and use cached data to speed up application startup time.
Data-driven: In addition to online scheduling, the system uses data from days, weeks, or larger time ranges for offline analysis. The purpose of offline analysis is to use the full amount of data to verify the effect of the online scheduling algorithm, provide a basis for parameter tuning, speed up the flow of resources through a data-driven approach, and improve the overall resource utilization of the cluster.

Load Balancing and Flow Control

Resource scheduling service is the key link of serverless system. In order to support nearly one million resource scheduling requests per second, the system needs to shard the load of the resource scheduling service and scale it horizontally to multiple machines to avoid single point bottlenecks. The shard manager monitors the sharding and server load of the entire cluster, and performs shard migration, splitting, and merging operations to achieve horizontal expansion and load balancing of cluster processing capabilities.

In a multi-tenant environment, traffic isolation control is the key to ensuring service quality. Since the user pays for the resources actually used, the computing resources should be shared by different applications of different users to reduce the system cost. This requires the system to have excellent isolation capabilities to prevent applications from interfering with each other.

safety

The positioning of the serverless computing platform is a general-purpose computing service that can execute arbitrary user code, so security is an insurmountable bottom line. The system should comprehensively guarantee the security of applications from various dimensions such as rights management, network security, data security, and runtime security . New virtualization technologies such as lightweight security containers have achieved smaller resource isolation granularity, faster startup speed, and smaller system overhead, making the resource usage of the data center more fine-grained and dynamic, so as to more fully Use fragmented resources.