In-depth demystification of function computing asynchronous task capabilities

Task scheduling

Task scheduling refers to the related operations that a system places different tasks into appropriate computing resources to perform based on the current load situation. A complete scheduling system often needs to balance the two requirements of isolation between tasks with different characteristics and optimal efficiency. The independent queue model and automatic load balancing strategy are adopted for asynchronous tasks in functional computing, which provides the ability to perform multi rent isolation without affecting processing performance.

Serverless Task Scheduling Model

After a user submits a task, the system will convert the task into a message and place it in the internal queue through asynchronous distribution. The processing flow of a message is shown in the following figure:

The entire system mainly relies on the scheduler's consumption and control of queues in terms of task scheduling, multi rent isolation, and message backlog control. We will divide an account level queue for each user in advance, and asynchronous calls (including task calls) to all functions of the user will share the queue.

Such a model structure ensures that each user's asynchronous execution requests (including task calls) are not affected by other users' invocation situations. However, in some large-scale application scenarios, such as when there are many functions for a user and the call volume of each function is large, it is inevitable that all asynchronous messages share a single queue, resulting in mutual impact between calls. Some long tail calls may consume excessive resources in the queue, leading to starvation in the execution of other functions.

To avoid this situation affecting the execution of important functions, function calculations provide a more granular queue - function level queue. You can set a separate queue for each different function to ensure that the consumption of high priority functions is not affected by the execution of other functions under the same account. The relationship between queues is shown in the following figure:

Suppose a user A has 2 different task functions. One task A requires execution of one message per message due to downstream service constraints; The other task B is a large concurrent task and hopes to be completed as soon as possible. In the default mode, tasks A and B share the same user queue; At this point, the following scenario will occur: Due to the concurrency limitation of task A, the function calculation side will control the outgoing rate of the entire task queue. This resulted in the task of Task B being delayed and unable to leave the team. After Task A finishes executing, Task B gets a chance to leave the queue. At this time, the concurrency increases, and Task B's messages preempt the resource pool for execution. Task A becomes difficult to leave the queue and cannot start execution for a long time. As a result, both A and B are severely disrupted by the other party's business.

After queue adjustment, tasks A and B occupy the queue exclusively. In this case, the consumption speed of tasks A and B is not affected by the other party, and both can meet their own demands.

Currently, Serverless Task provides a large inventory of task backlog. You can obtain the number of tasks that have already been backlog in the task interface and comprehensively analyze whether it is necessary to open the exclusive queue of the function.

Serverless Task Task Queue Load Balancing Model

The above section describes how to avoid the "Noise Neighbour" problem through function level queues. However, in some scenarios, if the concurrency level of a task is too large, even if the task is divided into a single queue, it can lead to a backlog of tasks. To solve this problem, it is necessary to introduce the load balancing strategy of Serverless Task.

The task processing module of function calculation has the concept of Partition. Each user belongs to a partition by default, and the scheduler responsible for that partition will listen to the user's corresponding task queue. When there is a serious backlog, we will allocate multiple partitions to users based on the load situation, and assign them to different schedulers for consumption, to improve the overall consumption speed of the task.

Figure 3

As can be seen, Alibaba Cloud functional computing has the ability to achieve multi tenancy and isolation in task queue management by default, which can be applied to most scenarios. For some scenarios with heavy load, long execution, and high concurrency, function computing also supports horizontal expansion to accelerate consumption. In task isolation, function calculation supports separate isolation of functions with different priorities to avoid the problem of Noisy Neighbour.

Observability

The observability of a task is one of the essential capabilities of a task system. Strong observability will help business parties reduce the additional workload required at various stages of task execution.

1. Development stage: The online debugging ability of tasks and the debugging ability of running results will directly affect the business online progress;

2. Routine business operation stage: various monitoring, traffic statistics, and runtime logs will help users quickly understand the development and changes of the business, as well as quickly locate and handle failures;

Periodic audit: The storage and retention of historical records of tasks will provide users with good traceability and enable subsequent business planning based on historical information.

ServerlessTask Observability Support - Development Test Phase

The main requirement in the development phase of a business is to quickly debug and locate issues. In support of this phase, ServerlessTask provides the ability to log in to instances and real-time logs. After the code is developed and uploaded, the process of testing, debugging, modifying the code, and retesting can all be completed on the console, greatly improving research and development efficiency. If there is a need for performance debugging, third-party Binary debugging (such as FFmpeg debugging in the field of audio and video processing), etc. can be completed using the login instance function. The operation process is as follows:

Select the task you want to log in to the instance and click the instance link.

You will enter the instance monitoring page and click the login instance function in the upper right corner to log in to the corresponding instance.

ServerlessTask observability support - post business launch operation phase

When a business goes online, it is often easy to encounter failures due to insufficient capacity estimates that result in downstream systems being unable to bear the pressure. Therefore, ServerlessTask provides runtime indicators, namely, the number of tasks submitted, completed, and executed over a period of time. Users can quickly understand the current business load based on this indicator chart. When the downstream consumption of a user's task is slow, it may cause a backlog of tasks, which can also be easily reflected in the indicator chart, and then quickly respond accordingly. Currently, the relevant indicators provided by ServerlessTask are as follows:

The task monitoring dashboard provides the following task monitoring data:

In terms of quickly locating problems, function computing supports real-time viewing of function logs and instance metrics. You can enter the task list page to find the task that actually failed to execute, and enter the log page and instance page to locate the problem:

ServerlessTask observability support - phased audit

After an online task has been running for a period of time, it is often necessary to conduct a series of periodic audits, such as the total number of tasks executed in the previous week, the number of failed tasks, and the time of failed execution. Currently, in addition to the console, functional computing provides rich API capabilities for task auditing. It mainly includes the following capabilities:

1. Filter by status and query only the execution of a certain status;

2. Filter based on the trigger time, such as querying tasks initiated within a certain period of time in the past;

3. Query by task name. If your task has a TraceID upstream and downstream of the business, you can specify a meaningful task ID when triggering the task. You can subsequently query the range based on the ID prefix;

The above filtering methods can be combined to achieve more convenient requirements. The filter conditions supported by the console are shown in the following figure:

ServerlessTask observability support - dead letter queues and service compensation

In the field of messaging, there is a very important concept - dead letter queues. When some messages cannot be consumed, they often need to be stored in a place for subsequent human intervention to avoid business losses caused by non processing. Serverless Task also supports such functions. You can set target functions for Serverless Tasks; When a task fails to execute, function calculation supports automatically pushing the context information of the execution failure to message services such as Message Queuing for subsequent processing. If your processing logic supports automation, function calculation also supports pushing the context information of failed tasks back to function calculation, and executing a section of your custom business logic to achieve business compensation.

You can configure success and failure targets on the Asynchronous Call Configuration page.

In summary, the observable capabilities provided by Serverless Task can effectively support the monitoring requirements of the entire mission lifecycle. All console capabilities can be customized using open APIs to meet more requirements. The target function of Serverless Task can not only compensate for task failures, but also serve as a data source for the Event Driven mode to automatically post processed events to downstream services.