Function Compute:Configure provisioned instances and auto scaling rules

On-demand mode

Introduction

In the on-demand mode, instances are allocated and released by Function Compute. Function Compute automatically scales instances based on the number of function invocations. Function Compute creates instances when the number of function invocations increases, and destroys instances when the number of requests decreases. During the entire process, instance creation is automatically triggered by requests. On-demand instances are destroyed when no requests are sent for processing for a period of time (usually 3 to 5 minutes). When you invoke an on-demand instance for the first time, you must wait for the cold start of the instance to complete.

By default, a maximum of 300 on-demand instances can be created for an Alibaba Cloud account in each region. If you want to increase the upper limit, join DingTalk group 11721331 to contact the technical support.

Billing method

You are charged only when your function is invoked. If no requests are sent to your function, no instances are allocated and you are not charged for any fees. For more information about the pricing and billing of Function Compute, see Billing overview.

Provisioned mode

Introduction

On-demand instances are automatically created when requests are sent. When you invoke on-demand instances for the first time, you must wait for the cold start of the instance to complete. If you want to eliminate the impact of cold starts, you can use provisioned instances.

In provisioned mode, you can manage the allocation and release of function instances. The provisioned instances are retained unless you release them. Provisioned instances takes precedence over on-demand instances. If the number of provisioned instances is not enough to process all requests, Function Compute allocates on-demand instances to process the remaining requests.

Billing method

You are charged for provisioned instances from when you create them to when you release them, regardless of whether they have processed any requests. For more information about the pricing and billing of Function Compute, see Billing overview.

Idle billing

By default, the idle billing feature is disabled. In this case, the provisioned instances in Function Compute are always allocated with CPU resources even when no requests are being processed. This ensures that the instances can run background tasks when no requests are made. After the idle billing feature is enabled, Function Compute freezes the CPU resources of the provisioned instances when the provisioned instances are not processing any request. This way, the instances are in the idle state and you are charged based on the idle billing unit prices. The unit prices for idle instance resources are 10% of the unit prices of active instance resources. For more information, see Billing overview.

You can enable the idle billing feature based on your business requirements.

• Costs

  If you want to use provisioned instances to eliminate cold stars and save costs, we recommend that you enable the idle billing feature. The idle billing feature allows you to pay fewer fees for provisioned instances when they are in the idle state, and requests can be responded without cold starts.

• Background tasks
  If your function needs to run background tasks, we recommend that you disable the idle billing feature. Example scenarios:

  • Some application frameworks rely on the built-in scheduler or background features. Some dependent middleware needs to regularly report heartbeats.
  • Sone asynchronous operations are performed by using Goroutine lightweight threads that use Go, the async functions that use Node.js, or the asynchronous threads that use the Java language.

Instance scaling limits

Scaling limits for on-demand instances

When the total number of instances or the scaling speed of the instances exceeds the limit, Function Compute returns a throttling error, for which the HTTP status code is 429. The following figure shows how Function Compute performs throttling in a scenario where the number of invocations increases rapidly.

• ①: Before the upper limit for burst instances is reached, Function Compute creates instances immediately. During this process, a cold start occurs but no throttling error is reported.
• ②: When the limit for burst instances is reached, the increase of instances is restricted by the growth rate. Throttling errors are reported for some requests.
• ③: When the upper limit of instances is reached, some requests are throttled.

By default, all functions within an Alibaba Cloud account in the same region share the preceding scaling limits. To configure the limit on the number of instances for a specific function, see Overview of configuring the maximum number of on-demand instances. After the configuration, Function Compute returns a throttling error when the total number of running instances for the function exceeds the configured limit.

Scaling limits for provisioned instances

Limits on scaling speeds of instances in different regions

Configure provisioned instances

1. Log on to the Function Compute console. In the left-side navigation pane, click Services & Functions.
2. In the top navigation bar, select a region. On the Services page, click the desired service.
3. On the Functions page, click the function that you want to modify.
4. On the Function Details page, click the Auto Scaling tab and click Create Rule.
5. On the page that appears, configure the following parameters and click Create.

   Parameter	Description
   Basic Configurations
   Version or Alias	Select the version or alias for which you want to create provisioned instances. Note You can create provisioned instances only for the LATEST version.
   Minimum Number of Instances	Enter the number of provisioned instances that you want to create. The minimum number of instances equals the number of provisioned instances to be created. Note You can set the minimum number of function instances to reduce cold starts and the time needed to respond to function invocation requests. This helps improve the service performance for online applications that are sensitive to response latency.
   Idle Billing	Enable or disable the idle billing feature. By default, the idle billing feature is disabled. Take note of the following items: When this feature is enabled, provisioned instances are allocated with CPU resources only when they are processing requests. The CPU resources of the instances are frozen when the instances are not processing any request. When this feature is disabled, provisioned instances are allocated with CPU resources whether they are processing requests or not.
   Maximum Number of Instances	Enter the maximum number of instances. The maximum number of instances equals the number of provisioned instances plus the maximum number of on-demand instances that can be allocated. Note You can set the maximum number of function instances to prevent a single function from using a large number of instances because of excessive invocations protect backend resources, and prevent unexpected charges. If you do not configure this parameter, the maximum number of instances is determined by the resource limits for the current region and Alibaba Cloud account.
   (Optional) Scheduled Setting Modification: You can create scheduled scaling rules to flexibly configure provisioned instances. You can configure the number of provisioned instances to be automatically adjusted to a specified value at a scheduled time. This way, the number of provisioned instances can meet the concurrency requirement of your business.
   Policy Name	Enter a custom policy name.
   Minimum Number of Instances	Enter the minimum number of provisioned instances.
   Schedule Expression (UTC)	Enter the expression of the schedule. Example: cron(0 0 20 * * *). For more information, see Parameters.
   Effective Time (UTC)	Set the time when the configurations of scheduled scaling start to take effect and expire.
   (Optional) Metric-based Setting Modification: Provisioned instances are scaled in or out every minute based on the metrics of instances or concurrency utilization of provisioned instances.
   Policy Name	Enter a custom policy name.
   Minimum Range of Instances	Specify the value range for the minimum number of provisioned instances.
   Utilization Type	This parameter is displayed only when GPU-accelerated instances are configured. Select the types of metrics based on which the auto scaling policy is configured. For more information about the auto scaling policies of GPU-accelerated instances, see Create an auto scaling policy for provisioned GPU-accelerated instances.
   Usage Threshold	Configure the scaling range. Scale-in is performed if the values of metrics or the concurrency utilization of provisioned instances is lower than the specified values. Scale-out is performed if the values of metrics or the concurrency utilization of provisioned instances is higher than the specified values.
   Effective Time (UTC)	Specify the time when the configurations of metric-based auto scaling start to take effect and expire.

   After the auto scaling rule is created, you can go to the Auto Scaling page of the service and view details of the rule.

You can modify or delete the number of provisioned instances as prompted.

Create an auto scaling rule for a provisioned instance

In provisioned mode, Function Compute creates a specified number of instances, but the instances may not be fully used. You can use Scheduled Setting Modification and Metric-based Setting Modification to make better use of provisioned instances.

Scheduled Setting Modification

• Definition: Scheduled scaling helps you flexibly configure provisioned instances. You can configure the number of provisioned instances to be automatically adjusted to a specified value at a specified time so that the number of instances can meet the concurrency of your business.
• Applicable scenarios: Functions work based on periodic rules or predictable traffic peaks. If the provisioned instances are insufficient to process all the function invocation requests, the remaining requests are processed by on-demand instances. For more information, see Instance types and instance modes.
• Sample configuration: The following figure shows two scheduled actions that are configured. The first scheduled action scales out the provisioned instances before the traffic peak, and the second scheduled action scales in the provisioned instances after the traffic peak.

{
  "ServiceName": "service_1",
  "FunctionName": "function_1",
  "Qualifier": "alias_1",
  "ScheduledActions": [
    {
      "Name": "action_1",
      "StartTime": "2022-11-01T10:00:00Z",
      "EndTime": "2022-11-30T10:00:00Z",
      "TargetValue": 50,
      "ScheduleExpression": "cron(0 0 20 * * *)"
    },
    {
      "Name": "action_2",
      "StartTime": "2022-11-01T10:00:00Z",
      "EndTime": "2022-11-30T10:00:00Z",
      "TargetValue": 10,
      "ScheduleExpression": "cron(0 0 22 * * *)"
    }
  ]
}

Metric-based Setting Modification

• Definition: Metric-based auto-scaling tracks metrics to dynamically scale provisioned instances.
• Scenario: Function Compute collects the concurrency utilization of provisioned instances or resource utilization metrics of instances on a regular basis, and uses this metrics together with the threshold values that you specify for scaling operations, to control the scaling of provisioned instances. This way, the number of provisioned instances can be adjusted to meet your business needs.
• Principle: Provisioned instances are scaled every minute based on the metric value.
  • If the metric value exceeds the threshold that you configure, the system rapidly performs scale-out operations to adjust the number of provisioned instances to the target value.
  • If the metric value is lower than the threshold that you configure, the system slightly performs scale-in operations to adjust the number of provisioned instances to the specified value.
  If the maximum and minimum numbers of provisioned instances are configured, the system scales the provisioned instances between the maximum and minimum numbers. If the number of instances reaches the maximum or minimum number, scaling stops.
• Sample configuration: The following figure shows an example of metric-based scaling for provisioned instances.
  • When the traffic increases and the threshold is triggered, provisioned instances start to be scaled out until the number of provisioned instances reaches the upper limit. Requests that cannot be processed by the provisioned instances are allocated to on-demand instances.
  • When the traffic decreases and the threshold is triggered, provisioned instances start to be scaled in.
  

Only the statistics on provisioned instances are collected to calculate the concurrency utilization of provisioned instances. The statistics on on-demand instances are not included.

The metric is calculated based on the following formula: The number of concurrent requests to which provisioned instances are responding/The maximum number of concurrent requests to which all provisioned instances can respond. The metric value ranges from 0 to 1.

{
  "ServiceName": "service_1",
  "FunctionName": "function_1",
  "Qualifier": "alias_1",
  "TargetTrackingPolicies": [
    {
      "Name": "action_1",
      "StartTime": "2022-11-01T10:00:00Z",
      "EndTime": "2022-11-30T10:00:00Z",
      "MetricType": "ProvisionedConcurrencyUtilization",
      "MetricTarget": 0.6,
      "MinCapacity": 10,
      "MaxCapacity": 100,
    }
  ]
}