Configure auto scaling to scale SAE instances - Serverless App Engine

For microservice applications in Serverless App Engine (SAE), you can add auto scaling policies to automatically scale application instances in response to traffic changes, such as sudden traffic spikes.

Overview

SAE supports the following scaling methods:

Manual scaling: Use manual scaling for urgent scaling needs, such as unexpected traffic spikes.
Auto scaling: Use auto scaling for non-urgent scaling needs, such as periodic traffic changes, by configuring an auto scaling policy. SAE supports scheduled, metric-based, and hybrid auto scaling policies.

Use cases

SAE supports three types of auto scaling policies:

Scheduled auto scaling policy: Suitable for applications with predictable, periodic resource usage patterns. This is common in industries such as finance, healthcare, government, and education.
Metric-based auto scaling policy: Suitable for applications that experience burst traffic or periodic traffic fluctuations. This is common in industries such as internet services, gaming, and social media.
Hybrid auto scaling policy: Suitable for applications that handle both periodic resource demand fluctuations and burst traffic. This is common in industries such as internet services, education, and food services.

Considerations

You can create a maximum of five scheduled auto scaling policies, one metric-based auto scaling policy, or one hybrid auto scaling policy. You cannot use these three types of policies at the same time.
When an auto scaling policy is enabled, do not perform manual application lifecycle operations, such as scaling, deploying, changing specifications, restarting, or stopping the application. To perform these operations, disable the auto scaling policy first.
If an application is currently scaling out, scaling in, being deployed (single-batch, phased, or canary release), changing specifications, restarting, or stopping, you must wait for the process to complete before adding or enabling an auto scaling policy.
The maximum number of instances for a single application is 50. To increase this quota, join the DingTalk group (ID: 32874633) and apply to be added to the whitelist.

Prerequisites

An application is deployed.

Procedure

In the SAE application list, select the target region and namespace at the top, and then click the target Application ID.
On the Basic Information page of the target application, click the Auto Scaling tab. In the Auto Scaling section, click Create Auto Scaling Policy.

Configure an auto scaling policy

Scheduled policy

In the Create Auto Scaling Policy panel, configure the following parameters, and then click Next: Preview Scheduled Auto Scaling Policy.

Parameter	Description	Example
Policy Type	Select Scheduled auto scaling policy.	Scheduled auto scaling policy
Policy Name	Enter a custom name for the policy.	demo
Timeframe	Specify whether the policy is short-term or long-term: Short-term: Specify a start time and an end time. Long-term: The policy remains active indefinitely.	Long-term
Recurrence	Specifies how often the scaling action repeats. Options are Daily, Weekly, or Monthly.	Daily
Daily Trigger Schedule	Trigger Time: The time of day when the scaling action occurs. Target Instance Count: The number of instances to maintain after the scaling action is triggered.	Trigger time: 08:00 and 20:00 Target instance count: 10 and 3

On the Preview Scheduled Auto Scaling Policy page, review the projected instance count for specific time periods, and then click Confirm.
After adding the scheduled auto scaling policy, enable it.

Metric-based policy

In the Create Auto Scaling Policy panel, configure the following parameters, and then click Confirm.

Parameter	Description	Example
Policy Type	Select Metric-based auto scaling policy.	Metric-based auto scaling policy
Policy Name	Enter a custom name for the policy.	demo
Trigger Conditions	Select a metric type: CPU utilization Memory usage TCP active connections Total TCP connections Application QPS Application response time (RT) Internet-facing CLB QPS Internet-facing CLB response time Internal-facing CLB QPS Internal-facing CLB response time Note For information about how metrics are aggregated, see the descriptions in the console. You can set multiple metric types.	CPU utilization
Trigger Conditions	A scaling action is triggered when the specified metric reaches this value.	70%
Instance Count	Set the Minimum application instances, Maximum application instances, and minimum available instances. Note Minimum available instances specifies the minimum number of instances that remain available during a rolling update. You can set this value by number or by ratio.	Minimum application instances: 6 Maximum application instances: 50 Minimum available instances: Set by number to 3.
Advanced Settings	(Optional) Configure the following parameters as needed: Scale-out step size: The maximum number of instances that can be added in a single scaling action. Scale-in step size: The maximum number of instances that can be removed in a single scaling action. Scale-out stabilization window: To prevent rapid fluctuations when scaling out, the algorithm uses the minimum required instance count calculated over this time window to make a scaling decision. Scale-in stabilization window: To prevent rapid fluctuations when scaling in, the algorithm uses the maximum required instance count calculated over this time window to make a scaling decision. Disable scale-in: If enabled, the application will never scale in. This can prevent service disruptions caused by scaling in during traffic peaks. This option is disabled by default.	See Best practices for advanced auto scaling parameters.

After adding the metric-based auto scaling policy, enable it.

Hybrid policy

Note

A hybrid auto scaling policy combines a scheduled auto scaling policy and a metric-based auto scaling policy.

For Policy Type, select Hybrid auto scaling policy and enter a custom Policy Name.
Configure the Metric Settings. For details, refer to the parameters for a metric-based auto scaling policy.

(Optional) Click Advanced Settings and configure the following parameters as needed.

Parameter	Description	Example
Scale-out step size	The maximum number of instances that can be added in a single scaling action.	3
Scale-in step size	The maximum number of instances that can be removed in a single scaling action.	2
Scale-out stabilization window	To prevent rapid fluctuations when scaling out, the algorithm uses the minimum required instance count calculated over this time window to make a scaling decision.	300 seconds
Scale-in stabilization window	To prevent rapid fluctuations when scaling in, the algorithm uses the maximum required instance count calculated over this time window to make a scaling decision.	300 seconds
Disable scale-in	If enabled, the application will never scale in. This can prevent service disruptions caused by scaling in during traffic peaks. This option is disabled by default.	Enabled

Configure Special Time Period Settings. For details, refer to the parameters for a scheduled auto scaling policy.

In the Preview Scheduled Auto Scaling Policy wizard, review the projected instance count for specific time periods, and then click Confirm.
After adding the hybrid auto scaling policy, enable it.

Verify the policy

Verify that the auto scaling policy is effective using one of the following methods:

Verify instance count

Go to the Basic Information page of the target application and click the Auto Scaling tab. Check if the number of instances matches the number specified in the auto scaling policy. If they match, the policy is effective.

The Instance Count Trend with Scaling Metrics chart at the top of the page displays the change curves for the current number of application instances and metrics such as CPU utilization. The blue area indicates the scale-out range, and the yellow area indicates the scale-in range. The Hybrid Auto Scaling Policy List table below shows the configuration details for each policy, such as the Metric Target Value, Maximum application instances, and Minimum application instances. You can use this information to compare the current instance count with the policy settings.

Perform a stress test

Add a public endpoint for the application.
1. Log on to the SAE console. On the Basic Information page, in the Application Access Settings section, click Add Internet-facing CLB Access.
2. In the Add Internet-facing CLB Access panel, select Create CLB (Pay-by-Specification), set the HTTP Port and Container Port, and then click OK.
  
  In this example, HTTP Port is set to 80 and Container Port is set to 3210.
3. After the endpoint is added, copy the public endpoint address.
  
  To the right of Public Endpoint, you can see the generated address (for example, 101.xxx:80) and the protocol type HTTP.
Perform a stress test on the application.
1. Log on to the Performance Testing Service (PTS) console. On the overview page, enter the endpoint address in the format http://<Public Endpoint Address>, and then click Test.
2. In the Test Settings panel, enter the number for Requests Per Second (RPS). Then, select the The test is permitted and complies with the applicable laws and regulations checkbox and click Start.
  
  In this example, the Total Duration is set to 10 minutes and Requests Per Second (RPS) is set to 1000.
Return to the SAE console. On the Basic Information page of the target application, go to the Instance List tab. Check that the number of application instances has increased. A successful scale-out confirms that the policy is effective.

Note
After the stress test is complete, the number of instances automatically scales in. This process might take some time.

View scaling events

Go to the Application Events page of the target application. Filter by Source Type and set the source type to HorizontalPodAutoscaler to view scaling events.

After filtering, the event list displays scaling-related events. An event with a level of Normal and a reason of SuccessfulRescale indicates a successful scaling action. You can expand the event details to view the scaling result (for example, New size: 5) and the trigger reason (for example, the external metric slb_incall_qps exceeded the target threshold).

More operations

On the Basic Information page of the target application, click the Auto Scaling tab.
In the Auto Scaling section of the Auto Scaling tab, find the target policy. In the Actions column, you can Enable, Disable, Edit, or Delete the policy, or view the Events triggered by the policy.