All Products
Search
Document Center

Function Compute:Configure instance concurrency

Last Updated:Mar 13, 2024

Instance concurrency specifies the maximum number of concurrent requests that each function instance can process at a time. You can use the concurrency feature in Function Compute to manage resource usage in an efficient manner during traffic peaks and reduce impact of cold starts. This helps improve performance and reduce costs.

Background

Function Compute is billed based on the execution duration of instances. For example, three requests need to be processed at the same time. The processing of each request lasts 10 seconds. The following items describe the execution duration for different concurrency values:

  • If instance concurrency is set to 1, each instance can process only one request at a time. Function Compute creates three instances to process the 3 requests. The total execution duration is 30 seconds.

  • If instance concurrency is set to 10, each instance can process 10 requests at a time. Function Compute creates only 1 instance to process the 3 requests. The total execution duration is 10 seconds.

Note

By default, instance concurrency of a function is 1, which indicates that a single instance can process only one request at a time. If you set instance concurrency to a value greater than 1, Function Compute creates a new instance only when the number of requests that are concurrently processed by existing instances exceeds the specified value.

The following figure shows the difference in request execution when instance concurrency is set to different values.

image

Common scenarios

If a large amount of time is required for a function to wait for responses from downstream services, we recommend that you use the concurrency feature to allow a single instance to concurrently process multiple requests. In most cases, waiting for responses does not consume resources. Concurrent processing of multiple requests by one instance saves costs and improves responsiveness and throughput of applications.

Benefits

  • Shorter execution duration and lower costs

    For example, for functions that involve a great number of input/output (I/O) operations, you can use a single instance to concurrently process multiple requests. This reduces the number of instances that are used to process requests and reduces the total execution duration of requests.

  • Status shareable among requests

    Multiple requests can share the connection pool of a database in one instance to minimize connections between requests and the database.

  • Lower frequency of cold starts

    One instance can process multiple requests, which reduces the number of new instances and reduces frequency of cold starts.

  • Fewer IP addresses used in virtual private clouds (VPCs)

    For a fixed number of requests to be processed, the number of required instances is reduced if each instance can process multiple requests. This reduces the number of IP addresses used in VPCs.

    Important

    Make sure that the vSwitch associated with your VPC has at least two available IP addresses. Otherwise, the service may be unavailable, leading to request errors.

Limits

Item

Description

Supported runtimes

  • Custom runtimes

  • Custom Container

Number of requests that can be concurrently processed by a single instance

1 to 200

Logs in X-Fc-Log-Result of the response header

Not supported if instance concurrency is set to a value greater than 1

Configure instance concurrency for a function

You can specify instance concurrency of a function when you create or update the function. For more information, see Manage functions.

Note

If you select Web Function or Container Images to create a function, you can configure instances concurrency when you create the function.

dg-instance-concurrency

Provisioned instances can also concurrently process multiple requests. For more information, see Configure provisioned instances.

Impacts

This section describes differences between the value of 1 and a value greater than 1 for instance concurrency.

Billing

The execution duration differs between a single instance with single concurrency (instance concurrency = 1) and a single instance with multiple concurrency (instance concurrency > 1). As a result, costs are different. For more information, see Billing overview.

Instance concurrency = 1

An instance can process only one request at a time. The billing duration starts when the first request starts to be processed and ends when the last request is processed.

image

Instance concurrency > 1

For a single instance that concurrently processes multiple requests, you are charged based on the execution duration of requests. The billing duration starts when the first request starts to be processed and ends when the last request is processed.

image

Concurrency throttling

By default, the maximum number of on-demand instances of Function Compute in a region is 300. The maximum number of requests that can be processed at a time in a region is 300 × Instance concurrency. For example, if you set instance concurrency to 10, a maximum of 3,000 concurrent requests can be processed at a time in a region. If the number of concurrent requests exceeds the maximum number of requests that Function Compute can handle, the throttling control error ResourceExhausted is reported.

Note

To increase the upper limit of on-demand instances in a region, contact us.

Logs

  • If instance concurrency is set to 1, Function Compute returns function logs in the X-Fc-Log-Result field of the response header if you specify X-Fc-Log-Type: Tail in the HTTP header when you invoke a function. If instance concurrency is set to a value greater than 1, the response header does not contain function logs because the logs of specific requests cannot be obtained among concurrent requests.

  • For a Node.js runtime, console.info() was used to return logs, which include request IDs. If instance concurrency is set to a value greater than 1, console.info() cannot print request IDs as expected. All the request IDs are printed as req 2. The following example shows a sample log:

    2019-11-06T14:23:37.587Z req1 [info] logger begin
    2019-11-06T14:23:37.587Z req1 [info] ctxlogger begin
    2019-11-06T14:23:37.587Z req2 [info] logger begin
    2019-11-06T14:23:37.587Z req2 [info] ctxlogger begin
    2019-11-06T14:23:40.587Z req1 [info] ctxlogger end
    2019-11-06T14:23:40.587Z req2 [info] ctxlogger end
    2019-11-06T14:23:37.587Z req2 [info] logger end
    2019-11-06T14:23:37.587Z req2 [info] logger end                    

    In this case, we recommend that you use context.logger.info() to print logs. This method allows request IDs to be printed as expected. The following sample code shows an example:

    exports.handler = (event, context, callback) => {
        console.info('logger begin');
        context.logger.info('ctxlogger begin');
    
        setTimeout(function() {
            context.logger.info('ctxlogger end');
            console.info('logger end');
            callback(null, 'hello world');
        }, 3000);
    };                   

Error handling

When an instance concurrently processes multiple requests, unexpected process quits caused by failed requests affect other concurrent requests. Therefore, you must compile logic to capture request-level exceptions in the function code and prevent impact on other requests. The following example shows sample Node.js code:

exports.handler = (event, context, callback) => {
    try {
        JSON.parse(event);
    } catch (ex) {
        callback(ex);
    }

    callback(null, 'hello world');
};                    

Shared variables

When an instance concurrently processes multiple requests, errors may occur if multiple requests attempt to modify a variable at the same time. You must use mutual exclusion in your code to protect non-thread-safe variables upon modifications. The following example shows sample Java code:

public class App implements StreamRequestHandler
{
    private static int counter = 0;

    @Override
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
        synchronized (this) {
            counter = counter + 1;
        }
        outputStream.write(new String("hello world").getBytes());
    }
}                    

Monitoring metrics

After you set instance concurrency to a value greater than 1, you can see that the number of used instances is reduced in the instance monitoring chart.实例数据监控图

FAQ