Function Compute:Configure instance concurrency

Background

Function Compute calculates fees based on the execution duration of instances. For example, if the database access latency is 10 seconds and three requests are concurrently processed by three instances, the total execution duration of the requests by the three instances is 30 seconds. If the three requests are concurrently processed by one instance, the total execution duration of the requests by the instance is 10 seconds. To save your costs, Function Compute allows you to use a single instance to concurrently process multiple requests. You can use the Instance Concurrency parameter to specify the number of requests that you want to concurrently process by an instance. The following figure shows the differences between the scenarios in which an instance processes one request at a time and the scenarios in which a single instance processes multiple requests at a time.

Scenarios

If it takes a long time for a function to obtain responses from downstream services, we recommend that you use a single instance to concurrently process multiple requests. In most cases, resources are not consumed when requests are waiting for responses. If you use a single instance to concurrently process multiple requests, costs can be reduced.

Benefits

• Reduces the execution duration and costs.

  For example, for functions that require I/O operations, you can use a single instance to concurrently process multiple requests. This reduces the number of instances that are used to process requests to reduce the total execution duration of requests.

• Shares the status among requests.

  Multiple requests can share the connection pool of a database in one instance to minimize the connections between requests and the database.

• Reduces the frequency of cold starts.

  One instance can process multiple requests, which reduces the number of new instances and lowers the frequency of cold starts.

• Reduces the number of IP addresses used in a virtual private cloud (VPC).

  For a fixed number of requests to be processed, the number of required instances is reduced if each instance can process multiple requests. This reduces the number of IP addresses used in the VPC.

  Important

  Make sure that the vSwitch associated with your VPC has at least two available IP addresses. Otherwise, services may be unavailable, which causes request errors.

Impacts

This section describes the differences between the scenarios in which a single instance processes a single request at a time (Instance Concurrency = 1) and the scenarios in which a single instance can process multiple requests at a time (Instance Concurrency > 1):

Billing

• A single instance processes a single request at a time

  An instance can process only one request at a time. The billing duration starts when an instance processes the first request and ends when the instance processes the last request.

• A single instance processes multiple requests at a time

  For a single instance that concurrently processes multiple requests, you are charged based on the execution duration of the requests processed by the instance. The billing duration starts when an instance processes the first request and ends when the instance processes the last request.

  

For more information, see Billing overview.

Concurrency throttling

By default, Function Compute supports a maximum of 300 on-demand instances in a region. The maximum number of requests that can be concurrently processed in a region is calculated based on the following formula: 300 × Value of Instance Concurrency. For example, if you set Instance Concurrency to 10, a maximum of 3,000 requests can be concurrently processed in a region. If the number of concurrent requests exceeds the maximum number of requests that Function Compute can process, the ResourceExhausted error is returned.

Logs

• If Instance Concurrency is set to 1, Function Compute returns the function logs in the X-Fc-Log-Result field of the response header if you specify X-Fc-Log-Type: Tail in the HTTP header when you invoke a function. For a single instance that concurrently processes multiple requests, the response header does not include function logs because the logs of a specific request cannot be obtained among concurrent requests.

• For a Node.js runtime, the function console.info() was used to return logs, which include request IDs. If Instance Concurrency is set to a value greater than 1, console.info() cannot display request IDs as expected. All the request IDs are displayed as req 2. The following sample log is displayed:

  2019-11-06T14:23:37.587Z req1 [info] logger begin
  2019-11-06T14:23:37.587Z req1 [info] ctxlogger begin
  2019-11-06T14:23:37.587Z req2 [info] logger begin
  2019-11-06T14:23:37.587Z req2 [info] ctxlogger begin
  2019-11-06T14:23:40.587Z req1 [info] ctxlogger end
  2019-11-06T14:23:40.587Z req2 [info] ctxlogger end
  2019-11-06T14:23:37.587Z req2 [info] logger end
  2019-11-06T14:23:37.587Z req2 [info] logger end                    

  In this case, we recommend that you use context.logger.info() to print logs. This method allows request IDs to be displayed as expected. The following sample code shows an example:

  exports.handler = (event, context, callback) => {
      console.info('logger begin');
      context.logger.info('ctxlogger begin');
  
      setTimeout(function() {
          context.logger.info('ctxlogger end');
          console.info('logger end');
          callback(null, 'hello world');
      }, 3000);
  };                   

Error handling

When an instance concurrently processes multiple requests, unexpected process quits caused by failed requests affect other concurrent requests. Therefore, you must compile logic to capture request-level exceptions in the function code and prevent impacts on other requests. The following example shows the sample Node.js code:

exports.handler = (event, context, callback) => {
    try {
        JSON.parse(event);
    } catch (ex) {
        callback(ex);
    }

    callback(null, 'hello world');
};                    

Shared variables

When an instance concurrently processes multiple requests, errors may occur if multiple requests attempt to modify the same variable at the same time. You must use mutual exclusion in your code to prevent variable modifications that are not safe for threads. The following example shows the sample Java code:

public class App implements StreamRequestHandler
{
    private static int counter = 0;

    @Override
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
        synchronized (this) {
            counter = counter + 1;
        }
        outputStream.write(new String("hello world").getBytes());
    }
}                    

Monitoring metrics

Limits

Configure the instance concurrency for a function

You can configure the Instance Concurrency parameter when you create or update a function. For more information, see Manage functions.

If you use provisioned instances, the function in provisioned mode can concurrently process multiple requests. For more information, see Configure provisioned instances and auto scaling rules.

Reference

For more information about how to use the SDK for Node.js to set the instance concurrency, see Specify the instance concurrency.