This topic describes the background information, benefits, scenarios, and impact of a single instance that concurrently processes multiple requests.

Background information

Function Compute calculates fees based on the execution duration of requests by an instance. Assume that database access latency is 10 seconds and three instances totally process three requests. The total execution duration of the requests by the three instances is 30 seconds. If one instance concurrently processes the three requests, the total execution duration of the requests by the instance is 10 seconds. Function Compute allows you to concurrently process multiple requests by using a single instance. This reduces the fees for resource usage of instances. Function Compute allows you to set the Single Instance Concurrency parameter for a function. This parameter specifies the maximum number of requests that a single instance can concurrently process. The following figure shows the differences between a single instance that processes a single request at a time and a single instance that concurrently processes multiple requests.instanceconcurrency

Assume that three requests need to be concurrently processed. If you set the Single Instance Concurrency parameter to 1, Function Compute must create three instances to process the three requests, and each instance processes one request. If you set the Single Instance Concurrency parameter to 10, one instance can concurrently process 10 requests, and Function Compute needs to create only one instance to process the three requests.

Note By default, the value of the Single Instance Concurrency parameter is 1. A value of 1 indicates that a single instance can process only one request at a time. If you set the Single Instance Concurrency parameter to a value greater than 1, Function Compute makes full use of the concurrency of an instance before Function Compute creates another instance.

Benefits

  • Reduces the execution duration and costs.

    For example, for functions that require I/O operations, you can use a single instance to concurrently process multiple requests. This reduces the number of instances that are used to process requests to reduce the total execution duration of requests by the instances.

  • Provides a shared state for requests.

    Multiple requests can share the connection pool of a database in one instance to minimize the connections between requests and the database.

  • Reduces the frequency of cold starts.

    Fewer instances need to be created because one instance can process multiple requests. This reduces the frequency of cold starts.

  • Reduces the number of IP addresses used in a virtual private cloud (VPC).

    For a fixed number of requests to be processed, the number of occupied instances is reduced when each instance can process multiple requests. This reduces the number of IP addresses used in the VPC.

Scenarios

This feature is not applicable to all functions. The following table describes the scenarios to which this feature is or is not applicable.
Scenario Applicable Reason
Requests are waiting for responses from the downstream service for an extended period of time. Yes Resources are generally not consumed when requests are waiting for responses. If you use a single instance to concurrently process multiple requests, costs can be reduced.
Requests are using a shared state that cannot be concurrently accessed. No If multiple requests are concurrently processed to modify the shared state, such as global variables, errors may occur.
A request consumes a large amount of CPU and memory resources. No Multiple requests compete for resources, which leads to out of memory or longer latency.

Impacts

If you set the Single Instance Concurrency parameter to a value greater than 1, the differences from a value of 1 lie in the following aspects:

  • Billing
    • A single instance that processes a single request at a time
      An instance can process only one request at a time. The billing duration starts when the first request starts to be processed and ends when the last request is processed.instanceconcurrency=1
    • A single instance that concurrently processes multiple requests

      For a single instance that concurrently processes multiple requests, Function Compute calculates fees based on the execution duration of the requests by the instance. This duration starts when the first request starts to be processed and ends when the last request is processed.

      instanceconcurrency

    For more information, see Billing.

  • Concurrent request limit

    By default, Function Compute supports a maximum of 300 pay-as-you-go instances in a region. Maximum number of requests that can be concurrently processed in a region = 300 × Value of the Single Instance Concurrency parameter. For example, if you set the Single Instance Concurrency parameter to 10, a maximum of 3,000 requests can be concurrently processed in a region. If the number of concurrent requests exceeds the maximum number of requests that Function Compute can process, the ResourceExhausted error occurs.

    Note If you want to increase the number of pay-as-you-go instances in a region, you can contact Function Compute engineers.
  • Logs
    • For a single instance that processes a single request at a time, if you specify X-Fc-Log-Type: Tail in the HTTP header when you invoke a function, Function Compute returns the function logs in the X-Fc-Log-Result field that is in the response header. For a single instance that concurrently processes multiple requests, the response header does not include function logs because the logs of a specific request cannot be obtained among concurrent requests.
    • For the Node.js runtime, the console.info() function is used to return the ID of the current request in the log. When an instance concurrently processes multiple requests, the console.info() function cannot display the correct IDs of all the requests. All the request IDs are changed to req 2. The following sample log is displayed:
      2019-11-06T14:23:37.587Z req1 [info] logger begin
      2019-11-06T14:23:37.587Z req1 [info] ctxlogger begin
      2019-11-06T14:23:37.587Z req2 [info] logger begin
      2019-11-06T14:23:37.587Z req2 [info] ctxlogger begin
      2019-11-06T14:23:40.587Z req1 [info] ctxlogger end
      2019-11-06T14:23:40.587Z req2 [info] ctxlogger end
      2019-11-06T14:23:37.587Z req2 [info] logger end
      2019-11-06T14:23:37.587Z req2 [info] logger end                    
      In this case, the context.logger.info() function can be used to display logs. This ensures that the correct ID of a request is returned. The following part shows the sample code:
      exports.handler = (event, context, callback) => {
          console.info('logger begin');
          context.logger.info('ctxlogger begin');
      
          setTimeout(function() {
              context.logger.info('ctxlogger end');
              console.info('logger end');
              callback(null, 'hello world');
          }, 3000);
      };                   
  • Troubleshooting

    When an instance concurrently processes multiple requests, unexpected process quits caused by failed requests affect other concurrent requests. Therefore, you must compile troubleshooting logic to prevent impacts on other requests. The following example shows the sample Node.js code:

    exports.handler = (event, context, callback) => {
        try {
            JSON.parse(event);
        } catch (ex) {
            callback(ex);
        }
    
        callback(null, 'hello world');
    };                    
  • Shared variables

    When an instance concurrently processes multiple requests, errors may occur if multiple requests attempt to modify the same variable at the same time. You must use the mutual exclusion method to prevent variable modifications that are not safe for threads when you define your functions. The following example shows the sample Java code:

    public class App implements StreamRequestHandler
    {
        private static int counter = 0;
    
        @Override
        public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
            synchronized (this) {
                counter = counter + 1;
            }
            outputStream.write(new String("hello world").getBytes());
        }
    }                    
  • Monitoring metrics
    After you set the instance concurrency for your function, you can see that the number of used instances is reduced in the instance monitoring chart.Instance monitoring chart

Limits

Item Description
Supported runtime environments
  • Node.js Runtime
  • Java Runtime
  • Custom Runtime
Valid values of instance concurrency 1–100
Function execution logs provided in the X-Fc-Log-Result field in the response header Not supported when the Single Instance Concurrency parameter is set to a value greater than 1

References

Set the request concurrency in a single instance