Function Compute:Configure instance concurrency

Benefits

• Lower costs. Multiple requests share a single instance, reducing total billed execution duration. This is especially effective for I/O-heavy functions.

• Shared state across requests. Requests processed by the same instance share resources such as database connection pools, reducing the total number of connections.

• Fewer cold starts. Fewer instances means fewer cold start events.

• Fewer VPC IP addresses consumed. For functions running in a virtual private cloud (VPC), fewer instances means fewer IP addresses used.

When to keep concurrency at 1

Keep instance concurrency at 1 in these scenarios:

• CPU-intensive functions where concurrent requests compete for CPU and degrade each other's performance.

• Memory-intensive functions where multiple concurrent requests may exceed instance memory limits and crash the process.

• Functions with non-thread-safe global state that cannot be easily synchronized.

• Functions that require per-request logging through the X-Fc-Log-Result response header (not supported when concurrency > 1).

Choosing a concurrency value

The right value depends on your workload type and instance specifications.

You can estimate an initial concurrency value with the following heuristic:

Estimated concurrency ≈ Expected QPS per instance x Average response time (seconds)

For example, if a single instance handles 20 requests per second and each request takes 0.5 seconds: estimated concurrency ≈ 20 x 0.5 = 10.

Monitor actual utilization and adjust based on observed performance.

Concurrency = 1

Each instance processes one request at a time. The billing duration spans from when the first request starts to when the last request finishes. Because requests run sequentially, the total billed duration equals the sum of all request durations.

Concurrency > 1

An instance processes multiple requests concurrently. The billing duration spans from when the first request starts to when the last request finishes. Because requests overlap, the total billed duration is typically much less than the sum of individual request durations.

Logging

When instance concurrency is set to 1, function logs are returned in the X-Fc-Log-Result response header if the request includes X-Fc-Log-Type: Tail. When concurrency is greater than 1, this header is not available because logs from concurrent requests cannot be isolated.

Node.js-specific issue: When concurrency > 1, console.info() cannot associate log entries with the correct request ID. All entries may show the same request ID. Example of interleaved logs:

2019-11-06T14:23:37.587Z req1 [info] logger begin
2019-11-06T14:23:37.587Z req1 [info] ctxlogger begin
2019-11-06T14:23:37.587Z req2 [info] logger begin
2019-11-06T14:23:37.587Z req2 [info] ctxlogger begin
2019-11-06T14:23:40.587Z req1 [info] ctxlogger end
2019-11-06T14:23:40.587Z req2 [info] ctxlogger end
2019-11-06T14:23:37.587Z req2 [info] logger end
2019-11-06T14:23:37.587Z req2 [info] logger end

Use context.logger.info() instead of console.info() to preserve correct request ID association:

exports.handler = (event, context, callback) => {
    console.info('logger begin');
    context.logger.info('ctxlogger begin');

    setTimeout(function() {
        context.logger.info('ctxlogger end');
        console.info('logger end');
        callback(null, 'hello world');
    }, 3000);
};

Error handling

When an instance processes multiple requests concurrently, an unhandled exception in one request can crash the process and affect all other in-flight requests. Wrap request processing logic in try-catch blocks to isolate failures.

Node.js example:

exports.handler = (event, context, callback) => {
    try {
        JSON.parse(event);
    } catch (ex) {
        callback(ex);
    }

    callback(null, 'hello world');
};

Shared variables

When multiple requests run concurrently on the same instance, concurrent modifications to shared variables can cause race conditions. Use synchronization mechanisms to protect shared state.

Java example:

public class App implements StreamRequestHandler
{
    private static int counter = 0;

    @Override
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
        synchronized (this) {
            counter = counter + 1;
        }
        outputStream.write(new String("hello world").getBytes());
    }
}