How to run stress testing based on access logs - Performance Testing

Overview

Stress testing based on access logs lets you play back real-world gateway traffic with simple configurations. This feature can parameterize test requests, automatically create a test scenario, and resolve issues with expired logon status through simple logon API settings. The configuration process involves four main modules:

Configure log producers: Your access logs may come from an ALB, CLB, MSE gateway, or self-managed web services on an ECS instance. First, you must ship the access logs to Log Service (SLS). For more information, see Configure ALB access logs, Configure CLB access logs, Enable log shipping for an MSE gateway, and Collect text logs by using the NGINX configuration mode.
Configure Log Service: Set up Log Service to receive logs shipped from the producers.
Configure stress testing based on access logs: Analyze the logs stored in Log Service to identify APIs for testing and create a test scenario.
Edit the PTS test scenario: After the test scenario is created, you are redirected to the PTS scenario editor. On this page, you can further configure the scenario, or directly debug it and start the test.

Log specifications

Important

The URL field must have both an index and a value. Its index name must be one of the following: "request_uri", "path", "request_url", "url", "uri", "requestURI", or "requestURL". If the URL path and parameters are recorded separately, the method field must have a value, and its index name must be one of the following: "method", "http_method", or "request_method".
To play back POST requests, you must meet the following requirements:
- The HTTP POST body field must have an index and a value, and the index name must be one of the following: "body", "http_body", "http_request_body", or "request_body". Only bodies in application/x-www-form-urlencoded, text/plain, and application/json formats are supported. Binary data bodies are not supported.
- The Content-Type field must have an index and a value, and the index name must be "content_type" or "contentType". Otherwise, PTS automatically infers the Content-Type.
- If you use an MSE gateway, you can log the body of POST requests to the gateway logs. For more information, see Add complete request and response information to access logs. PTS can directly play back POST requests that are logged in this way.
If you use simple mode, the following additional rules apply. Logs that do not meet these rules are ignored:
- The domain name field must have an index and a value, and the index name must be "host", "http_host", or "authority".
- The HTTP protocol field must have an index and a value. The index name must be "http_protocol", "protocol", "scheme", or "http_scheme", and the value must be http or https. Otherwise, http is used by default.

Prerequisites

You have activated Log Service, created a Project and a standard Logstore, and configured indexes and log collection. For more information, see Manage a Project, Create a standard Logstore, and Create indexes.
You are familiar with the Billing overview.

Step 1: Create a scenario from access logs

Log on to the PTS console. In the left-side navigation pane, choose Performance Test > Create Scenario, and then click Access Log.

Search & Analyze

Select the Log Region, Log Project, and Logstore where your access logs are stored. Set the query parameters and time range, and then click Search & Analyze to query the logs for playback. The log indexes must meet the log specifications. Logs that do not comply are ignored.

@timestamp: 2024-08-29T14:38:39+08:00
body_bytes_sent: 200
bytes_sent: 519
content_type: text/html;charset=UTF-8
host: www.example.com
http_referer: http://www.example.com/index.html
http_user_agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
remote_addr: xxx
request_body: username=user1&password=123456&verifycode=1234
request_length: 963
request_method: POST
request_time: 0.002000
request_uri: /login.php
scheme: http
server_addr: 172.26.150.228
status: 200
upstream_host: unix:/dev/shm/php-cgi.sock
upstream_status: 200
upstream_time: 0.003
uri: /login.php

Step 2: Select a test mode

On the Access Log Test Scenario page, turn on the Simple Playback Mode switch and click Select APIs for Testing to enter simple mode. To enter standard mode, turn off the Simple Playback Mode switch and click Select APIs for Testing.

Important

Simple mode directly replays traffic recorded in logs without parsing request structures. It does not limit the number of different domains and APIs, but you can only control the requests per second (RPS) for the entire scenario. This mode does not handle logon status.

In standard mode, you can configure logon status, analyze the request count and maximum RPS distribution for each API, and precisely control the load for each API. However, it limits the number of different domains and APIs.

You can choose either simple mode or standard mode for traffic playback based on your needs. The following table provides a comparison.

Test mode	Description
Simple mode	This mode supports playback of only GET and POST requests. PTS replays requests as recorded in the logs. For POST requests without a body, an empty body is used for playback. You can choose whether to play back POST requests. A traffic playback scenario categorizes requests based on the HTTP request method (GET/POST), protocol type (HTTP/HTTPS), and the Content-Type of the POST body. PTS uses different abstract APIs to play back different types of requests. For example, HTTP GET and HTTPS GET requests belong to two different abstract APIs. Each API runs in parallel and uses parameters extracted from the logs. Simple mode does not handle logon status. If you need to configure logon status, use standard mode.
Standard mode	This mode supports playback of only GET and POST requests. APIs that request static resources are automatically filtered out. POST requests without a request body are not played back. During traffic playback, each API runs in parallel and uses parameters extracted from the logs. By default, logon status is managed using cookies. The number of cookies used equals the number of rows in the parameter file for the logon API, up to a maximum of 100. By default, request headers are not configured. For advanced requirements, you can edit them on the scenario editor page after the PTS test scenario is generated.

Simple mode

Configure playback parameters

The analysis results in simple mode show the number of valid requests within the selected time period. You can configure the following parameters:
- Play Back POST Requests: Turn on this switch to allow PTS to play back POST requests. However, if the request body is not included in the access log, the replayed POST request will not have a body, which may cause errors. Use this option with caution.
- Repeat Playback: If enabled, the playback process loops back to the beginning of the logs after reaching the end. If disabled, load generation stops when the end of the logs is reached. However, the test does not automatically stop until the specified duration ends. To avoid unnecessary Virtual User-Minute (VUM) consumption, manually stop the test after all requests have been sent.
- Playback Duration (Minutes): The duration of the traffic playback. The default is 10 minutes.
- Requests per Second: Configure the rate for traffic playback. PTS replays your access log traffic at this rate, which is distributed evenly across each abstract API. The simple mode configuration form also includes fields for Selected Time Period (displays the time range for playback), Total Valid Requests (displays the total number of valid requests and POST requests in the selected period), and Requests per Second (sets the number of requests sent per second during playback).
Create Scenario

Click Create Scenario. PTS displays a summary of the playback configuration. Clicking Confirm redirects you to the PTS scenario editor page. You can further edit the generated scenario on this page, or you can skip Step 3: (Optional) Configure the PTS scenario and proceed directly to Step 4: Start the test.

Standard mode

Note

If the logs contain the request domain name and the field name is host, http_host, or authority, PTS automatically populates the host, which you can also modify. Currently, only a single host is supported, and all APIs share this host. To play back traffic for multiple domains, create separate scenarios for each API, then modify them as needed.

Select APIs for Testing

In the Select APIs for Testing dialog box, enter the target host and select a protocol, such as http://. PTS displays the top 10 APIs from the access log for the selected time period, ranked by frequency and RPS. The left list shows each API's path and its percentage of total traffic. The right list shows the maximum RPS for each API. Select the target APIs for the test from the list, for example, POST /login. The selected APIs are displayed in the Selected area.
Configure test parameters

If the target APIs do not require a logon, turn on the No Logon Required switch. If the APIs require logon status information, configure the logon API as follows:

a. Select from APIs: If the logon API is among the queried APIs, you can select it from this drop-down list. PTS automatically uses this as the logon API, applies parameters from the logs, and exports cookies for logon status without manual configuration. For example, select POST /login from the Select from APIs drop-down list and set the Logon API RPS to 1.

b. Set Logon API: If the logon API is not in the list, you can configure it manually. Enter the logon API address, such as http://example.com/login. Configure the body parameters for the logon API, such as username and password. You can use ${} to reference parameters. To specify the parameter file, click Select Parameter Files. For more information, see Data sources.
After confirming the configurations, clicking Create Scenario redirects you to the PTS scenario editor page. On this page, you can further edit the generated scenario, or you can skip Step 3: (Optional) Configure the PTS scenario and proceed directly to Step 4: Start the test.

(Optional) Step 3: Configure the PTS scenario

To set the number of cookies to use, click Data Export. In the Export Data configuration panel, select Cookie as the data field to export, set the Export Scale to 100, and select an array parameter from the Specify as array parameter drop-down list if needed. If you need to use other logon status information, such as a token, click the logon API, set the output parameters, and configure the data export. For more information, see Data export instructions.
Click any API to add a header. For the logon API, the request method is POST and the request URL is http://example.com/login. On the Header tab, set Content-Type to application/x-www-form-urlencoded.
For load settings, see Load models and Specify the number of source IPs.

Step 4: Start the test

You can debug the scenario to verify your configurations and prevent test failures. To do so, click Debug to go to the scenario debugging page. For more information, see Debug a scenario.
Click Save and Start. In the Note dialog box, select Execute Now, select the The test is permitted and complies with the applicable laws and regulations. checkbox, and then click Start.

Analyze stress testing results

After the test completes, PTS automatically collects the stress testing data and generates a stress testing report. The report includes the following sections:

Scenario metrics
Business details
Monitoring details
API sampling logs

For details, see View PTS stress testing reports.