Web Application Firewall:What to do when WAF log storage is full

Background

WAF logs record all access requests to web applications and the corresponding firewall responses. These logs contain detailed information, such as attack detection, filtering, and blocking. Sufficient log storage ensures the complete recording of all security events, helping administrators perform detailed analysis and audits. If the log storage reaches its capacity, the following issues may occur:

• Log loss

  When log storage is full, WAF cannot record new log data, resulting in the loss of important security event information.

• Delayed threat detection

  If you rely on WAF logs for other security analysis services, the inability to record new logs can prevent the timely detection of and response to potential security threats, increasing your security risk.

• Compliance issues

  If WAF cannot record new logs because the storage limit is reached, you may face compliance issues.

How to handle full log storage

When your WAF log storage is full, you can resolve the issue in one of the following four ways:

• Upgrade log storage capacity

  The most direct solution is to increase your log storage capacity. Configuring more storage space ensures that WAF has enough capacity to record all security events and network traffic data. For instructions, see Upgrade log storage capacity.

• Use fine-grained configurations to reduce storage usage

  Configure fine-grained logging policies based on the importance of different services. For example, for non-critical services, you can choose to log only attack events and required fields, or a few optional fields, to reduce storage consumption. For instructions, see Use fine-grained configurations to reduce storage usage.

• Reduce the data retention period

  Optimizing the log retention period is another effective method. You can shorten the log retention period based on your business needs to reduce storage usage. For example, changing the retention period from 30 days to 7 days can save a significant amount of storage space while still retaining recent critical data. For instructions, see Reduce the data retention period.

• Reduce indexed log fields

  You can significantly reduce index space usage by turning off automatic index updates and removing unnecessary field indexes. Although indexes improve log query and analysis speed, they also consume considerable storage space. By optimizing your index configuration, you can reduce storage usage while maintaining efficient query performance. For instructions, see Reduce indexed log fields.

Upgrade log storage capacity

1. Log on to the Web Application Firewall 3.0 console, and in the left navigation bar, select Detection and Response > Log Service. Click Upgrade Storage in the upper-right corner. On the Log Service page, click Upgrade Capacity.

2. Select a larger Log Storage Capacity plan and complete the purchase.

Reduce storage with fine-grained configurations

The LogShipper fields settings allow you to configure fields and the log storage class for each protected object. If you configure specific fields and a log storage class for a protected object, these settings take precedence over the default field settings.

1. Log on to the Web Application Firewall 3.0 console. In the left navigation bar, choose Detection and Response > Log Service. In the upper-right corner, click Log Configuration > Delivery Settings.

2. On the Delivery Settings tab, find the protected object that you want to configure, and click Field Settings in the Field of Delivery to Simple Log Service column. Configure the parameters as described in the following table.

   Parameter	Description
   Required Fields	Required fields are always included in WAF logs. You cannot edit required fields. For details about the fields, see Required log fields.
   Optional Fields	Optional fields can be manually selected for inclusion in WAF logs. WAF only records the optional fields that you enable. For details about the fields, see Optional log fields. Note Enabling more optional fields increases log storage usage. If you have sufficient storage capacity, we recommend enabling more optional fields for comprehensive log analysis.
   Log Type	Storage type settings allow you to select log storage types and sampling ratios. The sampling ratio determines the percentage of generated log entries that are collected for storage and analysis. After selecting a storage type, you can set the sampling ratio between 1% and 100%. The three storage types are described as follows: Block Logs: Records requests blocked by security policies such as Block, JS Challenge, Slider Challenge, or dynamic token verification that do not reach the origin server. Detection Logs: Records requests that trigger monitoring rules only. Normal Request Logs: Records normal requests only. Requests that pass JS verification, slider verification, or dynamic token verification are also recorded in normal request logs. Note For comprehensive auditing and analysis, we recommend selecting all log storage types to record full logs.

   For non-critical services, you can log only attack events and required fields, or a small number of optional fields, to reduce storage usage. After you configure the LogShipper fields, click OK. The message The operation is successful. appears.

Reduce the data retention period

1. Log on to the Web Application Firewall 3.0 console. In the left navigation bar, choose Detection and Response > Log Service. In the upper-right corner, click Storage Duration. You are redirected to the Simple Log Service console.

2. In the Logstore Attributes panel, click Modify in the upper-right corner. Decrease the Data Retention Period and then click Save.

   Important

   • When you adjust the data retention period, make sure to comply with all relevant laws and compliance requirements to avoid issues caused by insufficient data retention.

   • When logs reach the specified retention period, they are automatically deleted.

Reduce indexed log fields

1. Log on to the Simple Log Service console. In the Project list, click the target Project whose name starts with wafng.

2. In the left navigation bar, click Log Storage. In the Logstores list, click the target Logstore.

   You are redirected to the query and analysis page for the Logstore. The top of the page displays a query input field and a time range picker. The bottom of the page shows the query results and a time distribution histogram.

3. On the query and analysis page of the Logstore, choose Index Attributes > Attributes.

4. Turn off automatic index updates. For special cloud product Logstores or internal Logstores, the Auto Update switch is turned on by default, and the Field Search section is read-only. In the Search & Analysis panel, turn off the Auto Update switch.

   Warning

   If you have configured custom reports and alerts in Simple Log Service, deleting indexes from the dedicated WAF Logstore may affect these features. Proceed with caution.

5. After turning off the Auto Update switch, remove unnecessary field indexes from the Field Search section and click OK.