Web Application Firewall:Handling full WAF log capacity

Background

WAF logs record all access requests to your web applications and the corresponding firewall responses. These logs contain detailed information, including attack detections, filtering actions, and blocked requests. Sufficient log storage ensures that all security events are fully recorded, helping administrators perform detailed analysis and audits. If the storage capacity is exhausted, you may experience the following issues:

• Log data loss

  When storage is full, new logs cannot be written, leading to the loss of important security event data.

• Delayed threat detection

  Without new log data, security analysis services that depend on WAF logs may not detect and respond to threats quickly, increasing your security risk.

• Compliance risks

  Failing to record new logs can lead to compliance violations.

Solutions

When your WAF log capacity is full, you can use one of the following four methods to resolve the issue:

• Upgrade log storage capacity

  Upgrading your log storage capacity is the most direct solution. Configuring a larger storage space ensures that WAF has enough capacity to record all security events and network traffic. For instructions, see Upgrade log storage capacity.

• Use fine-grained configurations to reduce storage usage

  Configure logging policies based on service importance. For example, for non-critical services, you can log only attack events and required fields, or a smaller selection of optional fields, to reduce storage usage. For instructions, see Use fine-grained configurations to reduce storage usage.

• Reduce the log retention period

  Optimizing the log retention period is another effective method. Shortening the log retention period can significantly reduce storage usage. For example, reducing the retention period from 30 days to 7 days can save a large amount of storage space without losing recent critical data. For instructions, see Reduce the log retention period.

• Reduce indexed fields

  Disabling automatic index updates and removing unnecessary field indexes can significantly reduce the storage used by indexes. Although indexes help you query and analyze logs more quickly, they consume considerable storage space. By optimizing your index configuration, you can reduce storage usage while maintaining adequate query performance. For instructions, see Reduce indexed fields.

Upgrade log storage capacity

1. Log on to the WAF 3.0 console. In the left-side navigation pane, choose Detection and Response > Log Service. In the upper-right corner, click Upgrade Storage.

2. Select a larger Log Storage Capacity plan and complete the purchase.

Reduce storage with fine-grained configurations

Log delivery settings let you configure granular field and storage type settings for each protected object. If you configure specific settings for a protected object, they take precedence over the default field settings.

1. Log on to the WAF 3.0 console. In the left-side navigation pane, choose Detection and Response > Log Service. In the upper-right corner, click Log Configuration > Delivery Settings.

2. On the Delivery Settings tab, find the target protected object and click Field Settings in the Field of Delivery to Simple Log Service column. Configure the settings as described in the following table.

   Parameter	Description
   Required Fields	Required fields are always included in WAF logs. You cannot edit them. For more information, see Required log fields.
   Optional Fields	You can select which optional fields to include in your WAF logs. WAF records only the optional fields that you enable. For more information, see Optional log fields. Note Enabling more optional fields increases your log storage usage. If you have sufficient storage capacity, we recommend enabling more fields for comprehensive analysis.
   Log Type	The log type configuration allows you to select multiple log types and a sampling ratio. The sampling ratio determines the percentage of log entries that are collected for storage and analysis. You can set the sampling ratio from 1% to 100% after selecting a log type. The available log type options are: Block Logs: Records requests that are blocked by security policies, such as Block, JS Challenge, Slider Challenge, or dynamic token verification, and therefore do not reach the origin server. Detection Logs: Records requests that trigger only monitoring rules. Normal Request Logs: Records normal requests, including those that pass JS validation, slider verification, and dynamic token verification. Note For comprehensive auditing and analysis, we recommend selecting all log types.

   For non-critical services, you can log only attack events and required fields, or a small number of optional fields, to reduce storage usage. After you configure the log delivery fields, click OK. The The operation is successful. message indicates that the new configuration has taken effect for the specified protected object.

Reduce the log retention period

1. Log on to the WAF 3.0 console. In the left-side navigation pane, choose Detection and Response > Log Service. In the upper-right corner, click Storage Duration. You are redirected to the Simple Log Service console.

2. In the Logstore Attributes panel, click Modify in the upper-right corner. Decrease the value for Data Retention Period and then click Save.

   Important

   • When you adjust the log retention period, ensure the new setting complies with relevant laws and regulations to avoid compliance issues.

   • Logs older than the specified retention period are automatically deleted.

   

Reduce indexed fields

1. Log on to the Simple Log Service console. In the project list, click the target project whose name starts with wafng.

2. In the left-side navigation pane, click Log Storage. In the Logstores list, click the target Logstore.

   

3. On the Search & Analysis page of the Logstore, select Query & Analysis Properties > Properties.

4. Turn off automatic index updates. By default, the Auto Update switch is turned on for cloud product Logstores or internal Logstores. When it is on, the Field Search area is read-only. In the Query & Analysis panel, turn off the Auto Update switch.

   Warning

   If you have configured custom reports and alarms in SLS, deleting indexes from the WAF-specific Logstore may affect these features. Proceed with caution.

   

5. Delete indexes. After turning off the Auto Update switch, you can delete unnecessary field indexes in the Field Search area. After deleting the indexes, click OK.