This topic answers frequently asked questions (FAQs) about Cloud Firewall logs, including storage, export, and troubleshooting.
Can I export Cloud Firewall traffic logs to a third-party system?
Why do I see traffic logs from periodic ICMP probes from Alibaba Cloud?
Is log analysis data retained after I release my Cloud Firewall instance?
How do I use logs to find the total number of blocked attacks?
Why are some long-lived connection logs missing after a Cloud Firewall engine upgrade?
Why is the total number of Cloud Firewall logs different from Anti-DDoS Proxy or WAF logs?
Reducing log storage consumption
You can reduce your storage consumption by shortening the log retention period, delivering fewer log types, periodically shipping logs to an OSS bucket, or clearing your log storage.
Reduce the log retention period
After you enable log analysis, the default log retention period is 180 days. If your business does not require long-term retention of historical logs, you can manually shorten this period. For more information, see Modify the log retention period.
Deliver fewer log types
Cloud Firewall enables the delivery of all log types by default. If you only need to monitor specific log types, we recommend enabling delivery only for those types. For more information, see Configure log collection types.
Periodically ship logs to an OSS bucket
If you generate a large volume of logs and need to retain them, we recommend shipping the logs to an OSS bucket. For more information, see Create an OSS shipping task (new version).
Clear log storage
If you generate a large volume of logs during testing and do not need to retain them, we recommend clearing the stored logs. For more information, see Manage log storage capacity.
Exporting traffic logs to third-party systems
Yes. You can export logs by using the log analysis feature of Cloud Firewall and then ingest the exported log files into third-party systems, such as a security operations center (SOC).
You can choose an export method based on your needs.
For small data volumes
You can use the download feature in log analysis to download logs to your local computer and then upload them to your third-party system. For more information, see Export logs.
For large data volumes
You can use the Simple Log Service (SLS) console and a consumer group to programmatically export log data to your third-party system. For more information, see Use a consumer group to consume logs.
Viewing remaining log storage capacity
You cannot view log storage capacity if the log analysis feature is disabled. If you have enabled the feature, you can view the storage usage and remaining capacity in the Cloud Firewall console. For more information, see Manage log storage capacity.
Traffic logs from Alibaba Cloud ICMP probes
To ensure service quality, Cloud Firewall periodically sends Internet Control Message Protocol (ICMP) packets for network probes. These probes are not scanning attacks and do not affect your services.
You can log on to the Cloud Firewall console and find the Source address for SLA monitoring in the address book to view the probe addresses for Cloud Firewall service level agreement (SLA) monitoring. For more information, see Address book.
Traffic logs with an "Unknown" application
Cloud Firewall labels an application as "Unknown" when it cannot identify the application that generated the traffic. This can occur for several reasons:
The traffic consists of fewer than three packets and no session was established. This often indicates scanning traffic.
The traffic was blocked by a Layer 4 access control policy. In this case, no session is established.
The traffic was interrupted by a TCP reset triggered by the intrusion prevention feature or for other reasons, preventing it from matching a known signature.
The traffic is encrypted, or it belongs to a non-standard application, an internal application, or an application not supported by Deep Packet Inspection (DPI).
To avoid service disruptions, Cloud Firewall allows traffic from unidentified applications or domains by default. If you want to block this traffic, you can enable strict mode for the relevant firewall. For more information, see Introduction to the access control engine mode or Configure the ACL engine mode.
Log data retention after instance release
Data retention depends on the billing method: data is retained for pay-as-you-go instances but not for subscription instances.
After you release a Cloud Firewall instance, its configuration data, such as access control policies, attack protection policies, and traffic analysis policies, is retained for 15 days. If you use a subscription instance and need to retain log analysis data, export the logs to your local computer or ship them to a third-party system before you release the instance. For more information, see Export logs.
Exporting log audit records
You cannot export log audit records directly. However, you can go to the Log Analysis page and use a query statement to search for and export the raw logs.
For example, to export logs for inbound traffic of an internet firewall over the last 24 hours where the application is HTTPS, follow these steps:
Log on to the Cloud Firewall console.
In the left-side navigation pane, choose .
On the Log Analysis tab, enter a query statement in the search box and set the time range to 1 Day. For more information, see Query and analyze logs.
The following is a sample query statement:
log_type:internet_log and direction:"in" and app_name:"HTTPS"Export the query results. For more information, see Export logs.
Finding the number of blocked attacks
You can enter the query statement rule_result:drop and specify a time range. The Log Quantity in the query results indicates the total number of attacks intercepted by Cloud Firewall. For more information, see Query and analyze logs.
Due to factors such as query time, the total number of protected attacks retrieved from log queries may be inconsistent with the Total Blocked value on the Overview page. Please refer to the Total Blocked value on the Overview page as the authoritative source.
Log count differences across services
Cloud Firewall records inbound and outbound traffic logs at Layer 4, whereas Anti-DDoS Proxy and WAF record logs for HTTP requests at Layer 7.
Layer 4 logs focus on individual TCP or UDP connections and packets, while Layer 7 logs focus on complete HTTP requests and responses. A single HTTP request or response may be transmitted as multiple TCP packets due to network conditions. This means a single Layer 7 log entry may correspond to multiple Layer 4 log entries.
In addition, Layer 4 logging is more complex and generates more logs than Layer 7 logging due to factors such as retransmissions, network latency, fragmentation, and application-layer protocols like HTTP Keep-Alive. It is important to consider these inherent differences when comparing or analyzing logs from different layers.
Therefore, it is normal for the total number of Cloud Firewall logs to differ from the total number of Anti-DDoS Proxy or WAF logs.
Missing long-lived connection logs after upgrades
Cloud Firewall engine upgrades do not affect user services. However, during maintenance scenarios such as engine upgrades and scaling, traffic logs for some long-lived connections may no longer be reported, resulting in partial data loss.
You can enable the log analysis feature and go to the Log Analysis page. Set the time range to 1 minute and use the query new_conn=0 to filter for traffic from existing connections, which is the data that may be lost. In this scenario, logging resumes after the service re-establishes the connection.
For information about how to enable the log analysis feature, see Enable the log analysis feature.
Querying events from a service-linked role
Log on to the 操作审计控制台.
In the left-side navigation pane, choose .
In the top navigation bar, select the region where you want to query events.
From the filter options, select Operator, set its value to
aliyunserviceroleforcloudfw, specify a time range, and then click the
icon to query for related events.Find the event that you want to query, and click View Details in the Actions column to view the event details and record. For more information, see Event query.