Cloud Firewall:Log management

Prerequisites

Your assets have been connected. For more information, see Access management.

Enable log delivery

1. Log on to the Agentic NDR console.

2. In the left-side navigation pane, choose Logs.

3. Log Delivery: In the upper-right corner of the Log Analysis page, click Delivery Settings. Turn on both Alert Logs and Protocol Log.

4. Set log delivery rules: In the upper-right corner of the Log Analysis page, click Log Settings. The following operations are supported:

   • Modify the log storage region.

   • Modify the log storage duration.

   • Expand log storage space.

   For specific operations and precautions, see Modify Log Storage Configurations.

Configure filter rules and delivery fields

1. In the upper-right corner of the Log Analysis page, click Log Settings.

2. Configure the settings on the Custom Filter Rules and Custom Delivery Fields tabs.

   • Custom filter rules

     • Create a filter rule

       1. On the Custom Filter Rules tab, click Create Rule.

       2. In the Protocol Log Filter Rule panel, configure the Filter Logic and Stream Information, and then click OK.

          Important

          When multiple log filter rules exist:

          1. Multiple rules in an allowlist or a denylist are combined with OR logic.

          2. Denylist rules override allowlist rules. If a conflict occurs, the denylist rule is applied.

     • Edit a filter rule

       1. In the Actions column of the target filter rule, click Edit.

       2. In the Protocol Log Filter Rule panel, modify the filter rule, and then click OK.

     • Delete a filter rule

       1. In the Actions column of the target filter rule, click Delete.

       2. Review the warning message and click OK.

          Important

          Once deleted, a log filter rule is permanently removed and stops filtering matching logs.

   • Custom delivery fields

     On the Custom Delivery Fields tab, select the fields to deliver, and then click OK.

Log query

1. In the left-side navigation pane, choose Logs.

   By default, when you open the log analysis page, the system automatically runs a query and displays the results.

2. In the search box, enter a search statement and an analytic statement.

   • A search statement is used to search and filter log data. You can use criteria like a time range, request type, or keyword to find specific data. A search statement can be used on its own. For more information about the syntax, see Query syntax and features.

   • An analytic statement is used to filter, transform, count, and aggregate log data. For example, you can calculate the average value of data over a period or obtain year-over-year and month-over-month results. An analytic statement must be used with a search statement in the format search statement|analytic statement. For more information about the syntax, see Aggregate functions.

   • To the right of the search box, click the icon to switch to the interactive query and analysis mode. You can use Data Explorer to quickly build search and analytic statements without writing SQL. For more information, see High-performance and high-precision query and analysis (Dedicated SQL).

   Statements

   Log analysis statements consist of a search statement and an optional analytic statement, separated by a vertical bar (|).

   search statement | analytic statement

   • A search statement uses syntax specific to Simple Log Service. You can use it independently to query logs that meet specific conditions. A search statement can contain keywords, numeric values, ranges, spaces, or an asterisk (*). A space or an asterisk (*) matches all logs without filtering.

     The search syntax is divided into full-text search and field-specific search, depending on how the index is configured. For more information, see Query syntax and functions.

     Full-text search

     You can specify keywords for a full-text search. Enclose a keyword in double quotation marks ("") to query logs containing that keyword. You can also separate multiple keywords with spaces or and to find logs that contain all specified keywords.

     • Multi-keyword search

       Search for logs that contain both www.aliyun.com and pass.

       www.aliyun.com pass

       or

       www.aliyun.com and pass

     • Conditional search

       Search for all logs that contain www.aliyun.com and either pass or tcp.

       www.aliyun.com and (pass or tcp)

     • Prefix search

       Search for all logs that contain www.aliyun.com and start with tcp_.

       www.aliyun.com and tcp_*

       Note

       You can use an asterisk (*) as a suffix to perform a prefix search. Wildcard prefixes, such as in *_not_establish, are not supported.

     Field-specific search

     You can perform a field-specific search by specifying a field name and a value. This search supports comparison queries for numeric fields in the format field:value or field >= value. You can also use operators such as and and or to create combined queries and use them in combination with full-text searches.

     For a list of fields in Cloud Firewall log analysis that support an index, see Fields that support indexes.

     • Query multiple fields

       Query the access logs for requests from the specified client 192.XX.XX.22 to the destination address 192.XX.XX.54.

       src_ip: 192.XX.XX.22 and dst_ip: 192.XX.XX.54

     • Query for field existence

       • Query the logs that contain the cloud_instance_id field.

         cloud_instance_id: *

       • Query logs that do not contain the cloud_instance_id field.

         not cloud_instance_id: *

   • An analytic statement calculates and generates statistics from the results of a search statement or from the entire dataset. It must be used with a search statement. If the analytic statement is empty, only the search results are returned without any analysis. For more information about the syntax and functions of analytic statements, see Overview of log query and analysis.

   Common search statements

   This section provides common search statements and examples for Cloud Firewall logs.

   Traffic volume search statements

   • Query for network records of access initiated from the internet to the internal asset 1.2.*.*, and calculate the total size of inbound traffic and the total number of packets.

     log_type:internet_log and direction:"in" and dst_ip:1.2.*.* | select sum(in_packet_bytes) as flow, sum(in_packet_count) as packet

   • Query NAT firewall traffic, group the results by the source IP, destination IP, and destination port fields, and analyze the top 10 results ranked by the total size of inbound and outbound traffic.

     log_type:nat_firewall_log | select src_ip, dst_ip, dst_port, sum(in_packet_bytes) as in_bytes, sum(out_packet_bytes) as out_bytes, sum(total_packet_bytes) as total_bytes group by src_ip, dst_ip, dst_port order by total_bytes desc limit 10

   Access control statements

   • Query for traffic logs of access initiated from the internet to internal assets that match an access control policy.

     log_type:internet_log and direction:"in" and not acl_rule_id:00000000-0000-0000-0000-000000000000

     The condition not acl_rule_id... filters for logs that match an access control policy. An acl_rule_id of 00000000-0000-0000-0000-000000000000 indicates that no policy was matched.

   • Query for traffic logs of access initiated from internal assets to the internet that were blocked by an access control policy. Then, analyze the distribution of the top 10 destination IP addresses and destination ports.

     log_type:internet_log and direction:out and not acl_rule_id:00000000-0000-0000-0000-000000000000 and rule_result:drop | select dst_ip, dst_port, count(*) as cnt group by dst_ip, dst_port order by cnt desc limit 10

   • This query finds traffic logs of access initiated from internal assets to the internet with destination port 443, more than three packets (indicating a completed TCP three-way handshake), and no domain information detected. The results are then grouped by the destination IP segment and application to identify the top 10 source IPs, destination IPs, and applications.

     log_type:internet_log and direction:out and dst_port:443 and total_packet_count>3 and domain:""| select array_agg(distinct src_ip) as srcip, array_agg(distinct dst_ip) as dstip, slice(split(dst_ip,'.' ,4),1,3) as dstip_c, app_name, COUNT(1) as cnt GROUP by dstip_c,app_name order by cnt desc limit 10

   Attack prevention statements

   • Query for traffic logs of access initiated from the internet to internal assets that match an attack prevention policy.

     log_type:internet_log and direction:"in" and not ips_rule_id:00000000-0000-0000-0000-000000000000

     The condition not ips_rule_id... filters for logs that match an attack prevention policy. An ips_rule_id of 00000000-0000-0000-0000-000000000000 indicates that no policy was matched.

   • Query for traffic logs of access initiated from internal assets to the internet that match an attack prevention policy, and view information such as the IP address, port, application, domain, and IPS policy results.

     log_type:internet_log and direction:out and not ips_rule_id:00000000-0000-0000-0000-000000000000 | select src_ip, dst_ip, dst_port,app_name, domain,ips_rule_id, ips_rule_name, rule_result

3. Click Search & Analyze to view the query and analysis results. For more information, see View query and analysis results.

Query result analysis

You can view the results of your log query and analysis in modules such as the histogram and raw logs.

Related documentation

• For a detailed description of log fields, see Log fields.

• You can export the results of a query and analysis to your local computer for storage. For more information, see Export logs.

• If you have a large volume of logs that require long-term storage, you can periodically ship logs to an OSS bucket for storage and analysis. For more information, see Create an OSS data shipping job (New).