Internet Information Services (IIS) is an extensible web server used to build and host websites. You can collect and analyze IIS access logs to obtain data such as page views (PVs), unique visitors (UVs), client IP distribution, error requests, and inbound and outbound traffic, to monitor and analyze access to your website.

Prerequisites

  • Log Service is activated.
  • A project and a Logstore are created. For more information, see Preparations.
  • IIS access logs are in W3C format.

    To better analyze IIS access logs, we recommend that you use the W3C extended log file format. To select W3C fields to log, follow these steps: In IIS Manager, click Select Fields. In the dialog box that appears, select Bytes Sent (sc-bytes) and Bytes Received (cs-bytes) in the Standard Fields list.

Background information

Log format

You can use the following W3C format:
logExtFileFlags="Date, Time, ClientIP, UserName, SiteName, ComputerName, ServerIP, Method, UriStem, UriQuery, HttpStatus, Win32Status, BytesSent, BytesRecv, TimeTaken, ServerPort, UserAgent, Cookie, Referer, ProtocolVersion, Host, HttpSubStatus"
  • Field prefixes
    Prefix Description
    s- The server action.
    c- The client action.
    cs- The client-to-server action.
    sc- The server-to-client action.
  • Fields
    Field Description
    date The date on which the client sends the request.
    time The time when the client sends the request.
    s-sitename The Internet service name and instance number of the site visited by the client.
    s-computername The name of the server on which the log is generated.
    s-ip The IP address of the server on which the log is generated.
    cs-method The HTTP request method, such as GET or POST, used by the client.
    cs-uri-stem The URI resource requested by the client.
    cs-uri-query The query string that follows the question mark (?) in the HTTP request.
    s-port The port number of the server to which the client is connected.
    cs-username The username used by the client to access the server. Authenticated users are referenced as domain\username. Anonymous users are indicated by a hyphen (-).
    c-ip The IP address of the client that sends the request.
    cs-version The protocol version, such as HTTP 1.0 or HTTP 1.1, used by the client.
    user-agent The browser used by the client.
    Cookie The content of the cookie sent or received. A hyphen (-) is used if no cookie is sent or received.
    referer The site that the client last visited. This site provides a link to the current site.
    cs-host The header name of the host.
    sc-status The HTTP or FTP status code returned by the server.
    sc-substatus The HTTP sub-status code returned by the server.
    sc-win32-status The Windows status code returned by the server.
    sc-bytes The number of bytes sent by the server.
    cs-bytes The number of bytes received by the server.
    time-taken The processing time of the request. Unit: milliseconds.

Procedure

  1. Start the Import Data wizard.
    1. Log on to the Log Service console. On the homepage, click the target project name.
    2. In the left-side navigation pane, find the target Logstore and click the plus sign (+) next to Data Import.
  2. In the Import Data dialog box, select IIS - Text Log.
  3. Select a Logstore.
    You can select an existing Logstore, or create a project and a Logstore.
  4. Create a server group.
    Before creating a server group, make sure that Logtail is installed.
    • Servers of Alibaba Group: By default, Logtail is installed on these servers. If Logtail is not installed on a server, contact Alibaba Cloud as prompted.
    • ECS instances: Select ECS instances and click Install. ECS instances that are running in Windows do not support one-click installation of Logtail. In this case, you need to manually install Logtail. For more information, see Install Logtail in Windows.
    • On-premises servers: Install Logtail as prompted. For more information about how to install Logtail, see Install Logtail in Linux or Install Logtail in Windows based on your operating system.
    After installing Logtail, click Complete Installation to create a server group. If you have created a server group, click Use Existing Server Groups.
  5. Configure the server group.
    Select a server group and move the server group from Source Server Groups to Applied Server Groups.
  6. Create a Logtail configuration.
    1. Set Config Name and Log Path.
      You can view the log path in IIS Manager.
    2. Set Log format.
      Select the log format of your IIS access logs. Valid values:
      • IIS: Microsoft IIS log file format
      • NCSA: NCSA common log file format
      • W3C: W3C extended log file format
    3. Set IIS Configuration.
      • If you select the Microsoft IIS log file format or NCSA common log file format, the IIS configuration is preset.
      • If you select the W3C extended log file format, follow these steps to set IIS Configuration:
        1. Open the IIS configuration file.
          • Default path of the IIS5 configuration file: C:\WINNT\system32\inetsrv\MetaBase.bin
          • Default path of the IIS6 configuration file: C:\WINDOWS\system32\inetsrv\MetaBase.xml
          • Default path of the IIS7 configuration file: C:\Windows\System32\inetsrv\config\applicationHost.config
          Figure 1. View the configuration file
          View the configuration file
        2. In the IIS configuration file, copy the text inside the quotation marks (" ") in the logFile logExtFileFlags field.
        3. Paste the text to the quotation marks (" ") in the IIS Configuration field.
          Figure 2. Configure the data source
          Configure the data source
    4. Confirm IIS key names.
      Log Service automatically extracts the key names from the IIS log format.
      Figure 3. Extract IIS key names
      Extract IIS key names
    5. Set Drop Failed to Parse Logs.
      Specify whether to upload logs that fail to be parsed to Log Service.

      If you turn on this switch, the logs that fail to be parsed are not uploaded to Log Service. If you turn off this switch, the raw logs are uploaded to Log Service when log parsing fails. The key of a raw log is __raw_log__ and the value is the log content.

    6. Optional. Set advanced options as required.
      Parameter Description
      Upload Raw Log Specifies whether to upload the raw log. If you turn on this switch, the raw log content is uploaded as the __raw__ field with the parsed log content.
      Topic Generation Mode
      • Null - Do not generate topic: The default value, which specifies that the topic is set to a null string. You can query logs without entering the topic.
      • Machine Group Topic Attributes: sets the topic based on a machine group to differentiate log data generated on different frontend servers.
      • File Path RegEx: uses Custom RegEx to extract a part of the log path as the topic. This mode is used to differentiate log data generated by different users or instances.
      Custom RegEx The custom regular expression specified if you set Topic Generation Mode to File Path RegEx.
      Log File Encoding
      • utf8: specifies UTF-8 encoding.
      • gbk: specifies GBK encoding.
      Maximum Directory Monitoring Depth The maximum depth of the monitored directory when logs are collected from the log source, that is, at most how many levels of directories can be monitored. Valid values: [0, 1000]. A value of 0 indicates that only the current directory is monitored.
      Timeout Specifies whether the system considers that a log file has timed out if the file is not updated within the specified period. You can set Timeout as follows:
      • Never: specifies that all log files are continuously monitored without timeout.
      • 30 Minute Timeout: specifies that if a log file is not updated within 30 minutes, the system considers that the log file has timed out and no longer monitors the file.
      Filter Configuration The filter conditions that logs must completely meet before they can be collected.
      For example:
      • Collect logs that meet a condition: Set a condition Key:level Regex:WARNING|ERROR, which indicates that only logs whose level is WARNING or ERROR are collected.
      • Filter logs that do not meet a condition:
        • Set a condition Key:level Regex:^(?!.*(INFO|DEBUG)).* , which indicates that logs whose level is INFO or DEBUG are not collected.
        • Set a condition Key:url Regex:.*^(?!.*(healthcheck)).* , which indicates that logs with healthcheck in url are not collected. For example, logs in which the key is url and the value is /inner/healthcheck/jiankong.html are not collected.
      For more examples, see regex-exclude-word and regex-exclude-pattern.
      Confirm the configuration and click Next.
  7. Configure log query and analysis.
    By default, Log Service creates key names for you. You can set the actual key names based on previewed data and map them to the default key names. To create or modify indexes, choose Index Attributes > Modify on the Search & Analysis page.
    You can preview the collected log data if servers in the serve group have normal heartbeats.
    Figure 4. Preview logs
    Preview logs

    Log Service creates a default dashboard named LogstoreName-iis-dashboard for you. After the preceding configuration is completed, you can view real-time data on the dashboard, including the distribution of client IP addresses and the percentage of each HTTP status code.

    • Client IP distribution: Use the following SQL statement to collect statistics on the distribution of client IP addresses:
      | select ip_to_geo("c-ip") as country, count(1) as c group by ip_to_geo("c-ip") limit 100
    • PVs and UVs: Use the following SQL statement to count the number of PVs and UVs:
      *| select approx_distinct("c-ip") as uv ,count(1) as pv , date_format(date_trunc('hour', __time__), '%m-%d %H:%i') as time group by date_format(date_trunc('hour', __time__), '%m-%d %H:%i') order by time limit 1000
      Figure 5. PVs and UVs
      PVs and UVs
    • Percentages of HTTP status codes: Use the following SQL statement to count the percentage of each HTTP status code returned:
      *| select count(1) as pv ,"sc-status" group by "sc-status"
      Figure 6. Percentages of HTTP status codes
      Percentages of HTTP status codes
    • Inbound and outbound traffic: Use the following SQL statement to collect statistics on the inbound and outbound traffic:
      *| select sum("sc-bytes") as net_out, sum("cs-bytes") as net_in ,date_format(date_trunc('hour', time), '%m-%d %H:%i') as time group by date_format(date_trunc('hour', time), '%m-%d %H:%i') order by time limit 10000
      Figure 7. Inbound and outbound traffic
      Inbound and outbound traffic
    • Percentages of request methods: Use the following SQL statement to count the percentage of each request method used:
      *| select count(1) as pv ,"cs-method" group by "cs-method"
      Figure 8. Percentages of request methods
      Percentages of request methods
    • Percentages of browser types: Use the following SQL statement to count the percentage of each browser type used:
      *| select count(1) as pv, case when "user-agent" like '%Chrome%' then 'Chrome' when "user-agent" like '%Firefox%' then 'Firefox' when "user-agent" like '%Safari%' then 'Safari' else 'unKnown' end as "user-agent" group by case when "user-agent" like '%Chrome%' then 'Chrome' when "user-agent" like '%Firefox%' then 'Firefox' when "user-agent" like '%Safari%' then 'Safari' else 'unKnown' end order by pv desc limit 10
      Figure 9. Percentages of browser types
      Percentages of browser types
    • Top 10 most visited pages: Use the following SQL statement to count the top 10 visited pages with the most PVs:
      *| select count(1) as pv, split_part("cs-uri-stem",'?',1) as path group by split_part("cs-uri-stem",'?',1) order by pv desc limit 10
      Figure 10. Top 10 most visited pages
      Top 10 most visited pages