This topic describes how to use Grafana to display and analyze NGINX logs that are collected by Log Service.

Prerequisites

  • NGINX logs are collected. For more information, see Collect logs in NGINX mode.
  • The indexing feature is enabled, and indexes are created. For more information, see Collect and analyze NGINX access logs.
  • The aliyun-log-grafana-datasource-plugin software package is downloaded. For more information, visit https://github.com/aliyun/aliyun-log-grafana-datasource-plugin.
    Note In this topic, aliyun-log-grafana-datasource-plugin V2.9 is used.
  • Grafana is installed. For more information, see Grafana documentation.
    Note
    • In this topic, Grafana 8.0.6 is used.
    • If you install Grafana on your computer, you must enable port 3000 in your browser settings.
    • If you want to use pie charts, you can run the following command to install the Pie Chart plug-in:
      grafana-cli plugins install grafana-piechart-panel

Version compatibility between Grafana and aliyun-log-grafana-datasource-plugin

The following table describes the version compatibility between Grafana and aliyun-log-grafana-datasource-plugin.

Grafana aliyun-log-grafana-datasource-plugin
8.0.0 and later V2.x
Earlier than 8.0.0 V1.0

Step 1: Install the Log Service plug-in

The following procedure describes how to install the Log Service plug-in for Grafana:

  1. Run the following commands to decompress the project package to the plug-in directory of Grafana.
    • If Grafana is installed by using a YUM repository or an RPM package, run the following command:
      unzip aliyun-log-grafana-datasource-plugin-master.zip -d /var/lib/grafana/plugins
    • If Grafana is installed by using a .tar.gz file, run the following command:

      {PATH_TO} specifies the installation directory of Grafana.

      unzip aliyun-log-grafana-datasource-plugin-master.zip -d {PATH_TO}/grafana-8.0.6/data/plugins
  2. Modify the configuration file of Grafana.
    1. Open the configuration file.
      • If Grafana is installed by using a YUM repository or an RPM package, open the /etc/grafana/grafana.ini file.
      • If Grafana is installed by using a .tar.gz file, open the {PATH_TO}/grafana-8.0.6/conf/defaults.ini file.
    2. Configure the allow_loading_unsigned_plugins parameter.
      allow_loading_unsigned_plugins = aliyun-log-service-datasource,grafana-log-service-datasource
  3. Restart the Grafana service.
    1. Run the kill command to terminate the Grafana process.
    2. Run the following commands to start the Grafana service:
      • If Grafana is installed by using a YUM repository or an RPM package, run the following command:
        systemctl restart grafana-server
      • If Grafana is installed by using a .tar.gz file, run the following command:
        ./bin/grafana-server web

Step 2: Add a data source for Grafana

The following procedure describes how to add the Log Service plug-in as a data source for Grafana:

  1. Log on to Grafana.
  2. In the left-side navigation pane, choose G1 > Data Sources.
  3. On the Data Sources tab, click Add data source.
  4. On the Add data source page, click Select in the LogService card.
  5. Configure the data source.
    The following table describes the parameters.
    Parameter Description
    Name The name of the data source.
    Default The Default switch. In this example, turn on the switch.
    Endpoint The endpoint of the Log Service project. Example: http://cn-qingdao.log.aliyuncs.com. Enter an endpoint based on your business requirements. For more information, see Endpoints.
    Project The name of the project.
    Logstore The name of the Logstore.
    AccessKeyId The AccessKey ID provided by Alibaba Cloud. The AccessKey ID is used to identify the user. To ensure the security of your account, we recommend that you use the AccessKey pair of a RAM user. For more information about how to obtain an AccessKey pair, see AccessKey pair.
    AccessKeySecret The AccessKey secret provided by Alibaba Cloud. The AccessKey secret is used to authenticate the key of the user. To ensure the security of your account, we recommend that you use the AccessKey pair of a RAM user. For more information about how to obtain an AccessKey pair, see AccessKey pair.
  6. Click Save & Test.

Step 3: Add a dashboard

Perform the following steps to add a dashboard for Grafana:

  1. In the left-side navigation pane, choose Icon 1 > Dashboards.
  2. In the New Panel panel, click Choose Visualization.
  3. Configure template variables.

    After you configure template variables in Grafana, you can select different variable values to view different results in the same panel.

    1. Configure a template variable for a time interval.
      1. In the upper-right corner of the New dashboard page, select a time interval and click the Icon 2 icon.
      2. Click Variables.
      3. Click Add variable.
      4. Configure the parameters for the template variable. Then, click Add.
        The following table describes the parameters.
        Parameter Description
        Name Enter the name of the template variable. Example: myinterval. If you use a template variable in a query criterion, you must add two dollar signs ($$) preceding the name of the template variable. Example: $$myinterval.
        Type Select Interval.
        Label Enter time interval.
        Values Enter 1m,10m,30m,1h,6h,12h,1d,7d,14d,30d.
        Auto Option Turn on Auto Option and retain the default values of the other parameters.
    2. Configure a template variable for a domain.
      1. On the Variables page, click New.
      2. Configure the parameters for the template variable. Then, click Add. The following table describes the parameters.
        Parameter Description
        Name Enter the name of the template variable. Example: hostname. If you use a template variable in a query criterion, you must add a dollar sign ($) preceding the name of the template variable. Example: $hostname.
        Type Select Custom.
        Label Enter the name of the domain.
        Custom Options Enter *,example.com,example.org,example.net in the "Values separated by comma" field. This value specifies that you can view information about access to all domains. You can also view information about access to the example.com, example.org, or example.net domain.
        Selection Options Retain the default value of the parameter.
    3. In the left-side navigation pane, click Save.
  4. Add panels based on your business requirements.
    • Add a graph panel to display page views (PVs) and unique visitors (UVs).
      1. In the upper-right corner of the page, click the Icon 3 icon.
      2. In the New Panel panel, click Add Query.
      3. Click the Icon 4 icon and select Graph from the Visualization drop-down list.
      4. Click the Icon 5 icon. Then, enter UV&PV in the Title field.
      5. Click the Icon 6 icon, select Logservice from the Query drop-down list, and then configure the following parameters. grafana-pv&uv01
        Parameter Description
        Query Enter the query statement that you want to execute. Example:
        $hostname| select approx_distinct(remote_addr) as uv ,count(1) as pv , __time__ - __time__ % $$myinterval as time group by time order by time limit 1000
        In the query result, $hostname is replaced by the name of the domain that you specify, and $$myinterval is replaced by the time interval that you specify.
        Notice Two dollar signs ($$) must precede myinterval, and one dollar sign ($) must precede hostname.
        X-Column Enter time. The value is a UNIX timestamp with second precision.
        Y-Column Enter uv,pv.
      6. If the difference between the value of UV and the value of PV is significant, use a dual Y-axis panel to display UV and PV statistics. grafana-pv&uv02
      7. Click the Save icon icon to save your settings.
    • Add a graph panel to display inbound traffic and outbound traffic.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)." section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select sum(body_byte_sent) as net_out, sum(request_length) as net_in,__time__ - __time__ % $$myinterval as time group by __time__ - __time__ % $$myinterval limit 10000
      In the query result, $hostname is replaced by the name of the domain that you specify, and $$myinterval is replaced by the time interval that you specify.
      Notice Two dollar signs ($$) must precede myinterval, and one dollar sign ($) must precede hostname.
      X-Column Enter time. The value is a UNIX timestamp with second precision.
      Y-Column Enter net_in,net_out.
    • Add a pie chart panel to display the proportion of requests that are sent by using each HTTP request method.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select count(1) as pv ,method group by method

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter pie.
      Y-Column Enter method,pv.
    • Add a pie chart panel to display the proportion of responses that carry each HTTP status code.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select count(1) as pv ,status group by status

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter pie.
      Y-Column Enter status,pv.
    • Add a pie chart panel to display the request sources of frequently accessed pages.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select count(1) as pv , referer group by referer order by pv desc

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter pie.
      Y-Column Enter referer,pv.
    • Add a table panel to display the pages that are loaded at the highest latencies.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select URL as top_latency_URL ,request_time order by request_time desc limit 10

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column You do not need to configure this parameter.
      Y-Column Enter top_latency_url,request_time.
    • Add a table panel to display frequently accessed pages.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select count(1) as pv, split_part(URL,'?',1) as path group by split_part(URL,'?',1) order by pv desc limit 20

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column You do not need to configure this parameter.
      Y-Column Enter path,pv.
    • Add a table panel to display the pages that have the most access requests to which HTTP status code 200 is not returned.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname not status:200| select count(1) as pv , url group by url order by pv desc

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column You do not need to configure this parameter.
      Y-Column Enter url,pv.
    • Add a singlestat panel to display the average latency.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname | select avg(request_time) as response_time, avg(upstream_response_time) as upstream_response_time ,__time__ - __time__ % $$myinterval as time group by __time__ -  __time__ % $$myinterval limit 10000
      In the query result, $hostname is replaced by the name of the domain that you specify, and $$myinterval is replaced by the time interval that you specify.
      Notice Two dollar signs ($$) must precede myinterval, and one dollar sign ($) must precede hostname.
      X-Column Enter time.
      Y-Column Enter upstream_response_time,response_time.
    • Add a logs panel to display the details about logs.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Note Each page can display a maximum of 100 logs. Therefore, the maximum value of the Logs Per Page parameter is 100.
    • Add a pie chart panel to display client statistics.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname  | select count(1) as pv, case when  regexp_like(http_user_agent , 'okhttp') then 'okhttp' when  regexp_like(http_user_agent ,  'iPhone') then 'iPhone' when regexp_like(http_user_agent ,  'Android')  then 'Android' else 'unKnown' end as http_user_agent group by  http_user_agent order by pv desc limit 10

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter pie.
      Y-Column Enter http_user_agent,pv.
    • Add a graph panel to display the number of requests to which each HTTP status code is returned over a 1-minute time range.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname  | select to_unixtime(time) as time,status,count from (select time_series(__time__, '1m', '%Y-%m-%d %H:%i', '0')  as time,status,count(*) as count from log group by status,time order by time limit 10000)

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter time.
      Y-Column Enter col1#:#col2. col1 is the aggregated column, and col2 is one of the other columns.
    • Add a worldmap panel to display the distribution of source IP addresses.
      For more information about how to add a panel, see the "Add a graph panel to display page views (PVs) and unique visitors (UVs)" section of this topic. The following table describes the parameters.
      Parameter Description
      Query Enter the query statement that you want to execute. Example:
      $hostname  | select   count(1) as pv ,geohash(ip_to_geo(arbitrary(remote_addr))) as geo,ip_to_country(remote_addr) as country  from log group by country having geo <>'' limit 1000

      In the query result, $hostname is replaced by the name of the domain that you specify.

      X-Column Enter map.
      Y-Column Enter country,geo,pv.
      Parameter Description
      Location Data Select geohash.
      Location Name Field Enter country.
      geo_point/geohash Field Enter geo.
      Metric Field Enter pv.
  5. View the results.

    In the upper section of the Dashboard page, you can select a time interval. You can also filter the results by using the time interval and hostname query criteria.

FAQ

  • Where are Grafana logs stored?
    Grafana logs are stored in the following directories:
    • macOS: /usr/local/var/log/grafana
    • Linux: /var/log/grafana
  • What do I do if aliyun-log-plugin_linux_amd64: permission denied appears in logs?

    Grant the EXECUTE permission on the dist/aliyun-log-plugin_linux_amd64 directory of the plug-in directory.