After a CloudMonitor agent parses on-premises logs into specific metrics based on your settings, Hybrid Cloud Monitoring reports the metrics to the specified namespace and visualizes the related monitoring data in charts. You can use a monitoring dashboard in Hybrid Cloud Monitoring to create custom charts. You can also configure Prometheus alert rules to receive alert notifications.

Prerequisites

Step 1: Install a CloudMonitor agent on the host from which on-premises logs are collected

The CloudMonitor agent can be manually or automatically installed on the host from which on-premises logs are collected.

If you want to use the on-premises log monitoring feature of Hybrid Cloud Monitoring to monitor multiple hosts but the CloudMonitor agent fails to be automatically installed on the hosts or the hosts cannot access the Internet, you must manually install the CloudMonitor agent on the hosts. Then, you must create an application group and add the hosts to the application group. This way, you can manage the hosts in a centralized manner.

  1. Manually install the CloudMonitor agent on the host from which on-premises logs are collected.

    Make sure that a CloudMonitor agent whose version is V3.5.5 or later is installed on the instances of the selected application group. We recommend that you install a CloudMonitor agent of the latest version. For more information, see Manually install the CloudMonitor agent for C++ on an ECS instance or Manually install the CloudMonitor agent for C++ on a host that is not provided by Alibaba Cloud.

  2. View the installation result of the CloudMonitor agent.

    In the left-side navigation pane, click Host Monitoring. On the page that appears, you can view the version and status of the CloudMonitor agent that is installed on your host.

  3. Create an application group for the host from which on-premises logs are collected.
    1. In the left-side navigation pane, click Application Groups.
    2. In the upper-right corner of the Application Group tab, click Create Application Group.
    3. In the Create Application Group panel, set Creation Method to Manually Create, enter a name in the Application Group Name field, turn off Initialize Agent Installation, and then select an alert contact group from the Alert Contact Group drop-down list.
    4. Click OK.
  4. Add the host from which on-premises logs are collected to the application group.
    1. On the Group Resources page of the application group, click Manage Services and Resources.
    2. In the Add/Modify Group Resources panel, select the cloud services and resources that you want to add to the application group.
    3. Click OK.

      On the Group Resources page of the application group, you can view the information about the cloud services and instances that you added.

If you want to use the on-premises log monitoring feature of Hybrid Cloud Monitoring to monitor multiple hosts, you can create an application group and add the hosts to the application group. The CloudMonitor agent can be automatically installed on the hosts.

Note: If you use an application group to automatically install a CloudMonitor agent, the CloudMonitor agent of the latest version is installed on the instances of the application group.

  1. Create an application group for the host from which on-premises logs are collected.
    1. Log on to the CloudMonitor console.
    2. In the left-side navigation pane, click Application Groups.
    3. In the upper-right corner of the Application Group tab, click Create Application Group.
    4. In the Create Application Group panel, set Creation Method to Manually Create, enter a name in the Application Group Name field, turn on Initialize Agent Installation, and then select an alert contact group from the Alert Contact Group drop-down list.
    5. Click OK.
  2. Add the host from which on-premises logs are collected to the application group.
    1. On the Group Resources page of the application group, click Manage Services and Resources.
    2. In the Add/Modify Group Resources panel, select the cloud services and resources that you want to add to the application group.
    3. Click OK.

      On the Group Resources page of the application group, you can view the information about the cloud services and instances that you added.

  3. View the installation result of the CloudMonitor agent.

    In the left-side navigation pane, click Host Monitoring. On the page that appears, you can view the version and status of the CloudMonitor agent that is installed on your host.

Step 2: Create a metric import task for on-premises logs

After you create a metric import task, the CloudMonitor agent parses on-premises logs into specific metrics based on your settings for the metric import task. Then, Hybrid Cloud Monitoring reports the metrics to the specified namespace and visualizes the related monitoring data in charts.

  1. In the left-side navigation pane, choose Hybrid Cloud Monitoring > Business Monitoring.
  2. Click the On-premises Log Monitoring tab.
  3. In the upper-right corner of the On-premises Log Monitoring tab, click Metric Data.
  4. On the On-premises Log Monitoring tab, set the parameters. The following section describes the parameters.
    • Task Name: Enter the name of the metric import task for on-premises logs.
    • Application Groups: Select the application group that you created in Step 1.
    • Hosts: The rule that is used to match hosts from which on-premises logs are collected. Default value: All.
    • Log Path: The path where on-premises logs are stored.
      • Windows: The log path must start with a drive letter. Example:C:\UserData\logs\pre.cms.console.aliyun.com.@FUNC__DateFormat__YYYY-MM-DD@.log. You cannot specify wildcard characters in a log path.
      • Linux: The log path must start with a forward slash (/). Example: /home/logs/pre.cms.console.aliyun.com.@FUNC__DateFormat__YYYY-MM-DD@.log.
      Note pre.cms.console.aliyun.com.YYYY-MM-DD.log indicates the log file from which on-premises logs are collected. If you want to exclude the error.log error log file, use @FUNC__DateFormat__YYYY-MM-DD@ to filter the log file whose name contains a date.
    • Log parsing: On-premises logs are parsed into specific metrics. The following parsing modes are supported:
      • Full Regex Mode
        1. In the Sample Log section, enter a sample log.
        2. Click Confirm. Then, click Generate Regular Expression.
        3. In the Log Extraction Results section, find the key for which you want to configure translation values. Then, click Column Value Translation.
        4. Set the Column Name and Translation Value parameters and click OK.

          Replace the metric value that is parsed from the sample log with another value for display.

      • Delimiter Mode
        1. In the Sample Log section, enter a sample log.
        2. Click Confirm. Then, click Add Column.
        3. Enter a name for the column in the Column Name field and select the field that you want to parse in the Sample Log section. The system automatically generates values for the Column Value Sampling and Extraction Rule parameters. Then, set the Translation Value parameter and click OK.

          Replace the metric value that is parsed from the sample log with another value for display.

      • JSON Mode
        1. In the Sample Log section, enter a JSON-formatted sample log.
        2. Click Confirm.
        3. In the Log Extraction Results section, find the key for which you want to configure translation values and click Column Value Translation.
        4. Set the Translation Value parameter and click OK.

          Replace the metric value that is parsed from the sample log with another value for display.

    • Metric Settings: queries statistics on the metrics that are parsed from the sample log and displays the results in charts.
    • Namespace: Select a namespace where you want to store on-premises logs.
    • Tags: classifies metrics.
  5. Click OK.

Step 3: View the monitoring chart of on-premises logs

You can view the monitoring chart of a specified metric of on-premises logs.

  1. On the On-premises Log Monitoring tab, find the metric import task whose charts you want to view and click View Data in the Actions column.
  2. On the View Data page, select a metric.

Step 4: Create custom charts for on-premises logs

You can use the dashboard feature of Hybrid Cloud Monitoring to create a monitoring dashboard for the metrics of on-premises logs and create custom charts.

  1. In the left-side navigation pane, choose Hybrid Cloud Monitoring > Dashboards.
  2. On the Dashboards tab, click View Dashboard.
  3. In the left-side navigation pane of the dashboard, click the Create icon. Then, click Add new panel.
  4. On the Edit Panel page, configure metrics and monitoring charts.
    • On the Query tab, select the namespace and select metrics.
    • On the Panel tab, specify a panel title and enter a description. Then, configure visualization and other settings.
  5. In the upper-right corner of the Edit Panel page, click Apply.
    The monitoring chart is created.
  6. In the upper-right corner, click the Save the dashboard icon. Then, specify the name of the dashboard and the folder in which the dashboard resides.
  7. Click Save.
    The dashboard is created.
  8. In the left-side navigation pane of the dashboard, choose Dashboard > Manage to view the dashboard.

Step 5: Configure an alert rule for on-premises logs

You can configure alert rules for the metrics of on-premises logs. If the specified conditions are met, you can receive alert notifications.

  1. In the left-side navigation pane, choose Hybrid Cloud Monitoring > Namespace.
  2. In the upper-right corner of the Namespaces tab, click Create Alert Rule.
  3. In the Create Alert Rule panel, enter a name in the Rule Name field, select a namespace from the Namespace drop-down list, set the Alert Expression (PromQL) parameter, and then select an alert contact group from the Alert Contact Group drop-down list.
  4. Click OK.