You can use Telegraf to collect ClickHouse metrics, use Logtail to send the metric data to a MetricStore, and then create a dashboard to visualize the ClickHouse metrics. This topic describes how to use Log Service to collect metrics from a ClickHouse server and visualize the ClickHouse metrics.

Prerequisites

  • Linux Logtail 0.16.48 or later is installed on the ClickHouse server. For more information, see Install Logtail on a Linux server.
  • Telegraf is installed on a server that is connected to the ClickHouse server over a private network.

Procedure

  1. Log on to the Log Service console.
  2. In the Import Data section, click the ClickHouse Monitoring Data card.
  3. In the Specify Logstore step, select the project and the Metricstore that you want to use. Then, click Next.
    You can also click Create Now to create a project or a Metricstore. For more information, see Create a project and Create a Metricstore.
  4. In the Create Machine Group step, create a machine group.
    • If a machine group is available, click Using Existing Machine Groups.
    • If no machine groups are available, perform the following steps to create a machine group. In this example, an Elastic Compute Service (ECS) instance is used.
      1. Select the ECS instance on which you want to install Logtail. For more information, see Install Logtail on ECS instances.
        If Logtail is installed on the ECS instance, click Complete Installation.
        Note If you want to collect data from servers in self-managed clusters or servers on third-party clouds, you must manually install Logtail. For more information, see Install Logtail on a Linux server.
      2. After Logtail is installed, click Complete Installation.
      3. Create a machine group.

        For more information about how to create a machine group, see Create an IP address-based machine group or Create a custom ID-based machine group.

  5. Select the newly created machine group and move it from the Source Server Groups section to the Applied Server Groups section. Then, click Next.
    Notice If you apply a machine group immediately after it is created, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Log Service. In this case, you can click Automatic Retry. If the issue persists, see What do I do if no heartbeat connections are detected on Logtail?
  6. In the Specify Data Source step, set the required parameters. The following table describes the parameters.
    Parameter Description
    Configuration Name The name of the Logtail configuration file.
    Cluster Name The name of the ClickHouse cluster. After you set this parameter, Log Service adds the cluster=<cluster name> tag to your data.
    Note The cluster name must be unique. Otherwise, data conflicts may occur.
    Username The username of the account that is used to log on to the ClickHouse server.
    Password The password of the account.
    Server List Click the + icon to add the ClickHouse server.
    • Address: the endpoint of the ClickHouse server.
    • Port: the port number of the ClickHouse server. The default value is 8123. You do not need to modify the value.

    You can add multiple ClickHouse servers based on your business requirements.

    Custom Tags In a MetricStore, you can create multiple Logtail configuration files. You can add custom tags that are specified in the Custom Tags field to the data that is collected based on these Logtail configuration files.

    Click the + icon to add a custom tag. You can add multiple custom tags. These custom tags are added to each data entry.

FAQ

How can I check whether Telegraf collects data as expected?

You can check the logs of the /etc/ilogtail/telegraf/telegraf.log file on your server. You can also collect this log file to Log Service and search for the required information.

What to do next

  • Query and analyze metrics

    After the configuration, Telegraf uses Logtail to upload collected metrics to the MetricStore of Log Service. You can query the data on the Search & Analysis page of the MetricStore. For more information, see Query and analyze time series data.

  • Visualize metrics

    After the configuration, Log Service automatically creates a dashboard named ClickHouse Monitoring Data_<cluster name> in the related project. In the dashboard, you can perform the required operations. For example, you can view query results or configure alert rules.