Configure X-Pack Watcher

This topic describes how to configure X-Pack Watcher for Elasticsearch. X-Pack Watcher allows you to trigger specific actions when specified conditions are met. For example, you can create a watch for Elasticsearch to search the logs index for errors and send alerts through emails or DingTalk messages. X-Pack Watcher is a monitoring and alerting service based on Elasticsearch.

1. Configure a security group rule for the ECS instance.
   1. Log on to the Alibaba Cloud Elastic Compute Service console. In the left-side navigation pane, click Instances & Images. Then, click Instances.
   2. On the Instances page, find the target instance, click More, and then choose Network and Security Group > Configure Security Group in the Actions column.
   3. On the Security Groups tab, find the target security group and click Add Rules in the Actions column.
   4. On the Security Group Rules page, click Add Security Group Rule.
   5. Specify the required parameters.
      

      Parameter	Description
      Rule Direction	Select Inbound.
      Action	Select Allow.
      Protocol Type	Select Custom TCP.
      Priority	Retain the default value.
      Port Range	Set the port to your frequently used port. This parameter is required for NGINX configurations. In this example, port 8080 is used.
      Authorization Type	Select IPv4 CIDR Block.
      Authorization Object	Add the IP addresses of all nodes in your Elasticsearch cluster. Note To query the IP addresses of the nodes, log on to the Kibana console of your Elasticsearch cluster based on Log on to the Kibana console, click Monitoring in the left-side navigation pane, and then click Nodes.

   6. Click OK.
2. Configure an NGINX proxy.
   1. Install NGINX on the ECS instance.
   2. Configure the nginx.conf file.
      Replace the server configuration in the nginx.conf file with the following content:

      server
        {
          listen 8080;# Listening port
          server_name localhost;# Domain name
          index index.html index.htm index.php;
          root /usr/local/webserver/nginx/html;# Website directory
            location ~ . *\.(php|php5)? $
          {
            #fastcgi_pass unix:/tmp/php-cgi.sock;
            fastcgi_pass 127.0.0.1:9000;
            fastcgi_index index.php;
            include fastcgi.conf;
          }
          location ~ . *\.(gif|jpg|jpeg|png|bmp|swf|ico)$
          {
            expires 30d;
            # access_log off;
          }
          location / {
            proxy_pass <Webhook address of the DingTalk Chatbot>;
          }
          location ~ . *\.(js|css)? $
          {
            expires 15d;
            # access_log off;
          }
          access_log off;
        }

      <Webhook address of the DingTalk Chatbot>: Replace it with the webhook address of the DingTalk Chatbot that is used to receive alert notifications.

      Note To query the webhook address of the DingTalk Chatbot, create an alert group in DingTalk. Then, in the DingTalk group, click the More icon in the upper-right corner, click ChatBot, and then select Custom to add a ChatBot that is connected by using webhooks. You can then view the webhook address of the DingTalk Chatbot.
   3. Reload the NGINX configuration file and restart NGINX.

      /usr/local/webserver/nginx/sbin/nginx -s reload            # Reload the NGINX configuration file.
      /usr/local/webserver/nginx/sbin/nginx -s reopen            # Restart NGINX.

3. Create a watch.
   1. Log on to the Kibana console of your Elasticsearch cluster.
      Note For more information, see Log on to the Kibana console.
   2. In the left-side navigation pane, click Dev Tools.
   3. On the Console tab, run the following command to create a watch:
      The following example shows how to create a watch named log_error_watch to search the logs index for errors every 10 seconds. If more than 0 errors are found, an alert is triggered.

      PUT _xpack/watcher/watch/log_error_watch
      {
        "trigger": {
          "schedule": {
            "interval": "10s"
          }
        },
        "input": {
          "search": {
            "request": {
              "indices": ["logs"],
              "body": {
                "query": {
                  "match": {
                    "message": "error"
                  }
                }
              }
            }
          }
        },
        "condition": {
          "compare": {
            "ctx.payload.hits.total": {
              "gt": 0
            }
          }
        },
        "actions" : {
        "test_issue" : {
          "webhook" : {
            "method" : "POST",
            "url" : "http://<Private IP address of your ECS instance>:8080",
            "body" : "{\"msgtype\": \"text\", \"text\": { \"content\": \"An error has been found. Handle the issue immediately.\"}}"
          }
        }
      }
      }

      Notice

      • url specified in actions must contain the private IP address of your ECS instance that is deployed in the same region and VPC as your Elasticsearch cluster. You must also create a security group rule for the ECS instance by following the preceding procedure. Otherwise, the instance cannot connect to X-Pack Watcher.
      • If error No handler found for uri [/_xpack/watcher/watch/log_error_watch_2] and method [PUT] is returned when you run the preceding command, X-Pack Watcher is disabled for your Elasticsearch cluster. In this case, enable X-Pack Watcher and then run the command. For more information, see Modify the YML configuration.
      If you no longer need this watch, run the following command to delete the watch:

      DELETE _xpack/watcher/watch/log_error_watch