Elasticsearch:Configure a DingTalk robot to receive X-Pack Watcher alerts

Step 1: Create and configure a DingTalk robot

1. Create a DingTalk group to receive alerts.

2. In the upper-right corner of the group, click the icon. On the Group Settings panel, click Bot.

3. In the Robot Management dialog box, click Add a robot, select Custom, and click Add.

4. Enter a Robot Name, select Custom Keywords, and specify the keywords.

   Important

   The keywords must be included in the alert message that you set.

5. Select the "Custom robot services and disclaimers" and click Finished.

6. To copy the robot's webhook URL, click Copy next to Webhook.

   Keep the webhook URL secure for future use. Do not post it on external websites. Leaking the webhook URL poses a security risk.

Step 2: Configure an Nginx proxy and an ECS security group

1. Configure an Nginx proxy on the ECS instance.

   The Nginx proxy forwards alert messages. The X-Pack Watcher service of Elasticsearch sends alert messages to Nginx, which then forwards the messages to DingTalk or WeCom.

   1. Install Nginx on the ECS instance.

   2. Configure the nginx.conf file.

      Replace the configuration in the server section of the nginx.conf file with the following configuration.

      server
        {
          listen 8080; # Listener port
          server_name localhost; # Domain name
          index index.html index.htm index.php;
          root /usr/local/webserver/nginx/html; # Site folder
            location ~ .*\.(php|php5)?$
          {
            #fastcgi_pass unix:/tmp/php-cgi.sock;
            fastcgi_pass 127.0.0.1:9000;
            fastcgi_index index.php;
            include fastcgi.conf;
          }
          location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|ico)$
          {
            expires 30d;
            # access_log off;
          }
          location / {
            proxy_pass <Webhook URL>;
          }
          location ~ .*\.(js|css)?$
          {
            expires 15d;
            # access_log off;
          }
          access_log off;
        }

      Replace <Webhook URL> with the webhook URL of the DingTalk robot that receives alert messages.

   3. Load the modified configuration file and restart Nginx.

      /usr/local/webserver/nginx/sbin/nginx -s reload            # Reload the configuration file
      /usr/local/webserver/nginx/sbin/nginx -s reopen            # Restart Nginx

2. Configure the ECS security group.

   Allow the Nginx proxy on the ECS instance to receive alert messages from the Elasticsearch instance.

   1. Log on to the Alibaba Cloud ECS console.

   2. In the navigation pane on the left, choose Instances & Images > Instances.

   3. On the Instance page, click the name of the target instance.

   4. Click the Security Groups tab.

   5. On the Security Groups tab, click the name of the security group.

   6. On the Inbound tab, click Add Rule.

   7. Enter the required parameters.

      Parameter	Description
      Action	Select Allow.
      Priority	Keep the default value.
      Protocol	Select Custom TCP.
      Source	Add the IP addresses of all nodes in your Alibaba Cloud Elasticsearch instance. Note See View basic information about nodes to obtain the IP addresses of all nodes in the Elasticsearch instance.
      Destination (Current Instance)	Enter a commonly used port. This port is required when you configure Nginx. This topic uses port 8080 as an example.
      Description	Enter a description for the rule.

   8. Click OK.

Step 3: Configure a Watcher alert

1. Log on to the Kibana console of your Elasticsearch cluster and go to the homepage of the Kibana console as prompted.

   For more information about how to log on to the Kibana console, see Log on to the Kibana console.

   Note

   In this example, an Elasticsearch V6.7.0 cluster is used. Operations on clusters of other versions may differ. The actual operations in the console prevail.

2. In the left-side navigation pane of the page that appears, click Dev Tools.

3. In the Console, run the following command to create an alert document.

   The following example shows how to create a log_error_watch document. The system queries the logs index for error logs every 10 s. If more than 0 error logs are found, an alert is triggered.

   PUT _xpack/watcher/watch/log_error_watch
   {
     "trigger": {
       "schedule": {
         "interval": "10s"
       }
     },
     "input": {
       "search": {
         "request": {
           "indices": ["logs"],
           "body": {
             "query": {
               "match": {
                 "message": "error"
               }
             }
           }
         }
       }
     },
     "condition": {
       "compare": {
         "ctx.payload.hits.total": {
           "gt": 0
         }
       }
     },
     "actions" : {
     "test_issue" : {
       "webhook" : {
         "method" : "POST",
         "url" : "http://<yourAddress>:8080",
         "body" : "{\"msgtype\": \"text\", \"text\": { \"content\": \"An error log has appeared. Please handle it as soon as possible.\"}}"
       }
     }
   }
   }

   Table 1. Key parameter descriptions

   Parameter	Network type	Configuration description
   url	New network architecture	Set this parameter to the endpoint domain name. Requests are forwarded through the endpoint domain name. To obtain the endpoint domain name, see Configure private network peering for Alibaba Cloud Elasticsearch.
   	Old network architecture	Set this parameter to one of the following addresses: The IP address of the Nginx proxy: Requests are forwarded over the internet by the Nginx proxy in the same VPC. The webhook URL of the DingTalk robot.
   body	New network architecture Old network architecture	Configure this parameter based on the security settings of the DingTalk robot. For example, in Step 1: Create and configure a DingTalk robot, if you set Security Settings to Custom Keywords and add a custom keyword: error, the body must contain the content field that includes error for the DingTalk robot to push an alert message.

   Note

   If the No handler found for uri [/_xpack/watcher/watch/log_error_watch_2] and method [PUT] error occurs when you run the command, this indicates that the X-Pack Watcher feature is not enabled for your Alibaba Cloud Elasticsearch instance. Enable the feature and run the command again. For more information, see Configure YML parameters.

Step 4: View the alert result

If the configuration is correct, when the data in the cluster meets the alert conditions configured in Step 3: Configure a Watcher alert, you will receive an alert message such as An error log has appeared. Please handle it as soon as possible. in your DingTalk group.

DELETE _xpack/watcher/watch/log_error_watch