Application Real-Time Monitoring Service:Integrate a custom alert source with ARMS

Step 1: Create a custom integration

1. Log on to the ARMS console. In the left-side navigation pane, choose Alert Management > Integrations.

2. On the Alert Integration tab, click Custom Integration.

3. In the Create Custom Event Integration dialog box, enter a name for the integration, specify the period of time after which alerts are cleared, enter a description based on your business requirements, and then click Save and Configure.

   Note If an alert event is not triggered again within the specified period of time, the alert event is automatically cleared.

Step 2: Configure the API endpoint

You can configure an API endpoint to enable third-party alert sources to report alert events to ARMS.

1. In the Span Configuration section on the Integration Details page, copy the API endpoint and API key.

   

2. Access the API endpoint from a third-party alert source to report alert events.

   The following section shows an example of a reported alert event:

   curl -k -H "Content-Type: application/json" -d '{
     "trigger-type": "network",
     "trigger-location": "cn-hangzhou",
     "trigger-severity": "MAX",
     "trigger-policy": "package errors > 5%",
     "trigger-event": "inbound tcp package errors is 20%",
     "trigger-check": "tcp package error percentage",
     "trigger-value": "20",
     "trigger-time": "1629702922000",
     "metadata": [
       {
         "agent": "SERVER",
         "ip": "141.219.XX.XX",
         "fqdn": "websrv1.damenport.org",
         "microServiceId": "ms-login-2251",
         "accountId": "1504000433993",
         "service": "login-0"
       },
       {
         "agent": "CONTAINER",
         "ip": "172.1.XX.XX",
         "fqdn": "websrv2.damenport.org",
         "microServiceId": "ms-login-2252",
         "accountId": "129930302939",
         "service": "login-1"
       }
     ],
     "equipments": [
       {
         "equipmentId": "112"
       },
       {
         "equipmentId": "113"
       }
     ]
   }' "https://alerts.aliyuncs.com/api/v1/integrations/custom/ymQBN******"

   If you run the curl command to report alert events for the first time, the following error message is returned:

   { "requestId":null,"traceId":"0bc3b4a********71e4aa7","code":601,"data":{},"success":false,"errorCode":"Invalidincident,labels.alertnameisrequired","message":"Invalidincident,labels.alertnameisrequired","httpStatusCode":601,"errorCodeParams":null,"dynamicMessage":null

   You must configure alert field mappings first. Then, data is returned as expected. For more information, see Step 3.

   You can also use test data to report alert events. The first time that you use test data to report alert events, the following message appears. You need to configure alert field mappings as prompted. For more information, see Step 3.

Step 3: Configure alert field mappings

In the Event Mapping section, configure field mappings between the third-party alert source and ARMS.

The following scenario is used as an example:

Scenario: TCP packet errors occur on a server.

The following section describes the alert event reported by the third-party alert source:

{
  "trigger-type": "network",    //The type of the event: network. 
  "trigger-location": "cn-hangzhou",    //The region where the event occurs. 
  "trigger-severity": "MAX",    //The severity level of the event, such as MAX, MID, and LOW. 
  "trigger-policy": "package errors > 5%",     //The trigger policy. 
  "trigger-event": "inbound tcp package errors is 20%",    //The content of the event: The error rate of inbound TCP packets is 20%. 
  "trigger-check": "tcp package error percentage",    //The check item of the event: error rate of TCP packets. 
  "trigger-value": "20",    //The sample value of the alert event. 
  "trigger-time": "1629702922000",    //The start time of the alert event. 
  "metadata": [
    {
      "agent": "SERVER",    //The type of the agent: SERVER. 
      "ip": "141.219.XX.XX",   // The IP address. 
      "fqdn": "websrv1.damenport.org",    //The domain name. 
      "microServiceId": "ms-login-2251",    //The microservice ID. If the value of the agent field is CONTAINER, this field is mapped to the service field. 
      "accountId": "1504000433993",    //The user ID. The user ID is not included in an alert notification. 
      "service": "login-0" //The service ID. If the value of the agent field is SERVER, this field is mapped to the service field. 
    },
    {
      "agent": "CONTAINER",   //The type of the agent: CONTAINER. 
      "ip": "172.1.XX.XX",  // The IP address. 
      "fqdn": "websrv2.damenport.org",    //The domain name. 
      "microServiceId": "ms-login-2252",   //The microservice ID. If the value of the agent field is CONTAINER, this field is mapped to the service field. 
      "accountId": "129930302939",  //The user ID. The user ID is not included in an alert notification. 
      "service": "login-1"    //The service ID. If the value of the agent field is SERVER, this field is mapped to the service field. 
    }
  ],
  "equipments": [  //Another array node. 
    {
      "equipmentId": "112"
    },
    {
      "equipmentId": "113"
    }
  ]
}

1. In the Event Mapping section, click Send Test Data.

2. In the Send Test Data dialog box, enter the alert content of a third-party alert source in the JSON format and click Send.

   Note

   • If the message Uploaded. No events are generated. Configure mappings based on the original data. appears, the fields of the alert source are not mapped to the fields of ARMS alert events. The data that is sent is displayed in the left-side box. This allows you to select the source fields when you configure mappings.
   • If the message Uploaded. appears, the alert content is reported to the Alert Event History page. For more information, see View historical alert events.

   
3. In the Send Test Data dialog box, click Disable.
4. In the left part of the Event Mapping section, click the data records for which you want to configure mappings to view the details.

5. In the right part of the Event Mapping section, configure field mappings between the alert source and ARMS.

   1. Optional:In the Select Root Node section, specify whether to enable batch processing.

      If an array node exists in the alert data, you can specify the array node as the root node. The data that belongs to the root node is processed in a batch.

      After you select Use Batch Processing, select the array node to be processed as the root node.

      Note

      If multiple array nodes exist in the alert data, you can select only one of the array nodes for batch processing.

      In the preceding example, the metadata array node contains two service fields. If you select metadata as the root node, the values of the service fields are mapped to the specified alert field of ARMS.

      • If you select an array node as the root node, the values of all $.metadata[*].service fields in the array node are iteratively mapped to the service field of ARMS.

      • If you do not select an array node as the root node, you can choose to iteratively map the value of $.metadata[0].service or $.metadata[1].service in the array node to the service field of ARMS.

   2. Optional:Select Configure Alert Recovery Events and configure field conditions for clearing alerts.
      After ARMS receives events, it searches for alerts that contain specified field values and clears these alerts. The field that you specify to clear alerts must be a field that is equivalent to the alert severity in the event. You cannot use the $.severity field to clear alerts. For example, if the field that you specify to clear alerts is {$.eventType ="resolved"}, the system automatically clears all alerts whose value of eventType is resolved in the integration.

   3. In the Map Source Fields to Target Fields section, map the fields of the alert source to the alert fields of ARMS.

      Click the Map icon to change the field mapping method.

      • Direct: The specified field of the alert source is mapped to the specified alert field of ARMS.

      • Series: You can use delimiters to concatenate multiple fields of the alert source into one field, and then map this field to the specified alert field of ARMS. Only special characters can be used as delimiters.

        For example, you can use an underscore (_) to concatenate the $.trigger-type and $.trigger-policy fields to the $.trigger-type_$.trigger-policy field and then map the field to the alertname field of ARMS.

      • Condition: If the value of the specified field of the alert source meets the specified conditions, the field is mapped to the specified alert field of ARMS.

        For example, you can configure the following conditional mappings: If the value of the agent field in the metadata array node is CONTAINER, map the value of the microServiceId field to the service field of ARMS. If the value of the agent field is SERVER, map the value of the service field in the metadata array node to the service field of ARMS.

        

      • Mapping table: You must configure a mapping table that maps the severity levels of the alert source to the severity levels of ARMS alerts. You need to configure a mapping table only for the severity field.

      The following table describes the alert fields of ARMS.

      Alert field	Description	Example of mapping method	Example of alert source field
      alertname	The name of the custom alert.	Series	$.trigger-type and $.trigger-policy
      severity	The alert level. You must configure mappings for this field. The mapping method must be set to Direct.	Mapping method: Direct (required) Mapping table: MAX: P1 MID: P2 MIN: P3	$.trigger-severity
      message	The description of the alert. The description is used as the content of the alert message. The description cannot exceed 15,000 characters in length.	Direct	$.trigger-event
      value	The sample value of a metric.	Direct	$.trigger-value
      imageUrl	The URL of the line chart that contains Grafana metrics. The URL is used to map the line chart.	None	None
      check	The check item of the alert. Examples: CPU, JVM, Application Crash, and Deployment.	Direct	$.trigger-check
      source	The source of the alert.	Direct	$.metadata[*].ip
      class	The type of the object that triggers the alert event, for example, host.	Direct	$.trigger-type
      service	The source service of the alert. Example: Login Service.	Condition	If the value of the $.metadata[*].agent field is CONTAINER, the value of the $.metadata[*].microServiceId field is mapped to the service field of ARMS. If the value of the $.metadata[*].agent field is SERVER, the value of the $.metadata[*].service field is mapped to the service field of ARMS.
      startat	The timestamp that represents the start time of the event.	Direct	$.trigger-time
      endat	The timestamp that represents the end time of the event.	None	None
      Event URL (generatorUrl)	The URL of the event details.	None	None

      

Step 4: Configure deduplication

To reduce duplicate data, the system uses relevant fields as the basis for deduplication. ARMS Alert Management allows you to preview the deduplication grouping results of historical event data that is displayed in the Event Mapping section. You can change the fields whose values you want to deduplicate.

1. In the Event Deduplication section, select the fields whose values you want to deduplicate.

   If multiple events have the same value for a specified field, the events are merged into one alert notification.

   In this example, the $.metadata[*].ip field is mapped to the source field, and the $.trigger-check field is mapped to the check field. You can specify source and check as the fields that are used to deduplicate events. This way, events that have the same IP address and check item are merged into one event. Events that have different IP addresses or check items are not merged.

2. Click Deduplication Test to preview the alert group after deduplication.

   Note

   The deduplication test takes effect only on the latest 10 data records that are uploaded in the left part of the Event Mapping section.

   

3. Click Save.

Verify the result

After you complete the preceding operations, you can view the integration that you created on the Integrations page.

Manage the custom integration

In the left-side navigation pane, choose Alert Management > Integrations. On the Alert Integration tab, you can perform the following operations on the integrations that you created:

• View the details of an integration: Find the integration and then click the row. On the Integration Details page, view the integration details.

• Update a key: Find the integration and then choose More > Update Key in the Actions column. In the message that appears, click OK.

  Important

  After you update the key, modify the endpoint in the alert source that is specified in Step 2.

• Modify an integration: Find the integration and then click Edit in the Actions column. On the Integration Details page, modify the integration information and then click Save.
• Enable or disable an integration: Find the integration and then click Disable or Enable in the Actions column.
• Delete an integration: Find the integration and then click Delete in the Actions column. In the message that appears, click OK.
• Add an event processing flow to an integration: Find the integration and click Add Event Processing Flow in the Actions column. For more information, see Work with event processing flows.
• Create a notification policy: Find the integration for which you want to create a notification policy, and click More in the Actions column. In the list that appears, click Create Notification Policy. For more information, see Create and manage a notification policy.