Application Real-Time Monitoring Service:Custom alert integration

Step 1: Create a custom integration

1. Log on to the ARMS console. In the left-side navigation pane, choose Alert Management > Integrations.

2. On the Alert Integration tab, click Custom Integration.
3. In the Create Custom Event Integration dialog box, enter an integration name, set the alert auto-recovery time, add a description as needed, and then click Save and Configure.
   Note

   Automatic recovery time after timeout: If an alert event is not triggered again within the specified period of time, the alert event is automatically cleared.

Step 2: Configure API endpoint

Send alerts from your third-party source to ARMS alert management using the provided API endpoint URL.

1. On the Edit Integration page, in the API Configuration section, copy the endpoint URL and API key.
2. From your third-party alert source, use the endpoint URL to report alert content.

   Example:

   curl -k -H "Content-Type: application/json" -d '{
     "trigger-type": "network",
     "trigger-location": "cn-hangzhou",
     "trigger-severity": "MAX",
     "trigger-policy": "package errors > 5%",
     "trigger-event": "inbound tcp package errors is 20%",
     "trigger-check": "tcp package error percentage",
     "trigger-value": "20",
     "trigger-time": "1629702922000",
     "metadata": [
       {
         "agent": "SERVER",
         "ip": "141.219.XX.XX",
         "fqdn": "websrv1.damenport.org",
         "microServiceId": "ms-login-2251",
         "accountId": "1504000433993",
         "service": "login-0"
       },
       {
         "agent": "CONTAINER",
         "ip": "172.1.XX.XX",
         "fqdn": "websrv2.damenport.org",
         "microServiceId": "ms-login-2252",
         "accountId": "129930302939",
         "service": "login-1"
       }
     ],
     "equipments": [
       {
         "equipmentId": "112"
       },
       {
         "equipmentId": "113"
       }
     ]
   }' "https://alerts.aliyuncs.com/api/v1/integrations/custom/ymQBN******"

   The first time you run the curl command, it is expected to return the following error:

   { "requestId":null,"traceId":"0bc3b4a********71e4aa7","code":601,"data":{},"success":false,"errorCode":"Invalid incident, labels.alertname is required","message":"Invalid incident, labels.alertname is required","httpStatusCode":601,"errorCodeParams":null,"dynamicMessage":null

   After you complete the mapping in Step 3 and run the command again, the correct data is returned.

   You can also report alerts by sending test data. The first time you do this, a message appears, prompting you to complete the mapping in Step 3.

   In the Send Test Data pop-up window, after you click Send for the first time, the page displays the message Upload successful, but no event was generated. Please configure event mapping rules based on the raw data! Follow this prompt to complete the mapping in Step 3.

Step 3: Configure alert field mapping

In the Event Mapping section, map the fields from the third-party alert source to the ARMS alert event fields.

This section uses an example to illustrate the process.

Scenario: A TCP package error occurred on the machine network.

Third-party alert content:

{
  "trigger-type": "network",    // Event type: network.
  "trigger-location": "cn-hangzhou",    // Region where the event occurred.
  "trigger-severity": "MAX",    // Custom event severity level, such as MAX, MID, or LOW.
  "trigger-policy": "package errors > 5%",     // Trigger policy.
  "trigger-event": "inbound tcp package errors is 20%",    // Event content: The inbound TCP package error rate for the service is 20%.
  "trigger-check": "tcp package error percentage",    // Event check item: TCP package error rate.
  "trigger-value": "20",    // Alert sample value.
  "trigger-time": "1629702922000",    // Alert start time.
  "metadata": [
    {
      "agent": "SERVER",    // Agent type: server.
      "ip": "141.219.XX.XX",   // IP address.
      "fqdn": "websrv1.damenport.org",    // Service domain name information.
      "microServiceId": "ms-login-2251",    // Microservice ID. If 'agent' is 'CONTAINER', this field is conditionally mapped to the ARMS 'service' field.
      "accountId": "1504000433993",    // User ID. Not included in the alert notification.
      "service": "login-0" // Service ID. If 'agent' is 'SERVER', this field is conditionally mapped to the ARMS 'service' field.
    },
    {
      "agent": "CONTAINER",   // Agent type: container.
      "ip": "172.1.XX.XX",  // IP address.
      "fqdn": "websrv2.damenport.org",    // Service domain name information.
      "microServiceId": "ms-login-2252",   // Microservice ID. If 'agent' is 'CONTAINER', this field is conditionally mapped to the ARMS 'service' field.
      "accountId": "129930302939",  // User ID. Not included in the alert notification.
      "service": "login-1"    // Service ID. If 'agent' is 'SERVER', this field is conditionally mapped to the ARMS 'service' field.
    }
  ],
  "equipments": [  // Another array node.
    {
      "equipmentId": "112"
    },
    {
      "equipmentId": "113"
    }
  ]
}

1. In the Event Mapping section, click Send Test Data.

2. In the Send Test Data dialog box, enter the alert content from your third-party source in JSON format, and then click Send.
   Note

   • If the message Uploaded. No events are generated. Configure mappings based on the original data. appears, the fields of the alert source are not mapped to the fields of ARMS alert events. The data that is sent is displayed in the left-side box. This allows you to select the source fields when you configure mappings.

   • If the message Uploaded. appears, the alert content is reported to the Alert Event History page. For more information, see View alert event history.

3. In the Send Test Data dialog box, click Disable.

4. In the left part of the Event Mapping section, click the data records for which you want to configure mappings to view the details.

5. In the right pane of the Event Mapping section, configure the mapping between the alert source fields and the ARMS alert fields.
   Note

   If you do not receive a recovery notification or recovery email after an alert is resolved, check the following:

   1. Confirm that the recovery event and the original alert event are sent through the same custom integration channel.

   2. Confirm that the original alert is still in the Triggered state — it has not been automatically recovered due to timeout or manually closed.

   3. Check that the alertname value in the recovery event exactly matches the alertname value of the original alert.

   1. Optional: In the Select Root Node section, specify whether to use batch processing.
      If your alert data contains an array, you can specify it as the root node for batch processing.

      After you select Use Batch Processing, select the array node that you want to process in a batch as the root node.

      Note If the alert data contains multiple array nodes, ARMS alert management allows you to select only one array node for batch processing.

      For example, in the sample alert, the metadata array node contains two service fields. If you select metadata as the root node, the two service fields under this node are mapped to the same specified ARMS alert field.

      • If you select a root node, you can iteratively map the values of the $.metadata[*].service attribute for all elements in the array to the ARMS service field.
      • If you do not select a root node, you can map the value of a specific element, such as $.metadata[0].service or $.metadata[1].service, to the ARMS service field.

   2. Optional:Select Configure Alert Recovery Events and configure field conditions for clearing alerts.

      After ARMS receives events, it searches for alerts that contain specified field values and clears these alerts. The field that you specify to clear alerts must be a field that is equivalent to the alert severity in the event. You cannot use the $.severity field to clear alerts. For example, if the field that you specify to clear alerts is {$.eventType ="resolved"}, the system automatically clears all alerts whose value of eventType is resolved in the integration.

   3. In the Map Source Fields to Target Fields section, map the alert source fields to the ARMS alert fields.
      Click the mapping icon to change the field mapping method.

      • Direct: The specified alert source field is directly mapped to the corresponding ARMS alert field.
      • Series: Combines multiple source fields into one, separated by a delimiter, and maps the result to an ARMS alert field. Only special characters are supported as delimiters.

        For example, you can use an underscore (_) to concatenate the $.trigger-type and $.trigger-policy fields into $.trigger-type_$.trigger-policy, and then map the result to the ARMS alertname field.

      • Condition: Maps a source field to the target ARMS field only when a specified condition is met.

        For example, for the agent field in the metadata array node, you can set the following condition: If the value of agent is CONTAINER, map the value of the microServiceId attribute to the ARMS service field (If condition). If the value of agent is SERVER, map the value of the service attribute to the ARMS service field (Else If condition).

      • mapping table: Set up a mapping between the alert severity levels of the alert source and the severity levels in ARMS. A mapping table is required only for the severity field.

      The following table describes the ARMS alert fields.

      Parameter	Description	Example mapping method	Example source field
      alertname	The name of the custom alert.	Series	$.trigger-type, $.trigger-policy
      severity	The field for mapping alert severity levels. You must configure a mapping table for this field, and the mapping method must be set to Direct.	Method: Direct (required) mapping table: MAX: P1 MID: P2 MIN: P3	$.trigger-severity
      message	Detailed information about the alert event, used in notifications. Maximum length: 15,000 characters.	Direct	$.trigger-event
      value	The sample value of a metric.	Direct	$.trigger-value
      imageUrl	The URL of a metric line chart, used to map Grafana metric charts.	None	None
      check	The check item of the alert. Examples: CPU, JVM, Application Crash, and Deployment.	Direct	$.trigger-check
      source	The source of the alert event.	Direct	$.metadata[*].ip
      class	The object type of the alert event, such as a host.	Direct	$.trigger-type
      service	The business-related source service, such as logService.	Condition	If $.metadata[*].agent is CONTAINER, map $.metadata[*].microServiceId to the ARMS service field. If $.metadata[*].agent is SERVER, map $.metadata[*].service to the ARMS service field.
      startat	The event start timestamp.	Direct	$.trigger-time
      endat	The event end timestamp.	None	None
      generatorUrl	The URL for the event details.	None	None

      Note

      If you use multi-condition alert mode (where a single alert rule includes multiple conditions), and you observe that the alert count prediction displays 0 or the chart shows data anomalies, check your field mappings. Make sure that both the alertname and severity fields are correctly mapped and that the event labels are complete. Incomplete or incorrect mappings in multi-condition alert mode can cause the alert count prediction to return 0 or chart data to display incorrectly.

Step 4: Configure event deduplication

To reduce redundant data, the system uses specified fields as the basis for event deduplication. ARMS alert management allows you to preview the deduplication grouping results for the historical event data in the Event Mapping section. You can adjust the deduplication fields as needed.

1. In the Event Deduplication section, select the fields to be used for deduplication.
   When multiple events have the same values for the selected fields, they are merged into a single alert notification.

   For example, set the ARMS source and check fields as deduplication keys. In the example scenario, the $.metadata[*].ip field from the third-party alert source is mapped to the ARMS source field, and the $.trigger-check field is mapped to the ARMS check field. As a result, events with the same IP address and check item are merged into a single event, while events with different IP addresses or check items remain separate.

2. Click Deduplication Test to preview the alert groups after deduplication.
   Note The deduplication test applies to the last 10 test data records uploaded in the left pane of the Event Mapping section.

   In the Select fields for deduplication area, select source and check, and then click Deduplication Test. The test results are grouped by the selected fields. For example, the events are split into two groups based on different source IP addresses (141.219.xxx and 172.1.xxx) and the same check item (tcp package error percentage). The alert events within each group have the same values for the source and check fields.
3. After you complete the configuration, click Save.

Results

After the configuration is complete, the Integrations page shows the custom integration that you created.

Manage integrations

In the left-side navigation pane, choose Alert Management > Integrations. On the Alert Integration tab, you can perform the following operations on the integrations that you created:

• View the details of an integration: Find the integration and then click the row. On the Integration Details page, view the integration details.

• Update the key: In the Actions column for the target integration, choose More > Update Key. In the dialog box that appears, click Confirm.
  Important After you update the key, you must also update the endpoint URL in the alert source as described in Step 2.

• Modify an integration: Find the integration and then click Edit in the Actions column. On the Integration Details page, modify the integration information and then click Save.

• Enable or disable an integration: Find the integration and then click Disable or Enable in the Actions column.

• Delete an integration: Find the integration and then click Delete in the Actions column. In the message that appears, click OK.

• Add an event processing flow to an integration: Find the integration and click Add Event Processing Flow in the Actions column. For more information, see Event processing flow.

• Create a notification policy: Find the integration for which you want to create a notification policy, and click More in the Actions column. In the list that appears, click Create Notification Policy. For more information, see Notification policies.