E-MapReduce (EMR) Serverless Spark integrates with CloudMonitor, letting you subscribe to important events and configure custom alert notifications. This provides real-time updates on event status and progress to accelerate analysis and issue resolution.
Background information
CloudMonitor is a service that monitors Alibaba Cloud resources and internet applications. It offers a unified portal for managing and monitoring system events from various cloud services, providing status updates and alert notifications. For more information, see What is CloudMonitor?.
You can configure alert rules for all system events in CloudMonitor to receive timely notifications.
-
Send alerts by text message, email, DingTalk chatbot, and more.
-
Distribute events to your Message Queue, Log Service, Function Compute, or a webhook for automated handling.
Event subscription types
The following types of event subscriptions are supported.
|
Event type |
Use cases |
Monitoring granularity |
|
EMRSpark:JobRun:FAILED |
|
Task level |
|
EMRSpark:JobRun:TIMEOUT |
Monitor tasks that do not complete within a preset time. A task timeout triggers an alert. |
Task level |
|
EMRSpark:Workflow:TaskStateChange |
Monitor state changes of workflow nodes. An alert is triggered on each state change. |
Workflow node level |
For more information about system events, go to the CloudMonitor console and search for events whose names contain "EMRSpark".
Create a subscription policy
This topic provides an example of how to create a subscription policy. In this example, you subscribe to the EMRSpark:Workflow:TaskStateChange event and receive a notification when a workflow node fails. This ensures prompt detection and handling of node failures.
Workflow state change events may not appear immediately in the CloudMonitor console. A delay of a few minutes is normal.
-
Obtain the event resource ID.
-
Log on to the Cloud Monitor console.
-
In the navigation pane, choose .
-
On the Event Monitoring tab, select E-MapReduce for Product, select EMRSpark:Workflow:TaskStateChange for EMRSpark:Workflow:TaskStateChange, and then click Search.
-
In the list of filtered events, find the resource ID of your target event.
The resource ID is in the format:
acs:emr:cn-hangzhou:<your Alibaba Cloud account ID>:resourceId/workspaceId/<workspaceId>#workflowDefinitionName/<workflowDefinitionName>#taskDefinitionName/<taskDefinitionName>. You can also construct the resource ID using this format.
-
-
Subscribe to event notifications.
-
In the left-side navigation pane, choose Event Center > Event Subscription.
NoteYou can also create a subscription policy from the System Event menu as follows:
-
In the left-side navigation pane, choose Event Center > System Event.
-
In the Welcome to the New Event Center section, click Create Now.
-
-
On the Subscription Policy tab, click Create Subscription Policy.
-
On the Create Subscription Policy page, configure the following parameters.
-
Name: Enter a name for the subscription policy.
-
Subscription Type: Select System Event.
-
Subscription Scope:
-
Product: Select E-MapReduce.
-
Event Name: Select EMRSpark:Workflow:TaskStateChange.
-
Event Content: Enter toState: FAILED.
The subscription policy is triggered only when the task state changes to FAILED. The following table lists the supported states.
-
Event Resources: Enter the resource ID from the previous step.
-
Event Type, Event Level, and application group: Leave these parameters unconfigured.
-
-
Deduplication: Use the default settings.
-
Notification: Create a notification configuration and use the default Custom Notification Method.
To create a notification configuration, enter a name, set Notification Settings to Set notification group directly, select an alert contact group, and then click OK.
NoteFor more information about how to create a notification configuration, see Create a notification policy.
CloudMonitor sends notifications using the methods specified for contacts in the alert contact group. For example, if a contact's profile includes a mobile number and email address, and you use the default Custom Notification Method, that contact receives alerts by phone call, text message, and email only.
-
Push and Integration: Leave this section unconfigured.
NoteFor more information about how to create a push channel, see Create a push channel.
-
-
Click Submit.
-
Debug an event subscription
After you create a subscription policy, you can use the debug feature to verify that it triggers notifications correctly.
-
On the Subscription Policy tab, click Debug Event Subscription.
-
In the Create Event Debugging panel, select E-MapReduce for Product and EMRSpark:Workflow:TaskStateChange for Name.
The system automatically generates the debugging content in JSON format.
-
Click OK.
The system displays an Operation successful message, and CloudMonitor automatically sends a test alert to the specified contacts based on the subscription policy.