Learn about the available metrics for real-time synchronization task alerts and see a best practice example. - DataWorks

In DataWorks Data Integration, you can configure alert rules to monitor the status of real-time synchronization tasks and the real-time synchronization phase of full and incremental synchronization tasks. This topic describes the metrics available for these alert rules and provides a best-practice example.

Alert rule metrics

You can configure alert rules for real-time synchronization tasks in the DataWorks Operation Center. The available alert metrics include Status, Business delay, Failover, Not Supported by DDL Statements, DDL Notification, and Dirty Data. When you create a new rule, you must also configure parameters such as Name, Description, threshold, Alert Interval, WARNING, CRITICAL, and Recipients (Non-DingTalk).

Metric 1: Status

Description	Configuration recommendations
The Status metric uses a task's heartbeat to determine if it is functioning correctly.	Configure this alert for all critical real-time synchronization tasks. Set the alert threshold to 3 minutes or more to prevent false alarms from network jitter or other transient issues.

Metric 2: Business delay

Description	Configuration recommendations
The Business delay alert determines whether the synchronization rate can keep up with the data production rate at the source. This metric measures the time lag between data creation at the source and its processing by the task.	This is a core alert metric. We recommend configuring it for all critical real-time synchronization tasks. For tasks with a Kafka source, use the Accumulated Messages metric instead. Set the threshold based on your business requirements. We recommend a minute-level threshold to avoid false alarms caused by network jitter or temporary data spikes at the source.

Metric 3: Accumulated Messages (Kafka sources only)

Description	Configuration recommendations
This metric is available only for real-time synchronization tasks that use a Kafka source. It measures consumer lag by comparing the source offset in Kafka with the last consumed offset.	Configure this alert for all critical real-time synchronization tasks that use a Kafka source.

Metric 4: Failover

Description	Configuration recommendations
A Failover event occurs when a real-time synchronization process exits unexpectedly. To enhance process reliability, the Data Integration control service automatically restarts the process to attempt recovery.	If you need to track task failovers, configure this alert to be notified of frequent restarts, which may indicate an underlying stability issue.

Metric 5: Unsupported DDL statements (deprecated)

Description	Configuration recommendations
Data Integration real-time synchronization supports some DDL events from source systems. For unsupported DDL events, you can set a handling policy such as Ignore, Warning, or Critical on the DDL configuration page. When a DDL event with a Warning or Critical severity occurs, the Not Supported by DDL Statements metric triggers an alert.	This metric is deprecated. Use the DDL Notification metric instead, which includes this functionality.

Metric 6: DDL Notification

Description	Configuration recommendations
The DDL Notification metric allows you to monitor specific DDL types, independent of DDL handling policies. When a monitored DDL event occurs, it triggers an alert.	This metric includes the functionality of the Not Supported by DDL Statements alert. We recommend using this metric for all DDL-related monitoring and alerting scenarios.

Metric 7: Dirty Data

Description

Configuration recommendations

During real-time synchronization, data that fails to be written to the destination is classified as dirty data. Configure this metric to be alerted when dirty data is generated.

By default, real-time synchronization tasks do not tolerate dirty data, so you do not need to configure an alert for this metric. Configure it only if you change the dirty data policy to tolerate dirty data after restarting the task and you need to be alerted when dirty data occurs.
Changing the dirty data policy causes data that fails to be written to be discarded, leading to inconsistencies between the source and destination. We recommend not changing this policy unless you have a specific reason.

Alert notification methods

DataWorks supports several notification methods for real-time synchronization task alerts: email, text message, DingTalk, and webhook.

Email

DataWorks sends email alerts to the recipient's address configured on the Alert Contacts page in the DataWorks console. If no address is configured, DataWorks sends alerts to the email address of the associated Alibaba Cloud account.

Note

When checking for alert emails, make sure they have not been moved to your spam folder.

Text message

DataWorks sends text message alerts to the recipient's phone number configured on the Alert Contacts page in the DataWorks console. If no number is configured, DataWorks sends alerts to the phone number of the associated Alibaba Cloud account.

DingTalk

Alert notifications are sent as text messages to a specified DingTalk group through a custom DingTalk chatbot. Enter the chatbot token in the DingTalk Robot Token field. Use commas to separate multiple tokens. You can also enable the DingTalk Group Notification feature to ensure the alert is visible to all group members.

Important

After you add a custom DingTalk chatbot to a group, you must set a custom keyword. Other filtering methods are not supported. You must add DataWorks as a keyword (case-sensitive). Otherwise, the group will not receive alert notifications. For a step-by-step guide, see Use case: Send alert notifications to a DingTalk group.

Webhook

Alert notifications are sent to a specified web address. Enter the webhook URL of the target product in the Webhook URL field. Use commas to separate multiple webhooks.

Available edition: The webhook feature is available only in DataWorks Enterprise Edition.
Supported notification targets: WeCom and Lark.

Send alerts to a DingTalk group

Add a DingTalk chatbot

Note

The following steps may vary slightly depending on your version of DingTalk.

Open the target DingTalk group for alerts, and click the Group Settings icon in the upper-right corner.
Click Bot.
On the Bot page, click Add Robot.
On the Robot Management page, click Add Robot.
On the Please choose which robot to add page, click Custom.
In the Custom dialog box, click Add.

In the Add Robot dialog box, configure the following parameters.

Parameter	Description
Chatbot name	The display name of the custom chatbot.
Add to Group	The DingTalk group to which the chatbot is added. This cannot be changed after configuration.
Custom Keyword	After you set keywords, the chatbot will only post messages that contain at least one of the specified keywords. You must add DataWorks as a case-sensitive keyword. Note You can set up to 10 keywords. A message must contain at least one of the keywords to be sent successfully.

Select I have read and accepted the <<DingTalk Custom Robot Terms of Service>> and click Completed.
After you complete the security settings, copy the chatbot Webhook and click Completed.

Important
Keep the webhook URL confidential to avoid security risks.

Create an alert rule

Go to the alert settings page for the real-time synchronization task.
Log on to the DataWorks console. In Operation Center, on the Real-time Synchronization Task page, find the synchronization task for which you want to configure an alert, and click Alert Settings in the Actions column.
- The Alert Events tab shows triggered alarm events.
- The Alert Rules tab shows existing alert rules and allows you to create new ones.
Click the Alert Rules tab and then click Create Rule.
Configure the alert rule.
Follow the on-screen instructions to configure the rule name, description, and other parameters. This use case sends alerts to a DingTalk group. The key parameters are as follows.
- WARNING and CRITICAL: Select DingTalk.
- DingTalk Robot Token: Enter the token of the DingTalk chatbot.
- DingTalk Group Notification: Select the Enable checkbox.
Important
- The alert rule is enabled automatically after configuration.
- In production, we recommend configuring the following two alert metrics:
  
  Use the Task Status metric to monitor whether the task is running properly.
  
  Use the Business Latency or Accumulated Messages metric to monitor whether the synchronization throughput can keep up with the data production rate at the source.

Verify the alert rule configuration

After you add an alert rule, you can click Test on the alert rule page to verify that the configuration is correct.