Timeliness and accuracy are the core quality indicators for data business output. Quality rules monitoring ensures content integrity, while baseline monitoring safeguards timeliness, tailored to the data business's significance. Baseline monitoring automatically identifies and tracks tasks requiring monitoring based on scheduling dependencies. It issues alerts if there's a risk of delay to the data of the protected object, according to predefined warning and commitment times. Tasks within the baseline may be prioritized to ensure preferential resource allocation. This topic describes the steps to configure baseline monitoring.
Prerequisites
You need to subscribe to the artificial intelligence for IT operations value-added service and activate the artificial intelligence for IT operations module for the current tenant.
Required permissions
Only superusers, system administrators, or accounts with baseline monitoring configuration permissions are authorized to create baseline tasks.
Once a baseline task is established, its ownership can be transferred to other members, who then gain the ability to edit or delete the baseline.
Procedure
In the top menu bar of the Dataphin homepage, select Development > Task O&M.
In the left-side navigation pane, select Monitoring Management > Baseline Monitoring.
On the Baseline Monitoring page, click New Baseline. On the New Baseline page, configure the following parameters.
Basic Information
Parameter
Description
Baseline Name
Name the baseline task, ideally reflecting the monitored business scenario, with a maximum of 128 characters.
Owner
The owner can be any member within the current tenant.
Priority
You can select High or Highest.
NoteTasks assigned to the baseline adhere to its established priority, which influences the sequence of resource allocation in the scheduling process.
Baseline Type
At present, only the Daily Baseline feature is supported.
Description
Provide a brief description of the baseline, not exceeding 512 characters.
Protected Object
Parameter
Description
Protected Object List
The list displays current baseline's protected objects and includes management operations. Protected objects encompass Protected Object (name), Protection Details, Recurrence, Scheduled/expected Output Time, Belonging Project/section, Owner, and available actions.
Protection Details: Displays the number of protected fields and their details for logical tables.
Recurrence: Includes options for minute, hour, day, week, month, and year.
Scheduled/expected Output Time: For logical tables, it displays the earliest scheduled start time and the latest expected output time for selected fields. If the selected object has no historical records, the expected completion time cannot be calculated. The output time corresponds to the system time zone.
Click the
icon to view each logical table field's scheduled/expected output time.Belonging Project/section: Specifies the project for tasks or detail tables, and the section for aggregate tables.
Owner: The owner of the protected object.
Actions: Allows for the deletion of the current protected object.
+Add Protected Object
Click Add Protected Object in the protected object area to select Add Physical Task or Logical Table Fields as protected objects.
NoteEach baseline can include up to 50 protected objects.
A logical table is considered a single protection node, with the option to select multiple fields.
Add Physical Task: In the Add Physical Task dialog box, you can add physical tasks that need protection based on your business requirements. You can filter and search by project, node name, or ID.
Add Logical Table Fields: In the Add Logical Table Fields dialog box, select a logical table and choose the fields that meet your requirements. You can filter and search logical tables by data section or logical table name keyword, and search logical table fields by field name keyword.
Alert Configuration
Parameter
Description
Baseline Alert
Historical Expected Output Time: The system estimates this based on the successful execution records of the selected protected objects over the past 7 days. The output time corresponds to the system time zone.
NoteHourly tasks do not support this calculation, and it may be inaccurate when data is insufficient. This is for reference only.
Protection Output Time: You can select either Today or Next Day, and you need to select a time point.
When the system time zone (the time zone in the User Center) is different from the scheduling time zone, the system displays both time zones. After selecting a time, the system automatically calculates the corresponding scheduling time zone time. When the calculated scheduling time zone time is not on the same day as the system time zone time,
(-1)indicates the previous day and(+1)indicates the next day. For example, if the system time zone is GMT+08:00 and the scheduling time zone is GMT+01:00, and the specified time period is configured as00:00 ~ 01:00, the corresponding scheduling time zone time is17:00(-1) ~ 18:00(-1).Warning Margin: The default value is 0, with an allowable range of integers from 0 to 180.
NoteSetting the margin to 0 means no baseline warning will be triggered.
Alert Frequency: The default is once per minute, with an adjustable range of 1 to 59 minutes.
Alert Interval: You can select All Day or Specified Time Period, with the default being All Day. When you select a specified time period, you need to configure the start time. The time zone for the start time follows the same explanation as the Protection Output Time time zone.
Maximum Alert Times: The default is once, with an adjustable range from 1 to 10 times.
Receive Configuration: Click Add Recipient to set up the recipient and method of receipt.
Recipient Type: Options include Owner, Duty Schedule, or Customize.
Receiving Method: Supports phone, text message, email, and DingTalk as methods of notification.
NoteA maximum of three recipients can be configured.
Event Alert
Event switch: Configure event alerts, which are enabled by default.
If an upstream task of the protected task encounters failure or delays, it will trigger an event alert. This could prevent the protected task from completing by the committed time, yet it will not impact the configured task alerts.
Event Alert Type:
Error: An alert is issued if a node fails within the baseline link.
Slowdown: An alert for slowdown is triggered if the current runtime exceeds the historical average, calculated from the last seven successful executions, by 30% or more.
Alert And Receive Configuration:
Same As Baseline Alert Configuration: By default, the configuration includes settings for alert frequency, maximum number of alerts, alert intervals, and notification preferences.
Custom Configuration: Allows you to tailor settings for alert frequency, maximum number of alerts, alert intervals, and notification preferences.
Click OK to finalize the creation of the baseline task.