You can add monitored objects on the rule list page and perform operations such as viewing, editing, and deleting. This topic describes how to add and manage data table quality rules.
Limits
You can select a maximum of 200 monitored objects for batch addition at one time.
Only Dataphin tables and global tables support the configuration of quality score weights.
The quality score weight of monitored objects is used to calculate quality scores from perspectives such as project, data source, and individual.
Permission description
Super administrators and quality administrators can add monitored objects.
Only quality rules under monitored objects with view permissions can be exported.
Operation permissions vary for different objects. For more information, see Quality Rule Operation Permissions.
View quality rules from different perspectives
Dataphin supports viewing data table quality rules from three perspectives: Global, Project, and Individual.
Global: Super administrators, quality administrators, and custom global roles with Quality Rule-Management permissions can view all data table quality rules globally.
Project: Super administrators and quality administrators can switch to view data table quality rules for all projects. Project members can only select and view data table quality rules for projects they have joined.
Individual: Super administrators and quality administrators can switch to view data table quality rules for any member. Regular members can only view Dataphin data table quality rules for which they are the quality owner and cannot switch.
If permission restrictions are enabled in quality rules, the permission restrictions of the quality rules take precedence.
When viewing quality rules from the project perspective, only monitored objects and quality rules for the specified project are displayed.
When viewing quality rules from the individual perspective, only monitored objects and quality rules for which you are the quality owner are displayed.
Add monitored objects
On the Dataphin home page, select Administration > Data Quality from the top menu bar.
Click Quality Rule in the left-side navigation pane. On the quality rule page, click Add Monitored Object in the upper-right corner.
In the Add Monitored Object dialog box, filter monitored objects based on different conditions.
Dataphin Table: Filter data tables based on table type, environment, and associated project (for physical tables) or associated section (for logical tables).
Global Data Table: Filter data tables based on data source type and data source. For supported data sources, see Dataphin Supported Data Sources.
Metrics: Filter metrics based on data section and logical aggregate table.
Data Source: Filter data sources based on data source type.
Real-time Meta Table: Filter data tables based on environment and associated project.
Click Configure Quality Rule to enter the quality rule configuration page.
Manage monitored object list
On the Quality Rule list page, you can view the table name, validation switch status, the number of effective/total quality rules, and other information for configured monitored objects. You can also perform operations such as viewing, running, managing the quality owner, and deleting. The display is consistent across different monitored objects. The following example uses Dataphin Table.
Operation Item | Description |
① Single Operation Item | |
View Quality Rule Details | You can view the configuration details of data table quality rules. |
Run | You can run data tables based on the configured schedule or custom partition. |
View Quality Report | You can view the quality report corresponding to the data table. |
Manage Quality Owner | Enter the quality owner to be transferred. You can select up to 20. |
Edit Quality Score Weight | Support modifying the quality score weight. Important
|
Delete | Deleting this quality rule object will delete all quality rules under the object. Deletion is irreversible. |
② Batch Operation Item | |
Run | You can batch run data tables based on the configured schedule or custom partition. For non-partitioned tables, you can choose to validate the entire table.
Note
|
Batch Manage Quality Owner | Support batch Append or Modify quality owner.
|
Export Rules | Export custom SQL quality rules with view permissions under the selected monitored objects on the current page. |
Enable/Disable Quality Rule Validation | You can batch enable or disable quality rule validation. |
Edit Quality Score Weight | Support batch modification of quality score weight. Important
|
Alert Configuration | You can batch select quality rules under different monitored objects to configure alert methods. If the selected monitored objects already have alerts configured for the same monitoring range, the latest configuration will overwrite the existing configuration. If there is no alert configuration for the same monitoring range, a new alert configuration will be added. When configuring alerts in batches, the monitoring range only supports selecting: all rules, all strong rules, or all soft rules. The configuration details are the same as for a single object alert configuration. For more information, see Alert Configuration. Note
|
Permission Management | You can specify members to view validation record details, quality rule details, and quality reports. In the Quality Rule Permission Management dialog box, select members who can view details and click OK.
|
Delete | You can batch delete this quality rule object, which will delete all quality rules under the object. Deletion is irreversible. |
What to do next
After adding monitored objects, you can configure quality specifications for monitoring. For more information, see