Dataphin allows you to create quality rules for tables, simplifying table quality monitoring. This topic explains how to configure global quality rules for tables.
Prerequisites
You must add a monitored object before you can configure a quality rule. For more information, see Add a monitored object or .
Permissions
Super administrators, quality administrators, and custom global roles with the Quality Rule-Management permission can configure scheduling, alerts, exception archive tables, scoring weights, runtime configurations, and more for quality rules.
Quality owners can configure scheduling, alerts, exception archiving tables, scoring weights, and runtime configurations for quality rules on their monitoring objects.
Quality owners and standard users also need the synchronous read permission for the data source containing the global data table. To apply for this permission, see Apply for data source permission.
The supported operation permissions vary by object. For details, see Operation permissions for quality rules.
Validation Rules
When a table is validated against quality rules, if a weak monitoring rule is triggered, the system sends an alert so you can promptly address the anomaly. If a strong monitoring rule is triggered, the system automatically stops the associated task to prevent dirty data from flowing downstream and sends an alert.
Dry Run VS. Run
A dry run and a run differ in their execution and results. A dry run is a simulated execution of a quality rule to verify its behavior; its results do not appear in the quality report. A run checks a quality rule during a specified time window and outputs the results to the quality report.
Configure Quality Rules
On the Dataphin homepage, in the top menu bar, choose Governance > Data Quality.
In the left-side navigation pane, click Quality Rules. On the Global Data Table page, click the name of the target object to open the Quality Rule Details page and configure the quality rule.
On the Quality Rule Details page, click Create Quality Rule.
In the Create Quality Rule dialog box, configure the parameters.
Parameter
Description
Basic information
Rule name
A custom name for the quality rule. The name can be up to 256 characters long.
Rule strength
You can select weak rule or strong rule.
Weak rule: If you select weak rule, an alert is triggered when the validation result is abnormal, but downstream tasks are not blocked.
Strong rule: If you select strong rule, an alert is triggered when the validation result is abnormal. If downstream tasks exist (such as for code check scheduling or task-triggered scheduling), they are blocked to prevent the spread of corrupted data. If no downstream tasks exist (such as in periodic quality scheduling), only an alert is triggered.
Description
A custom description for the quality rule, up to 128 characters long.
Configuration method
Create from template: Quickly create a quality rule by using a general system template or a custom business template.
System template: The built-in parameters of the template are configurable. This is suitable for creating general-purpose rules.
Custom template: The parameters are preset in the template and do not require configuration. This is typically used for creating rules that contain business logic.
Custom SQL: Use SQL to flexibly define data quality monitoring rules. This method is suitable for complex scenarios.
Rule template
Select a rule template from the drop-down list. Options include Completeness, Uniqueness, Timeliness, Validity, Consistency, Stability, and Custom SQL.
Completeness: Includes field null value validation and field empty string validation.
Uniqueness: Includes field uniqueness validation, field group count validation, and duplicate field value count validation.
Timeliness: Includes time function comparison, time field comparison within a single table, and time field comparison between two tables.
Validity: Includes field format validation, field length validation, field value range validation, code table reference comparison, and data standard code table reference comparison (requires the Data Standard module).
Consistency: Includes field value consistency comparison in a single table, statistical value consistency comparison of fields in a single table, business logic consistency comparison for a single field, field value consistency comparison between two tables, statistical value consistency comparison of fields between two tables and business logic consistency comparison of fields between two tables, and statistical value consistency comparison of fields between two cross-source tables.
Stability: Includes table stability validation, table fluctuation validation, field stability validation, and field fluctuation validation.
Custom SQL: Includes custom statistical metric validation and custom data detail validation.
For more information, see Template type descriptions.
Rule type
A fundamental attribute of the template, used for description and filtering.
Monitoring granularity
When the configuration type is Custom SQL, you can configure the monitoring granularity. You can select Full Table or specify a specific field.
Template Configuration
Template Information
When you select a quality rule template, its configuration information is displayed. If you need to modify the configuration information, you can do so in Quality Rule Templates.
Rule configuration
Rule Configuration
The rule configuration varies depending on the selected rule template. For more information, see Table parameter configuration.
Data filtering for validation tables: Disabled by default. When this feature is enabled, you can configure filter conditions, partition filtering, or regular data filtering. The filter conditions will be directly appended to the validation SQL. If a validation table requires partition filtering, configure a partition expression in the scheduling configuration. After the expression is configured, the validation partition is used as the minimum granularity for viewing quality reports.
When you set the rule template to Consistency/Statistical Value Consistency Comparison for Two Tables or Consistency/Cross-Source Statistical Value Consistency Comparison for Two Tables, you can enable Comparison Table Data Filtering. This allows you to configure filter conditions, partition filtering, or regular data filtering for the comparison table. The filter conditions are directly appended to the validation SQL.
Validate Configuration
Rule validation
After a data quality rule is validated, the result is compared with the anomaly validation configuration. If the result meets the conditions, the validation fails and triggers subsequent processes, such as alerts.
The available metrics for anomaly validation are determined by the template and configuration content. The validation supports AND/OR logic for multiple criteria. We recommend that you use fewer than three criteria in an actual configuration.
For more information, see Validation configuration.
Archive Configuration
Exception Archiving
This feature is disabled by default. When you enable it, you can archive abnormal data to files or tables, and then download and analyze the archived data after a quality validation.
Archive Mode supports Archive Only Anomalous Fields and Archive Full Records.
Archive Only Abnormal Fields: Deduplicates and archives only the current monitored field. This is suitable for scenarios where abnormal data can be uniquely identified by a single field.
Archive Entire Record: Archives the entire record that contains anomalous data. Use this option when the entire record is required to locate the anomalous data. Note: Because archiving the entire record significantly increases the amount of archived data, we recommend that you archive only anomalous fields in most cases.
Archive Location supports Default File Server and Custom Exception Archive Table. If you have not created an exception archive table, you can click Manage Exception Archive Tables to create one. For more information, see Add an exception archive table.
Default file server: The system file server that is configured when you deploy Dataphin. You can directly download exception data from the Validation Record-Validation Details page or access the default file server to obtain the exception data. When you use the default file server, each validation run archives a maximum of 100 exception data records. This is suitable for scenarios where you validate small volumes of data.
Custom exception archive table: If you want to store more exception data or consolidate exception data from different validation records for subsequent comparative analysis, you can specify a custom archive table. Each quality rule can record a maximum of 10,000 exception records per run. In addition to downloading exception data for a single validation run from the validation records page, you can also directly access the archive table and customize its lifecycle for greater flexibility.
NoteYou can download a summary of the exception data generated by all rules in the current run. The download is limited to 10,000 entries. To view more data, archive the data to a specified exception archive table and then access the table directly to retrieve it.
An exception archive table must meet specific format requirements. Otherwise, an error may occur when you write data to the table, which affects its use. For more information, see Add an exception archive table.
When the compute engine is Transwarp TDH 6.x or Transwarp TDH 9.3.x and you select a custom exception archive table, you can select Display field descriptions in the exception data table header. After you select this option, the fields in the downloaded exception data table will contain comment information.
Service Property Configuration
Properties
The input format for business attributes depends on the configuration of the quality rule attributes. For example, if the Managing department attribute is configured as a multi-select enum with the options Big Data Department, Business Department, and Technology Department, then when you create a quality rule, this attribute is displayed as a multi-select dropdown list containing these options.
The Rule owner field is a custom input field with a maximum length of 256 characters. Therefore, when you create a quality rule, the value that you enter for this field cannot exceed this limit.
If the attribute field accepts a value range, the configuration method is as follows:
Range: Used for continuous numerical or date ranges. You can select from the following operators: >, >=, <, or <=. For more information, see Create and manage quality rule properties.
Scheduling Property Configuration
Scheduling Method
You can select a configured scheduling task. If you have not decided on a scheduling method, you can create the quality rule and configure the scheduling later. To create a new scheduling task, see Create a scheduling task.
Quality Score Configuration
Scoring Method
The service supports two scoring methods: quality validation status and data compliance ratio.
Quality Validation Status: The score is based on the status of the most recent validation record that was successfully executed according to the current rule. The score is 100 for a passing validation and 0 for a failing validation.
Data compliance percentage: The quality score is the percentage of valid data from the most recent successful validation record for the current rule. For example, if the data format validity is 80%, the quality score is 80.
Different rule templates support different scoring methods. The Quality Validation Status scoring method is described as follows:
Validation of the number of field groups and the number of duplicate field values for the uniqueness rule category.
The consistency rule category includes single-table field statistics consistency comparison and cross-source two-table field statistics consistency comparison.
Major category of stability rules.
Validation of custom statistical metrics for the custom SQL rule category.
Quality Score Weight
The quality score weight of a quality rule is used to calculate the quality score of a monitored object. You can select an integer from 1 to 10.
Click OK to complete the rule configuration.
Click Preview SQL to compare the SQL for the current configuration with the last saved configuration.
NotePreview SQL is unavailable if required information is incomplete.
The left side shows the SQL preview for the last saved configuration. This side is empty if no configuration was saved. The right side shows the SQL preview for the current configuration.
Rule Configuration List
On the Rule Configuration page, you can view your configured table rules and perform actions such as view, edit, test run, run, and delete.
Area | Description |
① Filter and search area | Search for objects or rules by name. Filter by rule type, rule template, rule severity, dry run status, and effective status. Note If a quality rule has searchable and filterable business attributes configured and enabled, you can search or filter by those attributes. |
② List area | This area displays the rule configuration list. The list includes the object type/name, rule name/ID, dry run status, effective status, rule type, rule template, rule severity, scheduling type, and related knowledge base document information. Click the
|
③ Operations area | You can perform the following operations: view, clone, edit, dry run, run, configure scheduling, associate knowledge base document, configure quality score, and delete.
|
④ Batch operations area | You can perform the following batch operations: dry run, run, configure scheduling, enable, disable, modify business attributes, associate knowledge base document, configure quality score, export rules, and delete.
|
icon before Refresh to select the columns to display in the rule list.
icon to view the dry run log.Create A Scheduling Task
When configuring a scheduling task for a rule, you can reuse an existing one to speed up the process. Each table can have a maximum of 20 scheduling tasks.
You can configure a maximum of 10 scheduling tasks for a single rule.
If scheduling configurations are identical, the system automatically deduplicates them.
The check scope acts as a filter condition to control the data range for each quality check. It also serves as the basic unit for downstream components, such as quality reports, defining the minimum viewing granularity.
On the Quality Rule Details page, click the Scheduling Configuration tab, and then click Create Scheduling Task to open the Create Scheduling Task dialog box.
In the Create Scheduling Task dialog box, configure the parameters.
Parameter
Description
Scheduling Name
Enter a custom name for the scheduling task, up to 64 characters.
Scheduling Type
The supported types are time-based scheduling and task-triggered scheduling.
time-based scheduling: Runs quality checks periodically based on a configured schedule. This type is suitable for scenarios where data is generated at relatively fixed times.
Scheduling cycle: Running a quality rule consumes computing resources. We recommend that you avoid running multiple quality rules concurrently to prevent impacts on your production tasks. The supported scheduling cycles are Day, Week, Month, Hour, and Minute.
When the system time zone (that is, the time zone in the user center) is different from the scheduling time zone (that is, the time zone configured in Management Center > System Settings > Basic Settings), the rule is executed based on the system time zone.
task-triggered scheduling: Executes the configured quality rule after a specified task successfully runs or before it runs. You can trigger the quality rule based on tasks of the following node types: SQL, offline pipeline, Python, Shell, Virtual, Dlink, Spark_jar, Hive_MR, and database SQL. This scheduling type is suitable for scenarios where table modification tasks are predictable.
NoteYou can only select tasks from the production environment for task-triggered scheduling. If a task fails a check from a strong rule, your online tasks may be affected. Proceed with caution.
The supported engine types are MaxCompute, E-MapReduce 3.X, E-MapReduce 5.x, CDH 5.X, CDH 6.X, FusionInsight 8.X, AsiaInfo DP 5.3, Cloudera Data Platform 7.x, Lindorm (compute engine), Transwarp TDH 6.x, and Transwarp TDH 9.3.x.
Triggering time: Select when the quality check is triggered. You can select Trigger after all tasks succeed, Trigger after each task succeeds, or Trigger before each task runs.
Triggering task: Select the task node from a production project to trigger the quality rule. You can also search for the task by its node output name. The following roles can select a triggering task:
Project administrators of Prod/Basic projects, users with the Ops system role in Prod projects, users with the Developer system role in Basic projects, and users with custom project roles that have Project Quality Management-Quality Rule Management permissions in Prod/Basic projects can select task nodes from the production project.
Users with custom global roles that have Quality Rule-Management permissions in Prod/Basic projects can select task nodes from all production projects.
NoteTo prevent delays in rule execution and result generation, we recommend that all triggering tasks share the same scheduling cycle when you use the Trigger after all tasks succeed option.
Scheduling Conditions
Disabled by default. If enabled, the system verifies that the scheduling conditions are met before running the task. If the conditions are not met, the scheduled run is skipped.
Business date/Execution date: You can configure a date condition for both time-based scheduling and task-triggered scheduling. Note that time-based scheduling does not support the Execution date condition. You can use a General Calendar or a Custom Calendar. For information about how to create a custom calendar, see Create a public calendar.
If you select General Calendar, you can set conditions based on Month, Day of Week, or Date. For example:
If you select Custom Calendar, you can set conditions based on Date Type or Tag. For example:
Instance type: If the scheduling type is task-triggered scheduling, you can configure the instance type. The options are periodic instance, backfill instance, and manual instance. For example:
NoteYou must configure at least one condition. To add a condition, click the + Add Condition button.
You can configure a maximum of 10 scheduling conditions.
The relationship between the scheduling conditions can be set to and or or.
Check range expression
This parameter is an editable drop-down list. You can directly enter the check range, such as
ds='${yyyyMMdd}'. You can also select a built-in partition expression and modify it to speed up configuration. For more information about partition expressions, see Built-in partition expression types.NoteIf there are multiple conditions for validation, you can use and or or to connect them, such as
province="Zhejiang" and ds<=${yyyyMMdd}.If a filter condition is configured in the quality rule, it is combined with the check range expression using an and operator. During the check, both conditions are applied to filter the data.
The check range expression supports a full table scan.
Note: A full table scan consumes significant resources and is not supported in some scenarios. We recommend that you configure a partition expression to avoid full table scans.
Check Date
Defaults to the current business date.
Click OK to complete the scheduling configuration.
Scheduling Configuration List
After a scheduling task is created, you can view, edit, clone, or delete it in the scheduling configuration list.
Area | Description |
① Filter and search area | You can search by scheduling name. You can filter by time-based scheduling or task-triggered scheduling. |
② List area | This area displays the Scheduling Name, Scheduling Type, Last Modified By, and Last Modified Time for each scheduling task. |
③ Actions area | You can edit, clone, or delete a scheduling task.
|
Alert Configuration
You can configure different notification channels for different quality rules to distinguish between alerts. For example, you can set phone call alerts for exceptions in strong rules and SMS alerts for exceptions in weak rules. If a single rule triggers multiple alert configurations, you can define a policy to determine which configuration takes effect.
You can create up to 20 alert configurations per monitoring object.
On the Quality rule details page, click the Alert configuration tab, and then click New alert configuration to open the New alert configuration dialog box.
In the New alert configuration dialog box, configure the parameters.
Parameter
Description
Scope
Specifies the rules to which this configuration applies. You can select All rules, All strong rules, All weak rules, or Custom.
NoteFor a single monitored object, you can configure one alert for each of the three scopes: all rules, all strong rules, and all weak rules. New rules are automatically matched with the corresponding alert based on their rule strength. If you need to change one of these alert configurations, you can modify the existing configuration.
For a custom scope, you can select up to 200 rules configured for the current monitored object.
Alert configuration name
The alert configuration name must be unique per monitoring object and cannot exceed 256 characters.
Alert recipient
Configure alert recipients and notification channels. You must select at least one alert recipient and one notification channel.
Alert recipient: You can select from three types of alert recipients: custom, on-call schedule, and quality owner.
You can configure up to 5 custom alert recipients and up to 3 on-call schedules.
Notification channel: Supported channels include phone call, email, SMS, DingTalk, Lark, WeCom, and custom channel. These channels can be managed in Configure Channel Settings.
Click OK to save the alert configuration.
Alert Configuration List
After you create alert configurations, you can sort, edit, or delete them in the list.
Area | Description |
① Sorting area | Configure the policy that determines which alert configuration takes effect when a quality rule matches multiple configurations:
|
② List area | Displays the name, scope, specific recipients, and their corresponding notification channels for each alert configuration. Scope: For custom alert configurations, you can view the names of the configured monitoring objects and quality rules. If a rule is deleted, its name is no longer displayed. In this case, we recommend updating the alert configuration.
|
③ Operations area | This area lets you edit or delete alert configurations.
|
icon next to a configuration name to reorder it. You can also use the Move to Top and Move to Bottom icons in the Actions column to reorder it. After reordering, click Done sorting to save the changes.
Add An Exception Archive Table
An exception archive table stores records that fail a quality rule check.
On the Quality Rule Details page, go to the Exception Archiving tab and click + Add Exception Archiving Table. The Add Exception Archiving Table dialog box appears.
In the Add Exception Archiving Table dialog box, configure the parameters.
Method, select Create New Table or Select Existing Table. The exception archive table contains special quality check columns so that exception data is not written to the original data table.
Create New Table: You can enter a custom name for the table. The default name is current_table_name_exception_data. After you add the table, the system creates a new table in the same database or data source. The name can contain letters, numbers, underscores (_), and periods (.), and cannot exceed 128 characters.
If the monitored table is a physical table, the system creates the archive table in the monitored table's project.
If the monitored table is a dimensional logical table or a fact logical table, the system creates the archive table in the same project by default. You can also manually specify a project within the same workspace, such as
projectA.table_name.If the monitored table is a summary logical table, specify a project in the table name. Otherwise, the system automatically creates the archive table in one of the projects within the same workspace.
The archive table must include all columns from the monitored table and the required validation columns. The script format is as follows:
create table 当前表名_exception_data (dataphin_quality_tenant_id varchar(64) comment 'tenant ID' , dataphin_quality_rule_id varchar(64) comment 'quality rule ID', dataphin_quality_rule_name varchar(256) comment 'quality rule name', dataphin_quality_column_name varchar(1024) comment 'checked column name', dataphin_quality_watch_task_id varchar(128) comment 'task ID of the monitored object', dataphin_quality_rule_task_id varchar(64) comment 'rule task ID', dataphin_quality_validate_time varchar(64) comment 'quality check time', dataphin_quality_archive_mode varchar(32) comment 'exception archiving mode, ONLY_ERROR_FIELD/FULL_RECORD', dataphin_quality_error_data string comment 'exception data', ljba_id bigint comment 'ljba_primary key', ljb_id bigint comment 'ljb_primary key', col_tinyint tinyint comment 'data type is TINYINT and in lowercase', col_tinyint_02 tinyint comment '2', col_smallint smallint comment 'data type is SMALLINT and in lowercase', col_smallint_02 smallint comment '4', col_int int comment 'data type is INT and in lowercase', col_int_02 int comment '6', col_bigint bigint comment 'data type is BIGINT and in lowercase', col_bigint_02 bigint comment '8', col_float float comment 'data type is FLOAT and in lowercase', col_float_02 float comment '10', col_double double comment 'data type is DOUBLE and in lowercase', col_double_02 double comment '11', col_decimal decimal(38,18) comment 'data type is DECIMAL(38,18) and in lowercase', col_decimal_02 decimal(38,18) comment '12', col_varchar varchar(500) comment 'data type is VARCHAR(500) and in lowercase', col_varchar_02 varchar(500) comment '13', col_char char(10) comment 'data type is CHAR(10) and in lowercase', col_char_02 char(10) comment '14', col_string string comment 'data type is STRING and in lowercase', col_string_02 string comment '15', col_date date comment 'data type is DATE and in lowercase', col_date_02 date comment '16', col_datetime datetime comment 'data type is DATETIME and in lowercase', col_datetime_02 datetime comment '17', col_timestmap timestamp comment 'data type is TIMESTAMP and in lowercase', col_timestmap_02 timestamp comment '18', col_boolean boolean comment 'data type is BOOLEAN and in lowercase', col_boolean_02 boolean comment '19', col_binary binary comment 'data type is BINARY and in lowercase', col_binary_02 binary comment '20', col_array array<int> comment 'data type is ARRAY<int> and in lowercase', col_array_02 array<string> comment '21', col_map map<string,string> comment 'data type is MAP<string, string> and in lowercase', col_map_02 map<string,int> comment '22', ds string comment 'date partition, yyyyMMdd' ) partitioned by (dataphin_quality_validate_date string comment 'validation date (partition column)');
Select Existing Table: You can select a table from the same project or data source. The archive table must include all columns from the monitored table and the required validation columns. You can click View Exception Archive Table DDL to see the required DDL statement. The script format is as follows:
create table 当前表名_exception_data (dataphin_quality_tenant_id varchar(64) comment 'tenant ID' , dataphin_quality_rule_id varchar(64) comment 'quality rule ID', dataphin_quality_rule_name varchar(256) comment 'quality rule name', dataphin_quality_column_name varchar(1024) comment 'checked column name', dataphin_quality_watch_task_id varchar(128) comment 'task ID of the monitored object', dataphin_quality_rule_task_id varchar(64) comment 'rule task ID', dataphin_quality_validate_time varchar(64) comment 'quality check time', dataphin_quality_archive_mode varchar(32) comment 'exception archiving mode, ONLY_ERROR_FIELD/FULL_RECORD', dataphin_quality_error_data string comment 'exception data', ljba_id bigint comment 'ljba_primary key', ljb_id bigint comment 'ljb_primary key', col_tinyint tinyint comment 'data type is TINYINT and in lowercase', col_tinyint_02 tinyint comment '2', col_smallint smallint comment 'data type is SMALLINT and in lowercase', col_smallint_02 smallint comment '4', col_int int comment 'data type is INT and in lowercase', col_int_02 int comment '6', col_bigint bigint comment 'data type is BIGINT and in lowercase', col_bigint_02 bigint comment '8', col_float float comment 'data type is FLOAT and in lowercase', col_float_02 float comment '10', col_double double comment 'data type is DOUBLE and in lowercase', col_double_02 double comment '11', col_decimal decimal(38,18) comment 'data type is DECIMAL(38,18) and in lowercase', col_decimal_02 decimal(38,18) comment '12', col_varchar varchar(500) comment 'data type is VARCHAR(500) and in lowercase', col_varchar_02 varchar(500) comment '13', col_char char(10) comment 'data type is CHAR(10) and in lowercase', col_char_02 char(10) comment '14', col_string string comment 'data type is STRING and in lowercase', col_string_02 string comment '15', col_date date comment 'data type is DATE and in lowercase', col_date_02 date comment '16', col_datetime datetime comment 'data type is DATETIME and in lowercase', col_datetime_02 datetime comment '17', col_timestmap timestamp comment 'data type is TIMESTAMP and in lowercase', col_timestmap_02 timestamp comment '18', col_boolean boolean comment 'data type is BOOLEAN and in lowercase', col_boolean_02 boolean comment '19', col_binary binary comment 'data type is BINARY and in lowercase', col_binary_02 binary comment '20', col_array array<int> comment 'data type is ARRAY<int> and in lowercase', col_array_02 array<string> comment '21', col_map map<string,string> comment 'data type is MAP<string, string> and in lowercase', col_map_02 map<string,int> comment '22', ds string comment 'date partition, yyyyMMdd' ) partitioned by (dataphin_quality_validate_date string comment 'validation date (partition column)');
Lifecycle: This setting is available only for tables that use MaxCompute as the compute engine. If you are creating a new table, you can enter a custom number of days or select from the presets: 7 Days, 14 Days, 30 Days, or 360 Days. If you select an existing table, its configured lifecycle is displayed.
Max Archive Count: This setting is available only for tables that use MaxCompute or the Hadoop series as the compute engine. This setting specifies the maximum number of exception records to archive from a single quality rule run. If the number of exceptions exceeds this threshold, the system archives only the specified number of records. Excess records are not saved to the archive table, but you can view them in the log. The default is 10,000 records per check. You can select 10,000, 100,000, 500,000, or All, or enter any positive integer.
NoteA large volume of data in the archive table can affect query performance. Configure this setting based on your business needs.
Click OK to add the exception archive table.
You can select Set as active archive table after creation. This automatically sets the table as the archive destination when you create quality rules later.
Exception Archive Table List
After you add a table, the first one in the list is set as the active archive table by default. Click the name of an exception archive table to view its table structure. You can also set other tables as the active archive table, delete them, or edit their settings.
Set as Active Archive Table: If you set a table as active, it becomes the destination for exception data from all quality rules configured to use a custom exception table for the monitored object.
Delete: This action only removes the reference to the exception archive table. It does not delete the physical table.
Edit: This option is available only for tables that use MaxCompute or the Hadoop series as the compute engine. You can modify the lifecycle and maximum archive count.
Quality Report
Click quality report to view the rule check overview and rule check details for the current quality rule.
You can quickly filter rule check details by abnormal results, partition time, or keywords in the rule or object name.
Click the
icon in the operation column to view the rule check details.Click the
icon in the operation column to view the execution log.
Permission Management For Quality Rules
Click Permission Management to configure the view details permission, which determines which members can view validation record details, quality rule details, and quality reports.
View details permission: Select All Members or Only members with quality management permissions for the current object.
Click OK to save the permission settings.
Quality Rule Run Configuration
If a quality rule task times out, its execution status is set to failed. You can view the corresponding execution log on the verification record details page.
For retried quality rule tasks, you can view the execution logs for each attempt on the verification record details page.
For a single monitoring object, all quality rules, including those that are newly created or have not yet run, use the latest run configuration for their next execution.
Click the Run Configuration tab, then click Edit at the bottom of the page to configure the parameters.
Parameter
Description
Scope
You can select All Rules, All Strong Rules, All Weak Rules, or Custom.
Run timeout
If the total run time of a quality rule task (from start to finish, excluding resource waiting and scheduling waiting) exceeds the configured threshold, the system automatically terminates the task and sets its status to failed. The default is 1 hour. You can set a custom duration between 0 and 2 hours, accurate to two decimal places.
Automatic retry
For failed quality rule tasks, you can configure automatic retries based on the retry count and retry interval. This feature is disabled by default. Enable this feature to configure the retry count and interval.
Retry count: The maximum number of automatic retries for a failed quality rule task. The default is 1. You can set an integer from 1 to 3.
Retry interval: The interval in minutes between automatic retry attempts. The default is 0. You can set an integer from 0 to 10.
Click OK to save the run configuration.
Next Steps
After configuring the quality rules, you can view them on the global data table rule list page. For more information, see View the monitoring object list.