In Data Quality, you can manage a set of custom rule templates and use the rule templates to improve the efficiency of rule configuration.

Background information

You can create a rule template on the Rule Templates and Monitoring Rules pages. After the rule template is created, you can manage and use it.

Create a rule template on the Rule Template Management page

  1. Go to the Data Quality page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. On the Workspaces page that appears, find the target workspace and click Data Analytics in the Actions column.
    4. On the DataStudio page that appears, click Icon in the upper-left corner and choose All Products > Data Quality.
  2. On the page that appears, choose Configuration > Rule Templates in the left-side navigation pane.
  3. On the Rule Templates page that appears, click Create icon and select Create Folder.
  4. In the Create Folder dialog box that appears, set the Name and Location parameters and click OK.
  5. Right-click the folder name and select Create Rule Template.
    You can also rename or delete a folder.
  6. In the Create Rule Template dialog box that appears, set the parameters.
    Create a rule template
    Parameter Description
    Template Name The name of the template. Enter a custom template name.
    Field The fields to be monitored. You can only set this parameter to Custom SQL.
    Sampling Method The statistical function. You can only set this parameter to Custom SQL.
    Set Flag The SET clause of the SQL statement for querying the field to be monitored.
    Note Separate multiple statements with commas (,). You do not need to add a semicolon (;) at the end of each statement.
    Check type The threshold type of the rule template. Valid values: Numeric type, Fluctuation, and Auto-Generated Threshold.
    Notice You can only select Auto-Generated Threshold in DataWorks Enterprise Edition or higher.
    Verification Method The verification method of the rule template. The verification methods that can be selected vary with the check type.
    • If you set Check Type to Numeric type, you can only set this parameter to Compare with a specified value.
    • If you set Check type to Fluctuation, the values that are optional for this parameter include Compare the current value with the average value of the last 7 days, Compare the current value with the average value of the last 30 days, Compare the current value with the value 1 day before, Compare the current value with the value 7 days before, Compare the current value with the value 30 days before, The variance between the current value and the value 7 days before, The variance between the current value and the value 30 days before, Compare with the value 1, 7, and 30 days before and Compare with the value of the previous cycle.
    • If you set Check Type to Auto-Generated Threshold, you can only set this parameter to Auto-Generated Threshold.
    Custom SQL The custom SQL statement. You can use ${tableName} as the table name.
    Location The name of the folder to which you want to store the custom rule template.
  7. Click OK.

Create a rule template on the Monitoring Rules page

  1. Go to the Data Quality page.
  2. On the page that appears, click Monitoring Rules in the left-side navigation pane.
  3. On the Monitoring Rules page that appears, select the computing engine or data store, find the target table or topic, and then click View Monitoring Rules in the Actions column.
    Note This topic uses a MaxCompute table as an example.
  4. Select a partition filter expression and click the Custom Rules tab.
    Custom rule
  5. On the Custom Rules tab, find the target custom rule and click Generate Template in the Actions column.
  6. In the Create Rule Template dialog box that appears, set the parameters.
    Create a rule template
    Parameter Description
    Template Name The name of the template. Enter a custom template name.
    Field The fields to be monitored. You can only set this parameter to Custom SQL.
    Sampling Method The statistical function. You can only set this parameter to Custom SQL.
    Set Flag The SET clause of the SQL statement for querying the field to be monitored.
    Note Separate multiple statements with commas (,). You do not need to add a semicolon (;) at the end of each statement.
    Check type The threshold type of the rule template. Valid values: Numeric type, Fluctuation, and Auto-Generated Threshold.
    Notice You can only select Auto-Generated Threshold in DataWorks Enterprise Edition or higher.
    Verification Method The verification method of the rule template. The verification methods that can be selected vary with the check type.
    • If you set Check Type to Numeric type, you can only set this parameter to Compare with a specified value.
    • If you set Check type to Fluctuation, the values that are optional for this parameter include Compare the current value with the average value of the last 7 days, Compare the current value with the average value of the last 30 days, Compare the current value with the value 1 day before, Compare the current value with the value 7 days before, Compare the current value with the value 30 days before, The variance between the current value and the value 7 days before, The variance between the current value and the value 30 days before, Compare with the value 1, 7, and 30 days before and Compare with the value of the previous cycle.
    • If you set Check Type to Auto-Generated Threshold, you can only set this parameter to Auto-Generated Threshold.
    Custom SQL The custom SQL statement. You can use ${tableName} as the table name.
    Location The name of the folder to which you want to store the custom rule template.
  7. Click OK.
  8. In the left-side navigation pane, choose Configuration > Rule Templates to view the created rule template.

Manage an existing rule template

On the Rule Templates page, you can click the name of a rule template to view, edit, delete, and copy the rule template.Manage an existing rule template
Action Description
View You can view the parameter configuration, the rules that use the rule template, and logs of the rule template:
  • The Application List tab displays the rules that use the rule template.
  • The View Log tab displays the logs of operations performed on the rule template, including the user who performed each operation, the time when each operation was performed, and the operation details.
Edit Click Edit in the upper-right corner. In the Edit Rule Template dialog box that appears, modify the required parameters, and click OK.
Delete Click Delete in the upper-right corner. When the Are you sure you want to delete the template? message appears, click OK.
Copy Click Copy in the upper-right corner. In the Copy Rule Template dialog box that appears, set the Template Name and Location parameters and click OK.

Use a rule template

When you create a monitoring rule, you can select a custom rule template to create the rule based on the rule template.

  1. Go to the Data Quality page.
  2. On the page that appears, click Monitoring Rules in the left-side navigation pane.
  3. On the Monitoring Rules page that appears, select the computing engine or data store, find the target table or topic, and then click View Monitoring Rules in the Actions column.
    Note This topic uses a MaxCompute table as an example.
  4. Select a partition filter expression and click Create rules.
  5. In the Create rules pane that appears, click the Template Rules tab and click Add Monitoring Rule.
  6. Set the parameters for the rule. Specifically, set Rule Source to Rule Templates and select a rule template. For more information about the parameter description, see Configure monitoring rules.
    Create a rule
  7. Click Batch Create.