GetDataQualityRule - DataWorks - Alibaba Cloud Documentation Center

Queries the information about a data quality monitoring rule.

Operation description

This API operation is available for all DataWorks editions.

Debugging

You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

Debug

Authorization information

There is currently no authorization information disclosed in the API.

Request parameters

Parameter	Type	Required	Description	Example
Id	long	Yes	The rule ID.	19715

Response parameters

Parameter	Type	Description	Example
	object	The response parameters.
RequestId	string	The request ID.	691CA452-D37A-4ED0-9441
DataQualityRule	object	The information about the rule.
Id	long	The rule ID.	16033
Name	string	The rule name.	The table cannot be empty.
ProjectId	long	The DataWorks workspace ID.	1948
Enabled	boolean	Indicates whether the rule is enabled.	true
Severity	string	Rule for the business level (corresponding to the strong and weak rules on the page), optional enumeration value: Normal High	High
Description	string	The description of the rule. The description can be up to 500 characters in length.	this is a odps _sql task
Target	object	The monitored object of the rule.
Type	string	Monitoring object type Table	Table
DatabaseType	string	The dataset of the table type. The database type to which the table belongs. maxcompute emr cdh hologres analyticdb_for_postgresql analyticdb_for_mysql starrocks	maxcompute
TableGuid	string	The ID of the table that is limited by the rule in Data Map.	odps.unit_test.tb_unit_test
PartitionSpec	string	The configuration of the partitioned table.	ds=$[yyyymmdd-1]
TemplateCode	string	The ID of the template used by the rule.	system::user_defined
SamplingConfig	object	The sampling settings.
Metric	string	The metrics used for sampling. Valid values: Count: the number of rows in the table. Min: the minimum value of the field. Max: the maximum value of the field. Avg: the average value of the field. DistinctCount: the number of unique values of the field after deduplication. DistinctPercent: the proportion of the number of unique values of the field after deduplication to the number of rows in the table. DuplicatedCount: the number of duplicated values of the field. DuplicatedPercent: the proportion of the number of duplicated values of the field to the number of rows in the table. TableSize: the table size. NullValueCount: the number of rows in which the field value is null. NullValuePercent: the proportion of the number of rows in which the field value is null to the number of rows in the table. GroupCount: the field value and the number of rows for each field value. CountNotIn: the number of rows in which the field values are different from the referenced values that you specified in the rule. CountDistinctNotIn: the number of unique values that are different from the referenced values that you specified in the rule after deduplication. UserDefinedSql: indicates that data is sampled by executing custom SQL statements.	Max
MetricParameters	string	The parameters required for sampling.	{ "Columns": [ "id", "name" ] , "SQL": "select count(1) from table;"}
SettingConfig	string	The statements that are used to configure the parameters required for sampling before you execute the sampling statements. The statements can be up to 1,000 characters in length. Only the MaxCompute database is supported.	SET odps.sql.udf.timeout=600s; SET odps.sql.python.version=cp27;
SamplingFilter	string	The statements that are used to filter unnecessary data during sampling. The statements can be up to 16,777,215 characters in length.	id IS NULL
CheckingConfig	object	The check settings for sample data.
Type	string	The threshold calculation method. Valid values: Fixed Fluctation FluctationDiscreate Auto Average Variance	Fixed
ReferencedSamplesFilter	string	The method that is used to query the referenced samples. To obtain some types of thresholds, you need to query reference values. In this example, an expression is used to indicate the query method of referenced samples.	{ "bizdate": [ "-1", "-7", "-1m" ] }
Thresholds	object	The threshold settings.
Expected	object	The expected threshold setting.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
Expression	string	The threshold expression.	$checkValue <= 0.01
Warned	object	The threshold settings for normal alerts.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
Expression	string	The threshold expression.	$checkValue > 0.01
Critical	object	The threshold settings for critical alerts.
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Value	string	The threshold value.	100.0
Expression	string	The threshold expression.	$checkValue > 0.05
ErrorHandlers	array<object>	The operations that you can perform after the rule-based check fails.
ErrorHandler	object	The operation that you can perform after the rule-based check fails.
Type	string	Processor type: SaveErrorData	SaveErrorData
ErrorDataFilter	string	The SQL statement that is used to filter failed tasks. If you define the rule by using custom SQL statements, you must specify an SQL statement to filter failed tasks.	SELECT * FROM tb_api_log WHERE id IS NULL

Examples

Sample success responses

JSONformat

{
  "RequestId": "691CA452-D37A-4ED0-9441\n",
  "DataQualityRule": {
    "Id": 16033,
    "Name": "The table cannot be empty.",
    "ProjectId": 1948,
    "Enabled": true,
    "Severity": "High",
    "Description": "this is a odps _sql task\n",
    "Target": {
      "Type": "Table",
      "DatabaseType": "maxcompute",
      "TableGuid": "odps.unit_test.tb_unit_test\n",
      "PartitionSpec": "ds=$[yyyymmdd-1]\n"
    },
    "TemplateCode": "system::user_defined\n",
    "SamplingConfig": {
      "Metric": "Max",
      "MetricParameters": "{ \"Columns\": [ \"id\", \"name\" ] , \"SQL\": \"select count(1) from table;\"}",
      "SettingConfig": "SET odps.sql.udf.timeout=600s; \nSET odps.sql.python.version=cp27;\n",
      "SamplingFilter": "id IS NULL\n"
    },
    "CheckingConfig": {
      "Type": "Fixed",
      "ReferencedSamplesFilter": "{ \"bizdate\": [ \"-1\", \"-7\", \"-1m\" ] }\n",
      "Thresholds": {
        "Expected": {
          "Operator": ">",
          "Value": "100.0",
          "Expression": "$checkValue <= 0.01"
        },
        "Warned": {
          "Operator": ">",
          "Value": "100.0",
          "Expression": "$checkValue > 0.01"
        },
        "Critical": {
          "Operator": ">",
          "Value": "100.0",
          "Expression": "$checkValue > 0.05"
        }
      }
    },
    "ErrorHandlers": [
      {
        "Type": "SaveErrorData\n",
        "ErrorDataFilter": "SELECT * FROM tb_api_log WHERE id IS NULL\n"
      }
    ]
  }
}

Error codes

For a list of error codes, visit the Service error codes.

Change history

Change time	Summary of changes	Operation
2024-12-19	The response structure of the API has changed	View Change Details
2024-12-19	The internal configuration of the API is changed, but the call is not affected	View Change Details