Retrieves the details of a data quality check task instance.
This API has been deprecated. We recommend migrating to dataworks-public(2024-05-18) - GetDataQualityScanRun for enhanced functionality and continued support. While the deprecated API remains temporarily accessible, no further updates or bug fixes will be provided.
Operation description
DataWorks Basic Edition or a higher edition is required.
Try it now
Test
RAM authorization
|
Action |
Access level |
Resource type |
Condition key |
Dependent action |
|
dataworks:* |
get |
*All Resource
|
None | None |
Request parameters
|
Parameter |
Type |
Required |
Description |
Example |
| Id |
integer |
Yes |
The data quality monitoring instance ID. |
7227550902 |
Response elements
|
Element |
Type |
Description |
Example |
|
object |
Schema of Response |
||
| RequestId |
string |
The request ID. You can use this ID to locate logs and troubleshoot issues. |
8abcb91f-d266-4073-b907-2ed67**** |
| DataQualityEvaluationTaskInstance |
object |
The details of the data quality evaluation task instance. |
|
| Status |
string |
The status of the data quality monitoring instance. Valid values:
|
Passed |
| Task |
object |
The data quality evaluation task. |
|
| Name |
string |
The name of the data quality evaluation task. |
Data quality OpenAPI monitoring test |
| Notifications |
object |
The notification settings. |
|
| Condition |
string |
The condition for triggering a notification. When this condition is met, a message notification is triggered. Currently, only two types of conditional expressions are supported:
|
${severity} == "High" AND ${status} == "Critical" |
| Notifications |
array<object> |
The notification methods. |
|
|
array<object> |
|||
| NotificationReceivers |
array<object> |
The values of the notification recipients. |
|
|
object |
|||
| ReceiverType |
string |
The type of the alert recipient. |
DingdingUrl |
| Extension |
string |
The additional parameter settings in JSON format when an alert is sent. Supported keys:
|
{ "atAll": true } |
| ReceiverValues |
array |
The alert recipients. |
|
|
string |
The value of the recipient.
|
https://api.fc.aliyuncs.com/webhook |
|
| NotificationChannels |
array<object> |
The notification channels. |
|
|
object |
|||
| Channels |
array |
The notification channels. |
|
|
string |
The alert methods. Valid values:
|
|
|
| Hooks |
array<object> |
The hook settings. |
|
|
object |
|||
| Type |
string |
The hook type. Currently, only one type is supported:
|
BlockTaskInstance |
| Condition |
string |
The condition for triggering the hook. When this condition is met, the hook action is triggered. Currently, only two types of conditional expressions are supported:
|
(${severity} == "High" AND ${status} == "Critical") OR (${severity} == "Normal" AND ${status} == "Critical") OR (${severity} == "Normal" AND ${status} == "Error") |
| RuntimeConf |
string |
The extension configuration, a JSON-formatted character string, takes effect only for EMR data quality monitoring.
|
{ "queue": "default" } |
| Trigger |
object |
The trigger configurations of the data quality evaluation task. |
|
| Type |
string |
The trigger type of the data quality evaluation task. Valid values:
|
ByScheduledTaskInstance |
| TaskIds |
array |
The list of scheduling task IDs. This parameter takes effect only when Type is set to ByScheduledTaskInstance. |
|
|
integer |
The scheduling task ID. |
70006236752324 |
|
| Target |
object |
The monitored object of the data quality evaluation task. For more information, see the DataQualityTarget example and DataQualityTarget. |
|
| Type |
string |
The monitored object type. Valid values:
|
Table |
| DatabaseType |
string |
The database type to which the table belongs. |
maxcompute |
| TableGuid |
string |
The unique ID of the table in DataWorks Data Map. |
odps.api_trace.ods_d_api_log |
| PartitionSpec |
string |
The partition range to be monitored. |
pt=$[yyyymmdd-1] |
| Description |
string |
The description of the data quality evaluation task. |
OpenAPI quality monitoring test |
| Id |
integer |
The ID of the data quality evaluation task. |
28544990 |
| ProjectId |
integer |
The ID of the workspace. |
20629 |
| Parameters |
string |
The execution parameters of the data quality evaluation task in JSON format. Available keys:
|
{ "triggerTime": 1733284062000 } |
| FinishTime |
integer |
The end time of the instance run. The value is a 13-digit number, such as |
1716344665000 |
| TriggerContext |
string |
The context information when the instance is triggered, in JSON format. Possible keys:
|
{ "triggerClient": "CWF2", "triggerClientId": 70001238945 } |
| CreateTime |
integer |
The creation time. The value is a 13-digit number, such as |
1716344665000 |
| Id |
integer |
The instance ID of the data quality monitoring. |
7234231689 |
| ProjectId |
integer |
The ID of the workspace. |
98330 |
| Results |
array<object> |
The list of data quality evaluation results. |
|
|
array<object> |
|||
| Id |
integer |
The ID of the evaluation result. |
10001 |
| Status |
string |
The status of the evaluation result. |
Passed |
| CreateTime |
integer |
The generation time of the evaluation result. The value is a 13-digit number, such as |
1708284916414 |
| TaskInstanceId |
integer |
The instance ID of the data quality check task. |
20001 |
| Sample |
string |
The sample value used for this evaluation. |
[ { "gender": "male", "_count": 100 }, { "gender": "female", "_count": 100 } ] |
| Details |
array<object> |
The details of this evaluation. |
|
|
object |
|||
| ReferencedValue |
string |
The referenced sample value used as the baseline for calculating the CheckedValue. |
0.0 |
| CheckedValue |
string |
The final value used to compare with the threshold. |
100.0 |
| Status |
string |
The status of the final comparison result. |
Passed |
| Rule |
object |
The snapshot of the rule configuration at the start of the evaluation. |
|
| Id |
integer |
The globally unique ID of the rule. |
100001 |
| Name |
string |
The name of the rule. The name can contain digits, letters, Chinese characters, and half-width or full-width punctuation marks. It can be up to 255 characters in length. |
Table cannot be empty |
| ProjectId |
integer |
The ID of the DataWorks workspace. |
100 |
| Enabled |
boolean |
Specifies whether the rule is enabled. |
|
| Severity |
string |
The severity level of the rule for the business, which corresponds to the strong and weak rules on the page. |
High |
| Description |
string |
The description of the rule. It can be up to 500 characters in length. |
this is a odps _sql task |
| TemplateCode |
string |
The rule template referenced when the rule is created. |
SYSTEM:user_defined_sql |
| Target |
object |
The object monitored by the rule. |
|
| Type |
string |
The monitored object type. Valid values:
|
Table |
| DatabaseType |
string |
The database type to which the table dataset belongs. Valid values:
|
maxcompute |
| TableGuid |
string |
The unique ID of the table in DataWorks Data Map. |
odps.unit_test.tb_unit_test |
| SamplingConfig |
object |
The settings required for sample collection. |
|
| Metric |
string |
The name of the sampling metric. Valid values:
|
Min |
| MetricParameters |
string |
The parameters required for sample collection. |
{ "Columns": [ "id", "name" ] } |
| SettingConfig |
string |
The runtime parameter setting statements inserted before the sampling statements are executed. It can be up to 1,000 characters in length. Currently, only MaxCompute is supported. |
SET odps.sql.udf.timeout=600s; |
| SamplingFilter |
string |
The condition for secondary filtering of uninterested data during sampling. It can be up to 16,777,215 characters in length. |
id IS NULL |
| CheckingConfig |
object |
The sample check settings. |
|
| Type |
string |
The threshold calculation method. Valid values:
|
Fixed |
| ReferencedSamplesFilter |
string |
The expression that represents the query method for reference samples. Some types of thresholds require querying reference samples and aggregating their values to obtain the threshold for comparison. |
{ "bizdate": [ "-1", "-7", "-1m" ] } |
| Thresholds |
object |
The threshold settings. |
|
| Expected |
object |
The expected threshold settings. |
|
| Value |
string |
The threshold value. |
100.0 |
| Operator |
string |
The comparison operator. Valid values:
|
> |
| Expression |
string |
The threshold expression. |
$checkValue > 0.01 |
| Warned |
object |
The threshold settings for warning alerts. |
|
| Value |
string |
The threshold value. |
100.0 |
| Operator |
string |
The comparison operator. Valid values:
|
> |
| Expression |
string |
The threshold expression. |
$checkValue > 0.01 |
| Critical |
object |
The threshold settings for critical alerts. |
|
| Value |
string |
The threshold value. |
100.0 |
| Operator |
string |
The comparison operator. Valid values:
|
> |
| Expression |
string |
The check expression. |
$checkValue > 0.01 |
| ErrorHandlers |
array<object> |
The error handlers for quality rule check issues. |
|
|
object |
|||
| Type |
string |
The handler type. |
SaveErrorData |
| ErrorDataFilter |
string |
The SQL statement used to filter problematic data for custom SQL rules. |
SELECT * FROM tb_api_log WHERE id IS NULL |
Examples
Success response
JSON format
{
"RequestId": "8abcb91f-d266-4073-b907-2ed67****",
"DataQualityEvaluationTaskInstance": {
"Status": "Passed",
"Task": {
"Name": "Data quality OpenAPI monitoring test\n",
"Notifications": {
"Condition": "${severity} == \"High\" AND ${status} == \"Critical\"",
"Notifications": [
{
"NotificationReceivers": [
{
"ReceiverType": "DingdingUrl",
"Extension": "{ \"atAll\": true }",
"ReceiverValues": [
"https://api.fc.aliyuncs.com/webhook"
]
}
],
"NotificationChannels": [
{
"Channels": [
"Mail"
]
}
]
}
]
},
"Hooks": [
{
"Type": "BlockTaskInstance",
"Condition": "(${severity} == \"High\" AND ${status} == \"Critical\") OR (${severity} == \"Normal\" AND ${status} == \"Critical\") OR (${severity} == \"Normal\" AND ${status} == \"Error\")"
}
],
"RuntimeConf": "{ \"queue\": \"default\" }",
"Trigger": {
"Type": "ByScheduledTaskInstance",
"TaskIds": [
70006236752324
]
},
"Target": {
"Type": "Table",
"DatabaseType": "maxcompute",
"TableGuid": "odps.api_trace.ods_d_api_log",
"PartitionSpec": "pt=$[yyyymmdd-1]"
},
"Description": "OpenAPI quality monitoring test",
"Id": 28544990,
"ProjectId": 20629
},
"Parameters": "{ \"triggerTime\": 1733284062000 }\n",
"FinishTime": 1716344665000,
"TriggerContext": "{ \"triggerClient\": \"CWF2\", \"triggerClientId\": 70001238945 }",
"CreateTime": 1716344665000,
"Id": 7234231689,
"ProjectId": 98330,
"Results": [
{
"Id": 10001,
"Status": "Passed",
"CreateTime": 1708284916414,
"TaskInstanceId": 20001,
"Sample": "[ { \"gender\": \"male\", \"_count\": 100 }, { \"gender\": \"female\", \"_count\": 100 } ]",
"Details": [
{
"ReferencedValue": "0.0",
"CheckedValue": "100.0",
"Status": "Passed"
}
],
"Rule": {
"Id": 100001,
"Name": "Table cannot be empty",
"ProjectId": 100,
"Enabled": false,
"Severity": "High",
"Description": "this is a odps _sql task",
"TemplateCode": "SYSTEM:user_defined_sql",
"Target": {
"Type": "Table",
"DatabaseType": "maxcompute",
"TableGuid": "odps.unit_test.tb_unit_test"
},
"SamplingConfig": {
"Metric": "Min",
"MetricParameters": "{ \"Columns\": [ \"id\", \"name\" ] }",
"SettingConfig": "SET odps.sql.udf.timeout=600s;",
"SamplingFilter": "id IS NULL"
},
"CheckingConfig": {
"Type": "Fixed",
"ReferencedSamplesFilter": "{ \"bizdate\": [ \"-1\", \"-7\", \"-1m\" ] }",
"Thresholds": {
"Expected": {
"Value": "100.0",
"Operator": ">",
"Expression": "$checkValue > 0.01"
},
"Warned": {
"Value": "100.0",
"Operator": ">",
"Expression": "$checkValue > 0.01"
},
"Critical": {
"Value": "100.0",
"Operator": ">",
"Expression": "$checkValue > 0.01"
}
}
},
"ErrorHandlers": [
{
"Type": "SaveErrorData",
"ErrorDataFilter": "SELECT * FROM tb_api_log WHERE id IS NULL"
}
]
}
}
]
}
}
Error codes
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.