Validates the quality of a Data Source against Validation Rules and returns the Validation Results.
This API has been deprecated. We recommend migrating to dataworks-public(2024-05-18) - ListDataQualityScanRuns for enhanced functionality and continued support. While the deprecated API remains temporarily accessible, no further updates or bug fixes will be provided.
Try it now
Test
RAM authorization
Request parameters
|
Parameter |
Type |
Required |
Description |
Example |
| RuleId |
integer |
Yes |
The ID of the monitoring rule. You can use the rule ID and a partition filter to query the monitoring results of the rule. |
152322134 |
| StartDate |
string |
Yes |
The beginning of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. You must configure this parameter together with the EndDate parameter. The time range to query cannot exceed 7 days. |
2020-09-20 00:00:00 |
| EndDate |
string |
Yes |
The end of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. You must configure this parameter together with the StartDate parameter. The time range to query cannot exceed 7 days. |
2020-09-22 00:00:00 |
| PageSize |
integer |
Yes |
The number of entries per page. Default value: 10. Maximum value: 20. |
10 |
| PageNumber |
integer |
Yes |
The page number. |
1 |
| ProjectName |
string |
Yes |
The name of the compute engine or data source for which you want to perform data quality monitoring. |
autotest |
| ProjectId |
integer |
Yes |
The ID of the DataWorks workspace. |
123456 |
Response elements
|
Element |
Type |
Description |
Example |
|
object |
The check result. |
||
| HttpStatusCode |
integer |
The HTTP status code. |
200 |
| ErrorMessage |
string |
The error message. |
You have no permission. |
| RequestId |
string |
The request ID. |
CBA58543-00D4-41**** |
| ErrorCode |
string |
The error code. |
401 |
| Success |
boolean |
Indicates whether the request was successful. |
true |
| Data |
object |
The top-level object of the check result. |
|
| PageNumber |
integer |
The page number. |
1 |
| PageSize |
integer |
The number of entries per page. Default value: 10. Maximum value: 100. |
10 |
| TotalCount |
integer |
The total number of records. |
200 |
| RuleChecks |
array<object> |
A list of check results. |
|
|
array<object> |
A check result. |
||
| BlockType |
integer |
The strength of the monitoring rule. A strong rule can block a downstream scheduling task if a critical alert is triggered. Valid values:
|
1 |
| WarningThreshold |
number |
The custom threshold for a warning alert. An alert is triggered if the deviation from the expected value exceeds this threshold. |
0.1 |
| Property |
string |
The column in the source data table that the rule checks. |
type |
| TableName |
string |
The name of the table being checked. |
dual |
| Comment |
string |
The description of the monitoring rule. |
The description of the rule. |
| CheckResultStatus |
integer |
The status of the check result. |
2 |
| TemplateName |
string |
The name of the rule template. |
Expected value verification |
| CheckerName |
string |
The name of the checker. |
fulx |
| RuleId |
integer |
The rule ID. |
123421 |
| FixedCheck |
boolean |
Indicates whether a fixed-value check is used. Valid values:
|
false |
| Op |
string |
The comparison operator. |
> |
| UpperValue |
number |
The predicted upper limit, which is automatically generated based on the configured threshold. |
22200 |
| ActualExpression |
string |
The actual data partition that was checked. |
ds=20200925 |
| ExternalId |
string |
The node ID of the scheduling task. |
123112232 |
| TimeCost |
string |
The duration of the check. |
10 |
| Trend |
string |
The trend of the check result. |
up |
| ExternalType |
string |
The type of the scheduling system. Currently, only CWF is supported. |
CWF2 |
| BizDate |
integer |
The business date. For an offline table, this is typically the day before the check is performed. |
1600704000000 |
| CheckResult |
integer |
The check result. |
2 |
| ResultString |
string |
The string representation of the check result. |
ResultString |
| MatchExpression |
string |
The partition filter expression. |
ds=$[yyyymmdd] |
| CheckerType |
integer |
The type of the checker. |
0 |
| ProjectName |
string |
The name of the engine or data source used for the quality check. |
autotest |
| BeginTime |
integer |
The start time of the check. |
1600704000000 |
| DateType |
string |
The scheduling cycle. For example, |
YMD |
| CriticalThreshold |
number |
The threshold for a critical alert. For a strong rule, exceeding this threshold blocks the downstream scheduling task. |
0.6 |
| IsPrediction |
boolean |
Indicates whether the result is a prediction. Valid values:
|
false |
| RuleName |
string |
The name of the rule. |
The name of the rule. |
| CheckerId |
integer |
The ID of the checker. |
7 |
| DiscreteCheck |
boolean |
Indicates whether a discrete value check is used. Valid values:
|
true |
| EndTime |
integer |
The end time of the check. |
1600704000000 |
| MethodName |
string |
The method used to collect sample data. Examples: |
max |
| LowerValue |
number |
The predicted lower limit, which is automatically generated based on the configured threshold. |
2344 |
| EntityId |
integer |
The ID of the partition filter expression. |
14534343 |
| WhereCondition |
string |
The filter condition for the check. |
type!='type2' |
| ExpectValue |
number |
The expected value. |
90 |
| TemplateId |
integer |
The ID of the rule template. |
5 |
| TaskId |
string |
The ID of the check task. |
16008552981681a0d6**** |
| Id |
integer |
The primary key ID. |
2231123 |
| ReferenceValue |
array<object> |
The historical sample values. |
|
|
object |
A historical sample. |
||
| DiscreteProperty |
string |
The value of the sample column after being grouped by the |
type1 |
| Value |
number |
The sample value. |
20 |
| BizDate |
string |
The business date. For an offline table, this is typically the day before the check is performed. |
1600704000000 |
| SingleCheckResult |
integer |
The result of a single check. |
2 |
| Threshold |
number |
The threshold. |
0.2 |
| SampleValue |
array<object> |
The current sample values. |
|
|
object |
A current sample. |
||
| DiscreteProperty |
string |
The value of the sample column after being grouped by the |
type2 |
| BizDate |
string |
The business date. For an offline table, this is typically the day before the check is performed. |
1600704000000 |
| Value |
number |
The sample value. |
23 |
| Open |
boolean |
Indicates whether the rule is enabled. |
true |
Examples
Success response
JSON format
{
"HttpStatusCode": 200,
"ErrorMessage": "You have no permission.",
"RequestId": "CBA58543-00D4-41****",
"ErrorCode": "401",
"Success": true,
"Data": {
"PageNumber": 1,
"PageSize": 10,
"TotalCount": 200,
"RuleChecks": [
{
"BlockType": 1,
"WarningThreshold": 0.1,
"Property": "type",
"TableName": "dual",
"Comment": "The description of the rule.",
"CheckResultStatus": 2,
"TemplateName": "Expected value verification",
"CheckerName": "fulx",
"RuleId": 123421,
"FixedCheck": false,
"Op": ">",
"UpperValue": 22200,
"ActualExpression": "ds=20200925",
"ExternalId": "123112232",
"TimeCost": "10",
"Trend": "up",
"ExternalType": "CWF2",
"BizDate": 1600704000000,
"CheckResult": 2,
"ResultString": "ResultString",
"MatchExpression": "ds=$[yyyymmdd]",
"CheckerType": 0,
"ProjectName": "autotest",
"BeginTime": 1600704000000,
"DateType": "YMD",
"CriticalThreshold": 0.6,
"IsPrediction": false,
"RuleName": "The name of the rule.",
"CheckerId": 7,
"DiscreteCheck": true,
"EndTime": 1600704000000,
"MethodName": "max",
"LowerValue": 2344,
"EntityId": 14534343,
"WhereCondition": "type!='type2'",
"ExpectValue": 90,
"TemplateId": 5,
"TaskId": "16008552981681a0d6****",
"Id": 2231123,
"ReferenceValue": [
{
"DiscreteProperty": "type1",
"Value": 20,
"BizDate": "1600704000000",
"SingleCheckResult": 2,
"Threshold": 0.2
}
],
"SampleValue": [
{
"DiscreteProperty": "type2",
"BizDate": "1600704000000",
"Value": 23
}
],
"Open": true
}
]
}
}
Error codes
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.