Queries monitoring results after the data quality of a data source or a compute engine instance is monitored based on monitoring rules.
Debugging
Request parameters
Parameter | Type | Required | Example | Description |
---|---|---|---|---|
Action | String | Yes | ListQualityResultsByRule | The operation that you want to perform. |
RuleId | Long | Yes | 152322134 | The ID of the monitoring rule. You can use the ID and information such as a partition filter expression to perform a joint query. |
StartDate | String | Yes | 2020-09-20 00:00:00 | The beginning of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. This parameter is used together with the EndDate parameter. The interval between the time specified by this parameter and the time specified by the EndDate parameter cannot exceed 7 days. |
EndDate | String | Yes | 2020-09-22 00:00:00 | The end of the time range to query. Specify the time in the yyyy-MM-dd HH:mm:ss format. This parameter is used together with the StartDate parameter. The interval between the time specified by this parameter and the time specified by the StartDate parameter cannot exceed 7 days. |
PageSize | Integer | Yes | 10 | The number of entries per page. Default value: 10. Maximum value: 100. |
PageNumber | Integer | Yes | 1 | The page number. |
ProjectName | String | Yes | autotest | The name of the compute engine instance or data source for which data quality is monitored. |
ProjectId | Long | No | 12345 | The ID of the workspace. |
Response parameters
Parameter | Type | Example | Description |
---|---|---|---|
HttpStatusCode | Integer | 200 | The HTTP status code returned. |
ErrorMessage | String | You have no permission. | The error message returned. |
RequestId | String | CBA58543-00D4-41**** | The request ID. |
ErrorCode | String | 401 | The error code returned. |
Success | Boolean | true | Indicates whether the request was successful. |
Data | Object | The data structure of the monitoring results returned. |
|
PageNumber | Integer | 1 | The page number. |
PageSize | Integer | 10 | The number of entries per page. Default value: 10. Maximum value: 100. |
TotalCount | Long | 200 | The total number of entries returned. |
RuleChecks | Array of RuleChecks | The monitoring results returned. |
|
BlockType | Integer | 1 | The type of the monitoring rule. The type of a monitoring rule indicates the importance of the rule. Valid values:
|
WarningThreshold | Float | 0.1 | The threshold for a warning alert. The threshold indicates the deviation of the monitoring result from the expected value. You can customize this threshold based on your business requirements. |
Property | String | type | The field of the rule attribute. This field is the column name of the data source table that is monitored. |
TableName | String | dual | The name of the table that is monitored. |
Comment | String | Description of the monitoring rule | The description of the monitoring rule. |
CheckResultStatus | Integer | 2 | The status of the monitoring result. |
TemplateName | String | Compare the Number of Unique Field Values Against Expectation | The name of the monitoring template. |
CheckerName | String | fulx | The name of the checker. |
RuleId | Long | 123421 | The ID of the monitoring rule. |
FixedCheck | Boolean | false | Indicates whether the monitoring is performed based on a fixed value. Valid values:
|
Op | String | > | The comparison operator. |
UpperValue | Float | 22200 | The upper limit of the predicted result. The value of this parameter is automatically generated based on the threshold that you specified. |
ActualExpression | String | ds=20200925 | The partition in the monitored data source table. |
ExternalId | String | 123112232 | The ID of the node. |
TimeCost | String | 10 | The time that was taken to run the monitoring task. |
Trend | String | up | The trend of the monitoring result. |
ExternalType | String | CWF2 | The type of the scheduling system. Only CWF scheduling systems are supported. |
BizDate | Long | 1600704000000 | The date on which the business was performed. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
CheckResult | Integer | 2 | The monitoring result. |
ResultString | String | ResultString | The string of the monitoring result. |
MatchExpression | String | ds=$[yyyymmdd] | The partition filter expression. |
CheckerType | Integer | 0 | The type of the checker. |
ProjectName | String | autotest | The name of the compute engine instance or data source for which data quality is monitored. |
BeginTime | Long | 1600704000000 | The time when the monitoring started. |
DateType | String | YMD | The type of the scheduling cycle. In most cases, the value of this parameter is YMD. This value indicates year, month, and day. |
CriticalThreshold | Float | 0.6 | The threshold for a critical alert. The threshold indicates the deviation of the monitoring result from the expected value. You can customize this threshold based on your business requirements. If a strong rule is used and a critical alert is reported, nodes are blocked. |
IsPrediction | Boolean | false | Indicates whether the monitoring result is the same as the predicted result. Valid values:
|
RuleName | String | Name of the monitoring rule | The name of the monitoring rule. |
CheckerId | Integer | 7 | The ID of the checker. |
DiscreteCheck | Boolean | true | Indicates whether the monitoring is discrete monitoring. Valid values:
|
EndTime | Long | 1600704000000 | The time when the monitoring ended. |
MethodName | String | max | The method used to collect sample data, such as avg, count, sum, min, max, count_distinct, user_defined, table_count, table_size, table_dt_load_count, table_dt_refuseload_count, null_value, null_value/table_count, (table_count-count_distinct)/table_count, or table_count-count_distinct. |
LowerValue | Float | 2344 | The lower limit of the predicted result. The value of this parameter is automatically generated based on the threshold that you specified. |
EntityId | Long | 14534343 | The ID of the partition filter expression. |
WhereCondition | String | type!='type2' | The filter condition of the monitoring rule. |
ExpectValue | Float | 90 | The expected value. |
TemplateId | Integer | 5 | The ID of the monitoring template. |
TaskId | String | 16008552981681a0d6**** | The ID of the monitoring task. |
Id | Long | 2231123 | The ID of the primary key. |
ReferenceValue | Array of ReferenceValue | The historical sample values. |
|
DiscreteProperty | String | type1 | The values of the sample field that are grouped by using the GROUP BY clause. For example, the values of the Gender field are grouped by using the GROUP BY clause. In this case, the values of DiscreteProperty are Male, Female, and null. |
Value | Float | 20 | The sample value. |
BizDate | String | 1600704000000 | The date on which the business was performed. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
SingleCheckResult | Integer | 2 | The string of the monitoring result. |
Threshold | Float | 0.2 | The threshold. |
SampleValue | Array of SampleValue | The sample values. |
|
DiscreteProperty | String | type2 | The values of the sample field that are grouped by using the GROUP BY clause. For example, the values of the Gender field are grouped by using the GROUP BY clause. In this case, the values of DiscreteProperty are Male, Female, and null. |
BizDate | String | 1600704000000 | The date on which the business was performed. If the monitored business entity is offline data, the value is usually one day before the monitoring is performed. |
Value | Float | 23 | The sample value. |
Open | Boolean | true | Indicates whether the monitoring rule is enabled. |
Examples
Sample requests
http(s)://[Endpoint]/?Action=ListQualityResultsByRule
&EndDate=2020-09-22 00:00:00
&PageNumber=1
&PageSize=10
&ProjectName=autotest
&RuleId=152322134
&StartDate=2020-09-20 00:00:00
&<Common request parameters>
Sample success responses
XML
format
HTTP/1.1 200 OK
Content-Type:application/xml
<ListQualityResultsByRuleResponse>
<RequestId>CBA58543-00D4-41****</RequestId>
<QualityResults>
<TotalCount>200</TotalCount>
<PageSize>10</PageSize>
<PageNumber>1</PageNumber>
<RuleChecks>
<Op>&gt;</Op>
<TaskId>16008552981681a0d6****</TaskId>
<EndTime>1600704000000</EndTime>
<DiscreteCheck>true</DiscreteCheck>
<CheckerName>fulx</CheckerName>
<ExpectValue>90</ExpectValue>
<ProjectName>autotest</ProjectName>
<RuleId>123421</RuleId>
<WarningThreshold>0.1</WarningThreshold>
<WhereCondition>type!='type2'</WhereCondition>
<BeginTime>1600704000000</BeginTime>
<CheckResultStatus>2</CheckResultStatus>
<MatchExpression>ds=$[yyyymmdd]</MatchExpression>
<UpperValue>22200</UpperValue>
<IsPrediction>false</IsPrediction>
<CheckResult>2</CheckResult>
<TemplateName>Compare the Number of Unique Field Values Against Expectation</TemplateName>
<TimeCost>10</TimeCost>
<FixedCheck>false</FixedCheck>
<MethodName>max</MethodName>
<ResultString>ResultString</ResultString>
<Property>type</Property>
<DateType>YMD</DateType>
<BizDate>1600704000000</BizDate>
<TemplateId>5</TemplateId>
<RuleName>Name of the monitoring rule</RuleName>
<Comment>Description of the monitoring rule</Comment>
<CriticalThreshold>0.6</CriticalThreshold>
<ExternalId>123112232</ExternalId>
<Trend>up</Trend>
<CheckerType>0</CheckerType>
<ExternalType>CWF2</ExternalType>
<CheckerId>7</CheckerId>
<EntityId>14534343</EntityId>
<TableName>dual</TableName>
<BlockType>1</BlockType>
<Id>2231123</Id>
<ActualExpression>ds=20200925</ActualExpression>
<LowerValue>2344</LowerValue>
</RuleChecks>
<RuleChecks>
<SampleValue>
<SingleCheckResult>2</SingleCheckResult>
<Value>20</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type1</DiscreteProperty>
<Threshold>0.2</Threshold>
</SampleValue>
<SampleValue>
<Value>23</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type2</DiscreteProperty>
</SampleValue>
<ReferenceValue>
<SingleCheckResult>2</SingleCheckResult>
<Value>20</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type1</DiscreteProperty>
<Threshold>0.2</Threshold>
</ReferenceValue>
<ReferenceValue>
<Value>23</Value>
<BizDate>1600704000000</BizDate>
<DiscreteProperty>type2</DiscreteProperty>
</ReferenceValue>
</RuleChecks>
</QualityResults>
</ListQualityResultsByRuleResponse>
JSON
format
HTTP/1.1 200 OK
Content-Type:application/json
{
"RequestId" : "CBA58543-00D4-41****",
"QualityResults" : {
"TotalCount" : 200,
"PageSize" : 10,
"PageNumber" : 1,
"RuleChecks" : [ {
"Op" : "&gt;",
"TaskId" : "16008552981681a0d6****",
"EndTime" : 1600704000000,
"DiscreteCheck" : true,
"CheckerName" : "fulx",
"ExpectValue" : 90,
"ProjectName" : "autotest",
"RuleId" : 123421,
"WarningThreshold" : 0.1,
"WhereCondition" : "type!='type2'",
"BeginTime" : 1600704000000,
"CheckResultStatus" : 2,
"MatchExpression" : "ds=$[yyyymmdd]",
"UpperValue" : 22200,
"IsPrediction" : false,
"CheckResult" : 2,
"TemplateName": "Compare the Number of Unique Field Values Against Expectation",
"TimeCost" : 10,
"FixedCheck" : false,
"MethodName" : "max",
"ResultString" : "ResultString",
"Property" : "type",
"DateType" : "YMD",
"BizDate" : 1600704000000,
"TemplateId" : 5,
"RuleName": "Name of the monitoring rule",
"Comment": "Description of the monitoring rule",
"CriticalThreshold" : 0.6,
"ExternalId" : 123112232,
"Trend" : "up",
"CheckerType" : 0,
"ExternalType" : "CWF2",
"CheckerId" : 7,
"EntityId" : 14534343,
"TableName" : "dual",
"BlockType" : 1,
"Id" : 2231123,
"ActualExpression" : "ds=20200925",
"LowerValue" : 2344
}, {
"SampleValue" : [ {
"SingleCheckResult" : 2,
"Value" : 20,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type1",
"Threshold" : 0.2
}, {
"Value" : 23,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type2"
} ],
"ReferenceValue" : [ {
"SingleCheckResult" : 2,
"Value" : 20,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type1",
"Threshold" : 0.2
}, {
"Value" : 23,
"BizDate" : 1600704000000,
"DiscreteProperty" : "type2"
} ]
} ]
}
}
Error codes
For a list of error codes, see Service error codes.