CreateDataQualityScan - DataWorks - Alibaba Cloud Documentation Center

Creates a data quality monitor.

Operation description

DataWorks Basic Edition or a higher edition is required.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

No authorization for this operation. If you encounter issues with this operation, contact technical support.

Request syntax

POST  HTTP/1.1

Request parameters

Parameter	Type	Required	Description	Example
Description	string	No	The description of the data quality monitor.	Daily data quality scanning of ods tables.
ProjectId	integer	No	The DataWorks workspace ID. You can log on to the DataWorks console and go to the workspace configuration page to obtain the workspace ID. This parameter is required to specify the target DataWorks workspace for this API operation.	101
Owner	string	No	The ID of the user who owns of the data quality monitor.	95279527****
Spec	string	No	Spec code for the content of the data quality monitoring.	{ "datasets": [ { "type": "Table", "dataSource": { "name": "odps_first", "envType": "Prod" }, "tables": [ "ods_d_user_info" ], "filter": "pt = $[yyyymmdd-1]" } ], "rules": [ { "assertion": "row_count > 0" }, { "templateId": "SYSTEM:field:null_value:fixed", "pass": "when = 0", "name": "The id cannot be empty.", "severity": "High", "identity": "a-customized-data-quality-rule-uuid" } ] }
Parameters	array<object>	No	The definition of execution parameters for the data quality monitoring.
	object	No	The definition of execution parameters for the data quality monitoring.
Value	string	No	The parameter values.	$[yyyymmdd-1]
Name	string	No	The parameter name.	triggerTime
RuntimeResource	object	No	The resource group used during execution of the data quality monitoring.
Id	string	No	The resource group ID.	0525242e-d0ee-4bda-bc73-765d82f6a34a
Cu	number	No	The default number of CUs configured for task running.	0.25
Image	string	No	The ID of the image configured for task running.	i-xxxxxx
ComputeResource	object	No	The compute engine used at runtime. If not specified, the data source defined in the Spec is used.
Name	string	No	The name of the compute engine, which is a unique identifier.	emr_first
Runtime	object	No	More settings for data quality monitoring at runtime.
Engine	string	No	The type of the compute engine. Only EMR compute engines support these settings. Valid values: Hive: Hive SQL Spark: Spark SQL Kyuubi	Hive
SparkConf	object	No	Additional Spark engine parameters. Currently, only the spark.yarn.queue parameter is supported.	spark.yarn.queue=dq_queue
HiveConf	object	No	Additional Hive engine parameters. Currently, only the mapreduce.job.queuename parameter is supported.	mapreduce.job.queuename=dq_queue
EnvType	string	No	The workspace environment to which the compute engine belongs. Valid values: Prod: production environment . Dev: development environment.	Dev
Trigger	object	No	The trigger configurations of the data quality monitoring task.
Type	string	No	The trigger mode of the monitoring task. Valid values: ByManual: Manual trigger. This is the default setting. BySchedule: Triggered by a scheduled task instance.	BySchedule
TaskIds	array	No	If the trigger mode is set to BySchedule, the scheduling task ID must be specified.
	integer	No	The ID of the scheduling task.	1004979033
Hooks	array<object>	No	The Hook configurations after the data quality monitoring run ends.
	object	No	The Hook configurations after the data quality monitoring run ends.
Condition	string	No	The Hook trigger condition. The hook will run if the condition is met. Currently, only one type of expression syntax is supported: You can specify multiple combinations of rule severity levels and validation statuses using an expression such as `results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }`. This expression means the condition is met if any executed rule has a result of Fail with severity Normal, Error with severity High, or Warn with severity High. In the condition expression, the values of severity and status are predefined enums. The values of severity must match those defined in the Spec, and the values of status must match those in DataQualityResult.	results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }
Type	string	No	The type of the Hook. Valid values: BlockTaskInstance: Blocks the scheduling of the task instance.	BlockTaskInstance
Name	string	No	The data quality monitoring name.	data_quality_scan_001
ClientToken	string	Yes	The idempotency token.	a-customized-uuid

Response elements

Element	Type	Description	Example
	object	Schema of Response
RequestId	string	Id of the request	0000-ABCD-EFG****
Id	integer	Returns the ID of the created data source sharing rule, which uniquely identifies the rule.	676303114031776

Examples

Success response

JSON format

{
  "RequestId": "0000-ABCD-EFG****",
  "Id": 676303114031776
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.