ListDataQualityScans - DataWorks - Alibaba Cloud Documentation Center

Queries the list of data quality scan tasks in a project.

Operation description

DataWorks Basic Edition or a higher edition is required.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

dataworks:ListDataQualityScans

list

*All Resource

*

None

Request syntax

GET  HTTP/1.1

Request parameters

Parameter	Type	Required	Description	Example
Name	string	No	The data quality scan task name for fuzzy match.	test
SortBy	string	No	The list of sorting fields. Supports fields such as last modified time and creation time. Format: "SortField+SortOrder (Desc/Asc)", where Asc is the default. Valid values: ModifyTime (Desc/Asc) CreateTime (Desc/Asc) Id (Desc/Asc)	ModifyTime Desc
PageSize	integer	Yes	The number of entries per page. Default value: 10.	10
PageNumber	integer	Yes	The page number. Default value: 1.	1
ProjectId	integer	Yes	The project ID.	10000
Table	string	No	Fuzzy match for the monitored table name.	video_album

Response elements

Element	Type	Description	Example
	object	Schema of Response
RequestId	string	The API request ID, which is generated as a UUID.	0bc14115***159376359
PageInfo	object	The page information.
PageNumber	integer	The page number.	1
PageSize	integer	The number of records per page. Default value: 10.	10
TotalCount	integer	The total number of records returned.	1
DataQualityScans	array<object>	The list of data quality monitors.
	array<object>	Information about the data quality monitor.
ModifyTime	integer	Last update time of the data quality monitor.	17236236472
Owner	string	The user ID of the owner responsible for the data quality monitor.	23782382795249
Description	string	The description of the data quality scan task. Maximum length: 65,535 characters.	This is a hourly run data quality evaluation plan.
ModifyUser	string	The user ID of the last person who updated the data quality monitor.	23782382795249
Parameters	array<object>	Execution parameter definitions for the data quality monitor.
	object	Execution parameter definitions for the data quality monitor.
Value	string	The parameter value.	$[yyyy-mm-dd-1]
Name	string	The parameter name.	dt
CreateTime	integer	The creation time of the data quality monitor.	1694512304000
ProjectId	integer	The project ID.	59094
CreateUser	string	The creator of the data quality monitor.	7892346529452
ComputeResource	object	The compute engine used during execution. If it is not specified, the data source connection defined in the Spec will be used.
Runtime	object	Additional runtime settings for the data quality monitor.
HiveConf	string	Additional parameters for the Hive engine. Currently, only mapreduce.job.queuename is supported to set the queue.	mapreduce.job.queuename=dq_queue
SparkConf	string	Additional parameters for the Spark engine. Currently, only spark.yarn.queue is supported to set the queue.	spark.yarn.queue=dq_queue
Engine	string	The engine type. These settings are only supported for the EMR compute engine. Valid values: Hive: Hive SQL Spark: Spark SQL Kyuubi	Hive
EnvType	string	Workspace environment of the compute engine. Valid values: Prod Dev	Prod
Name	string	The name of the computing engine. Uniquely identifies the engine.	emr_cluster_001
Name	string	The name of the data quality scan task. Can include digits, letters, Chinese characters, and both half-width and full-width punctuation marks. Maximum length: 255 characters.	Hourly partition quality monitoring
RuntimeResource	object	The resource group used during the execution of the data quality monitor.
Cu	number	CU consumption for task running.	0.25
Id	string	The ID of the resource group.	Serverless_resource_group_xxxxx
Image	string	The ID of the image configured for task running.	i-xxxxx
Trigger	object	Trigger settings for the data quality monitor.
Type	string	The trigger mode of the data quality monitor. Valid values: ByManual: Manually triggered. Default setting. BySchedule: Triggered by a scheduled task instance.	BySchedule
TaskIds	array	If the trigger mode is BySchedule, the ID of the scheduling task that triggers the monitor must be configured.
	integer	The scheduling task ID.	1023777390
Hooks	array<object>	The hook configuration after the data quality monitor stops.
	object	The hook configuration after the data quality monitor stops.
Condition	string	The hook trigger condition. When this condition is met, the hook is triggered. Valid expression format: Specifies multiple combinations of rule severity levels and rule validation statuses, such as `results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }`. This means the hook is triggered if any executed rule has Fail with Normal severity, Error with High severity, or Warn with High severity. The severity values must match those defined in the Spec. The status values must match those in DataQualityResult.	results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }
Type	string	The type of the hook. Valid values: BlockTaskInstance: Blocks the scheduling of the task instance.	BlockTaskInstance
Id	integer	The ID of the data quality monitor.	26433

Examples

Success response

JSON format

{
  "RequestId": "0bc14115***159376359",
  "PageInfo": {
    "PageNumber": 1,
    "PageSize": 10,
    "TotalCount": 1,
    "DataQualityScans": [
      {
        "ModifyTime": 17236236472,
        "Owner": "23782382795249",
        "Description": "This is a hourly run data quality evaluation plan.",
        "ModifyUser": "23782382795249",
        "Parameters": [
          {
            "Value": "$[yyyy-mm-dd-1]",
            "Name": "dt"
          }
        ],
        "CreateTime": 1694512304000,
        "ProjectId": 59094,
        "CreateUser": "7892346529452",
        "ComputeResource": {
          "Runtime": {
            "HiveConf": "mapreduce.job.queuename=dq_queue",
            "SparkConf": "spark.yarn.queue=dq_queue",
            "Engine": "Hive"
          },
          "EnvType": "Prod",
          "Name": "emr_cluster_001"
        },
        "Name": "Hourly partition quality monitoring",
        "RuntimeResource": {
          "Cu": 0.25,
          "Id": "Serverless_resource_group_xxxxx",
          "Image": "i-xxxxx"
        },
        "Trigger": {
          "Type": "BySchedule",
          "TaskIds": [
            1023777390
          ]
        },
        "Hooks": [
          {
            "Condition": "results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' || r.status == 'Error' && r.rule.severity == 'High' || r.status == 'Warn' && r.rule.severity == 'High' }",
            "Type": "BlockTaskInstance"
          }
        ],
        "Id": 26433
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.