GetDataQualityScan - DataWorks - Alibaba Cloud Documentation Center

Gets data quality monitoring details.

Operation description

DataWorks Basic Edition or a higher edition is required.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

dataworks:GetDataQualityScan

get

*All Resource

*

None

Request syntax

POST  HTTP/1.1

Request parameters

Parameter	Type	Required	Description	Example
Id	integer	No	The ID of the monitor.	10001

Response elements

Element	Type	Description	Example
	object	Schema of Response
RequestId	string	Id of the request	204EAF68-CCE3-5112-8DA0-E7A60F02XXXX
DataQualityScan	object	Data quality monitoring details.
Id	integer	The data quality monitoring ID.	10001
Name	string	The data quality monitor name.	data_quality_scan_001
Description	string	The data quality monitor description.	aily data quality scanning of ods tables.
ProjectId	integer	The workspace ID where the data quality monitor resides. You can obtain the workspace ID by calling the ListProjects operation.	101
CreateTime	integer	The creation time of the data quality monitor.	1731550150000
ModifyTime	integer	Last modified time of the data quality monitor.	1731550150000
CreateUser	string	The ID of the user who creates the data quality monitor.	2374924198591250
ModifyUser	string	The ID of the user who last modifies the data quality monitor.	23482597582479
Owner	string	The ID of the user who owns the data quality monitor.	231263586109857423
Spec	string	Spec code for the content of the data quality monitoring.	{ "datasets": [ { "type": "Table", "dataSource": { "name": "odps_first", "envType": "Prod" }, "tables": [ "ods_d_user_info" ], "filter": "pt = $[yyyymmdd-1]" } ], "rules": [ { "assertion": "row_count > 0" }, { "templateId": "SYSTEM:field:null_value:fixed", "pass": "when = 0", "name": "The id cannot be empty.", "severity": "High", "identity": "a-customized-data-quality-rule-uuid" } ] }
Parameters	array<object>	The definition of execution parameters for the data quality monitor.
	object	The parameter information.
Value	string	The parameter name.	10
Name	string	The parameter value.	e2e_autolabel
ComputeResource	object	The compute engine used at runtime. Optional. If not specified, the data source defined in the Spec is used.
Name	string	The name of the compute engine, which is a unique identifier.	polardb_to_holo
Runtime	object	More settings for data quality monitor at runtime.
Engine	string	The type of the compute engine. Only EMR compute engines support these settings. Valid values: Hive: Hive SQL Spark: Spark SQL Kyuubi	Hive
HiveConf	object	Additional Hive engine parameters. Currently, only the mapreduce.job.queuename parameter is supported.	mapreduce.job.queuename=dq_queue
SparkConf	object	Additional Spark engine parameters. Currently, only the spark.yarn.queue parameter is supported.	spark.yarn.queue=dq_queue
EnvType	string	The workspace environment to which the compute engine belongs. Valid values: Prod: production environment . Dev: development environment.	Prod
RuntimeResource	object	The resource group used during the running of the data quality monitor.
Id	string	The resource group ID.	122878
Cu	number	Reserved compute units (CU) for the resource group.	10
Image	string	The image ID used in the runtime configuration.	hp-tlp-e2e-repo-registry-vpc.cn-heyuan-acdr-1.cr.aliyuncs.com/hp-service/worker:9b28b6d-202506091008
Trigger	object	The trigger configurations of the data quality monitoring task.
Type	string	The trigger mode of the monitoring task. Valid values: ByManual: Manual trigger. This is the default setting. BySchedule: Triggered by a scheduled task instance.	BySchedule
TaskIds	array	If the trigger mode is set to BySchedule, the scheduling task ID must be specified.
	integer	The ID of the scheduling task.	1023777390
Hooks	array<object>	The Hook configurations after the data quality monitoring run ends.
	object	The Hook configurations after the data quality monitoring run ends.
Condition	string	The Hook trigger condition. The hook will run if the condition is met. Currently, only one type of expression syntax is supported: You can specify multiple combinations of rule severity levels and validation statuses using an expression such as `results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }`. This expression means the condition is met if any executed rule has a result of Fail with severity Normal, Error with severity High, or Warn with severity High. In the condition expression, the values of severity and status are predefined enums. The values of severity must match those defined in the Spec, and the values of status must match those in DataQualityResult.	results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' \|\| r.status == 'Error' && r.rule.severity == 'High' \|\| r.status == 'Warn' && r.rule.severity == 'High' }
Type	string	The type of the Hook. Valid values: BlockTaskInstance: BlockTaskInstance: Blocks the scheduling of the task instance.	BlockTaskInstance

Examples

Success response

JSON format

{
  "RequestId": "204EAF68-CCE3-5112-8DA0-E7A60F02XXXX",
  "DataQualityScan": {
    "Id": 10001,
    "Name": "data_quality_scan_001",
    "Description": "aily data quality scanning of ods tables.",
    "ProjectId": 101,
    "CreateTime": 1731550150000,
    "ModifyTime": 1731550150000,
    "CreateUser": "2374924198591250",
    "ModifyUser": "23482597582479",
    "Owner": "231263586109857423",
    "Spec": "{\n    \"datasets\": [\n        {\n            \"type\": \"Table\",\n            \"dataSource\": {\n                \"name\": \"odps_first\",\n                \"envType\": \"Prod\"\n            },\n            \"tables\": [\n                \"ods_d_user_info\"\n            ],\n            \"filter\": \"pt = $[yyyymmdd-1]\"\n        }\n    ],\n    \"rules\": [\n        {\n            \"assertion\": \"row_count > 0\"\n        }, {\n            \"templateId\": \"SYSTEM:field:null_value:fixed\",\n            \"pass\": \"when = 0\",\n            \"name\": \"The id cannot be empty.\",\n            \"severity\": \"High\",\n             \"identity\": \"a-customized-data-quality-rule-uuid\"\n        }\n    ]\n}",
    "Parameters": [
      {
        "Value": "10",
        "Name": "e2e_autolabel"
      }
    ],
    "ComputeResource": {
      "Name": "polardb_to_holo",
      "Runtime": {
        "Engine": "Hive",
        "HiveConf": {
          "test": "test",
          "test2": 1
        },
        "SparkConf": {
          "test": "test",
          "test2": 1
        }
      },
      "EnvType": "Prod"
    },
    "RuntimeResource": {
      "Id": "122878",
      "Cu": 10,
      "Image": "hp-tlp-e2e-repo-registry-vpc.cn-heyuan-acdr-1.cr.aliyuncs.com/hp-service/worker:9b28b6d-202506091008"
    },
    "Trigger": {
      "Type": "BySchedule",
      "TaskIds": [
        1023777390
      ]
    },
    "Hooks": [
      {
        "Condition": "results.any { r -> r.status == 'Fail' && r.rule.severity == 'Normal' || r.status == 'Error' && r.rule.severity == 'High' || r.status == 'Warn' && r.rule.severity == 'High' }",
        "Type": "BlockTaskInstance"
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.