This topic describes the event types, the formats of event messages, and the meaning of each field in an event message in DataWorks.
Background
In DataWorks, events are classified into regular events and extension point events based on whether the operation process related to an event in DataWorks can be blocked and whether the processing result of an event message can be returned after a self-managed service subscribes to the event message.
Regular events: You can subscribe to these event messages, but they do not support blocking operations in DataWorks. You can subscribe to these event messages by using OpenEvent. For more information, see Open events (OpenEvent).
Extension point events: After you subscribe to these event messages, you can customize the responses to the event messages by using the extension feature of the DataWorks open platform. When you use Extensions to control operations related to extension points, operations performed by DataWorks users on the platform for extension points are interrupted until your custom extension returns a processing result.
This list classifies the events supported by each module and the event types (regular events or extension point events) by module, and lists the EventBridge event type (Type) and extension event code (eventCode).
EventBridge event type (Type): The field used to filter event messages. For more information, see Enable message subscription.
Extension event code (eventCode): The event code that corresponds to the event type. During local development, this code can be obtained from the message content sent by DataWorks to determine the target event type.
Workspace-level events
Space-level modules generate space-level events, such as event messages produced by node execution in the Data Development module, event messages generated by status changes of recurring instances in the Operation Center module, and more. You can refer to the Event list below to understand which event messages supported by each module are regular events, which are extension point events, and the corresponding message format for sending these events.
The message format tabs below contain only partial content. For the complete messages sent to EventBridge or Function Compute, you need to refer to Appendix: Message formats.
DataStudio event list
Event list
Event type | Event (and operation that generates the event) | Regular event | Extension point event | EventBridge event type (Type) | Extension event code (eventCode) |
Node change | Add a node
|
|
|
|
|
Update a node
|
|
|
|
| |
File change
| Delete a file Delete a node in the development environment Note The deleted file will be moved to the recycle bin. |
|
|
|
|
Commit a file |
|
|
|
| |
Deploy a file |
|
|
|
| |
Run a file |
|
|
|
| |
Table change | Commit a table to the development environment |
|
|
|
|
Deploy a table to the production environment |
|
|
|
|
Message format
Node change events (add and update)
File change events (commit, deploy, run, and delete)
Table change events (commit a table to the development environment and deploy a table to the production environment)
Data Integration event list
Event list
Event type | Event (and operation that generates the event) | Regular event | Extension point event | EventBridge event type (Type) | Extension event code (eventCode) |
Node change event | Start a real-time task |
|
|
|
|
Start real-time tasks in batches |
|
|
|
|
Message format
Node change event-Start a real-time task
Node change event-Start real-time tasks in batches
Operation Center event list
Event list
Event type | Event (and operation that generates the event) | Regular event | Extension point event | EventBridge event type (Type) | Extension event code (eventCode) |
Node change | Delete a node
|
|
|
|
|
Undeploy a node Directly undeploy a node in the production environment Note This operation deletes the node in DataStudio and moves the node to the recycle bin. |
|
|
|
| |
Freeze a node |
|
|
|
| |
Unfreeze a node |
|
|
|
| |
Data backfilling for nodes | Data backfilling for nodes |
|
|
|
|
Node status change | Change the status of a scheduling task |
|
|
|
|
Instance change | Freeze an instance |
|
|
|
|
Unfreeze an instance |
|
|
|
| |
Terminate an instance |
|
|
|
| |
Rerun an instance
|
|
|
|
| |
Set the status of an instance to Succeeded |
|
|
|
| |
Remove specified upstream dependencies for an instance |
|
|
|
| |
Pre-event for freezing an instance |
|
|
|
| |
Pre-event for unfreezing an instance |
|
|
|
| |
Pre-event for rerunning an instance |
|
|
|
| |
Pre-event for setting the status of an instance to Succeeded |
|
|
|
| |
Pre-event for terminating an instance |
|
|
|
| |
Delete expired instances |
|
|
|
| |
Workflow status change | Change the status of a workflow
|
|
|
|
|
Monitoring alert | Monitoring alert |
|
|
|
|
Message format
Node change events (delete, undeploy, freeze, and unfreeze)
Data backfilling for nodes
Node status change
Instance change events (freeze, unfreeze, terminate, rerun, set to succeeded)
Instance change events (pre-freeze instance, pre-unfreeze instance, pre-rerun instance, pre-set instance to succeeded, pre-kill instance events)
Instance change events (freeze, unfreeze, terminate, rerun, set to Succeeded, and remove upstream dependencies)
Remove specified upstream dependency events for an instance
Workflow status change
Monitoring and alerts
Data Quality event list
Event list
Event type | Events (and operations that generate events) | Normal event | Extension point events | EventBridge event type (Type) | Extension program message type (eventCode) |
Approval center | Pre-event for creating permission request |
|
|
|
|
Complete request order |
|
|
|
| |
Security Center (table permission request) | Table permission request pre-event |
|
|
|
|
Message format
Approval center (Create and complete request orders)
Security Center (Table permission request pre-event)
Appendix: Authorization content list (MaxCompute)
Appendix: Authorization content list (DataService)
DataService Studio event list
Event list
Event type | Events (and operations that generate events) | Normal event | Extension point event | EventBridge event type (Type) | Extension program event type (eventCode) |
Data quality check | Feedback on check results
|
|
|
| |
Check completed
|
|
|
|
| |
Data Quality rule prerequisite | Batch create Data Quality rule prerequisite event |
|
|
|
|
Batch update Data Quality rule prerequisite event |
|
|
|
| |
Batch delete data quality rule event |
|
|
|
| |
Update data quality rule prerequisite event |
|
|
|
| |
Data Quality monitoring prerequisite | Create Data Quality monitoring prerequisite event |
|
|
|
|
Update Data Quality monitoring prerequisite event |
|
|
|
| |
Clone Data Quality monitoring event |
|
|
|
| |
Batch delete data quality monitoring prerequisite events |
|
|
|
| |
Data quality monitoring alert subscription prerequisite | Create data quality monitoring alert subscription prerequisite event |
|
|
| create-data-quality-evaluation-task-notification |
Update data quality monitoring alert subscription prerequisite events |
|
|
|
| |
Delete Data Quality monitoring alert subscription event |
|
|
|
|
Message format
API call
Data Quality rule pre-events (batch create data quality rules, batch update data quality rules, batch delete data quality rules, update data quality rule)
Data quality monitoring pre-events (Create data quality monitoring, Update data quality monitoring, Clone data quality monitoring, Batch delete data quality monitoring)
Data quality monitoring alert subscription pre-events (Create data quality monitoring alert subscription, Update data quality monitoring alert subscription, Delete data quality monitoring alert subscription)
Tenant-level events
Tenant-level modules generate tenant-level events, such as event messages generated when a workspace is deleted from the console. You can refer to the event list below to understand which event messages supported by each module are regular events, which are extension point events, and the corresponding message formats for these events.
The following message format tabs only contain partial content. For the complete message sent to EventBridge or Function Compute, you need to refer to Appendix: Message formats.
Console event list
Event list
Event type | Event (and operation that generates the event) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Delete a project | Delete a workspace (pre-event) |
|
|
|
|
Delete a workspace (post-event) |
|
|
|
|
Message format
Message entity format: Delete project event
Upload and download event list
Event list
Event type | Event (and operation that generates the event) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Data download and upload | Download data (pre-event) - file generation |
|
| dataworks:ResourcesDownload:DownloadResources | download-resources |
Download data (pre-event) - file download |
|
| dataworks:ResourcesDownload:DownloadResourcesExecute | download-resources-execute | |
Upload data (pre-event) |
|
| dataworks:ResourcesUpload:UploadDataToTable | upload-data-to-table |
Message format
Data download and upload
Appendix: Message formats
Message formats sent to EventBridge
After configuring an event distribution channel in Open Event (OpenEvent), when a workspace-level or tenant-level event is triggered in DataWorks, the event is filtered based on the event type (Type) configured in EventBridge. The following shows the message format that DataWorks sends to EventBridge through the event distribution channel.
{
"datacontenttype": "application/json;charset=utf-8",// The content format of data. datacontenttype supports only the application/json content type.
"data": {
// The message content varies based on the message type. The following fields are the fixed fields in an event message. For information about the message content of each type of event, see the preceding information.
"tenantId": 28378****10656,// The tenant ID. Each Alibaba Cloud account in DataWorks corresponds to a tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of the DataWorks Data Development page.
"eventCode": "xxxx"//
},
"id": "539fd8f4-4ea1-4625-aa8b-6c906674****",// The event ID. The ID is the unique identifier of an event.
"source": "acs.dataworks",// The event source, which indicates the service that produces events. In this example, event messages are pushed by DataWorks.
"specversion": "1.0",
"subject": "",
"time": "2020-11-19T21:04:41+08:00",// The event generation time.
"type": "dataworks:InstanceStatusChanges:InstanceStatusChanges",// The event type. The Type field is used in the EventBridge console to filter all event messages pushed by DataWorks. Each event has a different event type value. You can refer to the preceding information to obtain the event type.
"aliyunaccountid": "123456789098****",// The ID of the Alibaba Cloud account.
"aliyunpublishtime": "2020-11-19T21:04:42.179PRC",// The time when EventBridge receives an event.
"aliyuneventbusname": "default",// The name of the EventBridge event bus used to receive DataWorks event messages.
"aliyunregionid": "cn-hangzhou",// The region where events are received.
"aliyunpublishaddr": "172.25.XX.XX"
}A complete event message contains information such as the body, ID, source, and generation time of the event message. The following table describes the key fields in the preceding sample code.
Field name | Field type | Description |
data | object | The structure of a message body. The structure of a message body and the fields contained in a message body vary based on the type of the event. For more information, see the following sections in this topic: Message format of Data Development events: Data Development event list. Message format of Data Integration events: Data Integration event list. Message format of Operation Center events: Operation Center event list. Message format of Security Center events: Security Center event list. Message format of Data Quality events: Data Quality event list. |
id | String | The unique ID of the event message. You can identify an event message based on the ID of the event message. |
type | String | The event type. This field is used to describe the event type of the event source. Sample values:
The Type field is used in the EventBridge console to filter all event messages pushed by DataWorks. Each event has a different event type value. You can refer to the preceding information to obtain the event type. |
Formats of event messages sent to Function Compute
When you configure Extensions: Function Compute in DataWorks, DataWorks sends the messages of the triggered extension point events to Function Compute in JSON format. The following shows the format of event messages sent to Function Compute.
{
"blockBusiness": true,
"eventCategoryType": "resources-download",// The event category.
"eventType": "upload-data-to-table",// The event type.
"extensionBizId": "job_6603***070",
"messageBody": {
// The message content varies based on the message type. The following fields are the fixed fields in an event message. For information about the message content of each type of event, see the preceding information.
"tenantId": 28378****10656,// The tenant ID. Each Alibaba Cloud account in DataWorks corresponds to a tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of the DataWorks Data Development page.
"eventCode": "xxxx"//
},
"messageId": "52d44ee7-b51f-4d4d-afeb-*******"// The event ID. The ID is the unique identifier of an event.
}The following table describes the fields in the preceding sample code.
Field name | Field type | Description | |
messageId | String | The ID of the event. The ID is the unique identifier of an event. | |
messageBody | The specific event message pushed by DataWorks. This field is used when an extension is developed. The field value varies based on the message type. | ||
tenantId | The tenant ID. Each Alibaba Cloud account in DataWorks corresponds to a tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of the DataWorks Data Development page | ||
eventCode | The code of the security event. This field is used to identify a type of event message. For the eventCode that corresponds to each event type, see the Extension event type (eventCode) column in the Development reference: Event lists and event message formats table. | ||