This topic describes the event types and message formats in DataWorks and explains the fields in the messages. Use this topic to quickly find information about event lists and message formats.
Background
In DataWorks, events are classified as regular events or extension point events. This classification depends on whether an event can block an internal DataWorks operation and whether a self-managed service can return a processing result after subscribing to the event.
Regular events: You can subscribe to messages for these events, but you cannot block internal DataWorks operations. You can subscribe to these event messages through OpenEvent. For more information, see OpenEvent.
Extension point events: After you subscribe to messages for these events, you can customize the response to the event messages using the Extensions feature of the DataWorks open platform. When you use Extensions to control operations at an extension point, the operations are blocked in the DataWorks platform until your custom extension returns a processing result.
This topic lists the events supported by each module, their types (regular event or extension point event), their EventBridge event type (Type), and their extension event code (eventCode).
EventBridge event type (Type): A field used to filter event messages. For more information, see Enable message subscription.
Extension event code (eventCode): The event code that corresponds to an event type. When you develop locally, you can retrieve this code from the content of the message sent by DataWorks to identify the target event type.
Workspace-level events
Workspace-level modules generate workspace-level events. Examples include event messages generated when a node runs in Data Studio and event messages generated when the status of a recurring instance changes in Operation Center. The following Event list shows which events are regular events and which are extension point events for each module, along with the message format for each event.
The following Message format tabs show only partial content. For the complete message sent to EventBridge or Function Compute, see Appendix: Message format.
Data Development Event List
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event code (eventCode) |
Node change | Create node
|
|
|
|
|
Update node
|
|
|
|
| |
File change
| Pre-delete file event Delete a node in the development environment Note The deleted file is moved to the recycle bin. |
|
|
|
|
Pre-commit file event |
|
|
|
| |
Pre-publish file event |
|
|
|
| |
Pre-run code event |
|
|
|
| |
Table change | Pre-processing form submission event |
|
|
|
|
Pre-publish table to production environment event |
|
|
|
|
Message format
Node change events (create, update)
File change events (commit, publish, run, delete)
Table change events (commit table to development environment, publish table to production environment)
Data integration event list
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Node change event | Start task |
|
|
|
|
Start tasks in batches |
|
|
|
|
Message format
Node change event - Start task
Node change event - Start tasks in batches
Operation Center event list
Event list
Modifying the schedule resource group or the owner triggers a node-change-updated event.
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Node change | Delete node
|
|
|
|
|
Pre-unpublish node event Directly take a recurring task node offline in the production environment Note This operation also deletes the node in Data Studio and moves it to the recycle bin. |
|
|
|
| |
Pre-freeze node event Important This event also triggers a node-change-updated event. |
|
|
|
| |
Pre-unfreeze node event Important This event also triggers a node-change-updated event. |
|
|
|
| |
Node data backfill | Pre-data backfill event |
|
|
|
|
Task status change | Scheduling task status change |
|
|
|
|
Instance change | Freeze instance |
|
|
|
|
Unfreeze instance |
|
|
|
| |
Stop instance |
|
|
|
| |
Rerun instance
|
|
|
|
| |
Set instance to successful |
|
|
|
| |
Remove specified upstream dependency of an instance |
|
|
|
| |
Pre-freeze instance event |
|
|
|
| |
Pre-unfreeze instance event |
|
|
|
| |
Pre-rerun instance event |
|
|
|
| |
Pre-set instance to successful event |
|
|
|
| |
Pre-kill instance event |
|
|
|
| |
Delete expired instance |
|
|
|
| |
Workflow status change | Workflow status change
|
|
|
|
|
Monitoring and alerts | Monitoring and alerts |
|
|
|
|
Message format
Node change events (delete, unpublish, freeze, unfreeze)
Node data backfill
Task status change event
Instance change events (freeze, unfreeze, stop, rerun, set to successful)
Pre-instance change events (freeze, unfreeze, rerun, set to successful, kill instance)
Instance change event - Delete expired instance
Remove specified upstream dependency of an instance event
Workflow status change event
Monitoring and alerts
Security Center event list
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension message type (eventCode) |
Approval Center | Pre-create permission request event |
|
|
|
|
Complete request |
|
|
|
| |
Security Center (table permission request) | Pre-request table permission event |
|
|
|
|
Message format
Approval Center (create, complete request)
Security Center (Pre-request table permission event)
Appendix: Authorization content list (MaxCompute)
Appendix: Authorization content list (DataService)
Data Quality event list
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Data quality validation | Validation result feedback
|
|
|
| |
Validation completed
|
|
|
|
| |
Data quality rule pre-operation | Pre-batch create data quality rules event |
|
|
|
|
Pre-batch update data quality rules event |
|
|
|
| |
Batch delete prerequisite events for Data Quality rules |
|
|
|
| |
Pre-update data quality rule event |
|
|
|
| |
Data quality monitoring pre-operation | Create a prerequisite event for data quality monitoring |
|
|
|
|
Pre-update data quality monitoring event |
|
|
|
| |
Clone a pre-check event for Data Quality monitoring |
|
|
|
| |
Batch delete prerequisite events for Data Quality monitoring |
|
|
|
| |
Data quality monitoring alert subscription pre-operation | Create a prerequisite event for a subscription to Data Quality monitoring and alerting |
|
|
| create-data-quality-evaluation-task-notification |
Pre-update data quality monitoring alert subscription event |
|
|
|
| |
Pre-delete data quality monitoring alert subscription event |
|
|
|
|
Message format
Data quality validation
Data quality rule pre-operation events (batch create, update, delete)
Data quality monitoring pre-operation events (create, update, clone, batch delete)
Data quality monitoring alert subscription pre-operation events (create, update, delete)
Tenant-level events
Tenant-level modules generate tenant-level events, such as event messages generated when a workspace is deleted in the management console. The following Event list shows which events are regular events and which are extension point events for each module, along with the message format for each event.
The following Message format tabs show only partial content. For the complete message sent to EventBridge or Function Compute, see Appendix: Message format.
Management console event list
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Delete project space | Delete a prerequisite event for a project |
|
|
|
|
Delete trailing spaces. |
|
|
|
|
Message format
Message entity format: Delete project space event
Upload and download event list
Event list
Event type | Event (and triggering operation) | Regular event | Extension point event | EventBridge event type (Type) | Extension event type (eventCode) |
Data upload and download | Pre-data download event - File generation |
|
| dataworks:ResourcesDownload:DownloadResources | download-resources |
Pre-data download event - File download |
|
| dataworks:ResourcesDownload:DownloadResourcesExecute | download-resources-execute | |
Pre-data upload event |
|
| dataworks:ResourcesUpload:UploadDataToTable | upload-data-to-table |
Message format
Data upload and download
Appendix: Message format
Message format for messages sent to EventBridge
After you configure OpenEvent and add an event distribution channel, if a workspace-level or tenant-level event is triggered in DataWorks, the messages are filtered based on the event type (Type) that you configured in EventBridge. The following shows the message format that DataWorks uses to send messages to EventBridge through the event distribution channel.
{
"datacontenttype": "application/json;charset=utf-8",// The content format of the data parameter. datacontenttype only supports application/json.
"data": {
// The message content varies with the message type. The following are two fixed fields in the message. For specific event messages, see the preceding sections.
"tenantId": 28378****10656,// The tenant ID. Each Alibaba Cloud account corresponds to a DataWorks tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of Data Studio.
"eventCode": "xxxx"//
},
"id": "539fd8f4-4ea1-4625-aa8b-6c906674****",// The event ID. A unique value that identifies the event.
"source": "acs.dataworks",// The event source. The service that provides the event, indicating that this message is pushed by DataWorks.
"specversion": "1.0",
"subject": "",
"time": "2020-11-19T21:04:41+08:00",// The time when the event occurred.
"type": "dataworks:InstanceStatusChanges:InstanceStatusChanges",// The event type. This event type can be used to filter all messages pushed by DataWorks in the EventBridge console. The Type value varies for each event. See the preceding sections for different event message types.
"aliyunaccountid": "123456789098****",// The Alibaba Cloud account ID.
"aliyunpublishtime": "2020-11-19T21:04:42.179PRC",// The time when EventBridge received the event.
"aliyuneventbusname": "default",// The name of the EventBridge event bus used to receive DataWorks event messages.
"aliyunregionid": "cn-hangzhou",// The region where the event is received.
"aliyunpublishaddr": "172.25.XX.XX"
}A complete event message includes the message entity format and basic information such as the message ID, source, and creation time. The following table describes the important fields.
Field Name | Type | Description |
data | object | The message entity format. The message format and its fields vary for different event types. For more information, see: Data Studio event message format: Data Studio event list. Data integration event message format: Data integration event list. Operation Center event message format: Operation Center event list. Security Center event message format: Security Center event list. Data Quality event message format: Data Quality event list. |
id | String | The unique ID of the event message. Used to locate the event message. |
type | String | The event type. Describes the event type related to the event source. Examples:
This event type can be used to filter all messages pushed by DataWorks in the EventBridge console. The Type value varies for each event. See the preceding sections for different event message types. |
Message format for messages sent to Function Compute
When you configure Extensions in DataWorks using the Function Compute method, DataWorks sends messages for triggered extension point events to Function Compute in JSON format. The following shows the message format sent to Function Compute.
{
"blockBusiness": true,
"eventCategoryType": "resources-download",// Event category
"eventType": "upload-data-to-table",// Event type
"extensionBizId": "job_6603***070",
"messageBody": {
// The message content varies with the message type. The following are two fixed fields in the message. For specific event messages, see the preceding sections.
"tenantId": 28378****10656,// The tenant ID. Each Alibaba Cloud account corresponds to a DataWorks tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of Data Studio.
"eventCode": "xxxx"//
},
"messageId": "52d44ee7-b51f-4d4d-afeb-*******"// The event ID. A unique value that identifies the event.
}The key fields are described as follows:
Field Name | Field Type | Description | |
messageId | String | The event ID. A unique value that identifies the event. | |
messageBody | The specific event message pushed by DataWorks. You can use this when developing extensions. The content varies with the message type. | ||
tenantId | The tenant ID. Each Alibaba Cloud account corresponds to a DataWorks tenant, and each tenant has its own tenant ID. You can view this value in the user information in the upper-right corner of Data Studio. | ||
eventCode | The event code. Used to identify a specific type of event message. For the eventCode corresponding to each event type, see the Extension event type (eventCode) column in the Development reference: Event list and message format table. | ||