Trigger event checking during data development - DataWorks

After you locally develop an extension and register and publish the extension in the DataWorks console, if operations related to the specified extension points are performed in the workspaces in which the extension takes effect, DataWorks sends related event messages to the extension, and determines whether to allow the operations to take effect based on the results returned by the extension.

Prerequisites

The configurations related to OpenEvent messages and extensions are complete.

Background information

DataWorks sends messages to extensions when extension point events, such as the file deployment, file commitment, and table commitment events, are generated, and waits for extensions to automatically check and respond to the messages. For more information about extension point events, see Supported extension point events.

Extensions automatically check and respond to extension point events. DataWorks provides you with built-in extensions that you can determine whether to enable. For example, you can enable a built-in extension to prohibit simple processing or table creation. You can also develop extensions and register them in DataWorks. To view, enable, or disable the extensions that are associated with a workspace, go to the Extension configuration page of the workspace.

Limits

Built-in extensions can be displayed on the Extension configuration page of a workspace only after you enable Data Governance Center for the workspace. For more information about how to enable Data Governance Center and how it works, see Overview.

In scenarios in which you enable an extension to respond to messages related to extension point events generated during data development, take note of the following limits in terms of message notification and responding:

If you enable a built-in extension, the extension takes effect only for nodes of the ODPS SQL type.
That is, built-in extensions check the processing logic only for nodes of the ODPS SQL type. For nodes of the ODPS SQL type, if operations that trigger extension point events are performed, DataWorks interrupts the operations, sends messages to extensions, waits for extensions to check the processing logic and return check results, and then processes the operations based on the check results. For other types of nodes, if operations that trigger extension point events are performed, DataWorks interrupts the operations, but built-in extensions return success without checking the processing logic.
Extension point events related to table deployment and commitment can be triggered and checked only when MaxCompute tables are committed and deployed.
If node groups, such as Machine Learning Platform for AI nodes, do-while nodes, and for-each nodes, trigger extension checks, you must wait for all inner nodes of the node groups to pass the checks before you can perform subsequent operations.

Features

DataStudio allows local services to receive messages when events related to the following extension points are generated. In addition, you can register a local program as a DataWorks extension and use the extension to receive messages related to extension point events. The extension can process extension point events based on your custom logic and send the check result to the platform by calling the UpdateIDEEventResult operation. This implements process control in DataWorks. DataWorks supports the following extension points:

Extension points for operations on files, such as file execution, file commitment, file publishing, and file deletion
Extension points for operations on tables, such as table commitment and table publishing

Supported extension points

DataWorks sends messages to extensions and waits for extensions to check and respond to the messages when events related to the following extension points are generated during data development:

Run a file in DataStudio
Commit a file in DataStudio
Deploy a file in DataStudio
Delete a file in DataStudio
Commit a table in DataStudio
Deploy a table in DataStudio