The OpenEvent module allows you to subscribe to event messages so that you can detect various DataWorks change events and respond to the events at the earliest opportunity. For example, you can detect changes to core tables in real time by subscribing to table change events. You can also customize a dashboard to display the status of nodes in real time by subscribing to node change events. This topic describes how the OpenEvent module works and how to configure message subscription.
- Only DataWorks Enterprise Edition users can join the public preview program by submit a ticket . No fees are required for the program.
- This module is in public preview only in the China (Beijing), China (Hangzhou), China (Shenzhen), China (Shanghai) and China (Hong Kong) regions.
Supported event types
- Regular events: For this type of event, you can only subscribe to them.
- Extension point events: For this type of event, you can not only subscribe to them but also configure extensions to respond to them.
|DataWorks service||Event category||Event description and type||Example|
|Operation Center||Instance status change events||
||An instance status change event is triggered when the status of an instance changes from Not running to Running or from Running to Run failed.|
|Node change events||
||A node change event is triggered when a node is created, updated, or deleted in the production environment in Operation Center after the node is managed, committed, and deployed in DataStudio.|
|DataStudio||File change events||
|Table change events||
Logic for message subscription
- DataWorks serves as a producer, which publishes messages related to DataWorks change evens to a Kafka topic. A client serves as a consumer, which subscribes to the Kafka topic to receive messages. By default, DataWorks provides a shared Message Queue for Apache Kafka topic for you to subscribe to DataWorks change events. You only need to configure the topic.
- When you configure the topic, you must specify the workspaces and event types to which
the topic applies. After the configuration is complete, if events of the specified
types are triggered in the specified workspaces, DataWorks automatically publishes
messages to the topic for clients to consume.
Note For more information about supported event types, see Supported event types.
- DataWorks and clients can publish and receive messages by using the SSL endpoints of Message Queue for Apache Kafka. You can view the SSL endpoints on the OpenEvent tab of the Open Platform page in the DataWorks console.
- If a client subscribes to messages by using an SSL endpoint, the PLAIN mechanism is used for identity authentication. The PLAIN mechanism works based on a username and a password. The OpenEvent module automatically generates a username and a password when you enable message subscription. You can also delete the default user and create new user information for message subscription.
Main configuration steps
To use the features of the OpenEvent module, you must complete the following configuration steps:
- Enable message subscription in DataWorks.
In DataWorks, you must enable the message subscription feature, configure the username and password for identity authentication, and specify the workspaces and event types to which the topic applies. For more information, see Enable message subscription in DataWorks.
- Develop consumer programs on your clients.
After the configuration in DataWorks is complete, you can develop consumer programs on your clients to consume messages. For more information about the sample code, see Develop consumer programs on your clients. For information about the message formats for different types of DataWorks events, see Appendix: Message formats.
- Lower O&M costs:
You can subscribe to important DataWorks events with ease and have free access to Message Queue for Apache Kafka that features data persistence, high throughput, and enterprise-level data security protection.
- More efficient message publishing and receiving:
You only need to configure the Kafka topic in DataWorks and develop consumer programs on your clients. You do not need to develop producer programs for DataWorks. This reduces and simplifies manual operations.
- Applicable to various needs:
You can subscribe to various types of DataWorks events. This allows you to detect important DataWorks changes and respond to the changes at the earliest opportunity.