After you create a real-time synchronization node in DataStudio and deploy it to the production environment, run and monitor the node in Operation Center. This topic describes common Operations and Maintenance (O&M) operations for real-time synchronization nodes, including how to start and stop nodes, configure alerting, view running metrics, and modify node configurations.
Prerequisites
Before you begin, ensure that you have:
A real-time synchronization node created and deployed. For more information, see Create a real-time synchronization task in DataStudio and Configure a real-time synchronization node to synchronize all incremental data from a database
Run and manage the node
On the Real Time DI page in Operation Center, start, stop, or undeploy a real-time synchronization node.
To go to the Real Time DI page: log on to the DataWorks console, go to Operation Center, and choose Real-Time Node O&M > Real-time Synchronization Nodes in the left-side navigation pane.
For detailed steps, see Manage real-time synchronization tasks.
Start options
When you click Start in the Operation column, configure the following options:
Option | Description |
Offset | Resume data synchronization from a specified position. Supports resumable data synchronization and uploads from a specified offset. |
Failovers | Set the maximum number of failovers allowed within a specified period. If the threshold is exceeded, the node stops automatically to prevent resource exhaustion from repeated restarts. If no threshold is configured, the node stops automatically when failovers exceed 100 within 5 minutes. |
Configure alerting
Set up alerting to detect errors before they cause data output delays.
To open the alert settings: in Operation Center, choose Real-Time Node O&M > Real-time Synchronization Nodes, find the node, and click Alarm settings in the Operation column.
For detailed steps, see Manage real-time synchronization tasks.
Alert conditions
Configure alerts based on any of the following conditions:
Condition | Description |
Node status | Triggers an alert when the node status changes unexpectedly. |
Business delay | Triggers an alert when data synchronization latency exceeds a threshold. |
Failover | Triggers an alert when the node fails over. |
DDL synchronization | Triggers an alert when DDL operation changes are detected on the source. |
DDL synchronization support varies by destination type. If your destination does not support DDL change synchronization and you configure a DDL-based alert rule, a related error may be reported during node execution. For supported DDL operations, see Supported DML and DDL operations.
Notification methods
Alerts can be sent by Email, text message, or DingTalk.
Alerting frequency control
To avoid alert storms, configure a frequency control rule so that DataWorks sends only one alert notification within a specified period.
View node metrics with LogView
LogView is in invitational preview. To use this feature, contact technical personnel.
LogView in Data Integration collects event-based data from synchronization nodes, analyzes it, and displays the results visually. Use LogView to view data transmission rates and logs at fine-grained granularity.
To open LogView: in Operation Center, choose RealTime Task > Real Time DI in the left-side navigation pane, find the node, and click the node name.
The following tabs are available on the node details page:
Tab | Description |
Log | View log details and search logs by time. |
Progress | View synchronization progress by metric: business latency, number of synchronized records, number of synchronized bytes, data transmission rate for records, data transmission rate for bytes, and window waiting time. |
Database DDL event | View DDL operation records detected on the source. |
Database DML statistics | View statistical analysis results for source and destination tables. |
Failover | View failover status and a curve chart of failover counts. Click View in the Log Details column to view logs for a specific failover. |
More operations on the Progress tab
On the Progress tab:
Use the time picker to view synchronization details for a specific time range. Data is available for the last 15 days.
Click the icon on the right side of the node list to select which columns to display.
Switch between the list of workers and the list of tasks in the node list section:
List of workers: lists the threads running the node. In standard mode, one worker runs. In distributed execution mode, multiple workers run — the exact count depends on the configured parallelism.
List of tasks: lists the reader tasks and writer tasks within each worker. The number of tasks per worker depends on the configured parallelism.
Click a metric value in the node list to view its changes over time in a curve chart.
Modify the node configuration
To modify a real-time synchronization node's configuration:
Stop the node on the Real Time DI page in Operation Center.
Open DataStudio and make the changes.
Commit and redeploy the node.
Start the node on the Real Time DI page.
Modify DDL processing rules
Before you start the real-time synchronization node, you can modify the DDL message processing rules. To do so:
Stop the node on the Real Time DI page in Operation Center.
Open DataStudio and update the DDL processing rules. For more information, see Configure a real-time synchronization node to synchronize all incremental data from a database.
Commit and redeploy the node.
Start the node on the Real Time DI page.
FAQ
Why does my real-time synchronization task have high latency?