All Products
Search

This Product

    DataWorks:Manage real-time compute nodes

    Document Center

    DataWorks:Manage real-time compute nodes

    Last Updated:Aug 17, 2023

    The Stream Task page of the DataWorks console displays all real-time compute nodes. You can view the basic information and running information about the nodes on this page. You can also configure alert rules for the nodes that you want to monitor. This way, you can identify and handle exceptions at the earliest opportunity if an error occurs on a node.

    Limits

    DataWorks supports O&M only for EMR Spark Streaming and EMR Streaming SQL nodes.

    Go to the Stream Task page

    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. In the top navigation bar, select the region where your workspace resides. On the Workspaces page, find your workspace and click DataStudio in the Actions column.

    4. On the DataStudio page, click the 图标 icon in the upper-left corner and choose All Products > Data Development And Task Operation > Operation Center.

    5. In the left-side navigation pane of the Operation Center page, choose RealTime Task > Stream Task.

    View the real-time compute nodes in the node list

    The Stream Task page displays all real-time compute nodes in the production environment. You can view the basic information about, start, stop, or undeploy a real-time compute node. You can also configure alert rules for a real-time compute node.实时计算任务

    Section

    Description

    1

    In this section, you can search for a real-time compute node by node ID or node name. You can also specify one of the following filter conditions to perform the operation: Status, Task type, Engine Type, Engine Instance, and Responsible Person.

    Note

    • If you search for nodes by node name or node ID, the search result is affected by other filter conditions that you specified. Only nodes that meet all the filter conditions that you specified are displayed.

    • If you search for a real-time compute node by node name, fuzzy match is supported. After you enter a keyword, all real-time compute nodes whose names contain the keyword are displayed.

    2

    In this section, you can view the basic information and details of a real-time compute node.

    • Node Name: the name of the real-time compute node. You can click the node name to go to the details page of the node.

    • Node ID: the ID of the real-time compute node.

    • Status: the status of the real-time compute node. The node can be in one of the following states: Not Running, Generating, Pending (Resources), Starting, Running, Stopped, Failed, Restarting, and Undeploying.

    • Start running: the time when the real-time compute node started to run.

    • End Running: the time when the running of the real-time compute node was complete.

    • Runtime: the running duration of the real-time compute node. Unit: seconds.

    • Task type: the type of the real-time compute node.

      Note

      DataWorks supports O&M only for EMR Spark Streaming and EMR Streaming SQL nodes.

    • Engine Type: the type of the compute engine used to run the real-time compute node. DataWorks supports O&M only for real-time compute nodes that are run by using the E-MapReduce (EMR) compute engine.

    • Engine Instance: the name of the EMR compute engine instance that is associated with the workspace to which the real-time compute node belongs.

    • Deployed At: the time when the real-time compute node was deployed. The time is in the yyyy-MM-dd HH:mm:ss format.

    • Responsible Person: the owner of the workspace to which the real-time compute node belongs.

    Note

    You can rank all real-time compute nodes in ascending or descending order by Starting running, End Running, or Runtime.

    3

    In this section, you can perform the following operations on a real-time compute node:

    4

    In this section, you can perform an operation on multiple real-time compute nodes at a time. You can select multiple nodes and click Start, Terminate, Undeploy, or Monitoring Setting to perform the related operation on these nodes.

    View the details of a real-time compute node

    On the Stream Task page, find the real-time compute node whose details you want to view and click the node name to go to the details panel of the node. In the node details panel, you can view the details, run logs, operation logs, and code of the node on the Details, Runtime Log, Operation Log, and Code tabs. This allows you to quickly locate operations performed on the node and operation records, obtain details of the errors reported for the node, and fix the errors.实时计算任务详情页面