DataWorks:Nodes that are waiting for resources

The resources in a resource group are limited. If the resources are occupied by a node for an extended period of time, other nodes cannot be run until the occupied resources are released. For more information about how the scheduling system issues nodes, see Overview.

In the left-side navigation pane of Operation Center, click Intelligent Diagnosis. On the page that appears, enter the ID of an instance generated for the node in the search box and click Perform Diagnostics to perform an end-to-end diagnosis for the node. You can view the information such as the execution status of the node, the nodes that are waiting for resources, and the nodes that are running when the current node is waiting for resources. You can quickly find the nodes that are occupying resources for an extended period of time and block the execution of nodes involved in the directed acyclic graph (DAG) of the node. For more information, see Intelligent diagnosis.

To view the nodes that are running when the current node is waiting for resources, perform the following steps: On the Overview page of Operation Center, click Instance waiting for resource. On the page that appears, find the current node, right-click the node in the DAG, and then select Instance Diagnose. On the page that appears, click the Resources tab and view the nodes that are displayed in the Resource-consuming tasks section.

The logs of a node show that the node is waiting for gateway resources.

The number of nodes that are running in parallel on the related resource group for scheduling reaches the upper limit.

The logs of a data synchronization node in Data Integration show that the node is waiting for resources.

• Check whether the value of the concurrent parameter that is specified when you configure the node is excessively large. If the value is too large, you must configure the parameter based on the upper limit for the resource group for Data Integration and the number of nodes that are running in parallel.
• You may set the concurrent parameter of the node to a value that exceeds the upper limit for the resource group for Data Integration. In this case, stop the node and change the value of the concurrent parameter before you run the node again.
  Go to Operation Center. The Overview page appears. On the Overview page, click the Data Integration tab and view the details of data synchronization nodes in the Synchronization task execution details section.

  Note

  • You must set the concurrent parameter for a node based on the maximum number of nodes that can be run in parallel on an exclusive resource group for Data Integration. In addition, you must also consider the number of nodes that you want to run in parallel and the sum of the values of the concurrent parameter for each node.
  • A data synchronization node in Data Integration occupies the resources of a resource group for Data Integration. If resources are occupied by the node for an extended period of time, other nodes to be run on the resource group cannot be run.
• Wait until the nodes that are running release the occupied resources or scale out your exclusive resource group for Data Integration. You can use one of the following methods to view resource usage in the resource group:
  Note If a data synchronization node in Data Integration waits for resources in a resource group, you can resolve the issue by referring to Optimize the performance of batch synchronization nodes.
  • By using the intelligent diagnosis feature

    In the left-side navigation pane of Operation Center, click Intelligent Diagnosis. On the page that appears, enter the ID of an instance generated for the node in the search box and click Perform Diagnostics to perform an end-to-end diagnosis for the node. You can view the information such as the execution status of the node, the nodes that are waiting for resources, and the nodes that are running when the current node is waiting for resources. You can quickly find the nodes that are occupying resources for an extended period of time and block the execution of nodes involved in the DAG of the node. For more information, see Intelligent diagnosis.

    To view the nodes that are running when the current node is waiting for resources, perform the following steps: On the Overview page of Operation Center, click Instance waiting for resource. On the page that appears, find the current node, right-click the node in the DAG, and then select Instance Diagnose. On the page that appears, click the Resources tab and view the nodes that are displayed in the Resource-consuming tasks section.
  • In the DataWorks console
    Log on to the DataWorks console and click Resource Groups in the left-side navigation pane. On the Exclusive Resource Groups tab of the Resource Groups page, find the exclusive resource group and view the resource usage in the Resource Group Usage column. Then, you can click the percentage value in the Resource Group Usage column, and view the details of the resource group and the resource usage in the resource group.
  • On the Resource page of Operation Center
    Go to Operation Center. Click Resource in the left-side navigation pane and view the information about resource groups, such as the resource usage and the nodes that are running on the resource groups.

    Note The maximum number of nodes that can be run in parallel on an exclusive resource group for Data Integration varies based on the specifications of the resource group. For more information, see Billing of exclusive resource groups for Data Integration (subscription).