Removing or changing the output of a node can break scheduling dependencies for downstream nodes, causing them to fail or produce incorrect data. This topic explains what triggers output changes, what the downstream impacts are, and how to restore valid dependencies.
If a node has descendant nodes, changing its output can severely disrupt downstream scheduling. Assess the impact on your pipeline before making changes.
How node output affects scheduling
Removing a node's output does not affect whether the node runs or whether it produces table data. A node's output is used exclusively to define scheduling dependencies between nodes — the node's code logic alone determines what table data is produced.
Operations that trigger output removal or change
When a node is created, the system automatically generates two outputs for it. You can add outputs manually or enable the automatic parsing feature to let the system detect output tables from the node's code.
The following operations can remove or change a node's output:
| Operation | Description |
|---|---|
| Manually remove the output | Delete an output entry from the node directly. |
| Disable automatic parsing without verifying outputs | If you disable the automatic parsing feature and do not check the output of a node, and the node's code changes so it no longer produces a specific table, the corresponding output becomes invalid. |
Downstream impacts
When you remove the output of a node that has descendant nodes, a warning appears in the DataWorks console.
Review the warning and assess the downstream impact before proceeding. The following table describes the three impact scenarios.
| Scenario | Condition | Impact |
|---|---|---|
| Descendant node becomes isolated | Descendant nodes depend only on the current node | Descendant nodes become isolated nodes and cannot be scheduled. |
| Data quality issues | Descendant nodes depend on multiple ancestor nodes | Descendant nodes may run but fail to get the required data, causing data quality issues. |
| Dependencies become invalid | The node's code no longer produces a specific table | The output linked to that table becomes invalid. Descendant nodes that depend on it must be reconfigured with new dependencies. |
If descendant nodes have strong dependencies on the current node, severe impacts may occur. Also note that the output of a node automatically changes when the table generated by the node is changed. The automatic parsing feature maintains scheduling dependencies based on code lineage. If a node generates Table A and a descendant node depends on Table A, the system adds Table A to the node's Output and shows the IDs and names of dependent nodes. If the node no longer generates Table A, those dependencies become invalid and must be reconfigured manually.
If you remove the output of a node before the output table of the node is generated, data quality issues may occur.
Fix broken dependencies after a table changes
This example shows what happens when Node_A stops generating Table A due to a business change, and how to restore valid scheduling for Node_B.
Initial state
Node_A generates Table A. Node_B processes Table A and depends on Node_A. The automatic parsing feature configures the dependency between the two nodes.
What breaks
After a business change, Node_A no longer generates Table A. The dependency between Node_A and Node_B becomes invalid.
The following issues occur depending on Node_B's dependency configuration:
If
Node_Bdepends only onNode_A,Node_Bbecomes an isolated node and cannot be scheduled.If
Node_Bdepends on other nodes as well,Node_Bcan still be scheduled but may not get the data it needs. Note that Table A is still referenced inNode_B'sSELECTstatement, butNode_Bno longer has a dependency on the node that generates Table A.
The error details for the broken dependency are:
| Field | Detail |
|---|---|
| Error message | No ancestor node ID or ancestor node name is specified for Node_B. |
| Cause | Node_B cannot find the node that generates Table A from the output name of its ancestor node. |
| Business cause | Node_A no longer generates Table A, so Node_B cannot resolve the dependency using Node_A's output name. |
Fix
Add Table A as the output of Node_C — the node that now generates Table A. The automatic parsing feature then configures a dependency between Node_C and Node_B.
After adding Table A to Node_C's output, commit and deploy Node_C to activate the new dependency.
