When you deploy a node to the production environment, DataWorks can generate scheduling instances on the same day or wait until the next day. This page explains how the Immediately After Deployment mode works and what to expect in common scenarios.
How it works
DataWorks generates auto triggered instances for all auto triggered tasks every night for the following day. The instance generation mode you configure on a node controls when newly deployed or updated nodes take effect.
The following table compares the two modes using the same example: you deploy a node at 3:00 PM today.
| Mode | When it takes effect | Deploy at 3:00 PM — result |
|---|---|---|
| Next Day | Auto triggered instances for the next day reflect the change | Instances run starting tomorrow |
| Immediately After Deployment | Instances are generated on the current day, subject to a 10-minute delay | Instances with scheduled times after 3:10 PM run today; earlier times dry-run |
Avoid using Immediately After Deployment when modifying the scheduling configuration of production nodes. It can cause dependency changes, dependency confusion, instance replacement, instance deletion, and complex same-day dependency graphs. Dependencies return to normal the following day.
Usage notes
-
Workflow nodes: Nodes inside a workflow cannot be configured individually. Set the instance generation mode for the entire workflow in the scheduling configuration on the workflow editing page.
-
Daily maintenance window: Every day from 23:30 to 24:00, DataWorks generates all auto triggered instances for the next day. Nodes deployed during this window must wait until the third day for their instances to be generated.
-
10-minute activation delay: After deployment, immediate instance generation has a 10-minute delay. The node's scheduled time must be at least 10 minutes after the deployment time for the instance to run. If the difference is less than 10 minutes, or the scheduled time is already in the past, DataWorks generates a dry-run instance with status Expired Instance That Is Generated In Real Time.
-
Scope of immediate changes: Not all configuration changes take effect immediately on current-day instances. For example, if you change the data source instance associated with a node and deploy with Immediately After Deployment, existing instances on the current day still use the previous data source. To run the task with the latest configuration, perform a data backfill operation.
-
Inconsistent upstream/downstream modes: If an upstream node uses Next Day and a downstream node uses Immediately After Deployment, the downstream node becomes an isolated node. Isolated nodes are not automatically scheduled, which can disrupt dependent tasks.
Immediate instance generation behavior
Immediate instance generation only applies to future scheduled times — instances execute normally only when the scheduled time is later than the deployment time.
The following diagram shows the overall execution rules:
Normal execution
An instance runs normally when its scheduled time is more than 10 minutes after the deployment time.
| Scenario | Result |
|---|---|
| New node deployed; scheduled time is more than 10 minutes in the future | DataWorks generates executable auto triggered instances and schedules them normally |
| Node updated; new scheduled time is more than 10 minutes in the future | DataWorks generates instances per the new configuration, replacing instances from the previous configuration |
| Upstream task scheduling time changed; new instances are more than 10 minutes in the future | Downstream instances that have not yet run depend on the new upstream instances |
Dry-run (expired instances)
If the scheduled time is in the past relative to the deployment time, or less than 10 minutes in the future, DataWorks generates a dry-run instance. The instance status is set to Expired Instance That Is Generated In Real Time and no actual code logic runs.
Example 1: Node A has a scheduled time of 09:05 and a deployment time of 09:00. The difference is less than 10 minutes, so DataWorks generates a dry-run instance with status Expired Instance That Is Generated In Real Time.
Example 2: Node A has a scheduled time of 09:00 and a deployment time of 10:00. The scheduled time is already in the past, so DataWorks immediately generates a dry-run instance with status Expired Instance That Is Generated In Real Time.
To run the task with the current day's data, perform a data backfill operation. Data backfill executes with the latest task configuration.
Common scenarios
Deploy a new node
When you deploy a new node with Immediately After Deployment, DataWorks generates instances on the same day. Whether those instances actually run depends on the scheduled time relative to the deployment time.
Example: A node is deployed to the production environment at 12:00. The 10-minute delay means the mode takes effect at 12:10.
| Scheduled time | Result |
|---|---|
After 12:10 |
Instance is scheduled and executed normally |
Before 12:10 |
Instance dry-runs with status expired instance that is generated in real time |
This only affects instance execution on the current day. On subsequent days, DataWorks generates instances normally according to the scheduling configuration.
Change the scheduling cycle of a deployed task
After you update the scheduling time of a production task and deploy it, instances from before and after the change may coexist on the current day, creating complex instance dependencies.
This scenario only occurs on the day the task is deployed with Immediately After Deployment. The following day, DataWorks generates instances normally according to the updated configuration.
Example 1: Scheduling frequency changed from every 6 hours to daily at 08:00 (a past time), deployed at 09:00.
Example 2: Scheduling frequency changed from every 6 hours to daily at 18:00 (a future time), deployed at 09:00.
In this case:
-
New instance generated: Daily instance A3 is generated after 09:00 per the new configuration.
-
Replacement: New instance A3 replaces original instances A3 and A4.
-
Retained instances: Hourly instances generated before 09:10 are retained.
The general rules for scheduling time changes:
-
Scheduled time in the future: DataWorks replaces already-generated instances in future time periods with instances per the latest configuration.
-
Scheduled time in the past: DataWorks retains instances generated before the effective time of the change and replaces or deletes instances generated after the effective time.
Impact on downstream dependencies when changing a task's scheduled time
When you change an upstream task's scheduling time and deploy with Immediately After Deployment, downstream instances adjust their dependencies based on the latest scheduling configuration. Downstream instances mount dependencies for both newly generated instances and any unreplaced old instances, preserving correct scheduling logic.
This scenario only occurs when the node version being deployed has Immediately After Deployment set and includes a scheduling time change.
Example 1: An ancestor task's scheduling frequency changes from every 6 hours to every 8 hours.
Example 2: An ancestor task's scheduling frequency changes from every 6 hours to daily at 16:00.
For the dependency rules across hourly and minute-level scenarios, see Principles and samples of scheduling configurations in complex dependency scenarios.
Inconsistent instance generation modes between upstream and downstream tasks
If upstream and downstream nodes are both newly created but configured with different instance generation modes, the downstream node may become an isolated node.
Example: Upstream uses Next Day, downstream uses Immediately After Deployment. The downstream node's instances are generated on the current day, but the upstream node's instances aren't generated until the next day. The downstream node has no upstream instances to depend on and becomes isolated.
Isolated nodes are not automatically scheduled. If multiple downstream tasks depend on an isolated node, your pipelines may be severely affected.
Troubleshooting
If instances are not generated as expected after deploying with Immediately After Deployment, check the following in order:
-
Check the scheduled time vs. deployment time: The scheduled time must be at least 10 minutes after the deployment time. If the gap is less than 10 minutes, or the scheduled time is already past, the instance dry-runs with status Expired Instance That Is Generated In Real Time. Check the instance status in Operation Center.
-
Check if you deployed during the maintenance window: If you deployed between 23:30 and 24:00, instances are not generated until the third day, not the next day.
-
Check for isolated nodes: If upstream and downstream nodes have different instance generation modes, the downstream node may become isolated. Verify that upstream and downstream nodes use the same mode, or check the dependency graph in Operation Center for isolated node indicators.
-
Check what changes actually take effect: If you changed the data source instance associated with the node, current-day instances continue using the previous data source. Perform a data backfill to run the task with the latest configuration.