DataWorks Data Integration supports dynamic scaling of resources for real-time synchronization tasks. When the system scales out or in, it adjusts the number of workers while keeping the compute units (CUs) per worker unchanged. If the target CU count does not satisfy this constraint, the task may restart.
This topic describes how to calculate valid CU counts when configuring dynamic scaling.
How dynamic scaling works
When dynamic scaling is triggered, the system adds or removes workers. The CU count per worker must remain the same before and after scaling. The system derives this per-worker CU count from the initial CU configuration set at task startup.
If the scaled CU count violates this constraint, whether the task restarts depends on the exact CU value chosen and the task's scheduling sequence.
Scaling formula
Valid CU count after scaling
When only the number of workers changes, the CU count after scaling must satisfy:
CUs after scaling = ceil( (initial CUs ± N × CUs per worker) / 0.5 ) × 0.5Where N is a positive integer representing the number of workers added or removed.
The minimum sales unit of CUs for data synchronization is 0.5. The ceil(.../ 0.5) × 0.5 expression rounds up to the nearest 0.5.
Deriving CUs per worker
Calculate CUs per worker from the initial CU configuration:
Workers = floor(initial CUs / 3)
CUs per worker = (initial CUs - control process CUs) / workersConfiguration constants
| Constant | Value | Notes |
|---|---|---|
| Empirical constant (divisor for worker count) | 3 | Fixed; subject to change |
Control process CUs by initial CU range:
| Initial CUs configured | Control process CUs |
|---|---|
| 2 or fewer | 0.25 |
| Greater than 2, up to 10 | 0.5 |
| Greater than 10 | 0.75 |
Control process CU values may be adjusted in the future.
Worked example
Scenario: Initial CUs = 6, scale out by 1 worker.
Step 1 — Calculate workers:
Workers = floor(6 / 3) = 2Step 2 — Look up control process CUs:
Initial CUs = 6, which falls in the greater than 2 and up to 10 range. Control process CUs = 0.5.
Step 3 — Calculate CUs per worker:
CUs per worker = (6 - 0.5) / 2 = 2.75Step 4 — Calculate CUs after scaling (adding 1 worker):
CUs after scaling = ceil( (6 + 1 × 2.75) / 0.5 ) × 0.5
= ceil( 8.75 / 0.5 ) × 0.5
= ceil(17.5) × 0.5
= 18 × 0.5
= 9Result: To scale out by 1 worker, set the target CU count to 9.
Impact of non-compliant CU settings
If the target CU count does not satisfy the formula above, the task may restart during scaling. Whether a restart occurs depends on the exact CU value chosen and the scheduling sequence of the task.
Set the CU count to a value computed by the formula to avoid unplanned restarts.