When multiple jobs compete for the same resource quota, the scheduling policy controls which jobs are dequeued first and whether a blocked job stalls the rest of the queue. Platform for AI (PAI) provides four scheduling policies so you can match queue behavior to your workload — from strict ordering for dependent pipelines to throughput-first approaches for mixed batch jobs.
Each job submitted to a resource quota has a priority from 1 to 9, where a higher number means higher priority. Configure and manage scheduling policies on the Resource Quota page in the PAI console.
Note: Changing the scheduling policy after a resource quota is created only affects jobs that have not started running. Jobs already in the Running state are not affected.
Scheduling policies
FIFO (First In, First Out)
When the head-of-queue job cannot be dequeued, the scheduler retries it without skipping. All subsequent jobs in the queue wait until the blocked job is dequeued.
When to use: Workloads where strict ordering matters and no job should be bypassed — for example, dependent training pipelines where later jobs rely on the output of earlier ones.
Round Robin
When the head-of-queue job cannot be dequeued, the scheduler skips it and attempts to dequeue subsequent jobs in sequence.
When to use: Mixed workloads where throughput matters more than ordering. A blocked job at the head of the queue does not stall other ready jobs.
Balance
Balance starts with FIFO behavior and gradually expands the scheduling window to Round Robin as jobs exceed their timeout:
Phase 1 — FIFO retry: When the head-of-queue job cannot be dequeued, the scheduler retries it without skipping.
Phase 2 — Expand to Round Robin: If the job has not been dequeued after the timeout (typically a few minutes), the scheduler applies Round Robin across the first two jobs in the queue.
Phase 3 — Incremental expansion: If the second job also times out, the scheduler expands Round Robin to the first three jobs. The window continues to grow in the same manner.
When to use: General-purpose queues with a mix of long and short jobs. Balance protects high-priority jobs from being immediately bypassed while preventing a single blocked job from indefinitely stalling the rest of the queue.
Intelligent
Intelligent applies different policies based on each job's priority:
Priority >= 7: Scheduled using FIFO — high-priority jobs are never skipped.
Priority < 7: Scheduled using Round Robin. The priority of each waiting job increases by 1 based on their queue time until it reaches 6, preventing indefinite starvation.
In the figure above, Item1 and Item2 have a priority >= 7, and Items 3 to N have a priority < 7.
When to use: Queues with a clear distinction between critical and batch jobs. Assign priority >= 7 to production or time-sensitive jobs and lower priorities to batch or development workloads.
Choose a scheduling policy
| Policy | Head-of-queue blocked | Low-priority jobs | Best for |
|---|---|---|---|
| FIFO | Retried without skipping; queue is blocked | Wait behind blocked job | Dependent pipelines; strict ordering required |
| Round Robin | Skipped; subsequent jobs attempted in sequence | Run ahead of blocked jobs | Mixed independent jobs; maximize throughput |
| Balance | Retried first; Round Robin expands after timeout | Unblocked after timeout | General-purpose queues; blend of fairness and throughput |
| Intelligent | Priority >= 7: retried (FIFO); Priority < 7: skipped (Round Robin) | Priority rises over queue time, up to 6 | Mixed criticality; separate production from batch |
Configure a scheduling policy
To configure a scheduling policy when creating a resource quota, see Overview.