Workload API

The Workload API resource defines the scheduling requirements and structure of a multi-Pod application. While workload controllers such as Job manage the application's runtime state, the Workload specifies how groups of Pods should be scheduled. The Job controller is the only built-in controller that creates PodGroup objects from the Workload's PodGroupTemplates at runtime.

What is a Workload?

The Workload API resource is part of the scheduling.k8s.io/v1alpha2 API group and your cluster must have that API group enabled, as well as the GenericWorkload feature gate, before you can use this API.

A Workload is a static, long-lived policy template. It defines what scheduling policies should be applied to groups of Pods, but does not track runtime state itself. Runtime scheduling state is maintained by PodGroup objects, which controllers create from the Workload's PodGroupTemplates.

API structure

A Workload consists of two fields: a list of PodGroupTemplates and an optional controller reference. The entire Workload spec is immutable after creation: you cannot modify existing templates, add new templates, or remove templates from podGroupTemplates.

PodGroupTemplates

The spec.podGroupTemplates list defines the distinct components of your workload. For example, a machine learning job might have a driver template and a worker template.

Each entry in podGroupTemplates must have:

1. A unique name that will be used to reference the template in the PodGroup's spec.podGroupTemplateRef.
2. A scheduling policy (basic or gang).

If the WorkloadAwarePreemption feature gate is enabled each entry in podGroups can also have priority and disruption mode.

The maximum number of PodGroupTemplates in a single Workload is 8.

apiVersion: scheduling.k8s.io/v1alpha2
kind: Workload
metadata:
  name: training-job-workload
  namespace: some-ns
spec:
  controllerRef:
    apiGroup: batch
    kind: Job
    name: training-job
  podGroupTemplates:
  - name: workers
    schedulingPolicy:
      gang:
        # The gang is schedulable only if 4 pods can run at once
        minCount: 4
    priorityClassName: high-priority # Only applicable with WorkloadAwarePreemption feature gate
    disruptionMode: PodGroup # Only applicable with WorkloadAwarePreemption feature gate

When a workload controller creates a PodGroup from one of these templates, it copies the schedulingPolicy into the PodGroup's own spec. Changes to the Workload only affect newly created PodGroups, not existing ones.

Referencing a workload controlling object

The controllerRef field links the Workload back to the specific high-level object defining the application, such as a Job or a custom CRD. This is useful for observability and tooling. This data is not used to schedule or manage the Workload.

Gang scheduling with Jobs

When the EnableWorkloadWithJob feature gate is enabled, the Job controller automatically creates Workload and PodGroup objects for parallel indexed Jobs where .spec.parallelism equals .spec.completions. The gang policy's minCount is set to the Job's parallelism, so all Pods must be schedulable together before any of them are bound to nodes.

This is the built-in path for using gang scheduling with Jobs. You do not need to create Workload or PodGroup objects yourself as the Job controller handles it automatically. Other workload controllers (such as JobSet) may manage their own Workload and PodGroup objects independently.

What's next

• Learn about PodGroup scheduling policies.
• See how PodGroups are created from Workloads in the PodGroup API overview.
• Read about how Pods reference their PodGroup via the scheduling group field.
• Learn about Topology-aware workload scheduling.
• Understand the gang scheduling algorithm.