Container Compute Service:Kubernetes 1.26 release notes

Upgrade containerd to 1.6.0 or later

Upgrade containerd to 1.6.0 or later on all nodes before upgrading the cluster to Kubernetes 1.26.

Linux users can continue using iptables or IPVS. Windows users can use kernelspace mode.

Migrate away from deprecated beta APIs

Several beta APIs are removed in Kubernetes 1.25 and 1.26. If any controller or application in your cluster calls these APIs, update them to use the stable API versions before upgrading. See Deprecated APIs for the full list and migration paths.

Replace PodSecurityPolicy

The PodSecurityPolicy (PSP) admission controller was removed in Kubernetes 1.25. If your cluster uses PSP, set one of the following alternatives before upgrading:

• ACK policy management (recommended): Provides curated policies optimized for Kubernetes workloads. For more information, see Configure pod security policies.

• Built-in Pod Security Admission: A simpler alternative built into Kubernetes. For more information, see Pod Security Admission and Migrate from PodSecurityPolicy to the Built-in PodSecurity Admission Controller.

• Third-party admission webhook: Deploy and configure a webhook that matches your existing policies.

CVE fixes in 1.26.15-aliyunacs.1

The following CVEs are fixed in this release:

• CVE-2023-45288

• CVE-2024-3177

• CVE-2024-24786

Kubelet port change

The kubelet read-only port 10255 is no longer open by default in ACK clusters running Kubernetes 1.26 or later. The authenticated port 10250 is used instead. Update any monitoring or tooling that queries port 10255 directly.

Graduated to stable (GA)

Ephemeral containers — Introduced in Beta in Kubernetes 1.23, ephemeral containers reach GA in Kubernetes 1.25. Use kubectl debug to attach an ephemeral container to a running or crashed pod for troubleshooting without a pre-installed debug tool. For more information, see Ephemeral Containers.

cgroup v2 — Reaches stable in Kubernetes 1.25. cgroup v2 improves resource isolation and control compared to cgroup v1. For more information, see About cgroup v2.

StatefulSet minReadySeconds — The minReadySeconds field for StatefulSets reaches stable in Kubernetes 1.25. Use this field to introduce a deliberate delay between pod replacements during rolling updates.

DaemonSet maxSurge — Reaches stable in Kubernetes 1.25. maxSurge specifies how many extra pods can run on a node during a DaemonSet rolling update, reducing downtime. Note: maxSurge and hostPort cannot be used together because two active pods cannot share the same port on a node.

Local ephemeral storage capacity isolation — Reaches GA in Kubernetes 1.25. Pods have hard limits on emptyDir volume consumption and are evicted if they exceed those limits. For more information, see Local Ephemeral Storage Capacity Isolation.

Ephemeral inline CSI volumes — Reaches stable in Kubernetes 1.25. CSI volumes can be specified directly in a pod spec without a PersistentVolume (PV) or PersistentVolumeClaim (PVC). For more information, see Ephemeral Inline CSI volumes.

CPU manager — Reaches GA in Kubernetes 1.26 (Beta since 1.10). The CPU manager is part of the kubelet and allocates exclusive CPUs to containers. Three policies are supported. For more information, see Control CPU Management Policies on the Node.

CSI driver fsGroup delegation — Kubernetes 1.26 allows the kubelet to delegate fsGroup ownership changes to the CSI driver when mounting a volume. This is transparent to end users. For more information, see CSI Driver fsGroup Support.

New in Kubernetes 1.26 (Alpha)

Dynamic resource allocation — A new API for requesting and sharing resources between pods or containers, with custom initialization parameters. Enable the DynamicResourceAllocation feature gate and the resource.k8s.io/v1alpha1 API group, then install a resource driver for the resources to manage. For more information, see Alpha API For Dynamic Resource Allocation.

Pod scheduling gates — A new field spec.schedulingGates that marks a pod as not ready for scheduling. When set, the scheduler ignores the pod until an external controller removes the gate. This prevents a large backlog of unschedulable pending pods from degrading scheduler performance. For more information, see Pod Scheduling Readiness.

Cross-namespace storage data sources — Allows a PVC to reference a data source in a different namespace. For more information, see Kubernetes v1.26: Alpha support for cross-namespace storage data sources.

Unhealthy pod eviction policy for PodDisruptionBudget — Set .spec.unhealthyPodEvictionPolicy=AlwaysAllow on a PodDisruptionBudget (PDB) to allow eviction of unhealthy pods regardless of the disruption budget. Enable the PDBUnhealthyPodEvictionPolicy feature gate to use this feature. For more information, see Unhealthy Pod Eviction Policy.

Other notable changes

Non-graceful node shutdown — Reaches Beta in Kubernetes 1.26. When a node shuts down unexpectedly, pods get stuck in Terminating and StatefulSet pods cannot be rescheduled due to duplicate name constraints. Add the out-of-service taint to the node to force pod migration to another node. Remove the taint manually after the node recovers. For more information, see Non-graceful node shutdown.

RetroactiveDefaultStorageClass — Graduates to Beta in Kubernetes 1.26 and is enabled by default. A default StorageClass is automatically applied to existing PVCs that have no StorageClass assigned, eliminating the need to delete and recreate those PVCs.

JobPodFailurePolicy — Graduates to Beta in Kubernetes 1.26. Define a podFailurePolicy in a Job to handle pod failures based on exit codes and pod status, avoiding unnecessary retries and ignoring eviction-related failures. For more information, see Using Pod failure policy to ignore Pod disruptions.

API Priority and Fairness (APF) seat borrowing — Two new fields in .spec.limited: lendablePercent (percentage of seats this priority level can lend out) and borrowingLimitPercent (maximum percentage of seats this level can borrow from others).

httpGet lifecycle hook improvements — httpGet for preStop and postStart hooks now respects the scheme and headers fields, consistent with probes. Custom headers and HTTPS are supported. If HTTPS is misconfigured, the system falls back to HTTP without breaking existing hooks. Disable with --feature-gates=ConsistentHTTPGetHandlers=false on the kubelet.

HPA improvements — Two changes to HorizontalPodAutoscaler (HPA) behavior:

• Set --concurrent-horizontal-pod-autoscaler-syncs on kube-controller-manager to control the number of HPA controller workers.

• When multiple HPAs select overlapping pods or Deployments, the HPAs are disabled and an AmbiguousSelector event is generated.

Multiple default StorageClasses — When multiple StorageClasses are annotated as default with storageclass.kubernetes.io/is-default-class, Kubernetes 1.26 selects the newest one instead of returning an error.

Image registry redirect — In Kubernetes 1.25, k8s.gcr.io is redirected to registry.k8s.io. For more information, see k8s.gcr.io Redirect to registry.k8s.io.

NetworkPolicy EndPort — The endPort field reaches GA in Kubernetes 1.25, allowing a range of ports in a NetworkPolicy rule. If the network plugin does not support endPort, only the single port value is applied.

KMS v2 alpha — Introduced in Kubernetes 1.25. Replaces AES-CBC with AES-GCM for encrypting Secrets at rest using DEK. Existing data encrypted with AES-CBC can still be decrypted. For more information, see Use a KMS provider for data encryption.

Container Object Storage Interface (COSI) — Introduced in Alpha in Kubernetes 1.25 to standardize object storage provisioning and consumption. For more information, see COSI.

PodHasNetwork condition — Introduced in Alpha in Kubernetes 1.25. When this condition is True, the pod sandbox is initialized and the network is configured. Use this field to measure pod initialization latency excluding image pull and application startup. Enable the PodHasNetworkCondition feature gate on the kubelet. For more information, see PodHasNetwork.

User namespaces for pods — Alpha support added in Kubernetes 1.25. Maps a root user inside a pod to a non-zero host UID, improving isolation. Enable the UserNamespacesStatelessPodsSupport feature gate and make sure the container runtime supports this feature. For more information, see Kubernetes 1.25: alpha support for running Pods with user namespaces.

PodTopologySpread minDomains — Beta support for minDomains is added in Kubernetes 1.25, along with a fix for uneven pod spread during rolling updates.

kube-proxy performance improvements — In large clusters (1,000+ endpoints), unused iptables rules are retained until the next sync cycle instead of being scanned and deleted immediately. In small clusters, unused rules are removed immediately.

Windows improvements — Kubernetes 1.25 adds CI unit tests, conformance tests, and a new repository for Windows operational readiness. For more information, see windows-operational-readiness.

Deprecated APIs

APIs removed in Kubernetes 1.25 and 1.26 can no longer be used to create or manage the affected resources. Migrate to the replacement API versions before upgrading.

CronJob

batch/v1beta1 is removed in Kubernetes 1.25. Use batch/v1, available since Kubernetes 1.21.

EndpointSlice

discovery.k8s.io/v1beta1 is removed in Kubernetes 1.25. Use discovery.k8s.io/v1, available since Kubernetes 1.21.

Key field changes in discovery.k8s.io/v1:

Event

events.k8s.io/v1beta1 is removed in Kubernetes 1.25. Use events.k8s.io/v1, available since Kubernetes 1.19.

Key changes in events.k8s.io/v1:

• type accepts only Normal or Warning.

• involvedObject is renamed to regarding.

• action, reason, reportingController, and reportingInstance are required fields.

• eventTime replaces firstTimestamp (renamed to deprecatedFirstTimestamp, not allowed in v1).

• series.lastObservedTime replaces lastTimestamp (renamed to deprecatedLastTimestamp, not allowed in v1).

• series.count replaces count (renamed to deprecatedCount, not allowed in v1).

• reportingController replaces source.component (renamed to deprecatedSource.component, not allowed in v1).

• reportingInstance replaces source.host (renamed to deprecatedSource.host, not allowed in v1).

PodDisruptionBudget

policy/v1beta1 is removed in Kubernetes 1.25. Use policy/v1, available since Kubernetes 1.21.

Behavior change in policy/v1: an empty spec.selector ({}) selects all pods in the namespace. In policy/v1beta1, an empty selector selected no pods. Omitting spec.selector selects no pods in both versions.

PodSecurityPolicy

policy/v1beta1 for PodSecurityPolicy is removed in Kubernetes 1.25. Migrate to Pod Security Admission or a third-party admission webhook. See Migrate from PodSecurityPolicy to the Built-in PodSecurity Admission Controller.

RuntimeClass

node.k8s.io/v1beta1 is removed in Kubernetes 1.25. Use node.k8s.io/v1, available since Kubernetes 1.20.

HorizontalPodAutoscaler

• autoscaling/v2beta1 is removed in Kubernetes 1.25.

• autoscaling/v2beta2 is removed in Kubernetes 1.26.

Use autoscaling/v2, available since Kubernetes 1.23.

Flow control resources (FlowSchema and PriorityLevelConfiguration)

flowcontrol.apiserver.k8s.io/v1beta1 is removed in Kubernetes 1.26. Use:

• flowcontrol.apiserver.k8s.io/v1beta2, available since Kubernetes 1.23

• flowcontrol.apiserver.k8s.io/v1beta3, available since Kubernetes 1.26

Removed storage drivers

CSI migration reaches GA in Kubernetes 1.25, completing the move from in-tree volume plugins to out-of-tree Container Storage Interface (CSI) plugins.

The following in-tree storage drivers are removed or deprecated:

Removed command-line arguments and flags

kube-proxy

The userspace mode is removed in Kubernetes 1.26. Using --mode userspace returns an error. Linux users should use iptables or IPVS. Windows users should use kernelspace. Windows winkernel kube-proxy no longer supports Windows HNS v1 APIs.

kubectl

• --prune-whitelist is deprecated in Kubernetes 1.26 and replaced by --prune-allowlist per the Inclusive Naming Initiative.

• The following subcommands in kubectl run are deprecated and will be removed in a future version: --cascade, --filename, --force, --grace-period, --kustomize, --recursive, --timeout, and --wait.

API server

The --master-service-namespace flag is deprecated in Kubernetes 1.26. The flag is unused.

kube-controller-manager

Logging-related flags deprecated in earlier versions are removed in Kubernetes 1.26.

kubelet

The DynamicKubeletConfig feature gate is removed from the kubelet in Kubernetes 1.24 and from the API server in Kubernetes 1.26. To update kubelet configuration dynamically, edit the kubelet configuration file and restart the kubelet.

kubeadm

• UnversionedKubeletConfigMap reaches GA in Kubernetes 1.25. The ConfigMap name changes from kubelet-config-<x.yy> to kubelet-config.

• kubeadm no longer adds the node-role.kubernetes.io/master:NoSchedule label to control plane nodes in Kubernetes 1.25. The label is removed when running kubeadm upgrade apply.

• Seccomp annotations seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io are no longer supported in Kubernetes 1.25. Use SeccompProfile instead. For more information, see Restrict a Container's Syscalls with seccomp.

iptables chain ownership cleanup

Kubernetes does not guarantee that externally created iptables chains (such as KUBE-MARK-DROP, KUBE-MARK-MASQ, and KUBE-POSTROUTING in the NAT table) will persist across versions. The IPTablesCleanup feature gate introduced in Kubernetes 1.25 progressively stops creating these chains. For more information, see Kubernetes's IPTables Chains Are Not API.

Removed in-cluster credential management code

In-tree credential management code for Azure and Google Cloud is removed from client-go and kubectl in Kubernetes versions later than 1.26. Use authentication plugins instead. For more information, see Authentication plugins.

GA in Kubernetes 1.25

Beta in Kubernetes 1.25

Alpha in Kubernetes 1.25

GA in Kubernetes 1.26

Beta in Kubernetes 1.26

Alpha in Kubernetes 1.26