Container Service for Kubernetes:(Discontinued) Kubernetes 1.24 release notes

CoreDNS

Update to CoreDNS 1.9.3.6-32932850-aliyun does not affect running workloads. Changes include:

• Affinity settings for CoreDNS scheduling are optimized, enabling CoreDNS to run on clusters with auto-scaling nodes.

• Caching of ServError DNS resolution results is disabled.

• Pod anti-affinity based on hostnames changes from preferred to required, forcing anti-affinity rules within the topological domains of specified nodes.

• Custom parameters are supported.

• Log parsing is enabled by default.

Flannel

• An init container for installing the Flannel plugin is added.

• /var/run is now the default directory for caching IP addresses, preventing IP leaks when nodes restart.

• ARM64 instances are supported.

• CVE-2022-28391 and CVE-2022-37434 are fixed.

Ingress Controller

The update may temporarily interrupt workloads and cause compatibility issues with workload configurations. Evaluate the impact before upgrading.

Breaking changes

Dockershim removed

Dockershim is removed in Kubernetes 1.24. Docker is not supported as a container runtime. Docker images continue to work — only the runtime changes. Use containerd for new nodes. See Dockershim Removal FAQ and the related GitHub issue.

kube-apiserver memory increase (Go 1.18 garbage collection)

Kubernetes 1.24 compiles with Go 1.18, which changes the garbage collection algorithm. The result: 99th percentile API response latency is reduced by 10x, but kube-apiserver memory usage increases by roughly 25%. To reduce memory usage back to previous levels, set the GOGC environment variable to 63.

Service account token auto-generation disabled

The LegacyServiceAccountTokenNoAutoGeneration feature gate is enabled by default. The Secret API no longer auto-generates Secrets for service account tokens. Use the TokenRequest API to generate tokens. All tokens issued by the TokenRequest API expire by default. To create a non-expiring token, see service-account-token-secrets.

Control plane node label change (ACK dedicated clusters)

kubeadm no longer applies the node-role.kubernetes.io/master label to control plane nodes. Only node-role.kubernetes.io/control-plane is added. Both node-role.kubernetes.io/master:NoSchedule and node-role.kubernetes.io/control-plane:NoSchedule taints are applied to new control plane nodes. The node-role.kubernetes.io/master:NoSchedule taint will be removed in Kubernetes 1.25.

NodePort port conflict with kernel parameter

kube-proxy no longer reserves ports for NodePort Services. If a NodePort Service uses a port within the net.ipv4.ip_local_port_range kernel parameter range on a node, TCP connections may fail intermittently. This can cause health check failures and service exceptions. Before upgrading, verify that no NodePort port in your cluster conflicts with the net.ipv4.ip_local_port_range value on any node. See the Kubernetes community PR.

Security fix: CVE-2022-3172

New features

Storage capacity scheduling

CSIStorageCapacity reaches GA. The API exposes available storage capacity so that pods are scheduled only to nodes with sufficient storage, avoiding failures caused by volume creation or mounting errors. See Storage Capacity Constraints for Pod Scheduling.

gRPC probes (public preview)

gRPC probes are available for public preview. The GRPCContainerProbe feature gate is enabled by default. See Configure probes.

CSI plugin improvements

The CSI plugin now supports in-tree persistent volume (PV) deletion protection finalizers and migration of in-tree storage plugins to out-of-tree CSI drivers. See In-tree Storage Plugin to CSI Migration Design Doc.

CronJob time zone support

CronJobs now accept an optional timeZone parameter. Enable the CronJobTimeZone feature gate to run CronJobs in a specified time zone. This feature will reach public preview in Kubernetes 1.25.

kube-proxy Windows support

Two new kube-proxy options enable kube-proxy to run on Windows:

• --forward-healthcheck-vip: Forwards health check requests destined for a Service's virtual IP address (VIP) to the kube-proxy health check Service.

• --root-hnsendpoint-name: Specifies the name of the Host Networking Service (HNS) endpoint for the root network namespace.

OpenAPI V3

OpenAPI V3 is enabled by default.

SHA-1 certificate verification disabled

Certificates signed with the SHA-1 hash algorithm can no longer be verified by default. This is a consequence of Kubernetes 1.24 compiling with Go 1.18.

iptables chain for detecting iptables mode

kubelet creates a chain named KUBE-IPTABLES-HINT in the mangle table. Containerized components that need to modify iptables rules in the host network namespace can use this chain to reliably detect whether the system uses iptables-legacy or iptables-nft.

New and updated metrics

Features reaching GA in Kubernetes 1.23 (carried into 1.24)

• Structured logging (public preview since 1.23): Components such as kube-scheduler and kubelet now log in structured key=value text format — for example, verb="GET" URI="/healthz" latency="57.126µs" resp=200 instead of GET /healthz: (57.126µs) 200. Regular expressions are no longer needed to parse these logs. Print logs in structured text or JSON format. By default, JSON-format logs are written to stderr. See Introducing Structured Logs.

• StatefulSet `.spec.minReadySeconds`: Supported by default (public preview since 1.23). See Minimum Ready Seconds for StatefulSets.

• CSI `StorageCapacity` field editable: CSIDriver.Spec.StorageCapacity can now be modified.

• JobReadyPods: The number of pods in Ready state is shown in the Job status field.

• CRD validation with CEL: The CustomResourceValidationExpressions feature gate uses Common Expression Language (CEL) via the x-kubernetes-validations extension to validate CustomResourceDefinitions (CRDs).

• Windows ARM64 client binaries: Client-side binaries can be built for Windows on ARM64 (Go 1.17+ support).

• client-go workqueue `Processing` state: When shutting down a workqueue in the Processing state, shutdown waits for all in-progress tasks to complete.

• `admission_webhook_request_total` metric: Includes webhook name, admission type, operation, HTTP status code, rejection status, and resource namespace as labels.

• API server metrics reach GA: controller_admission_duration_seconds, step_admission_duration_seconds, webhook_admission_duration_seconds, apiserver_current_inflight_requests, apiserver_response_sizes. A new metric monitors LIST requests to the Kubernetes API server.

• Scheduler metrics reach GA: pending_pods, preemption_attempts_total, preemption_victims, schedule_attempts_total, scheduling_attempt_duration_seconds (formerly e2e_scheduling_duration_seconds), pod_scheduling_duration_seconds, pod_scheduling_attempts, framework_extension_point_duration_seconds, plugin_execution_duration_seconds, queue_incoming_pods_total.

• kube-controller-manager health checks: All controllers are now included.

• `pod.Spec.OS` enforcement: If the OS field is set on a pod but does not match the node's OS label, kubelet rejects the pod.

kubectl improvements (1.23–1.24)

• kubectl logs now shows the first container's log by default.

• kubectl describe ingress now shows IngressClasses.

• kubectl version now shows the embedded Kustomize version.

• kubectl get supports resource name tab-completion (for example, kubectl get pod pod1 <TAB>).

• fish and PowerShell are now supported for code completion and hinting.

Deprecated and removed features

Removed in Kubernetes 1.24

Deprecated in Kubernetes 1.24

Deprecated in Kubernetes 1.23

API changes

Kubernetes 1.24

Kubernetes 1.23

Other API changes in Kubernetes 1.23

• HorizontalPodAutoscaler v2 reaches GA. The autoscaling/v2beta2 API version is deprecated.

• Service.spec.ipFamilyPolicy is required when creating or updating IPv4/IPv6 dual stack Services. Set it to PreferDualStack or RequireDualStack.

• Components configured via LogFormatRegistry must update to use the logr v1.0.0 API. JSON logs are exported in go-logr/zapr format.

Feature gates

Feature gates move through three stages: Alpha (disabled by default), Beta (enabled by default), and GA (always enabled, cannot be disabled). For full details, see Feature Gates.

Reaching GA in Kubernetes 1.24

Public preview in Kubernetes 1.24 (enabled by default)

Public preview in Kubernetes 1.24

Enabled by default in Kubernetes 1.24

Deprecated in Kubernetes 1.24

Removed in Kubernetes 1.24

Reaching GA in Kubernetes 1.23

Public preview in Kubernetes 1.23

GA across Kubernetes 1.23–1.24 (cannot be disabled)

ConfigurableFSGroupPolicy, ControllerManagerLeaderMigration, CSIMigrationAzureDisk, CSIMigrationOpenStack, CSIStorageCapacity, CSIVolumeFSGroupPolicy, CSRDuration, CronJobControllerV2, DefaultPodTopologySpread, EfficientWatchResumption, ExpandCSIVolumes, ExpandInUsePersistentVolumes, ExpandPersistentVolumes, GenericEphemeralVolume, IPv6DualStack, IndexedJob, IngressClassNamespacedParams, NonPreemptingPriority, PodAffinityNamespaceSelector, PodOverhead, PreferNominatedNode, RemoveSelfLink, ServiceLBNodePortControl, ServiceLoadBalancerClass, SuspendJob, DynamicKubeletConfig, TTLAfterFinished.