v1.28.X
Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.
Release v1.28.8+rke2r1
This release updates Kubernetes to v1.28.8.
Canal uses flannel 0.24.3 which includes a bug: every 5 seconds it tries to add ipv6 iptables rules and fails if the node does not have an ipv6 address. The consequence is the log "Failed to ensure iptables rules: error setting up rules: failed to apply partial iptables-restore unable to run iptables-restore (, ): exit status 4" appears every 5 seconds in the flannel container of the canal pod.
Flannel daemonset is not tolerating node taints: "node-role.kubernetes.io/etcd:NoExecute", "node-role.kubernetes.io/control-plane:NoSchedule" and "node.cloudprovider.kubernetes.io/uninitialized:NoSchedule" which can create problems when deploying with Rancher in certain cloud-providers (e.g. vShpere or DigitalOcean).
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.7+rke2r1:
- Add a multus e2e test (#5545)
- Bump vsphere csi chart to 3.1.2-rancher101 and cpi to 1.7.001 (#5554)
- Bump coredns chart (#5561)
- Update 1.28 to r2 (#5566)
- Update Calico and Canal to v3.27.2 (#5583)
- Bump multus chart version (#5594)
- Bump K3s version for v1.28 (#5588)
- Fix: use correct wasm shims names
- Bump spegel to v0.0.18-k3s3
- Adds wildcard registry support
- Fixes issue with excessive CPU utilization while waiting for containerd to start
- Add env var to allow spegel mirroring of latest tag
- Bump helm-controller/klipper-helm versions
- Fix snapshot prune
- Fix issue with etcd node name missing hostname
- Fix additional corner cases in registries handling
- RKE2 will now warn and suppress duplicate entries in the mirror endpoint list for a registry. Containerd does not support listing the same endpoint multiple times as a mirror for a single upstream registry.
- Bump K3s version for v1.28 (#5606)
- Update k8s to 1.28.8 and Go (#5623)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.15.100 |
| rke2-canal | v3.27.2-build2024030800 |
| rke2-calico | v3.27.200 |
| rke2-calico-crd | v3.27.002 |
| rke2-coredns | 1.29.002 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051513 |
| rancher-vsphere-csi | 3.1.2-rancher101 |
| rancher-vsphere-cpi | 1.7.001 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1700 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.28.7+rke2r1
This release updates Kubernetes to v1.28.7.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.6+rke2r1:
- Fix validate-charts script (#5377)
- Windows (#5368)
- Avoid race condition when deleting HNS networks (#5383)
- Add CNI plugin flannel support for Windows (#5394)
- Bump coredns and multus/whereabouts versions (#5400)
- Fix: missing 'ip link delete cilium_wg0' in rke2-killall.sh (#5406)
- Update canal version (#5416)
- Improve calico in windows (#5427)
- Update Calico to v3.27.0 (#5436)
- Update Cilium to 1.15.0 (#5451)
- Backport agent containerd behavior 1.28 (#5457)
- Bump K3s version for v1.28 (#5459)
- Bump harvester-csi-driver to 0.1.7 (#5440)
- Update k8s and Go (#5467)
- Update Cilium to 1.15.1 (#5477)
- Bump rke2-coredns chart (#5497)
- Bump K3s for etcd-only fix (#5502)
- Add new network policy for ingress controller webhook (#5510)
- Backport bugfixes for 2024-02 rc (#5531)
- Bump wharfie to v0.6.6 to add support for bare hostname as endpoint, fix unnecessary namespace param inclusion
- Refactor netpol creation and add two new netpols for metrics-server and snapshot-validation-webhook
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.15.100 |
| rke2-canal | v3.27.0-build2024020601 |
| rke2-calico | v3.27.002 |
| rke2-calico-crd | v3.27.002 |
| rke2-coredns | 1.29.001 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051513 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1700 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.28.6+rke2r1
This release updates Kubernetes to v1.28.6.
Addresses the runc CVE: CVE-2024-21626 by updating runc to v1.1.12.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.5+rke2r1:
- Ensure charts directory exists in Windows runtime image (#5187)
- Use dl.k8s.io for getting kubectl (#5181)
- Update coredns chart to fix bug (#5200)
- Update multus chart to add optional dhcp daemonset (#5210)
- Update rke2-whereabouts to v0.6.3 and bump rke2-multus parent chart (#5242)
- Add e2e test about dnscache (#5226)
- Bump sriov image build versions (#5254)
- Enable arm64 based images for calico, multus and harvester (#5265)
- Improve kube-proxy and calico logging in Windows (#5284)
- Bump k3s for v1.28 (#5269)
- Update to 1.28.6 (#5295)
- Update base image (#5306)
- Bump K3s and runc versions for v1.28 (#5350)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.008 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051511 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.28.5+rke2r1
This release updates Kubernetes to v1.28.5.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.4+rke2r1:
- Update stable channel to v1.26.11+rke2r1 (#5099)
- Bump containerd and runc (#5117)
- Bumped containerd/runc to v1.7.10/v1.1.10
- Bump containerd to v1.7.11 (#5129)
- Added support for amazon linux 2023 (#4973)
- Added support for Amazon Linux 2023 (#4973)
- Update to 1.28.5 for december 2023 (#5150)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.007 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051511 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.28.4+rke2r1
This release updates Kubernetes to v1.28.4.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.3+rke2r2:
- Add chart validation tests (#4615)
- Update stable channel to v1.26.10+rke2r2 (#4995)
- Update canal to v3.26.3 (#5007)
- Update calico to v3.26.3 and fix nodeAddressAutodetectionV4 issue (#5022)
- Bump cilium chart to 1.14.400 (#5054)
- Bump K3s version for v1.28 (#5029)
- Containerd may now be configured to use rdt or blockio configuration by defining
rdt_config.yamlorblockio_config.yamlfiles. - Disable helm CRD installation for disable-helm-controller
- Omit snapshot list configmap entries for snapshots without extra metadata
- Add jitter to client config retry to avoid hammering servers when they are starting up
- Containerd may now be configured to use rdt or blockio configuration by defining
- Bump K3s version for v1.28 (#5069)
- Don't apply S3 retention if S3 client failed to initialize
- Don't request metadata when listing S3 snapshots
- Print key instead of file path in snapshot metadata log message
- Kubernetes patch release (#5066)
- Remove s390x steps since the runners are disabled (#5095)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.007 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051511 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.28.3+rke2r2
This release fixes an issue with identifying additional container runtimes.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.3+rke2r1:
- Update stable channel to v1.26.10+rke2r1 (#4977)
- Bump k3s, include container runtime fix (#4979)
- Fixed an issue with identifying additional container runtimes
- Update hardened kubernetes image (#4988)
Release v1.28.3+rke2r1
This release updates Kubernetes to v1.28.3.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.2+rke2r1:
- Add a time.Sleep in calico-win to avoid polluting the logs (#4723)
- Update stable channel to v1.26.9 (#4774)
- Bump actions/checkout from 3 to 4 (#4746)
- Fix .github regex to skip drone runs on gh action bumps (#4800)
- Add skip fapolicy option (#4673)
- Update calico chart to accept felix config values (#4802)
- Handle restart attempts in static pod manifest checks (#4784)
- Fixed an issue where static pod startup checks may return false positives in the case of pod restarts
- Remove unnecessary docker pull (#4820)
- Update charts to have ipFamilyPolicy: PreferDualStack as default (#4780)
- Use ipFamilyPolicy: PreferDualStack for system services: coredns, metrics-server, nginx and snapshot-validation-webhook
- Mirrored pause update (#4829)
- Fix function name on comment (#4668)
- Fix slemicro check for selinux (#4830)
- Write pod-manifests as 0600 in cis mode (#4831)
- Filter to not accept dependabot and updatecli branches (#4841)
- Bump k3s version in go.mod (#4850)
- Bump cilium to 1.14.2 (#4837)
- Bump K3s, Token Rotation support (#4866)
- Bump containerd to v1.7.7+k3s1 (#4879)
- Remove SECURITY.md (#4868)
- Bump K3s version for v1.28 (#4883)
- RKE2 now tracks snapshots using custom resource definitions. This resolves an issue where the configmap previously used to track snapshot metadata could grow excessively large and fail to update when new snapshots were taken.
- Bump dependencies (#4865)
- Bump k3s (#4896)
- Bump rke2-cloud-controller to v1.28.2-build20231016 (#4895)
- Bump K3s version for v1.28 (#4916)
- Re-enable etcd endpoint auto-sync
- Manually requeue configmap reconcile when no nodes have reconciled snapshots
- Update Kubernetes to v1.28.3 (#4923)
- Fix: upgrading Go in go.mod to 1.20 (#4911)
- Remove pod-manifests dir in killall script (#4929)
- Bump ingress-nginx to v1.9.3 (#4955)
- Bump K3s version for v1.28 (#4968)
Release v1.28.2+rke2r1
This release updates Kubernetes to v1.28.2.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.28.1+rke2r1:
- Support new generic "cis" profile (#4708)
- Update cilium to 1.14.1 (#4755)
- Update Kubernetes to v1.28.2 Go to v1.20.8 (#4760)