v1.27.X
Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.
Release v1.27.12+rke2r1
This release updates Kubernetes to v1.27.12.
Canal uses flannel 0.24.3 which includes a bug: every 5 seconds it tries to add ipv6 iptables rules and fails if the node does not have an ipv6 address. The consequence is the log "Failed to ensure iptables rules: error setting up rules: failed to apply partial iptables-restore unable to run iptables-restore (, ): exit status 4" appears every 5 seconds in the flannel container of the canal pod.
Flannel daemonset is not tolerating node taints: "node-role.kubernetes.io/etcd:NoExecute", "node-role.kubernetes.io/control-plane:NoSchedule" and "node.cloudprovider.kubernetes.io/uninitialized:NoSchedule" which can create problems when deploying with Rancher in certain cloud-providers (e.g. vShpere or DigitalOcean).
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.11+rke2r1:
- Add a multus e2e test (#5546)
- Bump vsphere csi chart to 3.1.2-rancher101 and cpi to 1.7.001 (#5555)
- Bump coredns chart (#5562)
- Update 1.27 to r2 (#5565)
- Update Calico and Canal to v3.27.2 (#5584)
- Bump multus chart version (#5595)
- Bump K3s version for v1.27 (#5589)
- Fix: use correct wasm shims names
- Bump spegel to v0.0.18-k3s3
- Adds wildcard registry support
- Fixes issue with excessive CPU utilization while waiting for containerd to start
- Add env var to allow spegel mirroring of latest tag
- Bump helm-controller/klipper-helm versions
- Fix snapshot prune
- Fix issue with etcd node name missing hostname
- Fix additional corner cases in registries handling
- RKE2 will now warn and suppress duplicate entries in the mirror endpoint list for a registry. Containerd does not support listing the same endpoint multiple times as a mirror for a single upstream registry.
- Bump K3s version for v1.27 (#5607)
- Update k8s to 1.27.12 and Go (#5622)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.15.100 |
| rke2-canal | v3.27.2-build2024030800 |
| rke2-calico | v3.27.200 |
| rke2-calico-crd | v3.27.002 |
| rke2-coredns | 1.29.002 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051513 |
| rancher-vsphere-csi | 3.1.2-rancher101 |
| rancher-vsphere-cpi | 1.7.001 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1700 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.27.11+rke2r1
This release updates Kubernetes to v1.27.11.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.10+rke2r1:
- Fix validate-charts script (#5376)
- Windows (#5369)
- Avoid race condition when deleting HNS networks (#5384)
- Add CNI plugin flannel support for Windows (#5395)
- Bump coredns and multus/whereabouts versions (#5401)
- Fix: missing 'ip link delete cilium_wg0' in rke2-killall.sh (#5407)
- Update canal version (#5417)
- Improve calico in windows (#5428)
- Update Calico to v3.27.0 (#5437)
- Update Cilium to 1.15.0 (#5452)
- Backport agent containerd behavior 1.27 (#5456)
- Bump K3s version for v1.27 (#5460)
- Bump harvester-csi-driver to 0.1.7 (#5441)
- Update k8s and Go (#5466)
- Update Cilium to 1.15.1 (#5478)
- Bump rke2-coredns chart (#5498)
- Bump K3s for etcd-only fix (#5503)
- Add new network policy for ingress controller webhook (#5511)
- Backport bugfixes for 2024-02 rc (#5532)
- Bump wharfie to v0.6.6 to add support for bare hostname as endpoint, fix unnecessary namespace param inclusion
- Refactor netpol creation and add two new netpols for metrics-server and snapshot-validation-webhook
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.15.100 |
| rke2-canal | v3.27.0-build2024020601 |
| rke2-calico | v3.27.002 |
| rke2-calico-crd | v3.27.002 |
| rke2-coredns | 1.29.001 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051513 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1700 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.27.10+rke2r1
This release updates Kubernetes to v1.27.10.
Addresses the runc CVE: CVE-2024-21626 by updating runc to v1.1.12.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.9+rke2r1:
- Use dl.k8s.io for getting kubectl (#5180)
- Ensure charts directory exists in Windows runtime image (#5186)
- Bump versions of different components (#5168)
- Update coredns chart to fix bug (#5201)
- Update multus chart to add optional dhcp daemonset (#5211)
- Add e2e test about dnscache (#5227)
- Update rke2-whereabouts to v0.6.3 and bump rke2-multus parent chart (#5245)
- Bump sriov image build versions (#5255)
- Enable arm64 based images for calico, multus and harvester (#5266)
- Improve kube-proxy and calico logging in Windows (#5285)
- Bump k3s for v1.27 (#5270)
- Update to 1.27.10 (#5294)
- Update base image (#5307)
- Bump K3s and runc versions for v1.27 (#5351)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.008 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051511 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.27.9+rke2r1
This release updates Kubernetes to v1.27.9.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.8+rke2r1:
- Bump containerd and runc (#5120)
- Bumped containerd/runc to v1.7.10/v1.1.10
- Bump containerd to v1.7.11 (#5130)
- Update to 1.27.9 for december 2023 (#5151)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.006 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051510 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.27.8+rke2r1
This release updates Kubernetes to v1.27.8.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.7+rke2r2:
- Add chart validation tests (#5001)
- Update canal to v3.26.3 (#5016)
- Update calico to v3.26.3 (#5026)
- Bump cilium chart to 1.14.400 (#5060)
- Bump K3s version for v1.27 (#5030)
- Containerd may now be configured to use rdt or blockio configuration by defining
rdt_config.yamlorblockio_config.yamlfiles. - Disable helm CRD installation for disable-helm-controller
- Omit snapshot list configmap entries for snapshots without extra metadata
- Add jitter to client config retry to avoid hammering servers when they are starting up
- Containerd may now be configured to use rdt or blockio configuration by defining
- Bump K3s version for v1.27 (#5070)
- Don't apply S3 retention if S3 client failed to initialize
- Don't request metadata when listing S3 snapshots
- Print key instead of file path in snapshot metadata log message
- Kubernetes patch release (#5065)
- Remove s390x steps since the runners are disabled (#5096)
Charts Versions
| Component | Version |
|---|---|
| rke2-cilium | 1.14.400 |
| rke2-canal | v3.26.3-build2023110900 |
| rke2-calico | v3.26.300 |
| rke2-calico-crd | v3.26.300 |
| rke2-coredns | 1.24.006 |
| rke2-ingress-nginx | 4.8.200 |
| rke2-metrics-server | 2.11.100-build2023051510 |
| rancher-vsphere-csi | 3.0.1-rancher101 |
| rancher-vsphere-cpi | 1.5.100 |
| harvester-cloud-provider | 0.2.200 |
| harvester-csi-driver | 0.1.1600 |
| rke2-snapshot-controller | 1.7.202 |
| rke2-snapshot-controller-crd | 1.7.202 |
| rke2-snapshot-validation-webhook | 1.7.302 |
Release v1.27.7+rke2r2
This release fixes an issue with identifying additional container runtimes.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.7+rke2r1:
- Bump k3s, include container runtime fix (#4980)
- Fixed an issue with identifying additional container runtimes
- Update hardened kubernetes image (#4987)
Release v1.27.7+rke2r1
This release updates Kubernetes to v1.27.7.
This release includes a version of ingress-nginx affected by CVE-2023-5043 and CVE-2023-5044. Ingress administrators should set the --enable-annotation-validation flag to enforce restrictions on the contents of ingress-nginx annotation fields.
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.6+rke2r1:
- Add a time.Sleep in calico-win to avoid polluting the logs (#4791)
- Support generic "cis" profile (#4797)
- Update calico chart to accept felix config values (#4814)
- Remove unnecessary docker pull (#4823)
- Mirrored pause backport (#4828)
- Write pod-manifests as 0600 in cis mode (#4838)
- Filter release branches (#4857)
- Bump k3s version (#4851)
- Update charts to have ipFamilyPolicy: PreferDualStack as default (#4845)
- Bump K3s, Token Rotation support (#4869)
- Bump containerd to v1.7.7+k3s1 (#4880)
- Bump K3s version for v1.27 (#4884)
- RKE2 now tracks snapshots using custom resource definitions. This resolves an issue where the configmap previously used to track snapshot metadata could grow excessively large and fail to update when new snapshots were taken.
- Fixed an issue where static pod startup checks may return false positives in the case of pod restarts.
- K3s bump (#4897)
- Bump rke2-cloud-controller to v1.28.2-build20231016 (#4903)
- Bump K3s version for v1.27 (#4917)
- Re-enable etcd endpoint auto-sync
- Manually requeue configmap reconcile when no nodes have reconciled snapshots
- Update Kubernetes to v1.27.7 (#4922)
- Remove pod-manifests dir in killall script (#4926)
- Revert mirrored pause backport (#4935)
- Bump ingress-nginx to v1.9.3 (#4956)
- Bump K3s version for v1.27 (#4969)
Release v1.27.6+rke2r1
This release updates Kubernetes to v1.27.5.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.5+rke2r1:
Release v1.27.5+rke2r1
This release updates Kubernetes to v1.27.5, and fixes a number of issues.
-
⚠️ This release includes support for remediating CVE-2023-32186, a potential Denial of Service attack vector on RKE2 servers. See https://github.com/rancher/rke2/security/advisories/GHSA-p45j-vfv5-wprq for more information, including mandatory steps necessary to harden clusters against this vulnerability.
-
If your server (control-plane) nodes were not started with the
--tokenCLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.4+rke2r1:
- Sync maintainers and PR template from K3s (#4474)
- Fix static pod UID generation and cleanup (#4508)
- Security bump to
docker/distribution(#4509) - Fix incorrect documented default value for INSTALL_RKE2_CHANNEL (#4500)
- Uninstall handle cases when directories are mounts and cannot be removed (#4470)
- Remove install_airgap_tarball grep error output (#4501)
- Update canal with resource bounds config (#4482)
- Channel server update (#4518)
- Fix default server address for rotate-ca command (#4548)
- Sync Felix and calico-node datastore (#4570)
- Update Calico and Flannel on Canal (#4535)
- Update cilium to v1.14.0 (#4585)
- Remove terraform test package (#4589)
- Bump versions for etcd, containerd, runc (#4552)
- Updated the embedded containerd to v1.7.3+k3s1
- Updated the embedded runc to v1.1.8
- Updated the embedded etcd to v3.5.9+k3s1
- Updated the rke2-snapshot-validation-webhook chart to enable VolumeSnapshotClass validation
- Update certs list for certificates test (#4597)
- Update to whereabouts v0.6.2 (#4590)
- Updated the embedded whereabouts to v0.6.2
- Fix non-working URL in issue template (#4606)
- Fix wrongly formatted files (#4605)
- Fix calico-node.log problem (#4609)
- Add support for commit installation in Windows quickstart file (#4614)
- N/A
- Use 'go list -m' instead of grep to look up versions (#4600)
- Install BGP windows packages in Windows image for tests (#4639)
- Bump k3s version to recent 1.27 (#4630)
- Bump K3s version for v1.27 (#4646)
- The version of
helmused by the bundled helm controller's job image has been updated to v3.12.3 - Bumped dynamiclistener to address an issue that could cause the supervisor listener on 9345 to stop serving requests on etcd-only nodes.
- The RKE2 supervisor listener on 9345 now sends a complete certificate chain in the TLS handshake.
- The version of
- Clean-up env variables and check OS env variables for felix and calico in Windows (#4640)
- Upgrade multus chart to v4.0.2-build2023081100 (#4661)
- Bug fix: Add VXLAN_VNI env var to Calico-node exec (#4670)
- Update to v1.27.5 (#4683)
- Bump K3s version for v1.27 (#4701)
- Added a new
--tls-san-securityoption. This flag defaults to false, but can be set to true to disable automatically adding SANs to the server's TLS certificate to satisfy any hostname requested by a client.
- Added a new
- Add additional static pod cleanup during cluster reset (#4724)
Release v1.27.4+rke2r1
This release updates Kubernetes to v1.27.4, and fixes a number of issues.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.3+rke2r1:
- Update channel server (#4397)
- Bump ingress-nginx charts to v1.7.1 (#4402)
- Add opensuse leap testing to install tests (#4364)
- Add log files for felix and calico in rke2-windows (#4412)
- Update multus to version v4.0.2 (#4428)
- Update Calico to v3.26.1 (#4420)
- Fix failure to set default audit-log-path (#4413)
- Update K3s for 2023-07 releases (#4447)
- Improve clone step retries (#4408)
- Add support for cni none on windows and windows-bgp backend (#4164)
- Updated Calico crd on Canal (#4463)
- Update to 1.27.4 (#4494)
Release v1.27.3+rke2r1
This release updates Kubernetes to v1.27.3 and fixes a number of issues.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.27.2+rke2r1:
- Preserve mode when extracting runtime data (#4288)
- Add el9 (#4303)
- Update channels.yaml (#4306)
- Bump alpine from 3.17 to 3.18 (#4232)
- Ignore untracked branch pushes (#4265)
- DynamicListener version bump (v0.3.3 -> v0.3.5) (#4324)
- Update canal chart (#4339)
- Add issue template for OS validation (#4346)
- Refactoring of Restart Cluster Server and Add the Certificate Rotation (#4226)
- Bump harvester cloud provider 0.2.1 (#4337)
- Fix broken links (#4300)
- Bump rke2-coredns chart version (#4325)
- Add arm64 support (#4335)
- Bump K3s version for v1.27 (#4354)
- Update rke2 (#4369)
- Bump harvester cloud provider 0.2.2 (#4373)
- Fix windows pause (#4381)
- Use our own file copy logic instead of continuity (#4388)
Release v1.27.2+rke2r1
This release updates Kubernetes to v1.27.2, and fixes a number of issues.
- If your server (control-plane) nodes were not started with the
--tokenCLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
- Many systems have updated their packages with newer version of container-selinux (> v2.191.0) which is incompatible with our rke2-selinux policy and require a change in policy. We have updated our policy; you will notice the rke2-selinux package being upgraded from version v0.11.1 to newer version v0.12.0.
Changes since v1.27.1+rke2r1:
- V1.27.2+rke2r1 (#4261)
- Update stable channel to v1.25.9+rke2r1 (#4138)
- Updating dev doc (#3111)
- Add dependabot (#4133)
- Add updatecli (#4135)
- Fix hardcoded file mount handling for default audit log filename (#4139)
- Add ability to have write custom files during TF tests for specialized configurations (#4132)
- Bump ubuntu from 20.04 to 22.04 (#4154)
- Add reviewers to dependabot PRs (#4156)
- Introduce updatecli to repo and validate basic functionality (#4155)
- Add label to dependabot PRs (#4169)
- Create 'upgrade traditional with workloads' test automated for RKE2 (#4118)
- Update Cilium to v1.13.2 (#4170)
- Fix drone dispatch step (#4147)
- Enable --with-node-id flag (#4131)
- Chore: replace
github.com/ghodss/yamlwithsigs.k8s.io/yaml(#4163) - Remove Trivy install from Dockerfile (#4187)
- Move Drone dispatch pipeline (#4202)
- Bump K3s/containerd/runc versions (#4210)
- The bundled containerd and runc versions have been bumped to v1.7.1-k3s1/v1.1.7
- Update Calico image on Canal (#4214)
- Upgrade docker/docker package (#4225)
- Add rke2-upgrade to image list (#4237)
- Bump metrics-server to v0.6.3 (#4244)
- Fix fapolicyd checks in install script (#4249)
- Bump vsphere csi/cpi and csi snapshot charts (#4271)
- Bump vsphere csi to remove duplicate CSI deployment. (#4295)
Release v1.27.1+rke2r1
This release is RKE2's first in the v1.27 line. This release updates Kubernetes to v1.27.1.
Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.
Important Note
If your server (control-plane) nodes were not started with the --token CLI flag or config file key, a randomized token was generated during initial cluster startup. This key is used both for joining new nodes to the cluster, and for encrypting cluster bootstrap data within the datastore. Ensure that you retain a copy of this token, as is required when restoring from backup.
You may retrieve the token value from any server already joined to the cluster:
cat /var/lib/rancher/rke2/server/token
Changes since v1.26.4+rke2r1:
- Bump to kubernetes v1.27.1 (#4108)