Upgrading Kubernetes on Huawei DCS

This guide explains how to complete Phase 2 of the upgrade workflow for clusters on Huawei DCS. Before you upgrade Kubernetes, complete the Distribution Version upgrade described in Upgrading Clusters.

INFO

Version

DCS provider v1.0.16 is the first release that supports pool-managed persistent disks.

INFO

Existing Cluster Migration

If your cluster runs ACP v4.2.1 or later and you are moving to DCS provider v1.0.16 or later, complete the migration procedure in Migrate Existing Huawei DCS Clusters to Pool-Managed Persistent Disks before you rely on upgrade-time disk preservation.

Upgrade Sequence

Upgrade DCS clusters in the following order:

  1. (Prerequisite) Upgrade the ACP platform on the management cluster first. This brings the cluster-api-provider-dcs controller and the related CAPI components (core, KubeadmControlPlane provider, bootstrap provider) to versions that understand the new schema. Trigger workload-cluster upgrades only after the management-side controllers have rolled out and become Ready.
  2. Upgrade the Distribution Version (Aligned Extensions) on the workload cluster. See Upgrading Distribution Version.
  3. Upgrade the control plane Kubernetes version.
  4. Upgrade worker nodes to the target Kubernetes version.

Cluster API orchestrates rolling updates with built-in safety mechanisms to reduce service disruption.

WARNING

Skipping step 1 risks two failure modes: the old controller silently ignores new schema fields written to DCSIpHostnamePool / DCSMachineTemplate; or a controller image swap mid-rollout interrupts persistent-disk state-machine progression. Always settle the management-side upgrade before touching workload rollout.

Prerequisites

Before you start, ensure all of the following prerequisites are met:

  • The Distribution Version upgrade is complete
  • The control plane is reachable
  • All nodes are healthy and in Ready state
  • The IP Pool has sufficient capacity for rolling updates
  • The VM template supports the target Kubernetes version. See OS Support Matrix for version mapping
  • The target Kubernetes version is compatible with your workloads and add-ons
  • DCS VM templates are 4.2.1 or later if you use pool-managed persistent disks, because safe shutdown and disk detach depend on guest tools
  • If you rely on pool-managed persistent disks, keep KubeadmControlPlane.spec.rolloutStrategy.rollingUpdate.maxSurge = 0 and each MachineDeployment.spec.strategy.rollingUpdate.maxSurge = 0
WARNING

What Is Preserved During Upgrades

Upgrades rely on Cluster API's rolling update mechanism.

  • Disks declared in DCSIpHostnamePool.spec.pool[].persistentDisk are detached from old VMs and reattached to replacement VMs.
  • This is the supported way to preserve node-local data such as the platform-required /var/cpaas disk.
  • Template disks in DCSMachineTemplate and any undeclared disks are not preserved across delete-recreate replacement.
  • The system disk is never treated as preserved state.
  • This preservation path requires one-by-one replacement, so keep maxSurge = 0 on the control plane and worker rollout strategies.

If your existing cluster still keeps preserved data in the old template-disk layout, migrate it first by following Migrate Existing Huawei DCS Clusters to Pool-Managed Persistent Disks.

Using YAML

YAML-based upgrades do not depend on Fleet Essentials.

Upgrade Control Plane Infrastructure

Upgrading the control plane machine template lets you roll out updated VM specifications, system patches, and infrastructure settings.

Procedure

  1. Create an updated machine template

    Copy the existing DCSMachineTemplate referenced by KubeadmControlPlane and save it as a new file:

    kubectl get dcsmachinetemplate <current-template-name> -n cpaas-system -o yaml > new-cp-template.yaml
  2. Modify the template specifications

    Update the new template as needed:

    • Set metadata.name to <new-template-name>
    • Update spec.template.spec.vmTemplateName
    • Update spec.template.spec.vmConfig.dcsMachineCpuSpec.quantity
    • Update spec.template.spec.vmConfig.dcsMachineMemorySpec.quantity
    • Update spec.template.spec.vmConfig.dcsMachineDiskSpec for system and template-local disks only

    Keep pool-managed persistent disks, including /var/cpaas, in DCSIpHostnamePool.spec.pool[].persistentDisk.

  3. Apply the updated template

    kubectl apply -f new-cp-template.yaml -n cpaas-system
  4. Update the control plane reference

    Modify the KubeadmControlPlane resource to reference the new template:

    kubectl patch kubeadmcontrolplane <kcp-name> -n cpaas-system --type='merge' -p='{"spec":{"machineTemplate":{"infrastructureRef":{"name":"<new-template-name>"}}}}'
  5. Monitor the rolling update

    kubectl get kubeadmcontrolplane <kcp-name> -n cpaas-system -w
    kubectl get machines -n cpaas-system -l cluster.x-k8s.io/control-plane

Upgrade Control Plane Kubernetes Version

Upgrading the control plane Kubernetes version requires both a compatible VM template and an updated KubeadmControlPlane version.

Procedure

  1. Prepare a compatible VM template

    Ensure the referenced DCSMachineTemplate uses a vmTemplateName that matches the target Kubernetes version.

  2. Update the control plane version

    Modify spec.version in the KubeadmControlPlane resource. Optionally update related rollout settings or the referenced infrastructure template at the same time to match your upgrade policy.

  3. Monitor the upgrade

    kubectl get kubeadmcontrolplane <kcp-name> -n cpaas-system
    kubectl get machines -n cpaas-system -l cluster.x-k8s.io/control-plane
    kubectl get nodes

Upgrade Worker Nodes

Worker node Kubernetes upgrades are managed through MachineDeployment resources.

Procedure

  1. Create an updated machine template

    • Create a new DCSMachineTemplate with a vmTemplateName that supports the target Kubernetes version
    • Ensure the VM template includes the correct Kubernetes binaries and dependencies
    • Keep /var/cpaas and any other upgrade-preserved disks in the IP pool rather than reintroducing them as template disks
  2. Update the MachineDeployment

    • Update spec.template.spec.version to the target Kubernetes version
    • Update spec.template.spec.infrastructureRef.name to reference the new machine template
    • Optionally update spec.template.spec.bootstrap.configRef.name if bootstrap configuration changes are required
  3. Monitor the rolling update

    • Verify that the rolling update completes successfully
    • Verify that the new worker nodes join the cluster with the target Kubernetes version

Using the Web UI

Use this workflow to upgrade Kubernetes from the web UI after Phase 1 is complete.

Version requirement: This workflow requires Fleet Essentials and Alauda Container Platform DCS Infrastructure Provider 1.0.13 or later. If the provider version is earlier than 1.0.13, use YAML manifests. If the upgrade relies on pool-managed persistent disks, use DCS provider v1.0.16 or later. In v1.0.16, the persistentDisk declaration on DCSIpHostnamePool remains YAML-only and is not exposed in the web UI.

Prerequisites

  • The Distribution Version upgrade is complete. See Upgrading Distribution Version
  • The Control Plane Node Pool is in Running state
  • The IP Pool has sufficient capacity for rolling updates
  • If the upgrade relies on pool-managed persistent disks, ensure the required DCSIpHostnamePool.spec.pool[].persistentDisk entries have already been created or updated through YAML

Upgrade Workflow

Kubernetes upgrades follow this sequence after the Distribution Version upgrade:

  1. Upgrade the Control Plane Node Pool.
  2. Wait for the Control Plane Node Pool upgrade to complete.
  3. Upgrade Worker Node Pools in any order.

Checking Available Upgrades

Navigation: Clusters → Clusters → Select cluster → Node Pools Tab

Node Pools with available upgrades show Upgrade available indicator. Click on the Node Pool card to view Current vs Target versions.

Upgrade the Control Plane Node Pool

Steps:

  1. In the Node Pools Tab, locate the Control Plane Node Pool
  2. Click Upgrade
  3. Review upgrade information:
    • Current Version: Current Kubernetes version
    • Target Version: Latest minor version supported (automatically selected)
  4. Click Confirm to start

Monitoring:

  • Watch the Node Pool status
  • Nodes will roll update one by one (maxSurge=0). This one-by-one replacement is also required when the cluster relies on persistent disks.
  • Upgrade time depends on node count and resources

Upgrade Worker Node Pools

WARNING

Worker Node Pools cannot be upgraded until the Control Plane Node Pool upgrade completes.

When Worker Pool Upgrade is Available:

  • Control Plane Kubernetes Version matches global cluster version
  • Control Plane is in Running state

Upgrade Steps:

  1. For each Worker Node Pool:
    • Click Upgrade button
    • Review and confirm
  2. Pools can be upgraded in parallel after Control Plane completes

Upgrade Constraints:

Pool StateUpgrade Button
Not Running❌ Disabled: "Upgrade is unavailable when the Worker Node Pool is not in the Running state"
Control Plane not started❌ Disabled: "Upgrade the Control Plane Node Pool first"
Control Plane upgrading❌ Disabled: "Wait for the Control Plane Node Pool upgrade to complete"
Control Plane upgraded✅ Enabled

Cross-Version Upgrades

When upgrading across multiple minor versions (for example, v1.32 → v1.34):

  1. Upgrade Control Plane to v1.33
  2. Wait for completion
  3. Upgrade Control Plane to v1.34
  4. Repeat for Worker Pools

Why: Kubernetes only supports single minor version upgrades.

Troubleshooting

IssueSolution
Upgrade button disabledCheck pool status and Control Plane version
Upgrade stuckCheck IP Pool availability, DCS platform resources
Nodes not joiningVerify network connectivity, DNS settings
Preserved disk did not reattachVerify the disk is declared in DCSIpHostnamePool.spec.pool[].persistentDisk, check status.persistentDiskStatus, and confirm the cluster has already been migrated to the pool-managed layout

Additional Resources