Troubleshooting Huawei DCS Workload Clusters
Start from the preflight error or the stage that ac adm alpha cluster status reports. For failures that every provider shares, see Troubleshooting. Run the ac commands on this page from a machine where ac is logged in to the platform, after ac config use-cluster global.
TOC
Self-built VIPControl Plane HAVM Stuck increatingDCSMachine Stuck in DeletingVM Stuck in stoppingVM Stuck in Mid-TransitionController Cannot Reach DCS APIDCS Account LockoutFC Site Policy: errorCode 102212808Verify RecoverySelf-built VIP
Control Plane HA
The Control Plane HA check prints the status, the reason, and a message:
VM Stuck in creating
A DCSMachine stays in Provisioning with status.cdRomFile set, so the sign-in and the ISO upload succeeded, while the DCS VM stays in creating with no host and its deploy task does not progress. The DCS scheduler cannot place the VM.
- In the DCS portal, check the free space of the datastores that the node group uses and the load of the hosts. A nearly full datastore, or hosts that are all heavily overcommitted, block placement; see DCS Objects, Capacity, and Addresses.
- Ask the DCS platform team to free space or rebalance the hosts. The DCS scheduler, not the provider, picks the host of each VM.
- If the deploy task does not respond to cancellation or to the VM deletion that the provider issues, a DCS administrator cancels it in the portal. Where DCS serializes VM deployment, one stuck task stalls every later one, so clear the first one.
DCSMachine Stuck in Deleting
Use this section when a DCSMachine stays in Deleting and blocks a scale-down, a rolling upgrade, or the deletion of the cluster. Before it deletes a VM, the provider stops it and detaches its persistent disks, and DCS must complete each step.
Read the VMStopPending condition of the DCSMachine:
VM Stuck in stopping
The provider issued a safe stop, and the VM does not reach stopped. In the DCS portal, open the VM detail page:
VM Stuck in Mid-Transition
The provider waits until the VM is running or stopped:
Controller Cannot Reach DCS API
Read the provider log:
DCS Account Lockout
errorCode: 10100116 means that DCS has locked the account of the credential Secret.
- If the password changed, update the credential Secret as described in DCS Credential Secret: while it is wrong, the provider's sign-ins keep the account locked.
- Contact technical support to recover from the lockout.
Do not use the DCS portal admin account in the credential Secret: a lockout then locks everyone out. Use a dedicated account, as described in DCS Credential Secret.
FC Site Policy: errorCode 102212808
On a DCS site whose FusionCompute platform deletes only stopped VMs, the provider log can show DeleteVm rejected by FC site policy with errorCode=102212808. The provider stops the VM again and retries on its own.
Verify Recovery
Continue when the Machine is no longer listed and the VM is gone from the DCS portal; otherwise contact technical support.