Pre-upgrade health checks during a Cloud Scale upgrade
The kubectl cloudscale upgrade command automatically runs pre-upgrade health checks before it upgrades a Cloud Scale environment. These checks verify that the Kubernetes cluster and Cloud Scale environment are ready for upgrade and help identify issues before changes are made.
You do not need to run the pre-upgrade health checks manually before an upgrade. The upgrade command runs them for you and uses the results to determine whether it is safe to continue.
To run the checks independently of an upgrade, use the following command:
kubectl cloudscale health-check -c pre-upgrade -n <namespace>
When you start an upgrade, the pre-upgrade health checks run after you specify the environment namespace and before the upgrade collects image tags. The upgrade sequence is as follows:
The upgrade displays the prerequisites message and prompts you to continue.
The prerequisite checks are performed, such as Helm, kubectl, container manager, and cluster connectivity checks.
You are prompted to enter the path of the extracted Cloud Scale folder and the environment namespace.
The pre-upgrade health checks run automatically.
If all checks pass, the upgrade continues and prompts you for image tags.
If any check fails, the upgrade stops before changes are made.
Each pre-upgrade health check returns one of the following results:
Table: Upgrade behavior based on check results
Result | Description |
|---|---|
OKAY | The check passed. |
NOT_OKAY | The check failed. |
SKIPPED | The check does not apply to the platform or configuration and was not run. |
The upgrade handles these results as follows:
If all checks are OKAY or SKIPPED, the upgrade continues automatically.
If one or more checks are NOT_OKAY, the upgrade stops before changes are made. Resolve the reported issues, and then run the upgrade again.
When a check fails, the upgrade displays the number of failed checks and prompts you to resolve the issues before you retry the upgrade.
The upgrade runs the following pre-upgrade health checks:
Table: Pre-upgrade checks
Check | Description |
|---|---|
HelmChartsInstalledCheck | Confirms that all required Helm charts are installed for the current Cloud Scale version. |
SupportingComponentValidation | Validates that cert-manager and trust-manager are installed, version-compatible, and healthy. |
ClusterEnvironmentCheck | Confirms that the Cloud Scale environment is ready. |
PolicySchedulerStateCheck | Verifies that the policy scheduler is suspended before the upgrade. |
SecretsValidation | Confirms that all required secrets are present. |
ServiceImageTagCheck | Confirms that the environment configuration does not contain the serviceImageTag field. |
ServiceValidationCheck | Validates service configurations, including cluster IPs and load balancer external IPs. |
PVCHealthCheck | Confirms that all expected persistent volume claims are created and bound. |
PrimaryDataVolumeSpaceCheck | Confirms that the primary pod data volume has enough free space for the upgrade. |
NodePoolCapacityCheck | Confirms that node pools use the recommended maximum pods-per-node setting. Runs only on Azure AKS only. |
EFSThroughputModeConfirmation | Confirms that the Amazon EFS throughput mode is set to elastic. Runs only on Amazon EKS only. |
IAMPermissionConfirmation | Confirms the permissions of the required IAM roles. Runs only on Amazon EKS only. |
The upgrade stops because one or more pre-upgrade health checks failed.
Open the detailed health-check log at the path displayed at the end of the run. Resolve the reported issues, and then run the kubectl cloudscale upgrade command again.
You want to investigate a failed check.
Run the standalone kubectl cloudscale health-check command to investigate the failed check in more detail.
You can run the health check command in multiple ways. For more information on the available options:
See Command options.