VMware Cloud Foundation (VCF) 5.1 delivers a significant leap forward in managing your private cloud environment. This upgrade offers enhanced capabilities across storage, networking, computing, and lifecycle management, empowering you to scale effectively and bolster resiliency.

VMware cloud foundation

This guide equips you with the knowledge to navigate the upgrade process for VCF 5.1, ensuring a smooth transition and maximizing the benefits of the new features.

Before You Begin: Planning and Preparation

A successful upgrade hinges on meticulous planning and preparation. Here are the crucial steps to take before initiating the upgrade process:

Understanding the Upgrade Process

VCF 5.1 offers two upgrade approaches

Key Considerations for Specific Components

Post-Upgrade Tasks and Validation

Once the upgrade is complete, it's essential to perform post-upgrade tasks and validations to ensure everything functions as expected. Here are some key steps:

Additional Resources and Best Practices

For a deeper dive into the upgrade process, refer to the following resources:

SDDC Manager

Upgrades will not be available until the Multi-Site Management bundles are removed. After upgrading SDDC Manager, I anticipated that NSX would be the next upgrade, but the loading for bundles was never completed.

SDDC Manager

If you are using an installation that has undergone multiple upgrades, such as from version 4.2 to 4.5.2, you may encounter the following error in the update view, which refers to a bundle software type called "MULTI_SITE_SERVICE":

{
    "type": "java.lang.IllegalArgumentException",
    "message": "No enum constant com.vmware.evo.sddc.lcm.model.bundle.BundleSoftwareType.MULTI_SITE_SERVICE"
}

Errors

Initially, this message was confusing to me. I couldn't find any relevant KBs or blog posts online that explained the error.

To understand what "MULTI_SITE_SERVICE" was, I had to dig deeper, which included performing a pg_dump on the lcm database to examine the upgrade_history table.

In earlier vCF deployments, there were bundles related to "multi-site management," for example.

{
    "id" : "f0c04887-dbf3-498a-b55a-12a28e668254",
    "version" : "1.5.14-vcf4210RELEASE-533",
    "description" : "VMware vCloud Foundation Multi-Site Management",
    "name" : "MULTI_SITE_SERVICE"
}

I viewed this as a chance to tidy up my SDDC Manager.

KB 94760 describes a bundle management tool that, when used with PowerVCF, offers a simple method for removing outdated bundles.

Bundle management

Administrator

After removing many older bundles from the SDDC Manager, totaling 400GB, NSX became a viable upgrade path.

Lesson learned: It's probably a good practice to delete old bundles before every update.

Compatibility data

Compatibility data

In VMware Cloud Foundation 4.5.x, the Compatibility Matrix Upload API call is not available. Because of this, I waited to upload it until SDDC Manager was on version 5.1.

It's worth noting that you can disable the compatibility matrix check by following KB 90074, but doing so may prevent you from upgrading NSX further.

vSAN HCL Update

I could only upload the latest vSAN HCL update after upgrading SDDC Manager to version 5.1 using the most recent lcm-bundle tools.

On my Windows jump-host, I encountered the following error when attempting the upload, so I chose to do it from SDDC Manager:

"Exception thrown when uploading vSAN HCL data: URI path begins with multiple slashes."

API Version Display

After upgrading to SDDC Manager 5.1.0, the Developer Center API will still display the 4.5.2 version. This appears to be a visual bug, as the 5.1.0 API is operational, although it cannot be accessed from the web interface.

VMware Cloud Foundation API Reference Guide

Clearing Tasks that Never Complete

Refer to KB 89911 for instructions on how to resolve tasks that appear to never finish after upgrading to SDDC Manager 5.x.

tasks

NSX

ESXi host cannot enter maintenance mode

Generic error: Virtual machine ‘app01’ on host ‘esxi02.lab.local’ would violate a virtual machine - host affinity rule. VM cannot migrate.

During the upgrade, I encountered a virtual machine with an affinity rule that couldn't be broken. To proceed, I simply powered off that virtual machine.

Install of offline bundle failed

Unable to get FS Attrs for /vmfs/volumes/ca3e7430-4611-41ac-8fa2-127349107360

One ESXi host displayed a disconnected datastore mounted from SDDC Manager that had not been removed because it was selected for use with vSphere HA.

lcm-bundle-repo

After removing the datastore, I had to reboot the host for the volume to disappear from /vmfs/volumes on the host.

To proceed, retry the update after taking the host out of both NSX Maintenance Mode and ESXi Maintenance Mode.

New authentication provider

The new version of VMware Validated Solutions for vCF/NSX now recommends using Active Directory for authentication instead of Identity Manager. Follow the guide below to migrate from Workspace ONE Access to LDAP Integration:

  1. Prepare Active Directory

    • Ensure that you have a dedicated service account in Active Directory for LDAP integration.
    • Make sure the service account has the necessary permissions to read user information.

  2. Configure LDAP Integration in SDDC Manager

    • Log in to SDDC Manager.
    • Navigate to the Administration tab.
    • Select the Authentication menu option.
    • Click on the "Configure" button for LDAP Integration.
    • Enter the LDAP server details (server address, port, and SSL settings).
    • Configure the Base DN and Bind DN settings based on your Active Directory configuration.
    • Enter the username and password for the service account.
    • Test the LDAP integration to ensure it's working correctly.

  3. Migrate Users from Identity Manager to Active Directory

    • Log in to Workspace ONE Access.
    • Navigate to the Users & Groups tab.
    • Export the list of users to a CSV file.
    • Import the CSV file into Active Directory to create user accounts.
    • Assign the appropriate permissions to the user accounts in Active Directory.

  4. Update VMware Validated Solutions Configuration

    • Update any references to Identity Manager in your VMware Validated Solutions configuration to use LDAP integration with Active Directory.
    • Ensure that all components of the solution are configured to use Active Directory for authentication.

  5. Testing and Validation

    • Test the new LDAP integration to ensure that users can authenticate correctly.
    • Validate that all components of the VMware Validated Solutions are functioning as expected with the new authentication method.

  6. Rollback Plan

    • Prepare a rollback plan in case any issues arise during or after the migration.
    • Ensure that you have a backup of the previous configuration and user information.

  7. Documentation and Training

    • Update documentation and provide training for users and administrators on the new authentication method.

  8. Post-Migration Cleanup

    • Once the migration is successful, remove any references to Identity Manager and decommission the Identity Manager infrastructure.

  9. Monitor and Maintain

    • Monitor the new LDAP integration for any issues and perform regular maintenance tasks as needed.

  10. Review and Audit

    • Regularly review the LDAP integration configuration and audit user access to ensure security and compliance.

Following these steps will help you migrate from Workspace ONE Access to LDAP integration with Active Directory successfully.

Alarms for Expired or Expiring Certificates in NSX

Following the upgrade, new alarms regarding certificate expiration were observed. A solution has been released to address this issue:

NSX Node Application Crash Alarm

If you encounter alarms indicating application crashes, refer to the instructions in KB 92493:

Potential LDAPS Disruption After NSX Upgrade

Upgrading NSX might cause LDAPS to cease functioning, which could affect the operation of the Identity Firewall and authentication.

Related KB LDAPS may stop working after upgrading NSX to version 4.1.0 (92869).

vCenter Server

Issues with ELM

Uppercase DNS Records

The source appliance FQDN vcsa01.lab.local must be the same as the source appliance primary network identifier vcsa01.lab.local

Related blog: Changing your vCenter Server’s FQDN

ESXi

Deployment Issue with vCenter LCM Plugin

The vCenter LCM plugin was not deployed correctly, possibly due to a race condition where the URL was not available in time.

DOWNLOAD_FAILED: Error downloading plugin package com.vmware.vlcm.client:8.0.2.22617221 from https://vcsa01.lab.local:9087/vci/downloads/vlcm-ui/plugin.zip. Reason: URL is unreachable. Make sure that the URL is reachable. com.vmware.vise.plugin.download.PluginDownloadException: org.apache.http.client.HttpResponseException: status code: 503, reason phrase: Service Unavailable

After a restart of the vCenter Server, the upgrade progressed further.

Error: vDS Port Not Found

Resolution