I spent some time today helping to refresh the great work of Ariel Sanchez Mora (arielsanchezmora.com) and Christina Griffus on vSphere 8.0 upgrade guidance. I added some specific info for vSphere/ESXi 8.0 Update 2.
Check out the new repo at https://github.com/vSphere8upgrade/7u3-to-8u2
Original source material at https://github.com/vSphere8upgrade/7u3-to-8u1
You can see the U2 content is still a Work in Progress. Follow along as it gets better.
UPDATE: VMware has issued a patch for ESX 7.0. This will address the issue and the MS patch can be safely deployed to VMs on updated hosts.
I’ve kept the information below to help you assess impact to your environment and provide mitigation information if upgrading hosts is not immediately possible.
Install ESXi 7.0U3k on hosts – https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3k-release-notes.html
If you already have VMs that will not boot you can migrate them to a host running 7.0 U3k and then boot them up.
Conditions where this will impact you (all must be true):
Current workarounds (choose one):
NOTE: Uninstalling KB50252842 will NOT resolve the boot issue. You must disable secure boot, or upgrade the host to return the VM to an operational state.
MS KB (KB5022842): https://support.microsoft.com/en-gb/topic/february-14-2023-kb5022842-os-build-20348-1547-be155955-29f7-47c4-855c-34bd43895940
VMware KB (90947): https://kb.vmware.com/s/article/90947
VMware 7.0 U3k release notes: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3k-release-notes.html
William Lam – Get & Set SecureBoot functions: https://github.com/vmware/PowerCLI-Example-Scripts/blob/master/Scripts/SecureBoot.ps1
As shown in William Lam’s code secure boot is exposed on a PowerCLI VM object within the ExtensionData property
$testVM = Get-VM <VMNAME>
$testVM.extensiondata.config.bootoptions.EfiSecureBootEnabled
This will return true if SecureBoot is enabled. Use this information as a starting point if you would like to scan your environment with powershell. You can further limit your search using the Guest.OSFullName property of the VM for 2022, or if you have hardware version <18 the OS value for Win2022 might be “…2016 or later…” refine your search terms as necessary
2/21/23
Had a problem this evening where an ESXi host would choke on the 6.5 U2 update. vSphere reported “The host returns esxupdate error code:15. The package manager transaction is not successful. Check the Update Manager log files and esxupdate log files for more details.” Which lead me to https://kb.vmware.com/kb/2030665
esxupdate.log had the Python errors listed in that KB. Quick fix right? No, that would be too easy. After following the easy instructions in that KB where you just delete the /locker/packages/6.5.0 directory. I kicked off VUM and was presented with the same error. Tried the long fix. Recreated the 6.5.0 structure from a good host. Again same error.
Some digging lead me to https://blog.definebroken.com/2017/07/28/patching-vsphere-esxi-to-6-5u1-failing-with-error-15-cause-ran-out-of-vfat-space-due-to-vsantrace-and-vsanobserver-files/
He had a problem with vsantrace files. I checked and that wasn’t my problem. But I did cause me to watch the vmkernel.log to see if there were any clues there.
Sure enough I was getting a bunch of out of space errors:
2018-08-11T04:28:18.037Z cpu23:124089)WARNING: VFAT: 313: VFAT volume mpx.vmhba32:C0:T0:L0:8 (UUID 597f645d-2327f2da-1218-246e963e79d0) is full. (585696 sectors, 0 free sectors)
Did some du work and found that /store/locker/core was taking up ~80MB
[root@HOSTA1:/vmfs/volumes/597f645d-2327f2da-1218-246e963e79d0] du -h . 256.0K ./packages/var/db/locker/vibs 256.0K ./packages/var/db/locker/profiles 768.0K ./packages/var/db/locker 1.0M ./packages/var/db 1.3M ./packages/var 1.5M ./packages 80.2M ./var/core 80.4M ./var 256.0K ./epd-new 82.4M .
Sure enough there was a zdump file in var/core
A simple rm and then everything worked.
I didn’t catch this in my initial troubleshooting because the VFAT partition reported as 28% in use.
Since I was patching ESX itself the first step is to delete and recreate the vmtools and floppy images vCenter uses for client OS installation. These are stored in the /store/packages/<ESXversion> directory. This freed a bit of space, but all the space on the partition was consumed when it copied the updated versions and packages for installation.
Sometimes it is just a different simple fix.
vChris 