ESXi upgrade from 5.5 to 6.7

ESXi 5.5 recently reached end-of-support (see End of General Support for vSphere 5.5), but my sales rep informed me that I was eligible for a free upgrade. Great! I set about doing just that.

First of all, I should note that you can’t upgrade directly from 5.5 to 6.7, so I upgraded to 6.5 first. I ran into several missteps along the way, which I have documented here:

Short version: use Rufus on Windows to build your bootable installation USB flash drive! Even if you are not a Windows user, find a Windows machine or download a Windows VM and use Rufus to build your boot media. Seriously, use Rufus and skip to the end of this article for a couple extra tips.

I found a succinct article, Create an ESXi 6.5 installation USB under two minutes, but the steps listed there expect you to use a Windows application to build the USB media. I was using MacOS, so I went looking for the MacOS alternative and found How to create a bootable VMware ESXi USB drive on Macs.

After following the steps there, I plugged the USB flash drive into my server, used the BIOS boot menu to boot from the USB drive, and got the following error:

Booting from Hard drive C:

Non-system disk
Press any key to reboot

I had run into a non-fatal error when using fdisk, maybe that had something to do with it? The error message was:

fdisk: could not open MBR file /usr/standalone/i386/boot0: No such file or directory

I also read that copying the files from the ISO to the USB using cp was not recommended, so I tried using dd as well:

$ sudo dd if=~/Downloads/VMware-VMvisor-Installer-6.7.0.update01-10302608.x86_64.iso of=/dev/rdisk2 bs=1m

After that, I got a different error message when trying to boot from the USB drive:

No boot device available or operating system detected.
Current boot mode is set to BIOS.
Please ensure a compatible bootable media is available.

Next I tried VMWare’s own documentation: Format a USB Flash Drive to Boot the ESXi Installation or Upgrade

This method required a Linux host with syslinux installed. I had a CentOS 7.5 host available, and installed syslinux 4.05 via yum:

$ sudo yum install syslinux

When I booted from the USB drive, I got the following error message:

menu.c32: not a COM32R image
boot:

I searched for that error and found the post Cannot boot from USB disk with “not a COM32R image” error on AskUbuntu, so I tried that:

[tab] install hddimage
boot: install

Same error.

The error mentioned menu.c32 specifically, so I copied /usr/share/syslinux/menu.c32 and /usr/share/syslinux/mboot.c32 to the USB key, overwriting the version from the VMWare ISO image. (There was also a safeboot.c32 on the USB key, but no corresponding file in /usr/share/syslinux.)

This time booting from the USB drive got slightly farther:

ESXi-6.7.0-201819929910 standard installer
Boot from local disk

Press (Tab) to edit options
Automatic boot in 1 second...

Loading -c...failed!
No files found!
boot:

I also tried keeping the mboot.c32 file from the VMWare ISO and replacing just the menu.c32 file, but that did not help.

I found some suggestions that VMWare is very picky about the version of syslinux that is used, so I also compiled a newer version of syslinux, 6.02, which is specifically mentioned in the VMWare documentation I was following (“For example, if you downloaded Syslinux 6.02, run the following commands”). That didn’t help either.

I raised a cry for help on the VMWare community forums, and someone chimed in right away and advised me to give up on Linux just use Rufus on Windows. I fired up a Windows 2012r2 VM — if you don’t have one, you can download an ISO from the Microsoft Evaluation Center and install one — and downloaded and installed Rufus. (Rufus looks a little sketchy, but is vouched for by many sane and security-minded people.)

In just a couple minutes, I had working boot media installed on my USB flash drive!

I booted my VMWare host and got a new error during the upgrade process:

Error(s)/Warning(s) Found During System Scan

The system encountered the following error(s).

Error(s)

<CONFLICTING_VIBS ERROR: Vibs on the host are conflicting with vibs in metadata. Remove the conflicting vibs or use Image Builder to create a custom ISO providing newer versions of the conflicting vibs.  ['LSI_bootbank_scsi-mpt3sas_04.00.00l00.lvmw-1OEM.500.0.0.472560', 'LSI_bootbank_scsi-mpt3sas_04.00.00l00.lvmw-1OEM.500.0.0.472560']>

Use the arrow keys to scroll

(F9) Back (F11) Reboot

A Reddit post, Trying to remove VIB, pointed out that the driver referenced by the VIB (vSphere Installation Bundle) is for hardware that isn’t compatible with my Dell hardware, so I would be safe to remove it. I removed it using the ESX command-line interface:

# esxcli software vib remove -n=scsi-mpt3sas

After that I was able to boot the system from the USB drive and upgrade the ESXi installation to 6.5.

A few additional notes:

• On my hardware, the ESXi OS was installed on an embedded SD card. The menu asked me to select upon which drive I wanted to install or upgrade ESXi, so I chose the smallest partition that corresponded to the embedded SD card. Your experience may differ.
• I was able to upgrade the 6.5 installation to 6.7 entirely from the web-based VMWare Update Manager.
• I was able to upgrade the license on My VMWare (see How to upgrade license keys in My VMware).