Just recently Nutanix released Nutanix Operating System (NOS) version 4.1.4, Magnus Andersson (part of the Performance and Solution engineering team just like me) wrote a small blogpost on the enhancements in 4.1.4 here. As my cluster was still on NOS 4.1.3 I finally had a moment to sit down and update my cluster.
I wanted to follow up on our own documentation so I gathered the information to do an upgrade as if in production and ran NCC which informed me that there was an issue with the health check service on one of my CVMs. Rather than going down the troubleshooting route I decided to reboot the CVM to see if the problem would still exist but after a reboot my CVM would boot any more, it said: ‘Operating System not found’ but because of the distributed nature of our architecture my VMs would operate without any issues while the performance stayed on par so I had some time to figure out the problem.
After talking to one of our unsung hero’s of support I was pointed to a KB article that could create a recovery ISO to see if there was hardware failing, during that process I noticed that this specific CVM did not have the service VM ISO mounted anymore and it all fell into place.
I recently ran a PowerCLI script to unmount the CD-ROM drives and apparently I didn’t exclude the CVMs causing the service VM ISO to be unmounted. Mounting the ISO file from the local storage did the trick and I was able to boot the CVM, ran the PreCheck and do a cluster upgrade within 20 minutes.
Take aways for me during this upgrade:
- Always run NCC before a cluster update
- Our support is absolutely awesome
- Exclude CVMs from PowerCLI scripts
- Modifications to our NCC check are already filed so we check if the ISO is actually mounted in feature releases of NCC.
- I’m an idiot 🙂
Latest posts by Kees Baggerman (see all)
- Recovering a Protection Domain snapshot to a VM - September 13, 2019
- Checking power settings on VMs using powershell - September 11, 2019
- Updated: VM Reporting Script for Nutanix with Powershell - July 3, 2019
- Updated (again!): VM Reporting Script for Nutanix AHV/vSphere with Powershell - June 17, 2019
- Updated: VM Reporting Script for Nutanix AHV with Powershell - April 8, 2019