Grub2 and UUID
24 July 2021
UUID kills restore from image
One of the benefits of virtual machines is the ease with with whcih you can take a snapshot and restore and restore to a new virtual machine. It mackes a backup a no brainer on the cloud. I’ve done it more times than I care to mention and it is magic… when it works.
A while back the default for various flavours of linux was to use UUIDs for fstab because you knew the partition you were mounting was the exact same one you want - except that destroyed one of the beauties of being able to mount things where you want, you could just plug in a different thing and it would be mounted there.
This is awesome for virtual environments and for bare metal backup, in fact it’s essential. You backup in case the disk dies, so the UUID IS going to change when you restore it somewhere else.
People in the linux world quickly changed their fstabs
back to devices or labels… now grub2 seems to really like UUIDs and I think new people come along ang say "hey UUID will solve this (rare) case, lets use that" and bugger everything up again.
I upgraded to Fedora 34 on Linode and as usual enabled backups (excellent service that) then the next time I had a production update to do I restored from backup to test the upgrade first and wham, the new Linode doesn’t boot, fuck!
Logging in via Lish shows it has dropped to the emergency shell because it couldn’t find a bloody UUID - OK go and re-learn how to mount the bits and reconfigure grub2.cfg so it has the right UUIDs now so it can boot, all the time muttering just use the bloody device!
Just in case you have the same issue and need to recover your image you need to do this after logging in (via Lish on linode, the serial terminal):
mkdir /mnt
mount /dev/sda /mnt
mount -t proc none /mnt/proc
mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys
chroot /mnt /bin/bash
dracut --force --no-hostonly
grub2-mkconfig -o /etc/grub2.cfg
So how do you fix it so that doesn’t happen again??
I had to go an look at the grub2-mkconfig script itself to work this out. "Simply" edit the /etc/default/grub file and add this line:
GRUB_DISABLE_UUID=true
then from the command line (as root):
grub2-mkconfig -o /etc/grub2.cfg
and that should be it. Reboot to see if it works and take a snapshot and do a test restore!