Debian not booting: ALERT /dev/disk/by-uuid does not exist

Written by Claudio Kuenzler - 0 comments

Published on February 2nd 2014 - Listed in Linux Proxmox Virtualization

---

On a test server running a Proxmox PVE installation (based on Debian Wheezy) I launched a apt-get dist-upgrade which installed a new kernel and launched grub-update.
In the output of dist-upgrade I saw some errors but didn't look closer at it and immediately rebooted the server. Mistake.

Once the grub menu appeared and the first entry was auto-selected, grub couldn't find the root disk and alerted with the following error message on the console:

ALERT! /dev/disk/by-uuid/5b9f271d-3e7a-49cf-b892-2c6c8e194684 does not exist.
Dropping to a shell!

Here the screenshot:

Argh. I have already seen such cases (see my article grub2 boot issues after hdd replacement) where grub2 didn't boot anymore.

I launched the Knoppix ISO from the virtual DVD drive (with boot option knoppix64) to see what's going on.

Mounted /dev/sda1 (the /boot partition of the Proxmox installation) as /mnt and verified some grub stuff:

mount /dev/sda1 /mnt

cat /mnt/grub/device.map
(hd0) /dev/sda

cat /mnt/grub/grub.cfg
[...]
menuentry 'Proxmox Virtual Environment GNU/Linux, with Linux 2.6.32-23-pve' --class proxmox --class gnu-linux --class gnu --class os {
[...]
set root='(hd0,msdos1)'
search --no-floppy --fs-uuid --set=root 29efb07c-be6e-44dc-b637-70efda447bd7
linux /vmlinuz-2.6.32-23-pve root=UUID=5b9f271d-3e7a-49cf-b892-2c6c8e194684 ro quiet
[...]
}

Let's check out the UUID of the root partition which is a Logical Volume. First activate the volume group pve:

vgchange -ay pve
  4 logical volume(s) in volume group "pve" now active

Where does the root LV point to?

ls -l /dev/mapper | grep pve-root
lrwxrwxrwx 1 root root 7 Feb  2 09:06 pve-root -> ../dm-1

Check out UUID of /dev/dm-1:

ls -l /dev/disk/by-uuid | grep dm-1
lrwxrwxrwx 1 root root 10 Feb  2 09:06 5b9f271d-3e7a-49cf-b892-2c6c8e194684

Hmm... that's the correct UUID also used in the grub config. So we need to find other errors by launching update-grub within the "real" Proxmox OS (Debian Wheezy). To do that, the root and runtime filesystems need to be mounted:

umount /mnt

mount /dev/mapper/pve-root /mnt
mount /dev/sda1 /mnt/boot
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys

Change root (get into the "real" Linux environment):

chroot /mnt

Launch update-grub command:

update-grub
Generating grub.cfg ..
/usr/sbin/grub-probe: error: Couldn't find PV pv1. Check your device.map.
Found linux image: /boot/vmlinuz-2.6.32-23-pve
Found initrd image: /boot/initrd.img-2.6.32-23-pve
/usr/sbin/grub-probe: error: Couldn't find PV pv1. Check your device.map.
[...]
done

pvdisplay didn't show anything. Activated VG pve within chroot environment again:

vgchange -ay pve
  4 logical volume(s) in volume group "pve" now active

pvdisplay
--- Physical volume ---
PV Name        /dev/sda2
VG Name        pve
[...]
PV UUID        1YMsls-ycW4-aqWL-MEsC-EOFM-VVyI-4UrgFY
[...]

Launch update-grub again:

update-grub
Generating grub.cfg ..
Found linux image: /boot/vmlinuz-2.6.32-23-pve
Found initrd image: /boot/initrd.img-2.6.32-23-pve
[...]
done

So this time it worked without an error message. Let's check the contents of the device.map and grub.cfg:

cat /boot/grub/device.map
(fd0) /dev/fd0
(hd0) /dev/sda
(hd1) /dev/sdb

cat /boot/grub/grub.cfg
[...]
menuentry 'Proxmox Virtual Environment GNU/Linux, with Linux 2.6.32-23-pve' --class proxmox --class gnu-linux --class gnu --class os {
[...]
set root='(hd0,msdos1)'
search --no-floppy --fs-uuid --set=root 29efb07c-be6e-44dc-b637-70efda447bd7
linux /vmlinuz-2.6.32-23-pve root=/dev/mapper/pve-root ro quiet
[...]
}

Well well well... device.map has changed a lot and so did the root path of the linux entry in grub.cfg.

Let's reboot and see... Hurray! Successfully booted:

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.