Unfortunately it is not as easy to mount a GlusterFS volume in a LXC container as it is with another device, for example an additional Logical Volume.
I first tried it with the LXC's fstab file:
localhost:/vol1 mnt glusterfs defaults,_netdev 0 2
This should (in theory) mount the GlusterFS volume "vol1" from localhost into the LXC container with mountpoint /mnt. Yes, the missing slash is correct given the relative path to the LXC container.
But unfortunately, this didn't work as a start of the container in debug mode showed:
lxc-start -n lxcname -o /var/lib/lxc/lxcname/stdout.log -l debug
lxc-start 1409577107.058 ERROR lxc_conf - No such device - failed to mount 'localhost:/vol1' on '/usr/lib/x86_64-linux-gnu/lxc/mnt'
lxc-start 1409577107.058 ERROR lxc_conf - failed to setup the mounts for 'lxcname'
lxc-start 1409577107.058 ERROR lxc_start - failed to setup the container
lxc-start 1409577107.058 ERROR lxc_sync - invalid sequence number 1. expected 2
As a second attempt, I tried it within the LXC container (as with a normal Linux host) in /etc/fstab:
# UNCONFIGURED FSTAB FOR BASE SYSTEM
10.10.11.10:/vol1 /mnt glusterfs defaults,_netdev 0 0
Where 10.10.11.10 is the IP address of the physical host of this LXC container.
Before rebooting the container, I tried to mount the gluster volume manually:
mount.glusterfs 10.10.11.10:/vol1 /mnt
Mount failed. Please check the log file for more details.
Ah crap! What now? I checked the glusterfs mount log:
I [glusterfsd.c:1910:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.4.2 (/usr/sbin/glusterfs --volfile-id=/vol1 --volfile-server=10.10.11.10 /mnt)
E [mount.c:267:gf_fuse_mount] 0-glusterfs-fuse: cannot open /dev/fuse (No such file or directory)
E [xlator.c:390:xlator_init] 0-fuse: Initialization of volume 'fuse' failed, review your volfile again
Indeed, the special character device /dev/fuse is missing in the container while on the physical host it exists.
At first I thought this is a device permission issue which needs to be solved in the container's config file but the relevant config for /dev/fuse is already there by default:
cat /usr/share/lxc/config/ubuntu.common.conf | grep -A 1 "## fuse"
lxc.cgroup.devices.allow = c 10:229 rwm
Then I stumbled across this Github issue: https://github.com/lxc/lxc/issues/80 where Stéphane Graber, one of the LXC's main developers, answered this:
Some modules will also require the creation of device nodes in the container which you'll need to do by hand or through init scripts.
So to solve this, I created /dev/fuse manually within the container:
mknod /dev/fuse c 10 229
And then tried the manual mount again:
mount.glusterfs 10.10.11.10:/vol1 /mnt
No error this time. Verification with df:
Filesystem Size Used Avail Use% Mounted on
/dev/lxc/lxcname 20G 830M 18G 5% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 13G 60K 13G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 63G 0 63G 0% /run/shm
none 100M 0 100M 0% /run/user
10.10.11.10:/vol1 99G 60M 99G 1% /mnt
To simplify such things, the /dev/fuse can of course already be created during "lxc-create" by modifying the relevant lxc template. Saves you a headache later.
Dino from wrote on Jan 12th, 2017:
I believe I ran into this issue as well with /dev/fuse
ck from Wil, Switzerland wrote on Dec 14th, 2015:
Hi Mark. In my case it worked. Once created within the container, the device node staid. Verify if the permissions (lxc.cgroup.devices.allow) are correct so the container is allowed to access the devices. I haven't tried glusterfs mounts yet with newer LXC releases.
Mark from wrote on Dec 13th, 2015:
But how do we actually get the 'mknod /dev/fuse c 10 229' to be persistent across reboots and also the glusterfs mounts? I haven't had any luck with either.
Ovidiu from San Francisco Bay Area wrote on May 22nd, 2015:
Sadly this doesn't seem to be working with LXD 0.9, where containers run as unprivileged users.
Adam from Boone, NC wrote on Jan 2nd, 2015:
Thanks for posting this quick guide. I just ran into this issue of mounting a GlusterFS inside a LXC container and would have spent a lot of time tracking down the /dev/fuse issue.