Glusterfs on Gentoo setup notes

For a long time I've been sharing the Portage tree over NFS to my virtual servers. However NFS is not exactly firewall friendly and feels a bit dated. So I've been looking for other alternatives. Today I stumbled across GlusterFS and took it for a spin.

It's quite easy to get a basic configuration going and there is already some Gentoo documentation about GlusterFS over at the Gentoo Wiki - GlusterFS. However they're installing GlusterFS from source and I only want to use ebuilds, so here is a short summary of how I did the setup with one server.

First ensure that you have FUSE enabled in the kernel (CONFIG_FUSE_FS=y)

Since glusterfs is not table yet, we'll add appropriate package keywords:

echo "sys-cluster/glusterfs ~amd64" >>\
/etc/portage/package.keywords/glusterfs

Then install with:

emerge -va glusterfs

We'll do most of the setup on the server so clients will download the configuration from the server. In this setup the server will have IP 192.168.2.1 and allow all connections from all clients in the 192.168.2.0/24 network.

First setup the server by editing /etc/glusterfs/glusterfsd.vol to contain the following:

volume portage
type storage/posix
option directory /usr/portage
end-volume

volume server
type protocol/server
option transport-type tcp
option transport.socket.listen-port 6996
option auth.addr.portage.allow 192.168.2.*
option client-volume-filename /etc/glusterfs/portage.vol
option volume-filename.portage /etc/glusterfs/portage.vol
subvolumes portage
end-volume

Note: that the above might contain wrong line breaks, almost all lines should start with option, and not with a file path.

Then create /etc/glusterfs/portage.vol:

volume portage
type protocol/client
option transport-type tcp
option remote-host 192.168.2.1
option remote-subvolume portage
end-volume

Then we just start the server:

/etc/init.d/glusterfsd start

Now we only need to configure the client:

echo "sys-cluster/glusterfs ~x86" >>\ /etc/portage/package.keywords/glusterfs

Note: Substitute appropriate keywords.

Then install glusterfs on the client:

emerge -va glusterfs

For automounting add the following to /etc/fstab:

192.168.2.1 /usr/portage glusterfs defaults 0 0

You could add the ro to the options, in addition to defaults, if only the server should be able to write to the Portage tree.

Then mount it the first time with:

mount /usr/portage

Also I've tried using the volfile-id=portage option, but it doesn't seem to work from fstab, though it works fine with glusterfs, glusterfs --volfile-server 192.168.2.1 --volfile-id portage /usr/portage/.

Now you should have your Portage tree shared with GlusterFS, if you have any problems consult the GlusterFS Troubleshooting