MooseFS taking shape

I am continuing experimenting with MooseFS. However, the final configuration looks somewhat different from what I had originally envisioned. For one thing, the idea of placing the master and metadata servers in VirtualBox VM's didn't quite work out. I guess that created just too many levels of execution and as a result that lead to the overall load growing too much and the performance suffering as soon as any serious load was applied.

So I switched to simply running all the processes (master, meta, chunkserver) on the same hardware and got rid of all the VM's. That worked fine. I defined a separate network - currently fully confined to the same host - in order to host the MooseFS installation. And MooseFS clients have started to run just fine. I got a performance of up to 80 MB/s for reading data from the MooseFS over a 1 Gbit/s network.

However, one problem remained. Running UNFSD on the same machine I got very poor performance.  As few as 5 clients could drive it down to just 30 KB/s! And that on an 8-core 48 GB RAM machine - while a MooseFS client would read at 3 orders of magnitude as much!

Surprisingly, the fix was simple: if I ran UNFSD on a separate physical machine the performance went back into the tens of MB/s range. So that was what I settled for. That NFS server machine is currently just a CentOS Linus MooseFS client sitting on the "general" network - different from the one hosting MooseFS - and sporting a mere 2 cores and 2 GB of RAM. So I guess for now I have a working solution.

SSHFS and AutoFS

Mounting an SSH-accessible remote directory automatically is a nifty capability. Here's a nice description of how one can do that - specifically under Ubuntu but it will work the same just fine under most other Linux distributions:

Automatically mounting a remote directory in Ubuntu using autofs + sshfs

One thing not mentioned there - and something I keep forgetting about between the instances I need to recall it - is that the passwordless SSH login will fail unless the user directory on the SSH server is writable by the owner only! So, using the same terms as in the example above one should do the following:

1) Log into example.com as remoteusername

2) Execute the following command:
chmod g-w,o-w ~