# - - # # - hmddfs - # https://romanrm.net/mhddfs https://github.com/crass/mhddfs https://www.tecmint.com/combine-partitions-into-one-in-linux-using-mhddfs/ http://zornsoftware.codenature.info/blog/why-i-ditched-raid-and-greyhole-for-mhddfs.html https://www.maketecheasier.com/mhddfs-combine-multiple-partitions-linux/ https://askubuntu.com/questions/586433/can-mhddfs-mounting-options-be-set-via-etc-fstab-to-respect-file-permissions http://man7.org/linux/man-pages/man8/mount.fuse.8.html http://techqa.info/ubuntu/question/586433/can-mhddfs-mounting-options-be-set-via-/etc/fstab-to-respect-file-permissions https://github.com/vdudouyt/mhddfs-nosegfault # - - # # - Very important for user permissions respect in the new disk - # default_permissions mhddfs#/pool/storage00,/pool/storage01,/pool/storage02 /storage fuse default_permissions,allow_other,mlimit=1G 0 0 Quick Reference apt-get update apt-get install mhddfs mkdir /mnt/virtual sudo gedit /etc/fstab mhddfs#/mnt/disk1,/mnt/disk2,/mnt/disk3,/mnt/disk4,/mnt/disk5 /mnt/virtual fuse allow_other,mlimit=20G 0 0 Alternatively MHDDFS could be mounted at user level by editing rc.local configuration file. sudo gedit /etc/rc.local mhddfs /mnt/disk1,/mnt/disk2,/mnt/disk3,/mnt/disk4,/mnt/disk5 /mnt/virtual -o allow_other -o mlimit=20G reboot Filesystem Size Used Avail Use% Mounted on /dev/sda1 511M 132K 511M 1% /boot/efi /dev/sda2 451G 92G 336G 22% / /dev/sdb1 1.9T 161G 1.7T 9% /media/avi/BD9B-5FCE /dev/sdc1 555M 555M 0 100% /media/avi/Debian 8.1.0 M-A 1 mhddfs /boot/efi, /, /media/avi/BD9B-5FCE/, /media/avi/Debian\ 8.1.0\ M-A\ 1/ /mnt/virtual_hdd -o allow_other # - - # # - CHAPTER 1 - From the original website - # # - df -hl - # # - Show you all disks in your computer - # Let's say the three hard drives you have are mounted at /mnt/hdd1 /mnt/hdd2 and /mnt/hdd3. Then, you might have something akin to the following: $ df -h Filesystem Size Used Avail Use% Mounted on ... /dev/sda1 80G 50G 30G 63% /mnt/hdd1 /dev/sdb1 40G 35G 5G 88% /mnt/hdd2 /dev/sdc1 60G 10G 50G 17% /mnt/hdd3 After you have installed the mhddfs package using your favourite package manager, you can create a new mount point, let's call it /mnt/virtual, which will join all these drives together for you. The beauty of FUSE means you don't really have to be root for this (can be just a member of the fuse group), but for the sake of examples' simplicity, let's suppose we are logged in as root here. # - - # # - All this in terminal - # sudo apt-get install mhddfs; mkdir /mnt/virtual mhddfs /mnt/hdd1,/mnt/hdd2,/mnt/hdd3 /mnt/virtual -o allow_other option: allow_other (1) mhddfs: directory '/mnt/hdd1' added to list mhddfs: directory '/mnt/hdd2' added to list mhddfs: directory '/mnt/hdd3' added to list mhddfs: move size limit 4294967296 bytes mhddfs: mount point '/mnt/virtual' # - - # # - Permanent mounting in the /etc/fstab file - # mhddfs#/mnt/hdd1,/mnt/hdd2,/mnt/hdd3 /mnt/virtual fuse defaults,allow_other 0 0 The «-o allow_other» option here means that the resulting filesystem should be visible to all users, not just to the one who created it. The result will look like this: $ df -h $ df -hl Filesystem Size Used Avail Use% Mounted on ... /dev/sda1 80G 50G 30G 63% /mnt/hdd1 /dev/sdb1 40G 35G 5G 88% /mnt/hdd2 /dev/sdc1 60G 10G 50G 17% /mnt/hdd3 mhddfs 180G 95G 85G 53% /mnt/virtual Filesystem Size Used Avail Use% Mounted on /dev/sda1 511M 132K 511M 1% /boot/efi /dev/sda2 451G 92G 336G 22% / /dev/sdb1 1.9T 161G 1.7T 9% /media/avi/BD9B-5FCE /dev/sdc1 555M 555M 0 100% /media/avi/Debian 8.1.0 M-A 1 mhddfs /boot/efi, /, /media/avi/BD9B-5FCE/, /media/avi/Debian\ 8.1.0\ M-A\ 1/ /mnt/virtual_hdd -o allow_other As you can see, the new filesystem has been created. It joined the total size of all drives together (180G), added together the space used by all files there (95G) and summed up the free space (85G). If you look at files in /mnt/virtual, you'll notice that it has files from all three drives, with all three directory structures “overlayed” onto each other. But what if you try to add new files somewhere inside that /mnt/virtual? Well, that is quite tricky issue, and I must say the author of mhddfs solved it very well. When you create a new file in the virtual filesystem, mhddfs will look at the free space, which remains on each of the drives. If the first drive has enough free space, the file will be created on that first drive. Otherwise, if that drive is low on space (has less than specified by “mlimit” option of mhddfs, which defaults to 4 GB), the second drive will be used instead. If that drive is low on space too, the third drive will be used. If each drive individually has less than mlimit free space, the drive with the most free space will be chosen for new files. It's even more than that; if a certain drive runs out of free space in the middle of a write (suppose, you tried to create a very large file on it), the write process will not fail; mhddfs will simply transfer the already written data to another drive (which has more space available) and continue the write there. All this completely transparently for to the application which writes the file (it will not even know that anything happened). Now you can simply work with files in /mnt/virtual, not caring about what is being read from which disk, etc. Also, the convenience of having large “contiguous” free space means you can simply drop any new files into that folder and (as long as there's space on at least one member of the virtual FS) not care about which file gets stored where. If you decide to make that mount point creating automatically for you on each boot, you can add the following line to /etc/fstab: # - - # # - Permanent mounting in the /etc/fstab file - # mhddfs#/mnt/hdd1,/mnt/hdd2,/mnt/hdd3 /mnt/virtual fuse defaults,allow_other 0 0 For more details, see man mhddfs. The last, but not the least important thing to mention, is the fact that it's very simple to stop using mhddfs, if you later decide to do so - and not lose any file data or directory structure. Let's say, at some point in time, you purchase a new 500 GB hard disk, and want to sell the smaller disks on Ebay. You can just plug in the new drive, copy everything from /mnt/virtual onto it, and then remove mhddfs mountpoint and disconnect old drives. All your folders, which were previously merged in a “virtual” way by mhddfs, will now be merged in reality, on the new disk. And thanks to the fact that files themselves are not split into bits which are stored on different drives, even in the unlikely event when mhddfs suddenly no longer works for you (or disappears from existence), you can still copy all your data from all three drives into one single folder, and have the same structure you previously had in that /mnt/virtual mountpoint. # - - # # - CHAPTER 2 - From another sample - # You can use mhddfs instead lvm as if one disk is failed then only data on that disk will be lost. It runs on user space using fuse modules but I'm using it for very heavy load infrastructure. # - df -hl - # # - Show you all disks in your computer - # [root@storagenode1 ~]# df -hl Filesystem Size Used Avail Use% Mounted on /dev/sdl2 259G 2.1G 244G 1% / tmpfs 48G 0 48G 0% /dev/shm /dev/sdl1 485M 30M 430M 7% /boot /dev/sda1 24T 23T 1.4T 95% /mnt/disk01 /dev/sdg1 24T 22T 2.6T 90% /mnt/disk02 /dev/sdf1 24T 22T 2.6T 90% /mnt/disk03 /dev/sdb1 24T 20T 4.5T 82% /mnt/disk04 /dev/sde1 39T 30T 8.3T 79% /mnt/disk07 /dev/sdh1 28T 6.6T 21T 24% /mnt/disk08 /dev/sdj1 39T 32T 6.5T 84% /mnt/disk09 /dev/sdi1 20T 792G 19T 5% /mnt/disk10 /mnt/disk01;/mnt/disk02;/mnt/disk03;/mnt/disk04;/mnt/disk07;/mnt/disk08;/mnt/disk09;/mnt/disk10 218T 153T 65T 71% /mnt/disk99 # - - # # - Install package - # sudo apt-get install mhddfs; # - Create mount points for local disks as normal and mount them - # # - Create another directory to hold all your disks, my case called disk99 - # # - Mount all disks into disk99 using mhddfs - # # - Add this line with vim/gedit -# sudo vim /etc/fstab mhddfs#/mnt/disk01,/mnt/disk02,/mnt/disk03,/mnt/disk04,/mnt/disk07,/mnt/disk08,/mnt/disk09,/mnt/disk10 /mnt/disk99 fuse defaults,allow_other,mlimit=10%,nonempty,logfile=/dev/null,loglevel=2 0 0 Warning: in my case with heavy load mhddfs was logfile size consuming and crashed the server many times, therefore I use /dev/null for logging. I couldn't make logrotate work with mhddfs because you need to remount when you change logfiles. # - - # # - uServerOffice - fstab Configuration File - # http://man7.org/linux/man-pages/man8/mount.fuse.8.html https://askubuntu.com/questions/586433/can-mhddfs-mounting-options-be-set-via-etc-fstab-to-respect-file-permissions # - - # # /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # # / was on /dev/sda1 during installation UUID=eb1103d9-86d3-407d-ae28-3347d236ec96 / ext4 errors=remount-ro 0 1 # swap was on /dev/sda5 during installation UUID=c3935e58-42ce-481f-ad0f-7906884479ae none swap sw 0 0 # - - # # - /home/Data-1 is on /dev/sda6 after installation - # UUID=4e1017a6-254e-427f-ba07-6ca5a9ebd9bf /home/Data-1 ext4 defaults 0 2 # - /home/Data-2 is on /dev/sdb1 after installation - # UUID=7be59294-ed5b-4531-bf41-f185aaed6b68 /home/Data-2 ext4 defaults 0 2 # - Put all disks in one - # # - /home/Data-1,/home/Data-2 - # #mhddfs#/home/Data-1,/home/Data-2 /home/AllDisks-Data fuse defaults,allow_other 0 0 mhddfs#/home/Data-1,/home/Data-2 /home/AllDisks-Data fuse default_permissions,allow_other,mlimit=1G 0 0