Installing Antergos Linux with a BTRFS filesytem and subvolumes
This is a tutorial on how to set up an Antergos Linux system with a BTRFS filesystem and several subvolumes, but this setup can applied to any Linux distro, as long as you know which different steps to take to perform the same actions (note that Ubuntu for example already automatically creates subvolumes for the root folder and the home folder if you choose BTRFS as your filesystem, so this setup wouldn’t apply there). These will allow you to take snapshots of important data and do rollbacks when necessary to restore modified and deleted data, but will also allow you to skip including these subvolumes in other snapshots. Note that this setup will not include any kind of RAID.
Unfortunately the information available on the internet is quite scarce – as a lot goes about the same kind of specific setup, is inconsistent and outdated. Even the official kernel page gives me the feeling some parts are outdated, because some parts of the technology are supposedly not supported, but still I see people apply them. I’m talking for example about mount options. I’m compiling all the useful information into this guide as much as possible.
Intro of BTRFS
So what is BTRFS actually? BTRFS is an awesome peace of technology and a storage system that provides way more important features than the traditional ones. It protects your files against bitrot by calculating checksums of files to check their integrity and tries to fix it in the case you have a RAID setup; it allows you to create subvolumes that can be mounted as filesystems are, and be specific about their mount options; you can create snapshots of subvolumes to have a quick kind of back-up (read more on that later) to rollback to, for example after files accidentally were deleted, modified etc.; You can also create subvolumes to make it so their contents are not included in other snapshots; you can use it to have a more solid RAID set (not the case for RAID 5 & 6 currently!); It uses a Copy On Write mechanism for better efficiency; It’s SSD aware and fully compatible; It provides automatic (but optional) file compression; supports online shrinking and extending of partitions, …
Some of these feature may be abstract for some people, so allow me to give some explanation why they are useful.
1. Bitrot prevention
Bitrot is the horrible and unpredictable damage to (part of) the data on your drives as a consequence of – for example – a hidden hardware failure. You need a filesystem that checks the integrity of your data to be able to detect such sudden changes to the data, and provide solutions to correct it. Btrfs does this by making (optionally hardware accelerated) checksums of the data and metadata. Bitrot in theory happens very rarely, but of course every situation is different and influential factors vary as much. RAID sometimes but not always protects you against such failures. This depends on your (hardware) RAID setup and used components. Note that the process of finding corrupt blocks and fixing them isn’t an automatic process. You’ll either need to start a scrub manually or automatically with a systemd timer or cronjob.
Compression allows – at the cost of sacrificing of a little CPU power – for potentially much faster file transfers to your drives as compressible data is reduced into smaller bytes. That means that less data has to be transferred to your drive and will take up less storage on that same drive. The compression will only be applied to data that has not been compressed already, and most of your data probably is if they’re mostly archives, videos and music files. If you’re working with gigabytes of text files for example, you’ll get a big benefit out of this feature! How does btrfs know it’s compressed data or not? Well, it performs a small test by compressing a small part of the data and compares the new versus the old. If there’s no (noticeable) difference, the process will be stopped.
You currently have the option between zlib and lzo compression, the first one using a bit more power to perform better compression. Lzo is the standard choice and you’ll have to make the calculation yourself if you’d rather want to use the other one. It’s mentioned that more algorithms could be supported in the future.
Compression still can’t be set on a subvolume basis according to the kernel wiki, but we’re setting it anyway to be ready for the future, as it’s a feature to be implemented.
3. Subvolumes and snapshots
One of the best features, if not THE best! With btrfs you can create subvolumes and mount them as so-called separate filesystems. This allows you to set different mount options for each one of them if wanted. You can have one for your virtual machine folder, so you can disable Copy On Write on it to prevent performance drawbacks and fragmentation due to constantly changing VM images. You can have another one for folders that contain a lot of compressed data, so you can disable the compress option on it.
With subvolumes you can also start taking snapshots of your system (limited to the subvolumes). These snapshots are atomic-level and restore the data in the subvolume exactly as it was at the time the snapshot was taken, including permissions and all other things. This is in particular extremely useful when installing updates. New Linux kernel patch available? Take a snapshot beforehand you can go back to how it was previously in case an issue is created as a result of that upgrade. Because of the COW mechanism, these snapshots take nearly no space, so you can go ahead and take plenty of them!
Do note that snaphots are not backups! If something happens on the hardware level of your drive and the data becomes corrupt, a snapshot won’t help you, because the original data isn’t available. RAID can help you out as a backup for hardware issues to one or more drives, depending on your setup. But as RAID isn’t a backup solution itself, you still need to take external backups of your data, external meaning not on the same drive. An off-site backup will cover you in the case of a burnt down house, exploding computers because Windows was installed on there too, theft, and other things that can happen to all your drives in one go. The send/receive are supposed to help you with that, but I haven’t researched that enough to be giving you advice here at this time. Otherwise dd, gnome-disks, rsync can help out too.
An interesting side of subvolumes is that you can create more of them of paths you don’t want to be included in global snapshots. You’ll learn more about that in my specific setup.
4. Copy On Write
This mechanism allows for fast file moving & copying because it will reference to the original file on the filesystem, this prevents an exact double copy on the same filesystem. So for file X process 1, 2 and 3 that made copies of it will be using the same file in the background. If one of them changes one byte, that file will be copied to a new location so that process uses the new file. This mechanism is very efficient for data that is not frequently changed, but it isn’t for the opposite. Be sure to set the nodatacow option on subvolumes or the NOCOW attribute to specific files. This also disables checksums and compression on that data.
5. Online resizing of partitions
With many filesystems you can extend their partitions while it’s in use (online), but you have to perform the shrinking when it’s not in use (offline), like at boot time or in an unmounted state. Btrfs allows you to do both while the partition is in use!
6. And more!
For the other features, I will forward you to the wiki page: https://btrfs.wiki.kernel.org/index.php/Main_Page#Major_Features_Currently_Implemented
Now let’s get to the real stuff! I want to achieve a solid filesystem where I can take snapshots and rollback to previous states of my machine. I want the compression to handle compressible data where possible. I want to be able to resize my filesystem easily so I can apply different ideas for my drive at a later time. I want, I want, I want! I’m not bothered by bitrot on my main drive, because I’m very much convinced that either my drive will fail first or I will perform a fresh installation before such a thing could ever occur. On my data disk HDD, bitrot is more of an issue because there all my media files are stored for long term storage, but still, I don’t think that’s really an issue, so for now, I’m not going to focus on that and build a RAID array. It’s an enterprise drive and it’s supposed to survive even the apocalypse. Even my consumer external 2TB drive kept my date stored properly for many years without any noticeable bitrot damage, although I was expecting a total hardware failure in the near future because of the age of the device. A RAID will only create more unwanted noise. Maybe later on I’ll change my mind and that’s the beauty of it, I easily can with btrfs.
So lots of words to say that, even though bitrot is considered the main boogieman in the ZFS scene, it’s not that much of an issue in real life for the non-business scene. So I’m not going to make that the main focus pointlessly.
So why BTRFS and not ZFS? Because BTRFS is more compatible with Linux license-wise, because it’s the main focus of SUSE and Synology (not Red Hat anymore though) and because I expect much of BTRFS in the future.
I’m going to start off by explaining what records will be added to the /etc/fstab file. This is the system configuration file that handles all the automatic filesystem mounting.
/ btrfs defaults,noatime,compress=lzo,ssd,space_cache,subvol=/_current/ROOT 0 0
Since this is the first line and the most important one of all – because the filesystem options are applied here -, I will explain my choice of options thoroughly.
inode_cache is only useful in some cases and can slow a system down otherwise.
noatime is a performance bottleneck and is only useful when the access times of files are actually used and they are mostly not. Also, it’s supposed to increase the size of your snapshots a lot, for what point if we don’t need it? So I leave it out!
compress is good. It uses up only a bit of CPU power (in a multithreaded way) to increase data throughput to your drive a lot and saves space when it leaves it there. Data that can be compressed, will be compressed and data that is already compressed (archives, videos, music …) it will leave alone. This option can also be deactivated per subvolume (if it’s actually already implemented in BTRFS)
Even though you see a lot of people adding the discard option, I leave it out. Discard depends a lot on queued trimming to do its job propery without any impact on your drive’s performance, so if your SSD doesn’t support it, it can affect the performance negatively because it freezes up your SSD a little bit every time it performs its task. I prefer to use a daily fstrim timer instead, instead of discard trimming the drive on every delete.
The autodefrag option is a confusing one. SSDs generally don’t need defragmentation because of the way the controllers handle the data, it’s even said that it even can reduce the lifetime of the drive, due to the defragmentation process wasting a lot of write cycles. But on the other side, this argument was very much mentioned and valid with the early generations of solid state drives. They have evolved quickly and can handle way more write cycles nowadays. Since fragmentation even happens on SSD, especially with COW filesystems like BTRFS, the autodefrag could help out to keep data organised on the drive in the case of very fragmented. I don’t believe there’ll be an advantage on my desktop system, so I leave it out. It’s always possible to do a manual defragmentation of specific data later on. If you use an HDD for the setup, add this option.
ssd allows btrfs to handle the drive as an SSD. Leave it out if you’re using an HDD. This option will be applied to all subsequent subvolume mounts on the same filesystem.
space_cache caches free space on your drive. This will prevent the drive from scanning the B-trees every time to check free space. Meaning: more performance! This option will be applied to all subsequent subvolume mounts on the same filesystem.
subvol is the subvolume option to mention it by path. There’s also subvolid as an alternative.
/home btrfs defaults,compress=lzo,subvol=/_current/home 0 0
Mostly the same as above. I made /home a separate subvolume so it isn’t included in the snapshots of the ROOT subvolume. Rollbacks in ROOT would otherwise overwrite our /home folders too and accidentally remove new files or modifications to files.
/var/cache/pacman/pkg btrfs default,subvol=/_current/pkg 0 0
The pacman cache folder is full of compressed files, so there’s no need for the compress option, as it doesn’t work on data that already has been compressed. So better save the extra cpu cycles in this case.
/tmp btrfs defaults,nodatacow,subvol=/_current/tmp 0 0
A folder containing content that changes very often, so no need for the COW feature to work here as it won’t do much good, because it can cause a lot of unnecessarily fragmentation on the filesystem and slow it down unnecessarily. Even though that’s not too much of an issue on SSD’s, it can still cause such an effect. The data also won’t be checksummed as a result, but that’s okay for this folder, but we won’t get much use out of that feature in a single drive setup anyway. It unfortunately also disables compression. So the nodatacow option is added here.
/var/tmp btrfs defaults,nodatacow,subvol=/_current/vartmp 0 0
Same as above
/var/run btrfs defaults,nodatacow,subvol=/_current/varrun 0 0
Same as above
/var/log btrfs defaults,compress=lzo,subvol=/_current/log 0 0
Logs should never be rollbacked, hence a different subvolume for that folder.
So here we are. Before we continue, I want to mention kudos to pyUser for giving the basic template of instructions for this setup (https://forum.antergos.com/topic/4157/antergos-on-a-btrfs-b-tree-file-system/11). Keep this person in mind after you experience euphoria with your complete BTRFS setup.
Let’s first start the Antergos Cnchi installer. But wait! Before you do that, if you are a non qwerty peasant like me, change your keyboard layout first. It will help you tons during the installation. Connect to the internet as the next step.
We’re ready for the installation now. Perform all steps until the partitioning. Choose “Choose exactly where Antergos should be installed. Click on New partition table, choose msdos (MBR). GPT is actually a better choice for reliability in combination with UEFI, but we’re not going to cover that now. If you want it, create a swap drive of at least the size of your RAM. Then format the rest as a BTRFS filesystem. Uncheck the bottom option of the bootloader installation because we’re going to do that later. Next. Perform the rest of the installation steps and in the end don’t restart the system.
Go into root user mode and get your partition location by running these commands:
My btrfs partition is located at /dev/sda1 so I will be using that for the rest of the tutorial now. Be sure to correct it with your own different location.
1. Create the subvolumes
mount /dev/sda1 /mnt
btrfs subvolume create ROOT
btrfs subvolume create home
btrfs subvolume create pkg
btrfs subvolume create tmp
btrfs subvolume create vartmp
btrfs subvolume create varrun
btrfs subvolume create log
2. Moving the files
Now move the data from the original folder to the subvolumes.
mv var/cache/pacman/pkg/* _current/pkg
mv home/* _current/home
mv var/log/* _current/log
mv * _current/ROOT
3. fstab modification
Open the fstab file:
Copy the above contents for the /etc/fstab file and add these to the file, by replacing the first line of the btrfs root filesystem record. If you also created a swap and other partitions at installation time, don’t remove them! Replace the x in the UUID with the real UUID of your partition.
UUID=x / btrfs defaults,noatime,nodiratime,compress=lzo,ssd,space_cache,subvol=/_current/ROOT 0 0
UUID=x /home btrfs defaults,compress=lzo,subvol=/_current/home 0 0
UUID=x /var/cache/pacman/pkg btrfs default,subvol=/_current/pkg 0 0
UUID=x /tmp btrfs defaults,nodatacow,subvol=/_current/tmp 0 0
UUID=x /var/tmp btrfs defaults,nodatacow,subvol=/_current/vartmp 0 0
UUID=x /var/run btrfs defaults,nodatacow,subvol=/_current/varrun 0 0
UUID=x /var/log btrfs defaults,compress=lzo,subvol=/_current/log 0 0
UUID=x /run/btrfs-root btrfs defaults 0 0
Save and close with
CTRL + X
4. Configuring boot
Go to the HOOKS section and add btrfs at the end of the line without the #.
HOOKS="base udev autodetect modconf block keyboard keymap filesystems"
HOOKS="base udev autodetect modconf block keyboard keymap filesystems btrfs"
Now mount the new filesystem to configure and install GRUB.
mount -o defaults,discard,ssd,subvol=_current/ROOT /dev/sda1 /install
mount -o defaults,discard,ssd,subvol=_current/home /dev/sda1 /install/home
mount -o defaults,discard,ssd,subvol=_current/pkg /dev/sda1 /install/var/cache/pacman/pkg
arch-chroot /install /bin/bash
pacman -S grub os-prober
grub-mkconfig -o /boot/grub/grub.cfg
mkinitcpio -p linux
Reboot and you’ll be able to use your new Antergos Linux OS with btrfs!
Note that when restarting or shutting down, you’ll get an error that the /var/log subvolume couldn’t be unmounted. Don’t worry about that, that’s just because the logs are still in use by the system during a shutdown process. So it’s safe to ignore that
Post-config: trim and snapper
Since we didn’t use the discard mount option, we have to enable the fstrim timer. By standard, fstrim runs weekly. If you’d like to change that, modify the timer file.
sudo nano /usr/lib/systemd/system/fstrim.timer
Change OnCalendar=weekly to daily or monthly.
sudo systemctl enable fstrim.timer
Install snapper to manage btrfs snapshots way easier. Also add configuration files for the root and home folders.
sudo pacman -S snapper
sudo snapper -c ROOT create-config /
sudo snapper -c home create-config /home
Create your first snapshots with these commands:
sudo snapper -c ROOT create -d "Fresh installation of Antergos"
sudo snapper -c home create -d "Fresh installation of Antergos"
- BTRFS Wiki:
- Recommended folders:
- Btrfs filesystem mount options:
- Mount subvolume options:
- Generic mount options:
- Arch btrfs installation manuals:
- COW & NOCOW benchmarks:
- BTRFS vs other filesystems benchmarks:
- Defragmentation with btrfs on SSDs: