Backup on Bastien Traverse

Moving an existing install into a virtual machine

Tue, 05 Apr 2022 00:00:00 +0200

This a follow-up to my tale of a system image.

For fun and convenience, I now wanted to have a bootable image of my old laptop.

So at first I thought of making a Live ISO out of it, but didn’t know the name of the process. I tried a few searches like “convert a configured system into ISO” or “create live ISO from running system”, but what I found looked strangely difficult and didn’t seem to really address my need.

Then in the course of writing the aforementioned post, I was reminded of an Archwiki article I had seen some time ago: Moving an existing install into (or out of) a virtual machine. This was exactly what I wanted, as I didn’t need a Live ISO so much as a VM that I could boot on my main machine whenever I wanted to check something, or bask nostalgically into the glow of my old wallpaper.

Looking into it, I stumbled upon presentation slides by Clonezilla and learned that this process had a name: “Physical-to-Virtual”, or P2V for short (here is another article by IBM from 2009 about this topic).

The ultimate OS conservation project: virtualizing to immortality

Indeed, with a VM image of your old install you can easily keep it around forever, as with the specifications of the machine written down into an .xml file or in a QEMU command line invocation, you can spin one up at will.

I’ll use this opportunity to simplify my system stack, namely to rid it of both LUKS and LVM and consolidate all the existing logical volumes into one single filesystem.

So how do we go about turning our physical system into a functional VM? Here is a rough outline of the process:

Preparing the VM image
Transferring the data
Making the necessary adjustments
Profit

Note

There are different ways to approach steps 1 & 2 depending on what you are working with, e.g. your target hypervisor, what kind of backup/file transfer program is available to you and whether or not you need to modify the image.

If you need to do any modifications to the partitions/filesystems, forget about Clonezilla/partclone/FSArchiver savepart mode and use a file-level transfer method.

Since this is my case I’ll describe this approach, but if e.g. for you the physical and virtual machines are connected via the network and no modifications are needed, then Clonezilla remote device cloning could work great.

(Heck, if no modifications are needed you could even just use a loopback file mounted on /home/partimag in the live environment on the physical machine and have Clonezilla make a local clone!)

Here I will target QEMU for the hypervisor as that’s what I use, but only little adjustments should be required for other hypervisors.

All right, let’s delve into it!

Step 1: Preparing the VM image

Here we will create and prepare the image file which will be used as the VM drive.

In this case I will directly create and work with QEMU qcow format, but will also illustrate how to do it all with a raw image file which can be later converted to the desired format if needed.

Creating the file

First we create an image of an appropriate size¹.

qcow image:

qemu-img create -f qcow2 -o cluster_size=2M --prealocation=falloc p2v.qcow2 15G

The -o cluster_size=2M option is used to increase performance.

Then we make the image available to the system:

sudo modprobe nbd max_part=1
sudo qemu-nbd -c /dev/nbd0 p2v.qcow2

Raw image:

On file systems supporting it (ext4, XFS, Btrfs, FAT – but not exFAT), fallocate is the best choice:

fallocate -l 20G p2v.img

For systems without fallocate but sparse file support, use truncate:

truncate -s 20G p2v.img

If even sparse files are too much to ask, then it’s time to get the disk destroyer out (warning: might take a while):

dd if=/dev/zero of=p2v.img bs=1M count=20000

If you really don’t want to know/choose and have QEMU installed, let it handle the details for you:

qemu-img create p2v.img 20G

Partitioning

Now to partition our image. It is easier to use the same partition table type as your physical machine (MBR or GPT), except if you want to convert it of course.

Warning

All of the following commands must be run with elevated privileges. Be careful with what you execute. Replace /dev/nbd0 with p2v.img (or whatever you named it) if using a raw disk image.

MBR

Here is the simplest way I found to create a single partition with MBR scheme aligned on the 1 MB boundary (with the default partition type for GNU/Linux).

printf ',' | sfdisk /dev/nbd0

Crazy right? If that (rightly) scares you, I’ve got an equivalent, more spelled-out parted one-liner:

parted --script --align optimal /dev/nbd0 mklabel msdos mkpart primary 1 100%

GPT

And here are their GPT equivalents (with sgdisk thrown into the mix):

printf ',' | sfdisk --label gpt /dev/nbd0
sgdisk --largest-new 0 /dev/nbd0
parted --script --align optimal /dev/nbd0 mklabel gpt mkpart $name 1 100%

See the $name in that parted command here? That’s because parted requires (yes, requires) that a partition be given a name in GPT mode, even though that’s absolutely not part of any specification… So do as you please to christen the first realm of this new digital kingdom!

Note however that if you are planning to use UEFI, a second EFI System Partition (ESP) is required as well. Here are three more one-liners to create both partitions in one go with a 512 MiB ESP, and the rest for the second partition (source for sfdisk example) :

printf ',512M,U\n,,' | sfdisk --label gpt /dev/nbd0
sgdisk -n 1:0:+512MiB -t 1:ef00 -n 2:0:0 /dev/nbd0
parted --script --align optimal /dev/nbd0 mklabel gpt mkpart EFI 0% 513MiB mkpart System 513MiB 100% set 1 esp

Formatting

Before mounting, we need to format our newly created partition(s) with a filesystem. It’s certainly a safe choice to go with the same one as the source system, albeit not strictly required. Ext4 example:

sudo mkfs.ext4 /dev/nbd0p1

If using a raw image file, we first need to use losetup to make the partition(s) available to the system:

sudo losetup --partscan --find --show --nooverlap p2v.img
--> /dev/loop0

From here on replace /dev/nbd0 with /dev/loop0 (or whatever the command returned).

In case of UEFI, the ESP needs to be formatted as FAT (FAT32 recommended):

sudo mkfs.fat -F32 /dev/nbd0p1
sudo mkfs.ext4 /dev/nbd0p2

Mounting

That’s it, now we can finally mount our image. For BIOS (single partition):

sudo mount /dev/nbd0p1 /mnt

For UEFI (two partitions), replace $ESP with the desired mount point:

sudo mount /dev/nbd0p2 /mnt
sudo mkdir -p /mnt/$ESP
sudo mount /dev/nbd0p1 !$

That’s it for the preparations, we are now ready to transfer the source system content!

Step 2: Transferring the data

This part depends on how you realized your full system backup on the source machine. For completeness’ sake and to create the counterpart to the previous article, I’ll show the restoration process of the various tools presented in the linked section.

partclone

Word of warning concerning partclone: it’s not good at restoring to a device smaller than the source, even if the actual content would fit in. Despite its offering of a “Don’t check device size and free space” (-C, --nocheck) option, my experience matched the web reports saying that restoration fails with a target seek ERROR:Invalid argument. Clonezilla FAQ even warns against this use case.

However, for completeness’ sake I’ll leave a quick restoration example to illustrate how to restore from a zstd-compressed image:

umount -R /mnt
zstdcat /path/to/partclone.img.zst | partclone.restore -o /dev/nbd0pY

Replace the Y at the end with the appropriate partition number for your use case.

Tip

partclone offers a --restore_raw_file option that aims at “creating special raw file for loop device”. This image wouldn’t have a partition table though so it’s not useful for our use case, but it’s still good to know it’s there 😉️

FSArchiver

I have a sweet spot for FSArchiver due to its versatility and very complete feature set. The only real downside I found to it are its exclusion rule patterns being less than clear.

Being a file-aware tool instead of block-aware one like partclone, it has absolutely no issues restoring to a destination smaller than the source (it is even one of the suggested workaround in the aforementioned Clonezilla FAQ entry!).

If you made a backup using its savepart mode, see the official docs for nice examples illustrating the different restoration cases.

On the other hand, if like me you used its savedir mode so as to have more flexibility, restoring is as simple as:

fsarchiver --jobs=$(nproc) restdir /path/to/backup.fsa /mnt/

May St IGNUcius bless FSarchiver and its author/team! 😇️

SquashFS

Another very straightforward option:

unsquashfs -force -dest /mnt/  backup.squashfs

Bonus points go to SquashFS compared with FSArchiver and tar because we can very simply access the archived content without having to restore/extract it: a simple mount backup.squashfs /path/to/mount/point suffices to make it browsable (with very good performance) 👍️

Rsync & tar

rsync is just about reversing the source and destination in its invocation, and tar about switching -c, --create with -x, --extract:

rsync -aHAXUUSh --info=progress2 --partial /path/to/backup/dir/* /mnt/
tar -x --zstd -p --acls --xattrs --atime-preserve=system -f /path/to/backup.tar.zst /mnt/

Alright, now that we have transferred the content of our physical system into the VM image, let’s move to the final adjustments.

Step 3: Adjusting the system config

Now it is time to chroot into our restored environment and make the needed adjustments. This step is very much dependent on your source system configuration and which modifications you wish to implement (if any).

arch-chroot /mnt

systemd-nspawn can replace arch-chroot if not using an Arch-based distro.

In my case, it equated to removing/commenting out references to the former LVM/LUKS devices in /etc/{fstab,crypttab} and configuring a single mount:

sed -i 's/^\([^#].*\)/# \1/g' /etc/crypttab
sed -i 's/^\([^#].*\)/# \1/g' /etc/fstab
echo "/dev/vda1  /  ext4  noatime  0  1" >> /etc/fstab

The two sed invocations comment out “active” lines while leaving blank lines alone.

For GPT replace the last line with:

printf "/dev/vda2  /  ext4  noatime  0  1\n/dev/vda1  /  vfat  defaults  0  2\n" >> /etc/fstab

Network-wise, I had nothing to do since NetworkManager was already handling automatic network configuration with DHCP.

I also removed the now unneeded lvm2, encrypt, keyboard and keymap mkinitcpio hooks as well as the i915 module from /etc/mkinitcpio.conf and regenerated the initramfs.

Finally, I removed the cryptdevice= kernel command line parameter from /etc/default/grub, installed GRUB in the image file boot sector and regenerated its config.

grub-install --target=i386-pc /dev/nbd0
sed -i '/cryptdev/s/.*/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/' /etc/default/grub
grub-mkconfig -o /boot/grub/grub.cfg

Don’t forget to replace references to nbd0 (or loop0) in /boot/grub/grub.cfg with whatever disk naming you will use (in my case /dev/vda):

sed -i 's/nbd0/vda/g' /boot/grub/grub.cfg

Then exit the chroot, unmount the image file and remove the nbd device and module:

exit
sudo umount -R /mnt/
sudo qemu-nbd -d /dev/nbd0
sudo modprobe -r nbd

Or loop mount:

losetup -d /dev/loop0

(Optionally, convert the image to a format suitable to your target hypervisor)

And voilà, you are now ready to try out your brand new virtualized system! Hopefully it will boot on the first try, otherwise just make the image available again as before, mount it and chroot in it to troubleshoot the issue.

Enjoy your nascent hall of past digital abodes! 😉️

Edit (2024-07-09): I’ve just stumbled upon virt-p2v from libguestfs, which is a “GUI interface to convert a physical machine to run as virtual machine on KVM”. It is a companion front-end to virt-v2v, and “comes as an ISO, CD or PXE image that can be booted on physical machines to virtualize those machines”. Looks like a great alternative to try as well!

At a minimum it should be able to accommodate all the data you’re going to transfer from the physical system, plus a little bit of spare space for system maintenance. ↩︎

A system imaging tale: LUKS, Clonezilla and friends

Mon, 14 Feb 2022 00:00:00 +0100

Today’s project: make a system image of my previous laptop for archival purposes before refurbishing it.

Conditions:

make a space efficient image, i.e. not a block-level copy: no need to retain encryption or logical volume setup, only their content matters. However:
all standard and extended file attributes must be correctly preserved, so that if I ever want to boot this image the system will be in a correct state.

The plan: boot up an Arch ISO, open the encrypted drives, set everything up for clonezilla – profit.

Here is how it actually went.

An unexpected obstacle to boot: Ventoy shenanigans

In order to boot into the Arch live environment, I unsheathed my very dear and trusted Sandisk Extreme multiboot USB thumb drive (not the currently sold, lower performing one but the ever chart-topping one from about 10 years ago), which had been irreproachably and impeccably fulfilling its duty in the two years since I had discovered Ventoy, the multiboot solution which surpassed them all and ended my long standing quest to find an easy to use, low-maintenance and low-friction multiboot USB solution. And yes, this was the longest sentence of the entire article 😁

Just recently I had updated it to the latest version using the provided Linux script, which completed without issue. But as soon as I booted on it, I was faced with an error: Warning! This is not a standard device and is not officially supported. Um, beg your pardon? It was working fine those past two years, so certainly it was the update which broke something…

Searching for this error, I eventually found this “kill switch” the dev introduced slightly after I used the software, to try and contain the flood of users with non-standard installs complaining to them when something got wrong.

I can’t really fault them for that, but here is the thing: mine was a pure product of the official tool from back in 2020, with no deviation from the prescribed layout/FS whatsoever. The first partition was starting at the 2048/1 MB boundary and formatted with the default exFAT, the second partition was 65536 sector large, the drive had an MBR partition table and all and all…

# fdisk -l /dev/sda
Disk /dev/sda: 59.63 GiB, 64023257088 bytes, 125045424 sectors
Disk model: Extreme  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd1a119ab

Device     Boot     Start       End   Sectors  Size Id Type
/dev/sda1  *         2048 124979887 124977840 59.6G  7 HPFS/NTFS/exFAT
/dev/sda2       124979888 125045423     65536   32M ef EFI (FAT-12/16/32)>

The official command line tool didn’t complain either while upgrading or when run manually:

# ventoy -l /dev/sda

**********************************************
      Ventoy: 1.0.65  x86_64
      longpanda admin@ventoy.net
      https://www.ventoy.net
**********************************************

Ventoy Version in Disk: 1.0.65
Disk Partition Style  : MBR
Secure Boot Support   : NO

Indeed but nope, I got axed by the kill switch all the same. Fuck artificial restrictions and treacherous computing. And I am burned by a free software project at that… What to expect from this world.

Anyhoo, after transfering out my 24 GB of GNU/Linux ISOs (God bless USB 3.0 & quality flash memory speeds), I forced a reinstall with the -I switch:

# ventoy -I /dev/sda

**********************************************
      Ventoy: 1.0.65  x86_64
      longpanda admin@ventoy.net
      https://www.ventoy.net
**********************************************

Disk : /dev/sda
Size : 59 GB
Style: MBR


Attention:
You will install Ventoy to /dev/sda.
All the data on the disk /dev/sda will be lost!!!

Continue? (y/n) y

All the data on the disk /dev/sda will be lost!!!
Double-check. Continue? (y/n) y

Create partitions on /dev/sda by parted in MBR style ...
Done
Wait for partitions ...
partition exist OK
create efi fat fs /dev/sda2 ...
mkfs.fat 4.2 (2021-01-31)
success
Wait for partitions ...
/dev/sda1 exist OK
/dev/sda2 exist OK
partition exist OK
Format partition 1 /dev/sda1 ...
mkexfatfs 1.3.0
Creating... done.
Flushing... done.
File system created successfully.
mkexfatfs success
writing data to disk ...
sync data ...
esp partition processing ...
Open ventoy efi file 0x610ac0
ventoy x64 efi file size 1757184 ...
Open bootx64 efi file 0x610ac0
Open ventoy ia32 efi file 0x610f10
ventoy efi file size 1183744 ...
Open bootia32 efi file 0x610ac0

Install Ventoy to /dev/sda successfully finished.

sudo ventoy -I /dev/sda  5,11s user 2,58s system 19% cpu 39,813 total

After transferring my files back and rebooting on the flash drive, everything was back in order and I was ready to rock 🤟

Optional step: setting up a comfortable environment

Being console-based, the archiso environment is somewhat austere. So the first thing I usually do is to set up SSH access so as to connect from my main machine:

# set french keymap
# the "-latin9" part is only needed if using e.g. accents
loadkeys fr-latin9
# directly connect to a known Wifi SSID
iwctl station wlan0 connect <SSID>
# Enable NTP to have accurate time
timedatectl set-timezone <Area/City>
timedatectl set-ntp true
# set root passhrase to enable SSH connection
passwd
# start SSH service
systemctl start sshd
# get IP address
iwctl station wlan0 show | grep IP | awk '{print $3}'
# or without iwd:
ip a | grep wlan | awk 'FNR == 2 {print substr($2, 1, length($2)-3)}'

Finally, connect to the live environment via SSH:

ssh -o PreferredAuthentications=password \
    -o PubkeyAuthentication=no \
    -o UserKnownHostsFile=/dev/null \
    root@<other-machine-IP>

👉️ The options used allow not to be affected by the local SSH configuration (SSH keys…), as well as well not polluting the local KnownHosts file with this one-time connection.

That’s it, now we can resume with the common procedure!

Clonezilla and LUKS/LVM

In order to efficiently image the system, we first need to open any LUKS container and make sure all the LVs are displayed correctly:

cryptsetup open --allow-discards /dev/sdb2 os
lvs

Then we create the /home/partimag directory (hardcoded backup destination in Clonezilla) and mount our drive on it:

mkdir /home/partimag
mount -o noatime /dev/sdc1 !$

Note

When booting the archiso, I selected the “Copy to RAM” option so as to free up the drive and reuse it as backup destination.

However, in order to do that I first needed to get rid of a lingering /dev/mapper/ventoy mapping on its first partition, which made eject/udisksctl throw errors despite the fact that it wasn’t mounted and lsof | grep -e ventoy -e sdb didn’t return anything. I didn’t want to pull it out on the off chance it’d corrupt something…

I wrote a separate post which goes into the details, but the gist is a simple dmsetup remove ventoy took care of the mapping, and then I was able to normally mount the drive.

And now it is on! Launch clonezilla from the command line and follow the on-screen instructions depending on your needs. Some personal recommendations:

use expert mode
avoid the 4G backup splitting by entering a large number when prompted (like 100000)
select “parallel zstd” (-z9p) as the compression algorithm

Or if you feel like keeping it CLI-only…

/usr/bin/ocs-sr \
    --use-partclone \
    --confirm \
    --clone-hidden-data \
    --zstdmt-compress
    --image-size 1000000 \
    --fsck-src-part \
    --skip-enc-ocs-img \
    --postaction choose \
    saveparts <backup-name> sdb1 sdb2

This is the long form equivalent of the

/usr/bin/ocs-sr -q2 -c -j2 -z9p -i 1000000 -fsck -senc -p choose saveparts <backup-name> sdb1 sdb

that got copied in a file under /tmp after validating my options in the TUI.

Then we just sit back and relax… Wait, what? dd mode? Why aren’t my OS logical volumes backed up by the regular free space-aware partclone mode? I explicitly decrypted the containers for that purpose, and saw the LVs being listed during the preparation screens! Although it’s true I could only select whole partitions to be backed up and not specific logical volumes, and saw a line flash by saying something like Shutting down the Volume manager… Oh come on.

One Ctrl+c later the backup of /dev/sdb2 was cancelled, but the unencrypted /dev/sdb1 with the arch LV (the only one in the boot VG) proceeded without issue. What the heck clonezilla?

Hum alright, according to this 2014 thread with the developer’s input, it seems this isn’t as straightforward as it looks. And it happened even much more recently to another user (although according to the dev’s answer it shouldn’t be the case), so anyway… Time for some manual partclone fun I guess!

System imaging, the manual way

Full disclosure: I had already stumbled upon the “manual partclone required” caveat when trying to efficiently backup LUKS-encrypted systems with Clonezilla, so I wasn’t really at a loss. All in all it’s pretty simple, you just have to use partclone like so:

partclone.$fstype --clone --source ... --output ...

Wich gave in my case:

partclone.ext4 --clone --source /dev/mapper/arch-root --output <backup path>/$(date -I)-old-laptop-archroot.ext4-ptcl.img

Tip

If you want to have the same nice ncurses-based UI as Clonezilla does, add the -N/--ncurses option (example).

Though if we really want to mimick Clonezilla and have space-efficient backups, we gotta add compression to the mix!

Compressing partclone images with zstd

Being a devout Zstandard Zealot© (this post summarizes well my feelings – no affiliation with the author), here is how to interface it with partclone for a nicely multi-threaded compression:

partclone.ext4 --clone --source /dev/mapper/arch-root | zstdmt -o <backup path>/$(date -I)-old-laptop-archroot.ext4-ptcl-img.zst

Info

From zstd man page: zstdmt is equivalent to zstd -T0, enabling auto-adjusted multi-threading.

If you are like me and have several LVs to clone and their names follow a pattern, you can use a for loop to optimize things a bit:

for i in arch-{root,home,opt,var} boot-arch; do partclone.ext4 --clone --source /dev/mapper/"${i}" | zstdmt -o <backup path>/$(date -I)-"${i}".ext4-ptcl-img.zst; done

Et voilà, space-efficient compressed images of your decrypted LVs!

Tip

Someone made a gist using somewhat older commands and touching upon restoration etc… Give it a look if you want to review the process!

Extra: comparing the process with different tools

A very precious thing in this world is diversity. Of people, ideas, food… And backup tools. Here is a rundown of the same task undertaken with four of them:

FSArchiver

The first time I realized I couldn’t use Clonezilla to properly back up my decrypted LVs, I stumbled upon a great alternative to partclone: FSArchiver.

The main reason for using it over the former? FSArchiver:

directly integrates multi-threaded compression with zstd (and recommends it over any others)
allows to specify multiple filesystems to be archived at once
also works with directories (creating a compressed and checksummed tarball of sorts)
allows to restore to a different filesystem and creating it in one go
allows to restore to a smaller partition by default, whereas partclone needs the -C option (and often just ends on a seek error)
accepts exclusion patterns to filter what’s archived/restored (the syntax is a bit tricky though)
offers to optionally encrypt data in the archive
and is generally really careful with data integrity… If that wasn’t already reason enough 😉

Its multi-threaded compression-ready invocation goes something like this:

fsarchiver --zstd=3 --jobs=$(nproc) savefs <backup path>/<archive-name>.fsa <list of devices to backup> --exclude=<exclude pattern>

--zstd=3 stands for zstd-compression at level 3
--jobs=$(nproc) works exactly like the eponymous make switch, enabling as many threads as there are logical cores on your system.

So for example:

fsarchiver -Z3 -j$(nproc) -v savefs <backup-path>/old-laptop.fsa /dev/mapper/arch-{root,home,opt,var} /dev/mapper/boot-arch --exclude="lost+found" --exclude="/var/cache/pacman/pkg/*"

The -v/--verbose option is useful to get a progress report, however it will list every file being archived and as such likely blow away your terminal history.

BTW, here is why clonezilla defaults to partclone instead of fsarchiver in the words of its author. Tldr: partclone supports pipes/stdin/stdout which are needed for other Clonezilla features, while fsarchiver doesn’t. Tough life.

Note however that FSArchiver (just like partclone) doesn’t deal with recreating a partition table on the destination, or any other form of partitioning/preparing the drive such as LUKS/LVM: you gotta do all that manually up to the point where you are ready to restore filsystems (that is to say, fsarchiver isn’t a disk cloning solution).

Also, you cannot merge the content of separate partitions created this way into one destination (e.g. if you want to consolidate separate LVs into one FS): they will overwrite each other. For that you’d rather use the savedir/restdir mode, which is akin to a tarball on steroids.

Squashfs

I discovered this method by stumbling upon this gem on the Arch wiki, “Full system backup with SquashFS”, and although the article is in dire need of a proper rewrite (which I’ve added to my todo list) I really wanted to give it a try.

In the words of TLDP, SquashFS is

“a read-only file system that lets you compress whole file systems or single directories, write them to other devices/partitions or to ordinary files, and then mount them directly (if a device) or using a loopback device (if it is a file). […] For archiving purposes, SquashFS gives you a lot more flexibility and performance speed than a tarball archive.”

Its selling points to me are:

random access & transparent decompression: no need to decompress and extract the entire image to browse its content (this is huge, especially when talking about gigabyte-sized images…)
file-level deduplication
exclusion patterns
zstd support 😊

However be aware that it doesn’t support ACLs, so if you rely on them you should look somewhere else.

After quickly perusing its manpage I gave it a go:

mount -o ro /dev/mapper/arch-root /mnt
mount -o ro /dev/mapper/boot-arch /mnt/boot
for i in home opt var; do mount -o ro /dev/mapper/arch-"${i}" /mnt/$i
cd /mnt

mksquashfs ./ <backup-path>/old-laptop.sqsh \
    -comp zstd \
    -no-exports \
    -progress \
    -mem 6G \
    -not-reproducible \
    -e var/cache/pacman/pkg

Results:

Squashfs 4.0 filesystem, zstd compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments,
	compressed xattrs, compressed ids
	duplicates are removed
Filesystem size 3912057.66 Kbytes (3820.37 Mbytes)
	35.64% of uncompressed filesystem size (10975717.90 Kbytes)
Inode table size 2886293 bytes (2818.65 Kbytes)
	24.42% of uncompressed inode table size (11819496 bytes)
Directory table size 3606490 bytes (3521.96 Kbytes)
	36.39% of uncompressed directory table size (9910813 bytes)
Xattr table size 621 bytes (0.61 Kbytes)
	28.98% of uncompressed xattr table size (2143 bytes)

To put things in perspective:

root@archiso ~ # du -sh /mnt
16G	/mnt
root@archiso ~ # du -sh /mnt/var/cache/pacman/pkg
4.5G	/mnt/var/cache/pacman/pkg
root@archiso ~ # du -h <backup-path>/old-laptop.sqsh
3.8G	<backup-path>/old-laptop.sqsh

So it went from 16-4.5=11.5 GiB to 3.8… Nice!

It produced a slightly smaller archive than both partclone and fsarchiver (even when accounting for the non-excluded pacman cache), but it took 11.5 minutes. After testing, fsarchiver savedir took 3min08s and produced a 4.2 GiB image (with --exclude='*.pkg.tar.zst'), but then fsarchiver doesn’t produce browsable archives and doesn’t do any deduplication 😊

Rsync

One step more remote from system imaging, the ever powerful Swiss-army knife of file transfer can evidently also be used to create full system backups. Here is my personal take on it:

rsync \
    --archive \
    --hard-links \
    --acls \
    --xattrs \
    --atimes \
    --open-noatime \
    --sparse \
    --human-readable \
    --info=progress2 \
    --partial \
    --exclude={"var/cache/pacman/pkg/*","*/lost+found"} \
    /mnt/* /path/to/backup/dir

Or as a one-liner:

rsync -aHAXUUSh --info=progress2 --partial --exclude={"var/cache/pacman/pkg/*","*/lost+found"} /mnt/* /path/to/backup/dir

See its man page for details on the options used. Note that the repeated -U is not a mistake but a shortcut to enable both --atimes and --open-noatime 😉

However, while doing so I realized rsync wasn’t exactly the best solution when the destination filesystem is {ex,}FAT… Timestamps and permissions/ACLs/xattr won’t carry over correctly¹ 😒 This is what got me for the very first time interested in using…

Tar

O venerable Lord of computerized backups! Thy name is reminiscent of the glorious days of yore, when backups were mere tape archives…

Okay, but how does it fare in 2022 for complete system backup with high fidelity? After delving into its manpage this is what I came up with:

cd /mnt
tar --create \
    --preserve-permissions \
    --acls \
    --xattrs \
    --atime-preserve=system \
    --exclude={"var/cache/pacman/pkg/*","*lost+found"} \
    -I zstdmt \
    -f /path/to/backup/dir/<backup-name>.tar.zst \
    *
tar: /var/run/nscd/socket: socket ignored
...
# repeat dozens of times

Hmmm okay, apart from socket files which apparently clearly don’t matter, what about the completeness of the backup? Sections verify and compare from tar manual cover this question. A few pitfalls to be aware of however:

--verify (only for archive creation) doesn’t work with compressed archives; if you don’t mind you can first create and verify an uncompressed archive, then compress it in a second step…
--compare “ignores files in the file system that do not have corresponding members in the archive”, so if something was left over during the backup, no errors will be reported.

If you have the space and will for it, you could also just extract the archive and diff --recursive it against the source, but all in all I feel this is an aspect where the original (and still current) purpose of tar as a tape specific archiving tool, makes itself felt.

From my superficial testing everything seemed mostly alright, but I can’t shake the feelings that tar isn’t making as complete a clone as fsarchiver or rsync (see its limitations too). But that’s probably because my beard isn’t grey enough I guess… 😁

Wrap up

In conclusion, if you are in a situation where Clonezilla doesn’t work for you (most notably when using LUKS), my recommendation would be to turn to either FSArchiver, partclone or SquashFS. They seem to provide the most complete and reliable feature-set of the lot for imaging systems in a correct and efficient way. Rsync and tar being more general purpose tools, they require more options not to leave something out or otherwise shoot yourself in the foot.

Also, while laboring over this article I realized that while it was all fine and dandy to have a static archive of my old system, the next step was to make an interactive image of it that I could boot on whenever I want to see which GTK theme I used, or otherwise take a nostalgic whiff and relapse in my former computing abode…

This reminded me of the Moving an existing install into a virtual machine article, and I have an entire article coming up with the fruits of this endeavour: creating a VM image from the backups made here, testing in the process the oh-so-important restoration side of all the different tools used here (which taught me that an un-restored backup is no backup at all), making the required adjustments to be able to boot from it… Stay stuned for the second part! [edit: there it goes]

Thanks for reading and happy imaging 🙏

One workaround is to use a loop device file, i.e. create a file sufficiently large to hold your backup, partition it, activate it via losetup, then finally format it with a more amenable filesystem and mount it as the backup destination (example). ↩︎