DwarFS vs. SquashFS
I couldn’t see if DwarFS was comparing against SquashFS with LZMA. So I did my own tests.
Results
| Time | Size | Options | Details | |
|---|---|---|---|---|
| mksquashfs | 52.991 s ± 1.483 s | 213932 | -comp zstd -Xcompression-level 22 | zstd:level=22 (all) |
| mksquashfs | 50.122 s ± 0.871 s | 199332 | -comp zstd -Xcompression-level 22 -b 1048576 | zstd:level=22 (all) |
| mkdwarfs | 61.023 s ± 1.207 s | 182864 | –compress-level 7 | zstd:level=22 (block/meta) zstd:level=12 (schema) nilsimsa inode order |
| mksquashfs | 41.936 s ± 0.630 s | 213944 | -comp zstd -Xcompression-level 19 -b 1048576 | zstd:level=19 (all) |
| mkdwarfs | 42.884 s ± 1.007 s | 188816 | –compress-level 5 | zstd:level=19 (block) zstd:level=12 (schema) none (meta) similarity inode order |
| mksquashfs | 3.870 s ± 0.040 s | 218100 | -comp zstd -Xcompression-level 5 -b 1048576 | zstd:level=5 (all) |
| mkdwarfs | 4.674 s ± 0.051 s | 253192 | –compress-level 2 | lz4hc:level=9 (block) zstd:level=12 (meta) path inode order |
TL;DR:
Choose mkdwarfs if you need to make smallest size possible. It’s also said to has advantages in. Also note
https://github.com/mhx/dwarfs#with-squashfs--xz
Notes:
- mksquashfs benefits a lot from increasing block size
- SquashFS does not support
systemxattr prefix or ACL - Dell XPS 13” 9310 , 32GB RAM, Ubuntu 22.04. Generic Kernel.
- DwarFS and SquashFS on 8 cores
Preparation
# We use Fedora 36 Cloud Image root as test file system
$ wget https://download.fedoraproject.org/pub/fedora/linux/releases/36/Cloud/x86_64/images/Fedora-Cloud-Base-36-1.5.x86_64.raw.xz
$ xz -d Fedora-Cloud-Base-36-1.5.x86_64.raw.xz
$ du Fedora-Cloud-Base-36-1.5.x86_64.raw.xz
353399556 Fedora-Cloud-Base-36-1.5.x86_64.raw
# We use DwarFS 0.6.1
wget https://github.com/mhx/dwarfs/releases/download/v0.6.1/dwarfs-0.6.1-Linux.tar.xz
sudo tar -C /usr/local -xvf dwarfs-0.6.1-Linux.tar.xz --strip-components=1
# We use SquashFS from U
squashfs-tools 1:4.5-3build1
# mount the filesystem root
$ sudo kpartx -arv Fedora-Cloud-Base-36-1.5.x86_64.raw
add map loop27p1 (253:1): 0 2048 linear 7:27 2048
add map loop27p2 (253:2): 0 2048000 linear 7:27 4096
add map loop27p3 (253:3): 0 204800 linear 7:27 2052096
add map loop27p4 (253:4): 0 8192 linear 7:27 2256896
add map loop27p5 (253:5): 0 8218624 linear 7:27 2265088
$ mkdir /mnt/testing
$ sudo mount -o ro /dev/mapper/loop27p5 /mnt/testing
Looking at comparable compression
DwarFS lists following
Compression level defaults:
-------------------------------------------------------------------------------
Level Block Compression Algorithm Window Inode
Size Block Data Schema Metadata Size/Step Order
-------------------------------------------------------------------------------
0 20 null null null 0 / 0 none
1 20 lz4 zstd:level=12 null 0 / 0 path
2 20 lz4hc:level=9 zstd:level=12 null 0 / 0 path
3 21 lz4hc:level=9 zstd:level=12 null 12 / 1 similarity
4 22 zstd:level=11 zstd:level=12 null 12 / 2 similarity
5 23 zstd:level=19 zstd:level=12 null 12 / 2 similarity
6 24 zstd:level=22 zstd:level=12 null 12 / 3 nilsimsa
7 24 zstd:level=22 zstd:level=12 zstd:level=22 12 / 3 nilsimsa
8 24 lzma:level=9 zstd:level=12 lzma:level=9 12 / 4 nilsimsa
9 26 lzma:level=9 zstd:level=12 lzma:level=9 12 / 4 nilsimsa
-------------------------------------------------------------------------------
Tests
DwarFS --compress-level 7
$ hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/dwarfs || true' \
--show-output 'sudo mkdwarfs -i /mnt/testing -o /tmp/dwarfs --compress-level 7'
…
I 10:09:22.937113 scanning /mnt/testing
I 10:09:24.288998 assigning directory and link inodes...
I 10:09:24.292942 waiting for background scanners...
I 10:09:24.976626 scanning CPU time: 9.148s
I 10:09:24.976682 finalizing file inodes...
I 10:09:24.986637 saved 12.66 MiB / 532.7 MiB in 1366/21195 duplicate files
I 10:09:24.986718 assigning device inodes...
I 10:09:24.987365 assigning pipe/socket inodes...
I 10:09:24.987729 building metadata...
I 10:09:24.987773 building blocks...
I 10:09:24.987790 saving names and symlinks...
I 10:09:24.987887 using a 4 KiB window at 512 B steps for segment analysis
I 10:09:24.987960 bloom filter size: 64 KiB
I 10:09:24.988098 ordering 17417 inodes using nilsimsa similarity...
I 10:09:24.988752 nilsimsa: depth=20000 (1000), limit=255
I 10:09:25.000510 pre-sorted index (16631 name, 320 path lookups) [11.7ms]
I 10:09:25.002358 updating name and link indices...
I 10:09:26.447139 17417 inodes ordered [1.459s, 1.382s CPU]
I 10:09:26.447827 waiting for segmenting/blockifying to finish...
I 10:09:32.641476 segmenting/blockifying CPU time: 5.457s
I 10:09:32.641858 bloom filter reject rate: 96.112% (TPR=0.025%, lookups=475438859)
I 10:09:32.642210 segmentation matches: good=2102, bad=2585, total=7423
I 10:09:32.642479 segmentation collisions: L1=0.020%, L2=0.008% [1022164 hashes]
I 10:09:32.642758 saving chunks...
I 10:09:32.650680 saving directories...
I 10:09:32.657453 saving shared files table...
I 10:09:32.673006 saving names table... [13.63ms]
I 10:09:32.677200 saving symlinks table... [3.467ms]
I 10:09:32.707513 waiting for compression to finish...
I 10:10:24.508446 compressed 532.7 MiB to 178.6 MiB (ratio=0.335197)
I 10:10:24.516629 compression CPU time: 393.2s
I 10:10:24.516736 filesystem created without errors [61.58s]
…
waiting for block compression to finish
3629 dirs, 3124/2412 soft/hard links, 21195/21195 files, 0 other
original size: 532.7 MiB, dedupe: 12.66 MiB (1366 files), segment: 20.76 MiB
filesystem: 499.2 MiB in 32 blocks (20716 chunks, 17417/17417 inodes)
compressed filesystem: 32 blocks/178.6 MiB written [depth: 20000]
…
Time (mean ± σ): 61.023 s ± 1.207 s [User: 0.005 s, System: 0.025 s]
Range (min … max): 59.630 s … 61.757 s 3 runs
$ du /tmp/dwarfs
182864 /tmp/dwarfs
SquashFS -comp zstd -Xcompression-level 22
hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/squashfs || true' \
--show-output 'sudo mksquashfs /mnt/testing /tmp/squashfs -comp zstd -Xcompression-level 22'
…
Exportable Squashfs 4.0 filesystem, zstd compressed, data block size 131072
compressed data, compressed metadata, compressed fragments,
compressed xattrs, compressed ids
duplicates are removed
Filesystem size 213929.25 Kbytes (208.92 Mbytes)
39.06% of uncompressed filesystem size (547735.68 Kbytes)
Inode table size 258823 bytes (252.76 Kbytes)
18.50% of uncompressed inode table size (1399362 bytes)
Directory table size 284870 bytes (278.19 Kbytes)
40.14% of uncompressed directory table size (709748 bytes)
Xattr table size 1873 bytes (1.83 Kbytes)
14.46% of uncompressed xattr table size (12949 bytes)
…
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on /tmp/squashfs, block size 131072.
…
Time (mean ± σ): 52.991 s ± 1.483 s [User: 0.010 s, System: 0.009 s]
Range (min … max): 51.732 s … 54.626 s 3 runs
$ du /tmp/squashfs
213932 /tmp/squashfs
SquashFS with increased block size -comp zstd -Xcompression-level 22 -b 1048576
$ hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/squashfs || true' \
--show-output 'sudo mksquashfs /mnt/testing /tmp/squashfs -comp zstd -Xcompression-level 22 -b 1048576'
…
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on /tmp/squashfs, block size 1048576.
Exportable Squashfs 4.0 filesystem, zstd compressed, data block size 1048576
compressed data, compressed metadata, compressed fragments,
compressed xattrs, compressed ids
duplicates are removed
Filesystem size 199328.03 Kbytes (194.66 Mbytes)
36.39% of uncompressed filesystem size (547705.80 Kbytes)
Inode table size 251168 bytes (245.28 Kbytes)
18.10% of uncompressed inode table size (1387699 bytes)
Directory table size 284914 bytes (278.24 Kbytes)
40.11% of uncompressed directory table size (710264 bytes)
Xattr table size 1873 bytes (1.83 Kbytes)
14.46% of uncompressed xattr table size (12949 bytes)
…
Time (mean ± σ): 50.122 s ± 0.871 s [User: 0.010 s, System: 0.009 s]
Range (min … max): 49.586 s … 51.127 s 3 runs
…
$ du /tmp/squashfs
199332 /tmp/squashfs
NOTE: ZSTD does not use latest ––ultra option
DwarFS --level 5
$ hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/dwarfs || true' \
--show-output 'sudo mkdwarfs -i /mnt/testing -o /tmp/dwarfs --compress-level 5'
…
I 10:16:38.169327 scanning /mnt/testing
I 10:16:39.434663 assigning directory and link inodes...
I 10:16:39.439035 waiting for background scanners...
I 10:16:39.534985 scanning CPU time: 3.844s
I 10:16:39.535066 finalizing file inodes...
I 10:16:39.545341 saved 12.66 MiB / 532.7 MiB in 1366/21195 duplicate files
I 10:16:39.545406 assigning device inodes...
I 10:16:39.546073 assigning pipe/socket inodes...
I 10:16:39.546476 building metadata...
I 10:16:39.546520 building blocks...
I 10:16:39.546535 saving names and symlinks...
I 10:16:39.546606 using a 4 KiB window at 1 KiB steps for segment analysis
I 10:16:39.546670 bloom filter size: 16 KiB
I 10:16:39.546831 ordering 17417 inodes by similarity...
I 10:16:39.556843 17417 inodes ordered [9.945ms, 9.895ms CPU]
I 10:16:39.556907 assigning file inodes...
I 10:16:39.560047 waiting for segmenting/blockifying to finish...
I 10:16:39.563787 updating name and link indices...
I 10:16:46.209654 segmenting/blockifying CPU time: 5.341s
I 10:16:46.209804 bloom filter reject rate: 96.010% (TPR=0.014%, lookups=477674412)
I 10:16:46.210132 segmentation matches: good=1906, bad=694, total=3253
I 10:16:46.210386 segmentation collisions: L1=0.023%, L2=0.006% [512991 hashes]
I 10:16:46.210661 saving chunks...
I 10:16:46.217260 saving directories...
I 10:16:46.223004 saving shared files table...
I 10:16:46.242544 saving names table... [18.18ms]
I 10:16:46.247582 saving symlinks table... [4.608ms]
I 10:16:46.284911 waiting for compression to finish...
I 10:17:22.054127 compressed 532.7 MiB to 184.4 MiB (ratio=0.346111)
I 10:17:22.061327 compression CPU time: 295.7s
I 10:17:22.061531 filesystem created without errors [43.89s]
…
3629 dirs, 3124/2412 soft/hard links, 21195/21195 files, 0 other
original size: 532.7 MiB, dedupe: 12.66 MiB (1366 files), segment: 18.78 MiB
filesystem: 501.2 MiB in 63 blocks (20308 chunks, 17417/17417 inodes)
compressed filesystem: 63 blocks/184.4 MiB written
…
Time (mean ± σ): 42.884 s ± 1.007 s [User: 0.010 s, System: 0.014 s]
Range (min … max): 42.046 s … 44.000 s 3 runs
$ du /tmp/dwarfs
188816 /tmp/dwarfs
SquashFS with increased block size -comp zstd -Xcompression-level 19 -b 1048576
$ hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/squashfs || true' \
--show-output 'sudo mksquashfs /mnt/testing /tmp/squashfs -comp zstd -Xcompression-level 19'
…
Exportable Squashfs 4.0 filesystem, zstd compressed, data block size 131072
compressed data, compressed metadata, compressed fragments,
compressed xattrs, compressed ids
duplicates are removed
Filesystem size 213941.82 Kbytes (208.93 Mbytes)
39.06% of uncompressed filesystem size (547735.68 Kbytes)
Inode table size 258854 bytes (252.79 Kbytes)
18.50% of uncompressed inode table size (1399362 bytes)
Directory table size 284852 bytes (278.18 Kbytes)
40.13% of uncompressed directory table size (709748 bytes)
Xattr table size 1873 bytes (1.83 Kbytes)
14.46% of uncompressed xattr table size (12949 bytes)
…
Time (mean ± σ): 41.936 s ± 0.630 s [User: 0.011 s, System: 0.006 s]
Range (min … max): 41.293 s … 42.552 s 3 runs
$ du /tmp/squashfs
213944 /tmp/squashfs
DwarFS --compress-level 2
$ hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/dwarfs || true' \
--show-output 'sudo mkdwarfs -i /mnt/testing -o /tmp/dwarfs --compress-level 2'
…
I 10:29:53.953142 scanning /mnt/testing
I 10:29:55.045638 assigning directory and link inodes...
I 10:29:55.049707 waiting for background scanners...
I 10:29:55.051185 scanning CPU time: 1.123s
I 10:29:55.051229 finalizing file inodes...
I 10:29:55.061802 saved 12.66 MiB / 532.7 MiB in 1366/21195 duplicate files
I 10:29:55.061887 assigning device inodes...
I 10:29:55.062932 assigning pipe/socket inodes...
I 10:29:55.064317 building metadata...
I 10:29:55.064507 building blocks...
I 10:29:55.065977 ordering 17417 inodes by path name...
I 10:29:55.064525 saving names and symlinks...
I 10:29:55.089163 17417 inodes ordered [23.13ms, 19.32ms CPU]
I 10:29:55.089394 assigning file inodes...
I 10:29:55.093949 waiting for segmenting/blockifying to finish...
I 10:29:55.105262 updating name and link indices...
I 10:29:55.927997 segmenting/blockifying CPU time: 676.1ms
I 10:29:55.928340 saving chunks...
I 10:29:55.933716 saving directories...
I 10:29:55.940146 saving shared files table...
I 10:29:55.955761 saving names table... [14.07ms]
I 10:29:55.959295 saving symlinks table... [3.269ms]
I 10:29:55.985698 waiting for compression to finish...
I 10:29:58.613134 compressed 532.7 MiB to 247.3 MiB (ratio=0.464114)
I 10:29:58.624627 compression CPU time: 25.04s
I 10:29:58.624699 filesystem created without errors [4.672s]
…
waiting for block compression to finish
3629 dirs, 3124/2412 soft/hard links, 21195/21195 files, 0 other
original size: 532.7 MiB, dedupe: 12.66 MiB (1366 files), segment: 0 B
filesystem: 520 MiB in 520 blocks (17935 chunks, 17417/17417 inodes)
compressed filesystem: 520 blocks/247.3 MiB written
…
Time (mean ± σ): 4.674 s ± 0.051 s [User: 0.005 s, System: 0.006 s]
Range (min … max): 4.628 s … 4.729 s 3 runs
$ du /tmp/dwarfs
253192 /tmp/dwarfs
SquashFS -comp zstd -Xcompression-level 5 -b 1048576
hyperfine -r 3 \
--prepare 'sync;echo 3 | sudo tee /proc/sys/vm/drop_caches;sudo rm /tmp/squashfs || true' \
--show-output 'sudo mksquashfs /mnt/testing /tmp/squashfs -comp zstd -Xcompression-level 19'
…
Time (mean ± σ): 3.870 s ± 0.040 s [User: 0.007 s, System: 0.003 s]
Range (min … max): 3.828 s … 3.908 s 3 runs
…
$ du /tmp/squashfs
218100 /tmp/squashfs