GNU Linux – what filesystem can store more bytes? – storage efficiency harddisk space utilization – dd benchmark in virtual machine

07.Jul.2017

Administration / Server, benchmark, filesystem / filesystems, GNU-Linux, Virtualization / KVM / VirtualBox / xenserver

who can store more bytes?

i created 10GByte partitions for each filesystem.

df -TH
Filesystem     Type      Size  Used Avail Use% Mounted on
udev           devtmpfs  505M     0  505M   0% /dev
tmpfs          tmpfs     104M  5.1M   99M   5% /run
/dev/sda1      ext4      133G   34G   93G  27% /
tmpfs          tmpfs     518M     0  518M   0% /dev/shm
tmpfs          tmpfs     5.3M     0  5.3M   0% /run/lock
tmpfs          tmpfs     518M     0  518M   0% /sys/fs/cgroup
/dev/sdb5      ext4       11G   38M   10G   1% /mnt/ext4
/dev/sdb6      ext3       11G   24M   10G   1% /mnt/ext3
tmpfs          tmpfs     104M   17k  104M   1% /run/user/1000
/dev/sdb1      xfs        11G   45M   11G   1% /mnt/xfs1
/dev/sdb2      jfs        11G  1.5M   11G   1% /mnt/jfs1
/dev/sdb7      btrfs      11G   18M  8.6G   1% /mnt/btrfs
/dev/sdb8      f2fs       10G  9.6G     0 100% /mnt/f2fs

not sure if dd is a good testing method…

yo can clearly see the harddisk RAM Caching on multiple levels 1. Linux 2. Hyper-V (?) 3. Windows

dd if=/dev/zero of=/mnt/f2fs/16GBybe.testfile bs=64M count=256 iflag=fullblock;
153+0 records in
152+0 records out
10200547328 bytes (10 GB, 9.5 GiB) copied, 7.53305 s, 1.4 GB/s
# 2nd run
10200547328 bytes (10 GB, 9.5 GiB) copied, 7.61236 s, 1.3 GB/s
# 3rd run
10200547328 bytes (10 GB, 9.5 GiB) copied, 7.07396 s, 1.4 GB/s

dd if=/dev/zero of=/mnt/btrfs/16GBybe.testfile bs=64M count=256 iflag=fullblock;
128+0 records in
127+0 records out
8572108800 bytes (8.6 GB, 8.0 GiB) copied, 6.53744 s, 1.3 GB/s
# 2nd run
8572108800 bytes (8.6 GB, 8.0 GiB) copied, 6.50127 s, 1.3 GB/s
# 3rd run
8572108800 bytes (8.6 GB, 8.0 GiB) copied, 6.6249 s, 1.3 GB/s

dd if=/dev/zero of=/mnt/xfs1/16GBybe.testfile bs=64M count=256 iflag=fullblock
160+0 records in
159+0 records out
10679549952 bytes (11 GB, 9.9 GiB) copied, 9.55563 s, (deleted)
# 2nd run
10678239232 bytes (11 GB, 9.9 GiB) copied, 10.639 s, 1.0 GB/s
# 3rd run
10681647104 bytes (11 GB, 9.9 GiB) copied, 10.2884 s, 1.0 GB/s

dd if=/dev/zero of=/mnt/jfs1/16GBybe.testfile bs=64M count=256 iflag=fullblock;
160+0 records in
159+0 records out
10693496832 bytes (11 GB, 10 GiB) copied, 14.0012 s, 764 MB/s
# 2nd run
10693496832 bytes (11 GB, 10 GiB) copied, 11.9995 s, 891 MB/s
# 3rd run
10693496832 bytes (11 GB, 10 GiB) copied, 12.1015 s, 884 MB/s

dd if=/dev/zero of=/mnt/ext3/16GBybe.testfile bs=64M count=256 iflag=fullblock;
156+0 records in
155+0 records out
10467930112 bytes (10 GB, 9.7 GiB) copied, 31.5958 s, 331 MB/s
# 2nd run
10467930112 bytes (10 GB, 9.7 GiB) copied, 28.2994 s, 370 MB/s
# 3rd run
10467930112 bytes (10 GB, 9.7 GiB) copied, 27.2546 s, 384 MB/s

dd if=/dev/zero of=/mnt/ext4/16GBybe.testfile bs=64M count=256 iflag=fullblock;
156+0 records in
155+0 records out
10447216640 bytes (10 GB, 9.7 GiB) copied, 26.8287 s, 389 MB/s
# 2nd rerun
10447216640 bytes (10 GB, 9.7 GiB) copied, 24.7539 s, 422 MB/s
# 3nd rerun
10447216640 bytes (10 GB, 9.7 GiB) copied, 23.9356 s, 436 MB/s

# they all fail with "dd: error writing '/mnt/xfs1/16GBybe.testfile': No space left on device" - but this is on purpse.

result:

and the winner is: ext4 with 10447216640 Bytes!

mkfs.ext4 -V
mke2fs 1.43.4 (31-Jan-2017)
Using EXT2FS Library version 1.43.4

ext3 stores -0.19788% less than ext4

mkfs.ext3 -V
mke2fs 1.43.4 (31-Jan-2017)
Using EXT2FS Library version 1.43.4 (actually the ext4 kernel module is handling ext3, it’s the same module)

xfs stores -2.17550% less than ext4

mkfs.xfs -V
mkfs.xfs version 4.9.0

Version: 4.9.0+nmu1

XFS is a high performance journaling filesystem which originated on the SGI IRIX platform.

It is completely multi-threaded, can support large files and large filesystems, extended attributes,
variable block sizes, is extent based, and makes extensive use of Btrees (directories, extents, free space) to aid both performance and scalability.

Refer to the documentation at http://oss.sgi.com/projects/xfs/

jfs stores -2.30309% less than ext4

mkfs.jfs -V
mkfs.jfs version 1.1.15, 04-Mar-2011

IBM’s journaled file system technology, currently used in IBM enterprise servers, is designed for high-throughput server environments, key to running intranet and other high-performance e-business file servers.

f2fs stores -2.36% less than ext4 (more overhead)

apt-cache show f2fs-tools
Version: 1.7.0-1.1
Depends: libf2fs0 (= 1.7.0-1.1), libc6 (>= 2.14), libselinux1 (>= 2.0.65), libuuid1 (>= 2.16)
Description-en: Tools for Flash-Friendly File System
F2FS is a new filesystem for Linux aimed at NAND flash memory-based storage
devices, such as SSD, eMMC, and SD cards. It is based on Log-structured File
System (LFS).

btrfs stores -21.87452% less than ext4

mkfs.btrfs –version
mkfs.btrfs, part of btrfs-progs v4.7.3

Btrfs is a new copy on write filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration.

… interesting.

interesting that with xfs maximum amount of bytes vary – while the amount of storable bytes of the other filesystems stays constant – could this be due to xfs large write cache – which also means – only use it with a USV that properly shuts down your server in case of an power outage.

also interesting that BTRFS seems to be the fastest of them (almost 3x times of ext3) – but at the costs of storage-space (20% less).

liked this article?

only together we can create a truly free world
plz support dwaves to keep it up & running!
(yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
really really hate advertisement
contribute: whenever a solution was found, blog about it for others to find!
talk about, recommend & link to this blog and articles
thanks to all who contribute!

admin