what for?

various reasons. for example maybe the user wants to test download bandwidth by downloading a large file with random data and measure the bandwidth speed.

lsb_release -a; # tested on
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 12 (bookworm)
Release:	12
Codename:	bookworm

# create non-empty small file for comparison
echo "this is a small text file that is not empty" > small.text.testfile
# verify that there is data
xxd -b small.text.testfile |less

# fallocate.man.txt: generate large file fast with fallocate
# preallocation is done quickly by allocating blocks and marking them as uninitialized
# requiring no IO to the data blocks
# This is much faster than creating a file by filling it with zeroes
fallocate -l 1G 1GByte.empty.testfile

# verify that there is no data (zero) (but sometimes there is X-D)
# manpage: xxd.man.txt
xxd -b 1GByte.empty.testfile |less

# try to compress it with zip
time tar fcvz 1GByte.empty.testfile.tar.gz 1GByte.empty.testfile

# it compresses well, because it's (mostly) full of zeros
du -hs 1GByte.empty.testfile*
1.1G	1GByte.empty.testfile
1020K	1GByte.empty.testfile.tar.gz

# generate empty (zero) but 1GByte large file with dd (compresses well)
time dd if=/dev/zero of=1GByte.zero.testfile bs=64M count=16 iflag=fullblock
# verify it's empty (full of zeros)
xxd -b 1GByte.zero.testfile |less

# try to compress it
time tar fcvz 1GByte.zero.testfile.tar.gz 1GByte.zero.testfile
du -hs 1GByte.zero*
# same same but different
1.1G 1GByte.zero.testfile
1020K 1GByte.zero.testfile.tar.gz

# generate 1GByte test file that contains random data (can not be compressed well)
time dd if=/dev/urandom of=1GByte.random.testfile bs=64M count=16 iflag=fullblock

# verify it's random data
xxd -b 1GByte.random.testfile |less

# try to compress it
time tar fcvz 1GByte.random.testfile.tar.gz 1GByte.random.testfile

# checkout what it did (how much zip was able to compress)
du -hs 1GByte.random.testfile*
1.1G 1GByte.random.testfile
1.1G 1GByte.random.testfile.tar.gz <- basically unable to compress it at all (which is good for bandwidth testing)

offtopic:

# in case the user want's zero-out all marked as free blocks with actual zeros (so image of harddisk compresses well)
# fill the disk with a large empty file to zero out empty space (it will stop automatically and then remove the file)
time dd if=/dev/zero of=./zero_out_space bs=1M; rm -rf ./zero_out_space

related links:

https://www.cyberciti.biz/faq/howto-create-lage-files-with-dd-command/

https://superuser.com/questions/470949/how-do-i-create-a-1gb-random-file-in-linux

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!
admin