tar gz (single core) -> tar pigz (multi core)
tar might be old, but still a great program for archiving files
to get multi core usage for compression (faster) the user tells tar to use a different compression algorithm that has multi-core support like pigz or xz.
# install pigz apt install pigz # create tar archive using pigz as "compressor" time tar -c --use-compress-program=pigz -f file.tar.gz directory # time meassures the time it needed using all cores # lenovo t440 i5 and ssd # took 6min to compress a 29 GByte home directory # size of home directory was compressed by -70% to 8.5 GByte # multi threaded decompress is not implemented # decompression seems to use way less CPU resources than compression # (have 25MByte/sec during compression and over 100MByte/sec during decompression) # time to compress without pigz: # real 13m13.490s # time to compression with pigz: # real 5m57.117s # ;) nice # time to decompress without pigz (not supported): # real 3min 37sec tar fxvz file.tar.gz
manpage: pigz.man.txt
want to safe disk space? have got the time? xz!
xz takes way more cpu time but compresses way more than gzip or pigz
# if user wants to use more than 4 cpu threads/cores just increase the number # compress "folder" into archive.tar.xz XZ_DEFAULTS="--threads=4"; export XZ_DEFAULTS; tar fcvJ /path/to/backup/archive.tar.xz /compress/this/folder # de compress archive.tar.xz into current directory XZ_DEFAULTS="--threads=4"; export XZ_DEFAULTS; tar fxvJ /path/to/backup/archive.tar.xz
manpage: xz.man.txt
real world example:
4x Core2Quad Q6600@2.4GHz managed to compress 62GBytes (random also binary data) into 26GBytes (41% of original) in 152min (2.53 hours).
liked this article?
- only together we can create a truly free world
- plz support dwaves to keep it up & running!
- (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
- really really hate advertisement
- contribute: whenever a solution was found, blog about it for others to find!
- talk about, recommend & link to this blog and articles
- thanks to all who contribute!
