split the files into smaler files (1GB e.g.)

i would create a new screen session on server and run the command in there (so you can logoff/close terminal and it will continue)

screen -S split # create new screen session with name split

# split the files into 1GB chunks
gzip -c LARGEBACKUP.file | split --suffix-length=5 -b 1073741824 - LARGEBACKUP.file.split.gz

# generate md5sum to check if transfered files were ok (will also be transfered)
md5sum *.file.split.gz > md5sums.txt

# you can have both in one go:
gzip -c LARGEBACKUP.file | split --suffix-length=5 -b 1073741824 - LARGEBACKUP.file.split.gz; md5sum *.file.split.gz > md5sums.txt;

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!