Tar-ing files across different host using SSH

It's been a while since a tech-blog was written.

Often when we need to migrate a large group file from one host to another we will:

• tar the files under a directory (e.g. /u01/myfiles in a giant tarball (/tmp/myfiles.tgz on the source host (A)
• transfer the tarball from the source host to the target (B)
• extract the tarball on the targe

# on host A
$ tar -zcf /tmp/myfiles.tgz /u01/myfiles
$ scp /tmp/myfiles.tgz john@B.host.com:/tmp/myfiles.tgz
#   unless you have SSH keys setup, you will be prompted for a password on that last command 
#   on host B
$ cd /; tar -zxf /tmp/myfiles.tgz

While this definitely works, it requires three separate commands, and requires additional space to create the tgz file on both hosts. Many people don't realize this, but you can pipe STDIO across SSH and we can use this to tar directly across hosts:

# syntax for this command is: tar -cf - [source files directory] | ssh username@target_host "(cd [target directory] ; tar -xf -)"
$ tar -cf - /u01/myfiles | ssh john@B.host.com "(cd / ; tar -xf -)"
# by doing this we dont ever create the tgz file on disk, which could be vital when your mount points are getting full
# in addition this command would get you A LOT of "street credibility" from the hackers!