Thursday, August 14, 2008

Tar Gzip On-The-Fly

I typically create tarballs by doing "tar cvf file.tar somedir" and then gzipping the resultant tarball. Sometimes there's not enough space on disk to have an intermediate uncompressed file sitting around. As an example, I just made a backup of 48 gigs of MySQL data yesterday. One handy way to avoid storing the uncompressed file on disk is to gzip on the fly.

tar cvf - somedir | gzip -c > somedir.tar.gz

In writing this post, I just noticed that you can also just give tar the -z flag to accomplish the same thing; however, explicitly piping to gzip allows you to specify options to gzip such as compression level.

4 comments:

Dr Foobario said...

the -z flag is pretty good when extracting .tgz/.tar.gz files... i.e. "tar xfzv foo.tgz", that and "tar cfzv foo bar" are the most common tar commands i use.

also, -j is for bzip

Anonymous said...

-z and -y only works on GNU's and BSD's tar implementation. If you stuck with some older UNIX (like Solaris 9 and earlier, and couldn't install GNU/BSD tar) you have to use the pipe method for on-the-fly compression/decompression.



Just my 2c

Travis Whitton said...

Good point. It's too easy to forget that Linux and FreeBSD aren't the only *nix operatings systems on the planet these days.

folti said...

Minor nitpick: Busybox's tar implementation may or may not supports the -z and -j options. It depends how it was configured. This could be a problem on an embedded system.

Also if you have free space problems on a machine, you could pipe tar's output into an ssh session to save the output on another machine:

tar cf[z] - importantstuff | ssh [user@]host 'cat > /path/to/backup.tar[.gz]