Visualizing progression of file operations using pv (Pipe Viewer)

Compressing large files from command line can be time consuming, and unless using a file manager like Midnight Commander, there is usually no way to know in advance how long the operation is going to last.

Enter pv (Pipe Viewer), a neat CLI tool allowing to monitor the progress of data through a pipeline, by displaying a progress bar and indicating both processing speed and estimated time until completion.

Here are a few examples showing how to use the tool:

Compressing using gzip:

pv access.log | gzip > access.log.gz
482MB 0:00:26 [18.3MB/s] [==================================>] 100%

Compressing using bzip2:

pv access.log | bzip2 > access.log.bz2
482MB 0:04:45 [1.69MB/s] [==================================>] 100%

Decompressing a gzip archive:

pv access.log.gz | gunzip > access.log
48.5MB 0:00:08 [5.85MB/s] [==================================>] 100%

Decompressing a bzip2 archive:

pv access.log.bz2 | bunzip2 > access.log
25.2MB 0:00:42 [ 607kB/s] [==================================>] 100%

One-liner to compress the current directory:

tar cpf - . | pv -s $(du -sb . | cut -f1) | gzip > archive.tar.gz
1.48GB 0:01:33 [16.3MB/s] [==================================>] 100%

This one is a little more complicated and requires an explanation: as pv cannot know in advance the amount of data to be transferred, we have to specify it beforehand using the du command.

There is so much more pv can do, for example limiting the transfer rate of a pipe (using the -L output modifier). Together with the -q display switch (which turns the progress bar off), it can be used to render text animations.

Playing a VT100 animation at 9600 bytes per second (a spinning and moving globe):

curl -s http://artscene.textfiles.com/vt100/movglobe.vt | pv -q -L 9600