A Weird Imagination

Transferring many small files

Posted in

The problem

Transferring many small files is much slower than you would expect given their total size.

The solution

tar c directory | pv -abrt | ssh target 'cd destination; tar x'

or

cd destination; ssh source tar c directory | pv -abrt | tar x

The details

Read more…

1 comment