E pur si muove

Resuming an scp file transfer

Thursday, August 20, 2009

Sometimes you have to transfer data over dodgy links. And when you're transferring a large file this can hurt when it fails. So as explained elsewhere rsync can safe the day:

$ scp host:remote_file local_file
# now interrupt and resume with
$ rsync --partial --progress --rsh=ssh host:remote_file local_file

You need rsync on the remote server for this however. But usually that's not too much of a hurdle.

Thursday, August 20, 2009 |


Unknown said...

Why not just use rsync over ssh in the first place?

And, FWIW, rsync has a -P option which is an alias for --partial --progress, so you can just use `rsync -P -essh host:remote_file local_file` (-e is equivalent to --rsh=) for all your file transfers.

Furthermore, ideally (since you probably don't want to be using rsync over a clear connection anyway), you can set `RSYNC_RSH` to `ssh`, and rsync will use `ssh` by default.

So with `RSYNC_RSH` set, you can just go with `rsync -P host:remote local`, which is barely longer than `scp host:remote local`, and will resume on relaunch if the connection crashes.

Also, if your transfer is BW-bound (rather than CPU) don't forget the `-z` option (= `--compress`)

New comments are not allowed.

Subscribe to: Post Comments (Atom)