Resuming an scp file transfer
Thursday, August 20, 2009
Sometimes you have to transfer data over dodgy links. And when you're transferring a large file this can hurt when it fails. So as explained elsewhere rsync can safe the day:
$ scp host:remote_file local_file
# now interrupt and resume with
$ rsync --partial --progress --rsh=ssh host:remote_file local_file
You need rsync on the remote server for this however. But usually that's not too much of a hurdle.
1 comments:
Unknown said...
Why not just use rsync over ssh in the first place?
And, FWIW, rsync has a -P option which is an alias for --partial --progress, so you can just use `rsync -P -essh host:remote_file local_file` (-e is equivalent to --rsh=) for all your file transfers.
Furthermore, ideally (since you probably don't want to be using rsync over a clear connection anyway), you can set `RSYNC_RSH` to `ssh`, and rsync will use `ssh` by default.
So with `RSYNC_RSH` set, you can just go with `rsync -P host:remote local`, which is barely longer than `scp host:remote local`, and will resume on relaunch if the connection crashes.
Also, if your transfer is BW-bound (rather than CPU) don't forget the `-z` option (= `--compress`)
New comments are not allowed.