How To Resume wget Downloads

Resume wget download

I’m downloading CentOS 8 Stream as we speak, and it’s a large enough ISO file – standard 8GB DVD image. I stopped download because I wanted to restart it in a tmux session, and realised that it’s a good opportunity to explain how this works.

How To Start Wget Download

Assuming you go to the CentOS Stream Download page and pick an image to download, you’ll end up with a download URL like this:

http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso

Using wget, here’s how you can start downloading it:

greys@redhat:/dist/iso $ wget http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:35:36--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 8572108800 (8.0G) [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’
CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 100.83M  1.77MB/s    eta 73m 43s
^C

At any time you can press Ctlr+C to stop the process. A file with unfinished download will be left in the current directory:

greys@redhat:/dist/iso $ ls -ald *iso
-rw-rw-r--. 1 greys 118453636 Sep 30 14:37 CentOS-Stream-x86_64-dvd1.iso

And if you just re-run the same wget command, your download will actually restart:

greys@redhat:/dist/iso $ wget http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:44:08--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 8572108800 (8.0G) [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso.1’
CentOS-Stream-x86_64-dvd1.iso.1  0%[+      ]  24.88M   775KB/s    eta 2h 13m
^C

If you notice though, the download restarts with a different filename: CentOS-Stream-x86_64-dvd1.iso.1 in my case. So wget kept your previous download attempt and started a new one:

greys@redhat:/dist/iso $ ls -ald iso
-rw-rw-r--. 1 greys 118453636 Sep 30 14:37 CentOS-Stream-x86_64-dvd1.iso
-rw-rw-r--. 1 greys  26088163 Sep 30 14:44 CentOS-Stream-x86_64-dvd1.iso.1

How To Resume wget Download

Assuming we want to resume download, you need to use the wget -c option:

greys@redhat:/dist/iso $ wget -c http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:45:27--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 8572108800 (8.0G), 8453655164 (7.9G) remaining [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’
CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 120.68M  1.49MB/s    eta 91m 20s
^C

So it picked up the original 100MB or so downloaded file and resumed downloading from 100MB+ location.

If we stop this with Ctlr+C again, we’ll see the following:

greys@redhat:/dist/iso $ ls -ald iso
-rw-rw-r--. 1 greys 126668812 Sep 30 14:45 CentOS-Stream-x86_64-dvd1.iso
-rw-rw-r--. 1 greys  26088163 Sep 30 14:44 CentOS-Stream-x86_64-dvd1.iso.1
greys@redhat:/dist/iso $ du -sh iso
121M    CentOS-Stream-x86_64-dvd1.iso
25M    CentOS-Stream-x86_64-dvd1.iso.1

So yes, it’s clearly the first file that is slightly larger down – meaning if we continue downloading, it will start from about 126MB location:

greys@redhat:/dist/iso $ wget -c http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
--2019-09-30 14:47:08--  http://ftp.heanet.ie/pub/centos/8-stream/isos/x86_64/CentOS-Stream-x86_64-dvd1.iso
Resolving ftp.heanet.ie (ftp.heanet.ie)… 87.44.34.235
Connecting to ftp.heanet.ie (ftp.heanet.ie)|87.44.34.235|:80… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 8572108800 (8.0G), 8445439988 (7.9G) remaining [application/octet-stream]
Saving to: ‘CentOS-Stream-x86_64-dvd1.iso’

CentOS-Stream-x86_64-dvd1.iso  1%[+      ] 127.76M  1.63MB/s    eta 98m 41s

And that’s how you resume wget downloads and continue when previous wget download left off.

See Also