Using dd command to quickly copy something or generate a file full of random or zero bytes is a really old trick. And almost every sysadmin knows how to use this command to erase disks:
dd if=/dev/zero of=/dev/sdX
But while this worked really well in the past, really large capacity of hard disks today means you need a trick or two if you want to track progress of running a dd against a 2TB or 3TB disk.
As part of upgrading NAS server in my home office, I wanted to re-sell some of the older disks, which means even disks with encrypted filesystems needed to be erased.
DISCLAIMER
use this article and examples at your own risk. It’s very easy to confuse disk names and accidentally erase wrong disk (including boot disk or the only disk you have on your server), so double-check disk device names and their sizes and mounted filesystems before using root superpowers for running any commands shown.
Please do your own research as I will accept no responsibility for any data corruption or data loss caused by using dd examples below.
General approach to erasing disks with dd
Best results are usually achieved when you zero the disk – write zeros to all the data blocks available on the storage device.
IMPORTANT: Test your approach and learn dd command basics on a regular file in your home directory and as regular user. For me, it can be /home/greys/file1:
mint ~ $ dd if=/dev/zero of=/home/greys/file1 bs=64M count=10
10+0 records in
10+0 records out
671088640 bytes (671 MB) copied, 0.484989 s, 1.4 GB/s
Once you are comfortable enough with dd, you can identify which disk you need and then erase it as shown below.
Here’s how one would erase the /dev/sdf disk:
mint ~ # dd if=/dev/zero of=/dev/sdf bs=64M
188+0 records in
188+0 records out
12616466432 bytes (13 GB) copied, 81.2798 s, 155 MB/s
199+0 records in
199+0 records out
13354663936 bytes (13 GB) copied, 86.0636 s, 155 MB/s
211+0 records in
211+0 records out
14159970304 bytes (14 GB) copied, 91.2245 s, 155 MB/s
553+0 records in
553+0 records out
37111201792 bytes (37 GB) copied, 239.566 s, 155 MB/s
1212+0 records in
1212+0 records out
81335943168 bytes (81 GB) copied, 525.101 s, 155 MB/s
2201+0 records in
2201+0 records out
147706609664 bytes (148 GB) copied, 958.028 s, 154 MB/s
3537+0 records in
3537+0 records out
237364051968 bytes (237 GB) copied, 1546.11 s, 154 MB/s
5219+0 records in
5219+0 records out
350241161216 bytes (350 GB) copied, 2296.43 s, 153 MB/s
5228+0 records in
But for even larger disks, it’s probably best to use pv command to show progress.
First, install the pv command:
$ sudo yum install pv
Now, split dd command in two using Unix pipe. What we do if use one of the default behaviours of most commands in Unix: if there’s no parameter specifying input or input file, the command checks standard input stream. Likewise, if there’s no output file specified, a standard output stream is used.
Unix pipe, as you may know already, takes standard output from one command and forwards it (pipes into) the standard input of the next command.
Thus our initial example of:
mint ~ $ dd if=/dev/zero of=/home/greys/file1 bs=64M count=10
can be rewritten with pipe and two dd commands like this:
mint ~ $ dd if=/dev/zero | dd of=/home/greys/file1 bs=64M count=10
dd: warning: partial read (66048 bytes); suggest iflag=fullblock
0+10 records in
0+10 records out
101376 bytes (101 kB) copied, 0.00823315 s, 12.3 MB/s
Excellent! Now, we simply insert pv between dd commands. This is how the command will look:
mint ~ $ dd if=/dev/zero | pv | dd of=/home/greys/file1 bs=64M count=10
dd: warning: partial read (2048 bytes); suggest iflag=fullblock
35.5kiB 0:00:00 [79.9MiB/s] [ <=> ]
0+10 records in
0+10 records out
36352 bytes (36 kB) copied, 0.00952251 s, 3.8 MB/s
If you compare the outputs, pv provides this transfer throughput in megabytes per second and a progress indicator (<=>) that will move simply to confirm that data transfer is still in progress.
Erase disk with dd and pv commands
Armed with the dd and pv knowledge, let’s start erasing the /dev/sdf disk.
VERY IMPORTANT: please pause and double-check that you know which disk you want to erase. I’ve specifically given you an example of /dev/sdf disk which won’t even be found unless you have multi-disk system, but please check again and again so that you don’t wipe the wrong disk.
mint ~ # dd if=/dev/zero | pv | dd of=/dev/sdf bs=64M
0+335852 records in6MB/s] [ <=> ]
0+335851 records out
5909268992 bytes (5.9 GB) copied, 134.987 s, 43.8 MB/s
0+343712 records in7MB/s] [ <=> ]
0+343712 records out
6043738624 bytes (6.0 GB) copied, 137.992 s, 43.8 MB/s
0+345803 records in5MB/s] [ <=> ]
0+345803 records out
Now, a really secure way to erase disk would be to fill it with random numbers, but using /dev/urandom (or even slower /dev/random) instead of /dev/zero will take so much longer than it’s not going to be feasible to secure erase 1TB disks this way.
So the next best thing is to overwrite each data block on the disk at least twice, preferably with different data. Assuming we let the previous command complete, got the disk filled with zeros.
This little trick below uses tr command to translate zeros into 1s. Specifically, this changes all the bits that were 0 into 1s. So you’ll end up writing a lot of charactes with hex code $FF instead of $00 into the resulting file. \377 is the octal value for hex number $FF (tr command doesn’t take hex values as parameters).
Our final command is this:
mint ~ # tr '\0' '\377' < /dev/zero | dd bs=64M of=/dev/sdf
0+93225 records in
0+93224 records out
768778240 bytes (769 MB) copied, 8.70987 s, 88.3 MB/s
0+111582 records in
0+111581 records out
919052288 bytes (919 MB) copied, 10.5316 s, 87.3 MB/s
0+120755 records in
0+120754 records out
994623488 bytes (995 MB) copied, 11.4484 s, 86.9 MB/s
0+126488 records in
0+126487 records out
1042391040 bytes (1.0 GB) copied, 12.0236 s, 86.7 MB/s
0+9158660 records in
0+9158659 records out
75524096000 bytes (76 GB) copied, 883.944 s, 85.4 MB/s
0+33863403 records in
0+33863403 records out
280563752960 bytes (281 GB) copied, 3354.46 s, 83.6 MB/s
0+35904256 records in
0+35904256 records out
297578594304 bytes (298 GB) copied, 3576.74 s, 83.2 MB/s
As you can see, the throughput is almost twice lower because of the translation, but the result is all the bits on the disk will be definitely overwritten now.
See Also
