When you're trying to clean up your filesystems and reclaim some space, one of the first things you'll want to do is to confirm the largest directories and individual files you have. This can be easily done using two Unix commands: find command and du command.
Find files larger than a certain size
It's very simply to find files which are larger than a specified size. The find command accepts a size parameter, and you can specify the limits for file sizes in your command line.
This example finds all the files under /etc directory which are larger than 100k:
root@ubuntu# find /etc -size +100k /etc/ssh/moduli /etc/ssl/certs/ca-certificates.cr /etc/bash_completio
If we look at their sizes, they really are above 100k:
root@ubuntu# ls -l /etc/ssh/moduli /etc/ssl/certs/ca-certificates.crt /etc/bash_completio -rw-r--r-- 1 root root 215938 Apr 10 2007 /etc/bash_completion -rw-r--r-- 1 root root 132777 Feb 19 2007 /etc/ssh/moduli -rw-r--r-- 1 root root 149568 Sep 7 2007 /etc/ssl/certs/ca-certificates.crt
Find files within specified size limits
The real beauty of using find command is that you can specify both the lower and the upper file size limit in one command line. Working off the previous example, we can limit the search to find only files with the size of 100k-150k, quite easily:
root@ubuntu# find /etc -size +100k -size -150k /etc/ssl/certs/ca-certificates.crt /etc/bash_completion
As you can see from the syntax, the size specification can contain a sign – plus or minus indicates whether you're looking for a file with the size above or under a given figure.
Show directory sizes using du
du command takes a little while to run, depending on what directory you pass it as a parameter, but then prints you a list of all the subdirectories along with their sizes. Most common usage is shown below, -s parameter makes the command report a summary of disk usage stats for only the specified directories matching the /usr/* mask (and not their subdirectories), and -k specifies that we want to see the results in kilobytes:
root@ubuntu# du -sk /usr/* 90824 /usr/bin 4 /usr/games 23644 /usr/include 404196 /usr/lib 0 /usr/lib64 116 /usr/local 22020 /usr/sbin 309516 /usr/share 301600 /usr/src
In most Linux systems, this command had been updated to support a -h parameter, which makes sizes even easier to interpret:
root@ubuntu# du -sh /usr/* 89M /usr/bin 4.0K /usr/games 24M /usr/include 395M /usr/lib 0 /usr/lib64 116K /usr/local 22M /usr/sbin 303M /usr/share 295M /usr/src
Sorting directory sizes
Now, the previous example would get a lot more useful if you sort the directories by their size. The only problem is that sort -n (numerical sorting) would sort by numbers but ignore the human-readable element (M for megabytes, K for kilobytes, G for gigabytes) thus giving you a complete mess:
root@ubuntu# du -sh /usr/* | sort -n 0 /usr/lib 644.0K /usr/games 22M /usr/sbin 24M /usr/include 89M /usr/bin 116K /usr/local 295M /usr/src 303M /usr/share 395M /usr/lib
So what do we to? Luckily for us, Linux implementation of sort supports -h option which does exactly the kind of sorting we needed:
root@ubuntu# du -sh /usr/* | sort -h 0 /usr/lib 644.0K /usr/games 116K /usr/local 22M /usr/sbin 24M /usr/include 89M /usr/bin 295M /usr/src 303M /usr/share 395M /usr/lib
See Also
Thanks a lot, this is the most useful and simple tuition I could find. I went through many UNIX handbooks in the past but it is always cool to refresh.
Glad to have helped you, Klaudio!
Thanks for a great tutorial! I am looking a solution how to display directory sizes in the same way: files can take a little space each other, but if there are 1 million small files in a folder? My question is how to list big folders?
Hi,
Your best bet would be to do something like du -sk /* for starters, and then narrow it down by du -sk /usr/* or something like that.
Thank for your reply! The problem is that I have too many small temporary files in the folder I'm searching and du hangs while executing such a command.
How do you search for a directory that could be in any other directory?
For example I want to find all "backup" directories but am not sure in which directories they all got saved.
Your tuition helped me on my EEEPC, a modest monster machine that just needed some extra UNIX scripts to perform fine!
A pleasure to be of some help!
Thanks very much. I couldn't remember what the du command was, and spent ages trying to find it again. Cheers 🙂
Hi Ben,
Glad to have helped!
Gleb
Please tell me how can i find the largest directory through a bash script who will sum the file in the folder(only the final directories). Thank you
Andrei, du -sk does exactly that, but you need to then sort the output so that it finds the largest.
thx,
i use this to find large files on my webserver
want to see the sizes directly and dont want to see files in the folder /download/, so i use:
find /var/www/ -type f -name "*" -size +100M -exec du -h '{}' \;|grep -v /download/
Find the sizes of only directories sorted by size:
find . -type d -print0 | xargs -0 -n1 du -sk | sort -rn > dir-sizes.txt&
If you just want the 50 or so largest add '| head -n50' before piping it to the output file. Sizes are in KB for ez sort and reading.
What a nice post. I really love reading these types or articles. I can?t wait to see what others have to say.
du -h | sort -n -r | head -n 20
For finding the directory size I use a small script.
for i in `ls -F | grep '/$'` ; do
du -sh `echo "$i"`
done
I am running this cleanup script to keep my Solaris drop box well, cleaned up.
#!/bin/ksh
#
userdir=/clocal/ftp/user
for i in `ls ${userdir}`
do
find ${userdir}/${i}/incoming -mtime +2 -type f -ls
find ${userdir}/${i}/incoming -mtime +2 -type f -exec rm {} \;
find ${userdir}/${i}/incoming -mtime +2 -type f -size +200557600c -ls
find ${userdir}/${i}/incoming -mtime +2 -type f -size +200557600c -exec rm {} \;
done
But, if I think it misses a lot of filenames with spaces in them (People t4ransferring DOS files). Any suggestions to catch those files as well?
Mike
Sorry. It's the mtime that is giving the issues. It is Nov 3 and it is now deleting files from Oct 31. I thought that mtime +2 would have deleted the files on Nov 1 or 2 at the latest?
Here is another cool way to find size of directory and subdirectory in UNIX.