How To Remove Files and Directories with Special Characters

Today is going to be a practical tip. If you’re managing many Unix systems, sooner or later you come across files with special characters – they can’t be deleted with rm command using standard approach and require a bit of trickery to be successfully removed.

Examples of files with special characters

Any language apart from English will probably have special characters in the alphabet, but for the purpose of today’s exercise I’ll give you more standard examples: files starting with dash (-) and hash (#) characters:

ubuntu$ ls -al
-rw-r--r-- 1 greys admin    0 Sep 25 05:50 #try
-rw-r--r-- 1 greys admin    0 Sep 25 05:48 -try

Now, if you try to access these files or remove them, you will get errors:

ubuntu$ cat -try
cat: invalid option -- r
Try `cat --help' for more information.
ubuntu$ rm -try
rm: invalid option -- t
Try `rm ./-try' to remove the file `-try'.
Try `rm --help' for more information.

These errors happen because commands treat file names as command line options because they start with dash (-).

With filenames starting with hash (#), you’ll get a different kind of error: your Unix shell will treat the rest of a filename (and anything that might follow it) as a comment because hashes are used to do it in shell scripts. That’s why your cat command will not show any error but will not finish until you Ctrl+C it:

ubuntu$ cat #try

… and if you try removing such a file, you’ll get a complaint from the rm command about missing command line parameters – because of the hash (#) sign, rm command receives no text as a parameter:

ubuntu$ rm #try
rm: missing operand
Try `rm --help' for more information.

How to remove a file when filename starts with dash (-)

First I’m gonna show you how to make your Unix shell interpret any filename directly instead of trying to analyze it as a set of command line options.

To make command ignore the leading dash (-) in a filename, use the — command line option:

ubuntu$ rm -- -try

As you can see, our file is gone:

ubuntu$ ls -al
-rw-r--r-- 1 greys admin    0 Sep 25 05:50 #try

Using backslash to escape special characters in a filename

Another option we have is to use a backslash (\), which will make shell interpreter ignore the special functionality of a character which immediately follows it. To escape the hash (#) our second file has, we should therefore do the following:

ubuntu$ rm \#try

Interesting to know: bash shell has an auto-completion functionality built in. When you type a filename, just press Tab key to make it auto-complete the name for you. Speaking of special characters in particular, quite a few of them are recognized by auto-completion and get escaped automatically.

So, if you start typing:

ubuntu $ rm #t

… and then press Tab, bash will not only auto-complete the name, but escape the leading hash (#):

ubuntu $ rm \#try

There’s a few more tricks you can use for escaping special characters, but they’re worth a separate post, so stay tuned! Until then, enjoy getting rid of annoying files with special characters in filenames!

See also:




Unix Tutorial Digest – September 21st, 2008

Here are the posts I’ve read and enjoyed this week:
* sshpass – automating SSH logins without passwordless RSA/DSA keys
* Tux Training: Get rid of command line histories with chattr command
* Tux Training: Speeding up your ext3 filesystem If you have any useful articles you’d like to share – just leave links in the comments area. Enjoy!




Easy date calculations in Unix scripts with GNU date

When I was writing a post about using date command to confirm date and time in your Unix scripts, I made a note in my future posts list to cover the date calculations – finding out the date of yesterday or tomorrow, and so on. Today I’ll show you the simplest way to calculate this.

GNU date command advantage

GNU version of the date command, although supporting a common syntax, has one great option: it allows you to specify the desired date with a simple string before reporting it back. What this means is that by default this specified date is assumed to be “now”, but you can use other keywords to shift the result of the date command and thus show what date it was yesterday or a week ago:

Here’s a default date output for the current date and time:

ubuntu$ date
Fri Sep 19 08:06:41 CDT 2008

Now, the parameter for specifying desired date is -d or –date, and if you use it with the “now” or “today” parameter, you’ll get similar output:

ubuntu$ date -d now
Fri Sep 19 08:06:44 CDT 2008
ubuntu$ date -d today
Fri Sep 19 08:06:50 CDT 2008

Showing tomorrow’s date

Similarly, you can get tomorrow’s date:

ubuntu$ date -d tomorrow
Sat Sep 20 08:02:12 CDT 2008

Obviously, if you feel like specifying a format for the date, you can do it:

ubuntu$ date -d tomorrow "+%b %d, %Y"
Sep 20, 2008

Find out yesterday’s date

Again, there’s no rocket science involved in showing yesterday’s date neither:

ubuntu$ date -d yesterday "+%b %d, %Y"
Sep 18, 2008

Show a date few days away

If you’re interested in a certain date a few days or even weeks away, here’s how you can do it:

Example 1: a date 5 days ago

ubuntu$ date -d "-5 days"
Sun Sep 14 08:45:57 CDT 2008

Example 2: a day 2 weeks from now

ubuntu$ date -d "2 weeks"
Fri Oct  3 08:45:08 CDT 2008

Example 3: a day two weeks ago from now

ubuntu$ date -d "-2 weeks"
Fri Sep  5 08:45:11 CDT 2008

Extreme example: a day 50 years ago

If you’re really curious about dates in Unix, you can even make GNU date go back a few years:

ubuntu$ date -d "-50 years"
Fri Sep 19 08:47:51 CDT 1958

That’s it for today, hope you like this little discovery – having mostly worked with Solaris systems most of my career, I didn’t know my Ubuntu had this functionality bonus. Really useful!

See also:




Another way to use math expressions in shell scripts

Today I’d like to expand a bit more on the basic calculations in Unix scripts.

Use Parenthesis to simplify math expressions

In Basic Arithmetic Operations post I’ve shown you how expression evaluation can be used to calculate simple math expressions in your Unix shell:

ubuntu$ START=1
ubuntu$ FINISH=10
ubuntu$ ELAPSED=$(($FINISH - $START))
ubuntu$ echo $ELAPSED
9

Although this approach looks clean enough, there’s a way to simplify it even further if you put everything in parenthesis. In other words, the same result (ELAPSED receiving a correct value of FINISH value minus START value) can be achieved this way:

ubuntu$ ((ELAPSED=FINISH-START))
ubuntu$ echo $ELAPSED
9

It’s a matter of preference, and I used to always do such calculations the former way shown, but now that one of the readers of this blog pointed out the latter way of using it, I think I might change my habits – I actually like this way much better.

See also:




Follow me on Twitter!

It was bound to happen: I’ve joined Twitter! It would be great to hear from you there!

Follow me or leave your Twitter name in comments below so that I can follow you:

http://twitter.com/UnixTutorial

Also, if you happen to know some interesting Twitter feeds for all of us interested in Unix-like systems, leave links to them as well!

See also:




List Installed Packages on Your Ubuntu Linux

If you’re interested in what exactly your Ubuntu system has got installed, there’s a command you can use to list the packages along with their versions and short descriptions.

How packages information is stored in Ubuntu

Essentially being a fork of the Debian Linux, Ubuntu inherited quite a lot of things from it. One of them is the way packages are installed and managed.

dpkg (Debian Package Manager) is a command found in every Ubuntu installation. While managing software packages, it stores all the files it depends upon in a /var/lib/dpkg directory. Most of these files you can look into using basic Unix commands, but there’s really no need because dpkg does it for you.

For example, status of all the installed packages is stored in /var/lib/dpkg/status file.
Here’s how it looks, just to give you an idea:

Package: bash
Essential: yes
Status: install ok installed
Priority: required
Section: shells
Installed-Size: 2012
Maintainer: Ubuntu Core developers <[email protected]>
Architecture: amd64
Version: 3.2-0ubuntu7
Replaces: bash-doc (<= 2.05-1), bash-completion
Depends: base-files (>= 2.1.12), debianutils (>= 2.15)
Pre-Depends: libc6 (>= 2.5-0ubuntu1), libncurses5 (>= 5.4-5)
Suggests: bash-doc
Conflicts: bash-completion
Conffiles:
 /etc/skel/.bashrc 52acca91b52f797661c89b181809b9f3
 /etc/skel/.profile 7d97942254c076a2ea5ea72180266420
 /etc/skel/.bash_logout 22bfb8c1dd94b5f3813a2b25da67463f
 /etc/bash.bashrc 860d464fca66fff1af4993962a253611
 /etc/bash_completion c8bce25ea68fb0312579a421df99955c
 /etc/skel/.bash_profile d1a8c44e7dd1bed2f3e75d1343b6e4e1 obsolete
Description: The GNU Bourne Again SHell
 Bash is an sh-compatible command language interpreter that executes
 commands read from the standard input or from a file.  Bash also
 incorporates useful features from the Korn and C shells (ksh and csh).
 .

As you can see, there’s all the possible information about bash package (the Bourne Again Shell), but you usually don’t need to know this much, so instead we’ll use dpkg command to confirm what packages are installed and which ones are not.

List installed packages with dpkg

The easiest way to confirm the list of packages installed on your Ubuntu OS is to run dpkg -l command. The output is quite long, so I’ll only show you a fragment of it:

ubuntu# dpkg -l | more
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name                       Version                                      Description
+++-==========================-============================================-============================================
ii  adduser                    3.100                                        Add and remove users and groups
ii  alsa-base                  1.0.13-3ubuntu1                              ALSA driver configuration files
ii  alsa-utils                 1.0.13-1ubuntu5                              ALSA utilities
ii  apache2                    2.2.3-3.2ubuntu2.1                           Next generation, scalable, extendable web se
rc  apache2-common             2.0.55-4ubuntu4                              next generation, scalable, extendable web se
ii  apache2-doc                2.2.3-3.2ubuntu2.1                           documentation for apache2
ii  apache2-mpm-prefork        2.2.3-3.2ubuntu2.1                           Traditional model for Apache HTTPD 2.1
ii  apache2-utils              2.2.3-3.2ubuntu2.1                           utility programs for webservers
ii  apache2.2-common           2.2.3-3.2ubuntu2.1                           Next generation, scalable, extendable web se
ii  apt                        0.6.46.4ubuntu10                             Advanced front-end for dpkg
ii  apt-utils                  0.6.46.4ubuntu10                             APT utility programs
ii  aptitude                   0.4.4-1ubuntu3                               terminal-based apt frontend
ii  at                         3.1.10ubuntu4                                Delayed job execution and batch processing
ii  autoconf                   2.61-3                                       automatic configure script builder
ii  automake1.4                1.4-p6-12                                    A tool for generating GNU Standards-complian
ii  automake1.9                1.9.6+nogfdl-3ubuntu1                        A tool for generating GNU Standards-complian
ii  autotools-dev              20060920.1                                   Update infrastructure for config.{guess,sub}
ii  awstats                    6.5+dfsg-1ubuntu3                            powerful and featureful web server log analy
ii  base-files                 4ubuntu2                                     Debian base system miscellaneous files

The legend at the very top of the output explains the first 3 charactes of each line in the dpkg output, the symbols there confirm whether each package is expected to be installed, and whether it’s actually installed or partially installed:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=b
||/ Name                       Version                                      Description
+++-==========================-============================================-============================================
ii  adduser                    3.100                                        Add and remove users and groups

A first letter of each option is used, so ii for the adduser package in this example means that the desired state for this package is “Installed” (first i) and the actual status is “Installed” as well. That’s the normal condition for most of your packages.

As you can also see, each line shows you the version of each package you have and provides a brief description of what a package is used for.

That’s it, this should be a good start for your Ubuntu exploration, I’ll post a few more things about dpkg in the future.

Related books

If you want to learn more, here’s a great book:


ubuntu-kung-fu-practical-guide
Ubuntu Kung Fu

See also: