I find it most useful when I approach any learning with a clear goal (or at least a specific enough task) to accomplish.

Here’s a list of simple Bash scripts that I think could be a useful learning exercise:

• what’s your name? – asks for your name and then shows a greeting
• hello, world! (that also shows hostname and list server IP addresses)
• write a system info script – that shows you a hostname, disks usage, network interfaces and maybe system load
• directory info – script that counts disks space taken up by a directory and shows number of files and number of directories in it
• users info – show number of users on the system, their full names and home directories (all taken from /etc/passwd file)
• list virtual hosts on your webserver – I actually have this as a login greeting on my webservers – small script that highlights which websites (domains) your server has in web server (Apache or Nginx) configuration.

Do you have any more examples of what you’d like to do in a Linux shell? If not, I’ll just start with the examples above. The plan is to replace each example name in the list above with a link to the Unix Tutorial post.

BSD stands for Berkeley Software Distribution, and it was a UNIX software compilation released by the Computer Software Research Group at the University of California, Berkeley between 1977 and 1995. The first one was 1BSD (First Berkeley Software Distribution) compiled by Bill Joy, but only as an add-on to Sixth Edition Unix from Bell Labs. In 2BSD Bill Joy added a C Shell and the iconic vi text editor.

As BSD evolved it became a complete UNIX operating system in its own right. The last release from Berkeley was 4.4BSD-Lite Release 2, and it contained no proprietary code from AT&T, and was freely available under the permissive BSD license. Since then other projects, descendants of the original BSD, continued the development and they are what BSD today generally refers to. Most popular of these are FreeBSD, NetBSD, and OpenBSD.

BSD was the first to include support for the Internet Protocol stack in form of its Berkeley sockets, which made it easy to read and write files over the network. It is also easier to natively run software from other operating systems on BSD thanks to its binary compatibility layer. The very permissive nature of the BSD open source license enabled widespread use of its code in various other software projects. Apple’s OSX and iOS, for example, are based on BSD code.

FreeBSD

Originally based on top of 386BSD, and since version 2.0 on 4.4BSD-Lite FreeBSD exists with a goal of providing a complete operating system that can serve any purpose without any strings attached, free in every sense of the word. FreeBSD originated the ports collection system for easy download, building, and installation of software packages that continues to be one of the easiest and most sophisticated ways to install software in the UNIX world. This was adopted by NetBSD and OpenBSD as well.

FreeBSD also uses a rather open model of development by letting hundreds of “committers” make changes to FreeBSD at any time as needed. The selection of committers and resolution of any disputes is managed by the elected Core Team.

FreeBSD is the most popular BSD version used, and also served as the basis of many other operating systems such as most notably the Apple’s OSX. There is also a number of FreeBSD variants such as the desktop oriented PC-BSD created to be easy for everyone to use.

NetBSD

NetBSD was founded after FreeBSD with major emphasis on portability at a time when FreeBSD was mostly focusing on the x86 architecture. NetBSD runs on so many platforms that they have a slogan saying “Of course it runs BSD”. This makes it particularly suitable for computer research because it readily runs on both old and new architectures alike. NetBSD also uses the pkgsrc package management system originally based on the Ports collection system from FreeBSD.

OpenBSD

In 1995 Theo De Radt forked the NetBSD project to create OpenBSD. Particular focus was put on security as well as strong emphasis on great documentation, code correctness, and open source licensing. The project has also spawned a number of key widely used security tools like OpenSSH, OpenNTPD, PF, and most recently the LibreSSL fork of OpenSSL after the Heartbleed bug fiasco. Unlike FreeBSD the project is more strictly managed by Theo De Radt himself. OpenBSD also supports about 20 different hardware architectures.

Changing the hostname in FreeBSD is very simple. It can be done immediately using the hostname command while logged in as root:

hostname new.hostname

Just replace new.hostname with your own desired name. It doesn’t have to have the dot in it, it can be just “myhost”, for example.

This changes the hostname for the current session only, which means that if you reboot it will be revert to the previous one. You can make sure the hostname change sticks permanently by editing the /etc/rc.conf file to add the following line:

hostname="new.hostname"

Again, just replace new.hostname with your own. Also, if this line already exists you can just edit it to change the hostname. Once you save this file your hostname change will stick even after you reboot the machine.

That’s all there is to it.

Finding out sizes of files and directories in Linux is done using the du command, which estimates their disk space usage. The du command can be used with options that allow you to customize the results you get.

Commands can also be combined with each other so the second command filters the results output by the first command. So to find directories that are larger than 1GB you can run this command:

du -h --max-depth=1 / | grep '[0-9]G\>'

The -h option displays the sizes in a more human readable format, in gigabytes rather than kilobytes. The –max-depth=1 option makes it display the sizes of only the directories immediately within the specified path, in this case the root path /, which in Linux includes directories like /home, /usr, /bin, /var, and so on. If we were to specify 2 it would go a level further and also look into directories like /home/user, /usr/bin, /var/log, etc.

The | sign is the pipe sign that allows combining the first command with the next command, in this case grep, which is used for searching through text output for the specified strings, which can be specified through regular expressions. So the grep ‘[0-9]G>’ searches through the du output and displays only the files that are larger than 1GB.

Here’s another example of this command, but with –max-depth=1 being shortened to -d 1, which works the same, and checking the /var directory instead of the root / directory.

du -h -d 1 /var | grep '[0-9]G\>'

This single command is all you need for this task, but there’s an even shorter one that works well too. You can use the built in -t or –threshold option of the du command like this:

du -h -d 1 -t 1G /

It will display first level directories larger than 1GB within the root / path, just like the first command. If you want it to display all directories, not just first level, just leave out the -d 1 option.

Finally, as a bonus, you can sort the results from largest to smallest by piping in the sort command:

du -h -d 1 -t 1G / | sort -hr 

You can also sort the first command like this too, in which case you’re combining the three commands into one:

du -h --max-depth=1 / | grep '[0-9]G\>' | sort -hr 

Those are some of the ways you find directories larger than 1GB, or any other size you desire for that matter. Just switch 1G to 2G or 500M and so on.

SSH, or Secure SHell, allows the user of one computer on the network to connect to and use the shell of another over a secure connection. It consists of two basic components, the SSH client used to connect to a remote server, and the SSH server daemon (sshd) running on the server to accept SSH connections from elsewhere.

Configuration for the sshd server is found in the /etc/ssh/sshd_config file. The client configuration is in /etc/ssh/ssh_config.

Here are some of the most important configuration options for an SSH server:

Port

The default port for SSH is 22, which is typically fine, but it could be changed to some other available port if you want to throw an extra obstacle to would be unauthorized attempts to connect.

PermitRootLogin

This option can be set to either yes or no. If it is set to yes then it will allow using SSH to log in directly as root by running something like ssh [email protected] from the client computer. It may be a good idea to set this to “no” in order to close even the remote possibility of someone cracking through the root password and wreaking havoc. Just a decent precaution.

AllowUsers

With this option you can set to allow only some users on the system to connect via SSH. For multiple users separate them by spaces. For example:

AllowUsers james kevin

That will allow only james and kevin users to connect.

LoginGraceTime

This is the amount of time SSH will wait on the user to authenticate before cutting the connection. By default it is set to 120, or 2 minutes, but it can be reduced if you want to diminish chances of someone successfully attempting a brute force attack.

PasswordAuthentication

Set to yes by default this enable password authentication, which definitely should be enabled unless you have public key authentication enabled, because otherwise basically anyone could connect.

PubkeyAuthentication

An alternative or an addition to PasswordAuthentication setting this to yes could significantly increase security. For it to work you also need an option that specifies where the authorized keys are:

AuthorizedKeysFile ~/.ssh/authorized_keys

TCPKeepAlive

Set to yes by default this option checks the status of your connection by sending keepalive messages to the client. If there are network interruptions it will then close the connection rather than continue to use up resources.

See Also

• SSH port
• ssh command
• Port Forwarding

Both wget and curl are command line tools for transferring files over the network via various network protocols like HTTP or FTP. Wget is a GNU Project by the Free Software Foundation licensed under the GNU GPL whereas Curl is an independent project licensed under a variant of the MIT license.

Curl is also based on a libcurl library, part of the same project, which makes it more suitable for use in programming various applications. It is generally more flexible and featureful whereas wget is simpler.

Simple download

Here’s how to initiate a simple download with both wget and curl.

wget ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

curl -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

This will download the file to the current working directory with its original file name (which is what the -O option passed to curl is for). You can run curl without any options as well, but it will dump the file contents on to the screen as it downloads.

Download multiple files at once

You can also download multiple files in one go by specifying multiple URLs:

wget ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

curl -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf -O ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont_upper-7.0.01.ttf

Specifying -O before each URL isn’t necessary, but we found that not specifying it results in the second file being downloaded being dumped into the command line.

That said, it is also possible to specify multiple files for download in accordance to a given range of numbers or letters in the file name.

With curl you can also download multiple files with sequential numbers or letters in their names like file1, file2, file3, and so on by specifying the range in the brackets:

curl -O ftp://example.com/file[1-50].txt

Multiple nested sequences are possible too, like this:

curl -O http://example.com/[2002-2014]/file{a-z}.txt

Resume downloads

Both wget and curl allow you to resume partial downloads, which is useful if a network outage in the middle of the download interrupted it, for example, and you want to continue where you left off.

With wget you simply use the -c or –continue option. Some have a habit of always passing this option just in case.

wget -c ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf

With curl you can use the -C – options:

curl -C - ftp://mirrors.kernel.org/gnu/unifont/unifont-7.0.01/unifont-7.0.01.ttf

That’s in a nutshell how to use wget and curl. They’re both powerful, curl a bit more powerful than wget, and you can find all about what they can do in their manual pages: man wget, and man curl.

Every process in UNIX runs with a particular priority assigned to it, which determines how much processing time should it be allowed to use. Priorities are represented as numbers from -20 to 19 where -20 represents the highest priority, because -20 comes before every other number on that scale, and 19 is actually the lowest priority because it comes last.

This is probably part of why this value is called the “nice value”, because the greater the number the “nicer” is the process when it comes to demanding resources. A process with the value of 19 is the nicest because it is the least selfish, so to speak.

The nice command allows you to start a process with a particular nice value, setting the process priority, whereas the renice command allows changing the nice value of an already running process.

nice

To start a process with a custom nice value you run the desired command prefixed by the appropriate nice command, like this:

nice -n 10 znc

So the -n option says we want to adjust the nice value, the 10 is the actual value, and znc is the command in this example. This will start the znc process with the nice value of 10, a fairly low priority.

Now if you use the ps command, which lists running processes, to search for a znc process you’ll see that its nice (NI) value is 10:

ps -l -C "znc"

F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
1 S  1008 13631     1  0  90  10 -  5251 -      ?        00:15:05 znc

The -l option tells it to show the process in the long format, listing all information about it, and the -C option followed by the command just tells it to look for a process started by that command.

renice

If you’ve got an already running process you can change its nice value, and therefore its priority, without restarting it. Simply run the renice command like this:

 renice -n 7 -p 13631  

So the syntax is the same except instead of specifying a command we’re specifying a process ID (PID), with the -p option. We need to have a process ID of the running process before we can renice it. You can find it in various ways, but the simplest is to just use the above mentioned ps command:

ps -l -C "znc"  

Then simply look for the PID field for the value to put after -p and you’re set.

When you run the ps command after renicing you’ll see the new NI value reflected.

In UNIX and Linux operating systems everything resides in a tree like structure rooted at /, the root directory. This includes storage devices with partitions and their own file systems. The mount command mounts these somewhere within this tree structure. For instance, a disk partition used for data storage can be mounted on /mnt/data or /media/data or wherever else you find it convenient or prudent.

The mount command accepts options that determine how is the file system mounted. A basic mount command without any options can be as simple as this:

sudo mount /dev/sdb1 /media/usb

Of course the directory you’re mounting to (in this example /media/usb) should exist beforehand. To unmount the device from it, thereby removing its file system from the file tree, just use the umount command the same way.

To mount a file system with options pass the -o option first followed by a comma separated list of mounting options, which may look something like this:

sudo mount -o noexec,auto /dev/sdb1 /media/usb/

In this case we’ve used the noexec and auto options, but we might have as well specified any number of other options depending on our needs and desires. Here is a quick overview of those you might find most useful, depending on the situation. They can be used alone or in combination with each other.

defaults

The defaults option is actually a shortcut to a number of different options that you’ll likely want if you have nothing specific in mind, and just want to mount a file system for normal use. They include rw, suid, dev, exec, auto, nouser, and async. In other words running..

sudo mount -o defaults /dev/sdb1 /media/usb  

..is the same as running..

sudo mount -o rw,suid,dev,exec,auto,nouser,async /dev/sdb1 /media/usb 

So what does each of these options mean?

rw – mount with both read and write access

suid – allows giving users who don’t own a specific file in a file system temporary permissions for running the file as if they did own it. In other words the owning user can give other users temporary permissions. This makes commands like passwd and ping usable for normal users even though their operation requires actions which normal users typically don’t have access to.

dev – allows creating devices nodes such as /dev/sdc1 within a mounted file system. The opposite option, nodev, would disallow this.

exec – allows executing binaries within the mounted file system. The opposite option, noexec, would make it impossible to execute programs from the mounted device.

auto – specifies that this file system will be automatically mounted on system startup or by running the mount -a command.

nouser – disallows the ordinary non-root user to mount this file system.

async – specifies that all input and output to the file system should be asynchronous as opposed to sync which would make them synchronous. The difference is that the asynchronous mode allows processing to run even as I/O operations are still ongoing rather than wait for them to finish, which makes sense most of the time, and is therefore a reasonable default option.

If all of these options sound fine to your specific need then just passing the defaults option will be good. However, if you don’t want one or more of these default options you may want to specify your own list, or use defaults with overriding subsequent options.

Here are some of the other useful options you might want to consider, some of which override or are the opposite of the above defaults, but could be useful in certain situations.

ro – mounts the file system as read-only, disallowing any write access to the mounted file system. This could be useful if you want to make sure to preserve the data on the file system as is, and especially prevent overwriting any data, when you are mounting the file system only for the purpose of accessing the files on it.

noexec – disallows executing binaries on the mounted file system, which could also be used for file systems used only as data storage where you don’t wish programs to run.

users – makes it possible for every user on the system to mount or unmount the file system.

group – allows non-root users to mount the file system if one of their groups is the same as the group to which the device belongs.

remount – useful when you want to mount an already mounted file system, but with different options than those specified previously or within the /etc/fstab configuration file, and without changing the mount point (the path where it is mounted).

noatime – prevents updating access times on inodes of the file system, which could speed up access on a frequently accessed file system for a specialized purpose.

umount options

Finally, you can pass the same options to the umount command using the -O option first, like this:

umount -O noexec /dev/sdb1 

Options passed to the umount command only mean that the command will apply to file systems which have the specified option within the /etc/fstab configuration file. In the above example it will not apply if /dev/sdb1 has no noexec option specified in /etc/fstab.

The chmod and chown commands are used to control access to files in UNIX and Linux systems.

The chmod command stands for “change mode”, and allows changing permissions of files and folders, also known as “modes” in UNIX. The chown command stands for “change owner”, and allows changing the owner of a given file or folder, which can be a user and a group. That’s the difference between them in a nutshell.

chmod and chown are working together: chmod can control which permissions are available to owner and owner’s group for a given file or directory, while chown quickly changes who on the system gets access to a file.

Let’s take a quick look at the basic usage of these commands.

chmod examples

The chmod command can be used in a couple of different ways, with permissions (or modes) set by numbers or by letters. Permissions can be given to a user who owns the file (u = user), group of said user (g = group), everyone else (o = others) or all users (a). And the basic permissions that can be given include read (r), write (w), and execute (x). There are also X, s, and t, but they’re less commonly used.

When using numbers you can use a numeric value such as 644 to set permissions. The position of the value represents to whom is the permission given, and the actual value represents which (or how much) permissions are given as a sum total of each permission’s unique value.

First position (in the above example 6) refers to the user. Second refers to the group of the user, and the third refers to all others.

Numeric values for permissions are:

4 = read 2 = write 1 = execute

So a value of 4 will only give read rights, but a value of 6 will give read and write rights because it is a sum of 4 and 2. 5 will give only read ane execute rights, and 7 will give all rights. Do this calculation for each numerical position and you’ll end up with the desired value. So in the example of 644 we’re giving the user who owns the file the permission to read and write (but not execute), the group of that user the permission to read only, and others the right to read only as well.

To set this mode with chmod on a file called important.txt we would simply run this command:

$ chmod 644 important.txt

Note that making a file executable, if it were a script or a program, amounts to simply giving someone or everyone a permission to execute. If this was an important.sh bash script we could allow the owner to execute, and others to read with the 744 mode, or everyone to execute with 755.

$ chmod 755 important.sh

Now, we can also use letters to accomplish the same thing, and we’ve already mentioned the relevant letters above. This is probably easier to remember than using numbers. For example, to accomplish the 644 permissions above we would run this:

$chmod u+rw,go+r important.txt

So we’re saying file owner user gets read and write permissions, group and others get to read.

The second example, with the important.sh file being made executable we could just run this:

$chmod u+rwx,go+rx important.sh

If important.sh already had permissions set to 644 we can add everyone execute rights by simply running:

$ chmod +x important.sh

Not specifying the letter for anyone is treated as if we said “a”, for all.

Finally, if we’re setting permissions to a folder we need to specify the -R option (standing for “recursive”):

$chmod -R 644 important-files/

chown command

Basic usage of chown is pretty straightforward. You just need to remember that first comes the user (owner), and then the group, delimited by a colon.

This command will set the user “daniel”, from the group of “admins” as owners of the directory “important-files”:

$chown -R greys:admins important-files

Just like with chmod, the -R is there when it’s a directory.

It’s possible to change just the owner of a file, without specifying a group (which will stay intact):

$chown -R greys important-files

See Also

• lrwxrwxrwx permissions
• Find Out File Type and Permissions in Perl
• Change Ownership of Files/Directories in Unix
• Basic unix commands
• Finding an Owner of a File in Unix

When you want to save yourself from typing an unwieldy command over and over again you can create and use an alias for it. It will then act as a shortcut to the larger command, which you can type and run instead.

Creating aliases in UNIX (and Linux) is done with a simple alias command which follows this format: alias name=’command you want to run’.

Replace the “name” with your shortcut command, and “command you want to run” with the larger command you want to create an alias of. Here’s a simple example:

alias accesslog='tail -f /var/log/lighttpd/access.log'  

In this example I’ve effectively created a new accesslog command which is an alias of the tail -f /var/log/lighttpd/access.log command. What it does is follow the access.log file and display new entries in it as they happen. Now instead of having to write the whole tail -f command every time I want to look at what’s happening in the access.log file I can simply run the accesslog alias command instead, which is pretty nifty.

What if I want to unset the alias once I no longer need it or wish to set a new better alias? Well, simply run:

unalias accesslog  

Quite logical. Now the accesslog alias no longer exists.

One thing to keep in mind though is that aliases that are set this way get lost the moment you close the command line session, or in other words, they are temporary. If you want to save aliases permanently you will have to edit the bash configuration file, which is usually .bashrc or .bash_profile residing in your user home directory. You can edit whichever you prefer, or whichever exists on your system.

To edit .bashrc just open it in a command line text editor such as nano, or any other you might prefer, and add the same exact alias command as in the above example at the bottom of it, or find where other aliases are already set and add yours after them.

nano .bashrc  

Once you add your aliases save the file, which in the nano editor is done by pressing the Сtrl-x keyboard shortcut, answering “y” when asked to save, and hitting enter.

Now your alias is saved permanently, and it will therefore work even after you close the session and come back. Of course, to remove the permanent alias just edit the file again and remove the line you’ve just added. If it’s still set run the unalias command as shown above and it will be gone.

Note that aliases are set for the currently active user. So you have to edit the .bashrc file in the home directory of that user. If you’re logged in as root that would be /root/.bashrc, and if you’re logged in as joe, for example, it will be in /home/joe/.bashrc. If you try to run root’s alias while acting as joe or vice versa you’ll get a “command not found” error.

Also note that aliases added to .bashrc aren’t active immediately after you save the file since that file is read on user’s login. If you log out and log back in then it will work.

Finally, once you have a bunch of aliases set up you might want to check up on which aliases are available. To do that just run the alias command by itself:

alias  

And it will list something like this:

alias accesslog='tail -f /var/log/lighttpd/access.log' 
alias ls='ls --color=auto'  

The list represents all of the aliases that have been set in .bashrc, or on the command line during the current session. In the above example we see my accesslog alias, and another one for the ls command associating it with the ls –color=auto command, which simply adds some coloring to our ls lists.

That brings us to the final point worth a mention, as demonstrated by the above ls alias, and that is that you can alias an already existing real command. For example if we have a nmon command installed, which shows various system activity information, we can actually turn it into an alias for the top command, which also shows system activity.

You probably don’t want to do this, or at least, you don’t want to keep this alias, but for the sake of demonstration:

alias nmon='top'  

And now when you run nmon, instead of opening the actual nmon program it will open top. In other words the alias is masking the original command.

This serves as a word of caution when it comes to setting names of aliases; try to avoid setting names that match existing commands. Chances are you’ll want those commands doing what they’re supposed to do, except in special cases like the above ls alias, which simply aliases to its own coloring options.

And that’s how aliases work in UNIX (and Linux).