tee: Replicate Standard Output

Now and then I come across a situation when I need to run a script or a Unix command and would like to not only see the output of it on the screen, but also save this output to some log file. Redirecting the standard output using standard Unix stream redirection isn’t always useful because your output will either be shown to you, or sent to the file – but not both at the same time

tee command

That’s where the tee command becomes really useful. You pipe your output to this command, and let it take care of the rest.

Here’s how you use it:

greys@simplyunix:~$ uname -a | tee /tmp/uname.txt
Linux simplyunix.com 2.6.16.29-xen #1 SMP Sun Sep 30 04:00:13 UTC 2007 x86_64 GNU/Linux
greys@simplyunix:~$ cat /tmp/uname.txt
Linux simplyunix.com 2.6.16.29-xen #1 SMP Sun Sep 30 04:00:13 UTC 2007 x86_64 GNU/Linux

What happens is that tee replicates the standard output to both your session and the specified file.

See also:




How To Find Out RedHat Version And More

If you’re a really curious mind, you won’t be satisfied with simply knowing the current release of your RedHat Linux, that’s why there’s a few more commands you could use to satisfy your interest.

RedHat release

If you simply want to confirm whether you’re using a RHEL4, RHEL5 or any of the previous RedHat Linux releases, this is the first place to look:

bash-3.1$ cat /etc/redhat-release
Red Hat Enterprise Linux Client release 5 (Tikanga)

RedHat kernel version and type

Next step is to find out the exact Linux kernel version on your system, and also confirm whether it’s 64-bit or not:

bash-3.1$ uname -a
Linux rhserver123 2.6.18-8.el5 #1 SMP Fri Jan 26 14:15:14 EST 2007 x86_64 x86_64 x86_64 GNU/Linux

RedHat kernel build

For the most curious ones, here’s the last command. Use it to confirm who and when compiled the RedHat kernel you’re using, and what gcc compiler was used in the build process.

bash-3.1$ cat /proc/version
Linux version 2.6.18-8.el5 ([email protected]) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Fri Jan 26 14:15:14 EST 2007

See also




How To Take A Screenshot in Unix (xwd)

Quite often there’s a need for you to take a screenshot of your Unix desktop, and as always there’s a number of ways to do it. Today I’m going to cover the command line approach to taking screenshots.

Taking a Screenshot with xwd

Most modern Unix desktop systems come with Gnome desktop environment by default, and use Xorg as their default X11 server. This means you are likely to have the xwd tool in your OS, which allows you to take screenshots.

Furthermore, many Unix distros come with hundreds of command-line tools bundled with the default OS install. Imagemagick is one of such bundled toolkits, and you can use the convert tool which is part of it for converting the xwd-generated screenshot into any graphics format of your preference.

Here’s how you use these two commands together:

bash-3.0$ xwd -root | convert - /tmp/screenshot.png

In this line, xwd command is invoked to take a full screenshot of your desktop. You then pipe its output to the convert tool and specify where you want this output saved to.

See Also




How To Mount An ISO image

Mounting an ISO image of a CD/DVD before burning it is one of the basic steps to verifying you’re going to get exactly the desired result. It’s also a neat trick to access files from a CD/DVD image when you only need a file or two and not a whole CD. Why burn it at all when you can access files much quicker and easier by simply mounting the ISO image?

Every Unix OS has a way to access ISO filesystem, and today I’ll only give you examples for Linux and Solaris. In both cases, the two things you need for the example to work are the ISO image itself and an available mount point (basically, an empty directory) on your filesystem to mount it under.

Here’s how to mount an ISO in Linux:

# mount -o loop /net/server/linux-bootcd.iso /mnt

Here’s how to mount an ISO in Solaris:

First, you need to associate your ISO image with a virtual device:

# lofiadm -a /net/server/linux-bootcd.iso

lofiadm approach allows you to have virtual devices associated with as many ISO images as you like, and you can view the list of current associations at any moment:

# lofiadm
Block Device File
/dev/lofi/1 /net/server/linux-bootcd.iso

To mount a virtual device, you use the following command:

# mount -F hsfs /dev/lofi/1 /mnt

See Also




script: Save Your Session Log

If you ever need to save the history of your Unix shell session, one of the easiest ways to do it is to use the script command, found in most Unix systems.

Simply provide the file name for your log as a command line parameter:

$ script /tmp/unix-session.log

After starting the command, you’ll be able to do whatever you want, and once you’re done you can use the standard Ctrl+D to end the session recording. The generated log file will have both the commands you have typed during your session and the output generated by these commands.

Your session log with also include the timestamps marking the start and the finish of your recording.

Here’s a sample session showing you how you use the script command:

$ script /tmp/unix-session.log
Script started, file is /tmp/unix-session.log
$ uname -a
Linux simplyunix.com 2.6.16.29-xen #3 SMP Sun Oct 15 13:15:34 BST 2006 x86_64 GNU/Linux
$ exit
Script done, file is /tmp/unix-session.log

Now, if we were to cat the log file generated in this example, we’d see the following:

$ cat /tmp/unix-session.log
Script started on Tue Nov 20 16:48:08 2007
$ uname -a
Linux simplyunix.com 2.6.16.29-xen #3 SMP Sun Oct 15 13:15:34 BST 2006 x86_64 GNU/Linux
$ exit
Script done on Tue Nov 20 16:48:12 2007

Have you got a better way to save your sessions? Let me know! (saving logs from gnome-terminal and Putty doesn’t count, mind you!)




Perl: Searching Through Directory Trees

I had a need to scan a huge directory tree today, identifying the users and Unix groups owning all the files. The problem I faced was too long usernames and group names which meant the

find /directory -ls

command which I normally use for such tasks wasn’t terribly useful because there was no space delimiter between a username and a group. Results of such scan of the directory tree will have to later be parsed by other tools, and that’s why proper splitting of the output into separate fields is so important.

 

This issue was motivational enough to refresh my Perl skills and sketch the following script (based entirely on this Never Run Unix Find Again article).

It’s a very simple piece of code which takes a directory to scan as a parameter.

How this works

As you can see, we’re using the standard File:Find functionality, and the two parameters find function takes are the wanted function, where you put conditions for your search.

Within this function, you call lstat to obtain all the necessary information about each directory entry, and then output the necessary fields.

Perl code

#!/usr/bin/perl
use File::Find;
if ($ARGV[0] ne "") {
        $dir = $ARGV[0];
} else {
        print "Please specify a directory!";
        exit;
}

find &wanted, $dir;

sub wanted {
  my $dev;         # the file system device number
  my $ino;         # inode number
  my $mode;        # mode of file
  my $nlink;       # counts number of links to file
  my $uid;         # the ID of the file's owner
  my $gid;         # the group ID of the file's owner
  my $rdev;        # the device identifier
  my $size;        # file size in bytes
  my $atime;       # last access time
  my $mtime;       # last modification time
  my $ctime;       # last change of the mode
  my $blksize;     # block size of file
  my $blocks;      # number of blocks in a file
  my $user;	# username
  my $group;	# unix group name

#Right below here your telling lstat to retrieve all this info on each and every file/directory. Each and every file/directory is written to $_.

  (($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks) = lstat($_));
  $user = getpwuid($uid);
  $group = getgrgid($gid);

  print $File::Find::name . ":$mode:$size:$user:$group:$ctime:$mtimen";
}

Hope you find this useful. Good luck with finding all your files! 🙂

For further reading, please consult the Perldoc section on File:Find.




Unix File Types

In Unix systems, there are 6 file types. Below I will give a very short description of each.

How to find out the type of file in Unix

The first and most obvious way to confirm the type of a particular file is to use the long-format output of ls command, invoked by the -l option:

$ ls -l * 
 -rw-r--r-- 1 greys greys       1024 Mar 29 06:31 text

The very first field of such output is the file type and access permissions field, I’ll cover them in a separate post in the future. For now, just concentrate on the first character in this field. In this particular case, it’s “-“, which means it’s a regular file. For other file types, this character will be different.

Regular file

This is the most common type of a file in Unix. Being a plain collection of bytes with arbitrary data. There’s nothing mysterious about this type. Most of the files you will ever work with are regular.

In long-format output of ls, this type of file is specified by the “-” symbol.

Directory

This is a special type of a file in Unix, which only contains a list of other files (the contents of a directory). You don’t work with directories directly, instead you manage them with standard commands provided with your OS. The whole directory structure of your Unix system is made of such special files with directory content in each of them.

In long-format output of ls, this type of file is specified by the “d” symbol:

$ ls -ld * 
 -rw-r--r-- 1 greys greys	1024 Mar 29 06:31 text
 drwxr-xr-x 2 greys greys	4096 Aug 21 11:00 mydir

Special Device File

This type of files in Unix allows access to various devices known to your system. Literally, almost every device has a special file associated with it. This simplifies the way Unix interacts with different devices – to the OS and most commands each device is still a file, so it can be read from and written to using various commands. Most special device files are owned by root, and regular users cannot create them,

Depending on the way of accessing each device, its special device file can be either a character (shown as “c” in ls output) or a block (shown as “b”) device. One device can have more than one device file associated, and it’s perfectly normal to have both character and block device files for the same device.

Most special device files are character ones, and devices referred by them are called raw devices. The simple reason behind such a name is that by accessing the device via its special device character file, you’re accessing the raw data on the device in a form the device is ready to operate with. For terminal devices, it’s one character at a time. For disk devices though, raw access means reading or writing in whole chunks of data – blocks, which are native to your disk. The most important thing to remember about raw devices is that all the read/write operations to them are direct, immediate and not cached.

Block device file will provide similar access to the same device, only this time the interaction is going to be buffered by the kernel of your Unix OS. Grouping data into logical blocks and caching such blocks in memory allows the kernel to process most I/O requests much more efficiently. No longer does it have to physically access the disk every time a request happens. The data block is read once, and then all the operations to it happen in the cached version of it, with data being synced to the actual device in regular intervals by a special process running in your OS.

Here’s how the different types of special device files look in your ls output:

$ ls -al /dev/loop0 /dev/ttys0
brw-rw---- 1 root disk 7,  0 Sep  7 05:03 /dev/loop0
crw-rw-rw- 1 root tty  3, 48 Sep  7 05:04 /dev/ttys0

Named Pipe

Pipes represent one of simpler forms of Unix interprocess communication. Their purpose is to connect I/O of two Unix processes accessing the pipe. One of the processes uses this pipe for output of data, while another process uses the very same named pipe file for input.

In long-format output of ls, named pipes are marked by the “p” symbol:

$ ls -al /dev/xconsole
prw-r----- 1 root adm 0 Sep 25 08:58 /dev/xconsole

Symbolic Link

This is yet another file type in Unix, used for referencing some other file of the filesystem. Symbolic link contains a text form of the path to the file it references. To an end user, symlink (sort for symbolic link) will appear to have its own name, but when you try reading or writing data to this file, it will instead reference these operations to the file it points to.

In long-format output of ls, symlinks are marked by the “l” symbol (that’s a lower case L). It also show the path to the referenced file:

$ ls -al hosts
lrwxrwxrwx 1 greys www-data 10 Sep 25 09:06 hosts -> /etc/hosts

In this example, a symlink called hosts points to the /etc/hosts file.

Socket

A Unix socket (sometimes also called IPC socket – inter-process communication socket) is a special file which allows for advanced inter-process communication. In essence, it is a stream of data, very similar to network stream (and network sockets), but all the transactions are local to the filesystem.

In long-format output of ls, Unix sockets are marked by “s” symbol:

$ ls -al /dev/log
srw-rw-rw- 1 root root 0 Sep  7 05:04 /dev/log

That’s it. Hope this gave you a better idea of what file types you can find working on your Unix system. I’ll obviously expand relevant topics in the future. Let me know if there’s anything in particular you’d like me to concentrate on!




How To: Use apt-get behind proxy

If you run your Ubuntu system behind a firewall and have to use proxy server for http and ftp access, then your apt-get on a newly installed Ubuntu system will probably not work.

To make it use proxy, simply set the http_proxy environment variable. Once you get it working (try something like apt-get update), you’ll probably want to add it to your .bashrc file.

Setting the http_proxy variable

Here’s how you set the variable:

bash$ export http_proxy=http://proxy:8080

If you’re required to use your username and password for your proxy, they can be specified this way. Obviously, port 8080 can and should be replaced with the valid port for your proxy server (sometimes it’s 3128):

bash$ export http_proxy=http://username:password@proxy:8080

See Also




URL file-access is disabled in the server configuration

I’ve recently upgraded Apache and PHP on my VPS, and one of the unpleasant surprises was that some scripts which tried including pages from remote sites (I know, not the most secure approach, but there were reasons for that) got broken.

allow_url_fopen

Traditionally, all the websites Google finds suggest that you double-check that your php.ini config has the allow_url_fopen enabled:

allow_url_fopen = On

Well, in my case it was enabled, but scripts were still broken. The really weird thing was that the upgrade procedure didn’t include changing the php.ini in any way, so it was fully working before and I kind of expected it to continue working.

allow_url_include

After some quick research, I’ve found out that PHP 5.1 introduced a new security option to accompany the allow_url_fope, and this was exactly the option which broke my scripts:

allow_url_include = On

There you have it, hope it helps you next time you come across this problem!




Ubuntu: Using Sudo to Grant User Privileges

If you have used your fresh Ubuntu install for longer than half an hour, chances are that you’ve discovered the sudo command already.

sudo allows certain users to execute a command under another user’s privileges. Most commonly, using sudo implies running a command as a superuser, but the approach works equally well for allowing you to inherit a user ID (uid) and group ID (gid) of any user on the system.

To gain access, a password is asked, and by default it is your password, and not the password of a user you’re trying to run a command as. This allows for the system’ s administrator to effectively manage user privileges without having any user share their password.

sudo is based off the /etc/sudoers file, which should be edited by root employing the visudo command. WARNING: although /etc/sudoers file is a regular text which root can edit manually, ONLY visudo way of updating it is recommended, as this command, apart from editing capabilities, also does a syntax check of the changes before applying them to prevent user privilege related disasters.

If you want to grant superuser privileges to a particular user, the following line should be added to the /etc/sudoers file (just type visudo to invoke the editor):

greys ALL=(ALL) ALL

In this example, greys is the username.

Related books

If you want to learn more, here’s a great book:

ubuntu-kung-fu-practical-guide
Ubuntu Kung Fu