How To: Use yum behind proxy

I’ve got a few RedHat and CentOS VMs running on internal network of my server, and since updating them quite regularly is a habit I thought this little piece of advice will help all of you who run VMs in a similar scenario.

To make yum command use proxy, your best best is to edit /etc/yum.conf and add your proxy server reference:

proxy=http://192.168.3.1:3128

You don’t have to restart anything but it may be a good idea to do yum clean all and then yum check-update:

[root@testvm1 ~]# yum check-update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: be.mirror.eurid.eu
* epel: epel.uni-oldenburg.de
* extras: be.mirror.eurid.eu
* updates: centosa5-msync-dvd.centos.org
base                                         | 3.7 kB 00:00
base/primary_db                              | 4.5 MB 00:01
cr                                           | 3.0 kB 00:00
cr/primary_db                                | 1.2 kB 00:00
epel                                         | 3.4 kB 00:00
epel/primary_db                              | 3.7 MB 00:00
extras                                       | 3.5 kB 00:00
extras/primary_db                            | 6.3 kB 00:00
updates                                      | 3.5 kB 00:00
updates/primary_db                           | 1.8 MB 00:01



Passwordless SSH with encrypted homedir in Ubuntu

Quite recently I came across a very interesting issue: while configuring passwordless SSH (it’s public key based, so depending on you have it configured it may not be completely passwordless) access to some of my VPS servers, I found that the same keypair just wouldn’t work on one of the servers.

Not only that, but the behaviour was quite bizzare: upon my first attempt to connect the public key would get rejected and a regular password would be requested by the ssh session. But once I successfully logged in with my password, any subsequent ssh connections would happily authenticate by my public key and would let me in without a problem.

Those of you using home dir encrypiton in Ubuntu are probably smiling right now! 🙂 But becase I have never consciously configured or used this feature, it took me a good few hours to troubleshoot the issue and come up with the fix.

Why public-key based SSH doesn’t work with encrypted home directories

The answer is quite simple: before your server can decide whether you are providing a valid and trusted SSH key, it must read your public key stored in your homedir. But if your homedir is encrypted, this becomes a classical chicken-and-egg scenario – until you log in and therefore decrypt your homedir the server won’t gain access to your public key. Only you wouldn’t be needing the public key by then, would you?

Store your authorized SSH keys outside your encrypted home directory

If you happen to like your homedir encryption AND would like to use public/private key SSH authentication,  there is a way out: you need to store your authorized keys outside of your encrypted homedir.

The usual access restrictions and directory/file permissions still apply, so the only thing you’re changing is moving your authorized keys outside of the encrypted homedir on your server. This way things will work exactly as you expect: you authenticate with your private key and this results in your automatically mounted and decrypted homedir.

Here are the steps to make this happen. You’re going to need superuser privileges for my scenario because it caters for all the users on your Ubuntu server, not just one account that belongs to you (use sudo to become root).

Step 1: create a directory structure for your authorized keys.

First, the main directory, I created it under /var – seems quite a safe choice since this directory is unlikely to grow and is equally unlikely to get removed by accident.

# mkdir /var/openssh

Perfect! Now we need to create user-specific directories, just to keep this dir really tidy. My username is “greys“, so here is the directory:

# mkdir /var/openssh/greys
# chown greys /var/openssh/greys

Step 2: copy existing authorized keys file into new location

(you must log in as your username for this, otherwise the homedir will stay encrypted)

$ cp /home/greys/.ssh/authorized_keys /var/openssh/greys

Step 3: update SSHd config with new location for authorized_keys file

You’re going to do this as root once again:

# vi /etc/ssh/sshd_config

update the value of the AuthorizedKeysFile so that it looks like this:

AuthorizedKeysFile        /var/openssh/%u/authorized_keys

Step 4: Restart SSH service

# service ssh restart
ssh start/running, process 3708

That’s it! Give it a try and let me know how it worked out.

Recommended books:

[AMAZONPRODUCTS asin=”1590594762″]

See also




Upgrading Ubuntu with do-release-upgrade

There comes a time (a couple of times a year, actually) when you may want to upgrade your Ubuntu distro (read here for instructions on confirming your version of Linux: Find Out Linux Version)

Once that’s done, you can use do-release-upgrade for a hassle free upgrade.

IMPORTANT: are you can see, I’ve used a really old Ubuntu server with 8.10, hence your procedure for upgrading more recent Ubuntu versions may be slightly different. For example, later upgrades will warn you if you’re doing a release upgrade over ssh.

What do-release-upgrade is and when you should use it

do-release-script is a Python script which automates the process of updating multiple packages. It relies upon Ubuntu’s core package management functionality.

Apart from downloading and installing updated versions of packages found on your system, this command attempts to take care of all the necessary Ubuntu-release related file changes.

Step 1: Run do-release-upgrade

Once you type the do-release-upgrade command name and press Enter, you should see how vital information about packages currently installed is being collected:

# do-release-upgrade
Checking for a new ubuntu release Done
Upgrade tool signature Done
Upgrade tool Done
downloading
extracting ‘jaunty.tar.gz’
authenticate ‘jaunty.tar.gz’ against ‘jaunty.tar.gz.gpg’
Reading cache
Checking package manager
Reading package lists: Done
Reading state information: Done
Updating repository information
Done http://archive.ubuntu.com jaunty Release.gpg
Done http://archive.ubuntu.com jaunty-updates Release.gpg
Done http://security.ubuntu.com jaunty-security Release.gpg
Done http://us.archive.ubuntu.com jaunty-backports Release.gpg
Done http://security.ubuntu.com jaunty-security Release

Checking package manager
Reading package lists: Done
jaunty-security/multiverse
Packages: 98  2
Reading state information: Done
Reading state information: Done
Reading state information: Done
Calculating the changes

 

2. Confirming what upgrading will do

This is your last change to change your mind. All the necessary information about your current Ubuntu release is collected, and now you’re presented with the exact upgrade details: how many packages will be removed, how many new ones will be installed, how many will be upgraded. You also are given details about the required amount of data to be downloaded should you decide to proceed with the upgrade;

Do you want to start the upgrade?

1 package is going to be removed. 23 new packages are going to be installed. 420 packages are going to be upgraded.

You have to download a total of 248M. This download will take about 7 minutes with your connection.

Fetching and installing the upgrade can take several hours. Once the download has finished, the process cannot be cancelled.

Continue [yN]  Details [d]

Ready? Press y for yes!

3. Downloading all the packages

Just like with apt-get, you will now see the progress of downloading all the updated packages for your Ubuntu OS. At the bottom of the screen you will see the overall completeness of the download (22% in my example), the current download speed (598kB/s in my case) and the ETA:

Done http://archive.ubuntu.com jaunty-updates/main libbz2-1.0 1.0.5-1ubuntu1.1
Done http://archive.ubuntu.com jaunty/main libdb4.7 4.7.25-6ubuntu1
Done http://archive.ubuntu.com jaunty/main libncursesw5 5.7+20090207-1ubuntu1
Done http://archive.ubuntu.com jaunty-updates/main libssl-dev 0.9.8g-15ubuntu3.6
Done http://archive.ubuntu.com jaunty-updates/main libssl0.9.8 0.9.8g-15ubuntu3.6
Done http://archive.ubuntu.com jaunty/main python2.6 2.6.2-0ubuntu1
[23%] 598kB/s 5min17s

4. Upgrade

Once package are downloaded, they will get installed once by one, with package-specific questions asked for software like postfix or apache.

5. Reboot

To finalize the distro upgrade, you will need to do a reboot. Once completed, you should have a shine next release available.

Recommended books:




How To Disable IPv6 in Red Hat Linux

Since it may be a while before I’m ready to use the IPv6 on my systems, I’ve been disabling IPv6 on most servers so far. And since there’s a particularly elegant way of doing this in Red Hat Linux, I think it’s worth sharing.

How to confirm if IPv6 is running on your system

IPv6 is implemented as a kernel module, so you can use the lsmod command to confirm if it’s currently running on your Red Hat system:

$ lsmod | grep ip
ipv6                  410913  36

If lsmod doesn’t return anything, it confirms that your system isn’t running IPv6.

Prevent IPv6 from getting started by modprobe

As you probably know, modprobe command is used for probing modules upon system boot. Probing simply means a module is loaded and an attempt is made to start it up. With any luck, the module starts successfully and its functionality becomes available to the Linux kernel.

For the boot stage, modprobe uses a special config file which helps you control its behavior: /etc/modprobe.conf. For now, let’s just concentrate on the IPv6 task at hand. To make sure modprobe doesn’t load the actual module next time your OS reboots, add the following line to the top of the /etc/modprobe.conf file (yes, you’re going to need root privileges for this):

install ipv6 /bin/true

WHY THIS WORKS: just to quickly explain the line above, here’s why we’re using the /bin/true command. install is rule of the modprobe config file which overrides the standard way of probing a module. Effectively, it tells modprobe to just run the specified command for starting a module. In the line above, we’re telling modprobe to use the /bin/true command for probing the ipv6 module. And as you remember, /bin/true is a command which doesn’t do anything but still returns a successful completion code – so in effect we’re doing nothing instead of loading the ipv6 module, and we’re looking good while doing this too.

Next, add the the following two lines to the same /etc/modprobe.conf file, and they will ensure that common aliases used for starting the IPv6 module are ignored by modprobe:

alias net-pf-10 off
alias ipv6 off

IMPORTANT: These change doesn’t immediately disable IPv6 on your system. Being a pretty important module, IPv6 isn’t easy to disable on a live system, and so the easiest is to always follow the changes above with a reboot.

Make sure your IPv6 firewall is disabled

The ip6tables service (/etc/init.d/ip6tables sciprt) is responsible for starting IPv6 firewall, and so you probably want to disable it:

# chkconfig ip6tables off

and then confirm that it won’t be started next time you reboot or switch to any runlevel:

# chkconfig --list ip6tables
ip6tables       0:off   1:off   2:off   3:off   4:off   5:off   6:off

That’s it, there really is nothing else to disabling IPv6. Let me know if you have any questions, otherwise I’ll talk to you soon!




Ubuntu SSH: How To Enable Secure Shell in Ubuntu

SSH (Secure SHell) is possibly the best way to remotely access a Unix system – it’s very secure thanks to automatic encryption of all the traffic, and it’s also quite universal because you can do all sorts of things: access remote command line shell, forward graphics session output, establish network tunnels, set up port redirections and even transfer files over the encrypted session.

Today I’m going to show you how to get started with SSH in Ubuntu.

Installing SSH server in Ubuntu

By default, your (desktop) system will have no SSH service enabled, which means you won’t be able to connect to it remotely using SSH protocol (TCP port 22). This makes installing SSH server one of the first post-install steps on your brand new Ubuntu.

The most common SSH implementation is OpenSSH. Although there are alternative implementations (closed source solutions and binary distributions maintained by various Unix and Unix-like OS vendors), OpenSSH is a de-facto standard in the secure transfers and connections industry. That’s exactly what you want to install.

Log in with your standard username and password, and run the following command to install openssh-server.

You should be using the same username that you specified when installing Ubuntu, as it will be the only account with sudo privileges to run commands as root:

ubuntu$ sudo apt-get install openssh-server
[sudo] password for greys:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  rssh molly-guard openssh-blacklist openssh-blacklist-extra
The following NEW packages will be installed:
  openssh-server0 upgraded, 1 newly installed, 0 to remove and 75 not upgraded.
Need to get 285kB of archives.
After this operation, 782kB of additional disk space will be used.
Get:1 http://ie.archive.ubuntu.com jaunty/main openssh-server 1:5.1p1-5ubuntu1 [285kB]
Fetched 285kB in 0s (345kB/s)
Preconfiguring packages ...
Selecting previously deselected package openssh-server.
(Reading database ... 101998 files and directories currently installed.)
Unpacking openssh-server (from .../openssh-server_1%3a5.1p1-5ubuntu1_i386.deb) ...
Processing triggers for ufw ...
Processing triggers for man-db ...
Setting up openssh-server (1:5.1p1-5ubuntu1) ...
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ... 
* Restarting OpenBSD Secure Shell server sshd                           [ OK ]

Public and Private keys in SSH

As you can see in the sample output above, the installation procedure created 2 sets of keys – SSH2 RSA keypair and SSH2 DSA keypair. The reason for this is that OpenSSH relies heavily on the public and private key (PPK) infrastructure.

The concept behind PPK is pretty cool: SSH allows you to create keypairs. They are generated to the maximum randomness achievable on your system. Keypairs can be created for your server or for your individual uses.

The idea is that public keys are shared with other servers, and they later can be used as a unique identificator to confirm your true identity. When you’re connecting to another server, it uses your public key to encrypt a short message and the secure session will only be established if on your side you have a private key that allows decrypting the message. No other system or user can decrypt the message because only you would have the private key. That’s why it’s called private – don’t ever share it with anyone.

As an additional security measure, when you’re generating personal keypairs you’ll be asked to supply a passphrase so that even if someone steals your private password they won’t be able to use it without knowing your passphrase.

Verifying your SSH server works

While you’re still on your local desktop session, you can use the ps command to confirm that SSH daemon (sshd) is running:

ubuntu$ ps -aef | grep sshd
root     24114     1  0 15:18 ?        00:00:00 /usr/sbin/sshd

Now that you see it’s there, it’s time to try connecting:

ubuntu$ ssh localhost

Since this is the first time you’re trying to connect using SSH, you’ll have to answer yes to the following question:

The authenticity of host 'localhost (::1)' can't be established.RSA key fingerprint is 18:4d:96:b3:0d:25:00:c8:a1:a3:84:5c:9f:1c:0d:a5.Are you sure you want to continue connecting (yes/no)? yes

… you’ll then be prompted for your own password (remember, the system treats such connection request as if you were connecting remotely, so it can’t trust you without confirming your password):

Warning: Permanently added 'localhost' (RSA) to the list of known hosts.greys@localhost's password:

.. and finally you’ll see the usual Ubuntu (Jaunty in this example) banner and prompt:

Linux ubuntu 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:57:59 UTC 2009 i686

The programs included with the Ubuntu system are free software;the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law.

To access official Ubuntu documentation, please visit:http://help.ubuntu.com

Last login: Fri May 15 15:18:34 2009 from ubuntu

ubuntu$

That’s it, providing you have your networking configured and you know your IP address or hostname, you can start connecting to your Ubuntu box from remote systems, using the same command. Enjoy!

Recommended books:

See also:




How to Confirm Disks Capacity in Linux

show disk size in Unix is a very popular request visitors use to arrive at my Unix Tutorial pages. Since I never addressed the question of confirming the number of hard drivers available on your system or the task of finding out a disk’s capacity, I’d like to document a quick and easy way of doing just that.

I hope that when someone looks for a way to show disk size, what’s really expected is a command to help you confirm the capacity of a disk in gigabytes.

Using fdisk command in Linux

One of the easiest ways to learn a lot about hard drives installed on your Linux system is to use the fdisk command:suse# fdisk -l

Disk /dev/sda: 145.4 GB, 145492017152 bytes
255 heads, 63 sectors/track, 17688 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         262     2104483+  82  Linux swap / Solaris
/dev/sda2   *         263       17688   139974345   83  Linux

As you can see, there are two sections in the output provided: disk information (capacity and geometry) and disk layout (partitions). The same pattern is repeated if you have more than one disk installed.

What you should look for is the lines starting with “Disk” word: they usually specify the device names for each drive and also provide the capacity in gigabytes. Thus, a time saver would be to grep the necessary information from the command above, this way:

suse# fdisk -l | grep Disk
Disk /dev/sda: 145.4 GB, 145492017152 bytes

On a system with multiple disks, the output will look more useful:

redhat# fdisk -l | grep Disk
Disk /dev/sda: 21.4 GB, 21474836480 bytes
Disk /dev/sdb: 4294 MB, 4294967296 bytes

That’s it – a very simple way for you to determine the number of disks in your system while also confirming the capacity available for your needs. fdisk command is actually a very powerful disks management tool which allows you to manage partitions – create and delete them or modify the type of each partition. I will be sure to revisit this command some other time cause usage above doen’t do this wonderful Unix command any justice.

See also:




How To Confirm if Your CPU is 32bit or 64bit

I had to download a piece of software today for one of the servers which I haven’t used in a while. A question of confirming the 64bit CPU capability came up, and I realized that I never mentioned it here on Unix Tutorial.

Some of you probably remember the uname command which also shows you similar information, but uname confirms the running kernel of your OS and not the CPU capability: if you’re booted into 32bit mode, it will not help you to recognize the 64bit potential of your system.

Obtaining CPU information from /proc/cpuinfo

Most Linux distros will have the special /proc/cpuinfo file which contains a textual description of all the features your processors have. This is a very useful file – depending on your task it may help you identify any features of your processors, as well as confirm the overall number of CPUs your system has installed.

Most commonly, the following information is obtained from /proc/cpuinfo:

  • processor model name and type
  • processor speed in Mhz
  • processor cache size
  • instruction flags supported by CPU

Here’s how the typical output will look:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      :                   Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 3192.320
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts
acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6388.78
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

The same block of information will be shown for each CPU visible to your system. There will be 2 processor instances for each physical CPU if hyper-treading is enabled, and there will be 2 or 4 processor entries for each physical CPU on dual- and quad-core systems configurations.

How to confirm the 64bit capability of your CPU in Linux

Based on /proc/cpuinfo file, it is quite easy to confirm whether your CPU is capable of 64bit or not. All you have to do is look at the flags which tell you what instruction sets your CPU is capable of.

All  the CPUs on your system will have the same type and therefore support the same instruction sets, that’s why in this example the grep command returns 4 similar lines – for the 4 CPU instances found on my system:

newserver# grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr

What you’re looking for is the following flag: lm. It stands for X86_FEATURE_LM, the Long Mode (64bit) support. If you can find the “lm” flag among your CPU flags, this means you’re looking at a 64bit capable processor.

As you can see from the example about, the processors in question are 64bit capable.

And this is how /proc/cpuinfo will look on an older system with only 32bit capable CPUs, there’s no lm flag as you can see:

oldserver# grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm

That’s it, hope this little tip helps you next time you’re puzzled about your CPU capabilities!

See also:




Use /proc/version to identify your Linux release

Hi everyone, I’m finally back from my holidays, and simply cannot wait to share some more Unix tips with all of you!

Today I’ll talk a bit more about yet another way of learning version information about your Linux OS: the /proc/version file. I mentioned it briefly in one of the previous posts, but would like to finish the explanations.

What you can learn from /proc/version

This file will not show you the name of the actual OS release, but will instead give you specifics about the version of Linux kernel used in your distribution, and confirm the version of a GCC compiler used to build it.

If you cat the /proc/version file, this is what you’re going to see (I’m using a RedHat 5.2 system for this):

rhel52# cat /proc/version
Linux version 2.6.18-92.el5 ([email protected]) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-41)) #1 SMP Tue Apr 29 13:16:15 EDT 2008

In this output, you get to see the following information:

  1. Exact version of the Linux kernel used in your OS: Linux version 2.6.18-92.el5
  2. Name of the user who compiled your kernel, and also a host name where it happened: [email protected]
  3. Version of the GCC compiler used for building the kernel: gcc version 4.1.2 20071124
  4. Type of the kernel – SMP here means Symmetric MultiProcessing kernel, the one that supports systems with multiple CPUs or multiple cpu cores
  5. Date and time when the kernel was built: Tue Apr 29 13:16:15 EDT 2008

It’s absolutely normal that the kernel is older than the overall release of yours. My example above, generated on the RedHat Enterprise Linux 5.2 system (RHEL5.2), shows the kernel birthday to be Apr 29, 2008. But the actual RHEL5.2 release became available to all the customers only a month later, on May 21st 2008 (here’s the original RedHat 5.2 announcement).

The reason your kernel is a bit older than the rest of the distribution is because kernel is only one part of the final product you’re getting – it may take a while to compile and integrate the rest of the OS before it can be used.

Different ways to find out Linux release information

By now, you should know quite a few ways of confirming release information about your Linux distro. Just to remind you, here they are:

This should be more than enough even for the most curious Linux users. Enjoy!

See also:




How To Monitor Linux Memory Usage with Watch Command

Hi all, today I’m going to teach you not one, but two really cool things in one post! First, I’ll introduce you to advanced memory usage stats available on Linux systems through /proc/meminfo file, and then I’ll explain the basics of using the watch command.

Memory usage with /proc/meminfo

As you know, quite a few Unix-like systems use the so-called pseudo file systems like /proc. It’s not a real filesystem, but just a convenient representation of processes managed by your Unix OS. In Linux systems, this directory also contains quite a few files allowing you to access various information about your system. /proc/meminfo is one of such files, it gives you access to most of the memory usage stats.

To get a snapshot of the current state of memory usage on your Linux system, simply cat the /proc/meminfo file:

ubuntu$ cat /proc/meminfo
MemTotal:       523008 kB
MemFree:         35336 kB
Buffers:         85560 kB
Cached:         137220 kB
SwapCached:      24480 kB
Active:         327420 kB
Inactive:        91308 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       523008 kB
LowFree:         35336 kB
SwapTotal:     1048568 kB
SwapFree:       998960 kB
Dirty:             504 kB
Writeback:           0 kB
Mapped:         212232 kB
Slab:            39140 kB
CommitLimit:   1310072 kB
Committed_AS:   655992 kB
PageTables:       4748 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       628 kB
VmallocChunk: 34359737739 kB

This probably gives you more information about memory usage that you’ll ever want to know, but there’s quite a few really useful stats there like the MemFree or SwapFree ones, they’re useful for making sure your OS environment is healthy enough in terms of having plenty of free memory for efficient operation.

Using watch command to track progress

watch command is a really neat tool which does a simple but incredibly useful thing: it repeatedly runs a given command line and shows you the output. So, you’re effectively monitoring a progress of some process by watching relevant files.

The default interval is 2, which gives enough dynamics for most of the needs.

Here’s how you use this command:

ubuntu$ watch cat /proc/meminfo

So it’s the same command we used in previous example, cat /proc/meminfo, but this time we’re asking the watch command to re-run the command every 2 seconds and show us the output.

The result of running a watch command is going to be a constantly refreshed console showing something like this:

Every 2.0s: cat /proc/meminfo       Fri Feb 13 03:51:01 2009

MemTotal:       523008 kB
MemFree:         46396 kB
Buffers:         82636 kB
Cached:         131044 kB
SwapCached:      24480 kB
Active:         308512 kB
Inactive:        99372 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       523008 kB
LowFree:         46396 kB
SwapTotal:     1048568 kB
SwapFree:       998960 kB
Dirty:             832 kB
Writeback:           0 kB
Mapped:         211076 kB
Slab:            39132 kB
CommitLimit:   1310072 kB
Committed_AS:   654860 kB
PageTables:       4856 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       628 kB
VmallocChunk: 34359737739 kB

This output gets refreshed every 2 seconds, so the numbers shown are constantly updated.

That’s it for today! There are limitless possibilities for monitoring various processes using watch command and I’ll be sure to cover them in the future, but for now – have a great weekend and hope Friday 13th turns out great!

See also:




How To Change Ownership of Files and Directories in Unix

I’ve just been asked a question about changing the ownership of files from one Unix user to another, and thought it probably makes sense to have a quick post on it.

File ownership in Unix

Just to give you a quick reminder, I’d like to confirm that every single file in Unix belongs to some user and some group. There simply isn’t a way to create a file without assigning ownership. I’ve briefly touched the topic of confirming file ownership in Unix before, so today I will simply build on that and show you how to change ownership of files.

Here’s a setup for today: I have created a temporary directory with a few files and made myself the owner of all the files:

ubuntu$ ls -al /home/greys/example/
total 12
drwxr-xr-x  3 greys admin 4096 Feb  9 03:55 .
drwxr-xr-x 13 greys greys 4096 Feb  9 03:54 ..
drwxr-xr-x  2 greys admin 4096 Feb  9 03:55 dir1
-rw-r--r--  1 greys admin    0 Feb  9 03:54 file1
-rw-r--r--  1 greys admin    0 Feb  9 03:55 file2

As you can see from this listing, the owner (third field in each line) is my username – greys. The next field is a Unix group of each file’s owner – admin in my example.

Changing owner of a file in Unix

Changing file ownership means only updating the association between a Unix user and a file, and nothing else. When you’re changing the owner of a file, no data contained in a file is changed.

To change the owner of a file, you need to use the chown command (easy enough to remember: CHange OWNer – chown), with the following syntax:

ubuntu$ chown nobody file1

In this command, nobody is the username of the new owner for a list of files. In my example, the only file we’d like to change ownership for is file1.

It is important to realize that you can only change file ownership as a super-user (root). Any regular Unix user cannot change the ownership of any file, and I’d like to explain why.

Indeed, some people are surprised: if I’m the owner of a given file, why can’t I change the ownership for it? That’s because transferring the ownership will mean some other Unix user will become the owner of the file(s) in question. So changing ownership is like making a decision not only for yourself, but for the new owner of the files.This is only something a super-user – special administrative account in Unix – can do.

The same logic applies to other people not being able to become owners of your files, even if they’re willing to assume the new responsibilities of owning files. They cannot revoke your ownership, because each Unix user is only allowed to make decisions and take actions on his/her own behalf.

That’s why you will probably see an error like this if you attempt to change ownership of a file as your own regular Unix user:

ubuntu$ id
uid=1000(greys) gid=113(admin) groups=33(www-data),113(admin)
ubuntu$ chown nobody file1
chown: changing ownership of `file1': Operation not permitted

But if we become root:

ubuntu$ sudo -i
[sudo] password for greys:
ubuntu#

… we’ll have no problem changing owners for any files:

ubuntu# cd /home/greys/example
ubuntu# chown nobody file1
ubuntu# ls -l file1
-rw-r--r-- 1 nobody admin 0 Feb  9 03:54 file1

Changing owner for multiple files

If you’re going to change owner of a few files, this can easily be done using either a full list of files or a mask.

First, here’s an example of updating ownership for a specified list of files (and as you can see, directories as well):

ubuntu# chown nobody file2 dir1
ubuntu# ls -al
total 12
drwxr-xr-x  3 greys  admin 4096 Feb  9 03:55 .
drwxr-xr-x 13 greys  greys 4096 Feb  9 03:54 ..
drwxr-xr-x  2 nobody admin 4096 Feb  9 03:55 dir1
-rw-r--r--  1 nobody admin    0 Feb  9 03:54 file1
-rw-r--r--  1 nobody admin    0 Feb  9 03:55 file2

IMPORTANT: here’s one thing which is often forgotten: when you’re changing an owner of a directory, this DOES NOT automatically change owner of all the files which already exist in this directory. So, if we check the file3 in dir1 after the example above, we can see that even though dir1 now belongs to user nobody, file3 in it still belongs to me:

ubuntu# ls -l dir1/file3
-rw-r--r-- 1 greys admin 0 Feb  9 03:55 dir1/file3

If your intention is to change ownership of all the files and directories of a certain location in your filesystem, you need to use a -R option of the chown command, which means recursive ownership change:

ubuntu# chown -R nobody dir1
ubuntu# ls -l dir1/file3
-rw-r--r-- 1 nobody admin 0 Feb  9 03:55 dir1/file3

And just to further demonstrate this, I’m going to change owner of all the files and directories in /home/greys/example directory back to my own username, greys:

ubuntu# chown -R greys /home/greys/example/
ubuntu# ls -l /home/greys/example/
total 4
drwxr-xr-x 2 greys admin 4096 Feb  9 03:55 dir1
-rw-r--r-- 1 greys admin    0 Feb  9 03:54 file1
-rw-r--r-- 1 greys admin    0 Feb  9 03:55 file2

Changing group ownership for a file

Similar to the chown command, there’s a command specifically helping you with changing not the owner (user) of a file.

IMPORANT: unlike chown command, chgrp can be used by non-privileged (regular) users of a system. So you don’t have to be root if you want to change a group ownership for some of your files, provided that you’re changing the ownership to a group you’re a member of.

For example, I’m a member of quite a few groups on one of my Ubuntu servers:

ubuntu$ id greys
uid=1000(greys) gid=1000(greys) groups=1000(greys),4(adm),20(dialout),24(cdrom),46(plugdev),114(lpadmin),115(sambashare),116(admin)

Now, if I create a new file, it will by default belong to my primary group (called greys, just like my username):

ubuntu$ touch file
ubuntu$ ls -al file
-rw-r--r-- 1 greys greys 0 2012-09-20 10:48 file

I can now change group ownership of this file, in this case to a group admin, which I’m also part of.

ubuntu$ chgrp admin file

and this is just to confirm that the change actualyl happened:

ubuntu$ ls -al file
-rw-r--r-- 1 greys admin 0 2012-09-20 10:48 file

That’s it for today, good luck with changing file owners on your Unix system!

Recommended books:

See also: