Colorized ls with grc

Colorized output of ls command

I blogged about Generic Colouriser (grc) last week, cause I’m using it now to monitor syslog messages in my centralised RSyslog setup. I also mentioned that grc suppors many standard commands in addition to parsing common types of log files.

Colorized ls Output

Many Linux distros and even macOS support colorized file listing of the ls command, etc. Here’s how it usually looks:

Colorized ls Output with grc

Compare above example to how grc colorizes the same list of files:

Colorized ls with grc

Obviously, focus is on file permissions and ownership info.

I really like this, must be of great use for those of us just getting familiar with file/directory permissions in Unix/Linux.

Have fun!

See Also




Unix Input/Output Redirection

Error Output STDERR Redirection

Input/output redirection is a fundamental functionality in Unix and Linux. It’s a great way of manipulating data exchange between commands that you run.

There’s lots of examples here and this probably calls for a Unix Input/Output Redirection Reference page (I’ll create it soon).

Today I just want to show you an example of using input/output redirection – follow my steps and let me know if you get the same results. Any questions – please get in touch and I’ll update this port and the Unix Input/Output Redirect reference.

Standard Output Redirect

Let’s say we want to create a simple text file with a message “Hello”. One way to do this would be to output the Hello message using echo command, and then to redirect its standard output using the > redirection:

greys@maverick:~ $ echo Hello > /tmp/try.hello

Basic use for redirection is: you run any commands you like, and then finish the command line with the > sign that invokes redirection, and specify the file where redirection should be written to.

USEFUL: Such standard output is called STDOUT.

If we check contents of the /tmp/try.hello file using cat command now, we can see our Hello:

greys@maverick:~ $ cat /tmp/try.hello
Hello

Since we can redirect output of any commands like this, we can redirect the result of this cat /tmp/try.hello command into another file, perhaps /tmp/try.hello2, and it will then also contain our Hello message:

greys@maverick:~ $ cat /tmp/try.hello > /tmp/try.hello2 greys@maverick:~ $ cat /tmp/try.hello2 Hello

Standard Input Redirect

Similar to Standard Output, there’s also a Standard Input – called STDIN. What it does is sources content of a specified file to use it for input to whatever you’re running.

So we use the < sign to redirect (take) input from a file. For instance:

greys@maverick:~ $ cat < /tmp/try.hello
Hello

Now, this is the simplest example and not the most useful one: most commands in Unix expect input file as one of the first parameters anyway. So we don’t really have to forward input like this – we could just run “cat /tmp/try.hello“.

But it’s important to recognise the difference: in this example with STDIN redirection above, cat command is not aware of any input files. It’s run without parameters and as such expects someone to type input or source it using redirection just like we did.

Standard Error Output Redirect

Now, what happens if the command you run generates an error? That’s not a standard command behaviour or standard output. It’s an error message, or standard error output: STDERR.

What this mean is that Unix/Linux is rather clever – error messages will be treated as a separate error destination. So even though in your console you’ll get both errors and standard output, the redirection will treat them separately.

Here’s an example. I’m trying to cat a non-existent file:

greys@maverick:~ $ cat /tmp/try.hello3
cat: /tmp/try.hello3: No such file or directory

This “cat: /tmp/try.hello3: No such file or directory” is an error message, not the standard output. That’s why, when I’m redirecting it to a file using standard output redirection, nothing is captured and put into the redirection output file:

greys@maverick:~ $ cat /tmp/try.hello3 > /tmp/redirected.out
cat: /tmp/try.hello3: No such file or directory
greys@maverick:~ $ cat /tmp/redirected.out
greys@maverick:~ $

Pretty cool, huh?

In order to redirect specifically error messages, we need to use special form of redirection, for STDERR error messages. We use number 2 before the redirection symbol, which refers to STDERR:

greys@maverick:~ $ cat /tmp/try.hello3 2> /tmp/redirected.out greys@maverick:~ $ cat /tmp/redirected.out 
cat: /tmp/try.hello3: No such file or directory 

Two things happened:

  1. Our command returned no output. Because all of the result (standard error it generated) got forwarded to the /tmp/redirected.out file
  2. The /tmp/redirected.out file now contains our error message

Think this is enough for one post. Will copy most of this into the Unix redirects reference page and come back some other day with more on this.

Have fun!

See Also




Show Next Few Lines with grep

Showing next few lines with grep

Now and then I have to find some information in text files – logs, config files, etc. with a unique enough approach: I have a hostname or parameter name to search by, but the actual information I’m interested in is not found in the same line. Simple grep will just not do – so I’m using one of the really useful command line options for the grep command.

Default grep Behaviour

By default, grep shows you just the lines of text files that match the parameter you specify.

I’m looking for the IP address of the home office server called “server” in .ssh/config file.

Here’s what default grep shows me:

greys@maverick:~ $ grep server .ssh/config
Host server

So it’s useful in a sense that this confirms that I indeed have a server profile defined in my .ssh/config file, but it doesn’t show any more lines of the config so I can’t really see the IP address of my “server”.

Show Next Few Lines with grep

This is where the -A option (A – After) for grep becomes really useful: specify number of lines to show in addition to the line that matches search pattern and enjoy the result!

I’m asking grep to show me 2 more lines after the “server” line:

greys@maverick:~ $ grep -A2 server .ssh/config
Host server
     HostName 192.168.1.55
     ForwardAgent yes

Super simple but very powerful way of using grep. I hope you like it!

See Also




How To Change Mac Hostname with scutil

Changing hostname with scutil in macOS

Standard command like hostname will still work in macOS, but perhaps it’s best to use the native way of updating system information on your Mac? scutil command is here to help.

Change Hostname with scutil

Simply run this command in your Terminal window:

greys@macbook:/ $ sudo scutil --set HostName "maverick"

if you now start new Terminal window or even just start new shell, you should see the new hostname:

greys@macbook:/ $ sudo scutil –set HostName “maverick”
greys@macbook:/ $ bash
greys@maverick:/ $ hostname
maverick

That’s it for today!

See Also




Ubuntu – Static IP

I’m finishing up the VM based Ubuntu 19.10 setup and decided to make a few screenshots to test new workflow. Instead of random topic, I documented the task of configuring Ubuntu to use static IP address.

Why use Static IP?

Desktops don’t benefit from this much, but servers benefit from having static (fixed) IP addresses because you configure DNS once and then use server hostname and same fixed IP address for using any services.

Even if all you need is connect to server via SSH, static IP address will make things much easier – plus it usually comes as default for most dedicated hosting plans.

In my home lab I prefer to configure most virtual machines with static IP addresses because this allows me to easily access them remotely (working from one of the laptops).

Things to Know Before Configuring Static IP

Assuming you’re going to assisg a static IP from your local network (LAN), you need to have the following information confirmed:

  • what’s the local network addressing in use? It most likely starts with 192.168 or 10.0, but you need to know for sure. You need the full network – so something like 192.168.0.x or 192.168.1.x
  • static IP address you plan to use (from the same network)
    • make sure this IP is available – currently not in use (and at least does not answer to ping)
  • default route (sometimes called gateway) address – this is usually the IP address of your broadband modem. In my case it’s 192.168.1.1
  • DNS server IP address – usually the same as your gateway IP
    • you can also easily use external DNS server like 1.1.1.1 (CloudFlare) or 8.8.8.8 (Google)

Configure Ubuntu 19.10 for Static IP

Here are the screenshots walking you through the static IP configuration:

Go to app listing:

… choose Settings:

and select Network:

Inspect current settings (they would be automatic IP address and DNS details via DHCP):

…and now go to IPv4 settings:

Now, select Manual option for IPv4 method:

…and provide new values:

Now click Apply (top right) and in the next screen turn the Wired network off:

Now turn it back on and you can review the result:

That’s it – it’s a rather simple procedure but I still find it more comfortable (and easier to manage) when I update configuration directly or via Ansible.

See Also




How To Confirm Symlink Destination

Today I show you how to confirm symlink destination using readlink command.
Showing symlink destination

Back to basics day: a very simple tip on working with one of the most useful things available in Unix and Linux filesystems: symbolic links (symlinks).

How Symlinks Look in ls command

When using long-form ls output, symlinks will be shown like this:

greys@mcfly:~/proj/unixtutorial $ ls -la
total 435736
-rwxr-xr-x 1 greys staff 819 14 Dec 2018 .gitignore
drwxr-xr-x 3 greys staff 96 20 Nov 09:27 github
drwxr-xr-x 4 greys staff 128 20 Nov 10:58 scripts
lrwxr-xr-x 1 greys staff 30 10 Dec 20:40 try -> /Users/greys/proj/unixtutorial
drwxr-xr-x 44 greys staff 1408 20 Nov 10:54 wpengine

Show Symlink Destination with readlink

As with pretty much everything in Unix, there’s a simple command (readlink) that reads a symbolic link and shows you the destination it’s pointing to. Very handy for scripting:

greys@mcfly:~/proj/unixtutorial $ readlink try
/Users/greys/proj/unixtutorial

It has a very basic syntax: you just specify a file or directory name, and if it’s a symlink you’ll get the full path to the destination as the result.

If readlink returns nothing, this means the object you’re inspecting isn’t a symlink at all. Based on the outputs above, if I check readlink for the regular scripts directory, I won’t get anything back:

greys@mcfly:~/proj/unixtutorial $ readlink scripts
greys@mcfly:~/proj/unixtutorial $

See Also




more or less

Unix Tutorial

I’m working hard to finish off the Basic Unix Commands section of this website – want to have every command there documented before I start recording videos about them. more and less are the most recent additions.

Both more and less are basic commands for viewing text files. Quite often they’re used together with cat command (you cat a file and pipe it into more or less), but that’s actually not necessary – you can get the same result by just using more or less on their own.

more command

more command helps you browse text files in Unix and Linux. Instead of throwing complete contents of the possibly large text file into terminal window, more shows you paginated content – fills up the screen and then waits for a key press to show the next screen’s worth of content.

more command showing /etc/lsb-release file

less command

less command continues traditions of a simple text file viewer of the more command (in many ways, less IS more) but does it with a number of key optimisations.

less command showing /etc/lsb-release file

more or less?

more is a much simpler command and will work on a relatively broken system – no complicated terminal emulation, no possibly broken Up/Down key sequences, etc. more will simply keep showing text output into your terminal session without any built-in scroll-back functionality.

less is more flexible but also expects terminal emulation to be operational for arrow-keys navigation to work. The advantage is you have a fully functional scroll-back.

more is great for small files, but it needs to read the complete text file before showing its paginated contents. So for large (hundreds of megabyes) files, less will be much faster as it only reads the part necessary to show on the screen (and reads more as you keep scrolling).

See Also




Sort Files by Size with ls

Sorting ls output by file size

I love reading man pages for even the most basic Unix commands like ls, because there’s always something interesting to learn. Today I discovered that it’s possible to sort ls output by file size.

Sort ls by file size

Simply add the capital S to command line options and you’ll see your output sorted by file size. Be sure to use long output form (-l option). I’m adding more command to the mix because I know that /usr/lib is a directory with quite a number of files:

greys@srv:/usr/bin $ ls -lS /usr/lib | more
total 492
-rwxr-xr-x  1 root 74688 Feb  1  2019 klibc-ae-2A4n9ZnfImcw9WIt7dF-3OvQ.so
-rw-r--r--  1 root 56480 Jan 14  2018 libdiscover.so.2.0.1
-rw-r--r--  1 root 22448 Aug 29  2016 libsupp.a
drwxr-xr-x 27 root 20480 Nov 24 01:00 python2.7
drwxr-xr-x 24 root 20480 Nov 26 22:12 x86_64-linux-gnu
-rw-r--r--  1 root 18336 Feb 26  2019 libau.so.2.10
drwxr-xr-x  3 root 12288 Sep 14 18:43 git-core
drwxr-xr-x 30 root 12288 Sep 14 18:44 python3.7
drwxr-xr-x 15 root 12288 Nov 26 22:12 systemd
drwxr-xr-x  2 root  4096 Aug 30 10:03 apparmor
drwxr-xr-x  5 root  4096 Aug 30 10:02 apt
drwxr-xr-x  2 root  4096 Aug 30 10:05 bfd-plugins

Reverse Sorting with ls command

Don’t know about you, but I prefer reverse sorting in most commands – this means largest objects are shown at the end of the output. The reason I do it this way is to avoid using pagers like more or less. Another reason is I can scroll up to explore further output of the ls command without having to rerun anything.

So here’s the same command run with the -r option for reverse output (I’m only showing last few lines):

...
drwxr-xr-x  2 root  4096 Aug 30 10:04 discover
drwxr-xr-x  2 root  4096 Aug 30 10:04 dbus-1.0
drwxr-xr-x  2 root  4096 Nov  3 20:02 console-setup
drwxr-xr-x  2 root  4096 Aug 30 10:05 compat-ld
drwxr-xr-x  2 root  4096 Nov 10 06:04 chrony
drwxr-xr-x  2 root  4096 May 24  2019 binfmt.d
drwxr-xr-x  2 root  4096 Aug 30 10:05 bfd-plugins
drwxr-xr-x  5 root  4096 Aug 30 10:02 apt
drwxr-xr-x  2 root  4096 Aug 30 10:03 apparmor
drwxr-xr-x 15 root 12288 Nov 26 22:12 systemd
drwxr-xr-x 30 root 12288 Sep 14 18:44 python3.7
drwxr-xr-x  3 root 12288 Sep 14 18:43 git-core
-rw-r--r--  1 root 18336 Feb 26  2019 libau.so.2.10
drwxr-xr-x 24 root 20480 Nov 26 22:12 x86_64-linux-gnu
drwxr-xr-x 27 root 20480 Nov 24 01:00 python2.7
-rw-r--r--  1 root 22448 Aug 29  2016 libsupp.a
-rw-r--r--  1 root 56480 Jan 14  2018 libdiscover.so.2.0.1
-rwxr-xr-x  1 root 74688 Feb  1  2019 klibc-ae-2A4n9ZnfImcw9WIt7dF-3OvQ.so
greys@srv:~ $

That’s it for today – hope you’ve learned something new today.

See Also




bat – syntax highlighter

I’m back online! Have been doing a bit of reading but was too busy to publish anything, so this week should be fun with lots of content.

bat command

bat is a simple tool that lets you show contents of plain text files with syntax highlighting. Using different colours and even colour themese, bat lets you read files much easier because you learn very quickly what to focus on based on the highlight colour.

Here’s how it differs from the standard cat command:

Installing bat in Ubuntu

I simply did this:

greys@xps: ~ $ sudo apt install bat

Basic syntax for bat command is the same as it is for cat command.

That’s it for today! Have fun!

See Also




chown example: recursive update

chown command

chown is a basic Unix command, super useful and very powerful. I have provided a chown example earlier, but would like to show another common way it’s used.

chown: update ownership recursively

chown command uses -R option to apply changes recursively.

For example, if I have a /Users/greys/unixtutorial directory with the following layout:

file1
file2
dir1/file3

… then it’s very easy to show the difference between chown and chown -R commands.

Let’s go to the /Users/greys/unixtutorial directory:

greys@maverick:~ $ cd /Users/greys/unixtutorial
greys@maverick:~/unixtutorial $

Now, let’s look at the files with the ls command:

greys@maverick:~/unixtutorial $ ls -al *
 -rw-r--r--  1 greys  staff  0  8 Oct 22:55 file1
 -rw-r--r--  1 greys  staff  0  8 Oct 22:55 file2
dir1:
 total 0
 drwxr-xr-x  3 greys  staff   96  8 Oct 22:55 .
 drwxr-xr-x  5 greys  staff  160  8 Oct 22:55 ..
 -rw-r--r--  1 greys  staff    0  8 Oct 22:55 file3

Everything belongs to my own user, greys and my primary group: staff.

Time to change ownership of everything in the current directory:

greys@maverick:~/unixtutorial $ sudo chown root:wheel *

Checking files with ls again, I can see that immediate contents of the /Users/greys/unixtutorial directory (where I was at the time of running chown) got updated ownership: file1 and file2, along with dir1, now belong to root:wheel.

But file3 was in a dir1 subdirectory, so it stayed intact and still belongs to greys:staff:

greys@maverick:~/unixtutorial $ ls -al *

-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file1

-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file2
dir1:
 total 0
 drwxr-xr-x  3 root   wheel   96  8 Oct 22:55 .
 drwxr-xr-x  5 root   wheel  160  8 Oct 22:55 ..
 -rw-r--r--  1 greys  staff    0  8 Oct 22:55 file3

That’s because without the -R (recursive) option, chown will only inspect and update files in the current directory, but not in any of its subdirectories (or their subdirectories, and so on).

Running the same command with -R option does the trick:

greys@maverick:~/unixtutorial $ sudo chown -R root:wheel *
greys@maverick:~/unixtutorial $ ls -al *
-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file1
-rw-r--r--  1 root  wheel  0  8 Oct 22:55 file2
dir1:
total 0
drwxr-xr-x  3 root  wheel   96  8 Oct 22:55 .
drwxr-xr-x  5 root  wheel  160  8 Oct 22:55 ..
-rw-r--r--  1 root  wheel    0  8 Oct 22:55 file3

Hope you learned something new, come back for more!

See Also