Review Latest Logs with tail and awk
Part of managing any Unix system is keeping an eye on the vital log files.
Today I was discussing one of such scenarios with a friend and we arrived at a pretty cool example involving awk command and eventually a bash command substitution.
Let’s say we have a directory with a bunch of log files, all constantly updated at different times and intervals. Here’s how I may get the last 10 lines of the output from the most recent log file:
root@vps1:/var/log# cd /var/log root@vps1:/var/log# ls -altr *log -rw-r--r-- 1 root root 32224 Jul 10 22:49 faillog -rw-r----- 1 syslog adm 0 Jul 25 06:25 kern.log -rw-r--r-- 1 root root 0 Aug 1 06:25 alternatives.log -rw-r--r-- 1 root root 2234 Aug 8 06:34 dpkg.log -rw-rw-r-- 1 root utmp 294044 Aug 15 22:32 lastlog -rw-r----- 1 syslog adm 12248 Aug 15 22:35 syslog -rw-r----- 1 syslog adm 5160757 Aug 15 22:40 auth.log
Ok, now we just need to get that filename from the last line (auth.log).
Most obvious way would be to use tail command to extract the last line, and awk to show the 9th parameter in that line – which would be the filename:
root@vps1:/var/log# ls -altr *log | tail -1 | awk '{print $9}'
auth.log
Pretty cool, but can be optimised using awk’s END clause:
root@vps1:/var/log# ls -altr *log | awk 'END {print $9}'
auth.log
Alright. Now we wanted to show the 10 lines of output, which we can use tail -10 for.
A really basic approach is to assing the result of the line we’re using to a variable in Bash, and then access that variable, like this:
root@vps1:/var/log# FILE=`ls -altr *log | tail -1 | awk '{print $9}'`
root@vps1:/var/log# tail -10 ${FILE}
Aug 15 22:40:37 vps1 sshd[26578]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=159.65.145.196
Aug 15 22:40:39 vps1 sshd[26578]: Failed password for invalid user Fred from 159.65.145.196 port 47934 ssh2
Aug 15 22:40:39 vps1 sshd[26578]: Received disconnect from 159.65.145.196 port 47934:11: Normal Shutdown, Thank you for playing [preauth]
Aug 15 22:40:39 vps1 sshd[26578]: Disconnected from 159.65.145.196 port 47934 [preauth]
Aug 15 22:41:15 vps1 sshd[26580]: Connection closed by 51.15.4.190 port 44958 [preauth]
Aug 15 22:42:02 vps1 sshd[26585]: Connection closed by 13.232.227.143 port 40054 [preauth]
Aug 15 22:43:23 vps1 sshd[26587]: Connection closed by 51.15.4.190 port 52454 [preauth]
Aug 15 22:44:08 vps1 sshd[26589]: Connection closed by 13.232.227.143 port 47542 [preauth]
Aug 15 22:45:01 vps1 CRON[26604]: pam_unix(cron:session): session opened for user root by (uid=0)
Aug 15 22:45:01 vps1 CRON[26604]: pam_unix(cron:session): session closed for user root
But an ever shorter (better?) way to do this would be to use the command substitution in bash: the output of a command becomes the command itself (or string value in our case).
Check it out:
root@vps1:/var/log# tail -10 $(ls -altr *log | tail -1 | awk '{print $9}')
Aug 15 22:42:02 vps1 sshd[26585]: Connection closed by 13.232.227.143 port 40054 [preauth]
Aug 15 22:43:23 vps1 sshd[26587]: Connection closed by 51.15.4.190 port 52454 [preauth]
Aug 15 22:44:08 vps1 sshd[26589]: Connection closed by 13.232.227.143 port 47542 [preauth]
Aug 15 22:45:01 vps1 CRON[26604]: pam_unix(cron:session): session opened for user root by (uid=0)
Aug 15 22:45:01 vps1 CRON[26604]: pam_unix(cron:session): session closed for user root
Aug 15 22:45:26 vps1 sshd[26610]: Connection closed by 51.15.4.190 port 59872 [preauth]
Aug 15 22:46:15 vps1 sshd[26612]: Connection closed by 13.232.227.143 port 55030 [preauth]
Aug 15 22:46:23 vps1 sshd[26608]: Connection closed by 18.217.190.140 port 40804 [preauth]
Aug 15 22:47:28 vps1 sshd[26614]: Connection closed by 51.15.4.190 port 39044 [preauth]
Aug 15 22:48:20 vps1 sshd[26616]: Connection closed by 13.232.227.143 port 34286 [preauth]
So in this example $(ls -altr *log | tail -1 | awk ‘{print $9}’) is a substituion – bash executes the command in the parenthesis and then passes the resulting value to further processing (becoming a parameter for the tail -10 command).
In our command above, we’re essentially executing the following command right now:
root@vps1:/var/log# tail -10 auth.log
only auth.log is always the filename of the log file that was updated the latest, so it could become syslog or dpkg.log if they’re updated before next auth.log update.