Unix/Linux tutorials for beginners
Безопасный ввод данных в скриптах BASH
I needed to create a very simple bash script that required a password to be provided using keyboard and then used further in the script. Turns out, there's just a way to do it with the bash built-in function called read.
[Read more…] about Secure Keyboard Input in bash Scripts
Now and then I have to find some information in text files – logs, config files, etc. with a unique enough approach: I have a hostname or parameter name to search by, but the actual information I'm interested in is not found in the same line. Simple grep will just not do – so I'm using one of the really useful command line options for the grep command.
By default, grep shows you just the lines of text files that match the parameter you specify.
I'm looking for the IP address of the home office server called "server" in .ssh/config file.
Here's what default grep shows me:
[email protected]:~ $ grep server .ssh/config Host server
So it's useful in a sense that this confirms that I indeed have a server profile defined in my .ssh/config file, but it doesn't show any more lines of the config so I can't really see the IP address of my "server".
This is where the -A option (A – After) for grep becomes really useful: specify number of lines to show in addition to the line that matches search pattern and enjoy the result!
I'm asking grep to show me 2 more lines after the "server" line:
[email protected]:~ $ grep -A2 server .ssh/config Host server HostName 192.168.1.55 ForwardAgent yes
Super simple but very powerful way of using grep. I hope you like it!
Back to basics day: a very simple tip on working with one of the most useful things available in Unix and Linux filesystems: symbolic links (symlinks).
When using long-form ls output, symlinks will be shown like this:
[email protected]:~/proj/unixtutorial $ ls -la
total 435736
-rwxr-xr-x 1 greys staff 819 14 Dec 2018 .gitignore
drwxr-xr-x 3 greys staff 96 20 Nov 09:27 github
drwxr-xr-x 4 greys staff 128 20 Nov 10:58 scripts
lrwxr-xr-x 1 greys staff 30 10 Dec 20:40 try -> /Users/greys/proj/unixtutorial
drwxr-xr-x 44 greys staff 1408 20 Nov 10:54 wpengine
Show Symlink Destination with readlink
As with pretty much everything in Unix, there's a simple command (readlink) that reads a symbolic link and shows you the destination it's pointing to. Very handy for scripting:
[email protected]:~/proj/unixtutorial $ readlink try /Users/greys/proj/unixtutorial
It has a very basic syntax: you just specify a file or directory name, and if it's a symlink you'll get the full path to the destination as the result.
If readlink returns nothing, this means the object you're inspecting isn't a symlink at all. Based on the outputs above, if I check readlink for the regular scripts directory, I won't get anything back:
[email protected]:~/proj/unixtutorial $ readlink scripts
[email protected]:~/proj/unixtutorial $
New Live Chat at Unix Tutorial feature I launched over the weekend is great! Even a few hours I'm online each moring and each evening are giving me plenty of opportunities to meet visitors and to help with Unix questions. Today I discussed awk fields delimiter with someone, speicifcally the part of showing the last field in a string.
Awk has a number of built-in variables. NF is one of them – it shows the total number of https://www.unixtutorial.org/awk-field-separatorfields in a given string that awk just finished parsing. Used with the $ macro, NF becomes $NF and gives you the awk field number NF – which is the last field in an awk string.
If we have a DIRNAME variable with a typical full path, like this:
[email protected]:~ $ DIRNAME=/Users/greys/proj/unixtutorial [email protected]:~ $ echo $DIRNAME /Users/greys/proj/unixtutorial
… then it's possible to get the name of the last subdirectory in DIRNAME using awk and $NF:
[email protected]:~ $ echo $DIRNAME | awk -F\/ '{print $NF}'
unixtutorial
That's it – as simple as that!
I'm slowly improving my Python skills, mostly by Googling and combining multiple answers to code a solution to my systems administration tasks. Today I decided to write a simpe converter that takes Epoch Time as a parameter and returns you the time and date it corresponds to.
Most of the functionality is done using the fromtimestamp function of the datetime module. But because I also like seeing time in UTC, I decided to use timezone module as well.
#!/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
import sys
from datetime import datetime, timezone
if len(sys.argv)>1:
print ("This is the Epoch time: ", sys.argv[1])
try:
timestamp = int(sys.argv[1])
if timestamp>0:
timedate = datetime.fromtimestamp(timestamp)
timedate_utc = datetime.fromtimestamp(timestamp, timezone.utc)
print ("Time/date: ", format(timedate))
print ("Time/date in UTC: ", format(timedate_utc))
except ValueError:
print ("Timestamp should be a positive integer, please.")
else:
print ("Usage: epoch.py <EPOCH-TIMESTAMP>")
FIXME: I'll revisit this to re-publish script directly from GitHub.
Here's how you can use the script:
[email protected]:~/proj/python $ ./epoch.py 1566672346 This is the Epoch time: 1566672346 Time/date: 2019-08-24 19:45:46 Time/date in UTC: 2019-08-24 18:45:46+00:00
I implemented basic checks:
Do you see anything that should be changed or can be improved? Let me know!
Removing files and directories is a very common task, so in some environments support engineers or automation scripts delete hundreds of files per day. That's why I think it's important to be familiar with different ways and safety mechanisms you should use when it comes to removing Unix directories. This article has a number of principles that should help you make your day-to-day files and directories operations safer.
[Read more…] about Advice: Safely Removing a File or Directory in Unix
Last night I finished a fun mini project as part of Unix Tutorials Projects. I have writted a basic enough script that can be added as root cronjob for automatically controlling keyboard backlight on my Dell XPS 9380.
As I've written just a couple of days ago, it's actually quite easy to turn keyboard backlight on or off on a Dell XPS in Linux (and this probably works with other Dell laptops).
Armed with that knowledge, I've written the following script:
#!/bin/bash
WORKDIR=/home/greys/scripts/backlight
LOCKFILE=backlight.kbd
LOGFILE=${WORKDIR}/backlight.log
KBDBACKLIGHT=`cat /sys/devices/platform/dell-laptop/leds/dell::kbd_backlight/brightness`
HOUR=`date +%H`
echo "---------------->" | tee -a $LOGFILE
date | tee -a $LOGFILE
if [ $HOUR -lt 4 -o $HOUR -gt 21 ]; then
echo "HOUR $HOUR is rather late! Must turn on backlight" | tee -a $LOGFILE
BACKLIGHT=3
else
echo "HOUR $HOUR is not too late, must turn off the backlight" | tee -a $LOGFILE
BACKLIGHT=0
fi
if [ $KBDBACKLIGHT -ne $BACKLIGHT ]; then
echo "Current backlight $KBDBACKLIGHT is different from desired backlight $BACKLIGHT" | tee -a $LOGFILE
FILE=`find ${WORKDIR} -mmin -1440 -name ${LOCKFILE}`
echo "FILE: -$FILE-"
if [ -z "$FILE" ]; then
echo "No lock file! Updating keyboard backlight" | tee -a $LOGFILE
echo $BACKLIGHT > /sys/devices/platform/dell-laptop/leds/dell::kbd_backlight/brightness
touch ${WORKDIR}/${LOCKFILE}
else
echo "Lockfile $FILE found, skipping action..." | tee -a $LOGFILE
fi
else
echo "Current backlight $KBDBACKLIGHT is the same as desired... No action needed" | tee -a $LOGFILE
fi
This is what my script does when you run it as root (it won't work if you run as regular user):
Log file looks like this:
[email protected]:~/scripts $ tail backlight/backlight.log ----------------> Tue May 28 00:10:00 BST 2019 HOUR 00 is rather late! Must turn on backlight Current backlight 2 is different from desired backlight 3 Lockfile /home/greys/scripts/backlight/backlight.kbd found, skipping action... ----------------> Tue May 28 00:15:00 BST 2019 HOUR 00 is rather late! Must turn on backlight Current backlight 2 is different from desired backlight 3 Lockfile /home/greys/scripts/backlight/backlight.kbd found, skipping action...
I have added this script to the root user's cronjob. In Ubuntu 19.04 running on my XPS laptop, this is how it was done:
[email protected]:~/scripts $ sudo crontab -e [sudo] password for greys:
I then added the following line:
*/5 * * * * /home/greys/scripts/backlight.sh
Depending on where you place similar script, you'll need to update full path to it from /home/greys/scripts. And then update WORKDIR variable in the script itself.
Here are just a few things I plan to improve:
I'm decorating my tmux setup and needed to confirm colour numbers for some elements of the interface. Turns out, it's simple enough to show all the possible colours with a 1-liner in your favourite Unix shell – bash shell in my case.
I'll explain how this works in full detail sometime in a separate post, but for now will just give you an example and show how it works:
So, in this example, this is how we achieve colorized text output:
Here's how we get the colours now: we create a loop from 1 until 255 (0 will be black) and then use the ESC syntax changing colour to $COLOR variable value. We then output the $COLOR value which will be a number:
for COLOR in {1..255}; do echo -en "\e[38;5;${COLOR}m${COLOR} "; done; echo;
Here's how running this will look in a propertly configured 256-color terminal:
Here's the same 1-liner converted into proper script for better portability and readability:
#!/bin/bash
for COLOR in {1..255}; do
echo -en "\e[38;5;${COLOR}m"
echo -n "${COLOR} "
done
echo
If you save this as bash-256-colours.sh and chmod a+rx bash-256-colours.sh, you can now run it every time you want to refresh your memory or pick different colours for some use.
If you're using CentOS, Fedora or Red Hat Linux, you are probably familiar with the yum package manager. One of the really useful options for yum is checking whether there are any available updates to be installed.
If you use check-update parameter with yum, it will show you the list of any available updates:
[email protected]:~ # yum check-update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: rep-centos-fr.upress.io * epel: mirror.in2p3.fr * extras: rep-centos-fr.upress.io * updates: ftp.pasteur.fr ansible.noarch 2.7.8-1.el7 epel datadog-agent.x86_64 1:6.10.1-1 datadog libgudev1.x86_64 219-62.el7_6.5 updates nginx.x86_64 1:1.15.9-1.el7_4.ngx nginx oci-systemd-hook.x86_64 1:0.1.18-3.git8787307.el7_6 extras polkit.x86_64 0.112-18.el7_6.1 updates systemd.x86_64 219-62.el7_6.5 updates systemd-libs.i686 219-62.el7_6.5 updates systemd-libs.x86_64 219-62.el7_6.5 updates systemd-python.x86_64 219-62.el7_6.5 updates systemd-sysv.x86_64 219-62.el7_6.5 updates
One thing that I didn't know and am very happy to discover is that yum check-update is actually meant for shell scripting. It returns a specific code after running, you can use the value to decide what do to next.
As usual: return value 0 means everything is fully updated, so no updates are available (and no action is needed). A value of 100 would mean you have updates available.
All we need to do is check the return value variable $? for its value in something like this:
#!/bin/bash
yum check-update
if [ $? == 100 ]; then
echo "You've got updates available!"
else
echo "Great stuff! No updates pending..."
fi
Here is how running this script would look if we saved the script as check-yum-updates.sh script:
[email protected]:~ # ./check-yum-updates.sh Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: rep-centos-fr.upress.io * epel: mirror.in2p3.fr * extras: rep-centos-fr.upress.io * updates: ftp.pasteur.fr ansible.noarch 2.7.8-1.el7 epel datadog-agent.x86_64 1:6.10.1-1 datadog libgudev1.x86_64 219-62.el7_6.5 updates nginx.x86_64 1:1.15.9-1.el7_4.ngx nginx oci-systemd-hook.x86_64 1:0.1.18-3.git8787307.el7_6 extras polkit.x86_64 0.112-18.el7_6.1 updates systemd.x86_64 219-62.el7_6.5 updates systemd-libs.i686 219-62.el7_6.5 updates systemd-libs.x86_64 219-62.el7_6.5 updates systemd-python.x86_64 219-62.el7_6.5 updates systemd-sysv.x86_64 219-62.el7_6.5 updates You've got updates available!
I'll revisit this post soon to show you a few more things that can be done with yum check-update functionality.
pwd command, and you probably know, reports the current working directory in your Unix shell. PWD stands for Present Working Directory. In addition to pwd, there's also a special variable – one of the user environment variables – called PWD, that you can also use.
Just to remind you, pwd command is a super simple way of confirming where you are in the filesystem tree.
Usually, your shell session starts in your home directory. For me and my username greys, this means /home/greys in most distributions:
[email protected]:~$ pwd /home/greys
If I then use cd command to visit some other directory, pwd command will help me confirm it:
[email protected]:~$ cd /tmp [email protected]:/tmp$ pwd /tmp
Most Unix shells have PWD as a variable. It reports the same information as pwd command but saves children processes the trouble of using pwd command and getpwd system call just to confirm the working directory of their parent process.
So, you can just do this to confirm $PWD value:
[email protected]:/tmp$ echo $PWD /tmp
… which really helps in shell scripting, cause you can do something like this:
#!/bin/bash echo "Home directory: $HOME" echo "Current directory: $PWD" if [ $HOME != $PWD ]; then echo "You MUST run this from your home directory!" exit else echo "Thank you for running this script from your home directory." fi
When we run this, the script will compare standard $HOME variable (your user's homedir) to the $PWD variable and will behave differently if they match.
I've created and saved pwd.sh in my projects directory for bash scripts: /home/greys/proj/bash:
[email protected]:~/proj/bash$ ./pwd.sh Home directory: /home/greys Current directory: /home/greys/proj/bash You MUST run this from your home directory!
If I now change back to my home directory:
[email protected]:~/proj/bash$ cd /home/greys/
… the script will thank me for it:
[email protected]:~$ proj/bash/pwd.sh Home directory: /home/greys Current directory: /home/greys Thank you for running this script from your home directory.
Have fun using pwd command and $PWD variable in your work and shell scripting!
