Just realised there’s been something I wanted to mention in my Unix Tutorial digest, but it appears a time limited deal so it won’t wait until the next edition. Please note I’m not paid for sharing these news and won’t be earning any affiliate comission from the links below.
Packt: Data Unlocked
Seems the month of September is pretty busy at Packt Publishing: every week there are lots of great books and educational videos made available for just $10 each.
If you’re considering a career in Data Science or just looking for a really interesting direction to learn something new – have a look at their data science section and perhaps grab a book or two!
or just looking for a really interesting direction to learn something new – have a look at their data science section and perhaps grab a book or two!
Really cool: there’s also a free book of the day at Packt, I think – just created my account to download one. This isn’t data science specific – so every day there is a different book.
CentOS 7.7 has just been released a few days ago and with any luck CentOS 8 will be released next week. I decided to quickly upgrade one of my dedicated servers from CentOS 7.6 to CentOS 7.7.
Confirm CentOS Version
Just a couple of steps to check what CentOS release we’re running:
root@s2:~ # cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
There should also be a package reflecting CentOS release already installed on your system. This package will be upgraded along with the rest of the OS when we’re stepping up to CentOS 7.7:
yum command has the check-update option for verifying if any packages are available for updating:
root@s2:~ # yum check-update
This will return a rather long list. To be super-sure we’ll actually get the CentOS 7.7 upgrade, look for the same centos-release package:
root@s2:~ # yum check-update | grep centos-release
centos-release.x86_64 7-7.1908.0.el7.centos base
Upgrade CentOS 7.6 to CentOS 7.7
We need the yum update command here. After you run it it will resolve dependencies and report something like this, prompting for your confirmation:
Transaction Summary
===============================================================
Install 3 Packages (+25 Dependent packages)
Upgrade 406 Packages
Total download size: 577 M
Is this ok [y/d/N]:
Sounds about right! We’ll press Y and let the server download and apply all the updates…
On my hosting it took about 1min to download all the packages!
Transaction Summary
===============================================================
Install 3 Packages (+25 Dependent packages)
Upgrade 406 Packages
Total download size: 577 M
Is this ok [y/d/N]:
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : libgcc-4.8.5-39.el7.x86_64 1/841
Installing : urw-base35-fonts-common-20170801-10.el7.noarch 2/841
Updating : 1:grub2-common-2.02-0.80.el7.centos.noarch 3/841
Updating : centos-release-7-7.1908.0.el7.centos.x86_64 4/841
Updating : langtable-0.0.31-4.el7.noarch 5/841
Updating : libreport-filesystem-2.1.11-43.el7.centos.x86_64 6/841
...
yum-plugin-fastestmirror.noarch 0:1.1.31-52.el7 yum-utils.noarch 0:1.1.31-52.el7
Replaced:
urw-fonts.noarch 0:2.4-16.el7
Complete!
It took less than 10 min to apply all the package updates, so the only things left are to capture current kernel version before and after the reboot:
root@s2:/ # uname -a
Linux s2 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Now let’s reboot:
root@s2:/ # shutdown -r now
… and confirm that we’re running newer Linux Kernel and sporting the CentOS 7.7 release now:
greys@s2:~ $ uname -a Linux s2 3.10.0-1062.1.1.el7.x86_64 #1 SMP Fri Sep 13 22:55:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Hmmm. This does look a bit conservative! It’s 3.10 branch of Linux kernel whereby desktop releases are sporting Linux Kernel 5.x already.
Anyway, that’s server s2 upgraded to CentOS 7.7 successfully!
We’re certainly running the CentOS 7.7 release now:
greys@s2:~ $ cat /etc/redhat-release CentOS Linux release 7.7.1908 (Core)
You may have noticed: I’m finally getting really close to organising the first few technologies of Unix and Linux kind into proper Unix Reference pages!
What I want Unix Tutorial Reference page to be is a collection of the most impactful and useful content here on my blog. For now, there’s less than 10 technologies I want to expand and link from the Reference page, but eventually we’ll see more.
Unix Reference Pages
Each Unix Tutorial Reference page will be found under the Reference section. Each such reference page will aim to be an ever-growing technical introduction to a chosen topic – complete with defintions and relevant support commands.
I’ve got only two reference pages properly started at this stage:
SSH Reference – everything I have written about SSH to date, eventually will be enough to act as complete introduction and how-to guide for managing both SSH server and SSH client.
SELinux Reference – more structured guide but a shorter page. I want it to read as a 1-pager explaining what SELinux is and how it can be used and managed.
Today is my son’s birthday, and we had a great party with many friends. We’ve been lucky to have his birthday on Sunday this year. Here’s a simple enough way of checking wich day of a week any day will be.
Show Current Day of The Week
date command has a special format code that shows current day of the week: %a for short name (like Sat or Sun) and %A for full name (like Saturday or Sunday).
Just typie it like this to confirm today’s day of the week:
greys@xps:~ $ date '+%a'
Sun
greys@xps:~ $ date '+%A'
Sunday
Show Day of the Week For a Specific Date
The same date command has another great option: using -d you can specify any date, so the date command will confirm the day of the week for it.
Here’s how you can check which day of the week it will be on September 15th next year:
I was sure this topic had been covered on Unix Tutorial pages before, but apparently it wasn’t – so without further ado, let me introduce you to one of the most fundamental ways of improving your daily sysadmin or developer life in Linux and Unix environments: passwordless SSH.
What Passwordless SSH Really Is
Passwords are used to protect all sorts of things in our digital lives. So anything that is passwordless sounds like a compromise and a degraded security. Want to make it clear: that is not what’s happening with passwordless SSH! There is no degraded security, only added flexibility to your daily SSH use.
Passwords are indeed the default authentication method for SSH access. You access remote server, it learns your username and asks for your password. If password is wrong, you get connection denied.
Passwordless SSH switches authentication mode: instead of using password, your client uses SSH key to authenticate against remote server. Just like in case with passwords, the remote SSH server must already have your SSH key in order to authenticate you and to accept your login. But depending on how you have your SSH keys management configured, you may not get asked for any password as part of logging in.
Passwordless SSH means you won’t get asked for your user password, but remote SSH server will instead query your client for your SSH key. If your SSH key is protected with a passphrase, you’ll still get asked for it. But if you are using SSH agent – then nothing may be asked.
Overall, passwordless SSH is a great improvement because it relies on SSH agent for managing your local SSH keys. You can load your SSH key into SSH agent once, and then SSH into hundreds of systems that accept that key, all without having to type passwords.
What passwordless SSH Isn’t
Passwordless SSH is not:
it’s not a security compromise – it’s just a different approach (you still need to specify a passphrase, but instead of controlling access to remote servers, it’s restricting access to your private SSH key)
it’s not a way to access unknown remote server with getting password asked
it’s not a setting on SSH server that magically makes everyone use passwordless SSH logins
it’s not a setting on SSH server that accepts logins without password (although there is such a setting!)
Configure Passwordless SSH
Okay, here is a brief example of configuring passwordless SSH. Remember: simplicity and flexibility are in the way this is used, not in the way this is deployed. So it takes a bit of effort to configure, but then becomes a breeze when it comes to adding new servers access.
This step is done on your local system: laptop or desktop. Better go into /home/USERNAME/.ssh directory (note the dot in front of ssh! it’s one of the dotfiles and dot directories mostly used), where these SSH keys (called identities) are usually stored. So for me, I’m in the /home/greys/.ssh directory.
DO NOT skip the passphrase – this is an important bit! Set it to some memorable phrase but not one of your existing passwords. You’ll need this pass phrase to access and use your SSH key.
greys@xps:~ $ ssh-keygen -t rsa -b 4096 -f unixtutorial
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in unixtutorial.
Your public key has been saved in unixtutorial.pub.
The key fingerprint is:
SHA256:tzMm/yh6rfiyEk7bfx04UDi3dxcZISljk/YWkbn2XRE greys@xps
The key's randomart image is:
+---[RSA 4096]----+
| . .o=E=|
| o oB =.+ |
| +o.= o o|
| . . .=. o|
| S..oo…o|
| o .o.. ..|
| o + ..=o . |
| + o.o+o+. |
| .+B++o.. |
+----[SHA256]-----+
Excellent. Here is our SSH identity, also called SSH keypair:
unixtutorial is a private key (protected by the passphrase)
Step 2: Deploy your SSH keypair onto remote server
Now it’s time to share the public key (unixtutorial.pub) with the remote system I want to access in the future. That system is server called becky, but you need to specify the IP or hostname of your own remote server. As you remember, you need to have a username/password access to that server already – if you don’t have them yet then you can’t proceed.
We use the ssh-copy-id command for this purpose: you specify the public key file and the hostname of remote server:
greys@xps:~ $ ssh-copy-id -i unixtutorial.pub becky
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "unixtutorial.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'becky'" and check to make sure that only the key(s) you wanted were added.
BY THE WAY: I have a number of small VPS servers online for the purpose of teaching Linux basics. If you want, I can create you an account there and you can test SSH access procedure there – just contact me.
Step 3: Connect to remote server using your new SSH keypair
Okay, now let’s try and connect to remote server using our new SSH identity. Just use the ssh command and specify the SSH private key (notice how it doesn’t say .pub at the end of unixtutorial filename):
greys@xps:~ $ ssh -i unixtutorial becky Enter passphrase for key 'unixtutorial': Linux becky 4.14.94-v7+ #1193 SMP Tue Jan 22 15:34:30 GMT 2019 armv7l
The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Wed Sep 11 09:00:27 2019 from 192.168.1.XX greys@becky:~ $ echo "Woohoo!" Woohoo!
And just like that – we’re on the remote server! So yes, we had to specify passphrase in order to use unixtutorial SSH key, but after that remote server becky didn’t ask us for a password to my account on it – it trusted my SSH key instead. So it’s been a password-less SSH access.
Step 4: Use ssh-agent for your SSH keypair
ssh-agent is a special tool that comes packaged with SSH. It asks for your passphrase to each private key you want to use, but then keeps the key in memory and uses it for remote access as needed. You type your passphrase once for ssh-agent, and then enjoy truly passwordless SSH to remote servers: no questions, passwords or passphrases asked.
Let’s add our key to the ssh-agent:
greys@xps:~ $ ssh-add unixtutorial
Enter passphrase for unixtutorial:
Identity added: unixtutorial (greys@xps)
Perfect. Now try accessing the remote server again, exactly the way we’ve done it in Step 3. Only this time we won’t get asked for a passphrase, because ssh-agent keeps it in memory:
greys@xps:~ $ ssh -i unixtutorial becky
Linux becky 4.14.94-v7+ #1193 SMP Tue Jan 22 15:34:30 GMT 2019 armv7l
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Sep 11 09:00:37 2019 from 192.168.1.XX
greys@becky:~ $ echo "Hurray!"
Hurray!
And we’re done! Congrats on your newly setup passwordless SSH!
Did you like this article? Leave a comment to let me know! If you have questions – feel free to ask and I’ll update the article.
Sometimes you need to quickly generate a number of files or directories with different, but not random, numbers and names. That’s when seq command is perfect!
Count Sequences using Seq
Simplest is to specify the start and end values for your sequence:
greys@xps:~ $ seq 1 5 1 2 3 4 5
So in this example we start with the first parameter (1) and count sequence until the value becomes the last parameter (5).
Specifying the increment step with seq
Sometimes you want to increment by more than just 1, this is how you can use step of 3, for instance:
greys@xps:~ $ seq 1 3 13
1
4
7
10
13
IMPORTANT: Strange thing to note is that when you use 3 parameters for the seq command, 2nd parameter stops acting as the last number in sequence and becomes an increment. So in our example we start with 1, increment by second parameter (3) and count until we reach 3rd parameter (13).
Counting Down with seq
It’s possible to specify a negative step for your sequence – instead of counting from 1 to 10, for example, you can be counting down from 10 to 1.
Here’s how we use -1 to count down from 10 to 0:
So t greys@xps:~ $ seq 10 -1 0
10
9
8
7
6
5
4
3
2
1
0
The RHEL 8 desktop PC I’m working on right now is going to be a testbed for all sorts of virtualization technologies – Docker, KVM and VirtualBox. Today I found a really cool command for showing CPU properties.
lscpu command is one of the Linux commands that helps you confirm hardware properties and key parameters of your CPU: number of cores, threads, 32-bit/64-bit support, architecture and many other useful things.
CPU information with lscpu
I’m using a relatively old AMD based system. Here’s what lscpu reports:
$ lscpu Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 19
Model name: AMD A8-6500 APU with Radeon(tm) HD Graphics
Stepping: 1
CPU MHz: 2815.703
CPU max MHz: 3500.0000
CPU min MHz: 1700.0000
BogoMIPS: 6987.00
Virtualization: AMD-V
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl no
nstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch os
vw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate ssbd vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pa
usefilter pfthreshold
Based on the above information I can see that my CPU is indeed going to support hardware virtualization and will allow me to host and run multiple 64-bit virtual machines.
I’m working on the next Unix Tutorial Project these days and need to configure new RHEL 8 based PC with Ansible. There’s always a bit of a chicken-and-egg with it because you need to have enough SSH and su or sudo access on remote server before Ansible can be used.
sshpass for Ansible
I got the following error in Ansible output:
fatal: [rhel8]: FAILED! => {“msg”: “to use the ‘ssh’ connection type with passwords, you must install the sshpass program”}
Hoping this would be easily solved, I tried installing sshpass with brew in macOS, but got this instead:
Error: No available formula with the name "sshpass" We won't add sshpass because it makes it too easy for novice SSH users to ruin SSH's security.
I think it’s pretty cool that someone took the time to leave this comment behind!
I’m developing Ansible playbooks for my new dedicated host and learning new Ansible features these days. One of the features required a more recent Ansible version, and that’s how I learned about rewriting symlinks for installed packages in Homebrew.
greys@maverick:~/proj/ansible $ brew install ansible
Updating Homebrew…
==> Auto-updated Homebrew!
Updated Homebrew from 37714b5ce to 78d74a8b2.
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
...
==> Deleted Formulae==> Downloading https://homebrew.bintray.com/bottles/ansible-2.8.4_1.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/93/93eac1e6a31cd36b229f3f66a051840512e198fd0123423d895de87c32ec13c6?gda=exp=1567440871~hmac=6a1d81f750325488002912e069cb58dcb04484e35ba49494b49279dfc2b03c35
################################################################## 100.0%
==> Pouring ansible-2.8.4_1.mojave.bottle.tar.gz
Error: The brew link step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/ansible
Target /usr/local/bin/ansible
already exists. You may want to remove it:
rm '/usr/local/bin/ansible'
To force the link and overwrite all conflicting files:
brew link --overwrite ansible
To list all files that would be deleted:
brew link --overwrite --dry-run ansible
Possible conflicting files are: /usr/local/bin/ansible /usr/local/bin/ansible-config -> /usr/local/bin/ansible /usr/local/bin/ansible-connection /usr/local/bin/ansible-console -> /usr/local/bin/ansible /usr/local/bin/ansible-doc -> /usr/local/bin/ansible /usr/local/bin/ansible-galaxy -> /usr/local/bin/ansible /usr/local/bin/ansible-inventory -> /usr/local/bin/ansible /usr/local/bin/ansible-playbook -> /usr/local/bin/ansible /usr/local/bin/ansible-pull -> /usr/local/bin/ansible /usr/local/bin/ansible-vault -> /usr/local/bin/ansible ==> Summary 🍺 /usr/local/Cellar/ansible/2.8.4_1: 15,147 files, 194.8MB You have new mail in /var/mail/greys
… and then realised Ansible version stayed the same (2.4.2.0) even though installed package was 2.8.4.1…
Overwrite symlinks in Homebrew
Here’s a useful command that re-creates (and overwrites) symlinks from /usr/local/bin directory to the latest version of a specified Homebrew bottle, Ansible in my case:
greys@maverick:~/proj/ansible $ brew link --overwrite ansible
Linking /usr/local/Cellar/ansible/2.8.4_1… 19 symlinks created
Now le’ts check the version of Ansible again, it should show the correct (latest) version:
I’m configuring a new dedicated server this weekend and think it will be great to capture small notes as I go. Today it’s a small basic example, perfect for Ansible beginners: creating a Unix group.
Ansible Playbook for Creating a Group
Here’s the sample playbook file groups.yaml in its entirety:
Ansible hosts file – specifically, group of hosts named techstack
Become method – meaning I give instructions to Ansible playbook for elevating privileges on remote hosts – effectively becoming another user. In this case, the user (become_user) is root, and the way to become this user is su (not sudo!)
group module – it’s one of the most standard modules in Ansible, allowing you to manage Unix groups. As always, you specify the name of the group (techstack2) and the group id for it (1002).
How To Run Group Playbook in Ansible
Because there’s su method involved, Ansible playbook will be expected to provide a password for root user on the remote hosts. But with no password definition in sight, we have to supply password using command line:
