Upgrading Raspbian OS from Stretch to Buster

With recently released Debian 10 Buster it's possible to upgrade Raspbian OS on your Raspberry Pi device to the same release.
Raspberry Pi 4

Debian 10 Buster got released a while back and this means it should be possible to upgrade Raspbian OS on Rasberry Pi devices to the Raspbian Buster as well. Raspbian OS is based on Debian as you remember.

Step 1: Confirm Your Current Linux Kernel and OS release

Always a good step: confirm Raspbian version and check your current Linux Kernel version:

greys@s7:~ $ uname -a
Linux s7 4.19.42-v7+ #1219 SMP Tue May 14 21:20:58 BST 2019 armv7l GNU/Linux
greys@s7:~ $ cat /etc/debian_version
9.9

Step 2: Update Existing Release Before Upgrading to the Next

Just like in the upgrade Jessie to Stretch procedure, we start with using apt-get command to upgrade existing packages available for the current release.

The reason it’s recommended to do this is by moving to the latest available version in your current release you are minimising the changes required for the upgrade to the next release. Changes and configuration file syntax are not likely to be as dramatic between last version in current release and first version in the next release.

We refresh list of available updates:

greys@s7:~ $ sudo apt-get update

and install these updates:

 greys@s7:~ $ sudo apt-get upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
The following packages will be upgraded:
cups-browsed cups-filters cups-filters-core-drivers dbus dbus-x11 fonts-opensymbol libcupsfilters1 libdbus-1-3 libexpat1 libexpat1-dev
libfontembed1 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 libreoffice libreoffice-avmedia-backend-gstreamer
libreoffice-base libreoffice-base-core libreoffice-base-drivers libreoffice-calc libreoffice-common libreoffice-core libreoffice-draw
libreoffice-gtk libreoffice-gtk2 libreoffice-impress libreoffice-java-common libreoffice-librelogo libreoffice-math libreoffice-nlpsolver
libreoffice-ogltrans libreoffice-pdfimport libreoffice-report-builder libreoffice-report-builder-bin libreoffice-script-provider-bsh
libreoffice-script-provider-js libreoffice-script-provider-python libreoffice-sdbc-hsqldb libreoffice-sdbc-postgresql libreoffice-style-galaxy
libreoffice-style-tango libreoffice-systray libreoffice-wiki-publisher libreoffice-writer libssl-dev libssl-doc libssl1.0.2 libssl1.1 libzmq5
nodered omxplayer openssl patch python3-uno raspberrypi-bootloader raspberrypi-kernel rpi-chromium-mods uno-libs3 ure vim vim-common vim-runtime
vim-tiny xxd
65 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 176 MB of archives.
After this operation, 9,103 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
...

Run apt-get dist-upgrade too, in my case it shows zero packages:

greys@s7:~ $ sudo apt-get dist-upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Step 3: Upgrade Raspbian OS to Buster

Three elements to this step:

  1. Update release code name in /etc/apt/sources.list
  2. Download list of available package updates
  3. Upgrade Raspbian OS

Update release name in /etc/apt/sources.list

First, let’s edit /etc/apt/sources.list file:

greys@s7:~ $ sudo vi /etc/apt/sources.list

I changed this:

to this:

Refresh packages info with apt-get update

Now we can refresh information about available package updates:

greys@s7:~ $ sudo apt-get update 

And, finally, we’re ready to…

Upgrade Raspbian packages to Buster release

greys@s7:~ $ sudo apt-get upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
...

… and finish it off with apt dist-upgrade:

greys@s7:~ $ sudo apt-get dist-upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
The following packages were automatically installed and are no longer required:
...
The following packages will be REMOVED:
gnome-themes-standard-data gstreamer1.0-omx gstreamer1.0-omx-rpi gstreamer1.0-omx-rpi-config idle-python3.5 libcupscgi1 libcupsmime1 libcupsppdc1
libcurl3 libgles1-mesa libllvm3.9 libmariadbclient18 libreoffice-gtk libreoffice-style-galaxy libreoffice-systray libsensors4 libservlet2.5-java
libtirpc1 netsurf-gtk python3-spidev raspi-copies-and-fills xserver-xorg-video-fbturbo
The following NEW packages will be installed:
...
533 upgraded, 459 newly installed, 22 to remove and 0 not upgraded.
Need to get 861 MB/862 MB of archives.
After this operation, 1,045 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Step 4: Confirm Successful Upgrade

We need to reboot the Raspberry Pi now:

greys@s7:~ $ sudo shutdown -r now

…and run uname -a command and inspect the /etc/debian_version file to confirm that our Raspberry Pi is indeed running the latest version of Raspbian OS now:

See Also




Unix Tutorial Digest – July 28th, 2019




Great Things You Can Confirm with dmidecode

dmidecode-command

I’ve just started working on the Linux Commands section of Unix Tutorial, and dmidecode is one of the best commands to mention when it comes to recent Linux distros. It’s found in most distributions and helps with learning lots of useful facts about your environments – both physical and virtual.

How To Use dmidecode Command

dmidecode command needs to be run as root and shows you hardware information about your system:

  • BIOS name and version
  • manufacturer of your server, desktop or laptop
  • model name and serial number of your system

Simply run the command and pipe it to a pager like more or less:

greys@xps:~ $ sudo dmidecode | less
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.2.1 present.
# SMBIOS implementations newer than version 3.2.0 are not
# fully supported by this version of dmidecode.
Table at 0x000E0000.

Handle 0x0000, DMI type 0, 26 bytes
BIOS Information
Vendor: Dell Inc.
Version: 1.2.1
Release Date: 02/14/2019
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 32 MB
...

Browsing dmidecode output and searching through the output are the best ways to use dmidecode command, but once you become familiar with your environment you’ll probably get a few dmidecode parameters you can grep for.

Hardware Vendor with dmidecode

This will report the manufacturer of your system. For my XPS laptop, it shows Dell:

greys@xps:~ $ sudo dmidecode | grep Vendor
Vendor: Dell Inc.
Vendor ID:

Model name with dmidecode

Look for the Product Name to confirm the name of your system. It shows my laptop’s model for me:

greys@xps:~ $ sudo dmidecode | grep Product
Product Name: XPS 13 9380
Product Name: 0KTW76

Motherboard with dmidecode

Finding the motherboard model will require you to search through the less pager output (press / and start typing word motherboard, then scroll up and down).

Here’s what one of my dedicated servers shows:

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: Supermicro
Product Name: X11SSE-F
Version: 1.01
Serial Number: ZM163S009892
Asset Tag: To be filled by O.E.M.
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: To be filled by O.E.M.
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0

Serial numbers with dmidecode

Just grep for the word Serial to find lots of serial numbers of various recognised devices. One of them (the first one in the output) is the Dell’s service tag that you usually need for hardware support:

greys@xps:~ $ sudo dmidecode | grep Serial
Serial services are supported (int 14h)
Serial Number: 50G8V**
Serial Number: /50G8V**/CN***00***00F8/
Serial Number: 50G8V**
Serial Number: Not Specified
Serial Number: Not Specified
Serial Number: To Be Filled By O.E.M.
Serial Number: 0A3E
Debug Use USB(Disabled:Serial)

There’s lots of other things dmidecode is useful for – I’ll be sure to update the dmidecode command page going forward.

See Also




Show Servers Across All OpenStack Projects

openstack-logo.png

I’m positive that sometime later this year I’ll revisit OpenStack basics, but for now there’s just no time. This post is one of those as-we-go technical notes taken for my own reference.

List OpenStack Servers using CLI

Once your OpenStack environment variables are configured, you can list servers using a very simple command:

$ openstack server list

But since one of the OpenStack variables is default project, the command above will probably just get you the list of servers within that project.

List OpenStack Servers Across All Projects

Provided your username and password have enough access, it should be possible to expand the previous command to get you a much longer list – namely every OpenStack server in every project available:

$ openstack server list --all-projects

See Also




Most Useful Basic Unix Commands

unix-tutorial

Seems the Basic Unix Commands section is gaining popularity online – that’s great to see, means I’m putting focus where it’s actually needed. Today I want to share my view of the most useful basic unix commands – the ones that I use daily.

What Defines a List of Daily Used Commands

The types of commands you will need depends on the tasks and how you go about them. Chances are, most of your work is done remotely – which means one of your most used commands could be git or ssh.

Apart from this, it’s unbelievably awesome how the majority of daily work can be accomplished using very basic unix commands, nothing fancy at all.

If you’re a software developer, you’ll probably have your share of programming language interpreters and compilers along with other components of a DevOps toolchain, but even with them in the picture there’s still quite a chunk of work done using the most beatifully simple commands like cd and ls.

Daily Tasks I Accomplish with Command Line

Here’s the list of things I get to do pretty much every day:

  • navigate around filesystems
    • where am I?
    • go to a specific directory
    • is this the same or separate filesystem?
    • check how much space is there available?
  • check files and directories
    • is there a file named such-and-such?
    • what’s the size of the file?
    • who owns the file (username and group)
    • when was the file last updated?
    • what’s the contents of this file?
  • manage files and directories
    • create a new directory
    • create or edit a file
    • change username or group ownership
    • remove a directory or a file
    • copy or move a file
  • remote access 
    • connect to a remote server
    • copy file from my system to remote server
    • transfer files from remote server to my local system

Basic Unix Commands For Daily Use

I’ll probably show examples of command lines for these tasks in some future post, but for now will just map the basic commands to each section:

See Also




Create swap from File on BTRFS Filesystem

mkswap-swapon

I didn’t create any swap during initial Ubuntu 19.04 install on Dell XPS laptop, so had to improvise when compiling a massive opensource project recently. This post shows you how to temporarily add swap memory using a regular file.

Create swap Using Regular File

The usual approach is fairly simple:

  • you create a large enough file (the desired size of your swap memory) – 1GB or 4GB or something like that
  • you initialise it as swap
  • you activated it as swap

Swap Files Support in BTRFS

Since I’ve used BTRFS filesystem for root in my Ubuntu setup, I discovered that there are additional steps needed: BTRFS filesystem creates and writes files in a way that’s been mostly incompatible with swap usage.

But as luck would have it, BTRFS supports swap files with Linux Kernel 5.0.x, so you just need to create file with specific attribute.

Swap Space Using File Procedure

Step 1: Create new empty file

We need to create a file with NOCOW (NO Copy-On-Write) property (no harm anywhere but is a requirement for BTRFS):

root@xps:/ # touch /swapfile
root@xps:/ # chattr +C /swapfile

Step 2: Allocate required amount of space to the file

root@xps:/ # fallocate -l 8G /swapfile

This file is 8GB now:

root@xps:/ # ls -ald /swapfile 
-rw------- 1 root root 8589934592 Jul 17 19:55 /swapfile

Step 3: Update permissions

root@xps:/ # chmod 0600 /swapfile 

Step 4: Initialise /swapfile as swap storage

root@xps:/ # mkswap /swapfile 
Setting up swapspace version 1, size = 8 GiB (8589930496 bytes)
no label, UUID=16d35c04-78de-4dd3-aeb0-e2228bb7ce36

Step 5: Activate swap space from /swapfile

root@xps:/ # swapon /swapfile

Step 6: Confirm newly activated swap space

root@xps:/ # free -h
total used free shared buff/cache available
Mem: 15Gi 6.1Gi 1.7Gi 3.1Gi 7.5Gi 5.8Gi
Swap: 8.0Gi 0B 8.0Gi

That’s it for today!

See Also




Free Up Reserved Space in Filesystems

tune2fs-reserved-space

You’re probably familiar with tune2fs command, it’s great for reporting filesystem layout and configuration options. It’s also helpful when you want to free up some of the reserved space.

Reserved Space in Unix/Linux Filesystems

Reserved space is one of the more mysterious things in Linux/Unix filesystems. A rather old tradition,  it is simply a practice of automatically reserving some filesystem space for system (superuser) use. When you’re formatting a new filesytem, it automatically reserves 5% of space for superuser access – meaning regular users and processes won’t be able to use this space (filesystem will report to be 100% full), but root user can still write and troubleshoot.

Reserved Space meant a lot more convenience when hard disks and filesystems were small – about 20 years ago your / or /var filesystem could be 1GB in total, and this meant it was absolutely crucial to reserve some space so that regular users could not cause an outage by generating some temporary file and filling up a key filesystem. These days sizes of filesystem are vastly larger and this means two things:

  1. You probably don’t benefit from reserved space as much as someone used to
  2. You may well never even know you had space reserved because you simply don’t run out of space

Here’s how this would look:

root@xps:/storage # df -h /storage
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 95G 90G 232M 100% /storage

Doesn’t quite add up, does it? It’s a 95GB filesystem with only 90GB used, yet filesystem is 100% full.

Change Reserved Space Percentage with tune2fs

Let’s use tune2fs command to reduce the percentage of reserved space (-m 1 means set reserve to 1% of total capacity):

root@xps:/storage # tune2fs -m 1 /dev/nvme0n1p8 
tune2fs 1.44.6 (5-Mar-2019)
Setting reserved blocks percentage to 1% (254279 blocks)

… and enjoy the results:

root@xps:/storage # df -h /storage 
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 95G 90G 4.2G 96% /storage

That’s it for today, hope you learned something new!

See Also

 




How To: Check AppArmor Status

apparmor-logo

AppArmor is a Linux Kernel security module that implements mandatory access control (MAC) security with per-application profiles in Debian based systems. It’s possible to confirm if AppArmor is enabled  in your Debian or Ubuntu system and to also find out the mode it’s running in.

AppArmor Status with aa-status Command

aa-status command will list the currently loaded AppArmor modules.

For instance, here’s how it looks on a system where AppArmor is inactive (Debian 9 in my case):

root@debian9:~# aa-status 
apparmor module is loaded. 
apparmor filesystem is not mounted.

And here is how AppArmor status is reported on Debian 10 system where it’s activated by default:

root@debian10:~# aa-status 
apparmor module is loaded.
20 profiles are loaded.
18 profiles are in enforce mode.
/usr/bin/evince
/usr/bin/evince-previewer
/usr/bin/evince-previewer//sanitized_helper
/usr/bin/evince-thumbnailer
/usr/bin/evince//sanitized_helper
/usr/bin/man
/usr/lib/telepathy/mission-control-5
/usr/lib/telepathy/telepathy-*
/usr/lib/telepathy/telepathy-*//pxgsettings
/usr/lib/telepathy/telepathy-*//sanitized_helper
/usr/lib/telepathy/telepathy-ofono
libreoffice-senddoc
libreoffice-soffice//gpg
libreoffice-xpdfimport
man_filter
man_groff
nvidia_modprobe
nvidia_modprobe//kmod
2 profiles are in complain mode.
libreoffice-oopslash
libreoffice-soffice
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

See Also




Debian 10 – Buster

debian-linux-logo

Debian 10 “Buster” got released ealier this week, culminating work of the past 2 years. It’s a massive upgrade to Debian 9 and a solid release available for 10 architectures (x86/x64/AMD64 and arm64/amrhf plus MIPS, PowerPC and IBM System Z.

I’m away from home for the past 2 weeks but will attempt distro upgrade in my Debian 9 VM, so stay tuned.

Notable Improvements in Debian 10 Buster

Secure Boot is finally here in Debian 10!

This means Debian 10 comes with signed and fully trusted version of GRUB boot loader that is activated by the Shim loader.

Secure Boot is a cool security feature found on most modern PCs and laptops, basically to ensure that trusted (secure) code is loaded by UEFI (Unified Extensible Firmware Interface) before Operating System.

AppArmor is Activated by Default

This means at least the core OS functionality will run with improved security. You’ll need to disable AppArmor using GRUB options if you want to continue running OS without AppArmor.

nftables is the Default Packet Filter

nftables tool will help with managing both IPv4 and IPv6 packets. I think iptables compatibility is there, meaning you can run iptables commands with common syntax – but implementation will actually be done using nftables.

Linux Kernel Updated to 4.19

Not quite the bleeding edge (I hear Linux Kernel 5.2 released recently), but that’s exactly the point: Debian always prides itself on being one of the most stable and reliable distros, meaning it won’t have latest features but will ensure that everything avaialbe will work as expected.

See Also




How To: Install Docker in Ubuntu Using Snap

docker-logo

One of the advantages snap package manager brings to Ubuntu is support for multiple versions of same software available and installable simultaneously. For example, you can decide if you want to run the stable or latest (edge) version of Docker.

Before You Install Docker with Snap

Apparently, snap model is not fully compatible with Docker right now, this means starting and using docker that you installed via snap is not as simple as you might expect. 

Specifically, Docker is no longer maintaining the Docker version so it’s built and supported by Canonical (Ubuntu vendor).

The way I see it: if you just need a hassle free Docker setup, install Docker using apt-get from the official Docker repositories.

Confirm Docker Versions Available via Snap

Running snap info docker will list all the available channels for Docker in snap:

greys@xps:~ $ snap info docker
name: docker
summary: Docker container runtime
publisher: Canonical✓
contact: [email protected]
license: Apache-2.0
description: |
Build and run container images with Docker.

This build requires all files that Docker uses, such as dockerfiles, to be in $HOME. Keep files
there for 'docker build', 'docker save' and 'docker load'. Use the 'docker-privilege' command for
'docker run --privileged'.

This snap is built by Canonical based on source code published by Docker, Inc. It is not endorsed
or published by Docker, Inc.

Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United
States and/or other countries. Docker, Inc. and other parties may also have trademark rights in
other terms used herein.
snap-id: sLCsFAO8PKM5Z0fAKNszUOX0YASjQfeZ
channels:
stable: 18.06.1-ce 2019-05-08 (384) 102MB -
candidate: 18.06.1-ce 2019-06-04 (392) 102MB -
beta: 18.06.1-ce 2019-05-31 (392) 102MB -
edge: 18.09.7-ce 2019-07-09 (404) 127MB -
17.09/stable: –
17.09/candidate: 17.09.1-ce 2017-12-22 (232) 44MB -
17.09/beta: 17.09.1-ce 2017-12-20 (232) 44MB -
17.09/edge: 17.09.1-ce 2017-12-14 (232) 44MB -
17.06/stable: 17.06.2-ce 2017-12-22 (179) 43MB -
17.06/candidate: 17.06.2-ce 2017-10-10 (179) 43MB -
17.06/beta: ↑
17.06/edge: 17.06.2-ce-1 2018-01-03 (237) 44MB -
17.03/stable: 17.03.2-ce-1 2017-07-20 (159) 42MB -
17.03/candidate: 17.03.2-ce-1 2017-06-30 (159) 42MB -
17.03/beta: ↑
17.03/edge: 17.03.2-ce-1 2017-06-30 (159) 42MB -


Install Latest (Edge) Version of Docker with Snap

Based on the above, let’s install the latest Docker package version available via edge channel:

greys@xps:~ $ sudo snap install docker --edge
[sudo] password for greys:
Download snap "docker" (404) from channel "edge" 98% 316kB/s 9.50s
docker (edge) 18.09.7-ce from Canonical✓ installed

Check the Version of Docker We Just Installed

greys@xps:~ $ docker --version
Docker version 18.09.7-ce, build 3a74934

See Also