Magic software that allows you to write a single program that runs on a wide range of hardware.
Bare metal programming is to run a program without an operating system below it.
Or in other words, it is basically implementing an operating system/firmware yourself ad hoc, together with your actual program.
Real hardware is for newbs. Real hardware is for newbs.
Follow all the installation instructions from: stopping before the "Flash the sample" section, as we don't flash QEMU. We just run it.
The installation procedure install all compiler toolchains for us, so we can then basically compile for any target. It also fetches the latest Git source code of Zephyr under:
The "most default" blinky hello world example which blinks an LED is a bit useless for us because QEMU doesn't have LEDs, so instead we are going to use one of the UART examples which will print characters we can see on QEMU stdout.
Let's start with the hello world example on an x86 target:
cd ~/zephyrproject/zephyr
zephyr west build -b qemu_x86 samples/hello_world -t run
and it outputs:
Hello World! qemu_x86
The qemu_x64 on the output comes from the CONFIG_BOARD macro
#include <zephyr/kernel.h>

int main(void)
	printk("Hello World! %s\n", CONFIG_BOARD);
	return 0;
You can also first cd into the directory that you want to build in to avoid typing samples/hello_world all the time:
cd ~/zephyrproject/zephyr/samples/hello_world
zephyr west build -b qemu_x86 -t run
You can also build and run separately with:
west build -b qemu_x86
west build -t run
Another important option is:
west build -t menuconfig
But note that it does not modify your prj.conf automatically for you.
Let's try on another target:
rm -rf build
zephyr west build -b qemu_cortex_a53 -t run
and same output, but on a completely different board! The qemu_cortex_a53 board is documented at:
The list of all examples can be seen under:
ls ~/zephyrproject/zephyr/samples
which for example contains:
So run another sample simply select it, e.g. to run zephyrproject/zephyr/samples/synchronization:
west build -b qemu_cortex_a53 samples/synchronization -t run
It ain't perfect, but it's decent enough.
From a technical point of view, it can do anything that Microsoft Windows can. Except being forcefully installed on every non-MacOS 2019 computer you can buy.
Ciro Santilli's conversion to Linux happened around 2012, and was a central part of Ciro Santilli's Open Source Enlightenment, since it fundamentally enables the discovery and contribution to open source software. Because what awesome open source person would waste time porting their amazing projects to closed source OSes?
Ciro's modest nature can be seen as he likes to compare this event Buddha's Great Renunciation.
Particularly interesting in the history of Linux is how it won out over the open competitors that were coming up in the time: MINIX (see the chat) and BSD Operating System that got legally bogged down at the critical growth moment.
Figure 1. xkcd 619: Supported Features. Source. This perfectly illustrates Linux development. First features that matter. Then useless features.
Video 1. Bill Gates vs Steve Jobs by Epic Rap Battles of History (2012) Source. Just stop whatever you are doing, and watch this right now. "I'm on Linux, bitch, I thought you GNU". Fandom explanations. It is just a shame that the Bill Gates actor looks absolutely nothing like the real gates. Actually, the entire Gates/Jobs parts are good, but not genial. But the Linux one is.
The fact that this foundation has a bunch of paid, closed, certification courses makes Ciro Santilli not respect them at all. They should be making open access content instead!
Documents the Linux kernel. Somewhat of a competitor to Linux Kernel Module Cheat, but more wordy and less automated.
As of 2020, no one knows how to build the major desktop distros fully from source into the ISO, and especially so in a reproducible build way. Everything is done in build servers somewhere with complicated layers of prebuilds. It's crap. has some very good comments on how snap is more closed than Flatpak.
However, many, many, many terrible horrors come with it:
The video folder is DCIM/Camera.
Respect. Big respect. Those people are hardcore from scratch hackers, and their wiki is amazing:
It's just that Buildroot is more hardcore ;-)
But can you build the ISO full from source: Linux distribution buildable from source
Buildroot is good.
This thing is sexy.
Ciro Santilli's Linux distro of choice as of 2019.
It ain't perfect, but it's decent enough.
The greatest advantage of it being that it has the likely largest desktop user base, and therefore the highest likelihood that your problems are solved on Ask Ubuntu, and goes together with Ciro's philosophy that "people should do everything in the same way to factor stuff out", especially the open source losers.
Ciro considers that the killer flaw of Ubuntu, and most desktop distros of 2020, is that no one under the Sun knows how to build them fully from source: Linux distribution buildable from source. This is why Ciro based the Linux Kernel Module Cheat on Buildroot, see also: Linux distribution buildable from source.
Their crash system does not have an amazing user interface.
Tested on Ubuntu 21.10.
After something crashes, look under /var/crash for a crash file, which helps to determine which package to report under on Launchpad.
E.g. a file /var/crash/_usr_sbin_gdm3.0.crash makes you want to file the bug under gdm at:
Then, while reporting the bug, you want to give the developpers access to that .crash file. But you can't publicly upload it because it contains memory dumps and could contain secret information. The way to do it is to look at the ID under:
sudo cat /var/crash/_usr_sbin_gdm3.0.uploaded
Ubuntu's crash report system has already uploaded the .crash for you, so you just have to confirm it and give the ID on the ticket.
You can view a list of all your uploaded errors at:
xdg-open$(sudo cat /var/lib/whoopsie/whoopsie-id)
and each of those contain a link to:<.uloaded error id>
which you yourself cannot see.
sudo apport-unpack /var/crash/_usr_sbin_gdm3.0.crash /tmp/app
splits it up into a few files, but does not make any major improvements.
sudo apt install apport-retrace
sudo chmod 666 /var/crash/_usr_sbin_gdm3.0.crash
apport-retrace -g /var/crash/_usr_sbin_gdm3.0.crash
opens GDB with the core dump. Debug symbols are supplied as separate packages, which is a really cool idea: so you should be able to download them after the crash to see symbols. mentions how to install them. Official docs at:
echo "deb $(lsb_release -cs) main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/ddebs.list
echo -e "deb $(lsb_release -cs)-updates main restricted universe multiverse\ndeb $(lsb_release -cs)-proposed main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/ddebs.list
sudo apt install ubuntu-dbgsym-keyring
but then sudo apt update fails with:
E: The repository ' impish-security Release' does not have a Release file.
Switching to the other installed kernel, 5.9 made boot work.
The solution on kernel 6.2 was:
sudo apt instal nvidia-driver-515
as per comments under: This also made the nvidia driver work: Find GPU information in Ubuntu.
Previously I had:
and it blew up before reaching disk decryption.
I also tried:
but that broke in a different way:
Finished apport-autoreport.service - Process error reports when automatic reporting is enabled.
nvidia-modeset: ERROR: GPU:0: Idling display engine timed out: 0x0000947d:0:0:407
(1 of 2) Job systemd-backlight@backlight: nvidia_e.service/start running (32s no limit)
GDM crashes sometimes when switching windows right after opening a new window:
Does not happen every time, only some times. Can't figure out why. Usually happens when has suspended for a longer time. sounds like a likely report, Nvidia driver version 470, but can't find those error messages anywhere. The last line of:
journalctl -o short-precise -k -b -1
once was:
PM: suspend entry (deep)
which is when sleep starts.
This suggests that it is not a video bug then, seems that it is not waking up at all? Gotta try to SSH into it. OK. I did SSH into it, and that was fine, so it is just the video that won't start.
PM: suspend exit is another possible bug, based on kernel version. I'm running 5.13, which is one of the failing versions on the report. Can't find any interesting dmesg though.
In another crash:
journalctl -o short-precise -k -b -1
had the following interesting lines:
nvidia-modeset: WARNING: GPU:0: Lost display notification (0:0x00000000); continuing.
[24307.640014] NVRM: GPU at PCI:0000:01:00: GPU-18af74bb-7c72-ff70-e447-87d48378ea20
[24307.640018] NVRM: Xid (PCI:0000:01:00): 79, pid=8828, GPU has fallen off the bus.
[24307.640021] NVRM: GPU 0000:01:00.0: GPU has fallen off the bus.
[24328.054022] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (LGD (DP-4)) is not supported on this GPU.
[repeats several more times]
[24328.056767] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (LGD (DP-4)) is not supported on this GPU.
[24328.056951] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000927c:0:0:0x0000000f
[24328.056955] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000927c:1:0:0x0000000f
[24328.056959] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000927c:2:0:0x0000000f
[24328.056962] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000927c:3:0:0x0000000f
[24328.056983] nvidia-modeset: ERROR: GPU:0: DP-4: Failed to disable DisplayPort audio stream-0
[24328.056992] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000947d:0:0:0x0000000f
and there was a corresponding /var/crash/_usr_sbin_gdm3.0.crash.
Sometimes it just feels like Ubuntu devs don't actually use Ubuntu as a desktop.
Some extremelly anoying problems are introduced and just never get fixed, even though they feel so obvious!
Would never happen on Mac...
This BS started after the move to ZFS. The temporary solution appears to be:
And then this to disable automatic snapshots in the future:
sudo mv /etc/apt/apt.conf.d/90_zsys_system_autosnapshot /etc/apt/apt.conf.d/90_zsys_system_autosnapshot.disabled
This has annoyed Ciro Santilli for many years, it is just too wasteful of screen space on laptops!
Or likely more generally, on GNOME desktop, which is the default desktop environment as of Ubuntu 22.10.
Legal issues stalled them at the turning point of the Internet, and Linux won. Can't change history.
Did Apple just fork it and made Mac OS X without giving anything back?
Non-POSIX only here.
If you are a programmer, grep becomes a verb: "to grep" means "to search text files", much like "to Google" means "to search random stuff online".
The OS that the Gods ordered be made.
One is reminded of Ulillillia, see also: The Story of Ulillillia by Atrocity Guide (2019)
Video 1. I like elephants and God likes elephants. Source.