= How computers work?
{tag=Essays by Ciro Santilli}
= Computer abstraction layer
{synonym}
A computer is a highly layered system, and so you have to decide which layers you are the most interested in studying.
Although the layer are somewhat independent, they also sometimes interact, and when that happens it usually hurts your brain. E.g., if <compilers> were perfect, no one optimizing software would have to know anything about <microarchitecture>. But if you want to go hardcore enough, you might have to learn some lower layer.
It must also be said that like in any industry, certain layers are hidden in commercial secrecy mysteries making it harder to actually learn them. In computing, the lower level you go, the more <closed source> things tend to become.
But as you climb down into the abyss of low level hardcoreness, don't forget that <backward design>[making usefulness is more important than being hardcore]: <image xkcd 378: Real Programmers>.
First, the most important thing you should know about this subject: https://cirosantilli.com/linux-kernel-module-cheat/should-you-waste-your-life-with-systems-programming
Here's a summary from low-level to high-level:
* <semiconductor physical implementation> this level is of course the most closed, but it is fun to try and peek into it from any openings given by commercials and academia:
* <photolithography>, and notably <photomask> design
* <register transfer level>
* interactive <Verilator> fun: https://stackoverflow.com/questions/38108243/is-it-possible-to-do-interactive-user-input-and-output-simulation-in-vhdl-or-ver/38174654#38174654[Is it possible to do interactive user input and output simulation in VHDL or Verilog?]
* more importantly, and much harder/maybe impossible with <open source>, would be to try and set up a open source <standard cell library> and supporting software to obtain <power, performance and area> estimates
* https://www.quora.com/Are-there-good-open-source-standard-cell-libraries-to-learn-IC-synthesis-with-EDA-tools/answer/Ciro-Santilli[Are there good open source standard cell libraries to learn IC synthesis with EDA tools?] on <Quora>
* the most open source ones are some initiatives targeting FPGAs, e.g. https://symbiflow.github.io/[], http://www.clifford.at/icestorm/[]
* <qflow> is an initiative targeting actual <integrated circuits>
* <microarchitecture>: a good way to play with this is to try and run some minimal userland examples on <gem5> userland simulation with logging, e.g. see on the <Linux Kernel Module Cheat>:
* https://cirosantilli.com/linux-kernel-module-cheat/gem5-event-queue-derivo3cpu-syscall-emulation-freestanding-example-analysis
This should be done at the same time as books/website/courses that explain the microarchitecture basics.
This is the level of abstraction that <Ciro Santilli> finds the most interesting of the hardware stack. Learning it for actual <CPUs> (which as of 2020 is only partially documented by vendors) could actually be useful in hardcore software optimization use cases.
* <instruction set architecture>: a good approach to learn this is to manually write some userland assembly with assertions as done in the <Linux Kernel Module Cheat> e.g. at:
* https://github.com/cirosantilli/linux-kernel-module-cheat/blob/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/userland/arch/x86_64/add.S
* https://cirosantilli.com/linux-kernel-module-cheat/x86-userland-assembly
* learn a bit about calling conventions, e.g. by calling C standard library functions from assembly:
* https://github.com/cirosantilli/linux-kernel-module-cheat/blob/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/userland/arch/aarch64/inline_asm/linux/asm_from_c.c
* https://stackoverflow.com/questions/16255608/calling-c-functions-from-x86-assembly-language/56328708#56328708[Calling C functions from x86 assembly language]
* you can also try and understand what some simple <C (programming language)> programs <compile> to. Things can get a bit hard though when `-O3` is used. Some cute examples:
* https://stackoverflow.com/questions/310974/what-is-tail-call-optimization/55230417#55230417[What is tail call optimization?]
* https://stackoverflow.com/questions/1345670/stack-smashing-detected/51897264#51897264[What is the "Stack smashing detected" error in GCC and how to solve it?]
* https://stackoverflow.com/questions/745870/realistic-usage-of-the-c99-restrict-keyword/30827311#30827311[Realistic usage of the C99 'restrict' keyword?]
* <executable file format>, notably <executable and Linkable Format>. Particularly important is to understand the basics of:
* address relocation: https://stackoverflow.com/questions/3322911/what-do-linkers-do/33690144#33690144[How do linkers and address relocation work?]
* position independent code: https://stackoverflow.com/questions/2463150/what-is-the-fpie-option-for-position-independent-executables-in-gcc-and-ld/51308031#51308031[What is the -fPIE option for position-independent executables in GCC and ld?]
* how to observe which symbols are present in object files, e.g.:
* how C++ uses name mangling https://stackoverflow.com/questions/1041866/what-is-the-effect-of-extern-c-in-c/30526795#30526795[What is the effect of extern "C" in C++?]
* how C++ template instantiation can help reduce link time and size: https://stackoverflow.com/questions/2351148/explicit-template-instantiation-when-is-it-used/59614755#59614755[Explicit template instantiation - when is it used?]
* <operating system>. There are two ways to approach this:
* learn about the Linux kernel <Linux kernel>. A good starting point is to learn about its main interfaces. This is well shown at <Linux Kernel Module Cheat>:
* system calls
* write some system calls in
* pure assembly:
* https://github.com/cirosantilli/linux-kernel-module-cheat/blob/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/userland/arch/x86_64/freestanding/linux/hello.S
* https://stackoverflow.com/questions/174942/how-should-strace-be-used/55397255#55397255[How should strace be used?]
* C GCC inline assembly:
* https://stackoverflow.com/questions/9506353/how-to-invoke-a-system-call-via-syscall-or-sysenter-in-inline-assembly/54956854#54956854
* https://github.com/cirosantilli/linux-kernel-module-cheat/blob/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/userland/arch/x86_64/inline_asm/freestanding/linux/hello.c
* learn about kernel modules and their interfaces. Notably, learn about to demystify special files such `/dev/random` and so on:
* https://stackoverflow.com/questions/22632713/how-to-write-a-simple-linux-device-driver/44640466#44640466
* https://github.com/cirosantilli/linux-kernel-module-cheat/tree/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/kernel_modules
* learn how to do a minimal Linux kernel disk image/boot to userland hello world: https://unix.stackexchange.com/questions/2692/what-is-the-smallest-possible-linux-implementation/203902#203902[What is the smallest possible Linux implementation?]
* learn how to GDB <Step debug the Linux kernel> itself. Once you know this, you will feel that "given enough patience, I could understand anything that I wanted about the kernel", and you can then proceed to not learn almost anything about it and carry on with your life
* write your own (mini-) OS, or study a minimal educational OS, e.g. as in:
* <x86 bare metal examples>
* https://stackoverflow.com/questions/22054578/how-to-run-a-program-without-an-operating-system/32483545#32483545
* <programming language>
\Image[https://web.archive.org/web/20191222121520if_/http://imgs.xkcd.com/comics/real_programmers.png]
{title=<xkcd> 378: Real Programmers}
{height=600}
{source=https://xkcd.com/378/}
\Video[https://youtube.com/watch?v=_6D05gCWh_I]
{title=How low can you go video by <Ciro Santilli> (2017)}
{description=In this infamous video Ciro has summarized the computer hierarchy.}
Back to article page