Debugging by Ciro Santilli 37 Updated 2025-07-16
Debugging sucks. But there's also nothing quite that "oh fuck, that's why it doesn't work" moment, which happens after you have examined and placed everything that is relevant to the problem into your brain. You just can't see it coming. It just happens. You just learn what you generally have to look at so it happens faster.
This is a simple hierarchical plaintext notation Ciro Santilli created to explain programs to himself.
It is usuall created by doing searches in an IDE, and then manually selecting the information of interest.
It attempts to capture intuitive information not only of the call graph itself, including callbacks, but of when things get called or not, by the addition of some context code.
For example, consider the following pseudocode:
f1() {
}

f2(i) {
  if (i > 5) {
    f1()
  }
}

f3() {
  f1()
  f2_2()
}

f2_2() {
  for (i = 0; i < 10; i++) {

    f2(i)
  }
}

main() {
  f2_2()
  f3()
}
Supose that we are interested in determining what calls f1.
Then a reasonable call hierarchy for f1 would be:
f2(i)
  if (i > 5) {
    f1()

  f2_2()
    for (i = 0; i < 10; i++) {
      f2(i)

    main
    f3
f3()
  main()
Some general principles:
  • start with a regular call tree
  • to include context:
    • remove any blank lines from the snippet of interest
    • add it indented below the function
    • and then follow it up with a blank line
    • and then finally add any callers at the same indentation level
One of the Holiest age old debugging techniques!
The cool thing about bisection is that it is a brainless process: unlike when using a debugger, you don't have to understand anything about the system, and it incredibly narrows down the problem cause for you. Not having to think is great!
Pernosco by Ciro Santilli 37 Updated 2025-07-16
Proprietary extension to Mozilla rr by rr lead coder Robert O'Callahan et. al, started in 2016 after he quit Mozilla.
Omniscient debugging by Ciro Santilli 37 Updated 2025-07-16
What it adds on top of reverse debugging: not only can you go back in time, but you can do it instantaneously.
Or in other words, you can access variables from any point in execution.
TODO implementation? Apparently Pernosco is an attempt at it, though proprietary.
GDB reverse debugging by Ciro Santilli 37 Updated 2025-07-16
The best open source implementation as of 2020 seems to be: Mozilla rr.
Ciro Santilli is obsessed by those in order to learn any new concept, not just for bug reporting.
This includes to learn more theoretical subjects like physics and mathematics.
Microsoft by Ciro Santilli 37 Updated 2025-07-16
And also their monopolistic practices: United States v. Microsoft Corp.
However, like all big tech companies with infinite money, they do end up doing some cool things in their research department, Microsoft Research, notably for Ciro Santilli being:
Microsoft Windows by Ciro Santilli 37 Updated 2025-07-16
A monopolistic operating system that only exists in the 2010's because of the IBM-linked historical lock-in and constant useless changes of the Microsoft Word document format to prevent cross operability.
It offers no technical advantages over free Linux distros in the late 2010's, and it is barely impossible to buy a non-Mac computer without paying for it, which should be illegal. European Union, time to use your regulatory powers.
The following anecdote illustrates Windows' pervasiveness. Ciro Santilli was once tutoring a high school student in Brazil, and decided to try and get her into programming. When the "Windows is not free" subject came up, the high school student was shocked: "I paid 100 dollars for this?". She never even knew it was there. To her, it was "just a computer".
Laws should really be passed forcing OEMs to allow you to not buy Microsoft Windows when buying a computer, European Union, why have you failed me in 2016??? en.wikipedia.org/wiki/Bundling_of_Microsoft_Windows
blog.zorinaq.com/i-contribute-to-the-windows-kernel-we-are-slower-than-other-oper/ I Contribute to the Windows Kernel. We Are Slower Than Other Operating Systems. Here Is Why. by Marc Bevand (2013) has some interesting remarks:
There's also little incentive to create changes in the first place. On linux-kernel, if you improve the performance of directory traversal by a consistent 5%, you're praised and thanked. Here, if you do that and you're not on the object manager team, then even if you do get your code past the Ob owners and into the tree, your own management doesn't care. Yes, making a massive improvement will get you noticed by senior people and could be a boon for your career, but the improvement has to be very large to attract that kind of attention.
Video 1.
Bomb disposal robot by The IT Crowd
. Source.
Video 2.
Gabe Newell Blue Screen
. Source. During Game Developers Choice Awards acceptance presentation. TODO year.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 2.
    You can publish local OurBigBook lightweight markup files to either https://OurBigBook.com or as a static website
    .
    Figure 3.
    Visual Studio Code extension installation
    .
    Figure 4.
    Visual Studio Code extension tree navigation
    .
    Figure 5.
    Web editor
    . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
    Video 4.
    OurBigBook Visual Studio Code extension editing and navigation demo
    . Source.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact