New articles - OurBigBook.com

www.youtube.com/watch?v=-_qNKbwM_eE Unsolved: Yang-Mills existence and mass gap by J Knudsen (2019). Gives 10 key points, but the truly hard ones are too quick. He knows the thing though.

Video 1.

Yang-Mills 1 by David Metzler (2011)

Source.

Playlist: www.youtube.com/watch?v=j3fsPHnrgLg&list=PL613A31A706529585&index=13

A bit disappointing, too high level, with very few nuggests that are not Googleable withing 5 minutes.

Breakdown:

1 www.youtube.com/watch?v=j3fsPHnrgLg: too basic
2 www.youtube.com/watch?v=br6OxCLyqAI?t=569: mentions groups of Lie type in the context of classification of finite simple groups. Each group has a little diagram.
3 youtu.be/1baiIxKKQlQ?list=PL613A31A706529585&t=728 the original example of a local symmetry was general relativity, and that in that context it can be clearly seen that the local symmetry is what causes "forces" to appear
- youtu.be/1baiIxKKQlQ?list=PL613A31A706529585&t=933 local symmetry gives a conserved current. In the case of electromagnetism, this is electrical current. This was the only worthwhile thing he sad to 2021 Ciro. Summarized at: local symmetries of the Lagrangian imply conserved currents.
4 youtu.be/5ljKcWm7hoU?list=PL613A31A706529585&t=427 electromagnetism has both a global symmetry (special relativity) but also local symmetry, which leads to the conservation of charge current and forces.
lecture 3 properly defines a local symmetry in terms of the context of the lagrangian density, and explains that the conservation of currents there is basically the statement of Noether's theorem in that context.

Video 2.

Millennium Prize Problem: Yang Mills Theory by David Gross (2018)

Source. 2 hour talk at the Kavli Institute for Theoretical Physics. Too mathematical, 2021 Ciro can't make much out of it.

Video 3.

Lorenzo Sadun on the "Yang-Mills and Mass Gap" Millennium problem

. Source. Unknown year. He almost gets there, he's good. Just needed to be a little bit deeper.

 Read the full article

x86 Paging Tutorial / Basic TLB operation by

Ciro Santilli  40 Updated 2025-07-16

 View more

After a translation between linear and physical address happens, it is stored on the TLB. For example, a 4 entry TLB starts in the following state:

  valid  linear  physical
  -----  ------  --------
> 0      00000   00000
  0      00000   00000
  0      00000   00000
  0      00000   00000

The > indicates the current entry to be replaced.

And after a page linear address 00003 is translated to a physical address 00005, the TLB becomes:

  valid  linear  physical
  -----  ------  --------
  1      00003   00005
> 0      00000   00000
  0      00000   00000
  0      00000   00000

and after a second translation of 00007 to 00009 it becomes:

  valid  linear  physical
  -----  ------  --------
  1      00003   00005
  1      00007   00009
> 0      00000   00000
  0      00000   00000

Now if 00003 needs to be translated again, hardware first looks up the TLB and finds out its address with a single RAM access 00003 --> 00005.

Of course, 00000 is not on the TLB since no valid entry contains 00000 as a key.

 Read the full article

x86 Paging Tutorial / TLB replacement policy by

Ciro Santilli  40 Updated 2025-07-16

 View more

When TLB is filled up, older addresses are overwritten. Just like CPU cache, the replacement policy is a potentially complex operation, but a simple and reasonable heuristic is to remove the least recently used entry (LRU).

With LRU, starting from state:

  valid  linear  physical
  -----  ------  --------
> 1      00003   00005
  1      00007   00009
  1      00009   00001
  1      0000B   00003

adding 0000D -> 0000A would give:

  valid  linear  physical
  -----  ------  --------
  1      0000D   0000A
> 1      00007   00009
  1      00009   00001
  1      0000B   00003

 Read the full article

x86 Paging Tutorial / CAM by

Ciro Santilli  40 Updated 2025-07-16

 View more

Using the TLB makes translation faster, because the initial translation takes one access per TLB level, which means 2 on a simple 32 bit scheme, but 3 or 4 on 64 bit architectures.

The TLB is usually implemented as an expensive type of RAM called content-addressable memory (CAM). CAM implements an associative map on hardware, that is, a structure that given a key (linear address), retrieves a value.

Mappings could also be implemented on RAM addresses, but CAM mappings may required much less entries than a RAM mapping.

For example, a map in which:

both keys and values have 20 bits (the case of a simple paging schemes)
at most 4 values need to be stored at each time

could be stored in a TLB with 4 entries:

linear  physical
------  --------
00000   00001
00001   00010
00010   00011
FFFFF   00000

However, to implement this with RAM, it would be necessary to have 2^20 addresses:

linear  physical
------  --------
00000   00001
00001   00010
00010   00011
... (from 00011 to FFFFE)
FFFFF   00000

which would be even more expensive than using a TLB.

 Read the full article

x86 Paging Tutorial / Invalidating TLB entries by

Ciro Santilli  40 Updated 2025-07-16

 View more

When the process changes, cr3 change to point to the page table of the new current process.

This creates a problem: the TLB is now filled with a bunch of cached entries for the old process.

A simple and naive solution would be to completely invalidate the TLB whenever the cr3 changes.

However, this is would not be very efficient, because it often happens that we switch back to process 1 before process 2 has completely used up the entire TLB cache entries.

The solution for this is to use so called "Address Space Identifiers" (ASID) as mentioned in sources such as:

Basically, the OS assigns a different ASID for each process, and then TLB entries are automatically also tagged with that ASID. This way when the process makes an access, the TLB can determine if a hit is actually for the current process, or if it is an old address coincidence with another process.

The x86 also offers the invlpg instruction which explicitly invalidates a single TLB entry. Other architectures offer even more instructions to invalidated TLB entries, such as invalidating all entries on a given range.

 Read the full article

x86 Paging Tutorial / ARM by

Ciro Santilli  40 Updated 2025-07-16

 View more

Information about ARM paging can be found at: cirosantilli.com/linux-kernel-module-cheat#arm-paging

 Read the full article

x86 Paging Tutorial / Play with physical addresses in Linux by

Ciro Santilli  40 Updated 2025-07-16

 View more

Convert virtual addresses to physical from user space with /proc/<pid>/pagemap and from kernel space with virt_to_phys:

Dump all page tables from userspace with /proc/<pid>/maps and /proc/<pid>/pagemap:

Read and write physical addresses from userspace with /dev/mem:

 Read the full article

x86 Paging Tutorial / Kernel vs process memory layout by

Ciro Santilli  40 Updated 2025-07-16

 View more

The Linux Kernel reserves two zones of virtual memory:

one for kernel memory
one for programs

The exact split is configured by CONFIG_VMSPLIT_.... By default:

on 32-bit:

the bottom 3/4 is program space: 00000000 to BFFFFFFF

the top 1/4 is kernel memory: C0000000 to FFFFFFFF, like this:

------------------ FFFFFFFF
Kernel
------------------ C0000000
------------------ BFFFFFFF


Process


------------------ 00000000

on 64-bit: currently only 48-bits are actually used, split into two equally sized disjoint spaces. The Linux kernel just assigns:
- the bottom part to processes 00000000 00000000 to 008FFFFF FFFFFFFF
- the top part to the kernel: FFFF8000 00000000 to FFFFFFFF FFFFFFFF, like this:
  ------------------ FFFFFFFF Kernel ------------------ C0000000 (not addressable) ------------------ BFFFFFFF Process ------------------ 00000000

Kernel memory is also paged.

In previous versions, the paging was continuous, but with HIGHMEM this changed.

There is no clear physical memory split: stackoverflow.com/questions/30471742/physical-memory-userspace-kernel-split-on-linux-x86-64

 Read the full article

x86 Paging Tutorial / Process memory layout by

Ciro Santilli  40 Updated 2025-07-16

 View more

For each process, the virtual address space looks like this:

------------------ 2^32 - 1
Stack (grows down)
v v v v v v v v v
------------------

(unmapped)

------------------ Maximum stack size.


(unmapped)


-------------------
mmap
-------------------


(unmapped)


-------------------
^^^^^^^^^^^^^^^^^^^
brk (grows up)
-------------------
BSS
-------------------
Data
-------------------
Text
-------------------

------------------- 0

The kernel maintains a list of pages that belong to each process, and synchronizes that with the paging.

If the program accesses memory that does not belong to it, the kernel handles a page-fault, and decides what to do:

if it is above the maximum stack size, allocate those pages to the process
otherwise, send a SIGSEGV to the process, which usually kills it

When an ELF file is loaded by the kernel to start a program with the exec system call, the kernel automatically registers text, data, BSS and stack for the program.

The brk and mmap areas can be modified by request of the program through the brk and mmap system calls. But the kernel can also deny the program those areas if there is not enough memory.

brk and mmap can be used to implement malloc, or the so called "heap".

mmap is also used to load dynamically loaded libraries into the program's memory so that it can access and run it.

Stack allocation: stackoverflow.com/questions/17671423/stack-allocation-for-process

Calculating exact addresses Things are complicated by:

Address Space Layout Randomization.
the fact that environment variables, CLI arguments, and some ELF header data take up initial stack space: unix.stackexchange.com/questions/145557/how-does-stack-allocation-work-in-linux/239323#239323

Why the text does not start at 0: stackoverflow.com/questions/14795164/why-do-linux-program-text-sections-start-at-0x0804800-and-stack-tops-start-at-0

 Read the full article

x86 Paging Tutorial / Copy-on-write by

Ciro Santilli  40 Updated 2025-07-16

 View more

en.wikipedia.org/wiki/Copy-on-write

Besides a missing page, a very common source of page faults is copy-on-write (COW).

Page tables have extra flags that allow the OS to mark a page a read-only.

Those page faults only happen when a process tries to write to the page, and not read from it.

When Linux forks a process:

instead of copying all the pages, which is unnecessarily costly, it makes the page tables of the two process point to the same physical address.
it marks those linear addresses as read-only
whenever one of the processes tries to write to a page, the makes a copy of the physical memory, and updates the pages of the two process to point to the two different physical addresses

 Read the full article

x86 Paging Tutorial / Linux source tree by

Ciro Santilli  40 Updated 2025-07-16

 View more

In v4.2, look under arch/x86/:

include/asm/pgtable*
include/asm/page*
mm/pgtable*
mm/page*

There seems to be no structs defined to represent the pages, only macros: include/asm/page_types.h is specially interesting. Excerpt:

#define _PAGE_BIT_PRESENT   0   /* is present */
#define _PAGE_BIT_RW        1   /* writeable */
#define _PAGE_BIT_USER      2   /* userspace addressable */
#define _PAGE_BIT_PWT       3   /* page write through */

arch/x86/include/uapi/asm/processor-flags.h defines CR0, and in particular the PG bit position:

#define X86_CR0_PG_BIT      31 /* Paging */

 Read the full article

x86 Paging Tutorial / Single level paging scheme visualization by

Ciro Santilli  40 Updated 2025-07-16

 View more

This is how the memory could look like in a single level paging scheme:

Links   Data                    Physical address

      +-----------------------+ 2^32 - 1
      |                       |
      .                       .
      |                       |
      +-----------------------+ page0 + 4k
      | data of page 0        |
+---->+-----------------------+ page0
|     |                       |
|     .                       .
|     |                       |
|     +-----------------------+ pageN + 4k
|     | data of page N        |
|  +->+-----------------------+ pageN
|  |  |                       |
|  |  .                       .
|  |  |                       |
|  |  +-----------------------+ CR3 + 2^20 * 4
|  +--| entry[2^20-1] = pageN |
|     +-----------------------+ CR3 + 2^20 - 1 * 4
|     |                       |
|     .    many entires       .
|     |                       |
|     +-----------------------+ CR3 + 2 * 4
|  +--| entry[1] = page1      |
|  |  +-----------------------+ CR3 + 1 * 4
+-----| entry[0] = page0      |
   |  +-----------------------+ <--- CR3
   |  |                       |
   |  .                       .
   |  |                       |
   |  +-----------------------+ page1 + 4k
   |  | data of page 1        |
   +->+-----------------------+ page1
      |                       |
      .                       .
      |                       |
      +-----------------------+  0

Notice that:

the CR3 register points to the first entry of the page table
the page table is just a large array with 2^20 page table entries
each entry is 4 bytes big, so the array takes up 4 MiB
each page table contains the physical address a page
each page is a 4 KiB aligned 4 KiB chunk of memory that user processes may use
we have 2^20 table entries. Since each page is 4 KiB == 2^12, this covers the whole 4 GiB (2^32) of 32-bit memory

 Read the full article

x86 Paging Tutorial / Single level paging scheme numerical translation example by

Ciro Santilli  40 Updated 2025-07-16

 View more

Suppose that the OS has setup the following page tables for process 1:

entry index   entry address       page address   present
-----------   ------------------  ------------   -------
0             CR3_1 + 0      * 4  0x00001        1
1             CR3_1 + 1      * 4  0x00000        1
2             CR3_1 + 2      * 4  0x00003        1
3             CR3_1 + 3      * 4                 0
...
2^20-1        CR3_1 + 2^20-1 * 4  0x00005        1

and for process 2:

entry index   entry address       page address   present
-----------   -----------------   ------------   -------
0             CR3_2 + 0      * 4  0x0000A        1
1             CR3_2 + 1      * 4  0x12345        1
2             CR3_2 + 2      * 4                 0
3             CR3_2 + 3      * 4  0x00003        1
...
2^20-1        CR3_2 + 2^20-1 * 4  0xFFFFF        1

Before process 1 starts running, the OS sets its cr3 to point to the page table 1 at CR3_1.

When process 1 tries to access a linear address, this is the physical addresses that will be actually accessed:

linear     physical
---------  ---------
00000 001  00001 001
00000 002  00001 002
00000 003  00001 003
00000 FFF  00001 FFF
00001 000  00000 000
00001 001  00000 001
00001 FFF  00000 FFF
00002 000  00003 000
FFFFF 000  00005 000

To switch to process 2, the OS simply sets cr3 to CR3_2, and now the following translations would happen:

linear     physical
---------  ---------
00000 002  0000A 002
00000 003  0000A 003
00000 FFF  0000A FFF
00001 000  12345 000
00001 001  12345 001
00001 FFF  12345 FFF
00004 000  00003 000
FFFFF 000  FFFFF 000

Step-by-step translation for process 1 of logical address 0x00000001 to physical address 0x00001001:

split the linear address into two parts:
```
| page (20 bits) | offset (12 bits) |
```
So in this case we would have:
*page = 0x00000. This part must be translated to a physical location.
*offset = 0x001. This part is added directly to the page address, and is not translated: it contains the position within the page.
look into Page table 1 because cr3 points to it.
The hardware knows that this entry is located at RAM address CR3 + 0x00000 * 4 = CR3:
*0x00000 because the page part of the logical address is 0x00000
*4 because that is the fixed size in bytes of every page table entry
since it is present, the access is valid
by the page table, the location of page number 0x00000 is at 0x00001 * 4K = 0x00001000.
to find the final physical address we just need to add the offset:
```
  00001 000
+ 00000 001
  ---------
  00001 001
```
because 00001 is the physical address of the page looked up on the table and 001 is the offset.
We shift 00001 by 12 bits because the pages are always aligned to 4 KiB.
The offset is always simply added the physical address of the page.
the hardware then gets the memory at that physical location and puts it in a register.

Another example: for logical address 0x00001001:

the page part is 00001, and the offset part is 001
the hardware knows that its page table entry is located at RAM address: CR3 + 1 * 4 (1 because of the page part), and that is where it will look for it
it finds the page address 0x00000 there
so the final address is 0x00000 * 4k + 0x001 = 0x00000001

 Read the full article

x86 Paging Tutorial / Multiple addresses translate to a single physical address by

Ciro Santilli  40 Updated 2025-07-16

 View more

The same linear address can translate to different physical addresses for different processes, depending only on the value inside cr3.

Both linear addresses 00002 000 from process 1 and 00004 000 from process 2 point to the same physical address 00003 000. This is completely allowed by the hardware, and it is up to the operating system to handle such cases.

This often in normal operation because of Copy-on-write (COW), which be explained elsewhere.

Such mappings are sometime called "aliases".

 Read the full article

x86 Paging Tutorial / Identity mapping by

Ciro Santilli  40 Updated 2025-07-16

 View more

FFFFF 000 points to its own physical address FFFFF 000. This kind of translation is called an "identity mapping", and can be very convenient for OS-level debugging.

 Read the full article

 Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.

Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!

Video 1.

Intro to OurBigBook

. Source.

We have two killer features:

topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
Articles of different users are sorted by upvote within each article page. This feature is a bit like:
- a Wikipedia where each user can have their own version of each article
- a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
Figure 1.
Screenshot of the "Derivative" topic page
. View it live at: ourbigbook.com/go/topic/derivative
Video 2.
OurBigBook Web topics demo
. Source.
local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
- to OurBigBook.com to get awesome multi-user features like topics and likes
- as HTML files to a static website, which you can host yourself for free on many external providers like GitHub Pages, and remain in full control
This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
Figure 2.
You can publish local OurBigBook lightweight markup files to either OurBigBook.com or as a static website
.
Figure 3.
Visual Studio Code extension installation
.
Figure 4.
Visual Studio Code extension tree navigation
.
Figure 5.
Web editor
. You can also edit articles on the Web editor without installing anything locally.
Video 3.
Edit locally and publish demo
. Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
Video 4.
OurBigBook Visual Studio Code extension editing and navigation demo
. Source.
Internal cross file references done right:
Infinitely deep tables of contents:
Figure 6.
Dynamic article tree with infinitely deep table of contents
.
Live URL: ourbigbook.com/cirosantilli/chordate
Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade