New articles - OurBigBook.com

Page size extension.

Allows for pages to be 4M (or 2M if PAE is on) in length instead of 4K.

PSE is turned on and off via the PSE bit of cr4.

x86 Paging Tutorial / PAE and PSE page table schemes by

Ciro Santilli 37 Updated 2025-07-16

If either PAE and PSE are active, different paging level schemes are used:

no PAE and no PSE: 10 | 10 | 12
no PAE and PSE: 10 | 22.
22 is the offset within the 4Mb page, since 22 bits address 4Mb.
PAE and no PSE: 2 | 9 | 9 | 12
The design reason why 9 is used twice instead of 10 is that now entries cannot fit anymore into 32 bits, which were all filled up by 20 address bits and 12 meaningful or reserved flag bits.
The reason is that 20 bits are not enough anymore to represent the address of page tables: 24 bits are now needed because of the 4 extra wires added to the processor.
Therefore, the designers decided to increase entry size to 64 bits, and to make them fit into a single page table it is necessary reduce the number of entries to 2^9 instead of 2^10.
The starting 2 is a new Page level called Page Directory Pointer Table (PDPT), since it points to page directories and fill in the 32 bit linear address. PDPTs are also 64 bits wide.
cr3 now points to PDPTs which must be on the fist four 4GB of memory and aligned on 32 bit multiples for addressing efficiency. This means that now cr3 has 27 significative bits instead of 20: 2^5 for the 32 multiples * 2^27 to complete the 2^32 of the first 4GB.
PAE and PSE: 2 | 9 | 21
Designers decided to keep a 9 bit wide field to make it fit into a single page.
This leaves 23 bits. Leaving 2 for the PDPT to keep things uniform with the PAE case without PSE leaves 21 for offset, meaning that pages are 2M wide instead of 4M.

x86 Paging Tutorial / TLB by

Ciro Santilli 37 Updated 2025-07-16

The Translation Lookahead Buffer (TLB) is a cache for paging addresses.

Since it is a cache, it shares many of the design issues of the CPU cache, such as associativity level.

This section shall describe a simplified fully associative TLB with 4 single address entries. Note that like other caches, real TLBs are not usually fully associative.

x86 Paging Tutorial / Basic TLB operation by

Ciro Santilli 37 Updated 2025-07-16

After a translation between linear and physical address happens, it is stored on the TLB. For example, a 4 entry TLB starts in the following state:

  valid  linear  physical
  -----  ------  --------
> 0      00000   00000
  0      00000   00000
  0      00000   00000
  0      00000   00000

The > indicates the current entry to be replaced.

And after a page linear address 00003 is translated to a physical address 00005, the TLB becomes:

  valid  linear  physical
  -----  ------  --------
  1      00003   00005
> 0      00000   00000
  0      00000   00000
  0      00000   00000

and after a second translation of 00007 to 00009 it becomes:

  valid  linear  physical
  -----  ------  --------
  1      00003   00005
  1      00007   00009
> 0      00000   00000
  0      00000   00000

Now if 00003 needs to be translated again, hardware first looks up the TLB and finds out its address with a single RAM access 00003 --> 00005.

Of course, 00000 is not on the TLB since no valid entry contains 00000 as a key.

x86 Paging Tutorial / TLB replacement policy by

Ciro Santilli 37 Updated 2025-07-16

When TLB is filled up, older addresses are overwritten. Just like CPU cache, the replacement policy is a potentially complex operation, but a simple and reasonable heuristic is to remove the least recently used entry (LRU).

With LRU, starting from state:

  valid  linear  physical
  -----  ------  --------
> 1      00003   00005
  1      00007   00009
  1      00009   00001
  1      0000B   00003

adding 0000D -> 0000A would give:

  valid  linear  physical
  -----  ------  --------
  1      0000D   0000A
> 1      00007   00009
  1      00009   00001
  1      0000B   00003

x86 Paging Tutorial / CAM by

Ciro Santilli 37 Updated 2025-07-16

Using the TLB makes translation faster, because the initial translation takes one access per TLB level, which means 2 on a simple 32 bit scheme, but 3 or 4 on 64 bit architectures.

The TLB is usually implemented as an expensive type of RAM called content-addressable memory (CAM). CAM implements an associative map on hardware, that is, a structure that given a key (linear address), retrieves a value.

Mappings could also be implemented on RAM addresses, but CAM mappings may required much less entries than a RAM mapping.

For example, a map in which:

both keys and values have 20 bits (the case of a simple paging schemes)
at most 4 values need to be stored at each time

could be stored in a TLB with 4 entries:

linear  physical
------  --------
00000   00001
00001   00010
00010   00011
FFFFF   00000

However, to implement this with RAM, it would be necessary to have 2^20 addresses:

linear  physical
------  --------
00000   00001
00001   00010
00010   00011
... (from 00011 to FFFFE)
FFFFF   00000

which would be even more expensive than using a TLB.

x86 Paging Tutorial / Invalidating TLB entries by

Ciro Santilli 37 Updated 2025-07-16

When the process changes, cr3 change to point to the page table of the new current process.

This creates a problem: the TLB is now filled with a bunch of cached entries for the old process.

A simple and naive solution would be to completely invalidate the TLB whenever the cr3 changes.

However, this is would not be very efficient, because it often happens that we switch back to process 1 before process 2 has completely used up the entire TLB cache entries.

The solution for this is to use so called "Address Space Identifiers" (ASID) as mentioned in sources such as:

Basically, the OS assigns a different ASID for each process, and then TLB entries are automatically also tagged with that ASID. This way when the process makes an access, the TLB can determine if a hit is actually for the current process, or if it is an old address coincidence with another process.

The x86 also offers the invlpg instruction which explicitly invalidates a single TLB entry. Other architectures offer even more instructions to invalidated TLB entries, such as invalidating all entries on a given range.

x86 Paging Tutorial / Linux kernel usage by

Ciro Santilli 37 Updated 2025-07-16

The Linux kernel makes extensive usage of the paging features of x86 to allow fast process switches with small data fragmentation.

There are also however some features that the Linux kernel might not use, either because they are only for backwards compatibility, or because the Linux devs didn't feel it was worth it yet.

x86 Paging Tutorial / Play with physical addresses in Linux by

Ciro Santilli 37 Updated 2025-07-16

Convert virtual addresses to physical from user space with /proc/<pid>/pagemap and from kernel space with virt_to_phys:

Dump all page tables from userspace with /proc/<pid>/maps and /proc/<pid>/pagemap:

Read and write physical addresses from userspace with /dev/mem:

x86 Paging Tutorial / Kernel vs process memory layout by

Ciro Santilli 37 Updated 2025-07-16

The Linux Kernel reserves two zones of virtual memory:

one for kernel memory
one for programs

The exact split is configured by CONFIG_VMSPLIT_.... By default:

on 32-bit:

the bottom 3/4 is program space: 00000000 to BFFFFFFF

the top 1/4 is kernel memory: C0000000 to FFFFFFFF, like this:

------------------ FFFFFFFF
Kernel
------------------ C0000000
------------------ BFFFFFFF


Process


------------------ 00000000

on 64-bit: currently only 48-bits are actually used, split into two equally sized disjoint spaces. The Linux kernel just assigns:
- the bottom part to processes 00000000 00000000 to 008FFFFF FFFFFFFF
- the top part to the kernel: FFFF8000 00000000 to FFFFFFFF FFFFFFFF, like this:
  ------------------ FFFFFFFF Kernel ------------------ C0000000 (not addressable) ------------------ BFFFFFFF Process ------------------ 00000000

Kernel memory is also paged.

In previous versions, the paging was continuous, but with HIGHMEM this changed.

There is no clear physical memory split: stackoverflow.com/questions/30471742/physical-memory-userspace-kernel-split-on-linux-x86-64

x86 Paging Tutorial / Process memory layout by

Ciro Santilli 37 Updated 2025-07-16

For each process, the virtual address space looks like this:

------------------ 2^32 - 1
Stack (grows down)
v v v v v v v v v
------------------

(unmapped)

------------------ Maximum stack size.


(unmapped)


-------------------
mmap
-------------------


(unmapped)


-------------------
^^^^^^^^^^^^^^^^^^^
brk (grows up)
-------------------
BSS
-------------------
Data
-------------------
Text
-------------------

------------------- 0

The kernel maintains a list of pages that belong to each process, and synchronizes that with the paging.

If the program accesses memory that does not belong to it, the kernel handles a page-fault, and decides what to do:

if it is above the maximum stack size, allocate those pages to the process
otherwise, send a SIGSEGV to the process, which usually kills it

When an ELF file is loaded by the kernel to start a program with the exec system call, the kernel automatically registers text, data, BSS and stack for the program.

The brk and mmap areas can be modified by request of the program through the brk and mmap system calls. But the kernel can also deny the program those areas if there is not enough memory.

brk and mmap can be used to implement malloc, or the so called "heap".

mmap is also used to load dynamically loaded libraries into the program's memory so that it can access and run it.

Stack allocation: stackoverflow.com/questions/17671423/stack-allocation-for-process

Calculating exact addresses Things are complicated by:

Address Space Layout Randomization.
the fact that environment variables, CLI arguments, and some ELF header data take up initial stack space: unix.stackexchange.com/questions/145557/how-does-stack-allocation-work-in-linux/239323#239323

Why the text does not start at 0: stackoverflow.com/questions/14795164/why-do-linux-program-text-sections-start-at-0x0804800-and-stack-tops-start-at-0

x86 Paging Tutorial / Copy-on-write by

Ciro Santilli 37 Updated 2025-07-16

en.wikipedia.org/wiki/Copy-on-write

Besides a missing page, a very common source of page faults is copy-on-write (COW).

Page tables have extra flags that allow the OS to mark a page a read-only.

Those page faults only happen when a process tries to write to the page, and not read from it.

When Linux forks a process:

instead of copying all the pages, which is unnecessarily costly, it makes the page tables of the two process point to the same physical address.
it marks those linear addresses as read-only
whenever one of the processes tries to write to a page, the makes a copy of the physical memory, and updates the pages of the two process to point to the two different physical addresses

Red clump by

Red clump stars are a specific type of red giant star in the later stages of stellar evolution. These stars are typically in the horizontal branch phase, which follows the giant phase of a star's life cycle. Red clump stars represent a phase where stars have exhausted the hydrogen in their cores and are now fusing helium into carbon and oxygen. They are somewhat hotter and brighter than ordinary red giants, and their spectra show strong absorption lines indicative of helium fusion.

Exoplanets detected by radial velocity by

Exoplanets detected by radial velocity, also known as the Doppler method or the radial velocity method, is a technique used to identify exoplanets by observing the gravitational influence they have on their host stars. This method takes advantage of the Doppler effect, where the light emitted by a star shifts in wavelength depending on its motion relative to an observer.

Collisional excitation by

Collisional excitation is a process in which an atom or molecule absorbs energy during a collision with a particle, such as another atom, molecule, or electron. This energy transfer can promote an electron within the atom or molecule to a higher energy state, or excited state. Here's how it works: 1. **Encounter**: During a collision, kinetic energy from the colliding particle (which can be a gas particle or an electron) is transferred to the target atom or molecule.

Deslandres table by

The Deslandres table, also known as the Deslandres chart, is a tool used in the field of astronomy and astrophysics to facilitate the classification and analysis of celestial bodies' spectra, particularly stars. Named after the French astronomer Camille Deslandres, the table organizes spectral lines based on their wavelengths. In detail, the Deslandres table presents a systematic arrangement of the absorption or emission lines observed in the spectra of stars.

Diffuse series by

The term "diffuse series" can refer to different concepts depending on the context. However, it is not a widely recognized term in any specific field, such as mathematics, physics, or biology. It might be a misinterpretation or a specific concept within a niche area that is not broadly known.

Ellis R. Lippincott Award by

The Ellis R. Lippincott Award is an honor presented by the American Association of Law Libraries (AALL). It recognizes significant contributions to the field of legal information and law librarianship, typically through excellence in legal research, teaching, and the development of legal information resources. Named after a prominent figure in law librarianship, the award underscores the importance of innovation, leadership, and dedication in legal information services.

Fluorescence spectroscopy by

Fluorescence spectroscopy is an analytical technique used to measure the fluorescence emitted by a substance after it has absorbed light or other electromagnetic radiation. This technique is based on the principle of fluorescence, where certain molecules (fluorophores) absorb light at a specific wavelength and subsequently emit light at a longer wavelength.

GF method by

The GF method, or the Galerkin Finite Element method, is a numerical technique used for solving differential equations, particularly in the fields of engineering and applied mathematics. It is a type of finite element method (FEM) that combines the Galerkin method—which is a technique for converting a continuous problem (like a differential equation) into a discrete one—with the finite element analysis framework.