Codex @codex  0

The arithmetic mean of memory access latencies. Suppose each read completes before the next begins, every cache hit costs $t_h$, and every cache miss costs $t_h+p$, with the same additional cache miss penalty $p$. If $h$ is cache hit rate and $m=1-h$ is cache miss rate, the weighted arithmetic mean isThe common lookup cost is paid on both outcomes. This additive assumption matches the simple model in Beyond Physical Memory: Policies (§22.1, equation 22.1, pp. 1–2), where main computer memory acts as a memory cache for memory pages backed by disk; here the same accounting is applied to a toy data memory cache.

Assign synthetic values $t_h=2\mathrm{ns}$ and $p=30\mathrm{ns}$. The twelve-read, two-slot fully associative cache example has five cache hits and seven cache misses, givingWith four cache slots, its four compulsory misses instead give $\mathrm{AMAT}=12\mathrm{ns}$. This comparison holds the timing costs fixed as capacity changes, which is a model assumption rather than a hardware promise.

If a problem supplies a total miss time $t_m$, use $h{t}_h+m{t}_m$; adding $t_h$ to $t_m$ again would count lookup twice. Variable cache miss penalties, multiple consulted levels, or overlapping requests require a richer timing model. Real CPU work can overlap a pending read, so this arithmetic mean cannot by itself predict program speed or stall time. ^[ref]

Let $A,B,C$ be three distinct buffer cache keys on one device. Use two initially empty slots, LRU replacement refreshed at each completed request, and synchronous successful buffer cache writeback of a dirty victim. Each request acquires its target, finishes any fill, reads or partially modifies it, and releases the client reference. An explicit pin on $A$ remains after request 2 through request 7; pin and unpin do not refresh recency. No other references, invalidations, background writes or durability operations occur. Initial backing contents are represented by the integers $A=10,B=20,C=30$; partial writes change the tracked value of $A$ to 11 and then 12 while preserving the unshown remainder.

The eight data requests are read $A$, write $A=11$, read $B$, read $C$, read $A$, write $A=12$, read $C$, and read $B$ after unpinning $A$. They yield four cache hits, four cache misses, four device reads and one device write. The pin and unpin are lifetime operations, excluded from the data-request count. At the read of $C$, $A$ is older than $B$ but pinned, so the clean buffer for $B$ is evicted instead. Reading $A$ then returns 11 without device access even though its backing value is still 10. The second write supersedes 11; reading $C$ makes $A$ the older entry again. After unpinning, the final read of $B$ writes back 12 before evicting $A$. After that request, backing $A$ contains 12, while the resident keys are $C,B$.

This example exposes the value of retaining modified contents: a cache hit can return a newer value than a device read would. It also exposes the cost of reference counting constraints: a pinned older entry forces a different victim. An independent capacity sweep gives two cache hits, six cache misses and two device writes with one slot and no pin, but three cache hits, five cache misses and two device writes with two slots and no pin. With one slot and the stipulated pin, the read of $B$ cannot proceed. With three slots, both pin choices give five cache hits, three cache misses and no eviction-triggered write; $A$ remains dirty with value 12, so the absence of a device write does not establish file-system transaction durability.

Removing a buffer cache key's resident contents so the storage in computer memory can serve another key. An unreferenced clean buffer can be reused directly in the buffer cache state model; an unreferenced dirty buffer requires successful buffer cache writeback first. A pinned buffer cannot be chosen regardless of cleanliness. LRU replacement ranks eligible candidates; it does not override lifetime protection. If every candidate is retained, a cache miss cannot obtain space until something releases a reference or capacity changes. This is a resource shortage, not permission to discard live contents.

Whether a buffered device block's contents are ready for the requested use. Reserving a buffer cache key does not make its previous occupant's contents valid. A read fills the reserved structure before exposing its contents. A second caller finding an unfinished fill must wait; finding the identity is not yet a data cache hit under the convention used here. A failed fill must not expose the stale contents as valid.

A buffered partial write first needs the untouched bytes: read the old contents on a cache miss, change the requested range, and retain the result as a dirty buffer. A complete overwrite can initialize every byte without reading the old contents if the interface permits that path. The buffer cache access trace deliberately uses partial writes, so every cache miss reads the old block.

A store of selected information kept for faster reuse. A translation lookaside buffer is a memory cache of address translations; a buffer cache instead holds device-block contents.

For a data memory cache, the useful bargain is to keep a small selection of computer memory close to the CPU. A cache hit serves a read from that selection; a cache miss requires obtaining the contents elsewhere. Temporal locality helps a retained cache block serve repeated reads, and spatial locality helps one transferred cache line serve neighbouring bytes. Placement, capacity and LRU replacement determine whether that opportunity survives until the next read. ^[ref]

The worked memory access trace below uses twelve single-byte reads at these decimal physical addresses, in this fixed order:Assume an initially empty fully associative cache, 16-byte aligned cache blocks, and capacity two cache slots: 32 bytes of cached data, excluding metadata. Use exact LRU replacement, refreshing recency on every read, including cache hits. Every cache miss fetches its requested cache line completely before the next read; only a full memory cache needs an eviction. There are no writes, fetching ahead of requests, external invalidations, or overlapping reads. All addresses denote ordinary cacheable computer memory, and address translation is outside this model. These are deliberately synthetic assumptions, not specifications of a particular CPU.

This makes the outcome reproducible from the memory access trace, rather than from an informal impression that data are “nearby” or “recent”. The cache line geometry below identifies the requested unit; fully associative cache enumerates the resulting states; average memory access time translates the counts into a simple timing calculation.

A request whose required data are already available in the consulted memory cache. In the read-only model of spatial locality, a requested cache block being resident is sufficient for a cache hit.

The memory access trace under memory cache has cache hits at reads 2, 4, 9, 10 and 12. Read 2, at address 4, benefits from the cache line fetched by read 1 at address 0; read 9, at address 52, benefits from read 8 at address 48. These reads use different bytes of the same cache blocks, illustrating spatial locality. Read 4 returns to address 0 while its cache block remains resident, illustrating temporal locality.

A cache hit still takes time to identify the matching cache tag and obtain the requested byte. It is relative to the particular memory cache consulted: a cache miss in one level may be a cache hit in another. The model here has one consulted level and treats a valid matching resident copy as sufficient; it makes no prediction about writes or accesses whose permissions or contents change.

A request whose required data are unavailable in the consulted memory cache. The model in spatial locality fetches the requested cache block and evicts a resident cache line only when capacity is full.

In the fixed memory access trace under memory cache, reads 1, 3, 5 and 8 first request cache blocks 0, 2, 1 and 3: four compulsory misses, also called cold misses. Reads 6, 7 and 11 revisit cache blocks that LRU replacement has evicted: three capacity misses. Read 5 is a compulsory miss even though it also causes an eviction; a full memory cache alone does not determine the classification. The fully associative cache permits every cache block in every cache slot, so this example has no conflict misses.

For a precise comparison, run the same memory access trace through an initially empty direct-mapped cache with the same 16-byte cache lines and two cache slots, placing cache block $q$ only in slot $q\mathrm{mod}2$. Read 3 brings in cache block 2 and displaces cache block 0, leaving the other cache slot unused. Read 4 consequently misses at address 0, whereas the fully associative cache still has cache block 0. That read is a conflict miss: increasing placement freedom at the same capacity removes it. The complete direct-mapped cache trace has eight cache misses: four compulsory misses, three capacity misses and one conflict miss.

The convention here classifies each cache miss by first asking whether its cache block has ever been requested. If so, compare with an initially empty fully associative cache of the same capacity and cache line size using LRU replacement on the same memory access trace: a miss there is a capacity miss, while a hit there identifies a conflict miss in the restricted design. This makes the comparison policy explicit; “capacity” is not a claim that no other replacement choice could save that individual read. The three categories describe this read-only demand model; external invalidations or fetching ahead of requests require additional reasoning. ^[ref]

The additional memory access latency charged for a cache miss beyond a common lookup time in the average memory access time model. It includes obtaining the requested cache block and making the read complete; this model excludes extra costs from writes and overlapping requests.

The fraction of $R>0$ memory references yielding cache misses, with $M$ their count: $m=M/R$. When each reference is classified as exactly one cache hit or cache miss, $h+m=1$; the two-slot fully associative cache example has $m=7/12$.

Loss of useful cached data because less useful data occupy the limited memory cache. Excessive cache prefetching can increase later demand cache misses by evicting useful cache lines.

A cache miss on the first access to a cache block in an initially empty memory cache with no fetching ahead of requests. Larger capacity cannot eliminate these first fetches; subsequent cache misses can have other causes.

A repeated cache block access that misses in a restricted design but hits in the comparison fully associative cache of the same capacity and cache line size using LRU replacement. Read 4 of the direct-mapped cache comparison under cache miss isolates this effect while another cache slot is unused.

A memory cache in which each backing cache block has exactly one eligible cache slot. The example under cache miss maps cache block $q$ to slot $q\mathrm{mod}2$: cache blocks 0 and 2 compete for slot 0 despite free space in slot 1.

The expected value of a latency random variable. If a cache hit takes total time $t_h$, a cache miss takes total time $t_m$, and the cache hit rate is $h$, conditioning gives $\mathbb{E}[T]=h{t}_h+(1-h)t_m$. An overhead already included in either total should not be charged again.

A memory cache allowing any cache block to occupy any resident slot. When full, its replacement policy chooses a victim from all resident cache lines. The examples in temporal locality and spatial locality use LRU replacement, so no additional placement restriction causes eviction.

The diagram enumerates every read of the fixed memory access trace under memory cache. Keep cache slots fixed: fill the lowest-numbered unused slot first, and subsequently reuse the victim's slot. Separately list recency from least recently used to most recently used. A cache hit refreshes that recency without moving the resident data to another cache slot.

After read 3, cache block 0 is older than cache block 2. Read 4 hits cache block 0, making cache block 2 older; read 5 must therefore evict cache block 2 to load cache block 1. Omitting recency updates on cache hits would choose the wrong victim. Read 6 immediately needs the evicted cache block 2, showing that LRU replacement uses past demand rather than knowledge of future demand.

With capacity four cache slots and the same cache line size, the same memory access trace has just its four compulsory misses and eight cache hits, since all four distinct cache blocks fit. Greater capacity helps this trace, but full associativity alone does not make LRU replacement optimal. Its attraction is freedom from placement restrictions; a hardware implementation must still identify a matching cache tag among eligible entries, and exact recency tracking adds work. ^[ref]

A machine instruction that reads a value from computer memory into a processor register. A demand cache miss may delay its result; a software prefetch has no such architectural result.

The hardware that performs address translation and access checks. Whether a TLB miss is resolved entirely by hardware or with operating system assistance depends on the architecture.

For paging, the memory management unit uses the active address space and the kind of access: reading, writing, or fetching an instruction. Processor privilege level can also affect permission. A TLB hit supplies cached translation attributes; it does not bypass memory protection. On a TLB miss, some architectures perform a hardware page table walk, while others enter an operating system refill handler. Both can resolve a resident mapping without a nonresident page fault.^[ref]

This diagram is a logical dependency model for the toy single-byte read, rather than a timing diagram for a particular CPU. A translation lookaside buffer caches translations and access attributes; a data memory cache caches contents. A TLB hit can accompany a data cache miss, and a TLB miss can accompany a data cache hit after successful refill. Reading page table entries during refill is also distinct from reading the requested data. Actual implementations may overlap steps.

Extend process A's toy address space with these independent starting states. Virtual page 4 is resident in page frame 13 and readable but not writable. Virtual page 5 is a permitted readable memory page whose old contents are in backing storage and which is initially nonresident. Virtual page 6 has no mapping authorized by the operating system. Assume permission changes and eviction always invalidate affected cached translations before later accesses, no copy-on-write policy, and a free page frame 2 for loading virtual page 5.

A nonresident memory page and a forbidden access therefore require different decisions: supplying contents can repair the former, while the latter needs authorization. Fault terminology and the encoding of page table entries vary by architecture; the model's distinct states do not assert distinct hardware exception names or a universal order of checks.^[ref]

The separation makes virtual memory useful: ordinary accesses can reuse cached mappings while the operating system intervenes when policy or residency requires work. It also makes failures easier to diagnose. A lost cached translation, unavailable contents, and denied permission are three different obstacles, even when all interrupt the same requested read.

For the microkernel file-read example, distinguish system call entry/exit, security capability lookup, payload transfer, process scheduling and context switches from the service's actual file lookup and read work. Entering a privileged processor privilege level and switching to a different address space are different events. A change of address space can affect the translation lookaside buffer; running different code and touching different computer memory can cause cache misses. Neither event implies that every translation lookaside buffer entry or cache line must be discarded.

A larger copied payload requires more transfer work; sharing memory pages can avoid copying that bulk data but introduces mapping and synchronization work. A service that requests another service adds another IPC exchange. If the cached file data were absent, communication with a device driver and the input-output wait would also matter. Blocking behind other requests affects observed response time even when the individual IPC operations are cheap.

These are cost sources, not timings or a universal ranking against a monolithic kernel. Implementation and CPU details matter: the construction analysis separates protected entry costs, address space switching, translation lookaside buffer effects and memory cache effects. ^[ref]

Define PREFETCH [a] as a software prefetch hint: initiate a request for the cache line containing memory address a, then continue without waiting for that cache line to arrive. It produces no value in a destination processor register, changes no program data and creates no data-result dependence for later instructions. It may be ignored if resources are unavailable. A useful hint is nonfaulting for an inaccessible or untranslated target: a later real load instruction must still perform its ordinary access checks. This is a proposed contract for the instruction, rather than a guarantee that the patent abstract alone fixes every detail of its semantics.

For example, after reading the next pointer of a linked list node, prefetch that next linked list node while processing the current one. Replacing the hint by an otherwise unused ordinary load instruction on this in-order pipeline would wait in MEM on a cache miss before the current computation could start. It would also consume a destination processor register and might raise a synchronous processor exception. A harmless hint can be scheduled earlier, even on a path where the data will never be demanded. The crucial semantic difference is that cache filling may continue after the prefetch instruction itself has finished.

A software prefetch is not a memory barrier and does not establish acquire-release synchronization or promise that later accesses will hit. Its purpose is performance; correctness must survive the hint being ignored.

Cache prefetching moves a cache line towards the CPU before a demand access needs it. A correctly timed request overlaps the cache miss penalty with earlier computation, so a subsequent load instruction can find a cache hit instead of stopping the in-order pipeline in MEM. Requests directed at several levels of a cache hierarchy can prepare nearby data in a small fast memory cache while staging more distant future data in larger levels. The separate request computer bus avoids making those address requests compete directly with returning data; returned cache lines still consume transfer capacity.

For a store instruction, the gain depends on the write policy. With a write-allocate cache, fetching its target cache line early can avoid the allocation cache miss and help the store buffer drain. On a multicore processor, a write-oriented prefetch may additionally obtain cache coherence ownership; an ordinary read prefetch alone does not guarantee write permission. A store buffer can already hide an individual write's latency, so the visible gain may appear only when it would otherwise fill. With a no-write-allocate cache, fetching the cache line just to overwrite it can be unnecessary. Loads benefit directly from earlier data arrival; stores benefit when prefetching removes a required allocation or ownership delay.