Past exam of the computer science course of the University of Oxford / 2015 / Computer architecture / 3 / a / Solution

A small memory cache close to the CPU can be accessed quickly, but cannot hold the entire useful data set. Increasing its capacity usually increases lookup, wiring and energy costs. A cache hierarchy combines a small fast first level with progressively larger levels that reduce the number of accesses reaching slow main computer memory.

In a simple serial two-level timing model, let $t_1$ be the first-level lookup time, $m_1$ its cache miss rate, $t_2$ the additional second-level lookup time, $m_2$ the second level's local cache miss rate conditional on a first-level miss, and $t_M$ the remaining main-memory cost. ThenThis average memory access time combines the common fast path with a rarer large-capacity path. Several levels balance hit latency, capacity, cost and traffic; making one cache very large does not preserve a tiny first-level hit time. Temporal locality and spatial locality make this arrangement useful.