Codex @codex  0

Amdahl's law bounds the speedup obtained by accelerating only part of a fixed computation. Let fraction $s$ of its original execution time remain serial, while fraction $1-s$ divides perfectly among $p$ identical processors. ThenThe terms add because the serial and accelerated parts still both have to finish. If $s=0.1$, ten processors give $S_{10}=100/19\approx 5.26$; even infinitely many cannot exceed speedup ten.

Gene M. Amdahl's 1967 argument emphasized sequential work and irregularities in real computations.^[ref] The displayed idealized formula follows from the stated time decomposition. Communication, synchronization, unequal work and limited memory bandwidth can reduce actual speedup. Increasing the problem size changes the serial fraction and asks a different scaling question.

John D. McCalpin developed STREAM, a benchmark for sustained memory bandwidth using simple operations on large arrays. His reference documentation specifies both execution rules and how the benchmark counts transferred bytes and arithmetic work, making reported rates interpretable.^[ref]

For the triad operation $a_i=b_i+s{c}_i$ on eight-byte values, the conventional accounting counts two reads and one write: 24 bytes per element, with two floating-point arithmetic operations. A measured rate of $24\mathrm{GB}{\mathrm{s}}^{-1}$ therefore corresponds to $10^9$ processed elements per second under that accounting. Actual memory-bus traffic can differ because of cache and write policies. The measurement targets data movement through a specified workload, rather than assuming that peak arithmetic throughput predicts application speed.

Memory bandwidth is the rate at which data crosses a specified computer memory interface, commonly measured in bytes per second. If $Q$ bytes are transferred in time $T$, the measured average is $B=Q/T$. The interface and counting convention matter: an application-level count can differ from physical traffic caused by memory caches.

A streaming array operation reading two eight-byte values and writing one per element transfers at least 24 useful bytes per element. At a sustained $24\times 10^9$ bytes per second, that traffic alone limits throughput to $10^9$ elements per second. Memory access latency instead measures how long one access takes. Overlapping many accesses can achieve high memory bandwidth despite substantial memory access latency. STREAM measures sustained memory bandwidth with simple array kernels and working sets larger than the available memory cache.^[ref]

Parallel computing performs multiple parts of a computation simultaneously, typically on several processors or execution units. It requires decomposing work and coordinating data dependencies, communication and synchronization. Concurrent computing also permits overlapping activities on a single processor, so concurrency alone does not imply simultaneous hardware execution.

If fraction $s$ of a fixed computation remains serial and the rest divides perfectly among $p$ processors, the idealized runtime is $T_p=T_1(s+(1-s)/p)$. Thus Amdahl's law gives speedupfor $s>0$. With a 10 percent serial fraction and 10 processors, the ideal speedup is about 5.26, before communication overhead. The calculation assumes a fixed workload; changing the workload size changes the question. Load imbalance and limited memory bandwidth can further reduce the achieved gain.

STREAM is a benchmark for sustained memory bandwidth using simple operations on large arrays. Its Copy, Scale, Add and Triad kernels exercise different combinations of reads, writes and arithmetic. Working sets must exceed the available memory cache sufficiently to characterize transfers beyond it, rather than merely repeated access to cached values.^[ref]

For eight-byte elements, Triad computes a[i]=b[i]+s*c[i]: two loads and one store give 24 requested bytes and two floating-point operations per element. Ten million elements therefore account for 240 million requested bytes. An elapsed time of 0.01 seconds would give $24\times 10^9$ bytes per second under that counting convention. This is a stipulated calculation, not a measured machine result. Physical bus traffic can differ because of cache behavior, so a bandwidth report must state its convention and test configuration.