Parallel computing
= Parallel computing
<Parallel computing> performs multiple parts of a computation simultaneously, typically on several processors or execution units. It requires decomposing work and coordinating data dependencies, communication and <synchronization>. <Concurrent computing> also permits overlapping activities on a single processor, so concurrency alone does not imply simultaneous hardware execution.
If fraction $s$ of a fixed computation remains serial and the rest divides perfectly among $p$ processors, the idealized runtime is $T_p=T_1(s+(1-s)/p)$. Thus <Amdahl's law> gives speedup
$$
S_p=\frac{1}{s+(1-s)/p}\leq\frac1s
$$
for $s>0$. With a 10 percent serial fraction and 10 processors, the ideal speedup is about 5.26, before communication overhead. The calculation assumes a fixed workload; changing the workload size changes the question. Load imbalance and limited <memory bandwidth> can further reduce the achieved gain.