Parallel computing

Parallel computing performs multiple parts of a computation simultaneously, typically on several processors or execution units. It requires decomposing work and coordinating data dependencies, communication and synchronization. Concurrent computing also permits overlapping activities on a single processor, so concurrency alone does not imply simultaneous hardware execution.

If fraction $s$ of a fixed computation remains serial and the rest divides perfectly among $p$ processors, the idealized runtime is $T_p=T_1(s+(1-s)/p)$. Thus Amdahl's law gives speedupfor $s>0$. With a 10 percent serial fraction and 10 processors, the ideal speedup is about 5.26, before communication overhead. The calculation assumes a fixed workload; changing the workload size changes the question. Load imbalance and limited memory bandwidth can further reduce the achieved gain.