Experiments explained by QED but not by the Dirac equation:
2s/2p energy split in the hydrogen emission spectrum, not predicted by the Dirac equation, but explained by quantum electrodynamics, which is one of the first great triumphs of that theory.
Note that for atoms with multiple electrons, 2s/2p shifts are expected: Why does 2s have less energy than 1s if they have the same principal quantum number?. The surprise was observing that on hydrogen which only has one electron.
Initial experiment: Lamb-Retherford experiment.
On the return from the train from the Shelter Island Conference in New York, Hans Bethe managed to do a non-relativistic calculation of the Lamb shift. He then published as The Electromagnetic Shift of Energy Levels by Hans Bethe (1947) which is still paywalled as of 2021, fuck me: journals.aps.org/pr/abstract/10.1103/PhysRev.72.339 by Physical Review.
The Electromagnetic Shift of Energy Levels Freeman Dyson (1948) published on Physical Review is apparently a relativistic analysis of the same: journals.aps.org/pr/abstract/10.1103/PhysRev.73.617 also paywalled as of 2021.
TODO how do the infinities show up, and how did people solve them?
Video 1. Lamb shift by Dr. Nissar Ahmad (2020) Source. Whiteboard Lecture about the phenomena, includes description of the experiment. Seems quite good.
Video 2. Murray Gell-Mann - The race to calculate the relativistic Lamb shift by Web of Stories (1997) Source. Quick historical overview. Mentions that Richard Feynman and Julian Schwinger were using mass renormalization and cancellation if infinities. He says that French and Weisskopf actually managed to do the correct calculations first with a less elegant method.
www.mdpi.com/2624-8174/2/2/8/pdf History and Some Aspects of the Lamb Shift by G. Jordan Maclay (2019)
Video 3. Freeman Dyson - The Lamb shift by Web of Stories (1998) Source.
Mentions that he moved to the USA from the United Kingdom specifically because great experiments were being carried at Columbia University, which is where the Lamb-Retherford experiment was done, and that Isidor Isaac Rabi was the head at the time.
He then explains mass renormalization briefly: instead of calculating from scratch, you just compare the raw electron to the bound electron and take the difference. Both of those have infinities in them, but the difference between them cancels out those infinities.
Video 4. Hans Bethe - The Lamb shift (1996) Source.
Ahh, Hans is so old in that video, it is sad to see. He did live a lot tough. Mentions that the shift is of about 1000 MHz.
The following video: www.youtube.com/watch?v=vZvQg3bkV7s Hans Bethe - Calculating the Lamb shift.
Video 5. Lamb shift by Vidya-mitra (2018) Source.
Published as "Fine Structure of the Hydrogen Atom by a Microwave Method" by Willis Lamb and Robert Retherford (1947) on Physical Review. This one actually has open accesses as of 2021, miracle! journals.aps.org/pr/pdf/10.1103/PhysRev.72.241
Microwave technology was developed in World War II for radar, notably at the MIT Radiation Laboratory. Before that, people were using much higher frequencies such as the visible spectrum. But to detect small energy differences, you need to look into longer wavelengths.
This experiment was fundamental to the development of quantum electrodynamics. As mentioned at Genius: Richard Feynman and Modern Physics by James Gleick (1994) chapter "Shrinking the infinities", before the experiment, people already knew that trying to add electromagnetism to the Dirac equation led to infinities using previous methods, and something needed to change urgently. However for the first time now the theorists had one precise number to try and hack their formulas to reach, not just a philosophical debate about infinities, and this led to major breakthroughs. The same book also describes the experiment briefly as:
Willis Lamb had just shined a beam of microwaves onto a hot wisp of hydrogen blowing from an oven.
It is two pages and a half long.
They were at Columbia University in the Columbia Radiation Laboratory. Robert was Willis' graduate student.
Previous less experiments had already hinted at this effect, but they were too imprecise to be sure.
This was one of the first two great successes of quantum electrodynamics, the other one being the Lamb shift.
In youtu.be/UKbp85zpdcY?t=52 from freeman Dyson Web of Stories interview (1998) Dyson mentions that the original key experiment was from Kusch and Foley from Columbia University, and that in 1948, Julian Schwinger reached the correct value from his calculations.
Bibliography:
Published on Physical Review by Polykarp Kusch and Foley.
TODO: in high level terms, why is QED more general than just solving the Dirac equation, and therefore explaining quantum electrodynamics experiments?
Also, is it just a bunch of differential equation (like the Dirac equation itself), or does it have some other more complicated mathematical formulation, as seems to be the case? Why do we need something more complicated than
Advanced quantum mechanics by Freeman Dyson (1951) mentions:
A Relativistic Quantum Theory of a Finite Number of Particles is Impossible.