Ciro Santilli @cirosantilli 40

Notable examples:

The good:

The bad:

Given a bunch of points in $n$ dimensions, PCA maps those points to a new $p$ dimensional space with $p\le n$.

$p$ is a hyperparameter, $p=1$ and $p=2$ are common choices when doing dataset exploration, as they can be easily visualized on a planar plot.

The mapping is done by projecting all points to a $p$ dimensional hyperplane. PCA is an algorithm for choosing this hyperplane and the coordinate system within this hyperplane.

The hyperplane choice is done as follows:

www.sartorius.com/en/knowledge/science-snippets/what-is-principal-component-analysis-pca-and-how-it-is-used-507186 provides an OK-ish example with a concrete context. In there, each point is a country, and the input data is the consumption of different kinds of foods per year, e.g.:so in this example, we would have input points in 4D.

The question is then: we want to be able to identify the country by what they eat.

Suppose that every country consumes the same amount of flour every year. Then, that number doesn't tell us much about which country each point represents (has the least variance), and the first PCA axes would basically never point anywhere near that direction.

Another cool thing is that PCA seems to automatically account for linear dependencies in the data, so it skips selecting highly correlated axes multiple times. For example, suppose that dry codfish and olive oil consumption are very high in Portugal and Spain, but very low in Germany and Poland. Therefore, the variation is very high in those two parameters, and contains a lot of information.

However, suppose that dry codfish consumption is also directly proportional to olive oil consumption. Because of this, it would be kind of wasteful if we selected:since the information about codfish already tells us the olive oil. PCA apparently recognizes this, and instead picks the first axis at a 45 degree angle to both dry codfish and olive oil, and then moves on to something else for the second axis.

We can see that much like the rest of machine learning, PCA can be seen as a form of compression.

This game is quite detailed: www.youtube.com/watch?v=w4Jmqp8a_bU

Or equivalently, the set of rows is orthonormal, and so is the set of columns. TODO proof that it is equivalent to the orthogonal group is the group of all matrices that preserve the dot product.

Smaller files, scalable image size, and editability. Why would you use anything else for programmatically generated images?!?!

Initially there were mathematical reasons why people suspected that all boson needed to have 0 mass as is the case for photons a gluons, see Goldstone's theorem.

However, experiments showed that the W boson and the Z boson both has large non-zero masses.

So people started theorizing some hack that would fix up the equations, and they came up with the higgs mechanism.