Ciro Santilli @cirosantilli 40

If you are going to live, you might as well chase one of them.

You might not achieve them in your lifetime, but you never know. At some point, the pieces just "fall into place", and they happen.

And they will all come from deep tech.

Ciro Santilli would like to contribute to them. but this is a bit less realistic than software projects.

But who knows? Maybe he can code some stuff in those areas.

And one can at least have some fun by learning deeply about those subjects.

Notable examples:

The good:

The bad:

Given a bunch of points in $n$ dimensions, PCA maps those points to a new $p$ dimensional space with $p\le n$.

$p$ is a hyperparameter, $p=1$ and $p=2$ are common choices when doing dataset exploration, as they can be easily visualized on a planar plot.

The mapping is done by projecting all points to a $p$ dimensional hyperplane. PCA is an algorithm for choosing this hyperplane and the coordinate system within this hyperplane.

The hyperplane choice is done as follows:

www.sartorius.com/en/knowledge/science-snippets/what-is-principal-component-analysis-pca-and-how-it-is-used-507186 provides an OK-ish example with a concrete context. In there, each point is a country, and the input data is the consumption of different kinds of foods per year, e.g.:so in this example, we would have input points in 4D.

The question is then: we want to be able to identify the country by what they eat.

Suppose that every country consumes the same amount of flour every year. Then, that number doesn't tell us much about which country each point represents (has the least variance), and the first PCA axes would basically never point anywhere near that direction.

Another cool thing is that PCA seems to automatically account for linear dependencies in the data, so it skips selecting highly correlated axes multiple times. For example, suppose that dry codfish and olive oil consumption are very high in Portugal and Spain, but very low in Germany and Poland. Therefore, the variation is very high in those two parameters, and contains a lot of information.

However, suppose that dry codfish consumption is also directly proportional to olive oil consumption. Because of this, it would be kind of wasteful if we selected:since the information about codfish already tells us the olive oil. PCA apparently recognizes this, and instead picks the first axis at a 45 degree angle to both dry codfish and olive oil, and then moves on to something else for the second axis.

We can see that much like the rest of machine learning, PCA can be seen as a form of compression.

This game is quite detailed: www.youtube.com/watch?v=w4Jmqp8a_bU

Or equivalently, the set of rows is orthonormal, and so is the set of columns. TODO proof that it is equivalent to the orthogonal group is the group of all matrices that preserve the dot product.