Generalization of AlphaGo Zero that plays Go, chess and shogi.

www.quora.com/Which-chess-engine-would-be-stronger-Alpha-Zero-or-Stockfish-12/answer/Felix-Zaslavskiy explains that it beat Stockfish 8. But then Stockfish was developed further and would start to beat it. We know this because although AlphaZero was closed source, they released the trained artificial neural network, so it was possible to replay AlphaZero at its particular stage of training.

www.gvgai.net (dead as of 2023)

The project kind of died circa 2020 it seems, a shame. Likely they funding ran out. The domain is dead as of 2023, last archive from 2022: web.archive.org/web/20220331022932/http://gvgai.net/ is marked as funded by DeepMind. Researchers really should use university/GitHub domain names!

Similar goals to Ciro's 2D reinforcement learning games, but they were focusing mostly on discrete games.

They have some source at: github.com/GAIGResearch/GVGAI TODO review

A published book at: gaigresearch.github.io/gvgaibook/

From QMUL Game AI Research Group:From other universities:TODO check:

This kind of died at some point checked as of 2023.

Julian Togelius cites it e.g. at: togelius.blogspot.com/2016/07/which-games-are-useful-for-testing.html

so what's that point of "Open" in the name anymore??

Development ceased in 2021 and was taken up by a not-for-profit as Farama Gymnasium.

The key takeaway is that setting an explicit value function to an AGI entity is a good way to destroy the world due to poor AI alignment. We are more likely to not destroy by creating an AI whose goals is to "do want humans what it to do", but in a way that it does not know before hand what it is that humans want, and it has to learn from them. This approach appears to be known as reward modeling.

Some other cool ideas:

Good points:

Given a bunch of points in $n$ dimensions, PCA maps those points to a new $p$ dimensional space with $p\le n$.

$p$ is a hyperparameter, $p=1$ and $p=2$ are common choices when doing dataset exploration, as they can be easily visualized on a planar plot.

The mapping is done by projecting all points to a $p$ dimensional hyperplane. PCA is an algorithm for choosing this hyperplane and the coordinate system within this hyperplane.

The hyperplane choice is done as follows:

www.sartorius.com/en/knowledge/science-snippets/what-is-principal-component-analysis-pca-and-how-it-is-used-507186 provides an OK-ish example with a concrete context. In there, each point is a country, and the input data is the consumption of different kinds of foods per year, e.g.:so in this example, we would have input points in 4D.

The question is then: we want to be able to identify the country by what they eat.

Suppose that every country consumes the same amount of flour every year. Then, that number doesn't tell us much about which country each point represents (has the least variance), and the first PCA axes would basically never point anywhere near that direction.

Another cool thing is that PCA seems to automatically account for linear dependencies in the data, so it skips selecting highly correlated axes multiple times. For example, suppose that dry codfish and olive oil consumption are very high in Portugal and Spain, but very low in Germany and Poland. Therefore, the variation is very high in those two parameters, and contains a lot of information.

However, suppose that dry codfish consumption is also directly proportional to olive oil consumption. Because of this, it would be kind of wasteful if we selected:since the information about codfish already tells us the olive oil. PCA apparently recognizes this, and instead picks the first axis at a 45 degree angle to both dry codfish and olive oil, and then moves on to something else for the second axis.

We can see that much like the rest of machine learning, PCA can be seen as a form of compression.

A parameter that you choose which determines how the algorithm will perform.

In the case of machine learning in particular, it is not part of the training data set.

Hyperparameters can also be considered in domains outside of machine learning however, e.g. the step size in partial differential equation solver is entirely independent from the problem itself and could be considered a hyperparamter. One difference from machine learning however is that step size hyperparameters in numerical analysis are clearly better if smaller at a higher computational cost. In machine learning however, there is often an optimum somewhere, beyond which overfitting becomes excessive.

Philosophically, machine learning can be seen as a form of lossy compression.

And if we make it too lossless, then we are basically overfitting.

Bibliography: