They seem to do some cool stuff.
They have also declined every one of Ciro Santilli's applications for software engineer jobs before any interview. Ciro always wondered what does it take to get an interview with them. Lilely a PhD? Oh well.
In the early days at least lots of gamedev experience was enough though: www.linkedin.com/in/charles-beattie-0695373/.
- www.quora.com/Will-Google-open-source-AlphaGo Will Google open source AlphaGo?
- www.nature.com/articles/nature16961 Mastering the game of Go with deep neural networks and tree search by Silver et al. (2016), published without source code
Generalization of AlphaGo Zero that plays Go, chess and shogi.
- www.science.org/doi/10.1126/science.aar6404 A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play by Silver et al. (2018), published without source code
- www.quora.com/Is-there-an-Open-Source-version-of-AlphaZero-specifically-the-generic-game-learning-tool-distinct-from-AlphaGo
www.quora.com/Which-chess-engine-would-be-stronger-Alpha-Zero-or-Stockfish-12/answer/Felix-Zaslavskiy explains that it beat Stockfish 8. But then Stockfish was developed further and would start to beat it. We know this because although AlphaZero was closed source, they released the trained artificial neural network, so it was possible to replay AlphaZero at its particular stage of training.
www.gvgai.net (dead as of 2023)
The project kind of died circa 2020 it seems, a shame. Likely they funding ran out. The domain is dead as of 2023, last archive from 2022: web.archive.org/web/20220331022932/http://gvgai.net/. Marks as funded by DeepMind. Researchers really should use university/GitHub domain names!
Similar goals to Ciro's 2D reinforcement learning games, but they were focusing mostly on discrete games.
They have some source at: github.com/GAIGResearch/GVGAI TODO review
A published book at: gaigresearch.github.io/gvgaibook/
From QMUL Game AI Research Group:From other universities:TODO check:
- Simon M. Lucas: gaigresearch.github.io/members/Simon-Lucas, principal investigator
- Diego Perez Liebana www.linkedin.com/in/diegoperezliebana/
- Raluca D. Gaina: www.linkedin.com/in/raluca-gaina-347518114/ from Queen Mary
- Ahmed Khalifa
- Jialin Liu
This kind of died at some point checked as of 2023.
Julian Togelius cites it e.g. at: togelius.blogspot.com/2016/07/which-games-are-useful-for-testing.html
In 2019, OpenAI transitioned from non-profit to for-profit
- www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/ "The AI moonshot was founded in the spirit of transparency. This is the inside story of how competitive pressure eroded that idealism."
- archive.ph/wXBtB How OpenAI Sold its Soul for $1 Billion
- www.reddit.com/r/GPT3/comments/n2eo86/is_gpt3_open_source/
Development ceased in 2021 and was taken up by a not-for-profit as Farama Gymnasium.
OpenAI Gym development by OpenAI ceased in 2021, and the Farama Foundation not for profit took up maintenance of it.
gymnasium==1.1.1 just worked on Ubuntu 24.10 testing with the hello world gym/random_control.py:just works and opens a game window on my desktop.
sudo apt install swig
cd gym
virtualenv -p python3
. .venv/bin/activate
pip install -r requirements-python-3-12.txt
./random_control.py
Lunar Lander environment of Farama Gymnasium with random controls
. This example just passes random commands to the ship so don't expect wonders. The cool thing about it though is that you can open any environment with it e.g.
./random_control.py CarRacing-v3To manually control it we can use gym/moon_play.py:
cd gym
./moon_play.pyManual control is extremely useful to get an intuition about the problem. You will notice immediately that controlling the ship is extremely difficult.
Lunar Lander environment of Farama Gymnasium with manual control
. We slow it down to 10 FPS to give us some fighting chance.
We don't know if it is realistic, but what is certain is that this is definitely not designed to be a fun video game!A good strategy is to land anywhere very slowly and then inch yourself towards the landing pad.
- the legs of the lander are short and soft, and you're not supposed to hit the body on ground, so you have to go very slow
- the thrusters are quite weak and inertia management is super important
- the ground is very slippery
The documentation for it is available at: gymnasium.farama.org/environments/box2d/lunar_lander/ The agent input is described as:so it is a fundamentally flawed robot training example as global x and y coordinates are precisely known.
The state is an 8-dimensional vector: the coordinates of the lander in x & y, its linear velocities in x & y, its angle, its angular velocity, and two booleans that represent whether each leg is in contact with the ground or not.
Variation in the scenario comes from:
- initial speed of vehicle
- shape of lunar surface, but TODO can the ship observe the lunar surface shape in any way? If not, once again, this is a deeply flawed example.
The actions are documented at:so we can make it spin like mad counter clockwise with:
- 0: do nothing
- 1: fire left orientation engine
- 2: fire main engine
- 3: fire right orientation engine
action = 1To actually play the games manually with keyboard, you need to define your own keybindings with gymnasium.utils.play.play. Feature request for default keybindings: github.com/Farama-Foundation/Gymnasium/discussions/1330
There is no C API, you have to go through Python: github.com/Farama-Foundation/Gymnasium/discussions/1181. Shame.
They have video recording support, minimal ex stackoverflow.com/questions/77042526/how-to-record-and-save-video-of-gym-environment/79514542#79514542
Announced at:
It would be cool if they maintained their own list!
github.com/DLR-RM/rl-baselines3-zoo seems to contain some implementations.
Suggested at: github.com/Farama-Foundation/Gymnasium/discussions/1331
Not-for profit that took up OpenAI Gym maintenance after OpenAI dropped it.
Articles by others on the same topic
There are currently no matching articles.