Incoming links: Two Minute Papers
- github.com/deepmind/meltingpot TODO vs DeepMind Lab2D? Also 2D discrete. Started in 2021.
- github.com/deepmind/ai-safety-gridworlds mentioned e.g. at www.youtube.com/watch?v=CGTkoUidQ8I by Rober Miles
Creating Multimodal Interactive Agents from DeepMind by Two Minute Papers (2023)
Source. www.deepmind.com/blog/building-interactive-agents-in-video-game-worldsOpen-Ended Learning Leads to Generally Capable Agents by DeepMind (2021)
Short name: XLand. Whitepaper: www.deepmind.com/blog/generally-capable-agents-emerge-from-open-ended-play.Prototype: github.com/cirosantilli/Urho3D-cheat
Prior art research: github.com/cirosantilli/awesome-reinforcement-learning-games
Top Down 2D Continuous Game with Urho3D C++ SDL and Box2D for Reinforcement learning by Ciro Santilli (2018)
Source. Source code at: github.com/cirosantilli/Urho3D-cheat.
Screenshot of the basketball stage of Ciro's 2D continuous game
. Source code at: github.com/cirosantilli/rl-game-2d-grid. Big kudos to game-icons.net for the sprites.Less good discrete prototype: github.com/cirosantilli/rl-game-2d-grid YouTube demo: Video 1. "Top Down 2D Continuous Game with Urho3D C++ SDL and Box2D for Reinforcement learning by Ciro Santilli (2018)".
Top Down 2D Discrete Tile Based Game with C++ SDL and Boost R-Tree for Reinforcement Learning by Ciro Santilli (2017)
Source. The goal of this project is to reach artificial general intelligence.
A few initiatives have created reasonable sets of robotics-like games for the purposes of AI development, most notably: OpenAI and DeepMind.
However, all projects so far have only created sets of unrelated games, or worse: focused on closed games designed for humans!
What is really needed is to create a single cohesive game world, designed specifically for this purpose, and with a very large number of game mechanics.
Notably, by "game mechanic" is meant "a magic aspect of the game world, which cannot be explained by object's location and inertia alone" in order to test the the missing link between continuous and discrete AI.
Much in the spirit of gvgai, we have to do the following loop:
- create an initial game that a human can solve
- find an AI that beats it well
- study the AI, and add a new mechanic that breaks the AI, but does not break a human!
The question then becomes: do we have enough computational power to simulation a game worlds that is analogous enough to the real world, so that our AI algorithms will also apply to the real world?
To reduce computation requirements, it is better to focus on a 2D world at first. Such world with the right mechanics can break any AI, while still being faster to simulate than a 3D world.
The initial prototype uses the Urho3D open source game engine, and that is a reasonable project, but a raw Simple DirectMedia Layer + Box2D + OpenGL solution from scratch would be faster to develop for this use case, since Urho3D has a lot of human-gaming features that are not needed, and because 2019 Urho3D lead developers disagree with the China censored keyword attack.
Simulations such as these can be viewed as a form of synthetic data generation procedure, where the goal is to use computer worlds to reduce the costs of experiments and to improve reproducibility.
Ciro has always had a feeling that AI research in the 2020's is too unambitious. How many teams are actually aiming for AGI? When he then read Superintelligence by Nick Bostrom (2014) it said the same. AGI research has become a taboo in the early 21st century.
Related projects:
- github.com/deepmind/lab2d: 2D gridworld games, C++ with Lua bindings
Related ideas:
- www.youtube.com/watch?v=MHFrhIAj0ME?t=4183 Can't get you out of my head by Adam Curtis (2021) Part 1: Bloodshed on Wolf Mountain :)
- www.youtube.com/watch?v=EUjc1WuyPT8 AI alignment: Why It's Hard, and Where to Start by Eliezer Yudkowsky (2016)
Bibliograpy:
- agents.inf.ed.ac.uk/blog/multiagent-learning-environments/ Multi-Agent Learning Environments (2021) by Lukas Schäfer from the Autonomous agents research group of the University of Edinburgh. One of their games actually uses apples as visual represntation of rewards, exactly like Ciro's game. So funny. They also have a 2d continuous game: agents.inf.ed.ac.uk/blog/multiagent-learning-environments/#mpe
- humanoid robot simulation
- Section "AI training game"
- Section "Software-based artificial life"
DeepMind Has A Superhuman Level Quake 3 AI Team by Two Minute Papers (2018)
Source. Commentary of DeepMind's 2019 Capture the Flag paper. DeepMind does some similar simulations to what Ciro wants, but TODO do they publish source code for all of them? If not Ciro calls bullshit on non-reproducible research. Does this repo contain everything?OpenAI Plays Hide and Seek... and Breaks The Game! by Two Minute Papers (2019)
Source. Commentary of OpenAi's 2019 hide and seek paper. OpenAI does some similar simulations to what Ciro wants, but TODO do they publish source code for all of them? If not Ciro calls bullshit on non-reproducible research, and even worse due to the fake "Open" in the name. Does this repo contain everything?- From Motor Control to Team Play in Simulated Humanoid Football
From Motor Control to Team Play in Simulated Humanoid Football by Ali Eslami (2023)
Source. Likely a reupload by DeepMind employee: www.linkedin.com/in/smalieslami.DeepMind’s AI Trained For 5 Years by Two Minute Papers (2023)
Source. The 5 years bullshit is of course in-game time clickbait, they simulate 1000x faster than realtime.One of Ciro Santilli's strongest feeling in education is that material often falls in either of the two categories:
- hundreds of too basic popular science, e.g.:
- a 5 minute popular science video trying to explain quantum electrodynamics (an advanced subject) for someone who doesn't know what a Riemann integral is (a basic subject)
- a few full university courses that takes 20 hours to deliver the first punchline of the course
Ciro believes that there is often an important missing link between them, e.g.:
- a 15 minute video that delivers the main end results and motivations for people who already know the very basic stuff
If we as a society are unable to provide this sweet Middle Way sweet-spot, it is unreasonable to expect that learners will ever have the motivation to advance, because it is just too boring! They are just more likely to go play video games instead.
It is Ciro's hope that OurBigBook.com will help to fill exactly that gap.
In Ciro's view, as of the 2020's this critical gap generally lies somewhere between the end of undergraduate studies, and at the start of postgraduate studies.
What we have to do is make this knowledge more accessible all way down to high school and earlier.
Let's take the gloves off more often, and give the full thing to interested students! Let students learn what they want to learn, and do that as soon as possible! Life is too short!
This problem is basically the knowledge version of the last mile problem. When we reach the end of graduate, there are enough directions of knowledge to go off into, that the probability that a great free tutorial exists is relatively low. Of course, as one approaches the realm of novel research, the branching is so wide that having perfect tutorials becomes impossible. Ciro's goal in life go push the last mile marker a bit further out.
Related:
- universityphysicstutorials.com/ by Adam Beatty mentions:
There are myriad resources for physics and maths. The Kahn Academy and Patrick JMT were the best for me. They really helped me out. The question is, what resources are there for the advanced undergraduate courses?
Being valued as a lab technician interview with Norman Greenwood by Web of Stories (2017)
Source. In this video, the noted chemist mentions how he managed to get into a chemistry research development before he even joined university, due to a somewhat exceptional situation. Section "The only reason for universities to exist should be the laboratories" also comes to mind. This is exactly the type of thing that Ciro Santilli wants to make much more widespread.
Stories of Ainan Celeste Cawley fighting to advance his kids education beyond school, and being forbidden to do so by a stupid educational system, also come to mind.
Bibliography: