Ciro Santilli has felt that perhaps what is missing in 2020's AGI research is:

the interface between:
- the continuous/noisy level (now well developed under artificial neural network techniques of the 2010's)
- and symbolic AI level AI
The key question is somewhat how to extract symbols out of the space-time continuous experiences.
more specialized accelerators that somehow interface with more generic artificial neural networks. Notably some kind of speialized processing of spacial elements is obviously hardcoded into the brain, see e.g. Section "Grid cell (2005)"

Forcing these boundaries to be tested was one of the main design goals of Ciro's 2D reinforcement learning games.

In those games, for example:

when you press a button here, a door opens somewhere far away
when you touch certain types of objects, a chemical reaction may happen, but not other types of objects

Therefore, those continuous objects would also have "magic" effects that could not be explained by "simple" "what is touching what" ideas.

Bibliography:

mitpress.mit.edu/9780262632683/the-algebraic-mind/

Intelligence is hierarchical

0 0

This point is beautifully argued in lots of different sources, and is clearly a pillar of AGI.

Perhaps one may argue that our deep learning layers do form some kind of hierarchy, e.g. this is very clear in certain models such as convolutional neural network. But many of those models cannot have arbitrarily deep hierarchies, which appears to be a fundamental aspect of intelligence.

How to Create a Mind:

The lists of steps in my mind are organized in hierarchies. I follow a routine procedure before going to sleep. The first step is to brush my teeth. But this action is in turn broken into a smaller series of steps, the first of which is to put toothpaste on the toothbrush. That step in turn is made up of yet smaller steps, such as finding the toothpaste, removing the cap, and so on. The step of finding the toothpaste also has steps, the first of which is to open the bathroom cabinet. That step in turn requires steps, the first of which is to grab the outside of the cabinet door. This nesting actually continues down to a very fine grain of movements, so that there are literally thousands of little actions constituting my nighttime routine. Although I may have difficulty remembering details of a walk I took just a few hours ago, I have no difficulty recalling all of these many steps in preparing for bed - so much so that I am able to think about other things while I go through these procedures. It is important to point out that this list is not stored as one long list of thousands of steps - rather, each of our routine procedures is remembered as an elaborate hierarchy of nested activities.

Human Compatible: TODO get exact quote. It was something along: life goal: save world from hunger. Subgoal: apply for some grant. Sub-sub-goal: eat, sleep, take shower. Sub-sub-sub-goal: move muscles to get me to table and open a can.