The key takeaway is that setting an explicit value function to an AGI entity is a good way to destroy the world due to poor AI alignment. We are more likely to not destroy by creating an AI whose goals is to "do want humans what it to do", but in a way that it does not know before hand what it is that humans want, and it has to learn from them. This approach appears to be known as reward modeling.
Some other cool ideas:
- a big thing that is missing for AGI in the 2010's is some kind of more hierarchical representation of the continuous input data of the world, e.g.:
- intelligence is hierarchical
- we can group continuous things into higher objects, e.g. all these pixels I'm seeing in front of me are a computer. So I treat all of them as a single object in my mind.
- game theory can be seen as part of artificial intelligence that deals with scenarios where multiple intelligent agents are involved
- probability plays a crucial role in our everyday living, even though we don't think too much about it every explicitly. He gives a very good example of the cost/risk tradeoffs of planning to the airport to catch a plane. E.g.:
- should you leave 2 days in advance to be sure you'll get there?
- should you pay an armed escort to make sure you are not attacked in the way?
- economy, and notably the study of the utility, is intrinsically linked to AI alignment
All pages below are from the second edition from 2018. It seems that there weren't any changes in the text, the updated preface mentions
As it happens, nearly 15 years have passed since the 1st edition of Power, Sex, Suicide was published, and I am resisting the temptation to make any lame revisions. Some say that even Darwin lessened the power of his arguments in the Origin of Species through his multiple revisions, in which he dealt with criticisms and sometimes shifted his views in the wrong direction. I prefer my original to speak for itself, even if it turns out to be wrong.
This is partly addressed in the preface of the second edition from 2018.
Central thesis:
- there are two sexes because of mitochondria
- the acquisition of mitochondria was one of the most important steps in the evolution of eukaryotes.There are no known eukaryotes which never had mitochondria. Having mitochondria appears to be a requisite for being an eukaryote.Contrast this for example with multicellularity, which is highly polyphyletic.
- Apoptosis is largely regulated by mitochondria
- there are two main theories for how the mitochondria endosymbiosis started:
- parsitic hypothesis of mitochondrial endosymbiosis: a parasitic option rather than cooperative
- hydrogen hypothesis: a cooperative option rather than parasitic
Smaller points:
- 10% of our body weight (dry presumably?) is mitochondria. Also quoted at: www.nature.com/scitable/blog/student-voices/mighty_mitochondria. TODO confirm.
- eukaryotes can do phatocytosis due to their cytoskeleton
- paints a colorful picture of Peter Mitchell. Some Wikipedia edits are warranted!
- it is hard for complex organisms to evolve because longer DNA means longer replication time
- cancer is natural selection gone wrong
- multicellular organisms are not utopias where every cell lives happily. Rather, they are dictatorships, where any dissident is forced to commit seppuku. Lu Xun's petition quote comes to mind.
Nitpicks:
- the book calls ATP synthase "ATPase" in several points, which is confusing because -ase means "something that breaks", and in 2020 parlance, there are ATPases which actually break ATP: en.wikipedia.org/wiki/ATPase. The book itself acknowledges that on page 135:
The ATPase is freely reversible. Under some circumstances it can go into reverse, whereupon it splits ATP, and uses the energy released to pump protons up the drive shaft, back across the membrane against the pressure of the reservoir. In fact the very name ATPase (rather than ATP synthase) signifies this action, which was discovered first. This bizarre trait hides a deep secret of life, and we’ll return to it in a moment.
Some criticisms:
- some of the later chapters are a bit more boring, like the stuff about warm-blooded animals. Perhaps is it that Ciro Santilli is more interested in the molecular aspects than macro
- the author talks about some very recent research at the time. While this does highlight his expertise, some of the points mentioned might still be in a state of flow. This is acknowledged by the author himself on the 2018 updated preface however.
Robert Noyce: The Man Behind the Microchip by Leslie Berlin (2006) Updated 2024-12-15 +Created 1970-01-01
Borrow from the Internet Archive for free: archive.org/details/manbehindmicroc000berl/page/n445/mode/2up
The Supermen: The Story of Seymour Cray by Charles J. Murray (1997) Updated 2024-12-15 +Created 1970-01-01
Borrow from the Internet Archive for free: archive.org/details/supermenstory00murr
Initial chapters put good clarity on the formation of the military-industrial complex. Being backed by the military, especially just after World War II, was in itself enough credibility to start and foster a company.
It is funny to see how the first computers were very artisanal, made on a one-off basis.
Amazing how Control Data Corporation raised capital IPO style as a startup without a product. The dude was selling shares at dinner parties in his home.
Very interesting mention on page 70 of how Israel bought CDC's UNIVAC 1103 which Cray contributed greatly to design, and everyone knew that it was to make thermonuclear weapons, since that was what the big American labs like this mention should be added to: en.wikipedia.org/wiki/Nuclear_weapons_and_Israel but that's Extended Protected... the horrors of Wikipedia.
Another interesting insight is how "unintegrated" computers were back then. They were literally building computers out of individual vacuum tubes, then individual semiconducting transistors, a gate at a time. Then things got more and more integrated as time went. That is why the now outdated word "microprocessor" existed. When processors start to fit into a single integrated circuit, they were truly micro compared to the monstrosities that existed previously.
Also, because integration was so weak initially, it was important to more manually consider the length of wire signals had to travel, and try to put components closer together to reduce the critical path to be able to increase clock speeds. These constraints are also of course present in modern computer design, but they were just so much more visible in those days.
The book does unfortunately not give much detail in Crays personal life as mentioned on this book review: www.goodreads.com/review/show/1277733185?book_show_action=true. His childhood section is brief, and his wedding is described in one paragraph, and divorce in one sentence. Part of this is because he was very private about his family most likely note how Wikipedia had missed his first wedding, and likely misattribute children to the second wedding; en.wikipedia.org/wiki/Talk:Seymour_Cray section "Weddings and Children".
Crays work philosophy is is highlighted many times in the book, and it is something worthy to have in mind:
- if a design is not working, start from scratch
- don't be the very first pioneer of a technology, let others work out the problems for you first, and then come second and win
Cray's final downfall was when he opted to try to use a promising but hard to work with material gallium arsenide instead of silicon as his way to try and speed up computers, see also: gallium arsenide vs silicon. Also, he went against the extremely current of the late 80's early 90's pointing rather towards using massively parallel systems based on silicon off-the-shelf Intel processors, a current that had DARPA support, and which by far the path that won very dramatically as of 2020, see: Intel supercomputer market share.
Talks about rebellion of the oppressed (and bandits), and therefore has been controversial throughout the many Chinese dictatorships.
The book is based on real events surrounding 12th century rebel leader Song Jiang during the Song dynasty.
It is also interesting that Mao Zedong was apparently a fan of the novel, although he had to hide that to some extent due to the controversial nature of the material, which could be said to instigate rebellion.
The incredible popularity of the novel can also be seen by the large number of paintings of it found in the Summer Palace.
This is a good novel. It appeals to Ciro Santilli's sensibilities of rebelling against unfairness, and in particular about people who are at the margin of society (at the river margin) doing so. Tax the rich BTW.
It also has always made Ciro quite curious how such novels are not used as a way to inspire people to rebel against the Chinese Communist Party.
Full text uploads of Chinese versions:
- www.gutenberg.org/cache/epub/23863/pg23863.html No table of contents.