Notes on Hotz vs. Yudkowski AI Safety Debate
You can watch the debate here.
AI take-off speed
George agrees that AI alignment is an unsolved and hard problem. However, he is much more optimistic about our ability to solve it in time, and one of the main favour of this view is that there won't be a fast take-off ("foom"), which could happen, e.g. if the AI starts to rapidly self-improve. For example, he sees AIs being better at programming soon, but not without bugs altogether. Similarly, he agrees that an AGI is conceivable if the trends continue, but we are far from superintelligence — or it may be impossible altogether. Eliezer is not convinced that the fast take-off is necessary for it to pose a problem.
Will super-intelligent AIs hate humans?
George also doesn't see why AIs would just want to kill humans. He goes on to quote Eliezer's line — "AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else" — but argues that there are atoms in the universe which are much more easily accessible for which superintelligent AIs will surely come first. Moreover, humans will defend themselves.
A related question is whether AIs will have goals and why? Eliezer's response is that goals (or drives) are useful to achieve things.
AI vs humans conflict
A future that George foresees is one with a conflict between humans enhanced with AIs fighting against other AIs, which may be trying to take control. There has been conflict throughout the whole history of humankind, and it is also how nature works. It will not be different this time, and we will prevail. Eliezer points out we have never been in conflict with entities more intelligent than us and that the fate of some animals is an obvious example of how such conflicts usually go for the less intelligent side.
Will we control AI, or will the AI control us?
Eliezer offers a metaphor which addresses the question of control. The more intelligent one controls the other one, and it is the “centre of gravity”. The AIs so far are not more intelligent than us, therefore we can control them. They are moons orbiting around the sun (us). However, this relationship may change in the future.
Note that this requires more general intelligence — a chess-playing engine is more intelligent than us in the narrow domain of playing chess but obviously can’t control us.
Closing summaries
In the end, both guests briefly summarized their positions and I paraphrase them here.
Eliezer: The argument is simple: very smart thing(s) that do not want humanity around eventually ends with humans gone.
George: Time matters. The situation is very different if this is a concern in 5, 50, or 500 years. AIs won't love us but won't consume our atoms either. Machines and humans (with machines) will fight with each other, but humankind will survive.