von Neumann’s Nightmare

Via the always excellent Stephen Hsu – apparently the concept of Technological Singularity can be traced back to von Neumann’s nightmare:
One night in early 1945, just back from Los Alamos, von Neumann woke in a state of alarm in the middle of the night and told his wife Klari:

“… we are creating … a monster whose influence is going to change history … this is only the beginning! The energy source which is now being made available will make scientists the most hated and most wanted citizens in any country.

The world could be conquered, but this nation of puritans will not grab its chance; we will be able to go into space way beyond the moon if only people could keep pace with what they create …”

He then predicted the future indispensable role of automation, becoming so agitated that he had to be put to sleep by a strong drink and sleeping pills. Source: Von Neumann, Morgenstern, and the Creation of Game Theory: From Chess to Social Science, 1900-1960.

In his obituary for John von Neumann, Ulam recalled a conversation with von Neumann:

[about the] “ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”

It is only fitting that the (co-)father of both game theory and computation also “discovered” their common endpoint.

As you recall the concept of intelligence explosion is attributed to I. J. Good and the term technological singularity to Vernor Vinge.

Biological boot loaders

Elon Musk a few months back summarized Nick Bostrom’s book Superintelligence: Paths, Dangers, Strategies:

While I find this issue indeed existentially important, I was a bit disappointed by the book because Bostrom turned out to be himself a biological boot loader – this time for Yudkowsky’s and Hanson’s memeplexes.

While these are by far not the worst memeplexes to be infected by, still in this important problem space one would like as many independent high-caliber search paths as possible.