(2015 Addendum is at bottom of page.)
In 2012 I posted the lead Amazon book review of Marshall Brain's new work,
Manna: Two Visions of Humanity's Future. Its theme - humanity's future - is one of my central concerns.
He and I agree that supervision by AIs is probable. The nature of that supervision (malevolent or benign) will reflect our evolving social structure in exactly the same way that our legal codes, morality, and statutes reflect who we are as a society.
Let me make that crystal clear — the AIs developed by North Korea will be of a fundamentally less benign nature than those developed by Google.The rise of AIs is inevitable - my hope is that they arise from and are molded by a benevolent human civilization. Whether their nature always continues to reflect that benevolence is anyone's guess.
Abou Ben Adhem (May His Tribe Increase!) - the rest of that lovely poem appears at the bottom of this page — a poem I learned from my father.
A few days ago (March 3, 2012) I heard Marshall Brain give a riveting talk to the Stanford Transhumanist Association. What will be the fate of humanity in a future world populated by supersmart AIs and robots. His message — basically, we're toast.
According to Brain, the future will bring increasing unemployment as broad swaths of humanity are replaced by robots. As a consequence, wealth will continue to concentrate in the hands of a privileged few, with the great majority (the 99%) being forced into grinding poverty. (You've heard this before, from, say, Karl Marx - but it gets even more bleak.)
As the AIs become ever more powerful they may come to regard us first as chimpanzees, then as chickens, and finally as bacteria. How will the AIs deal with us? Possibilities include extermination, zoos, prisons, and tailor-made virtual reality utopias (Heavenly or Eden-like versions of The Matrix.)
Scary stuff, but is it true? Yeah, it might happen.
In a brief chat afterward, I told him that the best I could see for humanity in a post-Singularity world was planet Earth as a retirement home for humanity — watched over by "machines of loving grace." Perhaps not the exuberant vision that you're used to, but not that different from real life. Now, you get old and die, but you get to see an improved, next generation carry the torch forward. Post-Singularity, it's just that the machines are carrying the torch (directing planetary affairs, doing the real innovation, and going to the stars.)
Ok, now back to the book. Having been primed for a nonstop trip to Hell, the vision portrayed in the book was actually a relief. Although the images of minimum wage employment enforced by a silicon Big Brother are bleak to my Western eyes, as expertly portrayed by Marshall, they might be a welcome relief to many in Asian sweatshops (think of the rash of suicides at Foxconn.)
Toward the end of this novella, he transitions to a far more optimistic, alternative future with greatly expanded freedom and opportunities. That alternative society (also involving tight integration with and supervision by Ais) is described in fascinating detail.
The work is a detailed dystopian extrapolation in the tradition of Orwell's 1984 and Huxley's Brave New World. The technical details are as expertly and believably detailed as one would expect from the inventor of HowStuffWorks. Brain is a social critic to be taken seriously, and this is an engaging work.
While I enjoyed reading a review copy on my Kindle, the work is also available for free on the internet. He wants the work to be widely read, and it should be.
Abou Ben Adhem by James Henry Leigh Hunt
Abou Ben Adhem (may his tribe increase!) Awoke one night from a deep dream of peace, And saw, within the moonlight in his room, Making it rich, and like a lily in bloom, An angel writing in a book of gold:— Exceeding peace had made Ben Adhem bold, And to the Presence in the room he said "What writest thou?"—The vision raised its head And with a look made of all sweet accord, Answered "The names of those who love the Lord." "And is mine one?" said Abou. "Nay, not so," Replied the angel. Abou spoke more low, But cheerly still, and said "I pray thee, then, Write me as one that loves his fellow men."
The angel wrote, and vanished. The next night It came again with a great wakening light, And showed the names whom love of God had blessed, And lo! Ben Adhem's name led all the rest.
(Addendum (April 2015):
There is a view, gaining traction in some circles, that superintelligent AI is so imminent that it dwarfs other more prosaic existential threats. I regard this view as erroneous (super-human AI is NOT imminent). Furthermore, this sensational forecast may serve to further weaken already anemic efforts to lessen real existential dangers: nuclear war, ecological calamity, and uncontrolled synthetic biology.
There is a danger from stupid AI — for example, the maniacal chess program described in a cautionary paper by Steve Omohundro.