Robert L. Blum, MD, PhD

 

Home

Photos

Sphere of Interest

WebBrain: AI
neurosci psych

Stanford Brain Lecture Notes

The RX Project:
Robotic Discovery

CV Biblio (1985)

Contact


Index of Essays

Psychology &
Neuroscience brain-icon

Computer Science,
Robotics, and AI
pr2-icon

Health & Biotech
insulin-icon

Earth Wisdom: Biosphere and Universe earth-icon

Be Saved by Bob!!!
(And Other Balms )
blowfish


Optimal Nutrition:
Are Fats Killers
or Saviors?

The Mystery of CONSCIOUSNESS

Consciousness Video:
Who, What, When?

Near Death Experiences: In the Desert With Pim Van Lommel

Is the UNIVERSE
Fine-Tuned for Life?

EUV 2013 - Future of Moore's Law

BAM: Brain Activity Map of Spikes

Beating Jeopardy!
What is Watson?
AI Overlord or Tool?

SETI: Search for Extraterrestrial Intelligence

KEPLER Seeks Earth-like Worlds

STEVE PINKER in the Amazon: photos

Billion Year Plan:
AI Formulation

AI Awakens

CONSCIOUSNESS as Global Resonance

SEAN's Accident

Coronary Artery CT
Scan: A Life Saver

Book Review: TRANSCEND

Book Review:
Create a Mind

Does Drug X
REALLY WORK?

TRANSCEND
DRUGS!!!

Total Recall:
Everything, Always

Ralph Triumphs:
Elbot Cheers

Scientists &
Evangelicals Unite

Thomas Berry,
Geologian: Obituary

Calorie Restriction
Works in Monkeys!

TheBrain &
WebBrain: Review

 

Beating Jeopardy!

What is WATSON?

 

           

Jeopardy Champs

AI Overlord or Tool?

 

In February 2011, when IBM's Watson defeated all time Jeopardy champions

 Ken Jennings and Brad Rutter, it joined a pantheon of iconic computer projects

 that have been milestones for AI.  Here is a partial list.

 

1956     Arthur Samuel's Checkers Player    

 

1971     Terry Winograd's SHRDLU: Language Understanding in Blocks World

 

1974     Harry Pople's and Jack Myer's INTERNIST, an internal medicine diagnostician

 

1975     Ted Shortliffe's MYCIN, a high performing, rule-based infectious disease expert system

 

1997     IBM's Deep Blue, a chess player that beat world champion Garry Kasparov

 

2000     ASIMO, Honda's deservedly famous android

 

2007     Stanford CSD’s Stanley, winner of the DARPA Grand Challenge

robotic car race in the Mojave Desert

 

            This victory by IBM's research team led by computer scientist David Ferrucci

is an important milestone for AI. Beating the best human players at Jeopardy

was a difficult engineering challenge that required harnessing several areas of AI:

natural language processing (NLP), knowledge representation and reasoning,

and parallel processing. Being a world champion at Jeopardy requires great accuracy,

a reliable estimate of confidence, and incredible speed.

 

            The best high level article on Watson was written by Ferrucci's IBM team,

and appeared in AI Magazine in 2010.  They describe the evolution of their

 DeepQA architecture from earlier QA systems including IBM's Piquant and NIST's Aquaint.
I also recommend KurzweilAI's Amara Angelica's interview with IBM's Eric Brown.

 

            The best tv show on Watson was this PBS NOVA, Smartest Machine on Earth.
These brief videos about Watson on IBM's website are also excellent.
And, if you missed the Jeopardy shows, you'll want to see this clip of the champs
in a preliminary bout.

 

            Here's what everybody wants: a system that (unlike a search engine)

gives you exactly the right answer as rapidly and accurately as possible.

How many cells are there in the human body?  What causes cancer?

Why is the rate of autism increasing? How would Sotomayor vote

if Roe v Wade was challenged?  Why does it get colder when you go up a mountain?

What is the current status of string theory?  Why did multicellularity take so long to evolve?

 

            You can see why the problem is so difficult.  The system must understand

the question, decide what kind of answer would be appropriate, search the internet,

and then display an accurate answer with the right supporting evidence.

This is the holy grail that IBM was trying to tackle with DeepQA, and Jeopardy was

an excellent motivating task.

 

   Watson Server Racks       

           

Below, I distill some key points on Watson, DeepQA, and the Jeopardy Challenge.

 

            Ambiguity in Jeopardy Questions: Consider this Jeopardy question:

 

                        Category: LINCOLN BLOGS

 

                        Clue:  Secretary Chase just submitted this to me for the third time;

                                    guess what, pal?  This time I'm accepting it.

 

                        Answer: his resignation.

 

            This Jeopardy question is a marvel of ambiguity.  The indefinite pronoun "this"

in "submitted this" is all that refers to the answer. Lots of things can be submitted:

documents, reports, letters - and who is Secretary Chase?  What does pal have to

do with it or blogs?   Who is the I in "I'm accepting it," and what is being accepted?
Even if Watson misidentifies the goal of an inquiry, it can still recover
by keeping track of alternative hyptheses.

 

            Memory and World of Discourse: Watson was not connected to the internet.

Everything it used in the contest was stored in memory. Your home computer has

at most 4 billion bytes (4 GB) of main memory (DRAM).  Watson used 15 trillion bytes

(15 TB) of DRAM shared by its 2880 cores.  It used 1 TB of DRAM to store

 the equivalent of about 1 million 200 page books including encyclopedias,

thesauri, dictionaries, movie databases, the Bible, and many other reference works.

 (The old estimate that human experts know about 50,000 facts is ludicrously low.

Ray Kurzweil’s estimate of 1.25 terabytes may be closer but is just a rough guess.)

 

            Level of Expertise in Jeopardy Champions: The best human players,

like contestants Ken Jennings and Brad Rutter, are stunningly accurate, consistent, and fast.

They answer perhaps 70% of the questions, and when they answer, they're right often

90% of the time.  Furthermore, they're blindingly fast, frequently buzzing in within 3 seconds
after the clue appears visually and within a few milliseconds after Alex Trebek reads it aloud.
(Test your own reaction times here or here.)

 

            Multiple Constraint Satisfaction:  The heart of Watson and DeepQA

is being able to score and rank-order  thousands of possible answers (candidates

or hypotheses) rapidly and accurately.  Doing this well is a tradeoff between

speed and accuracy.  The IBM team uses a multi-tiered approach, first using

quick and dirty methods to generate hypotheses, and then using increasingly

compute-intensive methods to hone the list.

 

            Consider the constraints that a final answer might have to satisfy.

It must be correct in person, place, time in history and year, accord with all the facts,

and accord with the logical structure of the question (ie its grammar and semantics).

It may even need to satisfy special constraints like word rhyming (eg soccer locker

 for the question “where Pele keeps his ball”) or spatial constraints (eg north of Beijing).

 

            Multiple constraint satisfaction is a key part of human knowledge acquisition.

We know when something’s not right. My favorite metaphor for this is fitting

a jig-saw puzzle piece: it needs to fit on all sides.

 

DeepQA Architecture

 

            Reliability of Evidence and Confidence:  Host Alex Trebek laughed when

Watson made a small but precise bet of $947 when it was winning $30,000.

As a rational game player, Watson had optimized the expected value of its bet.

How confident was it of its answer?  How much would it lose, if it was wrong?

 

            Watson uses multiple weighting schemes to assess its confidence.

Bayesian probabilities are popular in AI, but DeepQA uses a variety of weighting

schemes including Bayes to assess the accuracy of its answers. It must account for

source unreliability and partial satisfaction of multiple constraints.

             

            Hardware and Parallelism:  the team was using a single computer up to 2008.

Answering a question took 2 hours.  It now takes about 3 seconds.  Here's the math:

90 servers with a total of 3200 cores (single processors working in parallel).

 2 hours = 3600 seconds per hour * 2 = 7200 seconds.  Divide by 3200 and

 you get 2.25 seconds.

 

Watson's ninety Power 750 servers give it a throughput of 80 teraflops

 - hefty but not outrageous considering that 20 petaflops supercomputers (250X faster)
 are under construction at Lawrence Livermore National Lab to come online in 2012.

 

            Parallelism really helps.  That's the brain's great strength.

The human brain has over one million kilometers of wiring. That wiring

 (myelin insulation around axons = white matter) is most of the 3 pound weight of the brain.

The corpus callosum (our big interhemispheric cable) alone contains two hundred million fibers.

Your hundred billion neurons use only 20 watts of electricity and are liquid cooled.

Compare that to the giant AC units required in the Watson machine room.

 

            Common Sense:  One of the longest running and best-funded AI projects

of all time is CYC, an attempt by AI researcher Doug Lenat and his Cycorp team

to capture all of common sense on a computer.  Lenat estimates that CYC

now contains 6 million facts. A fact might be that living things require water

or that humans are mammals.

 

            When Lenat was asked about Watson versus CYC, he said (I paraphrase) suppose

 there was a Jeopardy category "goes uphill or downhill" and the clue was "spilled beer."

 Watson would utterly fail on this question.  It would have no basis on which to answer.

 In contrast CYC would know that "beer is a fluid,"  "fluids are liquids,"
"liquids usually flow downhill because of gravity."

 

            The IBM team hand-encoded a lot of common sense knowledge into Watson,

but that part of its database is tiny compared to what  CYC has.  (CYC has other crucial faults,

but do see Doug Lenat's defense of CYC in this video at Google HQ.)

 

            Turing Test:  The Turing Test is the most famous test for human intelligence in AI lore.

It just means having a computer that fools people into thinking they're talking to another person.

The Loebner Prize is awarded annually to the world's best efforts; however,

current annual winners, though amusing, are not yet close to passing the Turing Test,
which, I'm guessing, will happen within the next decade or two.
(Do try out ELBOT, one of my favorite chatbots, to a get a feel for the state of the art.)

Interestingly, Watson is too smart! If you were conversing with it, you might know
that you were talking with a superhuman. To pass the Turing Test, it would
need to be dumbed down.


Elbot

 

            Consciousness:  Even if an extended version of Watson passed the Turing Test,

it would NOT be conscious. Consciousness and intelligence are quite distinct.

In multicellular organisms emotion and consciousness arose  hundreds of

millions of years before human intelligence.  Computers have been highly

intelligent for decades back to Art Samuel's checkers playing programs in the 1950's.

They have never been conscious.

 

            Consciousness means perceiving the world around us in a highly detailed
(read gigabit per second), integrated fashion
.  See that world around you? It's computed
by your brain inside your skull and made to appear, virtual reality style, as the "real world."
It's such a flawless illusion that most people don't even get it after it's pointed out.

Mechanisms of consciousness are just now being uncovered by neuroscientists:
diffuse oscillatory resonances in multiple cortical lobes.
It uses highly recurrent networks of precisely timed spiking neurons with tight integration of local memory. The common Von Neumann computer architecture is grossly less efficient than the brain's billion-fold parallelism.

 

            On the other hand, robotics and computer vision researchers are beginning to
make progress by emulating the early stages of the human visual system (Serre et al.).

The robotic cars in the DARPA Grand Challenge and Microsoft's new Kinect
gaming system are great early applications of computer vision. I enthusiastically
follow their efforts, but they are the first faltering steps in a long journey toward
the coalescence of perception, imagination, and intelligence present in cerebration .

 

            Humans went from living in the African savannah 200,000 years ago

 to walking on the Moon.  AI researchers thought that animal sensory perception was

 the easy part and human intelligence was the hard part.  They had it backwards.

Capturing the perceptual ability of a bird or a mammal has proven to be the difficult part. 

After all, that part took evolution hundreds of millions of years to accomplish.

 

            An Intelligence Explosion:  Watson's success will trigger academic

 and corporate competition that will result in a spate of intelligent systems.

These QA systems will reside in "the cloud" in big server farms at IBM, Google,

Microsoft, Apple, Oracle, and others.  However, they will be available to all via the internet.
 (IBM discusses their plans for DeepQA healthcare systems, legal and governmental systems,
and scientific applications here
.)

 

 The future is bright (for the machines).  But take heart, fellow humans!

Between our ears, we’ve got the equivalent of a giant server farm that consumes
only 20 watts and can be powered by only coffee and donuts.

 

Copyleft 2011 BobBlum.com