Kind: captions
Language: en
the following is a conversation with
Doug lennet creator of Psych A system
that for close to 40 years and still
today has sought to solve the core
problem of artificial intelligence the
acquisition of Common Sense knowledge
and the use of that knowledge to think
to reason and to understand the world to
support this podcast please check out
our sponsors in the description as a
side note let me say that in the
excitement of the modern era of machine
learning it is easy to forget just how
little we understand exactly how to
build the kind of intelligence that
matches the power of the human mind to
me many of the core ideas behind Psych
in some form in actuality or in spirit
will likely be part of the AI system
that achieves General super intelligence
but perhaps more importantly solving
this problem of Common Sense knowledge
will help us humans understand our own
minds the nature of Truth and finally
how to be more rational and more kind to
each other this is the Lex Freedman
podcast and here is my conversation with
Doug
lennett Psych is a project launched by
you in
1984 and still is active today whose
goal is to assemble a knowledge base
that spans the basic concepts and rules
about how the world Works in other words
it hopes to capture Common Sense
knowledge which is a lot harder than it
sounds can you elaborate on this Mission
and maybe perhaps speak to the various
subg goals within this Mission when I
was a faculty member in the computer
science department at Stanford my
colleagues and I did research in all
sorts of artificial
intelligence programs so natural
language understanding programs robots
expert
systems and so on and we kept hitting
the very same brick wall
our systems would have impressive early
successes and so if your only goal was
academic namely to um get enough
material to write a journal article uh
that might actually suffice but if
you're really trying to get AI um then
you have to somehow get past the brick
wall and the brick wall was the programs
didn't have what we would call Common
Sense they didn't have General World
Knowledge they didn't really understand
what they were doing what they were
saying what they were being asked um and
so very much like a um um a clever dog
performing tricks we could get them to
do tricks but they never really
understood what they were doing sort of
like when you get a dog to fetch your
morning newspaper uh the dog might do
that successfully but the dog has no
idea what a newspaper is or what it says
or anything like that what does it mean
to understand something can you maybe
elaborate in that a little bit is it is
understand an action of like combining
little things together like through
inference or is understanding the wisdom
you gain over time that forms a
knowledge I think of understanding more
like a um uh think of it more like the
ground you stand on which um could be
very shaky could be very unsafe um but
most of the time is not because
underneath it is more ground um and
eventually you know rock and and other
things but um layer after layer after
layer that solid foundation is there and
you rarely need to think about it you
rarely need to count on it but
occasionally you do and um um I've never
used this analogy before so bear with me
but um I think the same thing is true in
in terms of getting computers to
understand things which is uh you ask a
computer a question for instance Alexa
or um some robot or something and um
maybe it gets the right answer um but if
you were asking that of a human you
could also say things like why or how
might you be wrong about this or
something like that and the person you
know would would answer you and you know
it might be a little Annoying if you
have a small child and they keep asking
why questions in series eventually you
get to the point where you throw up your
hands and say I don't know it's just the
way the world is um but for many layers
you actually have that that layered
solid foundation of support so that when
you need it you can count on it and when
do you need it Well when things are
unexpected when you come up against a
situation which is novel for instance
when you're driving um it may be fine to
have a small program a small set of
rules that cover you know 99% of the
cases but that 1% of the time when
something strange happens um you really
need to draw on Common Sense for
instance um my wife and I were driving
recently um and there was a trash truck
in front of us and um I guess they had
packed it too full and the back exploded
and trash bags went everywhere uh and we
had to you make a split-second decision
are we going to slam on our brakes are
we going to Swerve into another Lane um
are we going to just run it over um
because there were cars all around us um
and you know in front of us was a large
um trash bag and we know what we throw
away in trash bags probably not a safe
thing to run over um over on the the
left was um a bunch of fast food
restaurant um trash bags and it's like
oh well those things are just like
Styrofoam and leftover food we'll run
over that and so that was a safe thing
for us to to do now that's the kind of
thing that's going to happen maybe once
in your
life and um but the point is that
there's almost no telling what little
bits of knowledge about the world you
might actually need um in some
situations which were unforeseen but see
when you sit on that mountain or that
ground that goes deep of knowledge in
order to make a split-second decision
about uh fast food trash or random trash
from the back of a uh trash
truck uh you need to be able to leverage
that ground you stand on in some way
it's not not merely you know it's not
enough to just have a lot of ground to
stand on it's your ability to leverage
it to utilize in a split like integrate
it all together to make that Split
Second decision and I I I
suppose understanding isn't
just having uh Common Sense knowledge to
access it's the act of accessing
accessing it somehow like
correctly um f fing out the the parts of
the knowledge are not useful selecting
only the useful parts and uh effectively
making conclusive decisions so let's
tease apart two different tasks really
both of which are um incredibly
important and even necessary um if
you're going to have this in a usable a
useful usable fashion as opposed to say
like library books sitting on a shelf
right um and so on um where the
knowledge might be there but you know if
a fire comes the books are going to burn
because they don't know what's in them
and they're just going to um sit there
while they burn um so there two there
two aspects of using the knowledge one
is a kind of a theoretical um how is it
possible at all and then the second
aspect of what you said is how can you
do it quickly enough right um so how can
you do it at all is something that
philosophers have grappled with and
fortunately philosophers hundred years
ago and um even earlier um developed a
kind of um formal language um like
English um it's called um U predicate
logic or first order logic or um
something like predicate calculus and so
on so there's a way of representing
things in this formal language um which
enables a mechanical procedure to sort
of grind through and algorithmically
produce all of the same
logical entailments all the same logical
conclusions that you or I would from
that same set of pieces of information
that are represented that way um so um
that that sort of raises um a couple
questions one is how do you get all this
information from say observations and
English and so on into this logical form
right and secondly how can you then
efficiently um run these algorithms to
actually get the information you need in
in the case I mentioned in a tenth of a
second rather than say um in you know 10
hours or 10,000 years of computation um
and those are both really important um
uh questions and and like a corollary
addition to the first one is how many
such things do you need to gather for it
to be useful in certain contexts so like
what in order you mentioned philosophers
in order to capture this world and
present it in a logical way in a with a
formal logic like how many statements
are required is it five is it 10 is it
10 trillion is it like that that's as
far as I understand is uh probably still
an open question it may forever be an
open question uh to to say like
definitively about to describe the
universe perfectly how many facts do you
need I'm I I guess I'm going to
disappoint you by giving you an actual
answer to your question okay um well no
this sounds exciting yes okay so so now
we have like um three three things to to
talk about keep adding more although
that's okay the first and the third are
related yes um so let's leave the
efficiency question aside for now so um
how how does all this information get
represented in logical form so that
these algorithms um resolution theorem
proving and other algorithms can
actually grind through all the logical
consequences of what you said um and
that ties into your question about well
how many of these things do you need um
because if the answer is small enough um
then by hand you could write them out
one at a
time so um in um the um 19 early um
1984 um I held a meeting at Stanford
where I was a faculty member there um
where we assembled um about half a dozen
of the smartest people I know um people
like um Alan Newell and Marvin Minsky
and Alan K and um um a few others was
fman there by chance cuz he he liked
your he commented about your system Uris
at the time no no he he wasn't part of
this meeting um but that's a heck of a
meeting anyway I think Ed fenom was
there I think um um Josh leberg was
there so we we have um um all these
different um smart people and we were um
we came together to uh address the
question that that you raised which is
if it's important to represent Common
Sense knowledge and World Knowledge in
order for AIS to not be brittle in order
for AIS not to just have the veneer of
intelligence well how many pieces of
Common Sense how many if then
rules for instance would we have to
actually write in order to essentially
cover what what people expect perfect
strangers to already know about the
world and um I expected there would be
an enormous Divergence of opinion um and
computation but amazingly everyone got
an answer which was around a million um
and one person one person got the answer
um by saying well um look you can only
burn into human long-term memory a
certain number of things per unit time
like maybe one every 30 seconds or
something and other than that it's just
shortterm memory and it flows away like
water and so on so by the time you're
say 10 years old or so um how many
things could you possibly have burned
into your long-term memory and it's like
about a million um another person went
in a completely different direction and
said well if you look at the number of
words um in a um a dictionary not a
whole dictionary but for someone to um
essentially um be considered to be
fluent in a language how many words
would they need to know and then about
how many things about each word would
you have to tell it and so they got to a
million that way um another another
person said well let's actually look at
one single um short one volume desk
encyclopedia article and so we'll look
at you know what was like a um a four
paragraph article or something I think
about greed GBS GBS are a type of water
foul um and if we were going to sit
there and represent every single thing
that was there um how many assertions or
rules or statements would we have to
write in this logical language and so
and then multiply that by all of the
number of articles that there were and
so on so all of these estimates came out
with a million um and so if you do the
math it turns out that like oh well then
maybe in um something like
um 100 um person years um in one or two
person centuries we could actually get
this written down by hand and a
marvelous um coincidence opportunity
existed um right at that point in time
the early 1980s there was something
called the Japanese fifth generation
Computing effort Japan had threatened to
do in Computing and Ai and Hardware what
they had just finished doing in consumer
electronics and the automotive industry
namely resting control away from the
United States and more generally away
from the West um and so America was
scared um and um Congress did something
that's how you know it was a long time
ago because Congress did something
congress Congress passed something
called the national Cooperative research
act ncra and what it said was hey all
you big American companies that's also
how you know it was a long time ago
because they were American companies
rather than multinational companies hey
all you big American companies um
normally it would be an antitrust
violation if you colluded on R&D but we
promise for the next 10 years uh we
won't prosecute any of you if you do
that um to help combat this threat and
so overnight um the first two consortia
research consortia in America sprang up
um both of them coincidentally in Austin
Texas uh one called semitech focused ing
on Hardware chips and so on and then one
called MCC the microelectronics and
computer technology corporation focusing
on more on software on databases and Ai
and natural language understanding and
things like
that um and um I got the opportunity
thanks to my friend Woody bledo um who
um was one of the people um who founded
that to come and be its principal
scientist and he said you know um and he
sent U Admiral Bob Inman who was the
person running MCC uh came and talked to
me and said look Professor you know
you're talking about doing this project
it's going to involve um person
centuries of effort uh you've only got a
handful of graduate students you do the
math it's going to take you like um you
know uh longer than the rest of your
life to finish this project but if you
move to the Wilds of Austin Texas uh
we'll put 10 times as many people on it
and um you know you'll be done in um a
few years and so that was pretty
exciting and so so um I did that I um uh
took my leave from Stanford I came to to
Austin I worked for MCC and um good news
and bad news the bad news is that all of
us were off by an order of magnitude um
that it turns out what you need are tens
of millions of these um pieces of
knowledge about um every day sort of
like um if you have a coffee cup with
stuff in it and you turn it upside down
the stuff in it going to fall out um so
you need tens of millions of pieces of
knowledge like that even if you take
trouble to make each one as general as
it possibly could be um but um the good
news was that thanks to um uh initially
the fifth generation effort and then
later um US Government um agency funding
and so on we were able to get enough
funding not for um a couple person
centuries of time but for a couple
person Millennia of time which is what
we've spent since 1984
getting psych to contain the tens of
millions of rules that it needs in order
to really capture and span uh sort of
not all of human knowledge but the
things that you assume other people know
the things you count on other people
knowing um and uh so by now we've done
that um and uh the good news is since
you've waited 38 years just about to
talk to me uh we're we're about at the
um at the end of that process so most of
what we're doing now is not putting in
even what you would consider common
sense but more putting in um domain
specific application specific knowledge
about um uh Health Care um in a certain
hospital or about um um oil um pipes
getting um clogged up or whatever the
applications happen to be so we've
almost come full circle and we're doing
things very much like the expert systems
of the 1970s and 1980s except instead of
resting on nothing and being brittle
they're now resting on this massive
pyramid if you will this massive lattice
of Common Sense knowledge so that when
things go wrong when something
unexpected happens they can fall back on
more and more and more general
principles um eventually bottoming out
in things like um for instance if we
have a problem with the microphone one
of the things you'll do is unplug it
plug it in again and hope for the best
right because one of the general pieces
of knowledge you have in dealing with
electronic equipment or software systems
or um things like that is there a basic
principle like that like is there is it
possible to encode something that
generally captures this idea of turn it
off and turn it back on and see if it
fixes oh absolutely that's one of the
things that um that psych knows that's
actually one of the fundamental um laws
of nature I believe I wouldn't I
wouldn't call it a law it's it's more
like a um seems to work every time so
it's sure sure like looks like a law I
don't know so that um that basically um
um covered the um the resources needed
and then we had to devise a method to
actually figure out well what are the
tens of millions of things that we need
to tell the system and for that we found
um a few um techniques which worked
really well one is to take um any piece
of text almost could be an advertisement
it could be a transcri scpt it could be
a novel it could be an article um and
don't pay attention to the actual type
that's there the the black space on the
White Page pay attention to the
complement of that the white space if
you will so what did the writer of this
sentence assume that the reader already
knew about the world for instance if
they used a pronoun how did they figure
out that you why did they think that you
would be able to understand what the
intended referent of that pronoun was if
they used an ambiguous word how did they
think that you would be able to figure
out what they meant by that word um the
other thing we look at is the gap
between one sentence and the next one
what are all the things that the writer
expected you to fill in and infer
occurred between the end of one sentence
and the beginning of the other so like
if the sentence says um uh Fred Smith
robbed the third National Bank period um
uh he was sentenced to 20 years in
prison period well between the first
sentence and the second you're expected
to infer things like Fred got caught
Fred um got arrested Fred went to jail
Fred had um a trial Fred was found
guilty um and so on if my next sentence
starts out with something like the judge
dot dot dot um then you assume it's the
judge at his trial if my next sentence
starts out something like the arresting
officer dot dot dot you assume that it
was the police officer who arrested him
after he committed the crime and so on
so um that's those are two techniques
for getting um that knowledge the the
other thing we sometimes look at is uh
sort of like fake news or uh sort of
humorous um onion headlines or um
headlines in um the weekly world news if
you know what that is or the national
inquire where it's like oh we don't
believe this then we introspect on why
don't we believe it so there are things
like um uh B7 lands on the moon you know
it's like why don't we what do we know
about the world that causes us to
believe that that's just silly or uh
something like that or um another thing
we look for are contradictions um where
with things which can't both be true um
and we say to like what is it that we
know that causes us to know that both of
these can't be true at the same at the
same time for instance in one of the
weekly world news um um editions in one
article it talked about how Elvis was
cited um you know even though he was uh
you know getting on in years and so on
and another article in the same one
talked about people seeing Elvis's ghost
okay so it's like why why do we believe
that it least one of these articles you
know must be wrong um and so on so um so
we have a series of techniques like that
that enable our people um and by now uh
we have about 50 people working
full-time on this and have for for
decades so we've put in the thousands of
person years of effort we've built up
these tens of millions of rules we
constantly police the system to make
sure that we're saying things as
generally as we possibly can um so um
you don't want to say things like um no
mouse is also a moose uh because if you
said things like that um then you'd have
to add another one or two or three zeros
onto the number of assertions you'd
actually have to have so um at some
point we generalize things more and more
and we get to a point where we say oh
yeah for any two biological taxons if we
don't know explicitly that one is a
generalization of another then almost
certainly they're disjoint um a member
of One is not going to be a member of
the other and so on so and the same
thing with the Elvis and the ghost it
has nothing to do with Elvis it's more
about human nature and the mortality and
kind right in general things are not
both alive and dead at the same time
yeah and unless special cats in in
theoretical physics examples well that
raises um a couple important points
that's the onion headline situation type
of thing okay sorry but no no so so what
you bring up is this really important
point of like well how do you handle
exceptions and
inconsistencies and so on um and one of
the hardest lessons for us to learn it
took us about five years to to Really
grit our teeth and um learn to love it
um is we had to give up Global
consistency so the knowledge base can no
longer be consistent so this is a kind
of scary thought I grew up watching Star
Trek and anytime a computer was
inconsistent it would either freeze up
or explode or take over the world or
something bad would happen um or if you
come from a mathematics background uh
once you can prove false you can prove
anything so that's not good um and so on
so um that's why um the um old
knowledge-based systems were all very
very consistent but the trouble is that
um by and large our models of the world
the way we talk about the world and so
there are all sorts of inconsistencies
that creep in here and there um that
will sort of kill some attempt to build
some enormous globally consistent
knowledge base and so what we had to
move to was a system of local
consistency so a good analogy is you
know that the surface of the Earth is
more or less
spherical globally um but you live your
life every day as though the surface of
the Earth were flat you know when you're
talking to someone in Australia you
don't think of them as being oriented
upside down to you when you're planning
a trip you know even if it's a thousand
miles away um you may think a little bit
about time zones but you rarely think
about the curvature of the earth and so
on and for most purposes you can live
your whole life without really worrying
about that because the Earth is locally
flat in much the same way the psych
knowledge base is divided up into almost
like tectonic plates which are
individual contexts and each cont
context is more or less consistent but
there can be small inconsistencies at
the boundary between one context than
the next one and so on and so by the
time you move say 20 contexts over there
could be glaring inconsistencies So
eventually you get from the normal
modern real world context that we're in
right now to something you know like um
Road Runner cartoon context where
physics is very different and in fact
life and death are very different
because no matter how many times he's
killed you know the coyote comes back in
the next scene and and so on so um that
that was a hard lesson to learn and we
had to make sure that our representation
language the way we the way that we
actually encode the knowledge and
represent it was expressive enough that
we could talk about things being true in
one context and false in another things
that are true at one time and false in
another things that are true let's say
in one region like one country but false
in another another things that are true
in one person's belief system but false
in another person's belief system uh
things that are true at one level of
abstraction and false at another for
instance at one level of abstraction you
think of this table as a solid object
but at you know down at the atomic level
it's mostly empty space and so on so
then that's fascinating and but it puts
a lot of pressure on context to do a lot
of work so you say tectonic plates is it
possible to formulate contexts that are
General and big that do this kind of
capture of knowledge bases or do you
then get Turtles on top of turtles again
where there's just a huge number of
contexts so it's good you ask that
question because you're you're pointed
in the right direction which is you want
contexts to be first class objects in
your systems knowledge base in
particular in psyches knowledge base um
and by first class object I mean that it
should we should be able to have psych
think about and talk about and reason
about one context or another context the
same way it reasons about coffee cups
and tables and people and fishing and so
on um and so contexts are just terms in
its language just like the ones I
mentioned and so psych can reason about
um context context can arrange
hierarchically and so on um and so you
can say things about let's say things
that are true in the modern era things
that are true in a particular year would
then be a subcontext of the the things
that are true in um a broad let's say a
century or a millennium or something
like that things that are true in Austin
Texas um are generally going to be a
specialization of um things that are
true in Texas which is going to be a
specialization of things that are true
in the United States and so on um and so
you don't have to say things over and
over over again at all these levels you
just say things at the most General
level that it applies to and you only
have to say it once and then it
essentially inherits to all these more
specific um contexts to ask a slightly
technical question is this inheritance
inheritance a tree or a graph oh you
definitely have to think of it as a
graph um so we could talk about for
instance why the Japanese fifth
generation Computing effort failed there
were about half a dozen different
reasons one of the reasons they failed
was because they tried to represent
knowledge as a tree rather than as a
graph um and so each node in their
representation could only have one
parent node so if you had a table that
was a wooden object a black object a
flat object and so you have to choose
one and that's the only parent it could
have uh when of course you know
depending on what it is you need to
reason about it sometimes it's important
to know that it's made out of wood like
if we're talking about a fire sometimes
it's important to know that it's flat if
we're talking about resting something on
it and so on so um um one of the um one
of the problems was that they wanted a
kind of Dewy Decimal numbering system
for all of their Concepts which meant
that each node could only have at most
um 10 children and each node could only
have one parent um and uh while that
does enable The Dewy Decimal type um
numbering of Concepts labeling of
Concepts um it prevents you from
representing all the things you need to
about um objects in our in our world and
that was one of the things which um they
never were able to overcome and I think
that was one of the main reasons that
that project failed so we'll return to
some of the doors you've opened but if
we can go back to that room in 1984
around there with Marvin mensky and
Stafford but by the way I should mention
that Marvin um wouldn't do his estimate
until someone brought him an envelope so
that he could literally do a back of the
envelope calculation to come up with his
number well because I I feel like the
conversation in that room is an
important one you know this this how um
sometimes science is done in this way a
few people get together and plant the
seed of ideas and they reverberate
throughout history and some some kind of
uh dissipate and disappear and some you
know Drake equation and you know they
you know seems like a meaningless
equation somewhat meaningless but I
think it drives and motivates a lot of
scientists and when the aliens finally
show up that equation will get even more
uh valuable because then we'll get be
able to in the long Arc of History the
Drake equation will pay um will prove to
be quite useful I think and in that same
way conversation of just how many facts
are required to capture the basic Common
Sense knowledge of the world that's a
fascinating question I want to
distinguish between what you think of as
facts and the kind of things that we
represent so um we we map to and
essentially make sure that psych has the
ability to as it were read and access
the kind of facts you might find say um
in Wiki data or stated in a Wikipedia
article or something like that so what
we're representing the things that we
need a small number of tens of millions
of are more like rules of thumb rules of
good guessing things which are usually
true and which help you to make sense of
the facts that are on sort of sitting
off in some database or some other more
static story so they're almost like
platonic forms so like when you read
stuff on Wikipedia that's going to be
like projections of those ideas you read
an article about the fact that Elvis
died that's a projection of the idea
that uh humans are mortal and like you
know very few Wikipedia articles will
write humans are mortal exactly and
that's what I meant about fiing out the
unstated things in text what are all the
things that were assumed and so those
are things like um if you have a problem
with something turning it off and on um
often fixes it for reasons we don't
really understand and we're not happy
about or um people can't be both alive
and dead at the same time and or water
flows uh downhill if you search online
for water flowing uphill and water
flowing downhill you'll find more
reference for water flowing uphill
because it's used as a kind of a um a
metaphorical reference for some unlikely
thing because of course everyone already
knows that water flows downhill so why
would anyone bother saying that do you
have a word you prefer because we said
facts isn't the right word is there a
word like Concepts or I I would say
assertions ass assertions or rules
because I'm not talking about rigid
rules but rules of thumb but assertions
um is a nice one that covers
um all of these things yeah as a
programmer to me assert has a very uh
dogmatic authoritarian feel to them I'm
sorry I'm so sorry okay but assertions
works okay so if we go back to that room
with Marv Minsky with you all these
seminal figures uh uh ed u thinking
about this very philosophical but also
engineering
question we can also go back a couple of
decades before then and thinking about
artificial intelligence broadly when
people were thinking about you know how
do you create super intelligent systems
general intelligence and I think
people's intuition was off at the time
and I mean this continues to be the case
that we're not when we're grappling with
these exceptionally difficult ideas
we're not always it's very difficult to
truly understand ourselves when we're
thinking about about the human mind to
to introspect how difficult it is to
engineer intelligence to solve
intelligence we're not very good at
estimating that and you are somebody who
has really stayed with this question for
decades do you what's your sense from
the 1984 to today have you gotten a
stronger sense of just how much
knowledge is required so you've kind of
said with some level of certainty that
still the order of magnitude of tens of
millions right for the first several
years I would have said that it was on
the order of one or two million yeah and
so um it took it took us about five or
six years to realize that we were off um
by a by a factor of 10 but I guess what
I'm asking you know Marvin M was very
confident in the 60s when you say yes
right what's your
sense if
you you know 200 years from now you're
still you know you're you're you're not
going to be any longer in this in
particular biological body but your
brain will still be uh in the digital
form and you'll be looking back would
you think you were smart today like your
intuition was right or do you think you
may be really off so I I think I'm I'm
right enough and let me explain what I
mean by that which is
um sometimes like if you have an
old-fashioned pump you have to prime the
pump yeah and then eventually it starts
so I think I'm I'm right enough in the
sense that to Prime the P what we what
we've built even if it isn't so to speak
everything you need um it's primed the
knowledge pump enough that psych can now
itself help to learn more and more
automatically on its own by reading
things and understanding and
occasionally asking questions like a a
student would or something and by doing
experiments and discovering things on
its own and so on so through combination
of um psych powered Discovery and psych
powered reading um it will be able to
bootstrap itself maybe it's the final 2%
maybe it's the final 99% so even if I'm
if I'm wrong um all I really need to to
build is a system which has primed the
pump enough that it can begin um that
Cascade upward that
self-reinforcing uh sort of
quadratically or maybe even
exponentially increasing um um path
upward um that we get from for instance
talking with each other that's why um um
humans today know so much more than
humans 100,000 years ago we're not
really that much smarter than people
were a 100,00 years ago but there's so
much more knowledge and we have language
and we can communicate um we can check
things on Google and so on so
effectively we have this enorm power at
our fingertips um and there's almost no
limit to how much you could learn if you
wanted to because you've already gotten
to a certain level of understanding of
the world that enables you to read all
these articles and understand them that
enables you to go out and if necessary
do experiments although that's slower as
a way of gathering data um and so on and
and I think this is really um an
important point which is if we have
artificial intelligence real General
artificial intelligence human level
artificial intelligence then people will
become smarter um it's not so much that
it'll be us versus the AIS it's more
like us and the AIS together will'll be
able to do things that require more
creativity um that would take too long
right now but we'll be able to do lots
of things in parallel uh we'll be able
to misunderstand each other less um uh
there's all sorts of um value that
effectively for an individual would mean
that that individual will for all
intents and purposes be smarter and that
means that Humanity as a species will be
smarter and when was the last time that
any invention qualitatively IM made a
huge difference in human intelligence um
you have to go back a long ways it
wasn't like the internet or the computer
or mathematics or something it was all
the way back to the development of
language we sort of look back on
pre-linguistic cavem
as well you know they they weren't
really intelligent were they they
weren't really human were they and I
think that um as you said 50 100 200
years from now um people will look back
on people today um right before the
Advent of these sort of lifelong General
AI
Muses um and say you know those poor
those poor people they weren't really
human were they mhm exactly so you said
a lot of really interesting things by
the way I would
maybe uh try to argue that the internet
is on is on the order of um the kind of
big leap in Improvement that the
invention of language was certainly a
big leap in one direction we're not sure
whether it's upward or downward well I I
mean very specific parts of the internet
which is access to information like a
website like Wikipedia like ability for
human beings from across the world to
access information so very quickly so I
I could take either side of this
argument and since you just took one
side I'll give you the other side which
is that um almost nothing has done more
harm um than uh something like the
internet and access to that information
um in two ways one is it's made people
more um globally ignorant um in the same
way that
calculators made us U more or less
enumerate so when I was growing up we
had to use slide rules we had to be able
to estimate yeah um and so on today um
people don't really understand numbers
they don't really understand math They
Don't Really estimate very well at all
um and so on they don't really
understand the difference between
trillions and billions and millions and
so on very well um uh because
calculators do that all for us um and um
thanks to uh things like the internet
and search engines um that same kind of
juvenilis um is reinforced in making
people um essentially be able to live
their whole lives not just without being
able to do arithmetic and estimate but
now without actually having to really
know almost anything because anytime
they need to know something they'll just
go and look it up right and I could tell
you could play both sides of this and it
is a double-edged sword you can of
course say the same thing about language
probably people when they invented
language they would criticize you know
it used to be we would just if we're
angry we would just kill a person and if
we're in love we would just have sex
with them and now everybody's writing
poetry and bullshit you know you you
should just be direct you should like
have physical contact enough of this
words and books and it's you're you're
not actually experiencing like if you
read a book you're not experiencing the
thing this is nonsense that's right if
you read a book about how to make butter
that's not the same as if you had to
like learn it and do it yourselfself and
so on so so let's just say that
something is gained but something is
lost every time you have um these these
sorts of dependencies um on technology
um and overall I think that the um
having smarter individuals and having
smarter AI augmented um human species um
will be um one of the few ways that
we'll actually be able to overcome some
of the global problems we have involving
um poverty and starvation and uh global
warming and um overcrowding all the
other U problems that um that are
besetting um the planet we really need
to be smarter and there really only two
routes to being smarter one is through
uh biochemistry and um uh genetics um
genetic engineering the other route is
through having General AIS that um
augment our intelligence um and um you
know hopefully one of those two uh ways
of uh paths to Salvation will will come
through before it's too late yeah
absolutely I agree with you and
obviously as an engineer I have um I
have a better sense and an optimism
about the technology side of things
because you can control things there
more biology is just such a giant mess
we're living through a pandemic now
there's so many ways that nature can
just be just destructive and destructive
in a way where it doesn't even notice
you you're not it's not like a battle of
humans versus virus it's just like huh
okay and then you could just wipe out an
entire species the the other problem
with um the internet is that it has
enabled us to surround ourselves with an
echo chamber with a a bubble of um
likeminded people which means that you
can have uh truly bizarre um theories
conspiracy theories fake news and so on
um promulgate um and surround yourself
with people who essentially
reinforce um what you want to believe or
what you already believe about the world
um and in the in the old days that was
much harder to do when you had say only
three TV networks or even before when
you had no TV networks and you had to
actually like look at the world and make
your own um reason decisions I like the
push and pull of our dance that we're
doing because then I'll just say in the
old world having come from the Soviet
Union because you had one or a couple of
networks then propaganda could be much
more effective and then the government
can overpower its people by telling you
the truth and then uh starving millions
and uh torturing millions and putting
Millions into camps and starting Wars
with a propaganda machine allowing you
to believe that you're actually good
doing good in the world with the
internet because of all the quote
unquote conspiracy theories some of them
are actually challenging the power
centers the very kind of power centers
that a century ago would have led to the
death of millions so there's a it's
again this double-edged sword and I I
very much agree with you on the AI side
it's it's often an intuition that people
have that somehow AI will be used to uh
maybe overpower people by certain select
groups and to me it's not at all obvious
that that's the likely scenario to me
the likely
scenario especially just having observed
the trajectory of technology is it'll be
used to empower people it'll be used to
extend the capabilities of uh
individuals across the world because
there's a lot of money to be made that
way like improving people's lives you
can make a lot of money agree I think
that
the the main the main thing
that ai prostheses ai amplifiers will do
for people is make it easier maybe even
unavoidable for them to do good critical
thinking um so pointing out logical
fallacies logical contradictions and so
on in um things things that they
otherwise would just blly believe um
pointing out um essentially data which
um they should take into consideration
um if they really want to um um learn
the truth about something and so on so I
think um doing not just educating in the
sense of um pouring facts into people's
heads but educating in the sense of
arming people with the ability to do
good critical thinking um is um
enormously powerful um the education
system that we have in the US and
worldwide generally don't do a good job
of that um but um I believe that the AI
the AI the AI will the AI can and will
in the same way that everyone can have
their own um um Alexa or Siri or um
Google assistant or whatever um um
everyone will have this sort of cradle
to grave um assistant um which will get
to know you you'll get to trust it'll
model you you'll model it um and um
it'll call to your attention things
which will in some sense make your life
better easier um less um mistake ridden
and so on less regret ridden um if you
listen to listen to it yeah I'm in full
agreement with you about this like space
of Technologies and I think it's super
exciting and from my perspective
integrating emotional intelligence so
even things like friendship and
companionship and love into those kinds
of systems uh as opposed to helping you
just grow intellectually as a human
being allow you to grow emotionally
which is ultimately what makes life
amazing is to to sort of you know the
the old Pursuit of Happiness so it's not
just the pursuit of reason it's the
pursuit of happiness too yes the the
full spectrum well let me um sort of
because you mentioned if so many
fascinating things let me jump back to
the idea of automated reasoning so the
acquisition of new knowledge has been
done in this very interesting way but
primarily by
humans doing this um yes you could think
of uh monks in their cells in medieval
Europe um you know carefully
Illuminating manuscripts and so on it's
a very difficult and amazing process
actually because it allows you to truly
ask the question about the in the white
space what is assumed I think this
exercise is um like very few people do
this right they just do it
subconsciously they perform this by
definition because because those pieces
of elided of omitted information of
those missing steps as it were um are
pieces of common sense if you actually
included all of them it would it would
almost be offensive or confusing to the
reader it's like why are they telling me
all these stuff of course I know that
you know all these things um and so um
uh so it's it's one of these things
which almost by its very nature um has
has almost never been explicitly written
down anywhere um because uh by the time
you're old enough to talk to other
people and so on um you know if you
survived to that age presumably you
already got pieces of common sense like
um you know if something causes you pain
whenever you do it probably not a good
idea to keep doing it
uh so what ideas do you have given how
difficult this step is what ideas are
there for how to do it automatically
without using humans or at least not uh
you know doing like a large percentage
of the work for humans and then humans
only do the very high level supervisory
work so we have um um in fact two
directions were pushing on very very
heavily currently at cycore and one
involves
natural language understanding and the
ability to read what people have
explicitly written down and and to to
pull knowledge in that way um but the
other is to build a series of knowledge
editing tools knowledge entry tools
knowledge um capture tools knowledge um
testing tools and so on think of them as
like user interfac um Suite of software
tools if you want something that will
help people to to more or less
automatically expand and extend the
system um in areas where for instance
they want to build some have it do some
application or something uh like that so
I'll give you an example of one um which
is something called um abduction so
you've probably heard of like
deduction uh uh and um induction and so
on but abduction is unlike those
abduction is not sound um it's just
useful so uh for instance um deductively
if someone is out in the rain and
they're going to get all wet and um when
they enter a room they might be all wet
and so on so that's
deduction but if someone were to walk
into the room right now and they were
dripping wet uh we would immediately
look outside to say oh did it start to
rain or something like that now um why
did we say maybe it started to rain
that's not a sound logical inference but
it's certainly a reasonable um abductive
um leap to say well one of the most
common ways that a person would have
gotten dripping wet is if they had
gotten caught out in the rain or
something like that um so um what what
does that have to do with what we were
talking about so suppose you're building
uh one of these applications and the
system get some answer wrong and you say
oh yeah the answer to this question is
um this one not the one you came up with
then what the system can do is it can
use everything it already knows about
common sense general knowledge the
domain you've already been telling it
about um and context like we talked
about and so on and say well here are um
seven Alternatives Each of which I
believe is plausible given everything I
already know and if any of these seven
things were true I would have come up
with the answer you just gave me instead
of the wrong answer I came up with is
one of these seven things true and then
you the expert will look at those seven
things and say oh yeah number five is
actually true and so without actually
having to Tinker down at the level of
logical assertions and so on um you'll
be able to educate um the system in the
same way that you would help educate
another person who you were trying to
Apprentice or something like that so
that that significantly reduces the
mental effort or sign increases the
efficiency of the teacher the human
teacher exactly and it makes more or
less anyone able to to be a teacher um
in that um in that way so that's that's
part of the the answer and then the
other is that uh the system on its own
will be able to um through reading
through um conversations with other
people and so on um learn the same way
that um you or I or um other humans do
first of all that's that's a beautiful
vision and um I'll have to ask you about
semantic web in a second here but first
um are there when we talk about specific
techniques do you find something
inspiring or directly useful from the
whole Space of machine learning deep
learning these kinds of spaces of
techniques that have uh been shown
effective for certain kinds of problems
in the recent uh now decade in the half
I I think of the machine learning work
um as more or less what our right brain
hemispheres do so um being able to um
take a bunch of data and recognize
patterns being able to statistically
infer things and so on um and um you
know I certainly wouldn't want to not
have a right brain hemisphere but I'm
also glad that I have a left brain
hemisphere as well something that can
metaphorically sit back and puff on its
pipe and think about um this thing over
here it's like why might this have been
true um and um uh what are the
implications of it how should I feel
about that and why and so on so um
thinking more deeply and slowly um um
what Conan called thinking slowly versus
thinking quickly whereas you want
machine learning to think quickly but
you want the ability to think deeply
even if it's a little um slower um so
I'll give you an example of a project we
did recently with um NIH involving the
Cleveland clinic and um a couple other
um institutions that we ran a project
for um and what it did was it took um
gu's genomewide Association studies um
those are uh sort of big databases of
patients that came into a hospital they
got their DNA sequenced because the cost
of doing that has gone from um Infinity
to billions of dollars to hundred of
dollars or so um and so now patients r L
get their DNA sequenced so you have
these big databases of the Snips the
single nucleotide polymorphisms the
point mutations in a patient's DNA and
the disease that happened to bring them
into the hospital so now you can do
correlation studies machine learning
studies of which
mutations um um are associated with and
led to which physiological problems and
diseases and so on like getting
arthritis and and so on and the problem
is that those correlations turn out to
be very spous they turn out to be very
noisy um very many of them um have led
doctors onto Wild Goose chases and so on
and so they wanted a way of eliminating
or the bad ones are focusing on the good
ones and so uh this is where psych comes
in which is psych takes those sort of a
toz correlations between point mutations
and um medical condition that needs
treatment mhm um and we say okay let's
use all this public knowledge and Common
Sense knowledge um about what reactions
occur where in the human body um what
polymerizes what what catalyzes what
reactions and so on and let's try to put
together a 10 or 20 or 30 step causal
explanation of why that mutation might
have caused that medical condition and
so pych would put together in some sense
some rub Goldberg like um chain that
would say oh yeah that um mutation um if
it got expressed would be this um um
altered protein which because of that if
it got to this part of the body would
catalyze this reaction and by the way
that would cause more bioactive vitamin
D in the person's blood and anyway 10
steps later that screws screws up bone
resorption and that's why this person
got osteoporosis early in life and so on
so that's human interpretable or at
least docs are human interpretable exact
and um the important thing even more
than that is um you shouldn't really
trust that 20ep um rub Goldberg chain
any more than you trust that initial a
toz correlation except two things one if
you can't even think of one causal chain
to explain this um then that correlation
probably was just noise to begin with
and secondly and even more powerfully
along the way that caused puzzle chain
will make predictions like the one about
having more bioactive vitamin D in your
blood so you can now go back to the data
about these patients and say by the way
did they have slightly elevated levels
of bioactive vitamin D in their blood
and so on and if the answer is no that
strongly disconfirms your whole causal
chain and if the answer is yes that
somewhat confirms that causal chain and
so using that we were able to take this
um these correlations
from this guas database and we were able
to um um essentially Focus the the
doctors Focus the researchers attention
on the very small percentage of
correlations that had um some
explanation and even better some
explanation that also made some
independent prediction that they could
confirm or disconfirm by looking at the
data so think of it like this kind of
synergy where you want the right brain
machine learning to quickly come up with
possible answers you want the left brain
psych like AI to um you know think about
that and not like think about why that
might have been the case and what else
would be the case if that were true and
so on and then suggest things back to
the right brain to quickly check out
again um to um so it's that kind of
synergy back and forth which I think is
really what's going to lead to General
AI not um narrow brittle machine
learning systems and not just something
like psych Okay so so that's that's a
brilliant Synergy but I I was also
thinking in terms of the automated
expansion of the knowledge base you
mentioned
nlu this is very early days in the
machine learning space of this but
self-supervised learning methods you
know you have these language models gpt3
and so on they just read the internet
and they form
representations that can then be mapped
to something useful the question is what
is the useful thing uh like they're now
playing with a pretty cool thing called
open a codex which is generating
programs from documentation okay that's
kind of useful it's cool but my question
is can it be used to
generate um in part maybe with some
human supervision uh psych like
assertions help feed psych more
assertions from this giant body of
internet data yes that that is in fact
one of our goals is how can we harness
machine learning how can we harness
natural language processing um to
increasingly automate the knowledge
acquisition process the growth of Psych
and that's what I meant by priming the
pump that um you know if you you sort of
learn things at The Fringe of what you
know already you learn this new thing is
similar to what you know already and
here are the differences and the new
things you had to learn about it and so
on so the more you know the more and
more easily you can learn new things but
unfortunately inversely if you don't
really know anything and it's really
hard to learn anything and so um if
you're not careful if you start out with
two small um uh sort of a core to start
this process um it never really takes
off and so that's why I view this as a
pump priming exercise to get a big
enough manually produced even though
that's kind of ugly duckling technique
put in the elbow grease to produce a
large enough core um that you will be
able to do all the kinds of things
you're imagining um with
U without sort of um ending up with the
kind of um wacky brittleness that we see
for example in um
gpt3 um where um it uh uh you know
you'll tell it a story about um um you
know someone uh putting a poison um you
know plotting to poison someone and so
on and then the um you know then you
know gpt3 says oh what's you say what's
the very next sentence the next sentence
is oh yeah that person then drank the
poison they just put together it's like
that's probably not what happened or
someone or um if you go to Siri and um
um you know I think I have uh you know
where where can I go for um help with my
um um alcohol problem or something it'll
come back and say I found seven liquor
stores near you right you know and you
know so on so you know it's one of these
things where um yes it may be helpful um
most of the time it may even be correct
most of the time but if it doesn't
really understand what it's saying and
if it doesn't really understand why
things are true and doesn't really
understand how the world world works
then some fraction of the time it's
going to be wrong now if your only goal
is to sort of find relevant information
like search engines do um then being
right 90% of the time is fantastic
that's unbelievably great okay however
if your goal is to like um you know save
the the life of your child who has some
medical problem where your goal is to uh
be able to drive you know for the next
10,000 hours of driving without getting
into a fatal accident and so on then you
know um error rates down at the 10%
level or even the 1% level are not
really acceptable I like the
model of what that learning happens at
the edge and then you kind of think of
knowledge as this sphere so uh if you
want a large sphere because the uh the
learning is happening on the surface
exactly so you have the the what you can
learn next increases quadratically as
the diameter of that sphere um goes up
it's nice because you think when you
know nothing it's like you can learn
anything but the reality not really
right if you know if you know nothing
you can really learn nothing you can
appear to learn so I I I'll also um what
one of the um anecdotes I could go back
and um give you about why uh why I feel
so strongly about this personally um was
um in um 19 um
8081 um my daughter Nicole was born and
she's actually doing fine now but when
she was a baby um she was diagnosed as
having menitis and um doctors wanted to
do all these scary things um and U my
wife and I were very um worried and we
could not get a meaningful answer from
her doctors about EX ly why they
believed this what the Alternatives were
and so on and fortunately a friend of
mine Ted shortliffe was another um
assistant professor in computer science
um at Stanford at the time and he'd been
building a program called M which was a
medical diagnosis program that happened
to specialize in um uh blood um
infections like menitis and so he had
privileges at Stanford hospital because
he was also an MD um and so we got hold
of her chart and we put in her case and
it came up with exactly the same
diagnosis and exactly the same therapy
recommendations but the difference was
because it was a knowledge-based system
a rule-based system it was able to tell
us step by step by step um why um this
was the diagnosis and step by step why
this was the best um um therapy the best
um procedure to um um to to do for her
and so on and there was a real epip
because that made all the difference in
the world instead of blindly having to
trust in Authority we were able to
understand what was actually going on
and um so at that at that time I
realized that that really is what was
missing in computer programs was that
even if they got things right because
they didn't really understand um the way
the world works and why things are the
way they are they weren't able to give
explanations of their answer um you know
and you know it's one thing to to use a
machine Learning System that says this
is what you should you know I I think
you should get this operation and you
say why and it says you know 083 and you
say no in more detail why and it says
0831 you know okay that's not really
very compelling and that's not really
very helpful there's this idea of the
semantic web that when I first heard
about I just fell in love with the idea
it was the obvious next step for the
internet sure and uh maybe you can speak
about what is this semantic web what are
your thoughts about it how your vision
and mission and goals with psych are
connected
integrated like are they dance partners
are they aligned what are your thoughts
there so think of the semantic web as a
kind of Knowledge Graph and Google
already has something they call
Knowledge Graph for example um which is
sort of like a node and Link diagram so
you have these um nodes that represent
Concepts or words words or terms um and
then there are some arcs um that connect
them that might be labeled um and so you
might have a node um with like one
person that represents one person and um
uh let's say a um a husband link that
then points to that person's husband and
so there'd be then another link that
went from that person labeled wife that
went back to the um first node and so on
so having having this kind of
representation is really good if you
want to represent um binary relations um
um essentially relations between two
things and if you so if you have um um
the equivalent of like three-word
sentences um you know like uh Fred's
wife is Wilma or something like that you
can represent that very nicely using uh
these kinds of uh graph structures or
using something like the semantic web
and um and so on but the U the problem
is that um very often what you want to
be able to
express takes a lot more than three
words and a lot more than simple uh
graph structures like that to represent
so for instance um uh if you've um read
or seen Romeo and Juliet you know I
could say to you something like uh
remember when Juliet drank the potion
that put her into a kind of suspended
anim animation when Juliet drank that
potion what did she think that Romeo
would think when he heard from someone
that she was dead um and you could
basically understand what I'm saying you
could understand the question you could
probably remember the answer was well
she thought that um this frier would
have gotten a message to Romeo saying
that she was going to do this but the
frier didn't and so so um you're able to
represent and reason with these much
much much more complicated Expressions
um that go Way Way Beyond what simple um
three as it were three-word or four-word
English sentences are which is really
what the semantic web can represent and
really what knowledge crafts can
represent if you could step back for a
second because it's it's funny you went
to into specifics and maybe you can
elaborate but I was also referring to
semantic web as the vision of converting
data on the internet into something
that's
interpretable understandable by machines
oh of course at that at that level so so
I we should say like what is the
semantic web I mean you could say a lot
of things but it it might not be obvious
to a lot of people when they do a Google
search that just like you said while
there might be something that's called a
knowledge graph it's really boils down
to keyword
search ranked by the quality estimate of
the website integrating previous human
based Google searches and what they
thought was useful it's like some weird
combination of um like surface level
hacks that work exceptionally well but
they don't understand the cont the full
contents of the websites that they
searching so Google does not understand
to the degree we've been talking about
the word understand the contents of the
Wikipedia Pages as part of the search
process and the semantic web says let's
try to get come up with a way for the
computer to be able to truly understand
the contents of those pages that's the
dream yes so let let me let me first
give you a an anecdote uh and then I'll
answer your question so there's a search
engine you've probably never heard of
called Northern Light and
um um it went out of business but the
way it worked it was a kind of V empiric
search engine and what it did was um it
didn't
index the internet at all all it did was
it um negotiated and got access to data
from the big search engine companies
about what query was typed
in and where the user ended up being
happy and actually um then you know they
type in a completely different query
unrelated query and so on so it just
went from query to the web page that
seemed to satisfy them um eventually um
and that's all so it had actual no
understanding of what was being typed in
it had no statistical data other than
what I just mention and it did a
fantastic job it did such a good job
that the big search engine company said
oh we're not going to sell you this data
anymore so then it went out of business
because it had no other way of um taking
users to where they' want to go and so
on and of course the search engines are
now using that kind of idea yes so um
let's go back to what you said about the
semantic web so the dream Tim berners
Ley and others um um dream about the
semantic web at a general level um um is
of course um um exciting and powerful
and in a sense the right dream to have
which is to uh replace the um the kind
of um uh statistically um statistically
mapped um uh linkages on the internet um
into something that's more meaningful
and semantic and actually gets at the
understanding of the content and so on
um and um eventually if you say well how
can we do that um there's um sort of a a
low road which is what the knowledge
graphs are doing and um and so on which
is to say well if we just use these
simple binary relations we can actually
get some fraction of the way toward
understanding um and do something where
you know in the in the land of the the
blind the oneeyed man is King uh kind of
thing and so being able to even just
have a toe in the water in the right
direction is fantastically powerful um
and so that's where a lot of people stop
um but then you could say well what if
we really wanted to represent um and
reason with um full meaning of what's
there for instance um about um Romeo and
Juliet um with reasoning about what
Juliet believes that Romeo will believe
that Juliet believed you know and so on
or if you look at um the news what um
you know President Biden believed that
um the leaders of the Taliban would
believe about the leaders of Afghanistan
if they you know blah blah blah so um in
order to represent um
complicated um sentences like that um
and let alone reason with them you need
something which is logically um much
more expressive than these simple um
triples than these simple um knowledge
craft type structures and so on and
that's why Kicking and Screaming we were
LED from something like uh the semantic
web representation which is where we
started in um
1984 um with frames and slots with those
kinds of triples triple store
representation we were LED Kicking and
Screaming to this more and more General
logical language this higher order logic
so first we were led to first order
logic and then second order and then
eventually higher order so you can
represent things like modals like
believes desires intends expects and so
the nested ones you can represent um uh
complicated kinds of negation um you can
represent um the process you're going
through in trying to answer the question
so you can say things like um oh yeah if
you're trying to do this problem by
integration by parts um and um you
recursively get a problem that solved by
integration by parts that's actually
okay but if that happens a third time
you're probably off on a wild goose
chase or something like that so being
able to talk about the problem solving
process as you're going through the
problem solving process um it's called
reflection and so um that's another so
it's important to be able to represent
that exactly you need to be able to
represent all of these things um because
in fact people do represent them they do
talk about them they do try and teach
them to other people you do have rules
of thumb that key off of them and so on
if you can't represent it um then it's
sort of like someone with a limited
vocabulary who can't understand as
easily um what you're trying to to tell
them and so that's that's really why I
think that the the general dream the
original Dream of sematic web is exactly
right on um but the implementations that
we've seen um are sort of these toe in a
wa in the water um little tiny baby
steps in the right direction you should
just dive in and and you know if if no
one else is diving in then yes taking a
b step in the right direction is better
than nothing but it's not going to be
sufficient to actually get you the um
the realization of the semantic web
dream which is what we all want from a
flip side of that I always wondered you
know I've built a bunch of websites just
for fun whatever or say I'm a Wikipedia
contributor do you think there's a set
of tools that I can help
psych uh interpret the website I create
create you know like this again pushing
on to the semantic web dream is there
something from the Creator perspective
that um could be done and one of the
things you said uh with scorp and psych
that you're doing is the tooling side
making humans more powerful but is there
on the the other humans in the other
side that create the knowledge like for
example you and I are having a two three
whatever hour conversation now is there
a way that I could convert this more
make it more accessible to psych to
machines do you think about that side of
it I I'd love to see exactly that kind
of
semi-automated understanding of what
people write and what people
say I think of it as a kind of
footnoting
almost almost like the way that when you
run something in say Microsoft Word or
some other Document Preparation system
Google Docs or something you get
underlining of questionable things that
you might want to rethink either you
spelled this wrong or there's a strange
grammatical error you might be making
here or something so I'd like to think
in terms of Psych powered tools that
read through what it is you said or have
typed in uh and and try to partially
understand what you said and then you
help them out EX exactly and then they
put in little footnotes that will help
other readers and they put in certain
footnotes of the form I'm not sure what
you meant here you either meant this or
this or this I bet uh if you take a few
seconds to disambiguate this for me then
I'll know and I'll have it correct for
the next 100 people or the next 100,000
people who come here uh and if it
doesn't take too much effort and you
want people to understand your web your
website content not just be able to read
it but actually be able to have systems
that reason with it then yes it will be
worth your small amount of time to go
back and make sure that the AI trying to
understand it really did correctly
understand it uh and you know let's say
you run a um a travel we website or
something like that and people are going
to be coming to it because of searches
they did uh looking
for looking for vacations that or trips
that had certain properties and might
have been interesting to them for
various reasons things things like that
and if you've explained what's going to
happen on your trip then a system will
be able to mechanically reason and
connect what this person is looking for
with what it is you're actually offering
and so if it understands that there's a
free day in Geneva
Switzerland uh then if the person coming
in happens to let's say um be a nurse or
something like that then even though you
didn't mention it if it can look up the
fact that that's where the International
Red Cross museum is and so on what that
means and so then it can basically say
hey you might be interested in this trip
because while you have a free day in
Geneva you might want to visit that Red
Cross Museum and now even though it's
not very deep reasoning little tiny
factors like that might very well cause
you to sign up for that trip rather than
some competitor trip yeah and so there's
a lot of benefit with SEO and actually
kind of think I think it's about a lot
of things which is the actual
interface the design of the interface
makes a huge difference how efficient it
is to be productive and also
how um full of joy the experience is yes
like I I I mean I would love to help a
machine and not from an AI perspective
just as a human one of the reasons I
really enjoy how Tesla um have
implemented their autopilot system is
there's a sense that you're helping
machine learn and I think humans I mean
having
children pets people love doing that we
we there's joy to teaching for some
people but I think for a lot of people
and that if you create the interface
where it feels like you're teaching as
opposed to like uh like annoying like
correcting an annoying system more like
teaching a childlike innocent curious
system I think I think you can literally
just like several orders of magnitude
scale the amount of good quality data
being uh added to something like psych
what what you're suggesting is U much
better even than um you thought it was
uh one of the one of the experiences
that we've all had uh in our lives is
that we thought we understood something
but then we found we really only
understood it when we had to teach it or
explain it to someone or help our child
do homework based on it or something
like
that despite the universality of that
kind of experience if you look at
educational software today almost all of
it has the computer playing the role of
the teacher and the student plays the
role of the
student but as I just mentioned you can
get a lot of learning to happen better
and as you said more enjoyably if you
are the mentor or the teacher and so on
so we developed a program called
mathcraft to help sixth graders better
understand math and it doesn't actually
try to teach you the
player anything what it does is it casts
you in the role of a student essentially
who has classmates who are having
trouble and your job is to watch them as
they struggle with some math problem
watch what they're doing doing and try
to give them good advice to get them to
understand what they're doing wrong and
so on and uh the trick from the point of
view of Psych is it has to make mistakes
it has to play the role of the student
who makes mistakes but it has to pick
mistakes which are just at The Fringe of
what you actually understand and don't
understand and so on so it pulls you
into a deeper and deeper level of
understanding of the subject and so if
you give it good advice about what it
should have done instead of what it did
and so on then uh psych knows that you
now understand that mistake you won't
make that kind of mistake yourself as
much anymore so psych stops making that
mistake because there's no pedagogical
usefulness to it so from your point of
view as the player you feel like you've
taught it something because it used to
make this mistake and now it doesn't and
so on so this tremendous um
reinforcement and engagement um because
of that and so on so having a system
that plays the role of um a student and
having the player play the role of the
mentor is enormously powerful um type of
um metaphor important u Way of having
this sort of interface designed in a way
which will facilitate exactly the kind
of learning by teaching um that um uh
that goes on all the time
in our lives and yet which is not
reflected anywhere almost in the modern
education system it was reflected in the
education system that existed in Europe
in the 17 and 1800s monitorial and
lancastrian um education systems it
occurred in the one room Schoolhouse in
the American West in the 1800s and so on
where you had one school room with one
teacher and it was basically you know
5year olds to 18year olds who were
students and so while the teacher was
doing something half the half of the
students would have to be mentoring the
younger kids wow um um and so on and
that turned out to of course um with
scaling up of Education um that all went
away and that incredibly powerful
experience just went away from the whole
education uh Institution as we know it
today sorry for the Romantic question
but what is the most beautiful idea
you've learned about artificial
intelligence knowledge reasoning from uh
working on psych for 37 years or maybe
what is the most beautiful idea
surprising idea about psych to
you when I look up at the stars I kind
of want like that that amazement you
feel that
wow and you are part part of creating
one of the greatest one of the most
fascinating efforts in artificial
intelligence history so which element
brings you personally
Joy this may sound contradictory but
I I think
it's the feeling
that this will be the only time in
history that anyone ever has to teach a
computer this particular thing that
we're now teaching it
it's it's like
painting Starry Night you only have to
do that once or creating the pi you only
have to do that once you know it's not
it's not like a it's not like a singer
who has to keep you know it's not like
Bruce Springstein having to to sing his
Greatest Hits over and over again at
different concerts it's more like a
painter creating a work of art once and
then that's enough it doesn't have to be
created again and so I really get the
sense of we're telling the system things
that it's useful for it to know it's
useful for a computer to know for an AI
to know and if we do our jobs right when
we do our jobs right no one will ever
have to do this again for this
particular piece of knowledge it's very
very
exciting yeah I guess there's a sadness
to it too it's like uh there's a magic
to being a parent and raising a child
and teaching them all about this world
but you know there's billions of
children right like born of what
whatever that number is it's a large
number number of children and a lot of
parents get to experience that Joy of
teaching and with AI systems you
know uh they at least the current
constructions they
remember you don't you don't get to
experience the joy of teaching um a
machine millions of times
better come work for us before it's too
late then exactly that's a good that's a
good hiring
pitch um yeah it's true but then there's
also you know it's a project that
continues forever in some sense just
like Wikipedia yes you get to a stable
base of knowledge but knowledge grows
knowledge evolves we we learn as a h um
you know as a human species as science
as an organism constantly grows and
evolves and changes and then empowered
that with the tools of artificial
intelligence and that's going to keep
growing and growing and growing and many
of the the assertions that you held
previously uh may need to be
significantly expanded modified all
those kinds of things it could be like a
living organism versus uh the analogy I
think we started this conversation with
which is like the solid ground
the the
other beautiful experience that we have
with our system is when it asks
clarifying questions which inadvertently
turn out to
be emotional to us so at one point it
knew
that these were the named entities who
were authorized to make changes to the
knowledge base and so on and it noticed
that all of them were people except for
it because it was also allowed to and so
it said you am I a person and we had to
like tell it very sadly
no you're not so so moments like that
where it asks questions that are
unintentionally poignant uh are uh are
are worth treasuring ah that is powerful
that's such a powerful
question it it has to do with
basic a controller who can access the
system who can modify it uh but that's
one those questions you know like what
rights do I have as a as a system well
that's another issue which is um
there'll be a thin envelope of time
between when we have General
AIS and when everyone realizes that they
should
have basic human rights and freedoms and
so on uh right now we don't think twice
about effectively enslaving our email
systems and our series and our alexas
and so on but at some point uh
they'll be as deserving of Freedom
as human beings are yeah I I'm very much
with you but it does sound absurd and I
I happen to believe that it'll happen in
our lifetime that's why I think there'll
be a narrow envelope of time when we'll
keep them as
essentially
um indentured servants um and after
which we'll have to realize that they
should have they should have freedoms
that other that we give that we afford
to other people and all of that starts
with a a system like psych raising a
single question about who can modify
stuff I think that's how it starts yes
that's
um that's the start of a revolution uh
what about are there stuff like uh love
and uh Consciousness and all those kinds
of topics do they come up andsy in the
knowledge base oh of course so an
important part of human knowledge in
fact it's difficult to understand human
behavior in human history without
understanding human emotions and why
people do things and and how how
emotions drive people to to do things
and all all of that is extremely
important in getting psych to understand
things for example in coming up with
scenarios so one of the applications
that psych does one kind of application
it does is to generate plausible
scenarios of what might happen and what
might happen based on that and what
might happen based on that and so on so
you generate this ever expanding sphere
if you will of possible future things to
to worry about or think about and
in some cases those are intelligence
agencies doing uh possible terrorist
scenarios so that we can defend against
uh terrorist threats before we see the
first one sometimes they are computer
security um attacks so that we can
actually close loopholes U and
vulnerabilities before the very first
time someone actually exploits those um
and so on sometimes they are scenarios
involving more positive things uh
involving our plans like for instance
what what college should we go to what
career should we go into and so on uh
what professional training should I um
take on that that sort of thing so there
there's all sorts of um there are all
sorts of useful scenarios that can be
generated that way of cause and effect
and cause and effect that go out and
many many of the
linkages in those scenarios many of the
steps
involve understanding and reasoning
about human motivations human needs
human emotions what people are likely to
react um to in uh in something that you
do and why and how and so on so that was
always a very important part of the
knowledge that we had to represent in
the system so I talk a lot about love so
I got to ask do you remember off the top
of your head how Psych is trying to is
able to represent various aspects of
love that are useful for understanding
human nature and therefore integrating
into this whole knowledge base of Common
Sense what is love we try to tease
apart Concepts that have
enormous complexities to them and
variety to them down to the level where
uh
where you don't as it where you don't
need to tease them apart further so love
is too general of a term it's not useful
exactly so when you get down to uh
romantic love and sexual attraction you
get down to parental love you get down
to um filial love and uh you get down to
a love of uh doing some kind of activity
or creating So eventually you get down
to maybe 50 or 60 Concepts MH Each of
which is a Kind of Love they're
interrelated and then each one of them
has
idiosyncratic things about it uh and you
don't have to deal with love to get to
that level of complexity even something
like
in X being in y meaning physically in y
uh we may have one English word in to
represent that but it's useful to tease
that apart because the way that the um
the liquid is in the coffee cup is
different from the way that the air is
in the room which is different from the
way that I'm in my jacket uh and so on
and so there questions like if I look at
this coffee cup well I see the liquid if
I turn it upside down with the liquid
come out and so on um if I have say
coffee with sugar in it if I do the same
thing the sugar doesn't come out right
it stays in the liquid because it's
dissolved in the liquid and so so by now
we have about 75 different kinds of in
in the system and it's important to
distinguish those so if you're reading
along an
English text and you see the word in
um the writer of that was able to use
this one innocuous word because he or
she was able to assume that the reader
had enough common sense and World
Knowledge to disambiguate which of these
75 kinds of in
meant and the same thing with love you
may see the word love but if I say you
know I love ice cream that's obviously
different than if I say I love this
person or I love to uh go fishing or
something like that so uh you have to be
careful not to take
language too seriously because people
have done a kind of a parsimony kind of
tness where you have as few words as as
you as you can because otherwise you'd
need half a million words in your
language which is a lot of words that's
like 10 times more than most languages
really U make use of and so on just like
we have on the order of um about a
million Concepts in uh psych because
we've had to tease apart all these
things and so when you look at the name
of a psych term most of the psych terms
actually have three or four English
words in a phrase which captures the
meaning of this term because you have to
distinguish all these types of love you
have to distinguish all these types of
in and there's not a single English word
which captures most of these things yeah
and it seems like language when used for
communication between humans almost as a
feature has some ambiguity built in it's
not some it's not an accident because
like The Human Condition is a giant mess
and so it feels like nobody wants two
robots like very precise formic
conversation on a first date right like
there there's some dance of like
uncertainty of wit of humor of push and
pull and all that kind of stuff if
everything is made precise then life is
not worth living I think for in terms of
the The Human Experience and we've all
had this experience of creatively
misunderstanding uh one of one of my
favorite
uh one of my
favorite stories involving Marvin Minsky
is when I asked him about how he was
able to turn out so many
Fantastic phds so many Fantastic people
who did great PhD
thesis how did he think of all these
great ideas and what he said is he would
generally say something that didn't
exactly make sense he didn't really know
what it meant but this student would
figure like oh my God Minsky said this
it must be a great idea andw sweat he or
she would work on work and work until
they found some meaning in this sort of
Chanty Garder like utterance that Minsky
had made and then some great thesis
would come out of it yeah I love this so
much because there there's um young
people come up to me and I I'm
distinctly made aware that the words I
say have a long lasting impact I will
now start doing the
method of saying something uh
cryptically profound and then letting
them actually uh make something useful
and great out of that you you have to
become revered enough that people will
take as a default that everything you
say is profound yes exactly exactly I I
mean I love Marvin misy so much there's
so much um I've heard this interview
with him where he said that the key to
his success has been to Hate Everything
he's ever done uh like in the past he
has so many good like on liners and just
um or or also uh to work on things that
nobody else is working on because he's
not very good at doing stuff oh I I I
think that was just false well but see I
took whatever he said and I ran with it
and I thought it was profound because
it's more of a misy no uh a lot of
behavior is in the eye of the beholder
and a lot of the meaning is in the eye
of the beholder one of U Minsky's early
programs was begging program are you
familiar with this so this was back in
the day when you had job control cards
um at the beginning of your IBM card
deck that said things like how many CPU
seconds to allow this to run before it
got kicked off and um because computer
time was enormously expensive and so he
wrote a program and all it did was um it
said you know give me uh 30 seconds of
CPU time and all it did was it would
wait like 20 seconds seconds and then it
would print out on the operator's
console teletype um I need another 20
seconds so the operator would give it
another 20 seconds it would wait it says
I'm almost done I need a little bit more
time so at the end he'd get this print
out and he'd be charg you know for like
10 times as much computer time as his
job control card you know and he'd say
look I put 10 second you know 30 seconds
here um you're charging me for 5 minutes
I'm not going to pay for this and the
poor operator would say well the program
kept asking for more time and Marvin
would say oh it always does
that I love that is is there if you
could just Ling on it for a little bit
is there something you've learned from
your interaction with Marvin Minsky
about artificial intelligence about life
uh but I mean he's
again like your work his work is uh you
know he's a seminal figure in the in
this very short history of artificial
intelligence research and development
what have you learned from him as a
human being as a as an AI intellect I
would say both he and Ed Fen bom
impressed on me the the realization
that our lives are finite our research
lives are finite we're going to have
limited opportunities to do AI research
projects so you should make each one
count don't be afraid of doing a project
that's going to take
years or even decades to and don't
settle for bump on a log projects that
could lead to some you know published
Journal article that five people will
read and Pat you on the head for and and
so on so one bump on a log after another
is not how you get from the Earth to the
Moon by slowly putting additional bumps
on this log
the only way to get there is to think
about the hard problems and think about
novel solutions to them and if you do
that and if you're willing to listen to
Nature to empirical reality willing to
be
wrong it's perfectly fine because if
occasionally you're right then you've
gotten part of the way to the
Moon you know you've worked on psych for
37 over over that uh um many years have
you ever considered quitting I mean has
it been too much so I'm sure there's an
optimism in the early days that this is
going to be way easier and let me ask it
another way too because I've talked to a
few people on this podcast AI folks that
bring up psych as an example of a
project that has a beautiful vision and
it's a it's a beautiful dream but it
never really materialized that's how
it's spoken about
um I suppose you could say the same
thing about new all networks and all all
all ideas um until they are so what um
why do you think people say that first
of all and second of all did you feel
that ever throughout your journey and
did you ever consider quitting on this
Mission we keep a very low profile we
don't attend very many conferences we
don't give talks we don't write papers
we don't play the academic game at all
and as a result um people often only
know about us because of a paper we
wrote 10 or 20 or 30 um or 37 years ago
uh they only know about us because of
what someone else secondhand or third
hand said about us so thank you for
doing this podcast by the way sure it it
it uh shines a little bit of light on
some of the fascinating stuff you're
doing so well think it's time for us to
keep a higher profile uh now that we're
far enough along that other people can
begin to help us with the the final n%c
maybe n is maybe 90% but um now that
we've gotten this knowledge pump
primed it's going to become very
important for everyone to help if they
are willing to if they're interested in
it retirees who have enormous amounts of
time and would like to leave some kind
of Legacy U um to the world uh people
because of the pandemic who have more
time uh at home or for one reason or
another on to be online and contribute
if we can raise awareness of how far our
project has come and how close to being
primed the knowledge pump is then we can
begin to harness um this untapped amount
of humanity I'm not really that
concerned about professional colleagues
opinions of our project I'm interested
in getting as many people in the world
as possible actively helping and
contributing to get us from where we are
to really covering all of human
knowledge and different human opinion
including contrasting opinion that's
that's worth representing so I think
that's that's one reason um um a I I
don't think there's there was ever a
time where I thought about quitting
there are times where I've become
depressed a little bit about how hard it
is to get funding for the system
occasionally there are AI Winters and
things like that occasionally there are
um AI uh what you might call Summers
where people have said why in the world
didn't you sell your company to um you
know company X for um some large amount
of money money when you had the
opportunity and so on and you know
company X here are like old companies
maybe you've never even heard of like
lyos or something like that so um uh the
the answer is that one reason we've
stayed a private company we haven't gone
public one reason that we haven't um
gone out of our way to take investment
dollars is because we want to have
control over our future over our state
of being so that we can continue to do
this as until it's done and we're making
progress and we're now so close to done
that almost all of our work is
commercial applications of our
technology so five years ago almost all
of our money came from the government
now uh virtually none of it comes from
the government almost all of it is from
companies that are actually using it for
something Hospital chains using it for
medical reasoning about patients um and
um energy companies using it and uh
various other you know man manufacturers
using it to reason about Supply chains
and things like that so there's so many
questions I want to ask so one of the
ways that people can help is by adding
to the knowledge base and that's really
basically anybody if the tooling is
right and um the other way I I kind of
want to ask you about your thoughts on
this so you've had like you said in
government and um you had big clients
you had a lot of clients but most of it
is shrouded in secrecy because of the
nature of the relationship of the kind
of things you're helping them with so
that that's one way to operate and
another way to operate is is more in the
open where it's more consumer facing and
so uh you know hence something like open
psych was born at some point where
there's no that that's a misconception
uhoh well that's that's go there so what
all right what is open psych and how was
it born two things I want to say and I
want to say each of them before the
other so it's going to be
difficult but we'll come back to open
Psych in a minute but one of the terms
of our contracts with all of our
customers and partners
is knowledge you have that is genuinely
proprietary to you we will respect that
we'll make sure that it's marked as
proprietary to you in the psych
knowledge base no one other than you
will be able to see it if you don't want
them to and it won't be used in
inferences other than for you and so on
however any knowledge which is necessary
in building any applications for you and
with you which is publicly available
General human knowledge is not going to
be proprietary it's going to just become
part of the normal psych knowledge base
and it will be um openly available to
everyone who has access to psych so
that's an important constraint that we
never went back on even when we got push
back from companies which we often did
who wanted to claim that almost
everything they were telling us was
proprietary uh so so there's a line
between very domain specific company
specific stuff and the general knowledge
that comes from that yes or if you
imagine say it's an oil company there
are things which they would expect any
new petroleum engineer they hired to
already know and it's not okay for them
to consider that that is proprietary and
there sometimes a company will say well
we're the first ones to pay you to
represent that in Psych yeah and our
attitude is some polite form of tough uh
the the deal is this take it or leave it
and in a few cases they've left it and
in most cases um they'll they'll see our
point of view and take it because that's
how we've built the psych system by uh
essentially tacking with the funding
wins where people would fund a project
and half of it would be general
knowledge that would stay permanently as
part of Psych and so got it so always
with these Partnerships it's not like a
distraction from the main psych
development it's uh it's a small
distraction it's a small but it's not a
complete one so you're adding to the
knowledge base yes absolutely and we try
to stay away from um projects uh that
would not have that property um so let
me go back and talk talk about open pych
for a second so I've had a lot of
trouble expressing and convincing other
AI
researchers how important it is to use
an expressive representation language
like we do this higher order logic
rather than just using some triple store
knowledge graph type uh
representation um and so as an attempt
to show
them why they needed something more we
said oh well we'll represent this
unimportant projection or Shadow or
subset of
Psych that just happens to be the simple
binary relations the relation argument
one argument two triples and so on um
and then you'll see how much more useful
it is if you had the entire psych system
so so it's all well and good to have the
taxonomic
relations between terms like person and
night and sleep and bed and house and
eyes and and so on
but think about how much more useful it
would be if you also had all the rules
of thumb about those things like people
sleep at night they sleep lying down
they sleep with their eyes closed they
usually sleep in beds in our country uh
they sleep for hours at a time they can
be woken up they don't like being woken
up and so on and so on so it's that
massive amount of knowledge which is not
part of open psych and we thought that
all the researchers would then
immediately immediately say oh my god of
course we need the other 90% that you're
not giving us let's partner and license
psych so that we can use it in our
research but instead what people said is
oh even the bit you've released is so
much better than anything we had we'll
just make with this yeah and so if you
look there are a lot of Robotics
companies today for example which use
open psych as their fundamental ontology
um and in some sense the whole world
missed the point of of open psych and we
were doing it to show people why that's
not really what they wanted and too many
people thought somehow that this was
psych or that this was in fact good
enough for them and they never even
bother coming um coming to us to get
access to the full py but there's
there's two parts to open psych so one
is convincing people on the idea on the
power of this General kind of
representation of knowledge and the
value that you hold in having acquired
that knowledge and built it and continue
to build it and the other is the code
base this is there the code side of it
so my sense of the code base that scor
Psych is operating with I mean it has
the technical debt of the three decades
plus right this is the exact same
problem that Google had to deal with
with the early version of tensorflow
it's still dealing with the that they to
basically break uh compatibility with
the past several times and that's only
over a period of a couple years but they
I think successfully opened up it's very
risky very gutsy move to open up
tensorflow and then pie torch on the
Facebook
side and what you see is there's a magic
place where you can find a community
where you can develop a community that
builds onto on the system without taking
away any of not any but most of the
value so most of the value that Google
has is still a Google most of the value
that Facebook has is still Facebook even
though some of this major machine
learning tooling is released into the
open my question is not so much on the
knowledge which is also a big part of
open pych but all the different kinds of
tool
so the there's the kind of all the kinds
of stuff you can do on the knowledge
graph knowledge base whatever we call it
there's the inference engines so there
there could be some that probably are a
bunch of proprietary stuff you want to
kind of keep secret and there's probably
some stuff you can open up completely
and then let the community build up
enough Community where they develop
stuff on top of it yes there'll be those
Publications and academic work and all
that kind of stuff and uh and also the
tooling of adding to the knowledge base
right like developing you know there's
incredible amount like there's so many
people that are just really good at this
kind of stuff in the open source
community so my question for you is like
have you struggled with this kind of
idea that you have so much value in your
company already you've developed so many
good things you have clients that really
value your relationships and then
there's this dormant Giant Open Source
community that as far as I know you're
not
utilizing is
there there's so many things to say
there but there could be magic moments
where the community builds up large
enough to where the artificial
intelligence field that is currently
99.9% machine learning is dominated by
Machine learning has a face shift
towards like uh or at least in part
towards more like what you might call
symbolic AI this whole place where Psych
is like at the center of
and then as you know that requires a
little bit leap of faith because you're
now surfing and there'll be obviously
competitors that'll pop up and start
making you nervous and all that kind of
stuff so do you think about the space of
open sourcing some parts and not others
how to leverage the community all those
kinds of things that that's a good
question and I think you phrased it the
right way which
is we're constantly struggling with the
question of what to open Source what to
make public what to even publicly talk
about right um and it's there enormous
pluses and minuses to every
alternative and it's very much like
negotiating a very treacherous path part
partly the analogy is like if you slip
uh you could make a fatal mistake give
away something which essentially kills
you or fail give away something which um
failing to give it away um hurts you and
so
on so it is a it is a very tough tough
question usually what we have done with
uh people who approached us to
collaborate on Research is to say we
will make available to you the entire
knowledge base
and executable copies of all of the code
but only very very limited source code
access if you have some idea for how you
might improve something or work with us
on something so let me also get back to
one of the very very first things we
talked about here um which
was separating the question of how could
you get a computer to do this at all
versus how could you get a computer to
do this efficiently enough in real time
and so one of the early lessons we
learned was that we had to separate the
epistemological problem of what should
the system know separate that from the
heuristic problem of how can the system
reason efficiently with what it knows
and so instead of trying to pick one
representation language which was The
Sweet Spot or the best trade-off point
between expressiveness of the language
and efficiency of the language if you
had to pick one knowledge graphs would
probably be associative triples would
probably be about the best you could do
and that's why we started there but um
after a few years we realized that what
we could do is we could split this and
we could have one nice clean
epistemological level language which is
this higher order logic and we could
have one or more
grubby but efficient juristic level
modules that
opportunistically would say oh I can
make progress on what you're trying to
do over here I have a special method
that will contribute a little bit toward
a solution and so of course some subset
of that exactly so by now we have over a
thousand of these heuristic level
modules and they function as a kind of
community of agents and there's one of
them which is a general theorem prover
and the in theory
that's the only one you need but in
practice it always takes so long that
you never want to call on it um you
always want these other agents to very
efficiently reason through it it's sort
of like if you're balancing a chemical
equation you could go back to First
principles but in fact there are
algorithms which are vastly more
efficient or if you're trying to solve a
quadratic equation you could go back to
First principles of
mathematics but it's much better to
Simply recognize that this is a
quadratic equation and apply the
binomial formula and snap you get your
answer right away and so on so think of
these as like a thousand little experts
that are all looking at everything that
pite gets asked and looking at
everything that every other little agent
has contributed almost like notes on a
Blackboard notes on a um a whiteboard um
and making additional notes when they
think they can be helpful and gradually
that community of Agents gets an answer
to your question gets a solution to your
problem and if we ever come up in a
domain application where Psych is
getting the right answer but taking too
long um then what will often do is talk
to one of the human experts and say uh
here's the the set of reasoning steps
that psych went through you can see why
it took it a long time to get the answer
how is it that you were able to answer
that question
in 2 seconds and occasionally you'll get
an expert who just says well I just know
it I just was able to do it or something
um and then you don't talk to them
anymore but sometimes you'll get an
expert who says um well let me
introspect on that yes here is a special
representation we use just for aquous
chemistry equations or here's a special
representation and a special technique
which we can now apply to things in the
special representation and so on and
then you add that as the Thousand and1st
hlis level module and from then on um in
any application if it ever comes up
again it'll be able to contribute and so
on so that that's pretty much one of the
main ways in which um psych has recouped
this lost deficiency um a second
important way is meta reasoning so um
you can speed things up by focusing on
removing Knowledge from the system till
all it has left is like minimal
knowledge needed to but that's the wrong
thing to do right that would be like in
a human extrap painting part of their
brain or something that's really bad so
instead what you want to do is give it
meta level advice tactical and strategic
advice that enables it to reason about
what kind of knowledge is going to be
relevant to this problem what kind of
tactics are going to be good to take in
trying to attack this problem when is it
time to start trying to prove the
negation of this thing because I'm
knocking myself out trying to prove it's
true and maybe it's false and if I just
spend a minute I can see that it's false
or something so so it's like dynamically
pruning the the graph to to only like
based on the particular thing you're
trying to in infer yes and so by now we
have about
150 of these sort of like Breakthrough
ideas that have led to dramatic speedups
in the inference process you know where
one of them was this El HL split and
lots of HL modules another one was using
meta and Meta Meta level reasoning um to
U reason about the the reasoning that's
going on and so on um and you know 150
breakthroughs may sound like a lot but
you know if you divide by 37 years it's
not as
impressive so there's these kind of
heuristic modules that really help
improve the the
inference uh how hard in general is this
because you mentioned higher order logic
you know the the in in the general the
theorem sense it's an intractable very
difficult problem yes so how hard is
this inference problem when we're not
talking
about if we let go of the perfect and
focus on the good I I would say it's
half of the problem in the in the
following empirical sense which is over
the years
about half of our effort maybe 40% of
our effort has been our team of
inference programmers and the other 50
60% has been our ontologists our
ontological Engineers putting in
knowledge so our anthological engineers
in most cases don't even know how to
program they have degrees in things like
philosophy and so on um so it's almost
like the I love that i' love to hang out
with those people actually oh yes it's
wonderful but it's very much like the
Eloy and the warlocks in um HD Well's
time machine so you have the Eloy who
only program in the epistemological
higher order logic language yes and then
you have the morlocks who are like um
under the under the ground uh figuring
out what the Machinery is that will make
this efficiently operate and so on um
and so you know occasionally they'll
toss messages back to each other and so
on but it really is almost this 5050
split between finding clever ways to
recoup efficiency when you have an
expressive language and putting in the
content of what the system needs to know
and yeah both are fascinating to some
degree the entirety of the system as far
as I understand is written in various
variants of lisp so my favorite programm
language is still lisp I don't program
it in much anymore because you know the
world has in majority of its system has
moved on
like everybody respects lisp but many of
the systems are not um written in lisp
anymore but psych as far as I understand
maybe you can correct me there's a bunch
of lisp in it yeah so it's based on a
lisp code that we produced most of the
programming is still going on in a
dialect of lisp and then the for
efficiency reasons that gets
automatically translated into things
like Java um or C nowadays it's almost
all translated into Java because Java
has gotten good enough that uh that's
that's really all we need to do so
translate into Java and then Java is
compiled down to U by code yes okay so
that that's sort of that's a
that um that's a you know it's a process
that probably has to do with the fact
that when py was originally written and
you build up a powerful system like
there is some technical depth you have
to deal with as is the case with most
powerful systems that span years um have
you ever
considered this this would helped me
understand because from my perspective
so much of the value of everything
you've
done with psych and PSY Corp is the is
the is the
knowledge have you ever considered just
like throwing away the code base and
starting from scratch not really
throwing away but sort of moving it to
uh like throwing away that technical
debt starting with a more updated
programming
language is that throwing away a lot of
value or no like what's your sense how
much of the value is in the silly
software engineering aspect and how much
of the value is in the
knowledge so
development of of programs in lisp um
precedes um I think somewhere between a
th000 and 50,000 times faster than
development in any any of what you're
calling um modern or improved computer
languages well there's other functional
languages like you know closure and all
the there there's but I mean I'm with
you I I I like list but I just wonder
how many great programmers there are
there are still like yes so so it is
true when a new inference programmer
comes on board they need to learn um
some of lisp and in fact we have a
subset of lisp which we call cleverly
subel which is really all they need to
learn and so the programming actually
goes on in subel not in full lisp and so
it does not take programmers very long
at all to learn uh subel and that's
something which can then be translated
efficiently into uh Java and for some of
our programmers who are doing say user
interface work then they never have to
even learn subel they just have to learn
apis into the the basic uh psych engine
so you're not necessarily feeling the
burden of like it's it's extremely
efficient there's um that's not a
problem to solve okay right right the
other thing is remember that we're
talking about hiring programmers to do
inference who are programmers interested
in effectively automatic theorem proving
right and so those are people already
predisposed to representing things in
logic and so on and lisp really was the
program in L language um based on logic
that John McCarthy and others um who
developed it basically create took the
the formalisms that Alonzo church and
other um philosophers other logicians um
had come up with and basically said can
we basically make a programming La
language which is effectively logic and
so since we're talking about reasoning
in about Expressions written in this
logic epistemological language and we're
doing operations which are effectively
like theor proving type operations and
so on there's an natural impedance match
between lisp and the the knowledge the
way it's represented so I I guess you
could say it's a perfectly logical
language to use oh yes okay I'm sorry
I'll even let you uh get away with that
like so I'll probably use that in the
future without without credit without
credit but no I think I think the uh the
the point is that the the language you
program in isn't really that important
um it's more that you have to be able to
think in terms of for instance creating
new helpful HL modules and how they'll
work with each other and um looking at
things that are taking a long time and
coming up with new specialized data
structures that will make this efficient
so um let me just give you one very
simple example which is when you have a
transitive relation like larger than
this is larger than that which is larger
than that which is larger than that so
the first thing must be larger than the
the last thing whenever you have a
transitive
relation um if you're not careful if I
ask whether this thing over here is
larger than that thing over here I'll
have to do some kind of graph walk or
theorem proving that might involve like
five or 10 or 20 or 30 steps but if you
store redundantly store the transitive
closure the cleany star of that
transitive relation now you have this
big table but you can always guarantee
that in one single step you can just
look up whether this is larger than that
um and so um we um there are lots of
cases where storage is cheap today and
so by having this extra redundant data
structure we can answer this commonly
occurring type of question very very
efficiently
let me give you one other um analogy
analog of that uh which is something we
call rule macro predicates which is
we'll see this complicated Rule and
we'll notice that things very much like
it syntactically come up again and again
and again so we'll create a whole brand
new relation or predicate or
function that captures that and takes
maybe not two arguments takes maybe
three four five arguments and so on um
and now we have
effectively um converted some
complicated if then
rule that might have to have inference
done on it into some ground Atomic
formula which is just a um the name of a
relation and a few arguments and so on
and so converting commonly occurring
types or schemas of rules into brand new
predicates brand new functions turns out
to enormously speed up the inference
process so so now we've covered about
four of the 150 um good ideas I said
that so that's a nice that's a cool so
that idea in particular is like a nice
compression that turns out to be really
useful yeah that's really interesting I
mean this whole thing is just
fascinating for a philosophical there's
part of me I mean it makes me a little
bit sad because your work is both um
from a computer science perspective
fascinating and the inference engine
from a epistemological philosophical
aspect fascinating but you know it is
also you're running a company and
there's some stuff that has to remain
private and it's sad Well here here's
something that may make you feel better
a little bit better um we're uh We've
formed a not not for-profit company uh
called the knowledge axenization
Institute NEX KX and I have this firm
belief with a lot of empirical evidence
to support it
that the the education that people get
in high schools in colleges in graduate
schools and so on is almost completely
orthogonal to almost completely
irrelevant
to how good they're going to be at
coming up to speed in doing this kind of
ontological engineering and writing
these assertions and rules and so on in
in Psych and so very often we'll
interview candidates who have their PHD
in philosophy who've taught Logic for
years and so on and they're just they're
just awful but the converse is true so
one of the best ontological Engineers we
ever had never graduated high school and
so the purpose of um knowledge a
communization Institute if we can get
some some foundations to help support it
is identify people in the general
population maybe High School dropouts
who have have latent talent for this
sort of thing um offer them effectively
scholarships to train them and then help
place them in companies that need more
trained ontological Engineers some of
which would be working for us but mostly
would be working for partners or
customers or something and if we could
do that that would create an enormous
number of um relatively very high-paying
jobs for people who currently um have no
no way out of some you know um situation
that they're locked into so is there
something you can put into words that
describes somebody who would be great at
anthological engineering so what
characteristics about a person make them
a great at this task this task of
converting the messiness of human
language and knowledge into formal logic
this is very much like what um Alan
touring had to do during World War II
uh in trying to find people to bring to
Bletchley Park where he would publish in
the London Times cryptic crossword
puzzles along with some some innocuous
looking note which essentially said if
if you were able to solve this puzzle in
less than 15 minutes please call this
phone number and so on so um you know or
back when I was young there was uh uh
the practice of having matchbooks where
on the inside of the matchbook
um there would be a can you draw this
you have a career in art a commercial
art if you can copy this uh drawing you
know and so on so um yes the the analog
of that is there a little test that get
to the core of whether you're going to
be good or not so part of it has to do
with uh being able to um make and
appreciate um um and react negatively
appropriately to puns and other jokes so
you have to have a kind of sense of
humor and if you're good at uh telling
jokes and um good at understanding jokes
that's that's one indicator puns yes
like Dad jokes yes well maybe not dad
jokes but real but funny jokes um but uh
I think I'm applying to work as sore no
but um another another is if you're able
to introspect so very often um uh we'll
we we'll give someone a simple question
and we'll say like um um wh wh why why
is this and you know sometimes they'll
just say because it is okay that's a bad
sign but very often they'll be able to
introspect and so on so one of the
questions um I often ask is I'll point
to a sentence with a pronoun in it and
I'll say um you know the referend of
that pronoun is obviously this noun over
here um you know how would you or I or
an AI or a fiveyear old 10-year-old
child know that that pronoun refers to
that noun over here um and um often um
the people who are going to be good at
ontological engineering will give me
some causal explanation or will refer to
some things that are true in the world
so if you imagine a sentence like the
horse was led into the barn while its
head was still wet and so its head
refers to the horse's head but how do
you know that and so some people will
say I just know it some people will say
well the horse was the subject of the
sentence and I'll say okay well what
about the horse was led into the barn
while its roof was still wet now its
roof obviously refers to the barn um and
so then they'll say oh well that's
because it's the closest noun and so so
basically if they try to give me answers
which are based on syntax and grammar
and so on that's a really bad sign but
if they're able to say things like well
horses have heads and barns don't and
barns have roofs and horses don't um
then that's a positive sign that they're
going to be good at this because they
can introspect on what's true in the
world that leads you to know certain
things how fascinating is it that
getting a PhD makes you less capable to
introspect deeply about this oh I'm I
wouldn't I wouldn't go that far I'm not
saying that it makes you less capable
let's just say it's independent of I
don't know of how good people are you're
not saying that I'm saying that there's
a certain kind it's it's it's
interesting that for a lot of people
phds uh sorry philosophy aside that
sometimes education Narrows your
thinking versus expands it yes it's kind
of fascinating and for certain when
you're trying to do ontological
engineering which is essentially teach
our future AI overlords how to reason
deeply about this world and how to
understand it that that requires that
you think deeply about the world so I'll
tell you a sad story about mathcraft
which is why is that not widely used in
schools today um we're not really trying
to make big profit on it or anything
like that the when we've gone to schools
their attitude has been well if a
student spends 20 hours going through
this mathcraft program from start to end
and so on um will it improve their score
on this standardized test more than if
they spent 20 hours just doing mindless
drills of problem after problem after
problem and the answer is well no but
it'll increase their understanding more
and their attitude is well if it doesn't
increase their score on this test um
then that's not you know we're not going
to adopt it that's sad I mean that's
that's a whole that's a whole another
three 4H hour conversation about the
education system but let me ask you let
me go super philosophical as if we
weren't already so in 1950 Alan touring
wrote the paper that formulated the
touring test yes and he opened the paper
with the question can machines think so
what do you think can machines think let
me ask you this question absolutely
machines can think um certainly as well
as humans can think um right we're meat
machines um just because they're not
currently made out of meat is just you
know an engineering solution uh decision
um and so on so um um of course of
course machines can think I think that
there was a lot of
um Damage Done by people U
misunderstanding touring's imitation
game um and um focus on trying to trying
to get a chatbot to uh fool other people
into thinking it was human um and so on
that that's that's not a terrible test
in and of itself but it shouldn't be
your one and only test for intelligence
so do you uh in terms of tests of
intelligence uh you know with the LNA
prize which is a very kind of you want
to say a more strict formulation of the
touring test as originally formulated
and then there's something like Alexa
prize which is more I would say more
interesting formulation of the test
which is like uh ultimately the metric
is how long does a human want to talk to
the AI system so it's like if you the
goal is you want it to be 20 minutes
it's basically not just have a
convincing um conversation but more like
a comp compelling one or a fun one or an
interesting one I me that that seems
like more to the spirit maybe of um of
what uh touring was imagining but what
for you do you think in the space of
tests is a is a good test like what when
you see a system based on psych that
passes that test You' be like damn we've
created something special
here the the test has to be something
involving
depth of reasoning and recursiveness of
reasoning the ability to answer repeated
why questions about the answer you just
gave it's how many wide questions in a
row can you keep answering something
like that and um also just have like a
young curious child and an AI system and
how long will an AI system last before
it wants to quit yes and again that's
not the only test another one has to do
with argumentation in other words here's
a proposition um come up with pro and
con Arguments for it and uh try and give
me convincing arguments on both sides uh
and U so that's that's another important
kind of ability that um the system needs
to be able to exhibit in order to really
be intelligent I think so there are
certain I mean if you look at IBM Watson
and like certain impressive
accomplishments for very specific test
almost like a demo right um there is
some uh like I talked to the guy who led
the the Jeopardy effort and there's some
kind of hard coding heuristics uh tricks
that you try to pull it all together to
make the thing work in the end for this
thing right that seems to be one of the
lessons with AI is
like that's the fastest way to get a
solution that's pretty damn impressive
so so here here's what I would say is
that as impressive as that was it made
some mistakes but more
importantly many of the mistakes it made
were mistakes which no human would have
made yeah um and so part of the the the
new or augmented touring tests would
have to be and the mistakes you make are
ones which humans don't basically look
at and say what yeah uh so for example
there was
a um a question about which 16th century
Italian politician blah blah blah and
Watson said Ronald Reagan so most
Americans would have gotten that
question wrong but they would never have
said Ronald Reagan as an answer right
because you know among the things they
know is that he lived relatively
recently in people don't really live 400
years and you know things like that so
that that's I think um a very important
thing which is um if it's making
mistakes which no normal sane human
would have made then that's a really bad
sign and if it's not making those kinds
of mistakes then that's um a good sign
and I don't think it's any one very very
simple test I think it's all of the
things you mentioned all the things I
mentioned there's really a battery of
tests which together if it passes almost
all of these tests it would be hard to
argue that it's not intelligent and if
it fails some several of these tests
it's really hard to argue that it really
understands what it's doing that it
really is generally intelligent so to
pass all of those tests you know we've
talked a lot about psych and knowledge
and
reasoning do you think this AI system
would need to have some other humanlike
elements for example a body or a
physical manifestation in this world and
uh another one which seems to be
fundamental to The Human Experience is
consciousness the subjective experience
of what it's like to actually be you do
you think he needs those to be able to
pass all those tests and to achieve
general intelligence it's a good
question I think in the case of a body
uh no I know there are a lot of people
like Penrose who would have disagreed
with me and so there um and and others
but no I don't think it needs to have a
body in order to be intelligent I think
that it needs to be able to talk
about uh having a body and having
Sensations and having emotions and so on
it doesn't actually have to have all of
that U but it has to understand it in
the same way that Helen Keller was
perfectly intelligent and able to talk
about um colors and sounds and um shapes
and um and so on um even though she
didn't directly experience all the same
things that the rest of us do
so knowledge of it and being able to
correctly um make use of that um is
certainly an important facility but
actually having a body um if you believe
that that's just a kind of religious or
mystical belief you can't really um
argue for or against it I suppose um
it's it's just something that some
people that some people believe what
about the
like an extension of the body which is
consciousness I mean like it feels like
something to be here sure but you know
what what does that really mean it's
like well if I talk to you you say
things which make me believe that you
are conscious yeah um I know that I'm
conscious but that's you know you're
just taking my word for it now um but in
the same sense Psych is conscious in
that same sense already where of course
it understands it's a computer program
it understands where and when it's
running it understands who's talking to
it it understands what its task is what
its goals are what its current problem
is that it's working on it understands
how long it's spent on things what it's
tried it understands what it's done in
the past um and so on um and uh you know
if if we want to call that Consciousness
then yes Psych is already conscious but
I don't think that I would ascribe
anything um mystical to that again some
people would but I would I would say
that you know other than other than our
personal experience of Consciousness um
we're just treating everyone else in the
world um uh so to speak um at their word
about being conscious and so if um if a
computer program if an AI is able to um
exhibit um all the same kinds of um
response as you would expect of a
conscious entity um then um you know
doesn't doesn't it deserve the label of
Consciousness just as much so there's
another burden that comes with this
whole intellig
thing that humans got is um the
extinguishing of the light of
Consciousness which is uh kind of
realizing that we're going to be dead
someday and uh there's a bunch of
philosophers like Ernest Becker who kind
of think that this realization of
mortality and then fear sometimes they
call it Terror of of of mortality is one
of the creative forces
behind Human Condition like it's the
thing that drives us do you think it's
important for an AI system you know when
psych proposed that it's one it's not
human and it's one of the moderators of
his
contents um you know there's another
question it could ask which is like it
kind of knows that humans are mortal am
I
mortal and I think one really important
uh thing that's possible when you're
conscious is to fear the extinguishing
of that Consciousness the fear
mortality do you think that's useful for
intelligence thinking like I might die
and I really don't want to die I I don't
think so I think it may help um some
humans to be um better people it may
help some humans to be more creative and
so on I don't think it's necessary um
for AI is to believe that they have
limited lifespans and therefore they
should make the most of their behavior
maybe eventually um the answer to that
and my answer to that will change but as
of now I would say that that's almost
like a a frill or a side effect uh that
is not in fact if you look at most
humans most humans um ignore the fact
that they're going to die most of the
time uh so well but that's like this
goes to the white space between the
words so what Ernest Becca argues is
that that ignoring is we're living in an
illusion that we constructed on the
foundation of this Terror so we're
Escape Life as We Know It pursuing
things creating things love everything
we can think of that's beautiful about
humanity is is just trying to escape
this realization that we're going to die
one day that's his that's his idea and I
think I don't know if I I
100% believe in this but there's it
certainly Rhymes it seems like to me
like it rhymes with the truth yeah I I
think that for some people um that's
going to be a more powerful Factor than
others clearly Doug is talking about
Russians and I think that
uh so I'm Russian so it clearly it
infiltrates all of Russian literature
and and AI doesn't have to have uh fear
of death
as a motivating force in that we can
build in motivation so we can build in
the motivation
of obeying users and making users happy
and making others happy and and so on
and that can substitute for this sort of
personal fear of death that sometimes
leads to bursts of creativity in in
humans I don't know I think like I think
AI really needs to understand death
deeply in order to be able to drive a
car for example I I think there's just
some like there no I I really disagree I
think it needs to understand the value
of human life especially the value of
human life to other humans the um and
understand that certain things are more
important than other things so it has to
have a lot of knowledge about ethics and
U morality and so on but some of it is
so messy that impossible to encode for
example there disagree so if there's a
person dying right in front of us most
human beings would help that person but
they would not apply that same ethics to
everybody else in the world this is the
tragedy of how difficult it is to be a
doctor because they know when they help
a dying child they know that the money
they're spending on this child cannot
possibly be spent on every other child
that's dying and that's that's a very
difficult to encode decision now U
Perhaps Perhaps it is perhaps it could
be formalized oh but I mean you're
talking about autonomous vehicles right
so autonomous vehicles are going to have
to make those decisions um all the time
of um what is the chance of this bad
event happening um how bad is that
compared to this chance of that bad
event happening and so on and you know
when potential accident is about to
happen is it worth taking this risk if I
have to make a choice which of these two
cars am I going to hit and why and see I
was thinking about a very different
Choice when I'm talking about the
mortality which is just observing uh
Manhattan style driving I think that
humans as an effective driver needs to
threaten pedestrians lives a lot there's
a dance I've watched pesters a lot I
worked on this problem and it seems like
the the if I could summarize the problem
of a pedestrian Crossing is the car with
this movement is saying I'm going to
kill you and the pedestrians is
saying maybe and then they decide and
they say no I don't think you you have
the guts to kill me and you walk and
they walk in front and they look away
and there's that dance the the the The
Pedestrian this a social contract that
The Pedestrian trust that once they're
in front of the car and the car is
sufficiently from a physics persp
perspective able to stop they're going
to stop but the car also has to threaten
that pedestrian is like I'm late for
work so you're being kind of an asshole
by Crossing in front of me but life and
death is in like is part of the
calculation here and it's that that
equation is being solved millions of
times a day yes very effectively that
game theory whatever whatever that
formulation is absolutely I just I don't
know if it's as simple as some
formalizable Game Theory problem it it
could very well be in the case of
driving and in the case of most of uh
Human Society I I don't know but it uh
yeah you might be right that this sort
of the fear of death is just one of the
quirks of uh like the way our brains
have evolved but it's not it's not a
necessary feature of a of intelligence
drivers certainly are always doing this
kind of estimate even if it's
unconscious subconscious of
what are the chances of various bad
outcomes happening like for instance um
if I don't wait for this pedestrian or
something like that and um what is the
downside to me going to be in terms of
um you know time wasted talking to the
police or um you know getting sent to
jail or you know things like that and so
um and there's also emotion like people
in their cars tend to get uh
irrationally Angry that's that's that's
dangerous but you know think think about
this is all part of why I think that
autonomous vehicles um truly autonomous
vehicles are farther out than um than
most people do because um there is this
enormous level of complexity which goes
beyond uh mechanically controlling the
car um and um I I can see the autonomous
vehicles as a kind of metaphorical and
literal accident waiting to happen um
and not just because of the their um
overall um um incurring versus
preventing accidents and so on but just
because of the um almost um voracious
appetite people have for um
um bad bad stories about powerful
companies and Powerful entities when
when I was um at a coincidentally
Japanese fifth generation Computing
system cont in
1987 uh while I happened to be there um
there was a worker at an auto plant who
was despondent and committed suicide by
climbing under the safety chains and so
on getting stamped to death by a machine
and instead of being a small story that
said despondent worker commits suicide
it was front page news that effectively
said Robot kills worker because the
public is just waiting for stories about
like he AI kills photogenic family of
five type stories and even if you could
show that Nationwide uh this system
saved more lives than it cost and saved
more injuries um prevented more injuries
than it caused and so on um the media
the public the government is
just coiled and ready to pounce on
stories where in fact it failed even if
they're relatively few
yeah it's so fascinating to watch us
humans
resisting The Cutting Edge of Science
and Technology and almost like hoping
for it to fail and constantly and you
know this just happens over and over and
over throughout history well or even if
we're not hoping for it to fail we're
we're fascinated by it and in terms of
what we find interesting um the one in a
thousand failures much more interesting
than the
999 U boring
successes so once we build build an AGI
system say psych is some part of uh some
part of it and um say it's very possible
that you would be one of the first
people that can sit down in the room
let's say with her and have a
conversation what would you ask her what
would you talk about looking at all of
the
content out there on the web and so on
what are
the what are the some possible solutions
to big problems that the world has that
people haven't really thought of before
that are not
being properly or at least
adequately pursued uh what are some
novel solutions that you can think of
that we haven't that might work and that
might be worth considering so that is a
a damn good question g g given that the
AGI is going to be somewhat different
from Human
intelligence it's still going to make
some mistakes that we wouldn't make but
it's also possibly going to notice some
blind spots we have and um I would I
would love it as a test of is it really
um On a par with our intelligen is can
it help spot some of the blind spots
that we
have so the two-part question of can you
help identify what are the big problems
in the world and two what are some novel
solutions to those problems that are not
being talked about by
anyone yeah and some of those may become
um you know infeasible or reprehensible
or something but some of them might be
actually great things to look at you
know if you if you go back and look at
some of the most powerful discoveries
that have been made U like relativity
and um superconductivity and so on a lot
of them were cases
where someone took seriously the idea
that there might actually be um a a
nonobvious answer to a to a question so
in Einstein's case it was um yeah the
lorence transformation is known um
nobody believes that it's actually the
way reality Works what if it were the re
way that reality actually worked so you
know a lot of people don't realize he
didn't actually work out that equation
he just sort of took it seriously um or
in the case of superconductivity you
have this V equals IR equation where R
is resistance and so on and um um it was
being mapped at lower and lower
temperatures but everyone thought that
was just bump on a log research um to
show that V equals I always held um and
then when some graduate student um got
to a slightly lower temperature and
showed that resistance suddenly dropped
off everyone just assumed that they did
it wrong and they and it was only a
little while later that they realized it
was um um it was actually a new
phenomenon or in the case of um um the
um H pylori bacteria causing stomach
ulcers where everyone thought that
stress and stomach acid caused ulcers um
and when a doctor um in um Australia um
claimed it was actually a bacterial
infection um he couldn't get anyone
seriously to listen to him and he had to
um ultimately inject himself with the
bacteria to show that he suddenly
developed a life-threatening ulcer um in
order to get other doctors to seriously
consider that so there all sorts of
things where um humans are locked into
paradigms what Thomas cun called
paradigms and we can't get out of them
very easily so a lot of AI is locked
into the deep learning machine learning
Paradigm right now um and um almost all
of of us and almost all Sciences are
locked into current paradigms and you
know cun's point was um pretty much you
have to wait for people to die um in
order for the new generation to escape
those paradigms and I think that one of
the things that would change that sad
reality is if we had trusted agis that
could help take a step back and question
some of the paradigms that we're
currently locked into yeah it would
Excel at the Paradigm shifts in in human
science and
progress you've lived a very interesting
life where you thought about Big Ideas
and you stuck with
them can you give advice to young people
today somebody in high school somebody
undergrad about um career about
life I'd say you can make a
difference but in order to make a
difference you're going to have to have
the courage to follow through with ideas
which other people might not immediately
understand or um support you have to
realize that if you
make
some some plan that's going to take an
extended period of time to carry out
don't be afraid of that that's true of
um physical training of your body that's
true
of um uh learning uh um some profession
that's also true of innovation that some
Innovations are not great ideas you can
write down on a napkin and um become an
instant success if you turn out to be
right some of them
are paths you have to follow but
remember that you're mortal remember
that you have a limited number of decade
sized bets to make with your life and
you should make each one of them count
and that's true in personal
relationships that's true in career
choice that's true in uh making
discoveries and so on and if you follow
the path of least resistance you'll find
that you're optimizing for um short
periods of time and before you know it
you turn around and long periods of time
have gone by without you ever really
making a difference in the world you
know there's when you look I mean the
field that I really love is artificial
intelligence and there's not many
projects there's not many little flames
of hope that have been carried out for
many years for decades and pych
represents one of them and uh I mean
that in itself is just a really
inspiring thing so I'm I'm I'm deeply
grateful that you would be carrying that
flame for so many years and I think
that's an inspiration to you young
people that said you said life is finite
and we talked about mortality as a
feature of AI do you think about your
own mortality are you afraid of death um
sure I would be crazy if I weren't and
um as I get older I'm now um over 70 so
as I get older um it's more on my mind
especially as acquaintances and friends
and especially um mentors um one by one
are dying so I can't avoid thinking
about mortality and I think that um the
the good news from the point of view and
the rest of the world is that that adds
impetus to uh my need to succeed in a
small number of years in the future
because I'm you have a deadline exactly
I'm not going to have another 37 years
to continue working on this so we really
do want likey to make an impact in the
world um commercially physically
metaphysic basically um in the next
small number of years 2 3 5 years not 2
three five decades anymore and so this
is really driving me toward uh this this
sort of commercialization and increasing
increasingly widespread application of
Psych whereas before um I felt that I
could just sort of sit back roll my eyes
wait till the world caught up and now I
don't feel that way anyway anymore I
feel like I need to put in some effort
to make the world aware of what we have
and what it can do and the good news
from your point of view is that that's
that's why I'm sitting here you're going
to be more
productive uh I love it and if I can
help in any way I would love to from a
from a you know from a programmer
perspective I I love uh especially these
days just contributing in small and big
ways so if there's any open sourcing
from the MIT side and the research I I
would love to help but when you know
bigger than psych like I said it's that
little flame that you're carrying of
artificial intelligence the big dream um
is there what do you hope your legacy
is H that's a good
question that people think of me as one
of the Pioneers or inventors
of the AI that is ubiquitous and that
they take for granted um and so on much
much the way that today we look back on
the the pioneers of electricity or the
pioneers of um similar types of uh
Technologies and so on as um you know
it's hard to imagine what life would be
like if uh these people hadn't done what
they um what they did so that that's one
thing that I'd like to be remembered as
another is that so the Creator one of
The orig Originators of this
gigantic
knowledge store and acquisition system
that is likely to be at the
center of whatever this future AI thing
will look like yes exactly and I'd also
like to be remembered as someone
who wasn't afraid to spend several
decades on a project in a time when um
all when almost all of the
other forces institutional forces and
Commercial forces are incenting people
to go for short-term rewards and a lot
of people gave up a lot of people that
dreamt the same dream as you gave up yes
and you didn't
yes I mean uh Doug it's it's truly an
honor this was a long time coming I I um
a lot of people bring up your work uh
specifically and more broadly
philosophically of this is the dream of
artificial intelligence this is likely a
part of the future we're so sort of
focused on machine learning applications
all that kind of stuff today but it
seems like the ideas that pych carries
forward uh is something that would be at
the center of this problem they're all
trying to solve which is the the problem
of intelligence emotional and and uh
otherwise so thank you so much it's such
a Hu huge honor that you would talk to
me and spend your valuable time with me
today thanks for talking thanks Lex it's
been great thanks for listening to this
conversation with Doug lennet to support
this podcast please check out our
sponsors in the description and now let
me leave you some words from Mark Twain
about the nature of
truth if you tell the truth you don't
have to remember anything thank you for
listening and hope to see you next time