Kind: captions
Language: en
the following is a conversation with
michael i jordan a professor at berkeley
and one of the most influential people
in the history of machine learning
statistics and artificial intelligence
he has been cited over 170 thousand
times and has mentored many of the
world-class researchers defining the
field of ai today including andrew eng
zubin garamani bentascar and yoshio
banjo
all this to me is as impressive as the
over 32 000 points and the six nba
championships of the michael j jordan of
basketball fame
there's a non-zero probability that i
talked to the other michael jordan given
my connection to and love the chicago
bulls of the 90s but if i had to pick
one i'm going with the michael jordan of
statistics and computer science or as
john le calls him the miles davis
of machine learning
in his blog post titled artificial
intelligence the revolution hasn't
happened yet michael argues for
broadening the scope or the artificial
intelligence field
in many ways the underlying spirit of
this podcast is the same
to see artificial intelligence as a
deeply human endeavor to not only
engineer algorithms and robots but to
understand and empower human beings at
all levels of abstractions from the
individual to our civilization as a
whole
this is the artificial intelligence
podcast if you enjoy it subscribe on
youtube give it five stars at apple
podcast support it on patreon or simply
connect with me on twitter at lex
friedman spelled friday
as usual i'll do one or two minutes of
ads now and never any ads in the middle
that can break the flow of the
conversation i hope that works for you
and doesn't hurt the listening
experience
this show is presented by cash app the
number one finance app in the app store
when you get it use code lex podcast
cash app unless you send money to
friends buy bitcoin and invest in the
stock market with as little as one
dollar
since cash app does fractional share
trading let me mention that the order
execution algorithm that works behind
the scenes to create the abstraction of
the fractional orders is to me an
algorithmic marvel
so big props for the cash app engineers
for solving a hard problem that in the
end provides an easy interface that
takes a step up to the next layer of
abstraction over the stock market making
trading more accessible for new
investors and diversification much
easier
so once again if you get cash app from
the app store or google play and use the
code lex podcast you'll get ten dollars
and cash app will also donate ten
dollars the first one of my favorite
organizations that is helping to advance
robotics and stem education for young
people around the world
and now here's my conversation with
michael i jordan
given that you're one of the greats in
the field of ai machine learning
computer science and so on
you're
trivially called the michael jordan of
machine learning
although
as you know you were born first so
technically mj is the michael i jordan
of basketball but
anyway my my favorite is yan la
calling you the miles davis of machine
learning
because as he says you reinvent yourself
periodically and sometimes leave
fans scratching their heads after you
change direction so
can you put
at first your historian hat on and give
a history of computer science and ai as
you saw it as you experienced it
including
the four generations of ai successes
that i've seen you talk about
sure
yeah first of all i much prefer yon's
metaphor um miles davis is uh was
a real explorer in jazz and um he had a
coherent story so i think i have one and
but it's not just the one you live it's
the one you think about later what a
good historian does is they
look back and they revisit
um i think what happening right now is
not ai
that was an intellectual aspiration um
that's still alive today is an
aspiration but i think this is akin to
the development of chemical engineering
from chemistry or electrical engineering
from from electromagnetism
so if you go back to the 30s or 40s
there wasn't yet chemical engineering
there was chemistry there was fluid flow
there was mechanics and so on
but people
pretty clearly viewed
interesting goals try to build factories
that you make chemicals products and do
it viably safely
make good ones do it at scale
so people started to try to do that of
course and some factories worked some
didn't you know some were not viable
some exploded but in parallel developed
a whole field called chemical
engineering
right and chemical engineering is a
field it's it's no no bones about it it
has theoretical aspects to it it has
practical aspects it's not just
engineering quote unquote it's the real
thing real concepts are needed same
thing with electrical engineering you
know there was maxwell's equations which
in some sense were everything you know
about electromagnetism
but you needed to figure out how to
build circuits how to build modules how
to put them together how to bring
electricity from one point to another
safely and so on so forth so whole field
is developed called electrical
engineering
all right i think that's what's
happening right now is that we have we
have a proto
field which is statistics compute more
the theoretical side of the algorithmic
side of computer science that was enough
to start to build things but what things
systems that bring value to human beings
and use human data and mix in human
decisions the engineering side of that
is all ad hoc
that's what's emerging in fact if you
want to call machine learning a field i
think that's what it is that's a proto
form of engineering based on statistical
and computational ideas of previous
generations but do you think there's
something deeper about ai in his dreams
and aspirations as compared to chemical
engineering and electrical engineering
well the dreams and aspirations maybe
but those are from those are 500 years
from now i think that that's like the
greek sitting there and saying it would
be neat to get to the moon someday right
um i hate we have no clue how the brain
does computation uh we're just a
clueless we're like we're even worse
than the greeks
almost anything interesting
uh scientifically of our era can you
linger on that just for a moment because
you stand
not completely unique but a little bit
unique in that in the clarity of that
can you can you elaborate your intuition
of
why we
like where we stand in our understanding
of the human brain and a lot of people
say you know scientists say we're not
very far in understanding human brain
but you're like you're saying we're in
the dark here well i know i'm not unique
i don't even think in the clarity but if
you talk to real neuroscientists that
really study real synapses or real
neurons they agree
they agree it's a hundred year hundreds
of year tasks and they're building it up
slowly surely
what the signal is there is not clear we
think we have all of our metaphors we
think it's electrical
maybe it's chemical it's a whole soup
it's ions and proteins and it's a cell
and that's even around like a single
synapse if you look at a
electromicrograph of a single synapse
it's a it's a city of its own
and that's one little thing on a
dendritic tree which is extremely
complicated you know electrochemical
thing
and it's doing these spikes and voltages
have been flying around and then
proteins are taking that and taking it
down into the dna and who knows what
so it is the problem of the next few
centuries it is fantastic
but we have our metaphors about it is it
an economic device is it like the immune
system or is it like a layered you know
set of copy you know arithmetic
computations what we have all these
metaphors and they're fun
but that's not real science
per se there is neuroscience that's not
neuroscience all right that that's
that's like the greek speculating about
how to get to the moon fun
right and i think that i like to say
this fairly strongly because i think a
lot of young people think we're on the
verge
because a lot of people who don't talk
about it clearly let it be understood
that yes we kind of this is brain
inspired we're kind of close you know
breakthroughs are on the horizon
and unscrupulous people sometimes who
need money for their labs
um as i'm saying scrupulous but people
will oversell um i need money from a lab
i'm gonna i'm studying here you know
computational neuroscience um i'm gonna
oversell it and so there's been too much
of that so i'll step into the slight the
gray area between metaphor and
engineering with uh i'm not sure if
you're familiar with
brain computer interfaces
so
a company like elon musk has neural link
that's working on
putting electrodes into the brain and
trying to be able to read both read and
send electrical signals just as you said
even
the basic mechanism of communication
in the brain is not something we
understand but do you hope
without understanding the fundamental
principles of how the brain works we'll
be able to do something interesting
at that gray area of metaphor it's not
my area so i i hope in the sense like
anybody else hopes for some interesting
things to happen from research
i would expect more something like
alzheimer's will get figured out from
modern neuroscience
that you know a lot of there's a lot of
human suffering based on brain disease
and we throw things like lithium at the
brain it kind of works no one has a clue
why
that's not quite true but you know
mostly we don't know and that's even
just about the biochemistry of the brain
and how it leads to mood swings and so
on how thought emerges from that we just
we were really really completely dim so
that you might want to hook up
electrodes and try to do some signal
processing on that and try to find
patterns
fine you know by all means go for it
it's just not scientific at this point
it's just it's so it's like kind of
sitting in a satellite and watching the
emissions from a city and trying to
affirm things about the micro economy
even though you don't have microeconomic
concepts i mean it's really that kind of
thing and so yes can you find some
signals that do something interesting or
useful can you control a cursor
or mouse with your brain yeah absolutely
you know and then i can imagine business
models based on that and even you know
medical applications of that but from
there to understanding algorithms that
allow us to really tie in deeply to from
the brain to computer you know i just no
i don't agree with elon musk i don't
think that's even that's not for our
generation it's not even for the century
so
just uh in hopes of getting you to dream
uh you've mentioned kolmogorov and
touring might pop up
do you think that there might be
breakthroughs they'll get you to sit
back in five ten years and say
wow
oh i'm sure there will be but i don't
think that there'll be demos that
impress me
i don't think that having a computer
call a restaurant and pretend to be a
human
is a breakthrough
and people you know some people present
it as such
it's imitating human intelligence it's
even putting coughs
in the thing to make a bit of a pr stunt
and so fine the world runs on those
things too
and i don't want to diminish all the
hard work and engineering that goes
behind things like that and and the
ultimate value to the human race but
that's not scientific understanding
and and i know the people who work on
these things they are after a scientific
understanding you know in the meantime
they've got to kind of you know the
trains got to run and they got miles to
feed and they got things to do
and there's nothing wrong with all that
i would call that though just
engineering and i want to distinguish
that between an engineering field like
electoral internet chemical injury that
originally that originally emerged that
had real principles and you really knew
what you're doing and you had a little
scientific understanding maybe not even
complete
so it became more predictable and it was
really gave value to human life because
it was understood
and and so we have to we don't want to
muddle too much these waters of you know
what we're able to do versus what we
really can do in a way that's going to
impress the next so i don't i don't need
to be wowed but i i think that someone
comes along in 20 years
a younger person who's absorbed all the
uh the technology and for them to be
wowed i think they have to be more
deeply impressed a young kulmogorov
would not be wowed by some of the stunts
that you see right now coming from the
big companies the demos but do you think
the breakthroughs from kolmogorov
would be and give this question a chance
do you think they'll be in the
scientific fundamental principles arena
or do you think it's possible to have
fundamental breakthroughs in engineering
meaning
you know i would say some of the things
that elon musk is working with spacex
and then others sort of trying to
revolutionize the fundamentals of
engineering of manufacturing of
of saying here's a problem we know how
to do a demo of and actually taking it
to scale yeah so so there's going to be
all kinds of breakthroughs i just don't
like that terminology i'm a scientist
and i work on things day in and day out
and things move along and eventually say
wow something happened but it's i don't
like that language very much
also i don't like to prize theoretical
breakthroughs over practical ones um i
tend to be more of a theoretician and i
think there's lots to do in that arena
right now um and so i wouldn't point to
the komo gurus i might point to the
edisons of the era and maybe musk is a
bit more like that but um you know musk
god bless him also
we'll say things about ai that he knows
very little about and and he doesn't
know what he's he he is you know leads
people astray when he talks about things
he doesn't know anything about trying to
program a computer to understand natural
language to be involved in a dialogue
we're having right now that can happen
in our lifetime you could fake it you
can mimic sort of take old sentences
that humans use and retread them
with the deep understanding of language
now it's not going to happen and so from
that you know i hope you can perceive
that the deeper yet deeper kind of
aspects and intelligence are not going
to happen now will there be
breakthroughs you know i think that
google
was a breakthrough i think amazon's a
breakthrough you know i think uber is a
breakthrough you know that bring value
to human beings at scale in new brand
new ways based on data flows and and so
on a lot of these things are slightly
broken because there's not a kind of a
engineering field that takes economic
value in context of data and and at you
know planetary scale and and worries
about all the externalities the privacy
you know we don't have that field so we
don't think these things through very
well
but i see that is emerging and that will
be cons that will you know looking back
from 100 years that will be constituted
a breakthrough in this era just like
electrical engineering was a
breakthrough in the early part of the
last century and chemical injury was a
breakthrough so the scale the markets
that you talk about and we'll get to
will be seen as sort of breakthrough and
we're in the very early days of really
doing interesting stuff there and we'll
get to that but it's just taking a quick
step back
can you give
uh we kind of threw off the historian
hat
i mean you briefly said that uh in
the history of ai kind of mimics the
history of chemical engineering but i
keep saying machine learning you keep
want to say ai just to let you know i
don't you know i i'd resist that i don't
think this is about ai really was john
mccarthy as almost a philosopher
saying wouldn't it be cool if we could
put thought in a computer if we could
mimic the human
capability to think or put intelligence
in in some sense into a computer
that's an interesting philosophical
question and he wanted to make it more
than philosophy he wanted to actually
write down logical formula and
algorithms that would do that
and that is a perfectly valid reasonable
thing to do that's not what's happening
in this era
right so so the reason i keep saying ai
actually and i'd love to hear what you
think about it machine learning has uh
has a very particular set of methods and
tools
maybe your version of it is that mine
doesn't no it does it's very very open
it does optimization it does sampling it
does so systems that learn is what
machine learning is systems that learn
and make decisions and make decisions so
what is pattern recognition and from you
know finding patterns it's all about
making decisions in real worlds and
having close feedback loops so something
like symbolic ai expert systems reading
systems knowledge based representation
all of those kinds of things search does
that
neighbor fit into what you think of as
machine learning so i don't even like
the word but you know i think that with
the field you're talking about is all
about making large collections of
decisions under uncertainty by large
collections of entities yes right and
there are principles for that at that
scale you don't have to say the
principles are for a single entity
that's making decisions a single agent
or a single human it really immediately
goes to the network of decisions is a
good award for that or no no there's no
good words for any of this that's kind
of part of the problem um so we can
continue the conversation use ai for all
that i just want to kind of raise our
flag here
that this is not about we don't know
what intelligence is
and real intelligence we don't know much
about abstraction and reasoning at the
level of humans we don't have a clue
we're not trying to build that because
we don't have a clue eventually it may
emerge they'll make i don't know if
there'll be breakthroughs but eventually
we'll start to get glimmers of that it's
not what's happening right now though
okay we're taking data we're trying to
make good decisions based on that we're
trying to scale we're trying to do it
economically viably we're trying to
build markets we're trying to keep value
at that scale
and
aspects of this will look intelligent it
will look computers were so dumb
before they will seem more intelligent
we will use that buzz word of
intelligence so we can use it in that
sense but you know so machine learning
you can scope it narrowly is just
learning from data and pattern
recognition
but whatever i when i talk about these
topics i maybe data science is another
word you could throw in the mix
it really is important that the
decisions are
as part of it it's consequential
decisions in the real world are i have a
medical operation am i going to drive
down the street you know
things that where their scarcity things
that impact other human beings or other
you know the environment and so on how
do i do that based on data how do i do
that adaptively how i use computers to
help those kind of things go forward
whatever you want to call that so let's
call it ai let's agree to call it ai but
it's um
let's let's not say that what the goal
of that is is intelligence the goal of
that is really good working systems at
planetary scale we've never seen before
so reclaiming the word ai from the
dartmouth conference from many decades
ago of the dream of humanity i don't
want to reclaim it i want a new word i
think it was a bad choice i mean i i you
know i if you read one of my little
things um the history was basically that
uh mccarthy needed a new name because
cybernetics already existed and he
didn't like you know no one really liked
norbert wiener you know ravina was kind
of an island to himself and he felt that
he had encompassed all this and in some
sense he did you look at the language of
cybernetics it was everything we're
talking about it was control theory and
single processing and some notions of
intelligence and close feedback loops
and data it was all there it's just not
a word that lived on partly because of
maybe the personalities but mccarthy
needed a new word to say i'm different
from you i'm not part of your your show
i got my own
invented this word um and again as a
kind of a
thinking forward about the movies that
would be made about it
uh it was a great choice but thinking
forward about creating a sober academic
and real world discipline it was a
terrible choice because it led to
promises that are not true that we
understand we understand artificial
perhaps but we don't understand
intelligence it's a small tangent
because you're one of the great
personalities of machine learning
whatever the heck you call the field
the
do you think science progresses by
personalities or by the fundamental
principles and theories and
research that's outside of personality
both and i wouldn't say there should be
one kind of personality i have mine and
i have my preferences and
i have a kind of network around me that
feeds me and and some of them agree with
me and some disagree but all kinds of
personalities are needed
um right now i think the personality
that's a little too exuberant a little
bit too ready to promise the moon is a
little bit too much in ascendance
um and i do i do think that that's
there's some good to that it certainly
attracts lots of young people to our
field but a lot of those people come in
with strong misconceptions and they have
to then unlearn those and then find
something you know
to do
um and so i think there's just got to be
some multiple voices and there's i
didn't i wasn't hearing enough of the
more sober voice
so uh as a continuation of a fun tangent
and speaking of vibrant personalities
what would you say is the most
interesting disagreement you have with
yon lacoon
so john's an old friend and i just say
that i i don't think we disagree about
very much really
he and i both kind of have a let's build
that kind of mentality and does it work
kind of mentality and uh kind of
concrete um we both speak french and we
speak french more together and we have
we have a lot a lot in common
um
and so you know if one wanted to
highlight a a disagreement it's not
really a fundamental one i think it's
just kind of where we're emphasizing um
jan has uh emphasized pattern
recognition and
uh has emphasized prediction
all right so you know um and it's
interesting to try to take that as far
as you can if you could do perfect
prediction what would that give you kind
of as a thought experiment
um
and um i think that's
way too limited um
we cannot do perfect prediction we will
never have the data sets allow me to
figure out what you're about ready to do
what question you're going to ask next i
have no clue i will never know such
things moreover most of us find
ourselves during the day in all kinds of
situations we had no anticipation of
that are kind of very very
novel in various ways
and in that moment we want to think
through what we want and also there's
going to be market forces acting on us
i'd like to go down that street but now
it's full because there's a crane in the
street i gotta i gotta think about that
i gotta think about what i might really
want here and i gotta sort of think
about how much it cost me to do this
action versus this action i got to think
about the risks involved you know a lot
of our current pattern recognition and
prediction systems don't do any risk
evaluations they have no error bars
right i got to think about other
people's decisions around me i got to
think about a collection of my decisions
even just thinking about like a medical
treatment you know i'm not going to take
the prediction of a neural net
about my health about something
consequential am i about to have a heart
attack because some number is over 0.7
even if you had all the data in the
world never been collected about heart
attacks
better than any doctor ever had
i'm not going to trust the output of
that neural net to predict my heart
attack i'm going to want to ask what if
questions around that i'm going to want
to look at some us or other possible
data i didn't have causal things i'm
going to have a dialogue with a doctor
about things we didn't think about we
gathered the data
you know it i could go on and on i hope
you can see and i don't i think that if
you say predictions everything that that
you're missing all of this stuff
and so prediction plus decision making
is everything but both of them are
equally important and so the field has
emphasized prediction yan rightly so has
seen how powerful that is
but at the cost of people not being
aware that decision making is where the
rubber really hits the road where human
lives are at stake where risks are being
taken where you got to gather more data
you got to think about the arab bars you
got to think about the consequences of
your decisions on others you about the
economy around your decisions blah blah
blah
i'm not the only one working on those
but we're a smaller tribe and right now
we're not the the one that people talk
about the most um
but you know if you go out in the real
world in industry um you know at amazon
i'd say half the people there are
working on decision making and the other
half are doing you know the pattern
recognition it's important and the words
of pattern recognition and prediction i
think the distinction there
not to linger on words but the
distinction there is more a constrained
sort of in the lab data set versus
decision making is talking about
consequential decisions in the real
world under the messiness and the
uncertainty of the real world
and just
the whole of it the whole mess of it
that actually touches human beings and
scale like you said market forces that's
the that's the distinction yeah it helps
add those that perspective that broader
perspective you're right i totally agree
uh on the other hand if you're a real
prediction person of course you want it
to be in the real world you want to
predict real world events i'm just
saying that's not possible with just
data sets uh that it has to be in the
context of you know uh strategic things
that someone's doing data they might
gather things they could have gathered
the reasoning process around data it's
not just taking data and making
predictions based on the data so one of
the the things that you're working on
i'm sure there's others working on it
but i don't hear often
it talked about especially in the
clarity that you talk about it and i
think it's both the most exciting and
the most concerning area
of ai in terms of decision making so
you've talked about ai systems that help
make decisions that scale in a
distributed way millions billions
decisions
it's sort of markets of decisions can
you as a starting point sort of give an
example of a system that you think about
when you're thinking about these kinds
of systems
uh yeah so first of all you're
absolutely getting into some territory
which i will be beyond my expertise and
and there are lots of things that are
going to be very non-obvious to think
about just like just
again i like to think about history a
little bit but think about put yourself
back in the 60s there was kind of a
banking system that wasn't computerized
really there was there was database
theory emerging and database people had
to think about how do i actually not
just move data around but actual money
and have it be you know
valid and have transactions that atms
happen that are actually you know all
valid and
so on so forth so that's the kind of
issues you get into when you start to
get serious about sort of things like
this
i like to think about as kind of almost
a thought experiment to help me think uh
something simpler which is
a music market
and uh because there is
the first door there is no music market
in the world right now or in the con in
our country for sure uh there are uh
something called things called record
companies and they make money uh and
they prop up a few um really good
musicians and make them superstars and
they all make huge amounts of money
but there's a long tale of huge numbers
of people that make lots and lots of
really good music that is actually
listened to by more people than the
famous people
um
they are not in a market they cannot
have a career they do not make money the
creators the creators the creators the
so-called influencers or whatever that
diminishes who they are right so there
are people who make extremely good music
especially in the hip-hop or latin world
these days uh they do it on their laptop
that's what they do
on the weekend and they have
another job during the week and they put
it up on soundcloud or other sites
eventually it gets streamed it down gets
turned into bits it's not economically
valuable the information is lost it gets
put up there people stream it
you walk around in a big city you see
people with headphones all you know
especially young kids listen to music
all the time if you look at the data
none of them very little the music they
listen to is the famous people's music
and none of it's old music it's all the
latest stuff
but the people who made that latest
stuff are like some 16 year old
somewhere who will never make a career
out of this who will never make money of
course there will be a few counter
examples the record companies
incentivize to pick out a few and
highlight them
long story short there's a missing
market there there is not a consumer
producer relationship at the level of
the actual creative acts
um the pipelines and spotifys of the
world that take this stuff and stream it
along they make money off of
subscriptions or advertising and those
things they're making the money all
right and then they will offer bits and
pieces of it to a few people again to
highlight that you know they're
they simulate a market anyway a real
market would be if you're a creator of
music that you actually are somebody
who's good enough that people want to
listen to you you should have the data
available to you there should be a
dashboard showing a map of the united
states so in last week here's all the
places your songs were listened to it
should be transparent
um vettable so that if someone in down
in providence sees that you're being
listened to ten thousand times in
providence that they know that's real
data you know it's real data they will
have you come give a show down there
they will broadcast to the people who've
been listening to you that you're coming
if you do this right you could you could
you know go down there make twenty
thousand dollars you do that three times
a year you start to have a career so in
this sense ai creates jobs it's not
about taking away human jobs it's
creating new jobs because it creates a
new market
once you've created a market you've now
connected up producers and consumers you
know the new person who's making the
music can say to someone who comes to
their shows a lot hey i'll play your
daughter's wedding for ten thousand
dollars
you'll say eight thousand they'll say
nine thousand um then you again you you
can now get an income up to a hundred
thousand dollars you're not going to be
a millionaire
all right and and now even think about
really the value of music is in these
personal connections even so much so
that um
a young kid wants to wear a t-shirt with
their favorite musician's signature on
it right so if they listen to the music
on the internet the internet should be
able to provide them with a button as
they push and the merchandise arrives
the next day
we can do that right and now why should
we do that well because the kid who
bought the shirt will be happy but more
the person who made the music will get
the money
there's no advertising needed right
so you could create markets between
personal consumers take five percent cut
your company will be perfectly uh sound
it'll go forward into the future and it
will create new markets and that raises
human happiness um
now this seems like it was easy just
create this dashboard kind of create
some connections and all that but you
know if you think about uber or whatever
you think about the challenges in the
real world of doing things like this and
there are actually new principles going
to be needed you're trying to create a
new kind of two-way market at a
different scale that's ever been done
before there's going to be
you know
unwanted aspects of the market there'll
be bad people they'll be you know
the data will get used in the wrong ways
you know it'll fail in some ways it
won't deliver value you have to think
that through just like anyone who like
ran a big auction or you know ran a big
matching service in economics will think
these things through
and so that maybe doesn't get at all the
huge issues that can arise when you
start to create markets but it starts
for at least for me solidify my thoughts
and allow me to move forward in my own
thinking
yeah so i talked to how to research at
spotify actually i think their long-term
goal they've said is to uh have at least
1 million creators make a
make a comfortable living
putting on spotify so
in
and
i think you articulate a really nice
vision
of uh
the world and the digital and the cyber
space of markets what what do you think
companies like spotify or youtube or
netflix can do
to create such
markets is it an ai problem is it an
interface problem so interface design
is it uh
some other kind of it was an economics
problem who should they hire to solve
these problems well part of it's not
just top down so the silicon valley has
its attitude that they know how to do it
they will create the system just like
google did with the search box that will
be so good that they'll just everyone
will adopt that right
um it's not it's it's everything you
said but really i think missing the kind
of culture
all right so it's literally that 16 year
old who's able to create the songs you
don't create that as a silicon valley
entity you don't hire them per se
right you have to create an ecosystem in
which they are wanted and that they
belong right so you have to have some
cultural credibility to do things like
this you know netflix to their credit
wanted some of that sort of credibility
they created shows you know content they
call it content it's such a terrible
word but it's called it's culture
right and so with movies you can kind of
go give a large sum of money to somebody
graduate from the usc film school
it's a whole thing of its own but it's
kind of like rich white people's thing
to do
you know and
you know american culture has not been
so much about rich white people it's
been about all the immigrants all the
africans who came and brought
that culture and those those rhythms and
and that that to to this world and
created this whole new thing you know
american culture and and
so companies can't artificially create
that they can't just say hey we're here
we're going to buy it up
you got a partner right and um so but
anyway you know not to integrate these
companies are all trying and they should
and they they are i'm sure they're
asking these questions and some of them
are even making an effort but it is it
is partly a respect the culture as you
were as a technology person you got to
blend your technology with cultural with
cultural uh you know meaning how much of
a role do you think the algorithm
machine learning has in connecting the
consumer to the creator
sort of uh the recommender system aspect
of this yeah it's a great question i
think pretty high recommend you know um
there's no magic in the algorithms but a
good recommender system is way better
than a bad recommender system and uh
recommender systems was a billion dollar
industry back even you know 10 20 years
ago um
and it continues to be extremely
important going forward what's your
favorite recommender system just so we
can put something well just historically
i was one of the you know when i first
went to amazon and you know i first
didn't like amazon because they put the
book people out of business or the
library you know the local book sellers
went out of business
um i've come to accept that they're you
know there probably are more books being
selled now and more people reading them
than ever before
and then local books stores are coming
back so you know that's how economics
sometimes work you go up and you go down
but anyway when i finally started going
there and i bought a few books i was
really pleased to see another few books
being recommended to me that i never
would have thought of
and i bought a bunch of them so they
obviously had a good business model but
i learned things and i still to this day
kind of browse using that service
um
and i think lots of people get a lot you
know they're that that is a good aspect
of a recommendation system i'm learning
from my peers in a in an indirect way
and their algorithms are not meant to
have them impose what we what we learn
it really is trying to find out what's
in the data uh it doesn't work so well
for other kind of entities but that's
just the complexity of human life like
shirts you know i'm not gonna get
recommendations on shirts and uh but
that's that's that's interesting
uh if you try to recommend um
uh restaurants it's it's it's it's it's
hard it's hard to do it at scale and and
um
but uh a blend of recommendation systems
with other
economic ideas matchings and so on is
really really still very open
research-wise and there's new companies
that could emerge that do that well
what what do you think is going to the
messy difficult land of say politics and
things like that that youtube and
twitter have to deal with in terms of
recommendation systems
being able to suggest
i think facebook just launched facebook
news
so they're having
recommend the kind of news that are most
likely for you to be interesting
you think this is this ai solvable
again whatever term want to use do you
think it's a solvable problem for
machines or is it a deeply human problem
that's unsolvable uh so i don't even
think about it that level i think that
what's broken with some of these
companies it's all monetization by
advertising
they're not at least facebook let's i
want to critique them they didn't really
try to connect a producer and a consumer
in an economic way right no one wants to
pay for anything
and so they all you know starting with
google and facebook they went back to
the playbook of you know the the
television companies back in the day no
one wanted to pay for this signal they
will pay for the tv box but not for the
signal at least back in the day and so
advertising kind of filled that gap but
advertising was new and interesting and
it somehow didn't take over our lives
quite
right
fast forward google provides a service
that people don't want to pay for
um
and so somewhat surprising in the 90s
they made end up making huge amounts
they cornered the advertising market it
didn't seem like that was going to
happen at least to me um these little
things on the right hand side of the
screen just did not seem all that
economically interesting but that
companies had maybe no other choice the
tv market was going away and billboards
and so on um so they've they got it
and i think that sadly that uh google
just has me it was doing so well with
that and making such right they didn't
think much more about how wait a minute
is there a producer consumer
relationship to be set up here not just
uh between us and the advertisers market
to be created is there an actual market
between the producer and consumer
they're the producers the person who
created that video clip the person that
made that website the person who could
make more such things the person who
could adjust it and as a function of
demand the person on the other side
who's asking for different kinds of
things you know so you see glimmers of
that now there's influencers and there's
kind of a little glimmering of a market
but it should have been done 20 years
ago it should have been thought about it
should have been created in parallel
with the advertising ecosystem
and then facebook inherited that and i
think they also didn't think very much
about that
so fast forward and now they are making
huge amounts of money off of advertising
and the news thing and all these clicks
is just is feeding the advertising it's
all connected up to the advertiser
so you want more people to click on
certain things because that money flows
to you facebook
you're very much incentivized to do that
and when you start to find it's breaking
people are telling you well we're
getting into some troubles you try to
adjust it with your smart ai algorithms
right and figure out what are bad clicks
though maybe shouldn't be click-through
rate it should be something i find that
pretty much hopeless
it does get into all the complexity in
life and you can try to fix it you
should
but you could also fix the whole
business model
and the business model is that really
what are are there some human producers
and consumers out there is there some
economic value to be liberated by
connecting them directly is it such that
it's so valuable that uh people are
willing to pay for it
all right and micro payments like smart
micro but even have to be micro so i i
like the example suppose i'm going next
week i'm going to india never been to
india before right
uh i have a couple days in in mumbai um
i have no idea what to do there right
and i could go on the web right now and
search it's going to be kind of hopeless
i'm not going to find you know um i'll
have lots of advertisers in my face
right what i really want to do is
broadcast to the world that i am going
to mumbai and have someone on the other
side of a market
look at me and and there's a
recommendation system there so i'm not
looking at all possible people coming to
mumbai they're looking at the people who
are relevant to them so someone my age
group someone who kind of knows me in
some level
i give up a little privacy by that but
i'm happy because what i'm going to get
back is this person's going to make a
little video for me or they're going to
write a little two-page paper on here's
the cool things that you want to do and
move by this week especially
right i'm going to look at that i'm not
going to pay a micro payment i'm going
to pay you know 100 or whatever for that
it's real value it's like journalism um
as i'm not a subscription it's that i'm
gonna pay that person in that moment
company's gonna take five percent of
that and that person has now got it it's
a gig economy if you will but you know
done for in you know thinking about a
little bit behind youtube there was
actually people who could make more of
those things
if they were connected to a market they
would make more of those things
independently you have to tell them what
to do you don't have to incentivize them
any other way
um and so yeah these companies i don't
think have thought long long and hard
about that so i do distinguish on you
know facebook on the one side who just
not thought about these things at all i
think uh thinking that ai will fix
everything uh and amazon thinks about
them all the time because they were
already out in the real world they were
delivering packages people's doors they
were they were worried about a market
they were worrying about sellers and you
know they worry and some things they do
are great some things maybe not so great
but you know
they're in that business model and then
i'd say google sort of hover somewhere
in between i don't i don't think for a
long long time they they got it
i think they probably see that youtube
is more pregnant with possibility than
than than they might have thought and
that they're probably heading that
direction um
but uh you know silicon valley's been
dominated by the google facebook kind of
mentality and the subscription and
advertising and that is that's the core
problem
right the fake news actually rides on
top of that because it means that you're
monetizing with click-through rate and
that is the core problem you got to
remove that so advertisement
if you're going to linger on that i mean
that's an interesting thesis i don't
know if everyone really deeply thinks
about that
so
you're right the thought is
the advertising model is the only thing
we have the only thing we'll ever have
so we have to fix
we have to build algorithms that
despite that business model
you know find the better angels of our
nature and do good by society and by the
individual but you think we can
slowly you think first of all
there's a difference between should and
could so you're saying we should slowly
move away from the advertising model and
have a direct connection between the
consumer and the creator
the the question i also have is
can we because the advertising model is
so successful now in terms of just
making a huge amount of money and
therefore being able to build a big
company that provides has really smart
people working that create a good
service do you think it's possible and
just to clarify you think we should move
away well i think we should yeah but uh
we is you know me so society yeah will
the companies um i mean so first of all
full disclosure i'm doing a day a week
at amazon because i kind of want to
learn more about how they do things so
you know i'm not speaking for amazon in
any way but um you know i did go there
because i actually believe they get a
little bit of this are trying to create
these markets and they don't really use
advertising is not a crucial part of it
that's a good question so it has become
not crucial but it's become more and
more present if you go to amazon website
and you know without revealing too many
deep secrets about amazon i can tell you
that you know a lot of people company
question this and there's a huge
questioning going on
you do not want a world where there's
zero advertising that actually is a bad
world okay so here's a way to think
about it you're
a company that like amazon is trying to
bring products to customers all right
and the customer at any given you want
to buy a vacuum cleaner say you want to
know what's available for me and you
know it's not gonna be that obvious you
have to do a little bit of work at it
the recommendation system will sort of
help
all right but now suppose this other
person over here has just made the world
you know they spent a huge amount of
energy they had a great idea they made a
great vacuum cleaner they know they they
really did it they nailed it it's an mit
you know whiz kid that made a great new
vacuum cleaner all right it's not going
to be in the recommendation system no
one will know about it the algorithms
will not find it and ai will not fix
that okay at all
right how do you allow that vacuum
cleaner to start to get in front of
people
be sold well advertising and here what
advertising is it's a signal
that you're you believe in your product
enough that you're willing to pay some
real money for it and to me as a
consumer i look at that signal i say
well first of all i know these are not
just cheap little ads because we have
now right now i know that you know these
are super cheap you know pennies
uh if i see an ad where it's actually i
know the company is only doing a few of
these and they're making you know real
money is kind of flowing and i see an ad
i may pay more attention to it and i
actually might want that because i see
hey that guy spent money on his vacuum
cleaner
or maybe there's something good there so
i will look at it and and so that's part
of the overall information flow in a
good market uh so advertising has a role
um but the problem is of course that
that signal is now completely gone
because it just you know dominar by
these tiny little things that add up to
big money for the company
you know so i i think it will just i
think it will change because the
societies just don't you know stick with
things that annoy a lot of people and
advertising currently annoys people more
than it provides information
and i think that at google probably is
smart enough to figure out that this is
a dead this is a bad model even though
it's a hard huge amount of money and
they'll have to figure out how to pull
it away from it and slowly and i'm sure
the ceo there will figure it out but um
they need to do it and uh they need to
so if you reduce advertising not to zero
but you reduce it at the same time you
bring up
producer consumer actual real value
being delivered so real money is being
paid and they take a five percent cut
that five percent could start to get big
enough to cancel out the lost revenue
from the the kind of the poor kind of
advertising and i think that a good
company will will do that we'll realize
that
um
and they're com you know facebook you
know again god bless them they they
bring you know grandmother's uh you know
uh they bring children's pictures into
grandmother's lives it's fantastic
um
but they need to think of a new business
model and and they that's that's the
core problem there um until they start
to connect producer consumer i think
they will just just continue to make
money and then buy the next social
network company and then buy the next
one and the innovation level will not be
high and the health the health issues
will not go away
so i apologize that we kind of return to
words i don't think
the exact terms matter but in sort of
defensive advertisement
don't you think the kind of direct
connection between consumer
and creator producer
is the best
like the
is what advertisement strives to do
right so that is the best advertisement
is literally now
facebook is listening to our
conversation and heard that you're going
to india
and we'll be able to actually start
automatically for you making these
connections and start giving this offer
so like
i apologize if it's just a matter of
terms but just to draw a distinction is
it possible to make advertisements just
better and better and better
algorithmically to where it actually
becomes a connection almost address
that's a good question so let's
component all that push first of all i i
what we just talked about i was
defending advertising okay so i was
defending it as a way to get signals
into a market that don't come any other
way especially algorithmically it's a
sign that someone spent money on it it's
a sign they think it's valuable
and if i think that if other things
someone else thinks it's valuable and if
i trust other people i might be willing
to listen
i don't trust that facebook though is
who's an intermediary between this
i don't think they
care about me
okay i don't think they do and i find it
creepy that they know i'm going to india
next week because of our conversation
why do you think that can we so what
can you just
put your pr hat on
why do you think you find facebook uh
creepy and not trust them as as do
majority of the population
so they're out of the silicon valley
companies i saw like
not approval rate but there's there's
ranking of how much people trust
companies and facebook is in in the
gutter in the gutter including people
inside of facebook
so what
uh what do you attribute that to because
when i come on you don't find it creepy
that right now we're talking i might
walk out on the street right now that
some unknown person who i don't know
kind of comes up to me and says i hear
you going to india
i mean that's not even facebook that's
just a if
i want transparency in human society i
want to have if you know something about
me there's actually some reason you know
something about me that's something that
if i look at it later and audit it kind
of i approve
you know something about me because
you care in some way there's a caring
relationship even or an economic one or
something not just that you're someone
who could exploit it in ways i don't
know about or care about or or i'm
troubled by or or whatever and we're in
a world right now where that happens way
too much
and that facebook knows things about a
lot of people and could exploit it and
does exploit it at times
i think most people do find that creepy
it's not for them it's not it's not that
it's facebook that's not doing it
because they care about them right in
any real sense and they shouldn't they
should not be a big brother caring about
us that is not the role of a company
like that why not wait not the big
brother part but the sharing the trust
thing i mean don't those companies
just to linger because a lot of
companies have a lot of information
about us i would argue that there's
companies like microsoft that has more
information about us than facebook does
and yet we trust microsoft more well
microsoft is pivoting microsoft you know
under satya nadella has decided this is
really important we don't want to do
creepy things we really want people to
trust us to actually only use
information in ways that they really
would approve of that we don't decide
right
and um
i'm just kind of adding that the health
the health of a market is that uh when i
connect to someone who produces a
consumer it's not just a random producer
consumer it's people who see each other
they don't like each other but they
sense that if they transact
some happiness will go up on both sides
if a company helps me to do that
and moments that i choose
of my choosing
then fine so and also think about the
difference between you know browsing
versus buying right there are moments in
my life i just want to buy you know a
gadget or something i need something for
that moment i need some ammonia for my
house or something because i got a
problem a spill
i want to just go in i don't want to be
advertised at that moment i don't want
to be led down very dire you know that's
annoying i want to just go and have it
extremely easy to do what i want
um
other moments i might say no it's like
today i'm going to the shopping mall i
want to walk around and see things and
see people and be exposed to stuff so i
want control over that though i don't
want the company's algorithms to decide
for me
right and i think that's the thing we
it's a total loss of control if facebook
thinks they should take the control from
us of deciding when we want to have
certain kinds of information when we
don't what information that is how much
it relates to what they know about us
that we didn't really want them to know
about us
they're not i don't want them to be
helping me in that way i don't want them
to be helping them but they decide well
they have control over um um
what i want and when i totally agree so
facebook by the way i have this
optimistic thing where i think facebook
has the kind of personal information
about us that could create a beautiful
thing so i i'm really optimistic
of what facebook could do uh it's not
what it's doing but what it could do so
i don't see that i think that optimism
is misplaced
because there's not a bit you have to
have a business model behind these
things yes create a beautiful thing is
really let's be let's be clear it's
about
something that people would value and
and i don't think they have that
business model and i don't think they
they will suddenly discover it by what
you know have a long hot shower
i disagree i disagree in terms of uh you
can discover a lot of amazing things in
the shower so if i didn't say that i
said they won't come they won't they
won't do it but in the shower i think a
lot of other people will discover it i
think that this guy so i should also uh
full disclosure there's a company called
united masters which i'm on their board
and they've created this music market
yes they have a hundred thousand artists
now signed on and they've done things
like gone to the nba and the nba the
music you find behind me the eclipse
right now is their music right that's a
company that had the right business
model in mind from the get-go right
executed on that and and from day one
there was value brought to so here you
have a kid who made some songs who
suddenly their songs are on the nba
website
right
that that's real economic value to
people and uh so
you know
so you and i differ on the optimism of
being able to
sort of uh um
change the direction of the titanic
right
so i yeah
i'm older than you so i think titanic's
crash
got it but uh so and just to elaborate
because i totally agree with you and i
just want to know how difficult you
think this problem is of so for example
i um
i want to read some news
and i would there's a lot of times in
the day where something makes me either
smile or think in a way where i like
consciously think this really gave me
value like i sometimes listen to uh the
daily podcast in the new york times
way better than the new york times
themselves by the way for people
listening that's like real journalism is
happening for some reason in the podcast
space it doesn't make sense to me but
often i'll listen to it 20 minutes and i
i would be willing to pay for that like
five dollars ten dollars for that
experience absolutely and
how difficult that's kind of what you're
getting at is that little transaction
how difficult is it to create a
frictionless system like uber has for
example
for other things what's your intuition
there
uh so i first of all i pay little bits
of money to you know to say there's
something called courts that does
financial things i like medium as a site
i don't pay there but um i would you had
a great post on medium i would have
loved to pay you a dollar and but i
wouldn't want it i wouldn't have wanted
it per se because um
there should be also sites where that's
not actually the goal the goal is to
actually have a broadcast channel that i
monetize in some other way if i chose to
i mean i could now people know about it
i could i'm not doing it but um that's
fine with me there also the musicians
who are making all this music i don't
think the right model is that you pay a
little subscription fee to them all
right because because people can copy
the bits too easily and it's just not
that somewhere the value is the value is
that a connection was made between real
human beings then you can follow up on
that
right and create yet more value so no i
think um
there's a lot of open questions here hot
open questions but also yeah i do want
good recommendation systems that
recommend cool stuff to me and but it's
pretty hard right i don't like them to
recommend stuff just based on my
browsing history i don't like that based
on stuff they know about me quote quote
what's unknown about me is the most
interesting so this is the this is the
really interesting question we may
disagree maybe not
i think that
i love recommender systems and i want to
give them everything about me
in a way that i trust yeah but you but
you don't because so for example this
morning i clicked on i you know i was
pretty sleepy this morning um i clicked
on a story about the queen of england
yes right i do not give a damn about the
queen of england i really do not but it
was clickbait it kind of looked funny
and i had to say what the heck are they
talking about i don't want to have my
life you know heading that direction now
that's in my browsing history
the system in any reasonable system
we'll think about
history
right but but you're saying all the
trace all the digital exhaust or
whatever that's been kind of the models
if you collect all this stuff
you're gonna figure all of us out well
if you're trying to figure out like kind
of one person like trump or something
maybe you could figure him out but if
you're trying to figure out you know 500
million people
you know no way no way do you think so
no i do i think so i think we are humans
are just amazingly rich and complicated
every one of us has our little quirks
everyone else has our little things that
could intrigue us that we don't even
know and will intrigue us and there's no
sign of it in our past but by god there
it comes and you know you fall in love
with it and i don't want a company
trying to figure that out for me and
anticipate that okay well i want them to
provide a forum a market a place that i
kind of go and by hook or by crook this
happens
you know i i'm walking down the street
and i hear some chilean music being
played and i never knew i like chili
music but wow
so there is that side and i want them to
provide a limited but you know
interesting place to go right and so
don't try to use your ai to kind of
you know figure me out and then put me
in a world where you figured me out you
know no create huge spaces for human
beings where our creativity and our
style will be enriched and come forward
and it'll be a lot more transparency i
won't have people randomly anonymously
putting comments up and especially based
on stuff they know about me facts that
you know
we are so broken right now if you're you
know especially if you're a celebrity
but you know it's about anybody that
uh anonymous people are hurting lots and
lots of people right now and that's part
of this thing that silicon valley is
thinking that you know just collect all
this information and use it in a great
way so no i i'm i'm not i'm not a
pessimism i'm very much an optimist by
nature but i think that's just been the
wrong path for the whole technology to
take
be more limited create let humans rise
up don't don't try to replace them
that's the ai mantra don't try to
anticipate them don't try to predict
them
because you're you're not good at you're
not going to do those things you're
going to make things worse okay so
right now just give this a chance uh
right now the recommender systems are
the creepy people in the shadow watching
your every move
so they're looking at traces of you
they're not directly interacting with
you
sort of the your close friends and
family the way they know you is by
having conversation by actually having
interactions back and forth do you think
there's a place
for recommender systems sort of to step
because you you just emphasize the value
of human to human connection but yeah
just give it a chance ai human
connection is there a role for an ass
system to have conversations with you
in terms of
to try to figure out what kind of music
you like not by just watching what
you're listening but actually having a
conversation natural language or
otherwise yeah no i'm i'm so i'm not
against it i just want to push back
against them maybe you're saying you
have options for facebook so there i
think it's misplaced but but um i think
that this one pending facebook yeah now
so
good for you
um go for it that's a hard spot to be
yeah no good human interaction like on
our daily the context around me in my
own home is something that i don't want
some big company to know about at all
but i would be more than happy to have
technology help me with it
which kind of technology well you know
just alexa amazon well a good alexa's
done right i think alex is a research
platform right now more than anything
else but alexa done right you know could
do things like i i leave the water
running in my garden and i say hey alex
so the waters are in my garden um and
even have alexa figure out that that
means when my wife comes home that she
should be told about that
that's a little bit of a reasoning i
call that ai and by any kind of stretch
it's a little bit of reasoning and it
actually kind of makes my life a little
easier and better and
you know i don't i wouldn't call this a
wow moment but i kind of think that
overall rises human happiness up to have
that kind of thing um but not when
you're lonely alexa knowing loneliness
no no there i don't want to let you that
be feel intrusive and i and i don't want
just the designer of the system to kind
of work all this out i really want to
have a lot of control and i want
transparency and control
and if the company can stand up and give
me that in the context of new technology
i think they're good first of all be way
more successful than our current
generation and like i said i was
mentioning microsoft earlier i really
think they're pivoting to kind of be the
trusted old uncle but you know i think
that they get that this is the way to go
that if you let people find technology
empowers them to have more control and
have and have control not just over
privacy but over this rich set of
interactions um that that people gonna
like that a lot more and that's that's
the right business model going forward
what does control over privacy look like
do you think you should be able to just
view all the data that no it's much more
than that i mean first of all it should
be an individual decision some people
don't want privacy they want their whole
life out there other people's want it um
privacy is not a zero one it's not a
legal thing it's not just about which
data is available which is not
um
i like to
recall to people that you know a couple
hundred years ago everyone there was not
really big cities everyone lived down
the countryside and villages
um and in villages everybody knew
everything about you very you didn't
have any privacy is that bad are we
better off now well you know arguably no
because what did you get for that loss
of at least certain kinds of privacy um
well uh people helped each other if they
because they know everything about you
they know something's bad's happening
they will help you with that right and
now you live in a big city no one knows
their mouth you get no help
um
so uh it kind of depends the answer i
want certain people who i trust and
there should be relationships i should
kind of manage all those but who knows
what about me i should have some agency
there it shouldn't i shouldn't be a
drift in a sea of technology where i
have no idea i don't want to go reading
things and checking boxes
so i don't know how to do this and i'm
not a privacy researcher per se i just i
recognize the vast complexity of this
it's not just technology it's not just
legal scholars meeting technologists
there's got to be kind of whole layers
around it and so i when i allude to this
emerging engineering field this is a big
part of it
um like when electrical engineering come
came i'm not wasn't around in the time
but
you just didn't plug electricity you
know into walls and it all kind of
worked you don't have to have like
underwriter's laboratory that reassured
you that that plug's not going to burn
up your house
and that that machine will do this and
that and everything there'll be whole
people who can install things there'll
be people who can watch the installers
there'll be a whole layers you know an
onion of these kind of things
and for things as deeply
interesting as privacy which is his
least essential electricity
um that that's gonna take decades to
kind of work out but it's gonna require
a lot of new structures that we don't
have right now so it's kind of hard to
talk about it
and you're saying there's a lot of money
to be made if you get it right so
absolutely a lot of money to be made and
all these things that provide human
services and people recognize them as
useful parts
of their lives uh so yeah
um
so yeah the dialect sometimes goes from
the exuberant technologists
to the no technology is good kind of and
that's you know in our public discourse
you know in newspapers you see too much
of this kind of thing and and the sober
discussions in the middle which are the
challenging ones to have are where we
need to be having our conversations and
you know there's not actually there's
not many forum forum for those
um you know there's that's that's kind
of what i would look for maybe i could
go and i could read a comment section of
something and it would actually be this
kind of dialogue going back and forth
you don't see much of this right which
is why actually there's a resurgence of
podcasts out of all because good people
are really hungry for conversation but
their technology is not helping much so
comment sections of anything including
youtube yeah
is not
hurting i'm not hurting yeah
and you think technically speaking
it's possible to help i don't know the
answers but it's it's a it's a less
anonymity a little more locality um you
know worlds that you kind of enter in
and you trust the people there in those
worlds so that when you start having a
discussion you know not only is that
people not gonna hurt you but it's not
gonna be a total waste your time because
there's a lot of wasting of time that
you know a lot of us i i pulled out of
facebook early on because it was clearly
going to waste a lot of my time even
though there was some value
um and so yeah worlds that are somehow
you enter in you know what you're
getting and it's kind of appeals to you
might new things might happen but you
kind of have some some trust in that
world
and there's some deep interesting
complex psychological aspects around
anonymity uh how that changes human
behavior indeed quite dark and quite
dark yeah i think a lot of us are
especially
those of us who really love the advent
of technology i loved social networks
when it came out i was just i didn't see
any negatives there at all
but then i started uh seeing comment
sections i think it was maybe you know
cnn or something
and i started going wow this this
darkness i just did not know about and
um and our technology is now amplifying
it
so sorry for the big philosophical
question but on that topic do you think
human beings because you've also out of
all things had a foot in psychology too
the do you think human beings are
fundamentally good
like all of us have
good intent that could be mined or
is it
depending on context and environment
everybody could be evil
thought my answer is fundamentally good
um but fundamentally limited all of us
have very you know blinkers on we don't
see the other person's pain that easily
we don't see the other person's point of
view that easily we're very much in our
own head in our own world
and on my good days i think that
technology could open us up to you know
more perspectives and more less
blinkered and more understanding you
know a lot of wars in human history
happen because of just ignorance they
didn't they they thought the other
person was doing this well other person
wasn't doing this and we have huge
amounts of that um but in my lifetime
i've not seen technology really help in
that way yet and i do i do i do believe
in that but
you know no i think fundamentally humans
are good the people suffer people have
grievances because you have grudges and
those things cause them to do things
they probably wouldn't want
they regret it often um
so no i i i think it's a
you know part of the progress of
technology is to indeed allow it to be a
little easier to be the real good person
you actually are
well but
do you think individual human life or
society could be modeled as an
optimization problem
um not the way i think typically i mean
that's your time one of the most complex
phenomena in the whole you know in all
ways individual human life for society
as a whole both both i mean individual
human life is amazingly complex and um
so uh you know optimization is kind of
just one branch of mathematics that
talks about certain kind of things and
uh it just feels way too limited for the
complexity of
such things what properties of
optimization problems do you think so do
you think most interesting problems that
could be solved through optimization
uh what kind of properties does that
surface have
non-convexity convexity linearity all
those kinds of things saddle points well
so optimization's just one piece of
mathematics you know there's like you
just even in our era we're aware that
say sampling
um is coming up with examples of
something um coming up with a
description what's sampling
well you they you can if you're a kind
of a certain kind of mathematician you
can try to blend them and make them seem
to be sort of the same thing but
optimization is roughly speaking trying
to
uh find a point that um a single point
that is the
optimum of a criterion function of some
kind
um and sampling is trying to from that
same surface
treat that as a distribution or density
and find prop points that have high
density
so um i i want the entire distribution
and the sampling paradigm and i want the
um you know the the single point that's
the best point in the par in the sample
in the uh optimization paradigm now if
you were optimizing in the space of
probability measures the output of that
could be a whole probability
distribution so you can start to make
these things the same but in mathematics
if you go too high up that kind of
abstraction arc you start to lose
the uh you know the ability to do the
interesting theorems so you kind of
don't try to you don't try to overly
over abstract
so as a small tangent what kind of world
view do you find more appealing one that
is
deterministic or stochastic
well that's easy i mean i'm a
statistician you know the world is
highly stochastic wait i don't know
what's going to happen in the next five
minutes right because you're going to
ask what we're going to do
massive uncertainty yeah you know
massive uncertainty and so the best i
can do is have come rough sense or
probability distribution on things and
somehow use that in my reasoning about
what to do now
so
how does the distributed at scale when
you have multi-agent systems
look like so optimization
can optimize sort of it makes a lot more
sense
sort of uh at least from from a robotics
perspective for a single robot for a
single agent trying to optimize some
objective function
when you start to enter the real world
this game theoretic concept starts
popping up
and
that
how do you see optimization in this
because you've talked about markets in a
scale what does that look like do you
see this optimization do you see it as
sampling do you see like how how should
you modify these all blend together um
and a system designer thinking about how
to build an incentivized system will
have a blend of all these things so you
know a particle in a potential well is
optimizing a function called lagrangian
right the particle doesn't know that
there's no algorithm running
that does that it just happens it's so
it's a description mathematically of
something that helps us understand as
analysts what's happening
right and so the same will happen when
we talk about you know mixtures of
humans and computers and markets and so
on so forth there'll be certain
principles that allow us to understand
what's happening and whether or not the
actual algorithms are being used by any
sense it's not clear
now at some point i may have set up a
multi-agent or market kind of system
and i'm now thinking about an individual
agent in that system
and they're asked to do some tasks and
they're incentivized in some way they
get certain signals and they they have
some utility maybe what they will do at
that point is they just won't know the
answer they may have to optimize to find
an answer okay so an autism could be
embedded inside of an overall market
you know and game theory is is very very
broad
it is often studied very narrowly for
certain kinds of problems
but it's roughly speaking this is just
the i don't know what you're going to do
so i kind of anticipate that a little
bit and you anticipate what i'm
anticipating and we kind of go back and
forth in our own minds we run kind of
thought experiments
you talked about this interesting point
in terms of game theory you know most
optimization
problems really hate saddle points maybe
you can describe what saddle points are
but
i've heard you kind of mentioned that
there's a there's a branch of
optimization you could try to explicitly
look for saddle points that's a good
thing oh not optimization that's just
game theory that that's so uh there's
all kinds of different equilibria in
game theory and some of them are highly
explanatory behavior they're not
attempting to be algorithmic they're
just trying to say
if you happen to be at this equilibrium
you would see certain kind of behavior
and we see that in real life that's what
an economist wants to do especially
behavioral economist
um
uh
in in continuous uh
differential game theory you're in
continuous spaces a um some of the
simplest equilibria are saddle points a
nash equilibrium is a saddle point it's
a special kind of salon point
so
classically in game theory you were
trying to find nash equilibria and
algorithmic games here you're trying to
find algorithms that would find them and
so you're trying to find saddle points i
mean so that's literally what you're
trying to do
um but you know any economist knows that
nash equilibria have their limitations
they are definitely not that explanatory
in many situations they're not what you
really want um there's other kind of
equilibria and there's names associated
with these because they came from
history with certain people working on
them but there will be new ones emerging
so you know one example is a stackelberg
equilibrium so you know nash you and i
are both playing this game against each
other or for each other maybe it's
cooperative and we're both going to
think it through and then we're going to
decide and we're going to off you know
do our thing simultaneously
you know in a stackelberg no i'm going
to be the first mover i'm going to make
a move you're going to look at my move
and then you're going to make yours now
since i know you're going to look at my
move i anticipate what you're going to
do and so i don't do something stupid
but and but then i know that you were
also anticipating me so we're kind of
going back and so far am i but there is
then a first mover
thing
and so there's a those are different
equilibria all right and uh
so just mathematically yeah these things
have certain topologies certain shapes
they're like southwest and
algorithmically or dynamically how do
you move towards them how do you move
away from things
um you know so some of these questions
have answers they've been studied others
do not and especially if it becomes
stochastic
especially if there's large numbers of
decentralized things there's just uh you
know young people getting in this field
who kind of think it's all done because
we have you know tensorflow well no
these are all open problems and they're
really important and interesting and
it's about strategic settings how do i
collect data suppose i don't know what
you're going to do because i don't know
you very well right well i got to kind
of date about you so maybe i want to
push you in a part of the space where i
don't know much about you so i can get
data because and then later i'll realize
that you'll never you'll never go there
because of the way the game is set up
but you know that's part of the overall
you know data analysis context is that
yeah even the game of poker is
fascinating space
whenever there's any uncertainty your
lack of information is it's a
super exciting space yeah uh
just uh
lingard optimization for a second so if
we look at deep learning it's
essentially minimization of a
complicated
loss function so is there something
insightful or hopeful that you see
in the kinds of function surface that
loss functions that deep learning in
in the real world is trying to optimize
over is there something interesting this
is just
the usual kind of problems of
optimization
i think from an optimization point of
view that surface first of all it's
pretty smooth
um
and secondly if there's over if it's
over parameterized there's kind of lots
of paths down to reasonable optima
and so kind of the getting downhill to
the to an optimum is viewed as not as
hard as you might have expected in high
dimensions
the fact that some optima tend to be
really good ones and others not so good
and you tend to it's not sometimes you
find the good ones is sort of still
needs explanation
yes but but the particular surface is
coming from the particular generation of
neural nets i kind of suspect those will
this those will change
in 10 years it will not be exactly those
surfaces there'll be some others that
are and optimization theory will help
contribute to why other surfaces are why
other algorithms
layers of arithmetic operations with a
little bit of nonlinearity that's not
that didn't come from neuroscience per
se i mean maybe in the minds of some of
the people working on it they were
thinking about brains but uh they were
arithmetic circuits in all kinds of
fields you know uh computer science
control theory and so on and that layers
of these could transform things in
certain ways and that if it's smooth
maybe you could uh
you know find parameter values
um
you know it's a big is a is a sort of
big discovery that it's it's working
it's able to work at this scale but um
um i don't think that we're stuck with
that and we're certainly not stuck with
that because we're understanding the
brain
so in terms of uh on the algorithm size
of gradient descent do you think we're
stuck with gradient descent this is uh
variants of it what variants do you find
interesting or do you think there'll be
something else invented
that uh is able to
walk all over these optimization spaces
in more interesting ways so there's a
co-design of the surface and or the
architecture and the algorithm
so if you just ask if we stay with the
kind of architectures we have now and
not just neural nets but you know
phase retrieval architectures or
materials completion architectures and
so on um
you know i think we've kind of come to a
place where yeah a stochastic gradient
algorithms are dominant and
um there are versions uh they're you
know that are a little better than
others they you know have more
guarantees they're more robust and and
so on and there's ongoing research to
kind of figure out which is the best
downforce situation
um but i think that that'll start to
co-evolve that that'll put pressure on
the actual architecture and so we
shouldn't do it in this particular way
we should do it in a different way
because this other algorithm is now
available if you do it in a different
way
um
so
uh
that that i can't really anticipate that
co-evolution process but
you know gradients are amazing uh
mathematical objects um they uh have a
lot of
people who uh
start to study them more deeply
mathematically
are kind of shocked about what what they
are and what they can do um i mean to
think about this way if uh suppose that
i tell you if you move along the x-axis
you get uh
uh uh you know you go uphill in some
objective by you know three units
whereas if you move on the y-axis you go
uphill by seven units right now i'm
gonna only allow you to move a certain
you know unit distance
all right what are you gonna do well the
most not people will say i'm gonna go
along the y-axis i'm getting the biggest
bang for my buck you know and my buck is
only one unit so i'm gonna put all of it
in the y-axis right
and uh why should i even take any of my
strength my step size and put any of it
in the x-axis because i'm getting less
bang for my buck that seems like a
completely
you know clear cl argument and it's
wrong because the gradient direction is
not to go along the y-axis it's to take
a little bit of the x-axis
uh and that to understand that you have
to you have to know some math and um
so even a you know trivial so so-called
operator like grading is not trivial and
so you know exploiting its properties is
still very very important um now we know
that just providing descent has got all
kinds of problems it gets stuck in many
ways and it hadn't have you know good
dimension dependence and so on so um my
own line of work recently has been about
what kinds of stochasticity how can we
get dimension dependence how can we do
the theory of that
um and we've come up pretty favorable
results with certain kinds of
stochasticity
we have sufficient conditions generally
we know if you if you do this we will
give you a good guarantee
we don't have necessary conditions that
it must be done a certain way in general
so stochasticity how much randomness to
inject into the
into the walking along the gradient and
what kind of randomness
why is randomness good in this process
why is stochasticity good yeah so um i
give you simple answers but in some
sense again it's kind of amazing
stochasticity just uh
um you know
particular features of a surface that
could have hurt you if you were doing
one thing um deterministically it won't
hurt you because
uh you know by chance there's very
little chance that you would get hurt
and um
you know so here stochasticity um
you know is just kind of saves you from
some of the particular features of
surfaces that um
you know and in fact if you think about
you know surfaces that are discontinuous
in a first derivative like you know
absolute value function
um you will go down and hit that point
where there's non-differentiability
right and if you're running a
deterministic argument at that point you
can really do something bad
right whereas stochasticity just means
it's pretty unlikely that's going to
happen you're going to you're going to
hit that point
so you know it's again not trivially
analyzed but um
especially in higher dimensions also
stochasticity our intuition isn't very
good about it but it has properties that
kind of are very appealing in high
dimensions for a lot of large number of
reasons
um so it's it's all part of the
mathematics to kind of that's what's fun
to work in the field is that you get to
try to understand this mathematics and
um
but long story short you know partly
empirically it was discovered stochastic
gradient is very effective and theory
kind of followed i'd say um that
but i don't see that we're getting
clearly out of that uh
what's
the most beautiful mysterious a profound
idea to you in optimization
i don't know the most
but let me just say that uh you know
nestorov's work on nest drive
acceleration to me is uh pretty pretty
surprising and pretty deep
um
can you elaborate well install
acceleration is just that um
i suppose that we are
going to use gradients to move around
into space for the reasons i've alluded
to there there are nice directions to
move
and suppose that i tell you that you're
only allowed to use gradients you're not
going to be allowed to you'll see this
local person it can only sense kind of a
change in the surface
um but i'm going to give you kind of a
computer that's able to store all your
previous gradients and so you start to
learn some something about the the
surface
um and i'm going to restrict you to
maybe move in the direction of like a
linear span of all the gradients so you
can't kind of just move in some
arbitrary direction right so now we have
a well-defined mathematical complexity
model there's a certain classes of
algorithms that can do that and others
that can't and we can ask for certain
kinds of surfaces how fast can you get
down to the optimum
so there's an answers to these so for a
you know a smooth convex function
there's an answer which is one over the
number of steps squared
you will be within a ball of that size
after after k steps
um
gradient descent in particular has a
slower rate it's one over k
okay
um
so you could ask is gradient is said
actually even though we know it's a good
algorithm is it the best algorithm in
the sense of the answer is no well well
not clear yet because
what
one of our case score is a lower bound
that's that's probably the best you can
do what gradient is one over k but is
there something better
and so i think as a surprise to most
though nest drove discovered a new
algorithm that is got two pieces to it
it uses two gradients
um
and uh puts those together in a certain
kind of obscure way and uh the thing
doesn't even move downhill all the time
it sometimes goes back uphill
and if you're a physicist that kind of
makes some sense you're building up some
momentum and that is kind of the right
intuition but that that intuition is not
enough to understand kind of how to do
it and why it works
um
but it does it achieves one over k
squared and uh it has a mathematical
structure and it's still kind of to this
day a lot of us are writing papers and
trying to explore that and understand it
um so there are lots of cool ideas in
optimization but just kind of using
gradients i think is number one that
goes back you know 150 years um and then
nest drive i think has made a major
contribution with this idea so like you
said gradients themselves are in some
sense mysterious yeah they're not uh
they're not as trivial as they're not as
trivial
coordinate descent is more of a trivial
one you just pick one of the coordinates
that's how we think that's our human
mind that's our human minds think and
gradients are not that easy for our
human mind to grapple with
an absurd question
but uh what is statistics
so the here it's a little bit it's
somewhere between math and science and
technology it's somewhere in that convex
hole so it's a
set of principles that allow you to make
inferences that have got some reason to
be believed and also principles allow
you make decisions where you can have
some reason to believe you're not going
to make errors
so all of that requires some assumptions
about what do you mean by an error what
do you mean by you know the
probabilities and um
but you know you start after you start
making some of those assumptions you're
led to
uh
conclusions that yes i can guarantee
that you know you know if you do this in
this way your probability of making
error will be small
your probability of continuing to not
make errors over time will be small and
probability you found something that's
real will be small uh will be high so
decision making is a big part of the big
part yeah so uh the original so
statistics uh you know short history was
that you know it's kind of goes back
as a formal discipline you know 250
years or so
it was called inverse probability
because around that era
probability was developed sort of
especially to explain gambling
situations of course and um interesting
so
you would say well given the state of
nature is this there's a certain
roulette board that has a certain
mechanism in it uh what kind of outcomes
do i expect to see
uh and um especially if i do things long
long amounts of time what outcomes i see
and the physicists start to pay
attention to this
um and then people say well given let's
turn the problem around what if i saw
certain outcomes could i infer what the
underlying mechanism was that's an
inverse problem and in fact for quite a
while statistics was called inverse
probability that was the name of the
field
and i believe that uh it was laplace uh
who was working in napoleon's government
who was trying to who needed to do a
census of france
learn about the people there so he went
and gathered data and he analyzed that
data to determine policy and uh said
let's call this field that does this
kind of thing statistics because um
the the word state is in there in french
that's eta but
you know it's the study of data for the
state
so anyway that caught on and um it's
been called statistics ever since but um
uh but by the time it got formalized it
was sort of in the 30s um
and uh around that time there was game
theory and decision theory developed
nearby
people in that era didn't think of
themselves as either computer science or
statistics or controlled or econ they
were all they were all the above and so
you know von neumann is developing game
theory but also thinking of that as
decision theory wall is an
econometrician developing decision
theory and then you know turning that
into statistics
and so it's all about here's a
here's not just data and you analyze it
here's a loss function here's what you
care about here's the question you're
trying to ask
here is a probability model and here's
the risk you will face if you make
certain decisions
um and to this day in most advanced
statistical curricula you teach decision
theory is the starting point and then it
branches out into the two branches of
bazin or frequentist but um that's it's
all about decisions
in statistics what is the most
beautiful
mysterious maybe surprising idea that
you've come across
uh yeah good question um
i mean there's a bunch of surprising
ones there's something that's way too
technical for this thing but something
called james stein estimation which is
kind of surprising
and really takes time to wrap your head
around can you try to maybe i think i
don't even want to try um let me just
say a colleague
at steve steven stickler at university
of chicago wrote a really beautiful
paper on james stein estimation which
helps to its views of paradox it kind of
defeats the mind's attempts to
understand it but you can and steve has
a nice perspective on that
um
there uh so one of the troubles with
statistics is that it's like in physics
that are in quantum physics you have
multiple interpretations there's a wave
and particle duality in physics and
you get used to that over time but it
still kind of haunts you that you don't
really
you know quite understand the
relationship the electrons away when
electrons a particle well
um well the same thing happens here
there's bayesian ways of thinking and
frequentist and they are different they
they all they sometimes become sort of
the same in practice but they are
physically different and then in some
practice they are not the same at all
they give you rather different answers
um and so it is very much like wave and
particle duality and that is something
you have to kind of get used to in the
field can you define beijing and
frequencies yeah in decision theory you
can make i have a like i have a video
that people could see it's called are
you a bayesian or a frequentist and kind
of help try to
to make it really clear it comes from
decision theory so you know decision
theory
uh you're talking about loss functions
which are a function of
data x and parameter theta it's a
function of two arguments
okay
neither one of those arguments is known
you don't know the data a priori it's
random
and the parameter is unknown all right
so you have this function of two things
you don't know and you're trying to say
i want that function to be small i want
small loss
right
well um
what are you gonna do so you sort of say
well i'm gonna average over these
quantities or maximize over them or
something so that you know
i turn that uncertainty into something
certain
so you could look at the first argument
an average over it or you could look at
the second argument average over it
that's bayesian frequencies so the
frequencies says i'm going to look at
the x the data and i'm going to take
that as random and i'm going to average
over the distribution so i take the
expectation loss under x
theta is held fixed
all right that's called the risk and so
it's looking at other all the data sets
you could get
all right and say how well will a
certain procedure do under all those
data sets
that's called a frequency guarantee
all right so i think it is very
appropriate when like you're building a
piece of software and you're shipping it
out there and people are using all kinds
of data sets you want to have a stamp a
guarantee on it that as people run it on
many many data sets that you never even
thought about that 95 of the time it
will do the right thing
um perfectly reasonable
the bayesian perspective says well no
i'm going to look at the other argument
of the loss function the theta part okay
that's unknown and i'm uncertain about
it so i could have my own personal
probability for what it is you know how
many tall people are there out there i'm
trying to infer the average height of
the population well i have an idea
roughly what the height is
so i'm going to average over the um
the theta
so now that loss function has only now
again one argument's gone
now it's a function of x
and that's what a bayesian does is they
say well let's just focus on the
particular x we got the data set we got
we condition on that
conditional on the x i say something
about my loss that's a bayesian approach
to things and the bayesian will argue
that it's not relevant to look at all
the other data sets you could have
gotten and average over them the
frequentest approach
it's really only the data set you got
all right and i do agree with that
especially in situations where you're
working with a scientist you can learn a
lot about the domain and you really only
focus on certain kinds of data and
you've gathered your data and you make
inferences
i don't agree with it though that it you
know in the sense that there are needs
for frequency guarantees you're writing
software people are using it out there
you want to say something so these two
things have to go out to fight each
other a little bit but they have to
blend
so long story short there's a set of
ideas that are right in the middle
they're called empirical bays
and empirical base sort of starts with
the bayesian framework
it's
it's kind of arguably philosophically
more
you know reasonable and kosher
write down a bunch of the math that kind
of flows from that and then realize
there's a bunch of things you don't know
because it's the real world then you
don't know everything so you're
uncertain about certain quantities at
that point ask is there a reasonable way
to plug in an estimate for those things
okay and in some cases there's quite a
reasonable thing to do
to plug in there's a natural thing you
can observe in the world that you can
plug in
and then do a little bit more
mathematics and assure yourself it's
really good so my math are based on
human expertise what's what are good
they're both going in the bayesian
framework allows you to put a lot of
human expertise in
but the math kind of guides you along
that path and then kind of reassures you
at the end you could put that stamp of
approval under certain assumptions this
thing will work so perhaps you asked
question what's my favorite you know or
what's the most surprising nice idea so
one that is more accessible is something
called false discovery rate which is um
you know you're making not just one
hypothesis test or making one decision
you're making a whole bag of them
and in that bag of decisions you look at
the ones where you made a discovery you
announced that something interesting it
happened all right that's gonna be some
subset of your big bag
in the ones you made a discovery which
subset of those are bad
there are false false discoveries
you like the fraction of your false
discoveries among your discoveries to be
small
that's a different criterion than
accuracy or precision or recall or
sensitivity and specificity it's it's a
different quantity
those latter ones are almost all of them
um
have more of a frequencies flavor they
say given the truth
is that the null hypothesis is true
here's what accuracy i would get or
given that the alternative is true
here's what i would get so it's kind of
going forward from the state of nature
to the data
the bayesian goes the other direction
from the data back to the state of
nature and that's actually what false
discovery rate is it says given you made
a discovery
okay that's condition on your data
what's the probability of the hypothesis
it's going the other direction
and so um the classical frequency look
at that so i can't know that there's
some priors needed in that and the
empirical bayesian goes ahead and plows
forward and starts writing down these
formulas and realizes at some point some
of those things can actually be
estimated in a reasonable way oh
and so it's kind of it's a beautiful set
of ideas so i i this kind of line of
argument has come out it's not certainly
mine but it it sort of came out from
robin's around 1960. uh brad ephron has
written beautifully about this in
various papers and books and uh and the
fdr is you know ben yamini
in israel um john storey did this
bayesian interpretation and so on so
i've just absorbed these things over the
years and find it a very healthy way to
think about statistics
let me ask you about intelligence to
jump slightly back out
into philosophy perhaps
you said that uh
maybe you can elaborate but uh you said
that defining just even the question of
what is intelligence is a
word is as a very difficult question
is that a useful question do you think
we'll one day understand the
fundamentals of human intelligence and
what it means
you know have good uh benchmarks for
general intelligence that we put before
our machines
so i don't work on these topics so much
you're really asking a question for a
psychologist really and i just studied
some but i don't consider myself
at least an expert at this point
you know a psychologist aims to
understand human intelligence right and
i think
many psychologists i know are fairly
humble about this they they might try
and understand how a baby understands
you know whether something's a solid or
liquid or uh whether something's hidden
or not and um
maybe how
you know a child starts to learn the
meaning of certain words what's a verb
what's a noun and also you know
slowly but surely trying to figure out
things
um but human's ability to take a really
complicated environment reason about it
abstract about it find the right
abstractions communicate about it
interact and so on is just you know
really staggeringly rich and complicated
um
and so you know i think in all humidity
we don't think we're kind of aiming for
that in the near future certainly
psychologists doing experiments with
babies in the lab or with people talking
is is has a much more limited aspiration
and you know conor mcversky would look
at our reasoning patterns and they're
they're not deeply understanding all the
how we do our reasoning but they're sort
of saying here's some here's some
oddities about the reasoning and some
things you should you need to think
about it but also i as i emphasize and
things some things i've been writing
about um you know ai the revolution
hasn't happened yet yeah um great blog
post i've i've been emphasizing that you
know if you step back and look at uh
intelligent systems of any kind whatever
you mean by intelligence it's not just
the humans or the animals or you know
the
plants or whatever you know so a market
that brings goods into a city you know
food to restaurants or something every
day
uh is a system it's a decentralized set
of decisions looking at it from far
enough away it's just like a collection
of neurons everyone every neuron is
making its own little decisions
presumably in some way and if you step
back enough every little part of an
economic system is making us all of its
decisions
and just like with the brain who knows
what the individual neuron doesn't know
what the overall goal is
right but something happens at some
aggregate level same thing with the
economy people eat in a city and it's
robust
it works at all scales small villages to
big cities it's been working for
thousands of years uh it works rain or
shine so it's adaptive
um so all kind of you know those are
adjeeves one tends to apply to
intelligent systems robust adaptive you
know you don't need to keep adjusting it
it's self self healing whatever plus not
perfect you know intelligences are never
perfect and markets are not perfect
um but i do not believe in this area
that you cannot that you can say well
our computers our humans are smart but
you know no markets are not more markets
are so they are intelligent
uh now um we humans didn't evolve to be
markets
we've been participating in them right
but we are not ourselves a market per se
um the neurons could be viewed as the
market you can't there's economic you
know neuroscience kind of perspectives
that's interesting to pursue all that
the point though is is that if you were
to study humans and really be the
world's best psychologist study for
thousands of years and come up with the
theory of human intelligence you might
have never discovered principles of
markets you know spy demand curves and
you know matching and auctions and all
that uh those are real principles and
they lead to a form of intelligence
that's not maybe human intelligence it's
arguably another kind of intelligence
there probably are third kinds of
intelligence or fourth that none of us
are really thinking too much about right
now
so if you really and then all those are
relevant to computer systems in the
future certainly the market one is
relevant right now whereas understand
human intelligence is not so clear that
it's relevant right now probably not
um so if you want general intelligence
whatever one means by that or you know
understand the intelligence in a deep
sense and all that it is definitely has
to be not just human intelligence it's
got to be this broader thing and that's
not a mystery markets are intelligent so
you know
it's definitely not just a philosophical
stance to say we gotta move beyond and
tell who intelligence that sounds
ridiculous yeah but it's not and in that
blog post you define different kinds of
like intelligent infrastructure iii
which i really like that's some of the
concept you've just been
describing do you see ourselves if we
see earth human civilization is a single
organism do you think the intelligence
of that organism when you think from the
perspective of markets and
intelligence infrastructure is
increasing
is it increasing linearly is it
increasing exponentially what do you
think the future of that intelligence i
don't know i don't tend to think i don't
tend to answer questions like that
because you know that's science fiction
hoping to catch you off guard
well again because you said it's so far
in the future it's fun to ask and you'll
probably you know like you said
predicting the future is really nearly
impossible but
say
as an axiom one day we create
a human level superhuman level
intelligent not the scale of markets but
the scale of an individual
what do you think is
is
what do you think it would take to do
that or maybe to ask another question
is how would that system be different
than the biological
human beings that we see around us today
is it possible to say anything
interesting to that question or is it
just a stupid question it's not stupid
question but it's science fiction
science fiction and so i'm totally happy
to read science fiction and think about
it from time my own life
i loved there was this like brain in a
vat kind of you know little thing that
people were talking about when i was a
student i remember you know imagine that
uh
um you know between your brain and your
body there's you know there's a bunch of
wires right
and suppose that every one of them was
replaced with a
uh uh literal wire and then suppose that
wire was turning actually a little
wireless you know there's a receiver and
sender so the brain has got all the
senders and receiver you know on all of
its exiting uh you know axons and all
the dendrites down the body have
replaced with syndrome receivers now you
could move the body off somewhere and
put the brain in a vat
right and
then you could do things like start
killing off those centers of receivers
one by one and after you've killed off
all of them where is that person you
know they thought they were out in the
body walking around the world and they
moved on so those are science fiction
things those are fun to think about it's
just intriguing about where's what is
thought where is it and all that
and
i think every 18 year old it's to take
philosophy classes and think about these
things
and i think that everyone should think
about what could happen in society
that's kind of bad and all that but i
really don't think that's the right
thing for most of us that are my age
group to be doing and thinking about
i really think that we have so many more
present you know
first challenges and dangers and real
things to build and all that
um such that uh you know uh spending too
much time on science fiction at least in
public fora like this i think is is not
what we should be doing maybe over beers
in private that's right i'm well welcome
welcome
i'm not gonna broadcast where i have
beers because this is gonna go on
facebook
a lot of people showing up there but um
yeah i'll uh i love facebook twitter
amazon youtube i have i'm optimistic and
hopeful but uh maybe
maybe i don't have grounds for such
optimism and hope
let me ask
term you've mentored
some of the brightest
sort of some of the seminal figures in
the field can you uh
give advice to people who undergraduates
today
what does it take to take you know
advice on their journey if they're
interested in machine learning and ai in
in
[Music]
the ideas of markets from economics and
psychology and all the kinds of things
that you're exploring what what what
steps should they take on that journey
well yeah first of all the door is open
and second it's a journey i like your
language there
uh it is not that you're so brilliant
and you have great brilliant ideas and
therefore that's that's just you know
that's how you have success or that's
how you enter into the field uh it's
that you apprentice yourself you you
spend a lot of time you work on hard
things you
try and pull back and you be as broad as
you can you talk lots of people um
and it's like entering any kind of a
creative community there's um
years that are needed and uh human
connections are critical to it so you
know i think about you know being a
musician or being an artist or something
you don't just you know immediately from
day one you know you you're a genius and
therefore you do it no you
um
you know
practice really really hard on basics
and you uh be humble about where you are
and then and you realize you'll never be
an expert on everything so you kind of
pick and there's a lot of randomness and
a lot of kind of
luck but
luck just kind of picks out which branch
of the tree go down but you'll go down
some branch
um
so yeah it's it's a community so the
graduate school is i still think is one
of the wonderful phenomena that we have
in our in our world it's it's very much
about apprenticeship with an advisor
it's very much about a group of people
you belong to
it's a four or five year process so it's
plenty of time
to start from kind of nothing to come up
to something you know more expertise and
then start to have your own creativity
start to flower even surprise into your
own self
um and it's a very cooperative endeavor
it's i think a lot of people uh think of
science as highly competitive and i
think in some other fields it might be
more so
here it's way more cooperative than you
might imagine
and people are always teaching each
other something and people are always
more than happy to uh be clear that so i
i feel i'm an expert on certain kind of
things but i'm very much not expert on
lots of other things and a lot of them
are relevant and a lot of them are i
should know but it should in some sense
i you know you don't so
um i'm always willing to reveal my
ignorance to people around me so they
can teach me things
and uh i think a lot of us feel that way
about our field so it's very cooperative
uh i might add it's also very
international because it's so
cooperative we see no barriers and uh so
that the nationalism that you see
especially in the current era and
everything is just at odds with the way
that most of us think about what we're
doing here where this is a human
endeavor and we we cooperate
and are very much trying to do it
together for the you know the benefit of
everybody
so last question
where and how and why did you learn
french
and which language is more beautiful
english or french um great question so
um first of all i think italian's
actually more beautiful than french and
english and i also speak that so i'm i'm
i'm married to an italian and i have
kids and we speak italian
um
anyway though
all kidding aside that every language
allows you to express things a bit
differently um and it is one of the
great fun things to do in life is to
explore those things so
in fact when i kids
or you know teens or uh college students
ask me what they just
study i say well
do what your heart where your heart is
certainly do a lot of math math is good
for everybody but do some poetry and do
some history and do some language too um
you know throughout your life you'll
want to be a thinking person you'll want
to have done that
um
for me uh yeah french i learned when i
was i'd say
a late teen um i was living in the
middle of the country in kansas and uh
not much was going on in kansas with all
due respect to kansas but uh
and so my parents happen to have some
french books on the shelf and just in my
boredom i pulled them down and i found
this is fun and i kind of learned the
language by reading and
when i first heard it spoken i had no
idea what was being spoken but i
realized i somehow knew it from some
previous life and so i made the
connection
um but then you know i traveled and just
i i love to go beyond my own barriers
and uh my own comfort or whatever and i
found myself in you know on trains in
france next to say older people who
would you know live the whole life of
their own and
the ability to communicate with them was
was you know special and uh
ability to also see myself in other
people's shoes and have empathy and kind
of work on that language as part of that
um
so um so after that kind of experience
um and also embedding myself in french
culture which is you know quite quite
amazing you know languages are rich not
just because there's something
inherently beautiful about it but it's
all the creativity that went into it so
i learned a lot of songs read poems read
books
um and then i was here actually at mit
where we're doing the podcast today and
uh
young professor um
you know not yet married and uh um
you know not having a lot of friends in
the area so i just didn't have i was
getting kind of a bored person i said i
heard a lot of italians around there's
happened to be a lot of italians at mit
behind professor for some reason
and so i was kind of vaguely
understanding what they were talking
about i said well i should learn this
language too so i i did
and then later met my spouse and uh you
know wow italian became a more important
part of my life but um but i go to china
a lot these days i go to asia i go to
europe and um
every time i go i kind of uh i'm amazed
by the richness of human experience and
the the
people don't have any idea if you
haven't traveled kind of how i'm you
know amazingly rich and i love the
diversity
it's not just a buzzword to me it really
means something i love the you know
you know embed myself with other
people's experiences and uh so
yeah learning language is a big part of
that i think i've said in some interview
at some point that if i had you know
millions of dollars on the infinite time
whatever what would you really work on
if you really wanted to do ai and for me
that is natural language and really done
right you know deep understanding of
language um that's to me an amazingly
interesting scientific challenge and uh
when we're very far away one we're very
far away but good natural language
people are kind of really invested then
i think a lot of them see that's where
the core of ai is that if you understand
that you really help human communication
you understand something about the human
mind the semantics that come out of the
human mind and i agree i think that will
be such a long time so i didn't do that
in my career just because i kind of i
was behind in the early days i didn't
kind of know enough of that stuff i was
at mit i didn't learn much
language
and it was too late at some point to
kind of spend a whole career doing that
but i admire that field and uh
um
and so in my little way by learning
language you know kind of
that part of my brain has um has been
trained up
jan was right you truly are the miles
davis and machine learning i don't think
there's a better place than it was mike
is a huge honor talking to you today
merci beaucoup all right it's been my
pleasure thank you
thanks for listening to this
conversation with michael i jordan and
thank you to our presenting sponsor cash
app download it use code lex podcast
you'll get ten dollars and ten dollars
will go to first an organization that
inspires and educates young minds to
become science and technology innovators
of tomorrow
if you enjoy this podcast subscribe on
youtube give it five stars on apple
podcast support it on patreon or simply
connect with me on twitter at lex
friedman
and now let me leave you with some words
of wisdom from michael i jordan from his
blog post titled artificial intelligence
the revolution hasn't happened yet
calling for broadening the scope of the
ai field we should embrace the fact that
what we are witnessing is the creation
of a new branch of engineering the term
engineering is often invoked in a narrow
sense in academia and beyond
with overtones of cold effectless
machinery and negative connotations of
loss of control by humans
but an engineering discipline can be
what we want it to be
in the current era we have a real
opportunity to conceive of something
historically new a human-centric
engineering discipline
i'll resist giving this emerging
discipline a name but if the acronym ai
continues to be used
let's be aware of the very real
limitations of this placeholder let's
broaden our scope tone down the hype
and recognize the serious challenges
ahead
thank you for listening and hope to see
you next time
you