Kind: captions
Language: en
the following is a conversation with
wojciech zaramba co-founder of openai
which is one of the top organizations in
the world doing artificial intelligence
research and development
wojciech is the head of language and
cogeneration teams building and doing
research on github copilot openai codex
and gpt
three and who knows
four five
six
n and
n plus one and he also previously led
openai's robotic efforts
these are incredibly exciting projects
to me that deeply challenge and expand
our understanding of the structure and
nature of intelligence the 21st century
i think may very well be remembered for
a handful of revolutionary ai systems
and their implementations
gpt codex and applications of language
models and transformers in general
to the language and visual domains may
very well be at the core of these ai
systems
to support this podcast please check out
our sponsors
they're listed in the description
this is a lex friedman podcast and here
is my conversation with wachek zaremba
you mentioned that sam altman asked
about the fermi paradox
and the people at open ai had really
sophisticated interesting answers so
that's when you knew this is the right
team to be working with so let me ask
you about the fermi paradox about aliens
why have we not found overwhelming
evidence for aliens visiting earth
i don't have a conviction in the answer
but rather kind of probabilistic
perspective on what might be a let's say
possible answers it's also interesting
that the question itself even
can't touch on the you know your typical
question of what's the meaning of life
because like if you assume that like we
don't see aliens because they destroy
themselves that kind of upwards their
focus on making sure that we won't
destroy ourselves yeah and at the moment
the
place where i am actually with my belief
and these things also change over the
time
is i think that we might be alone in the
universe which actually makes
life more or less a consciousness life
more kind of valuable and that means
that we should more appreciate it
have you always been alone so what's
your intuition about our galaxy our
universe is it just
sprinkled with graveyards of intelligent
civilizations or are we truly is is life
intelligent life truly unique
at the moment my belief that it is
unique but i would say i could also
you know there was like some footage
released with ufo objects which makes me
actually doubt my own belief yes
yeah i can tell you one crazy answer
that i have heard yes
so
apparently when you look actually at the
limits of computation
you can compute more
if the temperature of the universe would
drop down
so one of the things that
aliens might want to do if they are
truly optimizing to maximize amount of
compute which you know maybe can lead to
or let's say simulations or so it's
instead of wasting current entropy of
the universe because you know we by
living we are actually somewhat wasting
entropy
then you can wait for the universe to
cool down such that you have more
computation that's kind of a funny
answer i'm not sure if i believe in it
but that would be one of the reasons why
you don't see aliens it's also possible
see some people say that maybe there is
not that much point in actually going to
other galaxies
if you can go inwards
so there is no limits of what could be
an experience
if we could you know connect machines to
our brains
while there are still some limits if we
want to explore universe yeah there
could be a lot of
ways to go inwards too
once you figure out some aspect of
physics we haven't figured out yet maybe
you can travel to different dimensions i
mean
travel in three-dimensional space may
not be the most fun kind of travel there
may be like just a huge amount of
different ways to travel and it doesn't
require a spaceship going
slowly in 3d space space time it also
feels you know one of the problems is
that speed of light is low and universe
is vast yeah and um
it seems that actually most likely if we
want to travel very far
then then we would instead of actually
sending spaceships with humans that wait
a lot we would
send something similar to what yuri
miller is working on these are like a
huge uh sail which is at first powered
power there is a shot of laser from an
earth and it can propel it to a quarter
of speed of light and uh sail itself
contains a
few grams of equipment and that might be
the way to actually
transport matter through universe but
then when you think what would it mean
for humans it means that
we would need to actually put their 3d
printer and you know 3d print a human on
other planet i don't know play them
youtube or let's say or like a pre 3d
print like a huge human right away or
maybe a womb or so um yeah
with our current techniques of
archaeology
if
a civilization was born and died
long long enough ago on earth we
wouldn't be able to tell and so that
makes me really sad
and so i think about earth in that same
way how can we leave some remnants if we
do destroy ourselves how can we leave
remnants for aliens in the future to
discover
like here's some nice stuff we've done
like wikipedia and youtube do we have it
like
in a satellite orbiting earth
with a hard drive like how how do we say
how do we back up human civilization
uh for the good parts or
all of it is good parts
so that uh
it can be preserved longer than our
bodies can that's a
that's kind of a
it's a difficult question it also
requires the difficult acceptance of the
fact that we may die and if we die we
may
die suddenly as a civilization
so let's see i think it kind of depends
on the cataclysm we have observed in
other parts of the universe that births
of gamma rays
these are
high energy
rays of light that actually can
apparently kill entire galaxy
so there might be actually nothing even
to
nothing to protect us from it i'm also
when i'm looking actually at the past
civilization so it's like aztecs or so
they disappear from the
surface of the earth and one can ask
why is it the case
and
the way i'm thinking about it is
you know that definitely they had some
problem that they couldn't solve
and maybe there was a flat and all of a
sudden they couldn't drink there was no
potable water and they all died
and
i think that
so far
the best solution to such a problems is
i guess technology so i mean if they
would know that you can just boil water
and then drink it after then that would
save their civilization and even now
when we look actually at the current
pandemic it seems that once again
actually science comes to rescue and
somehow science increases size of the
action space and i think that's a good
thing
yeah but nature
has a vastly larger action space but
still it might be a good thing for us to
keep on increasing action space
okay
looking at past civilizations yes
but looking at the destruction of human
civilization
perhaps expanding the action space will
add
actions that are easily
acted upon easily executed and as a
result destroy
us
so let's see
i was pondering
why actually even
we have negative impact on the
globe because you know if you ask every
single individual they would like to
have clean air
they would like healthy planet but
somehow it actually is not the case that
as a collective we are not going this
direction
i think that there exists very powerful
system to describe what we value that's
capitalism it assigns actually monetary
values to various activities at the
moment the problem in the current system
is that there are some things which we
value there is no cost assigned to it so
even though we value clean air or maybe
we also
value
lack of destruction on the internet or
so at the moment
these quantities you know companies
corporations can pollute them uh for
free
so in some sense
i wish
or like and that's i guess purpose of
politics to
align the incentive systems and we are
kind of maybe even moving in this
direction the first issue is even to be
able to measure the things that we value
then we can actually assign the monetary
value to them
yeah and that's so it's getting the data
and also
probably through technology enabling
people to vote
and to
move money around in a way that is
aligned with their values and that's
very much a technology question so like
having one president
and congress
and voting that happens every four years
or something like that
that's a very outdated idea there could
be some technological improvements to
that kind of idea so
i'm thinking from time to time about
these topics but it also feels to me
that it's it's a little bit like a
it's hard for me to actually make
correct predictions what is the
appropriate thing to do i extremely
trust uh sam altman our ceo
on these topics he um okay i'm more on
the side of being i guess
naive hippie that
yeah
that's your life philosophy um
well like i think self-doubt
and uh
i think hippie implies optimism those
those two things are pretty pretty good
way to operate
i mean still it is
hard for me to actually
understand how the politics works or
like uh how this like
exactly how the things would play out
and sam is a really excellent with it
what do you think is rarest in the
universe you said we might be alone
what's hardest to build is another
engineering way to ask that
life
intelligence or consciousness so like
you said that we might be alone
which is the thing that's hardest to get
to
is it just the origin of life is it the
origin of intelligence is it the origin
of consciousness
so
um let me at first explain you my kind
of mental model what i think is needed
for life to appear
um
so
i imagine that at some point there was
this primordial
zoop of
amino acids and maybe some proteins in
the ocean
and you know some proteins were turning
into some other proteins through
reaction
and you can almost think about this
uh cycle of what turns into what as
there is a graph essentially describing
which substance turns into some other
substance and essentially life means
that all the sudden in the graph has
been created a cycle such that the same
thing keeps on happening over and over
again that's what is needed for life to
happen and in some sense you can think
almost that you have this gigantic graph
and it needs like a sufficient number of
edges for the cycle to appear
then um from perspective of intelligence
and consciousness
my current intuition is that they might
be
quite intertwined first of all it might
not be that it's like a binary thing
that you have intelligence or
consciousness it seems to be a
more a
continuous component let's see if we
look for instance on the even networks
recognizing images and people are able
to show that the activations of these
networks correlate very strongly
with activations in visual cortex
of some monkeys the same seems to be
true about language models
also if you for instance
look
if you train agent in a 3d world
at first you know it it it it barely
recognizes what is going on over the
time it kind of recognizes foreground
from a background over the time it kind
of knows where there is a foot
and it just follows it
over the time it actually starts having
a 3d perception so it is possible for
instance to look inside of the head of
an agent and ask what would it see if it
looks to the right and the crazy thing
is you know initially when the agents
are very trained these predictions are
pretty bad over the time they they
become better and better you can still
see
that if you ask what happens when the
head is turned by 360 degrees for some
time they think that the different thing
appears and then at some stage they
understand actually that the same
thing's supposed to appear so they get
like a understanding of 3d structure
it's also you know very likely that they
have inside some
level of and of like a symbolic
reasoning like they're particularly
symbols for other agents so when you
look at dota agents they collaborate
together and uh
and
now they they they have some
anticipation of uh if if they would win
battle they have some some expectations
with respect to other agents i might be
you know too much anthropomorphizing
um the the how the things look
look for me but then the fact that they
have a symbol for other agents
and makes me believe that
at some stage as the uh you know as they
are optimizing for skills they would
have also symbol to describe
themselves this is like a very useful
symbol to have and this particularity i
would call it like a self-consciousness
or self-awareness
and still it might be different from the
consciousness so i guess the the way how
i'm understanding the word consciousness
let's say the experience of drinking a
coffee or let's say experience of being
a butt
that's the meaning of the word
consciousness it doesn't mean to be
awake
yeah it feels
it might be also somewhat related to
memory and recurrent connections so um
it's kind of okay if you look at
anesthetic drugs they might be
uh like they essentially
they disturb
brain waste
such that
[Music]
maybe memory is not not formed
so there's a lessening of consciousness
when you do that correct and so that's
one way to intuit what is consciousness
there's also kind of another
element here it could be that it's
you know this kind of self-awareness
module that you described
plus the actual subjective experience
is a storytelling module
that tells us a story about uh
what we're experiencing
the
crazy thing so let's say i mean in
meditation they teach people
not to speak story inside of the head
and there is also some fraction of
population
who doesn't have actually narrator i
know people who don't have a right
narrator and you know they have to use
external people in order to
kind of
solve tasks that require internal
narrator
so
it seems that it's possible to have the
experience without the talk
what are we talking about when we talk
about the internal narrator is that the
voice when you're like yeah i thought
that that that's what you are referring
to well i was referring more on the like
not an actual voice
i meant like
there's some kind of
like subjective experience
feels like it's
it's fundamentally about storytelling to
ourselves
it feels like
like the feeling is a story
that is much
much simpler abstraction than the raw
sensory information
so it feels like it's a very high level
abstraction
that
is useful
for me to feel like
entity in this world
most
useful aspect of it is that
because i'm conscious
i think there's an intricate connection
to me not one
wanting to die
so like
it's a useful hack to really
prioritize not dying
like those seem to be somehow connected
so i'm telling the story of like it's
richly feels like something to be me and
the fact that me exists in this world i
want to preserve me
and so that makes it a useful agent hack
so i will just refer maybe to the first
part as you said about the kind of story
of describing who you are
i was
thinking about that even so you know
obviously i'm i'm i
like thinking about consciousness uh i
like thinking about the ai as well and
i'm trying to see analogies of these
things in ai what would it correspond to
so um
you know openly i trained a
a
model called gpt
which
can generate a
pretty amusing text on arbitrary topic
and um
and one way to control gpd
is uh by putting into prefix at the
beginning of the text some information
what would be the story about
you can have even chat with uh
you know with gpt by saying that the
chat is with lex or elon musk or so
and gpt would just
pretend to be you or elon musk or so
and
it almost feels that this uh
story that we give ourselves to describe
our life it's almost like a
things that you put into context of gpt
yeah the primary it's the and but the
the context we provide to gpt
is uh
is multimodal it's so gpt itself is
multimodal gpt itself uh hasn't learned
actually from experience of single human
but from the experience of humanity it's
a chameleon you can turn it into
anything and in some sense by providing
context uh
it you know
behaves as the thing that you wanted it
to be and it's interesting that the
you know people have a stories of who
they are and as i said these stories
they help them to operate in the world
but it's also you know interesting
i guess various people find it out
through meditation or so that
there might be some patterns that you
have learned
when you were a kid that actually are
not serving you anymore
and you also might be thinking that
that's who you are and that's actually
just the story
yeah so it's a useful hack but sometimes
it gets us into trouble it's a local
optima
you wrote that stephen hawking he
tweeted stephen hawking asked what
breathes fire into equations which meant
what makes given mathematical equations
realize the physics of a universe
similarly
i wonder what breathes fire into
computation what makes given computation
conscious
okay so how do we engineer consciousness
how do you breathe fire
and magic into the machine
so
it seems clear to me that not every
computation is conscious i mean you can
let's say just keep on multiplying one
matrix over and over again and my
gigantic matrix you can put a lot of
computation i don't think it would be
conscious so in some sense the question
is
what are the computations which could be
conscious
uh i mean so one assumption is
that it has to do purely with
computation that you can abstract away
matter and other possibilities that it's
very important was the realization of
computation that it has to do with some
uh uh force fields or so and they bring
consciousness at the moment my intuition
is that it can be fully abstracted that
way so in case of computation you can
ask yourself what are the
mathematical objects or so that could
bring such a properties so for instance
if we think about the
models uh ai models then what they truly
try to do
or like models like gpt is uh
you know they try to predict a next word
or so and this turns out to be
equivalent to
compressing
text
and because in some sense compression
means that
you learn the model of reality and you
have just to uh
remember where are your mistakes the
better you are in predicting the
and
and in some sense when we look at our
experience also when you look for
instance the car driving you know in
which direction it will go you are good
like a in prediction and um
you know it might be the case that the
consciousness
is intertwined with compression it might
be also the case that self-consciousness
has to do with compressor trying to
compress itself so
um
okay i was just wondering what are the
objects in you know mathematics or
computer science which are mysterious
that could uh that that could have to do
with consciousness and then i thought um
you know you you see in uh mathematics
there is something called cadal theorem
which means okay you have if you have
sufficiently complicated mathematical
system it is possible to point the
mathematical system back on itself in
computer sense there is uh something
called helping problem it's it's
somewhat similar construction so i
thought that you know if we believe that
that the that
under assumption that consciousness has
to do with uh with compression
uh
then you could imagine that the the as
you keep on compressing things then at
some point it actually makes sense
for the compressor to compress itself
metacompression yeah consciousness is
metacompression
that's uh that's and i and an idea
and in some sense you know the creation
of it
thank you so uh
but do you think if we think of a
touring machine a universal touring
machine
can that achieve
consciousness
so is there some
thing beyond our traditional definition
of computation that's required so it's a
specific computation and i said this
computation has to do with compression
and
the compression itself maybe other way
of putting it is like you are internally
creating the model of reality
in order like a it's like a you try
inside to simplify reality in order to
predict what's going to happen
and
that also feels somewhat similar to how
i think actually about my own conscious
experience so clearly i don't have
access to reality the only access to
reality is through you know cable going
to my brain and my brain is creating a
simulation of reality and i have access
to the simulation of reality
are you by any chance uh aware of uh
the harder prize marcus hutter
he he made this prize
for compression
of wikipedia pages
and
there's a few qualities to it
one i think has to be perfect
compression which makes
i think that little quirk makes it much
less um
applicable to the general task of
intelligence because it feels like
intelligence is always going to be messy
uh
like perfect compression is feels like
it's not the right goal but it's
nevertheless a very interesting goal so
for him intelligence equals compression
and so
the smaller you make the file
given a large wikipedia page
the more intelligent the system has to
be yeah that makes sense so you can make
perfect compression if you store errors
and i think that actually what he meant
is you have algorithm plus errors and by
the way hooter hatter is a he was pa uh
phd advisor of shenleck who is the mind
uh
uh deep mind co-founder yeah yeah so
there's an interesting
and now he's a deep mind there's an
interesting uh network of people he's
one of the people that
i think
seriously took on the task of what would
an agi system look like
i think for a longest time
the question of agi was not
taken
seriously or rather rigorously
and he did just that like mathematically
speaking what would the model look like
if you remove the constraints of it
having to be
having to have a
reasonable amount of memory reasonable
amount of running time complexity uh
computation time what would it look like
and essentially it's it's a
half math half philosophical discussion
of uh how would like a reinforcement
learning type of framework look like for
an agi yeah so he developed a framework
even to describe what's optimal with
respect to reinforcement learning like
there is a theoretical framework which
is as you said
under assumption there is infinite
amount of memory and compute and there
was actually one person before his name
is solomonov hutter extended
amount of work to reinforcement learning
but there exists a
theoretical algorithm which is optimal
algorithm to build intelligence and i
can actually explain you the algorithm
yes
let's go let's go so the task itself can
i just
pause
how absurd it is
for brain in a skull trying to explain
the algorithm for intelligence just go
ahead it is pretty crazy it is pretty
crazy that you know the brain itself is
actually so small and it can ponder
how to design algorithms that optimally
solve the problem of intelligence okay
all right so what's the algorithm so
let's see so first of all the task
itself is
described as
you have infinite sequence of zeros and
ones
okay you read n bits and you are about
to predict n plus one bit
so that's the task and you could imagine
that every task could be casted as such
a task so if for instance you have
images and labels you can just turn
every image into sequence of zeros and
ones then label you concatenate labels
and you and that that's actually the the
and you could you could start by having
training data first and then afterwards
you have test data
so theoretically any problem could be
casted as a problem of predicting zeros
and ones on this infinite type so um
so let's say you read already n bits and
you want to predict n plus one bit
and i will ask you to write
every possible program that generates
these end bits okay so
and you can have you you choose
programming language it can be in python
or c
and the difference between programming
languages
might be there is a difference by
constant
asymptotically your predictions will be
equivalent
so you you read and beats you enumerate
all the programs that produce these and
end bits in their output
and then in order to predict n plus one
bit you actually weight
the programs according to their length
and there is like some specific formula
how you weight them and then the n plus
one bit prediction is the prediction uh
from each of this program according to
that weight
like statistically you statistically
pick so the smaller the program the more
likely you you are to pick the its
output
so uh that's that algorithm is grounded
in the hope
or the intuition that the simple answer
is the right one it's a formalization of
it yeah um it also
means like if you would ask the question
after
how many years
would you know
sun explode
you can say
it's more likely the answer is
to some power because it's a shorter
program
yeah
and then other
well i don't have a good intuition about
how different the space of short
programs are from the space of large
programs
like
what is the universe where short
programs
uh like run things
uh so as i said the things have to agree
with end beats so even if you have
you you need to start okay if if you
have very short program and they're like
uh still some as if it's not perfect
with prediction of n bits you have to
start errors what are the errors and
that gives you the full program that
agrees on end beats
oh so you don't agree perfectly with the
end bits and you store
that's like a longer a longer program
slightly longer program
because it contains these extra bits of
errors that's fascinating what's what's
your intuition
about
the the programs
that are able to do cool stuff like
intelligence and consciousness are they
uh
perfectly like is is it uh
is there if then statements in them so
like is there a lot of exceptions that
they're storing so um you could imagine
if there would be tremendous amount of
if statements yeah then they wouldn't be
that short in case of neural networks
you could imagine that
what happens is uh
they
when you start with an uninitialized
neural network uh it stores internally
many possibilities how the
how the problem can be solved and sgd is
kind of magnifying some some
some
paths which are slightly
similar to the correct answer so it's
kind of magnifying correct programs and
in some sense hdd is a search algorithm
in the program space and the program
space is represented by uh you know kind
of the wiring inside of the neural
network and there's like an insane
number of ways how that features can be
computed
let me ask you the high level basic
question that's not so basic
what is deep learning
is there a way you'd like to think of it
that is different than like a generic
textbook definition
the thing that i hinted just a second
ago is maybe the uh closest to how i'm
thinking these days about um deep
learning so
now the statement is
uh neural networks can represent some
programs
uh it seems that various modules that we
are actually adding up to are like a you
know we we want networks to be deep
because we we want multiple steps of the
computation
and
and deep learning provides the way to
represent space of programs which is
searchable and it's searchable with
stochastic gradient descent so we have
an algorithm to search over a humongous
number of programs
and gradient descent kind of bubbles up
the things that are tend to give correct
answers so
a neural network
with a with fixed weights that's
optimized do you think of that as a
single program um so there is a
work by christopher olach where he
so he works on interpretability of
neural networks and he was able to
uh
to identify inside of the neural network
for instance a detector of a wheel for a
car or the detector of a mask for a car
and then he was able to separate them
out and assemble them uh together using
a simple program uh for the detector for
a car detector that's like uh if you
think of traditionally defined programs
that's like a function within a program
that this particular neural network was
able to find and you can tear that out
just like you can copy and paste from
stack overflow
that
so uh any program is a composition of
smaller programs
yeah i mean the nice thing about the
neural networks is that it allows the
things to be more fuzzy than in case of
programs
in case of programs you have this like a
branching this way or that way and the
neural networks they they have an easier
way to
to be somewhere in between or to share
things
what to use the most beautiful or
surprising idea in deep learning
in the utilization of these neural
networks which by the way for people who
are not familiar
neural networks is a bunch of uh
what would you say it's inspired by the
human brain there's neurons there's
connection between those neurons there's
inputs and there's outputs and there's
millions or billions of those neurons
and
the learning
happens
uh by adjusting the weights on the edges
that connect these neurons thank you for
giving definition that
i supposed to do it but i guess you have
enough empathy to listeners to actually
know that that might be useful no that's
like
so i'm asking plato of like what is the
meaning of life he's not going to answer
you're being philosophical and deep and
quite profound talking about the space
of programs which is just very
interesting but also for people who are
just not familiar with the hell we're
talking about when we talk about deep
learning anyway sorry what is the most
beautiful
or surprising idea to you in in um in
all the time you've worked at deep
learning and you worked on a lot of
fascinating projects
applications of neural networks
it doesn't have to be big and profound
it can be a cool trick yeah i mean i'm
thinking about the trick but like it's
still amusing to me that it works at all
yeah that let's say that the extremely
simple algorithm stochastic gradient
descent which is something that i would
be able you know to derive on the piece
of paper to high school student uh when
put at the
ins at the scale of you know thousands
of machines actually
uh can create
the
behaviors we which we called kind of
human like behaviors
so in general
any applications to cast a gradient
descent to neural networks is
is amazing to you so that or is there a
particular application
in natural language
reinforcement learning
uh
and also would you attribute
that success too is it just scale
what profound insight can we take from
the fact that
the thing works for
gigantic
uh sets of variables
i mean the interesting thing is these
algorithms they were
invented uh decades ago
and
people actually
gave up on the idea yeah and um
you know back then they thought that we
need profoundly different algorithms and
they spent a lot of cycles on very
different algorithms and i believe that
you know we have seen that various
various innovations that say like
transformer or or dropout or so they can
uh you know pass the help but it's also
remarkable to me that this algorithm
from 60s or so
or i mean you can even say that the
gradient descent was invented by leibniz
in i guess 18th century or so that
actually
is the
core of learning
in the past people are
it's almost like a out of the maybe an
ego people are saying that it cannot be
the case that such a simple algorithm is
there you know
uh
could solve complicated problems
so they were in search for the
other algorithms and as i'm saying like
i believe that actually we are in the
game where there is there are actually
frankly three levels there is compute
there are algorithms and there is data
and if we want to build intelligent
systems we have to
pull all three levers
and they are actually multiplicative
and it's also interesting so you ask is
it only compute
people internally they did the studies
to determine how much gains they were
coming from different levels and so far
we have seen that more gains came from
compute than algorithms but also we are
in the world that in case of compute
there is a kind of you know exponential
increase in funding and at some point
it's impossible to
invest more it's impossible to you know
invest 10 trillion dollars
because we are speaking about that
let's say all taxes in u.s
uh but you're talking about money there
could be innovation
in the compute that's that's true as
well
so i mean they're like a few pieces so
one piece is human brain is an
incredible super computer
[Music]
and they're like a
it
it has
100 trillion
parameters or like a if you try to count
various quantities in the brain there
are like a neurons synapses that small
number of neurons there is a lot of
synapses yeah it's unclear even how to
map
synapses
to
two parameters of neural networks but
it's clear that there are many more yeah
so it might be the case that our
networks are still somewhat small
it also might be the case that they are
more efficient than brain or less
efficient by some by some huge factor
i also believe that there will be like a
you know at the moment we are at the
stage that the these neural networks
they require 1000x or like a huge factor
of more data than humans do and it will
be a matter of
there will be algorithms that
vastly decrease sample complexity i
believe so but the place where we are
heading today is dark domains which
contains million x
more
data and even though computers might be
1 000 times slower than humans in
learning that's not the problem okay for
instance
i believe that
it should be possible to create super
human therapies
uh by uh
and and then they're like even simple
steps of of doing what of of doing it
and you know that the core reason is
there is just machine will be able to
read way more
transcripts of therapies and then it
should be able to speak simultaneously
with many more people and it should be
possible to optimize it uh all in
parallel
and well there's now you're touching on
something i deeply care about and think
is way harder than we imagined
um
what's the goal of a therapist what's it
called therapies
so okay so one goal now this is
terrifying to me
but there's a lot of people that
contemplate suicide suffer from
depression
and they could significantly be helped
with therapy
and the idea that an ai algorithm might
be in charge of that
it's like a life and death task
it's uh
the stakes are high
so one
goal for a therapist whether human or ai
is to prevent suicide ideation to
prevent suicide how do you achieve that
so
let's see
so
to be clear i don't think that the
current models are good enough for such
a task because it requires insane amount
of understanding and patty and the
models are far from this place but it's
but do you think that understanding
empathy that signal is in the data um i
think there is some signal in the data
yes i mean there are plenty of
transcripts of conversations
and it is possible to
it is possible from it to understand
personalities it is possible from it to
understand uh if conversation is
a friendly
uh amicable uh
antagonistic it is i believe that the
you know given the fact that the models
that we train now
they can
they can have
they are chameleons that they can have
any personality they might turn out to
be better in understanding
uh personality of other people than
anyone else and they feel pathetic to be
empathetic yeah
interesting uh but i wonder if there's
some level
of
multiple modalities required
to be able to
be empathetic of the human experience
whether language is not enough to
understand death to understand fear to
understand
uh childhood trauma
to understand uh wit and humor required
when you're dancing with the person who
might be depressed or suffering
both humor and hope and love and all
those kinds of things
so there's another underlying question
which is self-supervised versus
supervised
so can you get
that from the data by just reading
a huge number of transcripts i actually
so i think that reading huge number of
transcripts is a step one it's like the
same way as you cannot learn to dance if
just from youtube by watching it you
have to actually try it out yourself
yeah and so i think that here that's a
similar situation i also wouldn't deploy
the system in the high-stakes situations
right away but kind of see gradually
where
it goes and
obviously initially
it would have to go hand with a hand in
hand with humans but
at the moment we are in the situation
that actually
there is many more people who actually
would like to have a therapy or
or speak with with someone then there
are therapies out there okay you know i
was
so so
fundamentally i was thinking what are
the things that
can vastly increase people well-being
therapy is one of them i think
meditation is other one i guess maybe
human connection is a third one and i
guess
pharmacologically it's also possible
maybe direct brain stimulation or
something like that but these are pretty
much options out there then let's say
the way i'm thinking about the agi
endeavor is by default that's an
endeavor to
increase amount of wealth and i believe
that we can vastly increase amount of
wealth
for everyone and simultaneously so i
mean they're like two endeavors that
make sense to me one is like essentially
increase amount of wealth and second one
is uh increase overall human well-being
and those are coupled together and they
they can okay i would say these are
different topics one can help another
and uh you know therapist is a funny
word because i see friendship and love
as therapy i mean so therapist broadly
defined as just friendship as a friend
so like therapist is has a very kind of
clinical sense to it but
what is human connection
you're like uh
not to get all camus and dostoyevsky on
you but you know life is suffering and
we draw
we
seek connection with other humans as we
desperately try to make sense of this
world
in the deep overwhelming loneliness that
we feel
inside
so i think connection has to do with
understanding
and i think that almost like a lack of
understanding causes suffering if you
speak with someone and you
do you feel ignored that actually causes
pain if you are feeling deeply
understood that actually
they they might not even tell you what
to do in life but like a pure
understanding or just being heard
understanding is a kind of
it's a lot you know just being heard
feel like you're being heard
like somehow
that's uh alleviation temporarily of the
loneliness
that if somebody
knows you're here
with their body language with the way
they are with the way they look at you
with the way they talk
you feel less alone for a brief moment
yeah very very much agree so i thought
in the past about uh somewhat similar
question to yours which is what is love
uh rather what is connection yes and um
and obviously i think about these things
from ai perspective what would it mean
um
so i said that the you know intelligence
has to do with some compression which is
more or less like i can say almost
understanding of what is going around it
seems to me that uh other aspect is
there seem to be reward functions and
you can have a you know reward for
uh food for maybe human connection for
uh let's say warmth
uh
sex and so on
and um
and it turns out that the various people
might be optimizing slightly different
reward functions they essentially might
care about different things
and um
in case of
love at least the love between two
people you can say that the um you know
boundary between people dissolves to
such extent that
they end up optimizing each other reward
functions
yeah
oh that's interesting um
the success of each other yeah in some
sense i would say love means
uh
helping others to optimize their uh
reward functions not your reward
functions not the things that you think
are important but the things that the
person cares about you try to help them
to optimize it so love is uh
if you think of two reward functions you
just it's a condition yeah you combine
them together yeah pretty much maybe
like with a weight and it depends like
the dynamic of the relationship yeah i
mean you could imagine that if you are
fully uh optimizing someone's reward
function without yours then yeah then
maybe are creating code dependency or
something like that yeah
i'm not sure what's the appropriate
weight but the interesting thing is i
even
i even think that the
individual person
we ourselves we are actually
less of a
unified insight so for instance if you
look at the donut on the one level you
might think oh this like it looks tasty
i would like to eat it on another level
you might tell yourself i shouldn't be
doing it because
i want to gain muscles so and you know
you might do it regardless kind of
against yourself so it seems that even
within ourselves they're almost like a
kind of intertwined personas
and
i believe that the self-love
means that
the love between all these persons which
also means being able to
love love
yourself when we are angry or stressed
or so combining all those reward
functions of the different selves you
have yeah and accepting that they are
there okay you know often people they
have a negative self-talk or they say i
don't like when i'm angry and like i try
to imagine
try to imagine if there would be
like a
small baby alex like a five years old
who's angry angry and then you're like
you shouldn't be angry like stop being
angry yeah but like instead actually you
want the legs to come over give him a
hug and he's like i say it's fine okay
you can't be angry as long as you want
yeah then he would stop
or
or maybe not or maybe not but you cannot
expect it even yeah
but still that doesn't explain the why
of love like why is love part of the
human condition why is it useful
to combine the reward functions
it seems like
that doesn't i mean
i don't think reinforcement learning
frameworks can give us answers to why
even even the hudder
framework has an objective function
that's static so we came to existence as
a consequence of evolutionary process
and in some sense the purpose of
evolution is survival and then the
this
complicated optimization objective
baked into us let's say compression
which might help us
operate in the real world and it bake
into us various reward functions yeah
and then to be clear at the moment we
are operating in the regime which is
somewhat out of distribution where the
event evolution optimized us it's almost
like love is a consequence of
cooperation that we've discovered is
useful correct in some way it's even the
case if you i just love the idea that
love is like the out of distribution
or it's not out of distribution it's
like as you that it evolved for
cooperation
yes and i believe that the cop like a in
some sense cooperation ends up helping
each of us individually so it makes
sense evolutionary and there is a in
some sense and you know love means there
is this dissolution of boundaries that
you have a shared reward function and we
evolve to actually identify ourselves
with larger groups so we we can identify
ourselves you know with a family we can
identify ourselves with a country to
such an extent that people are willing
to give away their life for country
[Music]
so there is we are wired actually even
uh
for love and at the moment i guess
the
maybe
it would be somewhat more beneficial if
you will if we would identify ourselves
with all the humanity as a whole so so
you can clearly see when people travel
around the world when they run into
person from the same country they say oh
which ctr and all this like all of a
sudden they find all these similarities
they they they find some they befriend
those folks earlier than others so there
is like a sense some sense of the
belonging and i would say i think it
would be overall good thing to the word
for people
to
move towards
i think it's even called open
individualism and move toward the
mindset of a larger and larger groups so
the challenge there
that's a beautiful vision and i share it
to expand that circle of empathy that
circle of love towards the entirety of
humanity but then you start to ask well
where do you draw the line
because why not expand it to other
conscious beings and then at the finally
for our discussion
something i think about
is why not expand it to ai systems
like we we start respecting each other
when the other the person the entity on
the other side
has the capacity to suffer because then
we develop a capacity to sort of
empathize
and so
i could see ai systems that are
interacting with humans
more and more having conscious like
displays
so like they display consciousness
through language and through other means
and so then the question is like well is
that consciousness
because they're acting conscious
and so
you know the reason we don't like
torturing animals
is because
they look like they're suffering when
they're tortured
and if ai looks like it's suffering
when it's tortured
how is that not
requiring of the same kind of empathy
from us and respect and rights
that animals do and other humans do i
think it requires empathy as well i mean
i would like
i guess us or humanity or so make a
progress in
understanding what consciousness is
because i don't want just to be speaking
about that the philosophy but rather
actually make a scientific uh to have a
like a you know there was a time that
people thought that
there is a force
of life
and
the
things that have this force they are
alive
and
i think that there is actually a path to
understand exactly what consciousness is
and
um in some sense it might require
essentially putting probes inside of a
human brain
what neuralink
does so the goal there i mean there's
several things with consciousness that
make it a real discipline which is one
is rigorous measurement of consciousness
and then the other is the engineering of
consciousness which may or may not be
related i mean you could also run into
trouble like for example in the united
states
for the department d.o.t department of
transportation and a lot of different
places put a value on human life
i think dot's
uh values nine million dollars per
person
sort of in that same way you can get
into trouble
if you put a number on how conscious a
being is
because then you can start making policy
if a cow
is uh 0.1
or like um
10 as conscious as a human then you can
start making calculations and might get
you into trouble but then again that
might be a very good way to do it
i would like uh
to move to that place that actually we
have scientific understanding what
consciousness is yeah and then we'll be
able to actually assign value and i
believe that there is even the path for
the experimentation in it so uh you know
we said that you know you could put the
probes inside of the brain there is
actually few other things that you could
do with devices like neuralink so you
could imagine that the way even to
measure if ai system is conscious
is by literally just plugging into the
brain
and i mean that that seems that's kind
of easy but the plugging into the brain
and asking person if they feel that
their consciousness expanded
this direction of course has some issues
you can say you know if someone takes a
psychedelic drug they might feel that
their consciousness expanded even though
that drug itself is not conscious
right so like you can't fully trust the
self-report of a person saying their
their consciousness is expanded or not
let me ask you a little bit about
psychedelics because uh there's been a
lot of excellent research on uh
different psychedelics psilocybin mdma
yeah even dmt
drugs in general marijuana too
uh what do you think psychedelics do to
the human mind it seems they take
the human mind to some interesting
places
is that just a little uh hack
a visual hack
or is there some profound expansion of
the mind
so let's see i i don't believe in magic
i believe in that i believe in
in science in
in causality
still let's say and then as i said like
i think that the brain
that the our subjective experience of
reality is uh
we live in the simulation run by our
brain and the simulation that our brain
runs
they can be very pleasant or very
hellish
drugs they are changing some hyper
parameters of the simulation it is
possible thanks to change of these hyper
parameters to actually look back on your
experience and even see that the given
things that we took for granted they are
changeable
so they allow to have a
amazing perspective there is also
for instance the fact that after dmt
people can see the
full movie inside of their head
gives me further belief
that the brain can generate that full
movie that the brain is actually
learning the model of reality to such
extent that it tries to predict what's
going to happen next yeah very high
resolution so it can replay realities
actually extremely high resolution
and it's also kind of interesting to me
that somehow there seems to be some
similarity between
these uh drugs and meditation itself and
i actually started even these days to
think about meditation as a psychedelic
and do you practice meditation
i i practice meditation i mean i once
few times on the
retreats and it feels after like after
second or third day of meditation
there is a there is almost like a sense
of you know tripping
what does the meditation retreat entail
so
i mean you you wake up early in the
morning and you meditate for extended
period of time
and alone
yeah so it's optimized even though there
are other people it's optimized for
isolation so you don't speak with anyone
you don't actually look into other
people's eyes
and
you know you sit on the chair and
say the passage meditation tells you uh
to focus on the breath so you try to put
all the all attention into breathing and
breathing in and breathing out
and the
crazy thing is that as you focus
attention like that
after some time
their stamps starts coming back like
some
memories that you completely forgotten
it almost feels like um that you have a
mailbox and then you
you know you are just like a archiving
email one by one
and at some point at some point there is
like a
amazing feeling
of getting to mailbox zero
zero emails and uh it's very pleasant
it's it's kind of it's it's
it's
crazy to me
that
that once
you resolve these
inner stories or like inner traumas
then once there is nothing
uh left
the default state of human mind is
extremely peaceful and happy extreme
like some sense it it feels that
it feels
at least to me in the way how when i was
a
child that i can look at any object and
it's very beautiful i have a lot of
curiosity about the simple things and
that's where usually meditation takes me
are you
what are you experiencing are you just
taking in simple sensory
information and they're just enjoying
the rawness of that sensory information
so there's no
there's no memories all that kind of
stuff you're just enjoying
being
yeah pretty much i mean still there is a
there it's it's thoughts are slowing
down sometimes they pop up but it's also
somehow the extended meditation takes
you to the space that they are
way more friendly you know way more
positive um
there is also this uh this thing that
we've actually
it almost feels that the
it almost feels that the we are
constantly getting a little bit of a
reward function and we are just
spreading this reward function on
various activities but if you stay still
for extended period of time it kind of
accumulates accumulates accumulates
and
there is a there is a sense there is a
sense that at some point it passes some
threshold and it feels as
drop is falling into kind of ocean of
love and bliss and that's like a
this is like a very pleasant and as i'm
saying okay
that corresponds to the subjective
experience
some people
uh i guess in spiritual community they
describe it that that's the reality and
i would say i believe that they're like
all sorts of subjective experience that
one can have and
i believe that for instance meditation
might take you to the subjective
experiences which are very pleasant
collaborative and i would like a word to
move toward a more collaborative uh
place
yeah i would say that's very pleasant
that i enjoy doing stuff like that i i
i wonder how that maps to your uh
mathematical model of love with the
the reward function combining a bunch of
things
it seems like our life
then is we're just we have this reward
function and we're accumulating a bunch
of stuff in it
with weights
it's like um
like multi-objective
and
what meditation is is you just remove
them remove them until the weight on one
or just a few is is very high and that's
where the pleasure comes from yeah so
something similar how i'm thinking about
this so i told you that there is like a
there is a story of who you are
and i think almost about it as a you
know text prepended to gpt
yeah and
some people refer to it as ego okay it's
like a story
who who you are okay so ego is the
prompt for gpt three gpg yes yes and
that's description of you and then with
meditation you can get to the point that
actually you experience things without
the prompt
and you experience things like as they
are you are not biased over the
description how they supposed to be
uh
that's very pleasant and then with
respect to the reward function uh it's
possible to
get to the point that the there is
dissolution of self
and therefore you can say that they are
you you're having a you're or like your
brain attempts to simulate the reward
function of everyone else or like
everything that's there is this like a
love which feels like a oneness with
everything
and that's also you know very beautiful
very pleasant at some point
you might have a lot of altruistic
thoughts during that moment and then
the self uh always comes back how would
you recommend
if somebody is interested in meditation
like a big thing to take on as a project
would you recommend a meditation retreat
how many days what kind of thing would
you recommend i think that actually
retreat is the way to go and it almost
feels that
as i said like a meditation is a
psychedelic but
when you take it in the small dose you
might barely feel it once you get the
high dose actually you're gonna feel it
um
so even cold turkey if you haven't
really seriously meditated for a
prolonged period of time just go to a
retreat yeah how many days how many days
start the weekend one weekend so like
two three days
and it's like it's interesting that
first or second day it's hard and at
some point it becomes easy
there's a lot of seconds in a day how
hard is the meditation retreat just
sitting there in a chair
so the thing is actually
it literally just depends on your uh
on death your own framing like if you
are in the mindset that you are waiting
for it to be over or you are waiting for
nirvana to happen it will be very
unpleasant yeah and in some sense even
the
difficulty it's not even in
the lack of being able to speak with
others like
you are sitting there your
legs will hurt from sitting
in terms of like the practical things do
you experience kind of discomfort like
physical discomfort of just sitting like
your your butt being numb your
legs being sore all that kind of stuff
yes you experience it and then the
they teach you to observe it
rather and it's like a the crazy thing
is
you at first might have a feeling toward
trying to escape it yeah and that
becomes very apparent that that's
extremely unpleasant and then you just
just observe it and
at some point it it just becomes uh it
just is
it's like a i remember with ilya told me
some time ago that uh you know he takes
a cold shower and his mindset of taking
a court cold shower was to
embrace suffering yeah excellent i do
the same there's the art style yes my
style
i like this
so my style is actually i also sometimes
take cold showers it is purely observing
how the water goes through my body like
a purely being present not trying to
escape from there yeah and i would say
then it actually becomes pleasant
it's not like ah well that that's
interesting um
i i'm also that mean that's that's the
way to deal with anything really
difficult especially in the physical
space is
to observe it
to say it's pleasant
it's a i would use a different word
your uh
you're accepting of the full beauty of
reality i would say because say pleasant
but yeah i mean in some sense it is
pleasant that's the only way to deal
with a cold shower
is to to become an observer and to find
joy in it
same with like really difficult physical
uh exercise or like running for a really
long time endurance events
just anytime you're exhausted any kind
of pain i think the only way to survive
it is not to resist it just to observe
it
you mentioned ilya elias discover
he's very he's our chief scientist but
also he's very close friend of mine he
co-founded open air with you i've spoken
with him a few times he's brilliant i
really enjoy talking to him
his mind just like yours works in
fascinating ways
now both of you are not able to define
deep learning simply
uh what's it like having him
as somebody you have technical
discussions with
on in space machine learning
deep learning ai but also life
what's it like when these two uh agents
get into a self-play situation in in a
room what's it like collaborating with
him
so i believe that we have
extreme uh respect to each other so
um
i mean
i love ilia's insight both like uh
i guess about consciousness uh life ai
but uh in terms of the it's interesting
to me because
you're
a brilliant
uh
thinker in the space of machine learning
like intuition like digging deep
in what works
what doesn't why it works why it doesn't
and so is ilia i'm wondering if there's
interesting
deep discussions you've had with him in
the past or disagreements that were very
productive so i can say
i also understood over the time where
are
my strengths so obviously we have plenty
of ai discussions and
um
and you know i myself have plenty of
ideas but like i consider ilya
one of the most prolific ai scientists
in the entire world
and
i think that
um i realized that maybe my super skill
is
being able to bring people to
collaborate together that i have some
level of empathy that is unique in ai
world and that might come you know from
either meditation psychedelics or let's
say i read just hundreds of books on
this topic so and i also went through a
journey of you know i develop all sorts
of algorithms so i think that
maybe i can
that's my
super human skill uh
ilia is
one of the best ai scientists but then
i'm pretty good in assembling teams and
i'm also not holding two people like i'm
growing people and then people become
managers that open yeah there's room any
of them like a research manager
so you you find
you find places where you're excellent
and and he finds like his his deep
scientific insights is where he is and
you find ways you can
the puzzle pieces fit together correct
okay you know ultimately for instance
let's say ilia he doesn't manage people
uh that's not
what he likes or so um
i i like i like hanging out with people
by default i'm an extrovert and i care
about people oh interesting okay
okay cool so that that fits perfectly
together but i i mean uh i also just
like your intuition about various
problems in machine learning
he's definitely one i really enjoy
i remember talking to him
about something i was struggling with
which is
coming up with a good model for
pedestrians
for human beings across the street in
the context of autonomous vehicles
and he immediately started to like
formulate a framework within which you
can evolve a model for pedestrians like
through self-play all that kind of
mechanisms
the depth of thought on a particular
problem especially problems he doesn't
know anything about
is fascinating to watch
it makes you realize like um
yeah the the limits of the
that the human intellect might be
limitless
or it's just impressive to see a descent
on the vape come up with clever ideas
yeah i mean so even in the space of deep
learning when you look at various people
there are people you know who
invented
some breakthroughs once but there are
very few people who did it multiple
times and you can think if someone
invented it once
that might be just a shared luck
and if someone invented it multiple
times you know if a probability of
inventing it once is one over a million
then probability of inventing it twice
or three times would be one over a
million square
or to the power of three
which which would be just impossible so
it literally means that it's it's given
that uh it's not the luck yeah and ilea
is one of these few people who um
who have uh a lot of these inventions in
his arsenal it also feels that the
now for instance if you think about
folks like gauss or euler
and you know
at first they read a lot of books
and then they did thinking and then they
figure out math
and that's how it feels with ilya yeah
you know at first he read stuff and then
like he spent his thinking cycles
and
that's a really good way to put it
when i talk to him
[Music]
i
i see thinking
he's actually thinking
like he makes me realize that there's
like deep thinking that the human mind
can do like most of us are not thinking
deeply
like you really have to put a lot of
effort to think deeply like i have to
really put myself in a place where i
think deeply about a problem it takes a
lot of effort it's like a it's like an
airplane taking off or something you
have to achieve deep focus he he's just
uh
he's what is it
his brain is like a vertical takeoff
in terms of airplane analogy so it's
interesting but
it i mean cal newport talks about this
as ideas of deep work
it's you know most of us don't work much
at all in terms of like
like deeply think about particular
problems whether it's math engineering
all that kind of stuff
you want to go to that place often and
that's real hard work and some of us are
better than others at that so i think
that the big piece has to do with
actually even engineering your
environment such that it's conducive to
that yeah so um
see both ilia and i uh on the frequent
basis we kind of disconnect ourselves
from the world in order to be able to do
extensive amount of thinking yes so ilia
usually
he just
leaves ipad
at hand he loves his ipad
and
for me i'm even
sometimes you know just going for a few
days to different location to airbnb i'm
turning off my phone
and there is no access to me yeah
and
that's extremely important for me to be
able to actually just formulate new
thoughts to do deep work rather than to
be reactive and the the older i am the
more of these like random tasks are at
hand
before i go on to that uh thread let me
return
to our friend gpt
let me ask you another ridiculously big
question
can you give an overview of what gpt 3
is
or like you say in your twitter bio gpt
n plus one
how it works
and why it works so um gpt 3 is a
humongous neural network and let's
assume that we know what is neural
network okay by the definition
and it is trained on the entire internet
and just to predict
next word so let's say it sees part of
the uh article and it the only task that
it has at hand it is to say what would
be the next word uh what would be the
next word
and it becomes uh
really exceptional at the task of
figuring out what's the next word so you
might ask
why would this be an important task why
would it be important to predict what's
the next word
and it turns out that a lot of problems
uh can be formulated
uh
as a text completion problem so gpt is
purely uh learning to complete the text
and you could imagine for instance if
you are asking a question who is a
president of united states
then gpt can give you an answer to it
it turns out that many more things can
be formulated this way you can format
text
in the way that you have sentence in
english
you make it even look like a some
content of a website uh elsewhere which
would be teaching people how to
translate things between languages so it
would be en colon
text in english fr colon and then you uh
and then you ask people and then you ask
model to to continue and it turns out
that the such a model is predicting
translation from english to french the
crazy thing is that
this model
can be used for way more sophisticated
tasks so you can format text such that
it looks like a conversation between two
people and that might be a conversation
between you and elon musk and because
the model read all the texts about elon
musk
it will be able to predict elon musk
words as it would be elon musk it will
speak about colonization of
mars
about sustainable future and so on and
it's also possible to
to even give arbitrary personality to
the model you can say here is a
conversation with a friendly ai bot
and the model uh will complete the text
as a friendly ai bot so i mean
how do i express how
amazing this is so
just to clarify
a conversation generating a conversation
between me and elon musk
it wouldn't just generate good
examples of what elon would say
it would get the syntax all correct so
like interview style you would say like
elon colon and lex con like it it's not
just like uh
inklings of
semantic
correctness
it's like the whole thing grammatical
syntactic
semantic
it's just really really impressive
uh generalization
yeah i mean i also want to you know
provide some caveats so it can generate
few paragraphs of coherent text but as
you go to uh longer pieces it actually
goes off the rails okay if you would uh
try to write a book it won't work out uh
this way what way does it go off the
rails by the way is there interesting
ways in which it goes off the rails like
what falls apart first so the model is
trained on the all the existing data
that is out there which means that it is
not trained on its own mistakes so for
instance if it would make a mistake then
uh i kept so to give give you an example
so let's say i have a conversation with
a
model pretending that is elon musk
and then i start putting some i'm start
actually making up things which are not
factual
um i would say like twitter
but i gotcha sorry yeah um okay
i don't know i would say that elon is my
wife
and the model will just
keep on carrying it on and as if it's
true
yes and in some sense if you would have
a normal conversation with elon he would
be what the fuck
yeah there would be some feedback
between so the the model is trained on
things that humans have written but
through the generation process there's
no human in the loop feedback correct
that's fascinating makes sense so it's
magnified it's like the errors get
magnified and magnified right and it's a
it's also interesting
i mean first of all humans have the same
problem it's just that we
uh we make
fewer errors and magnify the errors
slower i think that actually what
happens with humans is if you have a
wrong belief about the world as a kid
then very quickly you will learn that
it's not correct because you are
grounded in reality and you are learning
from your new experience yes
but do you think the model can correct
itself too
it through the power of the
representation
and so the absence
of
elon musk being your wife
information on the internet want to
correct itself
there won't be examples like that so the
errors would be subtle at first
saddle at first and in some sense
you can also say that the data that is
not out there is the data which would
represent how the human learns
that's an a and and maybe model would be
trained on such a data then it would be
better off how intelligent is gpt 3 do
you think like when you think about the
nature of intelligence
it seems exceptionally
impressive
but then if you think about the big agi
problem is this footsteps along the way
to agi
so
let's see seems that intelligence itself
is there are multiple axis of it and
i would expect that the
the systems that we are building they
may end up being super human on some
axis
and sub human on some other axis it
would be surprising to me on all axis
simultaneously they would become
superhuman
of course people ask this question is
gpt a spaceship that
that would take us to moon or are we
putting a building a ladder to heaven
that we are just building bigger and
bigger ladder and we don't know in some
sense
uh which one of these two which one is
better
i'm trying to i like stairway to heaven
that's a good song so i'm not exactly
sure which one is better but you're
saying like the the spaceship to the
moon is actually effective
correct so people who criticize gpt yeah
they say jarga is just
building a
taller a ladder
and it will never reach the moon
and
at the moment i would say the way i'm
thinking is this like a scientific
question
and i'm also in heart i'm a builder
creator and like i'm thinking let's try
out let's see how far it goes and so far
we see constantly that there is a
progress yeah
so what do you think
gpt4
gpt5 gpt n plus one
will uh
there'll be a phase shift like a
transition to a to a place where
we'll be truly surprised then again like
gpt3 is already very like truly
surprising the people that criticize
gpg3 as it's there as a what is it
ladder to heaven
i think too quickly get accustomed to
how impressive it is that the prediction
of the next word can achieve such
depth of semantics accuracy of syntax
grammar and semantics
um do you do you think
gpt four and five and six will continue
to surprise us
i mean definitely there will be more
impressive models there is a question of
course if there will be a phase shift
and
the also even the way i'm thinking about
the about these models is that
when we build these models
you know we see some level of the
capabilities but we don't even fully
understand everything that the model can
do and actually one of the best things
to do is to
allow other people to probe the model to
even see what is possible
hence the
using gpg as an api
and opening it up to the world yeah i
mean so when i'm thinking from
perspective of
there like a obviously various people
are that have concerns about agi
including myself
and then when i'm thinking from
perspective what's the strategy even to
deploy these things to the world
the
the one strategy that i have seen many
times working is the iterative
deployment that you deploy
um slightly better versions and you
allow other people to criticize you so
you actually are tried out you see where
are their fundamental issues and it's
almost you don't want to be in that
situation that you are holding into
powerful system and there's like a huge
overhang then you deploy it and it might
have a random chaotic impact on the
world so you actually want to be in the
situation that they are gradually
deploying systems
i asked this question of ilio let me ask
you
you this question
i've been reading a lot
about stalin and power
if you're in possession of a system
that's
like agi that's exceptionally powerful
do you think your character integrity
might become corrupted
like famously power corrupts and
absolute power corrupts absolutely
so i believe that
you want at some point to
work toward distributing the power
i think that
you want to be in the situation
that actually agi is not controlled by a
small number of people
but
essentially
by a larger collective so the thing is
that requires a george washington style
move
in the ascent to power there's always a
moment when somebody gets a lot of power
and they have to have the integrity
and uh the moral compass to give away
that power
that humans have been
good and bad throughout history at this
particular step and i wonder
i wonder we like blind ourselves in uh
for example
between nations a race
uh towards uh
yeah ai race between nations we might
blind ourselves and justify to ourselves
the development of ai without
distributing the power
because we want to defend ourselves
against china against russia that kind
of that kind of logic
and
i wonder
how we um
how we design governance mechanisms that
um prevent us from
becoming power hungry and in the process
destroying ourselves
so let's see i have been thinking about
this topic quite a bit but i also want
to admit that
uh once again i actually want to rely
way more on sam outman on it hero than a
heroed an excellent block
on how even to distribute wealth
and his proper he proposed in his block
to tax
equity of the companies rather than
profit and to distribute it and this is
this is an example of
uh washington move
i guess i personally have insane trust
in some
he already spent plenty of money running
a
universal basic income
project
that like gives me i guess
maybe some level of trust to him but i
also
i guess
love him as a friend yeah
i wonder because we're sort of summoning
a new set of technologies
i wonder if we'll be
cognizant like you're describing the
process of open ai but it could also be
at other places like in the us
government right
both china and the us are now
full steam ahead on autonomous weapons
systems development
and that's really worrying to me because
in the framework of something being
a national security danger or military
danger you can do a lot of pretty dark
things
that blind our moral compass
and i think ai will be one of those
things
in some sense the the mission
and the work you're doing at openai
is like the counterbalance to that so
you want to have more open ai and less
autonomous weapon systems i i i like
these statements like to be clear like
this interesting and i'm thinking about
it myself but uh
this is a place that i i okay
i put my trust actually
in some hence because it's extremely
hard for me to reason about it yeah i
mean one important statement to make is
um
it's good to think about this yeah no
question about right no question even
like
low-level quote-unquote engineer
like there's such a
i remember i i programmed a car uh our
rc car
they went really fast like 30 40 miles
an hour
and i remember i was like sleep deprived
so i programmed it
pretty crappily and it like uh
the the code froze so it's doing some
basic computer vision and it's going
around on track but it's going full
speed
and uh there's a bug in the code that uh
the car just
went it didn't turn it went straight
full speed and smashed into the wall i
remember thinking
the seriousness with which you need to
approach the design of artificial
intelligence systems and the programming
of artificial intelligence systems
is high because the consequences are
high like that little car smashing it to
the wall
for some reason i immediately thought of
like an algorithm that controls nuclear
weapons
having the same kind of bug and so like
the lowest level engineer and the ceo of
a company all need to have the
seriousness
in approaching this problem and thinking
about the worst case consequences so i
think that is true i mean the
what i also recognize in myself and
others even asking this question is that
it evokes a lot of fear
and fear itself ends up being actually
quite debilitating
the place where i arrived at the moment
might sound cheesy or so but it's almost
to
build things out of love rather than
fear yeah
i can focus on how
i can you know maximize the value how
the systems that i'm building might be
uh
useful
i'm not saying that the fear doesn't
exist out there and like it totally
makes sense to minimize it
but i don't want to be working because
uh i'm scared i want to be working out
of passion out of curiosity out of the
you know looking forward for the
positive future
with uh
the definition of love arising from a
rigorous practice of empathy so not just
like your own conception of what is good
for the world but uh always listening to
others
correct like at the love where i'm
considering reward functions of others
others
to infil limit to infinity is like a sum
like one to n where n is uh seven
billion or whatever it is not not
projecting my reward functions on others
yeah exactly
okay
can we just take a step back to
something else super cool which is uh
opening up codex
can you give an overview of what
open-air codecs and github co-pilot is
how it works
and why the hell it works so
well so with gpd3 we noticed that the
system
um you know that system training all the
language out there started having some
rudimentary coding capabilities so we're
able to ask it you know to
implement addition function between two
numbers and indeed it can write python
or javascript code for that and then we
thought um we might as well just go full
steam ahead and try to create a system
that is actually good
at what we are doing every day ourselves
which is programming
we optimize models
for proficiency in coding we actually
even created models that both have a
comprehension of language and code
and codex is api for these models so
it's first pre-trained on language
and then
i don't know if you can say fine-tuned
because there's a lot of code
but it's language and code it's language
and code
it's also optimized for various things
like let's say low latency and so on
codex is the api that's similar to gpd3
we expect that there will be
proliferation of the potential products
that can use coding capabilities and i
can
i can speak about it in a second
compiled is the first product
and developed by github so as we're
building uh models we wanted to make
sure that these models are useful
and we work together with github on
building the first product co-pilot is
actually as you code it suggests you
code completions and we have seen in the
past they're like a various tools that
can suggest how to like a few characters
of the code or the line of code the the
thing about copilot is it can generate
10 lines of code you
it's often the way how it works is you
often write in the comment what you want
to happen because
people in comments they describe what
happens next so
um these days when i code instead of
going to google to search
for the appropriate code to solve my
problem i say oh for this array could
you smooth it and then you know it
imports some appropriate libraries and
say it uses numpy convolution or so i
that i was not even aware that exists
and it does the appropriate thing
um so you you write a comment maybe the
header of a function and it completes
the function
of course you don't know what is the
space of all the possible
small programs it can generate
what are the failure cases how many edge
cases how many subtle
errors there are how many big errors
there are it's hard to know but the fact
that it works at all on in a large
number of cases is incredible it's like
a
it's a kind of search engine
into code that's been written on the
internet
correct so for instance
when you search things online then
usually you get to the
some particular
case like if you go to stack overflow
people describe that one particular
situation uh and then they seek for a
solution but in case of uh co-pilot it's
aware of your entire context and in
contexts oh these are the libraries that
they are using that's the set of the
variables that is initialized and on the
spot it can actually tell you what to do
so the interesting thing is
and we think that the copilot is one
possible product using codex but there
is a place for many more so
internally we tried out you know to
create other fun products so it turns
out that a lot of tools out there
let's say google calendar or microsoft
word or so
they all have uh internal api to build
plugins around them
so there is a way in the sophisticated
way to control calendar or microsoft
word today if you want
if you want more complicated behaviors
from these programs you have to add a
new button for every behavior
but it is possible to use codex and
tell for instance to calendar
could you schedule an appointment with
blacks
next week after 2 pm and either writes
corresponding piece of code
and that's the thing that actually you
want so interesting so
what you figure out is there's a lot of
programs with which you can interact
through code
and so there you can generate that code
from natural language
that's fascinating and that's somewhat
like also closest to
uh what was the promise of siri or alexa
yeah so previously all these behaviors
they were had
hard coded yeah and it seems that codex
on the fly can pick up the api of let's
say given software yeah and then it can
turn the language into use of this api
without hard coding you can find it can
translate to machine language correct it
to uh so for example this would be
really exciting for me like for um adobe
products like photoshop
uh which is the i think actionscript i
think there's a scripting language that
communicates with them same with
premiere
and you could imagine that that allows
event to
do coding by voice on your phone
so for instance in the past okay as of
today i'm not editing word documents on
my phone because it's just the keyboard
is too small but if i would be able to
tell
to my phone you know uh make the header
large and then move the paragraphs
around and it does actually what i want
so i can tell you one more cool thing or
even how i'm thinking about codex
so if you look actually at the evolution
of
of computers
we started with very primitive
interfaces which is a punch card and
punch card essentially
you make a holes in the
in the plastic card to indicate zeros
and ones
and
during that time there was a small
number of specialists who were able to
use computers and by the way people even
suspected that there is no need for many
more people to use computers
but then we moved from punch cards to
at first assembly then c
and these programming languages they
were slightly higher level they allowed
many more people to code and they also
led to more of a proliferation of
technology and
you know further on there was a jump to
say from c plus plus to java and python
and every time it has happened
more people are able to code and we
build more technology and it's even you
know
hard to imagine now if someone will tell
you that you should write code in
assembly instead of let's say python or
or
or java or javascript and codex is yet
another step toward kind of bringing
computers closer to humans such that you
communicate with a computer
with your own language
rather than with a specialized language
and
i think that it will lead to
an increase of number of people who can
code
yeah and then and the kind of
technologies that those people will
create is
like it's innumerable it could you know
it could be a huge number of
technologies we're not predicting at all
because that's less and less requirement
of uh
having a technical mind
a programming mind you're not opening it
to the world of
um
other kinds of minds creative minds
artistic minds all that kind of stuff i
would like for instance biologists who
work on dna to be able to program and
not to need to spend a lot of time uh
learning it and i i believe that's a
good thing to the word and i would
actually add out that so at the moment
i'm a managing codex team and also
language team and i believe that there
is like a plenty of brilliant people out
there
and they should apply
oh okay yeah awesome so what's the
language in the codexes so those are
kind of
they're overlapping teams so it's like
gpt the raw language and then the codex
is like applied to programming
correct and they are quite intertwined
there are many more teams involved
making these uh
models
extremely efficient and deployable for
instance there are people who are
working to you know
make our data centers uh amazing or
there are people who work on pro putting
these models into production
or uh
or even pushing it at the very limit of
the scale
so all aspects from from the
infrastructure to the actual machine
learning so i'm just saying that
multiple teams while the
and the team working on codex and
language uh i guess i'm i'm directly
managing them i would like i would love
to hire yeah if you're interested in
machine learning
this is probably one of the most
exciting uh problems and like systems to
be working on because it's actually it's
it's pretty cool like what what uh the
program synthesis like generating of
programs is very interesting very
interesting problem that has echoes of
reasoning and intelligence in it
it and i think there's a lot of
fundamental questions that you might be
able to sneak
sneak up to by generating programs yeah
the one more exciting thing about the
programs is that so i said that the
um you know the in case of language that
one of the troubles is even evaluating
language so when the things are made up
you you need somehow
either a human
to say that this doesn't make sense or
so in case of program there is one extra
level that we can actually execute
programs and see what they evaluate to
so that process might be somewhat
more automated in in order to improve
the uh qualities of generations and
that's not saying so like the wow that's
really interesting so for the language
that you know the simulation to actually
execute it as a human mind yeah for
programs there is a there is a computer
on which you can evaluate it
wow
that's a
brilliant little
insight that the thing compiles and runs
that's first
and second you can evaluate on a like do
automated unit testing
and in some sense
it seems to me that we will be able to
make a tremendous progress you know
we are in the paradigm that there is
way more data and there is like a
transcription of millions of uh of uh
software engineers yeah
yeah
so
i mean you just me because i was going
to ask you about reliability the thing
about programs is you don't know if
they're going to
like a program that's controlling a
nuclear power plant has to be very
reliable so i i wouldn't start with
controlling nuclear power plant can i be
one day but that that's not actually
that's not on the current roadmap that's
not that's step one and you know it's
the russian thing you just want to go to
the most powerful destructive thing
right away
run by javascript but i got you so it's
a lower impact but nevertheless what
you're making me realize
it is possible to achieve some levels of
reliability by doing testing
and i thought you could imagine that
them you know maybe there are ways for a
model to
write even code for testing itself and
so on
and there exists a ways to create the
feedback loops that the model could keep
on improving
by writing programs that generate tests
for the instance for instance
and that's how we get consciousness
because it's meta compression that's
what you're going to write that's the
comment that's the prompt that generates
consciousness
compressor of compressors you just write
that
do you think the code that generates
consciousness would be simple
so
let's see i mean ultimately the core
idea behind will be simple but there
will be also decent amount of
engineering
involved like in some sense
it seems that you know spreading these
models on many machines
and it's not that trivial yeah and
we find all sorts of innovations that
make our models more efficient
i believe that
first models
that i guess are conscious are like a
truly intelligent they will have all
sorts of
tricks
but then again there's uh
which is certain argument that maybe
the tricks are temporary thing yeah they
might be temporary things and in some
sense it's also even important
to um
to
know that even the cost of a trick so
sometimes people are eager to put the
trick
while forgetting that there is a cost of
maintenance
or like a long-term cost long-term cost
or maintenance or maybe even
flexibility of code to actually
implement new ideas so even if you have
something that gives you 2x but it
requires you know 1000 lines of code i'm
not sure if it's actually worth it so in
some sense you know if it's five lines
of code and 2x i would take it
and and we we we see many of this but
also you know that requires some level
of
i guess lack of attachment to code that
we are willing to remove it yeah
so you led the open ai robotics team can
you give an overview of of the cool
things you're able to accomplish what
are you most proud of
so when we started robotics we knew that
actually reinforcement learning works
and it is possible to
solve very complicated problems
like for instance alphago is an evidence
that it is possible to to build
superhuman and gold players dota 2 is a
an evidence that is possible to
build superhuman uh
agents playing dota so i asked myself a
question you know what about robots out
there could we train machines to solve
arbitrary tasks in the physical world
our approach was i guess let's pick a
complicated problem that
if we would solve it that means that we
made some uh significant progress in the
domain and then we went after the
problem
so um we noticed that actually the
robots out there they are kind of at the
moment optimized per task so you can
have a robot that it's like if you have
a robot opening a battle it's very
likely that the end factor is a battle
opener
and
and in some sense that's a hack to be
able to solve a task which makes any
task easier and um ask myself so what
would be a robot that can actually solve
many tasks yeah and we conclude that
that
like a human hands have such a quality
that indeed they are you know you have
five kind of tiny arms attached
individually they can manipulate
pretty broad spectrum of objects so we
went after a single hand like a trying
to solve rubik's cube single-handed we
picked this task because we thought that
there is no way to
harcode it and it's also we picked the
robot on which it would be hard to
hardcode it and
we went after the solution such that
it could generalize to other problems
and just to clarify it's
one robotic hand solving the rubik's
cube the hard part isn't the solution to
the rubik's cube is the manipulation of
the uh of like having it not fall out of
the hand having it
use the uh
five baby arms
to uh what is it like rotate different
parts of the rubik's cube to achieve the
solution correct yeah so what uh what
was the hardest part about that
what was the approach taken there what
are you most proud of obviously we have
like a strong belief in reinforcement
learning
and uh
you know one path it is to do
reinforcement learning the real world
other path is to
the simulation in some sense the
tricky part about the real world is at
the moment our models they require a lot
of data there is essentially no data
and i did we decided to go through the
path of the simulation and in simulation
you can have infinite amount of data the
tricky part is the fidelity of the
simulation and also can you in
simulation represent everything that you
represent otherwise in the real world
and you know it turned out that uh
that you know because there is lack of
fidelity it is possible to that what we
what we
arrived at is training a model that
doesn't solve one simulation but it
actually solves the
entire range of simulations which uh
vary uh in terms of like uh what's the
exactly the friction of that cube or the
weight or so
and the
single ai that can solve all of them
ends up working well with the reality
how do you generate the different
simulations so
you know there's plenty of parameters
out there we just pick them randomly and
and in simulation model just goes for
thousands of years and keeps on solving
rubik's cube in each of them and the
thing is the neural network that we used
it has a memory
and as it presses for instance the side
of the of the cube it can sense oh
that's actually this side was
uh difficult to press i should press it
stronger and throughout this process
kind of
learns even how to
how to solve this particular instance of
the rubik's cube back even mass it's
kind of like a
you know sometimes when you
go to a gym and after
after bench press you
try to lift the
and you kind of forgot uh and and your
hand goes like yeah right away because
kind of you got this to maybe different
weight yeah and it takes a second to
adjust yeah
and this kind of of a memory that model
gained through the process of
interacting with the cube in the
simulation
i appreciate you speaking to the
audience with the bench press all the
bros in the audience
probably working out right now there's
probably somebody listening to this
actually doing bench press
so maybe
uh put the bar down and pick up the
water bottle and you'll know exactly
what uh what check is talking about okay
so what uh
what was the hardest part of getting the
whole thing to work so the hardest part
is
at the moment when it comes to a
physical world
when it comes to robots
they require maintenance it's hard to
replicate a million times it's
it's also it's hard to replay things
exactly
i remember this situation that
one guy
at our company he had like a model that
performs way better than other models in
solving rubik's cube and
you know we kind of didn't know what's
going on
why it's that
and
it turned out
that you know he was running it from his
laptop that had better cpu
or
uh or better or maybe local gpu as well
and uh because of that there was less of
a latency and the model was the same
and that actually
made solving rubik's cube more reliable
so in some sense there might be some
saddlebacks like that when it comes to
running things in the real world
even hinting on that
you could imagine that the initial
models you would like to have models
which are insanely huge neural networks
and you would like to give them even
more time for thinking
and when you have these real-time
systems
then
you might be constrained actually by the
amount of latency
and
ultimately i would like to build the
system that it is
worth for you to wait five minutes
because it gives you the answer
that you are willing to wait for five
minutes so latency is a very unpleasant
constraint underwish to operate correct
and also there is actually one more
thing which is tricky about robots
there is actually
no not much data so the data that i'm
speaking about would be a data of
first person experience from the robot
and like a gigabytes of data like that
if we would have gigabytes of data like
that of robot solving various problems
it would be very easy to make a progress
on robotics and you can see that in case
of text or code there is a lot of data
like a first person perspective data on
the writing code
yeah so you had this
you mentioned this really interesting
idea that
if you were to build like a successful
robotics company so open as mission is
much bigger than robotics this is one of
the
one of the things you've worked on
but if it was a robotics company they
you wouldn't so quickly dismiss
supervised learning i correct that you
would build a robot
that
was perhaps one like
um an empty shell like dumb and they
would operate under tele operation
so you would invest
that's just one way to do it invest in
human super like direct human control of
the robots as it's learning and over
time add more and more automation
that's correct so let's say that's how i
would build a robotics company today
if i would be building a robotics
company which is you know spent 10
million dollars or so
recording human trajectories controlling
a robot after you find
a thing that the robot should be doing
that there's a market fit for like that
you can make a lot of money with that
product correct correct yeah
so
i would record data and then i would
essentially train supervised learning
model on it
that might be the path today
long term i think that actually what is
needed is to train powerful models over
video
so
um you have seen maybe a models that can
generate images like dali
and people are looking into models
generating videos they're like various
algorithmic questions even how to do it
and it's unclear if there is enough
compute for this purpose
but
i i suspect that the models that which
would have a
level of understanding of video same as
gpt has the level of understanding of
text
could be used
to train robots to solve tasks they
would have a lot of common sense
if one day
i'm pretty sure one day
there will be a robotics company
by robotics company i mean the primary
source of income is is from robots
that is worth over
1 trillion dollars
what do you think that company will do i
think self-driving cars no
it's interesting because my mind went to
personal robotics robots in the home
it seems like there's much more market
opportunity there
i think it's very difficult to achieve
i mean this this
this might speak to something important
which is i understand self-driving much
better than understand robotics in the
home so i understand how difficult it is
to actually solve self-driving
to uh to a level not just the actual
computer vision and the control problem
and just the basic problem self-driving
but
creating a product
that would undeniably
be um
that will cost less money like it will
save you a lot of money like orders the
magnitude less money that could replace
uber drivers for example so car sharing
that's autonomous that creates
a similar or better experience in terms
of how quickly you get from a to b or
just whatever the the pleasantness of
the experience
the efficiency of the experience the
value of the experience and at the same
time the car itself costs cheaper
i think that's very difficult to achieve
i think there's a lot more
um low hanging fruit in the home
that that could be i also want to give
you perspective on
like how challenging it would be at home
or like it maybe kind of depends on the
exact problem that you'd be solving okay
if we are speaking about these robotic
arms
and hence
these things they cost tens of thousands
of dollars or maybe 100k
and
you know maybe obviously maybe there
would be economy of scale these things
would be cheaper
but actually for any household to buy
the price would have to go down to maybe
thousand bucks
yeah i personally think
that uh
so self-driving car it provides a clear
service i don't think robots in the home
they'll be a trillion dollar company
will just be all about service
meaning it will not necessarily be about
like a robotic arm that
helps you i don't know open a bottle
or wash the dishes or
any of that kind of stuff it has to be
able to take care of that whole the
therapist thing you mentioned
i i think that's um of course there's a
line between what is a robot and what is
not
like doesn't really need a body but you
know some
uh ai system with some embodiment i
think
so the tricky part when you think
actually what's the difficult part is
um
when the robot has
like when there is a diversity of the
environment with which the robot has to
interact that becomes hard so you know
on one spectrum you have
industrial robots as they are doing over
and over the same thing it is possible
to some extent to prescribe the
movements and with very small amount of
intelligence the the movement can be
repeated millions of times um the it
there are also you know various pieces
of industrial robots where it becomes
harder and harder like for instance in
case of tesla it might be a matter of
putting a a rack inside of a car
and you know because the rack kind of
moves around it's it's not that easy
it's not exactly the same every time it
ends up being the case that you need
actually humans to do it
and while you know welding cars together
it's a very repetitive process
and then in case of self-driving itself
the difficulty has to do with the
diversity of the environment but still
the car itself and the problem that you
are solving is
you try to avoid even interacting with
things you are not touching anything
around because touching itself is hard
and then if you would have in the home
uh robot that you know has to touch
things and like if these things they
change the shape if there is a huge
variety of things to be touched then
that's difficult if you are speaking
about the robot which there is you know
head that is smiling in some way with
cameras that it doesn't you know touch
things that's relatively simple
okay so
to both agree and to push back
so you're referring to touch like
soft robotics like the actual touch
but
i would argue that you could formulate
just basic interaction
between um like non-contact interaction
is also a kind of touch and that might
be very difficult to solve that's the
basic this not disagreement but that's
the basic open question to me
with self-driving cars and disagreement
with elon which is how much interaction
is required to solve self-driving cars
how much touch is required you said that
in your intuition touch is not required
and my intuition to create a product
that's compelling to use you're going to
have to uh
interact with pedestrians not just avoid
pedestrians but interact with them
when we drive around in major cities
we're constantly threatening everybody's
life with our movements
and that's how they respect us there's a
game theoretically going on with
pedestrians
and
i am afraid you can't just
formulate
autonomous driving as a collision
avoidance problem so i i think it goes
beyond like a collision avoidance is the
first order approximation
but then at least in case of tesla they
are gathering data from people driving
their cars
and i believe that's an example of
supervised learning data that they can
train their models uh on and they are
doing it
which you know can give the model this
like
another level of
of a behavior that is needed to actually
interact with the real world yeah it's
interesting how much data
is required to achieve that
um
what do you think of the whole tesla
autopilot approach the computer vision
based approach with multiple cameras and
there's a data engine it's a multi-task
multi-headed neural network and it's
this fascinating process of uh similar
to what you're talking about
with the the robotics approach
uh which is you know you deploy neural
network and then there's humans that use
it
and then it runs into trouble in a bunch
of places and that stuff is sent back so
like
the deployment discovers a bunch of edge
cases and those edge cases are sent back
for supervised annotation thereby
improving the neural network and that's
deployed again
it goes over and over until the the
network becomes really good at the task
of driving becomes safer and safer what
do you think of that kind of approach to
robotics i believe that's the way to go
so in some sense even when i was
speaking about you know collecting
trajectories from humans that's like a
first step and then you deploy the
system and then you have humans revising
the
all the issues and in some sense
like this approach converges to system
that doesn't make mistakes because for
the cases where there are mistakes you
got their data how to fix them and the
system will keep on improving so there's
a very
to me difficult question of how hard
that you know how long that converging
takes how hard it is
uh the other aspect of autonomous
vehicles this probably applies to
certain robotics applications
is society right they put
as as the quality of the system
converges
so one there's a human factors
perspective of psychology of humans
being able to supervise those uh even
with teleoperation those robots and the
other society willing to accept robots
currently society is much harsher on
self-driving cars than it is on human
driven cars in terms of the expectation
of safety so the bar is set much higher
than for humans and we're so if there's
a death in an autonomous vehicle that's
seen as much more
much more dramatic than a death in a
human driven vehicle
part of the success of deployment of
robots is figuring out how to make
robots part of society both on the just
the human side on the media journalist
side and also on the policy government
side and that seems to be uh
maybe you can put that into the
objective function to optimize
but that is that is definitely um
a tricky one and i wonder if that is
actually the trickiest part for
self-driving cars or any system that's
safety critical
it's not the algorithm it's the society
accepting it
yeah i i would say
i believe that
the part of the process of deployment is
actually showing people that the given
things can be trusted yeah and you know
trust is also like a glass that is
actually really easy to crack it yeah
and damage it and
i think that's actually
very common with uh
with innovation
that there is some resistance toward it
yeah
and
it's just the natural progression so in
some sense people will have to keep on
proving that indeed these systems are
worth being used and i would say
i also found out that
often the best way to convince people
is by letting them experience it yeah
absolutely that's the case for tesla
autopilot for example
that's the case with uh yeah with
basically robots in general it's it's
kind of funny to hear people talk about
robots like
there's a lot of fear
even like legged robots but when they
actually interact with them
there's joy
i love interacting with them and the
same with the car
with the robot
if it starts being useful
i think people immediately understand
and if the product is designed well they
fall in love you're right
it's actually even similar when i'm
thinking about co-pilot the github
co-pilot there was a spectrum of
responses that people had and uh
ultimately
uh
the important piece was to let people
try it out and then many people just
loved it especially like
programmers yeah programmers but like
some of them you know they came with a
fear
yeah but then you try it out and you
think actually that's cool okay and you
know you can try to resist the same way
as you know you could resist moving from
punch cards to let's say
c plus or so
and
it's a little bit futile
so we talked about generation program
generation of language
even
self-supervised learning in the visual
space for robotics and then
reinforcement learning what do you and
like this whole beautiful
spectrum of ai
do you think is a good
benchmark a good test to strive for
to achieve intelligence that's a strong
test of intelligence you know it started
with alan turing and the touring test
maybe you think natural language
conversation is a good test
so you know it would be nice if for
instance machine would be able to solve
riemann hypothesis in math
that would be i think that would be very
impressive so theorem proving
is that to you proving theorems is a
good
oh oh like one thing that the machine
did you would say damn
exactly
okay
that would be quite
quite impressive i mean the the tricky
part about the benchmarks is
um you know as we are getting closer
with them we have to invent new
benchmarks there is actually no ultimate
benchmark out there yeah see my thought
with the riemann hypothesis would be
the moment the machine proves it would
say okay well then the problem was easy
that's what happens and i mean in some
sense um that's actually what happens
over the years in ai that like uh
we get used to things very quickly you
know something i talked to rodney brooks
i don't know if you know that is
he called alpha zero homework problem
because he was saying like there's
nothing special about it it's not a big
leap and i i didn't
well he's coming from one of the aspects
that we referred to as he was part of uh
the founding of irobot which deployed
now tens of millions of robot in the
home so
if you see robots
that are actually in the homes of people
as the legitimate
instantiation of artificial intelligence
then yes maybe an ai that plays a silly
game like going chess is not a real
accomplishment but to me it's it's a
fundamental leap but i think we as
humans then say okay well then
that uh that game of chess or go wasn't
that difficult compared to the thing
that's currently unsolved so my
intuition is that
from perspective of the evolution of
you know these ai systems we'll at first
see the tremendous progress in digital
space and the you know the main thing
about digital space is also that you can
everything is that there is a lot of
recorded data plus you can very rapidly
deploy things to billions of people
while in case of uh physical space the
deployment part takes multiple years you
have to manufacture things and
you know delivering it to actual people
it's very hard
so i'm expecting that the first and the
prices in digital space
of goods they would go you know down to
the let's say marginal costs are to zero
and also the question is how much of our
life will be in digital because it seems
like we're heading towards more and more
of our lives being in the digital space
so like
innovation in the physical space might
become less and less significant like
why do you need to drive anywhere
if most of your life is spent in virtual
reality i still would like you know to
at least at the moment my impression is
that i would like to have a physical
contact with other people and that's
very important to me and we don't have a
way to replicate it in the computer it
might be the case that over the time it
will change
like in 10 years from now why not have
like an arbitrary infinite number of
people you can interact with some of
them are real some are not
with uh arbitrary characteristics that
you can define based on your own
preferences i think that's maybe where
we are heading and maybe i'm resisting
the future yeah
i'm telling you
i if i got to choose
if i could live in elder scrolls skyrim
versus the real world i'm not so sure i
would stay with the real world
yeah i mean the question is so will vr
be sufficient to get us there or do do
you need to you know plug electrodes in
the brain
and it would be nice if these electrodes
wouldn't be invasive yeah
or at least like provably
non-destructive
but in in the digital space do you do
you think we'll be able to solve the
touring test the spirit of the touring
test which is do you think we'll be able
to
achieve compelling natural language
conversation between people like have
friends that are ai systems on the
internet
i thought i think it's doable do you
think the current approach of gbt
will take us there so there is you know
the the part of at first learning all
the content out there and i i think that
steel system should keep on learning as
it speaks with you yeah
and i think that should work the
question is how exactly to do it and you
know obviously
we have people at open air asking these
questions
and kind of at first pre-training on all
existing content is like a backbone and
it's a decent backbone
do you think ai
needs a body
connecting to our robotics question
to uh truly connect with humans or can
can most of the connection be in the
digital space
so let's see
we know that there are people who met
each other online and they felt in love
yeah
so it seems that it's conceivable to
establish connection which is purely
through internet
and of course it might be more
compelling than more modalities you add
so it would be like you're proposing
like a tinder but for ai
are you like swipe right and left and
half the systems are ai and the other is
uh
humans and you don't know which is which
that would be ours that would be our
formulation of touring test the the
moment ai is able to achieve more swipe
right or left whatever
the the moment is able to be more
attractive than other humans
it passes the torrent test then you
would pass the turing test in
attractiveness that's right well no like
attractiveness just declare conversation
not just visual right it's also
attractiveness
with wit and humor and uh whatever
whatever makes conversations pleasant
for humans
okay all right um
so so you're saying uh it's possible to
achieve in the digital space in some
sense i would almost ask that question
why wouldn't that be possible
right
well
i have this argument with my dad all the
time he thinks that touch and smell are
really important
so they can be very important and i'm
saying the initial systems they won't
have it
still i wouldn't like their people being
born
without these senses
and
you know i believe that they can still
fall in love and have meaningful life
yeah i wonder if it's uh possible to go
close to all the way by just training on
transcripts of conversations
like i wonder how far that takes us so i
i think that actually still you want
images like i would like so i don't have
kids but like i could imagine the
having ai tutor it has to see you know
kids
drawing some pictures on their paper
and and also facial expressions and all
that kind of stuff we use uh dogs and
humans use their eyes and
uh to communicate with each other i
think this that's that's a really
powerful mechanism of communication body
language too
that uh words are much uh lower
bandwidth and for body language we still
you know we kind of have a system that
displays an image
of its or facial expression on the
computer it doesn't have to move you
know mechanical pieces or so so i think
that uh you know there is like kind of a
progression you can imagine that text
might be the simplest to tackle
but
this is not a complete
human experience at all you expand it to
let's say
images both for input and output and
what you describe is actually the final
i guess frontier what makes us human the
fact that we can touch each other or
smell or so and it's the hardest from
perspective of data and deployment
and
i okay i believe that these things might
happen gradually
are you excited by that possibility this
particular application of
human to ai
friendship and interaction so let's see
like would you uh do you look forward to
a world you you said you're living with
a few folks and you're very close
friends with them
do you look forward to a day where one
or two of those friends are ai systems
so if the system would be truly wishing
me well
rather than being in the situation that
it optimizes for my time to interact
with the system
the line
between those is it's a gray
it's a gray area i i think that's the
distinction between
love and possession and these things
they might be often correlated for
humans but it's it like like a you you
might find that there like some friends
with whom you haven't spoken for months
yeah and then you know you pick up the
phone it's as the time hasn't passed
they are not holding to you
and i will i wouldn't like to have ai
system that you know it's
it's
trying to convince me to spend time with
it i would like the system to optimize
for what i care about and help me
in achieving my own goals
but there's some i mean i don't know
there's some manipulation there's some
possessiveness there's some insecurities
this fragility all those things are
necessary
to form a close friendship over time to
go through some dark shit together some
bliss and happiness together i feel like
there's a lot of greedy self-centered
behavior within that process
my intuition but i might be wrong is
that
human computer interaction doesn't have
to go through uh
computer being greedy possessive and so
on it is possible to train systems maybe
that they actually
you know they are i guess prompted or
fine-tuned or so
to truly optimize for what you care
about and you could imagine that you
know
that the way how the process would look
like is at some point
we as a human as we look at the
transcript of the conversation or like
an entire interaction and we say
actually here there was more loving way
to go about it and we supervise system
toward being more loving
or maybe we train the system such that
it has a reward function toward being
more loving yeah or maybe the
possibility of the system being an
asshole
and manipulative and possessive every
once in a while is a feature not a bug
because some of the
happiness that we experience when two
souls meet each other when two humans
meet each other is a kind of break from
the assholes in the world
and so you need assholes and ai as well
because like
it'll be like a breath of fresh air to
discover an ai that
the three previous ais you had are too
friendly
or no no or or cruel or whatever it's
like some kind of mix
and then this one is just right but you
need to experience the full spectrum
like i think you need to be able to
engineer assholes so let's see
because there's some level to us of
being appreciate to appreciate the human
experience
we need the dark
and the light so that kind of reminds me
um i met a while ago at the meditation
retreat uh
one woman and
um you know beautiful beautiful woman
and she had a she had a crutch okay she
had the trouble uh walking on one deck i
asked her what has happened
and
she said that five years ago she was in
maui hawaii
and she was eating a salad and some
snail fell into the salad and apparently
there are neurotoxic
snails over there and she got into coma
for a year okay oh wow
and
apparently there is you know high chance
of even just dying but she was in the
coma at some point
she regained partially consciousness she
was able to hear people in the room
people behave as she wouldn't be there
you know at some point she started being
able to speak but she was mumbling like
a barely able to to express herself and
at some point she got into wheelchair
then at some point she actually noticed
that she can move her uh
a toe
and then she knew that she will be able
to walk
and then you know that's where she was
five years after and she said that since
then she appreciates the fact that she
can move her toe
and i was thinking
do i need to go through such experience
to appreciate that i have i can move my
toe well that's really good story a
really deep example yeah
and in some sense it might be the case
that we don't see
light if we haven't went through the
darkness but i wouldn't say that we
should
we shouldn't assume that that's the case
which
may we maybe will do engineer shortcuts
yeah ilia had
this you know belief that maybe one has
to go for a week or six months
to some challenging camp yeah
to just experience you know a lot of
difficulties and then comes back and
actually
everything is bright everything is
beautiful i'm with iliana it must be a
russian thing where are you from
originally i'm i'm polish polish
okay
i'm tempted to say that explains a lot
but uh yeah there's something about the
russian the necessity of suffering i
believe i believe suffering or rather
struggle is necessary i believe that
struggle is necessary i mean in some
sense
you
even look at the story of any superhero
in that movie it's not that it was like
everything like it goes easy easy i like
how that's your ground truth
it's the story of superheroes okay
uh you mentioned that you used to do
research at night and go to bed at like
6 a.m or 7 a.m
i still do that
often
um
what uh sleep schedules have you tried
to make for a productive and happy life
like is there
um is there some interesting wild
sleeping patterns that you engaged that
you found that works really well for you
i tried at some point
decreasing number of hours of sleep like
gradually
like a half an hour every few days less
you know i was hoping to just save time
that clearly didn't work for me like at
some point there's like a phase shift
and
i felt tired all the time
uh
you know there was a time that i used to
work during the nights the nice thing
about the nights is that no one disturbs
you
and
even i remember
when i was
meeting for the first time with greg
brookman his cto and chairman of openai
our meeting was scheduled to 5 pm
and i overstepped for the meeting
over slept for the meeting yeah 5 p.m
yeah now you sound like me that's
hilarious okay yeah and uh at the moment
in some sense uh
my sleeping schedule also has to do with
the fact that i'm
interacting with people
i sleep without an alarm
so
so yeah the the team thing you mentioned
extrovert thing because
most humans operate during a certain set
of hours
you're forced to then operate at the
same set of hours
but
i'm not quite there yet
i found a lot of joy just like you said
working through the night
because it's quiet
because the world doesn't disturb you
and there's some aspect
counter to everything you're saying
there's some joyful aspect to sleeping
through the mess of the day
because uh people are having meetings
and sending emails and there's drama
meetings i can sleep through all the
meetings you know i have meetings every
day and they prevent me from having
sufficient amount of time for
focus work
and
then
i modified my calendar and i said that
i'm out of office wednesday thursday and
friday every day and i'm having meetings
only monday and tuesday
and that vastly
positively influenced my mood that i
have literally like had three days for
fully focused work yeah so there's
better solutions to this problem than
staying awake all night okay
you've been part of development of some
of the greatest ideas in artificial
intelligence what would you say is your
process for developing good novel ideas
you have to be aware that
clearly there are many other brilliant
people around
so
you have to ask yourself a question
why the given idea
let's say wasn't uh tried by someone
else
and in some sense it has to
do with
you know kind of simple it might sound
simple but like i'm thinking outside of
the box and what do i mean here
so for instance for a while
people in academia they assumed
that
you have a fixed data set
and then you optimize the algorithms
in order to get the best performance
and
that was so in great assumption
that no one thought about
training models on anti-internet
or like that that maybe some people
thought about it but if it felt too too
many as unfair
and in some sense that's almost like a
it's not my idea or so but that's an
example of breaking a typical assumption
so you want to be in the paradigm that
you are breaking a typical assumption
in the context of the ai community
getting to pick your dataset as cheating
correct and in some sense so that was a
that was assumption that many people had
out there
and then if you free yourself from
assumptions
then
they are
likely to achieve something that others
cannot do and in some sense if you are
trying to do exactly the same things as
others
it's very likely that you're gonna have
the same results yeah i
but there's also that kind of tension
which is uh
asking yourself the question why
haven't others done this
because um
i mean i get a lot of good ideas
but i think probably most of them suck
when they meet reality
so so actually i think the other big
piece
is
uh getting into habit of generating
ideas training your brain toward
generating ideas and not even
suspending judgment of the ideas
so in some sense i noticed myself that
even if i'm in the process of generating
ideas if i tell myself oh that was a bad
idea
then that actually interrupts the
process and i cannot generate more ideas
because i'm actually focused on the
negative part why it won't work yes
but
i created also environment in the way
that it's very easy for me to to store
new ideas so for instance next to my bed
i have a
voice recorder
and it happens to me often like i wake
up in that during the night and i have
some idea in the past i was
writing them down on my phone but that
means you know turning off this turning
on the screen and that wakes me up or
like pulling a paper which requires you
know turning on the
light these days i just start recording
it
what do you think i don't know if you
know who jim keller is i know team color
he's a big proponent of thinking hard on
a problem right before sleep so that he
can sleep through it and solve it in a
sleep
or like come up with radical stuff in
his sleep he was trying to get me to do
this so
it happened from
my experience perspective it happened to
me many times during the high school
days when i was doing mathematics
that i had the solution to my problem as
i woke up
at the moment regarding thinking hard
about the given problem is
i'm trying to actually devote
substantial amount of time to think
about important problems not just before
the sleep
like i'm organizing amount of the huge
chunks of time such that i'm not
constantly working on the urgent
problems but i actually have time to
think about the important one so you do
it naturally but his idea is that you
kind of
prime your brain
to make sure that that's the focus you
know oftentimes people have other
worries in their life that's not
fundamentally deep problems like i don't
know uh just stupid drama in your life
and even at work all that kind of stuff
he wants to kind of
pick the most important problem
that you're thinking about and go to bed
on that i think that's why i mean the
other thing that comes to my mind is
also i feel the most fresh in the
morning
so during the morning i try to work on
the most important things rather than
i'm just being pulled by urgent things
or checking email or so
what do you do with the cause i've been
doing the voice recorder thing too but i
end up recording so many messages it's
hard to organize
i have the same problem now i have heard
that
google pixel is really good in
transcribing text and i might get a
google pixel just for the sake of
transcribing text yeah people listening
to this if you have a good voice
recorder suggestion that transcribed
please let me know
i it's some of it is uh this has to do
with uh
uh open ai codex too like some of it is
simply like the friction
i need uh
apps that remove that friction between
voice
and the organization of the resulting
transcripts and all that kind of stuff
um but yes you're right absolutely like
during uh for me it's walking sleep too
but walking and running
especially running
get a lot of thoughts during running and
there's there's no
good mechanism for recording thoughts so
one more thing that i do i have a
separate phone
uh which i
which has no apps
and maybe it has like a
audible or let's say kindle no one has
this phone number this kind of my
meditation phone yeah and
i try to
expand the amount of time that that's
the phone that i'm having i it has also
google maps if i need to go somewhere
and i also use this phone to write down
ideas
ah that's really good idea
that's a really good idea often actually
what i end up doing is even sending a
message from that phone to the other
phone so that's actually my way of
recording messages or i just put them
into notes i love it
what advice would you give to a young
person high school
college
about how to be successful you've done a
lot of incredible things in the past
decade
so maybe maybe of some
something there might be something there
might be something
i mean
might sound like a
simplistic or so but i would say
literally just
follow your passion double down on it
and if you don't know what's your
passion just figure out what could be a
what could be a passion so this that
might be an exploration
when i was in elementary school was math
and chemistry
and i remember
for some time i gave up on math because
my school teacher she told me that i'm
dumb
and i i i guess maybe an advice would be
just ignore people if they tell you that
you're dumb
you mentioned something offline about
chemistry and explosives
um
what was that about so let's see
so a story goes like that i can
i got into chemistry maybe i was like a
second grade of my elementary school
third grade
uh i started going to chemistry classes
uh
i i really love building stuff
and
i did all the experiments that they
described in the book okay you know how
to create oxygen with vinegar and
and baking soda so okay
so i did all the experiments
and at some point i was you know so
what's next what can i do
and uh
explosives they also it's like a you
have a clear reward signal you know if
the thing worked or not
so i remember
at first i got i got interested in
producing hydrogen that was kind of
funny experiment from school you can
just burn it and then i moved to
uh nitroglycerin so that's also
relatively easy to synthesize
i started producing essentially
dynamite and detonating with it with my
friend i remember there was a you know
there was at first like maybe
two attempts that i went with a friend
to detonate what we built
and it didn't work out and like a third
time he was like ah it won't work like
uh let's don't waste time
and um now we were
i was carrying this uh
this
you know that tube with dynamite i don't
know pound or so
dynamite in my backpack or like riding
on the bike to the edges of the city
[Laughter]
yeah and
attempt number three
this would be to number three attempt
number three
and uh
now we we dig a hole to
uh put it inside it actually had the uh
you know electrical detonator
we we draw a cable behind the tree
i even i never had i haven't ever seen
like a explosion before so i thought
that there will be a lot of sound
and but you know we're like laying down
and i'm holding the cable and the
battery at some point you know it kind
of like a three to one
and uh
i just connected it and it felt like at
the ground shake it was like a more like
a
sound and then
the soil started kind of lifting up and
started falling on us yeah
wow and then uh
now the friends said let's let's make
sure next time we have helmets
but also you know
i'm happy that nothing happened to me it
could have been the case that i i lost
the limb or so yeah but that's childhood
of an engineering mind
with a strong reward signal
of an explosion
i love it
my there's some aspect of uh chemists
the the the chemist i know like my dad
with plasma chemistry plasma physics he
was very much into explosives too it's a
worrying quality of
people that work in chemistry that they
love i think it is that exactly is the
the strong signal that the thing worked
there is no doubt there's no doubt
there's some magic it's almost like a
reminder that physics works that
chemistry works it's cool it's almost
like a little glimpse at nature that you
yourself engineer i that's why i really
like artificial intelligence especially
robotics is you create
a little piece of nature
and in some sense even for me with
explosives the motivation was creation
rather than distraction yes exactly
in terms of advice i forgot to ask
about just machine learning and deep
learning for people who are specifically
interested in
machine learning how would you recommend
they get into the field
so um i would say implement everything
and also there is plenty of courses
so like from scratch um so on different
levels of abstraction in some sense but
i would say brain or implement something
from scratch or implement something from
a paper or implement something you know
from podcasts that you have heard about
i would say that's a powerful way to
understand things so it's often the case
that you read the description and you
think you understand but you truly
understand once you build it
then you actually know what really meant
that in the description
is there particular topics that you find
people just
fall in love with so i i've seen
i tend to uh really enjoy reinforcement
learning
because it's it's much more
it's much easier to get to a point where
you feel like you created something
special
like like fun games kind of things that
are rewarding it's rewarding yeah
uh as opposed to like
uh reimplementing from scratch
more like supervised learning kind of
things it's it's yeah so you know if if
someone would optimize for things to be
rewarding then it feels that the things
that are somewhat generative they have
such a property so yes you have for
instance yes adversarial networks or you
have just even generated language models
and you could you can even see
um internally we have seen this thing
with our releases so we have a we
released recently two models there is
one model called dali that generates
images and there is other model called
clip that actually uh
you provide
various possibilities what could be the
answer to what is on the picture and it
can tell you which one is the most
likely okay
and in some sense in case of the
first one dali
it is very easy for you to understand
that actually there is magic going on uh
and in the in case of the second one
even though it is insanely powerful and
you know people from a vision community
they as they started probing it inside
they actually understood
how far it goes it's difficult for
person at first to see
how well it works
and that's the same as you said that in
case of supervised learning models you
might not kind of see
or it's not that easy for you to
understand the the strength
even though you don't believe in magic
to see the magic let's say that magic
it's a generative that's really
brilliant so anything that's generative
because then you are
at the core of the creation you get to
experience creation
without much effort unless you have to
do it from scratch but and it feels that
you know humans are wired there is some
level of reward for creating stuff yeah
like of course different people have a
different weight on this reward yeah
in the big objective function in the big
objective
of a person of a person uh you wrote
that beautiful
is what you intensely pay attention to
even a cockroach is beautiful if you
look very closely
can you expand on this what is
beauty
so
what i'm i wrote here actually
corresponds to my subjective experience
that i had
through extended periods of meditation
it's it's pretty crazy that at some
point the meditation gets you to the
place that you have really
increased uh focus increase attention
and then you look at the very simple
objects that were all the time around
you can look at the table or on the pen
or at that nature
and
you notice more and more details
and it becomes very pleasant to look at
it
and it once again it kind of reminds me
my childhood
uh like i just pure joy
of being
it's also i have seen even the reverse
effect that
by default regardless of what we possess
we very quickly get used to it
and you know you can have a very
beautiful house
and
if you don't put sufficient effort
you're just gonna get used to it
and it doesn't bring any more joy
regardless of what you have yeah
well i actually
i find that material possessions
get in the way of that experience of
pure joy
so i've always i've been very fortunate
to just find joy in simple things just
just like you're saying
just like i don't know objects in my
life just stupid objects like this cup
like thing you know just objects sounds
okay i'm not being eloquent but
literally objects in the world
they're just full of joy because it's
like
i can't believe
one i can't believe that i'm
fortunate enough to be alive to
experience these objects and then two i
can't believe
humans are clever enough to have built
these objects
the the hierarchy of pleasure that that
uh provides is infinite i mean even if
you look at the cup of water so you know
you see first like a level of like a
reflection of light but then you think
you know man there's like a trillions
upon of trillions of particles bouncing
uh against each other there is also uh
the tension on the surface that you know
if the back back could like a stand on
it and move around and you think it also
has this like a magical property that as
you decrease temperature
it actually expands in volume which
allows for the
you know legs to freeze on the on the
surface and then at the bottom to have
actually uh not freeze which allows for
life like a
crazy yeah you look
in detail at some object and you think
actually you know this table that was
just the figment of someone's
imagination at some point and then there
was like thousands of people involved to
actually manufacture it yeah and put it
here and by default no one cares
[Laughter]
and then you can start thinking about
evolution how it all started from single
cell organisms
that led to this table and and okay
these thoughts they give me life
appreciation yeah and
even lack of those just the pure raw
signal also gives their life
appreciation see the thing is
and then that's coupled
for me with the sadness that the whole
ride ends
and perhaps is deeply coupled in that
the fact that this experience this
moment ends
gives it
gives it an intensity that i'm not sure
i would otherwise have
so in that same way i try to meditate on
my own death often
do you think about your mortality
are you afraid of death
so
fear of death is like one of the most
fundamental fears that each of us has we
might be not even aware of it it
requires to look inside to even
recognize that it's
out there and there is still let's say
this property
of uh nature that if things would last
forever then they would be also boring
to us
the fact that the things change in some
way
gives any meaning to them
i also you know found out that
it seems to be
very healing to people to
have
this short experiences
uh like i guess psychedelic experiences
in which they experience
death of self
in which they let go of this fear and
then maybe can even increase the
appreciation of the moment and it seems
that many people they uh
they
they can easily comprehend fine the fact
that their money is finite while they
don't see that time is finite
i have this like a discussion with ilya
from time to time he's saying you know
man like uh
the lack will pass very fast at some
point i will be 40 50 60 70 and then
it's over
this is true which also makes me believe
that you know that every single moment
it is so unique
that
should be appreciated and this also
makes me think that i should be acting
on my life
because otherwise it will pass
i also like this framework of thinking
from jeff bezos on regret minimization
that like i would like
if i will be at that death bed to look
back on my life
and
and not regret that i haven't
done something it's usually you might
regret that you haven't
tried i'm fine with failing
i haven't tried
uh what's the nature eternal occurrence
tried to live a life that if you had to
live it infinitely many times
that would be the
you'll be okay with
that kind of life
so try to live it optimally
i can say that
it's almost like i'm
unbelievable to me
where i am in my life i'm extremely
grateful for actually people
whom i met i would say i think that i'm
decently smart and so on
uh
but i think that
actually to great extent where i am has
to do with that people who i met
would you be okay
if after this conversation you died
so
if i'm dead then it kind of
i don't have a choice anymore
so in some sense there's like plenty of
things that i would like to try out in
my life
[Music]
i feel that you know i'm gradually going
one by one and i'm just doing them
i think that the list will be always
infinite yeah
so might as well go today
yeah i mean to be clear i'm not looking
forward to die
i would say if there is no choice i
would accept it
but like uh in some sense i'm
if there would be a choice if there
would be possibility to live i would
fight for a living
i find um
it's more honest than real to think
about you know dying today at the end of
the day
that seems to me to at least to my brain
more honest slap in the face
as opposed to i still have 10 years
like today
then then i'm much more about
appreciating the cup and the table and
so on and less about like silly worldly
accomplishments and all those kinds of
things
we we have in the company a person who
say at some point found out that they
have cancer and that also gives you know
huge perspective with respect to what
matters now yeah and you know often
people in situations like that they
conclude that actually what matters is
human connection
and
love and uh that's people conclude also
if you have kids because kids is family
you uh i think tweeted
we don't assign the minus infinity
reward to our death
such a reward would prevent us from
taking any risk we wouldn't be able to
cross the road in fear of being hit by a
car so in the objective function you
mentioned fear of death might be
fundamental to the human condition
so
as i said let's assume that they're like
a reward functions in our brain
and
and the interesting thing is
even realization how different reward
functions can play with your behavior
as a matter of fact i wouldn't say that
you should assign infinite
negative reward to anything because that
messes up the math
the math doesn't work out it doesn't
work out and as you said even you know
uh government or some insurance
companies you said they assign 99
million dollars to human life yeah and
i'm just saying it with respect to
that might be a harsh statement to
ourselves but in some sense that there
is a finite value of our own life
i'm trying to put it from perspective of
being
less
of being more egoless
and realizing fragility of my own life
and
in some sense
the
fear of death
might prevent you from acting
because anything can cause death
yeah and i'm sure actually if you were
to put death in the objective function
there's probably so many aspects to
death and fear of death and
realization of death and mortality
there's just whole components of
finiteness
of not just your life but every
experience and so on you're gonna have
to formalize mathematically and also you
know that might lead to
um
you spending a lot of compute cycles
on this like a
and deliberating this terrible future
instead of experiencing now
and that in some sense is also kind of
unpleasant simulation to run in your
head yeah
do you think there's an objective
function that describes the entirety of
uh
human life
so you know usually the way you ask that
is what is the meaning of life
is there um
a universal objective functions that
captures the why of life so yeah i mean
i suspected that they will ask this
question but it's also a question that i
asked myself many many times
see i can tell you a framework that i
have these days to think about these
questions so i think that fundamentally
meaning of life has to do
with some of our reward functions that
we have in brain and they might have to
do with
let's say for instance curiosity
or
human connection which might mean
understanding others
it's also possible for a person to
slightly modify their reward function
usually they mostly stay fixed but it's
possible to modify reward function and
you can pretty much choose so in some
sense reward functions optimizing reward
functions they will give you life
satisfaction
is there some randomness in the function
i think when you are born there is some
randomness like you can see that some
people for instance they
they care more about building stuff some
people care more about caring for others
some people
that there are all sorts of uh default
reward functions and then in some sense
you can ask yourself what's this like
what is the satisfying way for you to go
after this reward function and you just
go after this reward function and you
know some people also ask are these
reward functions real
i almost think about it
as
let's say
if you would have to discover
mathematics
in mathematics you are likely to run
into various objects like a complex
numbers
or differentiation some other objects
and these are very natural objects that
arise and similarly the reward functions
that we are having in our brain they are
somewhat very natural that you know
there is a reward function for
for understanding
like a comprehension yeah uh
curiosity and so on so in some sense
they are in the same way natural as
their natural objects in mathematics
interesting so you know there's the uh
the old
sort of debate is mathematics invented
or discovered you're saying reward
functions are discovered so nature so
nature's provided some you can still
let's say expanded throughout the life
some of the reward functions they might
be futile like for instance there might
be a reward function maximize amount of
wealth
yeah and this is more like a a learning
reward function
and but we know also that some reward
functions if you optimize them you won't
be
quite satisfied
well i don't know which part of your
reward function resulted in you coming
today but i am deeply appreciative that
you did spend your valuable time with me
watching is really fun talking to you
you're you're brilliant you're a good
human being and it's an honor to meet
you and an honor to talk to you thanks
for talking today brother
thank you alex a lot i appreciated your
questions here
i had a lot of time being here
thanks for listening to this
conversation with welch and ramba to
support this podcast please check out
our sponsors in the description
and now let me leave you some words from
arthur c clarke who is the author of
2001 a space odyssey
it may be that our role on this planet
is not to worship god but to create him
thank you for listening and hope to see
you next time
you