Kind: captions
Language: en
the following is a conversation with
wojciech zaramba co-founder of openai
which is one of the top organizations in
the world doing artificial intelligence
research and development
wojciech is the head of language and
cogeneration teams building and doing
research on github copilot openai codex
and gpt
three and who knows
four five
six
n and
n plus one and he also previously led
openai's robotic efforts
these are incredibly exciting projects
to me that deeply challenge and expand
our understanding of the structure and
nature of intelligence the 21st century
i think may very well be remembered for
a handful of revolutionary ai systems
and their implementations
gpt codex and applications of language
models and transformers in general
to the language and visual domains may
very well be at the core of these ai
systems
to support this podcast please check out
our sponsors
they're listed in the description
this is a lex friedman podcast and here
is my conversation with wachek zaremba
you mentioned that sam altman asked
about the fermi paradox
and the people at open ai had really
sophisticated interesting answers so
that's when you knew this is the right
team to be working with so let me ask
you about the fermi paradox about aliens
why have we not found overwhelming
evidence for aliens visiting earth
i don't have a conviction in the answer
but rather kind of probabilistic
perspective on what might be a let's say
possible answers it's also interesting
that the question itself even
can't touch on the you know your typical
question of what's the meaning of life
because like if you assume that like we
don't see aliens because they destroy
themselves that kind of upwards their
focus on making sure that we won't
destroy ourselves yeah and at the moment
the
place where i am actually with my belief
and these things also change over the
time
is i think that we might be alone in the
universe which actually makes
life more or less a consciousness life
more kind of valuable and that means
that we should more appreciate it
have you always been alone so what's
your intuition about our galaxy our
universe is it just
sprinkled with graveyards of intelligent
civilizations or are we truly is is life
intelligent life truly unique
at the moment my belief that it is
unique but i would say i could also
you know there was like some footage
released with ufo objects which makes me
actually doubt my own belief yes
yeah i can tell you one crazy answer
that i have heard yes
so
apparently when you look actually at the
limits of computation
you can compute more
if the temperature of the universe would
drop down
so one of the things that
aliens might want to do if they are
truly optimizing to maximize amount of
compute which you know maybe can lead to
or let's say simulations or so it's
instead of wasting current entropy of
the universe because you know we by
living we are actually somewhat wasting
entropy
then you can wait for the universe to
cool down such that you have more
computation that's kind of a funny
answer i'm not sure if i believe in it
but that would be one of the reasons why
you don't see aliens it's also possible
see some people say that maybe there is
not that much point in actually going to
other galaxies
if you can go inwards
so there is no limits of what could be
an experience
if we could you know connect machines to
our brains
while there are still some limits if we
want to explore universe yeah there
could be a lot of
ways to go inwards too
once you figure out some aspect of
physics we haven't figured out yet maybe
you can travel to different dimensions i
mean
travel in three-dimensional space may
not be the most fun kind of travel there
may be like just a huge amount of
different ways to travel and it doesn't
require a spaceship going
slowly in 3d space space time it also
feels you know one of the problems is
that speed of light is low and universe
is vast yeah and um
it seems that actually most likely if we
want to travel very far
then then we would instead of actually
sending spaceships with humans that wait
a lot we would
send something similar to what yuri
miller is working on these are like a
huge uh sail which is at first powered
power there is a shot of laser from an
earth and it can propel it to a quarter
of speed of light and uh sail itself
contains a
few grams of equipment and that might be
the way to actually
transport matter through universe but
then when you think what would it mean
for humans it means that
we would need to actually put their 3d
printer and you know 3d print a human on
other planet i don't know play them
youtube or let's say or like a pre 3d
print like a huge human right away or
maybe a womb or so um yeah
with our current techniques of
archaeology
if
a civilization was born and died
long long enough ago on earth we
wouldn't be able to tell and so that
makes me really sad
and so i think about earth in that same
way how can we leave some remnants if we
do destroy ourselves how can we leave
remnants for aliens in the future to
discover
like here's some nice stuff we've done
like wikipedia and youtube do we have it
like
in a satellite orbiting earth
with a hard drive like how how do we say
how do we back up human civilization
uh for the good parts or
all of it is good parts
so that uh
it can be preserved longer than our
bodies can that's a
that's kind of a
it's a difficult question it also
requires the difficult acceptance of the
fact that we may die and if we die we
may
die suddenly as a civilization
so let's see i think it kind of depends
on the cataclysm we have observed in
other parts of the universe that births
of gamma rays
these are
high energy
rays of light that actually can
apparently kill entire galaxy
so there might be actually nothing even
to
nothing to protect us from it i'm also
when i'm looking actually at the past
civilization so it's like aztecs or so
they disappear from the
surface of the earth and one can ask
why is it the case
and
the way i'm thinking about it is
you know that definitely they had some
problem that they couldn't solve
and maybe there was a flat and all of a
sudden they couldn't drink there was no
potable water and they all died
and
i think that
so far
the best solution to such a problems is
i guess technology so i mean if they
would know that you can just boil water
and then drink it after then that would
save their civilization and even now
when we look actually at the current
pandemic it seems that once again
actually science comes to rescue and
somehow science increases size of the
action space and i think that's a good
thing
yeah but nature
has a vastly larger action space but
still it might be a good thing for us to
keep on increasing action space
okay
looking at past civilizations yes
but looking at the destruction of human
civilization
perhaps expanding the action space will
add
actions that are easily
acted upon easily executed and as a
result destroy
us
so let's see
i was pondering
why actually even
we have negative impact on the
globe because you know if you ask every
single individual they would like to
have clean air
they would like healthy planet but
somehow it actually is not the case that
as a collective we are not going this
direction
i think that there exists very powerful
system to describe what we value that's
capitalism it assigns actually monetary
values to various activities at the
moment the problem in the current system
is that there are some things which we
value there is no cost assigned to it so
even though we value clean air or maybe
we also
value
lack of destruction on the internet or
so at the moment
these quantities you know companies
corporations can pollute them uh for
free
so in some sense
i wish
or like and that's i guess purpose of
politics to
align the incentive systems and we are
kind of maybe even moving in this
direction the first issue is even to be
able to measure the things that we value
then we can actually assign the monetary
value to them
yeah and that's so it's getting the data
and also
probably through technology enabling
people to vote
and to
move money around in a way that is
aligned with their values and that's
very much a technology question so like
having one president
and congress
and voting that happens every four years
or something like that
that's a very outdated idea there could
be some technological improvements to
that kind of idea so
i'm thinking from time to time about
these topics but it also feels to me
that it's it's a little bit like a
it's hard for me to actually make
correct predictions what is the
appropriate thing to do i extremely
trust uh sam altman our ceo
on these topics he um okay i'm more on
the side of being i guess
naive hippie that
yeah
that's your life philosophy um
well like i think self-doubt
and uh
i think hippie implies optimism those
those two things are pretty pretty good
way to operate
i mean still it is
hard for me to actually
understand how the politics works or
like uh how this like
exactly how the things would play out
and sam is a really excellent with it
what do you think is rarest in the
universe you said we might be alone
what's hardest to build is another
engineering way to ask that
life
intelligence or consciousness so like
you said that we might be alone
which is the thing that's hardest to get
to
is it just the origin of life is it the
origin of intelligence is it the origin
of consciousness
so
um let me at first explain you my kind
of mental model what i think is needed
for life to appear
um
so
i imagine that at some point there was
this primordial
zoop of
amino acids and maybe some proteins in
the ocean
and you know some proteins were turning
into some other proteins through
reaction
and you can almost think about this
uh cycle of what turns into what as
there is a graph essentially describing
which substance turns into some other
substance and essentially life means
that all the sudden in the graph has
been created a cycle such that the same
thing keeps on happening over and over
again that's what is needed for life to
happen and in some sense you can think
almost that you have this gigantic graph
and it needs like a sufficient number of
edges for the cycle to appear
then um from perspective of intelligence
and consciousness
my current intuition is that they might
be
quite intertwined first of all it might
not be that it's like a binary thing
that you have intelligence or
consciousness it seems to be a
more a
continuous component let's see if we
look for instance on the even networks
recognizing images and people are able
to show that the activations of these
networks correlate very strongly
with activations in visual cortex
of some monkeys the same seems to be
true about language models
also if you for instance
look
if you train agent in a 3d world
at first you know it it it it barely
recognizes what is going on over the
time it kind of recognizes foreground
from a background over the time it kind
of knows where there is a foot
and it just follows it
over the time it actually starts having
a 3d perception so it is possible for
instance to look inside of the head of
an agent and ask what would it see if it
looks to the right and the crazy thing
is you know initially when the agents
are very trained these predictions are
pretty bad over the time they they
become better and better you can still
see
that if you ask what happens when the
head is turned by 360 degrees for some
time they think that the different thing
appears and then at some stage they
understand actually that the same
thing's supposed to appear so they get
like a understanding of 3d structure
it's also you know very likely that they
have inside some
level of and of like a symbolic
reasoning like they're particularly
symbols for other agents so when you
look at dota agents they collaborate
together and uh
and
now they they they have some
anticipation of uh if if they would win
battle they have some some expectations
with respect to other agents i might be
you know too much anthropomorphizing
um the the how the things look
look for me but then the fact that they
have a symbol for other agents
and makes me believe that
at some stage as the uh you know as they
are optimizing for skills they would
have also symbol to describe
themselves this is like a very useful
symbol to have and this particularity i
would call it like a self-consciousness
or self-awareness
and still it might be different from the
consciousness so i guess the the way how
i'm understanding the word consciousness
let's say the experience of drinking a
coffee or let's say experience of being
a butt
that's the meaning of the word
consciousness it doesn't mean to be
awake
yeah it feels
it might be also somewhat related to
memory and recurrent connections so um
it's kind of okay if you look at
anesthetic drugs they might be
uh like they essentially
they disturb
brain waste
such that
[Music]
maybe memory is not not formed
so there's a lessening of consciousness
when you do that correct and so that's
one way to intuit what is consciousness
there's also kind of another
element here it could be that it's
you know this kind of self-awareness
module that you described
plus the actual subjective experience
is a storytelling module
that tells us a story about uh
what we're experiencing
the
crazy thing so let's say i mean in
meditation they teach people
not to speak story inside of the head
and there is also some fraction of
population
who doesn't have actually narrator i
know people who don't have a right
narrator and you know they have to use
external people in order to
kind of
solve tasks that require internal
narrator
so
it seems that it's possible to have the
experience without the talk
what are we talking about when we talk
about the internal narrator is that the
voice when you're like yeah i thought
that that that's what you are referring
to well i was referring more on the like
not an actual voice
i meant like
there's some kind of
like subjective experience
feels like it's
it's fundamentally about storytelling to
ourselves
it feels like
like the feeling is a story
that is much
much simpler abstraction than the raw
sensory information
so it feels like it's a very high level
abstraction
that
is useful
for me to feel like
entity in this world
most
useful aspect of it is that
because i'm conscious
i think there's an intricate connection
to me not one
wanting to die
so like
it's a useful hack to really
prioritize not dying
like those seem to be somehow connected
so i'm telling the story of like it's
richly feels like something to be me and
the fact that me exists in this world i
want to preserve me
and so that makes it a useful agent hack
so i will just refer maybe to the first
part as you said about the kind of story
of describing who you are
i was
thinking about that even so you know
obviously i'm i'm i
like thinking about consciousness uh i
like thinking about the ai as well and
i'm trying to see analogies of these
things in ai what would it correspond to
so um
you know openly i trained a
a
model called gpt
which
can generate a
pretty amusing text on arbitrary topic
and um
and one way to control gpd
is uh by putting into prefix at the
beginning of the text some information
what would be the story about
you can have even chat with uh
you know with gpt by saying that the
chat is with lex or elon musk or so
and gpt would just
pretend to be you or elon musk or so
and
it almost feels that this uh
story that we give ourselves to describe
our life it's almost like a
things that you put into context of gpt
yeah the primary it's the and but the
the context we provide to gpt
is uh
is multimodal it's so gpt itself is
multimodal gpt itself uh hasn't learned
actually from experience of single human
but from the experience of humanity it's
a chameleon you can turn it into
anything and in some sense by providing
context uh
it you know
behaves as the thing that you wanted it
to be and it's interesting that the
you know people have a stories of who
they are and as i said these stories
they help them to operate in the world
but it's also you know interesting
i guess various people find it out
through meditation or so that
there might be some patterns that you
have learned
when you were a kid that actually are
not serving you anymore
and you also might be thinking that
that's who you are and that's actually
just the story
yeah so it's a useful hack but sometimes
it gets us into trouble it's a local
optima
you wrote that stephen hawking he
tweeted stephen hawking asked what
breathes fire into equations which meant
what makes given mathematical equations
realize the physics of a universe
similarly
i wonder what breathes fire into
computation what makes given computation
conscious
okay so how do we engineer consciousness
how do you breathe fire
and magic into the machine
so
it seems clear to me that not every
computation is conscious i mean you can
let's say just keep on multiplying one
matrix over and over again and my
gigantic matrix you can put a lot of
computation i don't think it would be
conscious so in some sense the question
is
what are the computations which could be
conscious
uh i mean so one assumption is
that it has to do purely with
computation that you can abstract away
matter and other possibilities that it's
very important was the realization of
computation that it has to do with some
uh uh force fields or so and they bring
consciousness at the moment my intuition
is that it can be fully abstracted that
way so in case of computation you can
ask yourself what are the
mathematical objects or so that could
bring such a properties so for instance
if we think about the
models uh ai models then what they truly
try to do
or like models like gpt is uh
you know they try to predict a next word
or so and this turns out to be
equivalent to
compressing
text
and because in some sense compression
means that
you learn the model of reality and you
have just to uh
remember where are your mistakes the
better you are in predicting the
and
and in some sense when we look at our
experience also when you look for
instance the car driving you know in
which direction it will go you are good
like a in prediction and um
you know it might be the case that the
consciousness
is intertwined with compression it might
be also the case that self-consciousness
has to do with compressor trying to
compress itself so
um
okay i was just wondering what are the
objects in you know mathematics or
computer science which are mysterious
that could uh that that could have to do
with consciousness and then i thought um
you know you you see in uh mathematics
there is something called cadal theorem
which means okay you have if you have
sufficiently complicated mathematical
system it is possible to point the
mathematical system back on itself in
computer sense there is uh something
called helping problem it's it's
somewhat similar construction so i
thought that you know if we believe that
that the that
under assumption that consciousness has
to do with uh with compression
uh
then you could imagine that the the as
you keep on compressing things then at
some point it actually makes sense
for the compressor to compress itself
metacompression yeah consciousness is
metacompression
that's uh that's and i and an idea
and in some sense you know the creation
of it
thank you so uh
but do you think if we think of a
touring machine a universal touring
machine
can that achieve
consciousness
so is there some
thing beyond our traditional definition
of computation that's required so it's a
specific computation and i said this
computation has to do with compression
and
the compression itself maybe other way
of putting it is like you are internally
creating the model of reality
in order like a it's like a you try
inside to simplify reality in order to
predict what's going to happen
and
that also feels somewhat similar to how
i think actually about my own conscious
experience so clearly i don't have
access to reality the only access to
reality is through you know cable going
to my brain and my brain is creating a
simulation of reality and i have access
to the simulation of reality
are you by any chance uh aware of uh
the harder prize marcus hutter
he he made this prize
for compression
of wikipedia pages
and
there's a few qualities to it
one i think has to be perfect
compression which makes
i think that little quirk makes it much
less um
applicable to the general task of
intelligence because it feels like
intelligence is always going to be messy
uh
like perfect compression is feels like
it's not the right goal but it's
nevertheless a very interesting goal so
for him intelligence equals compression
and so
the smaller you make the file
given a large wikipedia page
the more intelligent the system has to
be yeah that makes sense so you can make
perfect compression if you store errors
and i think that actually what he meant
is you have algorithm plus errors and by
the way hooter hatter is a he was pa uh
phd advisor of shenleck who is the mind
uh
uh deep mind co-founder yeah yeah so
there's an interesting
and now he's a deep mind there's an
interesting uh network of people he's
one of the people that
i think
seriously took on the task of what would
an agi system look like
i think for a longest time
the question of agi was not
taken
seriously or rather rigorously
and he did just that like mathematically
speaking what would the model look like
if you remove the constraints of it
having to be
having to have a
reasonable amount of memory reasonable
amount of running time complexity uh
computation time what would it look like
and essentially it's it's a
half math half philosophical discussion
of uh how would like a reinforcement
learning type of framework look like for
an agi yeah so he developed a framework
even to describe what's optimal with
respect to reinforcement learning like
there is a theoretical framework which
is as you said
under assumption there is infinite
amount of memory and compute and there
was actually one person before his name
is solomonov hutter extended
amount of work to reinforcement learning
but there exists a
theoretical algorithm which is optimal
algorithm to build intelligence and i
can actually explain you the algorithm
yes
let's go let's go so the task itself can
i just
pause
how absurd it is
for brain in a skull trying to explain
the algorithm for intelligence just go
ahead it is pretty crazy it is pretty
crazy that you know the brain itself is
actually so small and it can ponder
how to design algorithms that optimally
solve the problem of intelligence okay
all right so what's the algorithm so
let's see so first of all the task
itself is
described as
you have infinite sequence of zeros and
ones
okay you read n bits and you are about
to predict n plus one bit
so that's the task and you could imagine
that every task could be casted as such
a task so if for instance you have
images and labels you can just turn
every image into sequence of zeros and
ones then label you concatenate labels
and you and that that's actually the the
and you could you could start by having
training data first and then afterwards
you have test data
so theoretically any problem could be
casted as a problem of predicting zeros
and ones on this infinite type so um
so let's say you read already n bits and
you want to predict n plus one bit
and i will ask you to write
every possible program that generates
these end bits okay so
and you can have you you choose
programming language it can be in python
or c
and the difference between programming
languages
might be there is a difference by
constant
asymptotically your predictions will be
equivalent
so you you read and beats you enumerate
all the programs that produce these and
end bits in their output
and then in order to predict n plus one
bit you actually weight
the programs according to their length
and there is like some specific formula
how you weight them and then the n plus
one bit prediction is the prediction uh
from each of this program according to
that weight
like statistically you statistically
pick so the smaller the program the more
likely you you are to pick the its
output
so uh that's that algorithm is grounded
in the hope
or the intuition that the simple answer
is the right one it's a formalization of
it yeah um it also
means like if you would ask the question
after
how many years
would you know
sun explode
you can say
it's more likely the answer is
to some power because it's a shorter
program
yeah
and then other
well i don't have a good intuition about
how different the space of short
programs are from the space of large
programs
like
what is the universe where short
programs
uh like run things
uh so as i said the things have to agree
with end beats so even if you have
you you need to start okay if if you
have very short program and they're like
uh still some as if it's not perfect
with prediction of n bits you have to
start errors what are the errors and
that gives you the full program that
agrees on end beats
oh so you don't agree perfectly with the
end bits and you store
that's like a longer a longer program
slightly longer program
because it contains these extra bits of
errors that's fascinating what's what's
your intuition
about
the the programs
that are able to do cool stuff like
intelligence and consciousness are they
uh
perfectly like is is it uh
is there if then statements in them so
like is there a lot of exceptions that
they're storing so um you could imagine
if there would be tremendous amount of
if statements yeah then they wouldn't be
that short in case of neural networks
you could imagine that
what happens is uh
they
when you start with an uninitialized
neural network uh it stores internally
many possibilities how the
how the problem can be solved and sgd is
kind of magnifying some some
some
paths which are slightly
similar to the correct answer so it's
kind of magnifying correct programs and
in some sense hdd is a search algorithm
in the program space and the program
space is represented by uh you know kind
of the wiring inside of the neural
network and there's like an insane
number of ways how that features can be
computed
let me ask you the high level basic
question that's not so basic
what is deep learning
is there a way you'd like to think of it
that is different than like a generic
textbook definition
the thing that i hinted just a second
ago is maybe the uh closest to how i'm
thinking these days about um deep
learning so
now the statement is
uh neural networks can represent some
programs
uh it seems that various modules that we
are actually adding up to are like a you
know we we want networks to be deep
because we we want multiple steps of the
computation
and
and deep learning provides the way to
represent space of programs which is
searchable and it's searchable with
stochastic gradient descent so we have
an algorithm to search over a humongous
number of programs
and gradient descent kind of bubbles up
the things that are tend to give correct
answers so
a neural network
with a with fixed weights that's
optimized do you think of that as a
single program um so there is a
work by christopher olach where he
so he works on interpretability of
neural networks and he was able to
uh
to identify inside of the neural network
for instance a detector of a wheel for a
car or the detector of a mask for a car
and then he was able to separate them
out and assemble them uh together using
a simple program uh for the detector for
a car detector that's like uh if you
think of traditionally defined programs
that's like a function within a program
that this particular neural network was
able to find and you can tear that out
just like you can copy and paste from
stack overflow
that
so uh any program is a composition of
smaller programs
yeah i mean the nice thing about the
neural networks is that it allows the
things to be more fuzzy than in case of
programs
in case of programs you have this like a
branching this way or that way and the
neural networks they they have an easier
way to
to be somewhere in between or to share
things
what to use the most beautiful or
surprising idea in deep learning
in the utilization of these neural
networks which by the way for people who
are not familiar
neural networks is a bunch of uh
what would you say it's inspired by the
human brain there's neurons there's
connection between those neurons there's
inputs and there's outputs and there's
millions or billions of those neurons
and
the learning
happens
uh by adjusting the weights on the edges
that connect these neurons thank you for
giving definition that
i supposed to do it but i guess you have
enough empathy to listeners to actually
know that that might be useful no that's
like
so i'm asking plato of like what is the
meaning of life he's not going to answer
you're being philosophical and deep and
quite profound talking about the space
of programs which is just very
interesting but also for people who are
just not familiar with the hell we're
talking about when we talk about deep
learning anyway sorry what is the most
beautiful
or surprising idea to you in in um in
all the time you've worked at deep
learning and you worked on a lot of
fascinating projects
applications of neural networks
it doesn't have to be big and profound
it can be a cool trick yeah i mean i'm
thinking about the trick but like it's
still amusing to me that it works at all
yeah that let's say that the extremely
simple algorithm stochastic gradient
descent which is something that i would
be able you know to derive on the piece
of paper to high school student uh when
put at the
ins at the scale of you know thousands
of machines actually
uh can create
the
behaviors we which we called kind of
human like behaviors
so in general
any applications to cast a gradient
descent to neural networks is
is amazing to you so that or is there a
particular application
in natural language
reinforcement learning
uh
and also would you attribute
that success too is it just scale
what profound insight can we take from
the fact that
the thing works for
gigantic
uh sets of variables
i mean the interesting thing is these
algorithms they were
invented uh decades ago
and
people actually
gave up on the idea yeah and um
you know back then they thought that we
need profoundly different algorithms and
they spent a lot of cycles on very
different algorithms and i believe that
you know we have seen that various
various innovations that say like
transformer or or dropout or so they can
uh you know pass the help but it's also
remarkable to me that this algorithm
from 60s or so
or i mean you can even say that the
gradient descent was invented by leibniz
in i guess 18th century or so that
actually
is the
core of learning
in the past people are
it's almost like a out of the maybe an
ego people are saying that it cannot be
the case that such a simple algorithm is
there you know
uh
could solve complicated problems
so they were in search for the
other algorithms and as i'm saying like
i believe that actually we are in the
game where there is there are actually
frankly three levels there is compute
there are algorithms and there is data
and if we want to build intelligent
systems we have to
pull all three levers
and they are actually multiplicative
and it's also interesting so you ask is
it only compute
people internally they did the studies
to determine how much gains they were
coming from different levels and so far
we have seen that more gains came from
compute than algorithms but also we are
in the world that in case of compute
there is a kind of you know exponential
increase in funding and at some point
it's impossible to
invest more it's impossible to you know
invest 10 trillion dollars
because we are speaking about that
let's say all taxes in u.s
uh but you're talking about money there
could be innovation
in the compute that's that's true as
well
so i mean they're like a few pieces so
one piece is human brain is an
incredible super computer
[Music]
and they're like a
it
it has
100 trillion
parameters or like a if you try to count
various quantities in the brain there
are like a neurons synapses that small
number of neurons there is a lot of
synapses yeah it's unclear even how to
map
synapses
to
two parameters of neural networks but
it's clear that there are many more yeah
so it might be the case that our
networks are still somewhat small
it also might be the case that they are
more efficient than brain or less
efficient by some by some huge factor
i also believe that there will be like a
you know at the moment we are at the
stage that the these neural networks
they require 1000x or like a huge factor
of more data than humans do and it will
be a matter of
there will be algorithms that
vastly decrease sample complexity i
believe so but the place where we are
heading today is dark domains which
contains million x
more
data and even though computers might be
1 000 times slower than humans in
learning that's not the problem okay for
instance
i believe that
it should be possible to create super
human therapies
uh by uh
and and then they're like even simple
steps of of doing what of of doing it
and you know that the core reason is
there is just machine will be able to
read way more
transcripts of therapies and then it
should be able to speak simultaneously
with many more people and it should be
possible to optimize it uh all in
parallel
and well there's now you're touching on
something i deeply care about and think
is way harder than we imagined
um
what's the goal of a therapist what's it
called therapies
so okay so one goal now this is
terrifying to me
but there's a lot of people that
contemplate suicide suffer from
depression
and they could significantly be helped
with therapy
and the idea that an ai algorithm might
be in charge of that
it's like a life and death task
it's uh
the stakes are high
so one
goal for a therapist whether human or ai
is to prevent suicide ideation to
prevent suicide how do you achieve that
so
let's see
so
to be clear i don't think that the
current models are good enough for such
a task because it requires insane amount
of understanding and patty and the
models are far from this place but it's
but do you think that understanding
empathy that signal is in the data um i
think there is some signal in the data
yes i mean there are plenty of
transcripts of conversations
and it is possible to
it is possible from it to understand
personalities it is possible from it to
understand uh if conversation is
a friendly
uh amicable uh
antagonistic it is i believe that the
you know given the fact that the models
that we train now
they can
they can have
they are chameleons that they can have
any personality they might turn out to
be better in understanding
uh personality of other people than
anyone else and they feel pathetic to be
empathetic yeah
interesting uh but i wonder if there's
some level
of
multiple modalities required
to be able to
be empathetic of the human experience
whether language is not enough to
understand death to understand fear to
understand
uh childhood trauma
to understand uh wit and humor required
when you're dancing with the person who
might be depressed or suffering
both humor and hope and love and all
those kinds of things
so there's another underlying question
which is self-supervised versus
supervised
so can you get
that from the data by just reading
a huge number of transcripts i actually
so i think that reading huge number of
transcripts is a step one it's like the
same way as you cannot learn to dance if
just from youtube by watching it you
have to actually try it out yourself
yeah and so i think that here that's a
similar situation i also wouldn't deploy
the system in the high-stakes situations
right away but kind of see gradually
where
it goes and
obviously initially
it would have to go hand with a hand in
hand with humans but
at the moment we are in the situation
that actually
there is many more people who actually
would like to have a therapy or
or speak with with someone then there
are therapies out there okay you know i
was
so so
fundamentally i was thinking what are
the things that
can vastly increase people well-being
therapy is one of them i think
meditation is other one i guess maybe
human connection is a third one and i
guess
pharmacologically it's also possible
maybe direct brain stimulation or
something like that but these are pretty
much options out there then let's say
the way i'm thinking about the agi
endeavor is by default that's an
endeavor to
increase amount of wealth and i believe
that we can vastly increase amount of
wealth
for everyone and simultaneously so i
mean they're like two endeavors that
make sense to me one is like essentially
increase amount of wealth and second one
is uh increase overall human well-being
and those are coupled together and they
they can okay i would say these are
different topics one can help another
and uh you know therapist is a funny
word because i see friendship and love
as therapy i mean so therapist broadly
defined as just friendship as a friend
so like therapist is has a very kind of
clinical sense to it but
what is human connection
you're like uh
not to get all camus and dostoyevsky on
you but you know life is suffering and
we draw
we
seek connection with other humans as we
desperately try to make sense of this
world
in the deep overwhelming loneliness that
we feel
inside
so i think connection has to do with
understanding
and i think that almost like a lack of
understanding causes suffering if you
speak with someone and you
do you feel ignored that actually causes
pain if you are feeling deeply
understood that actually
they they might not even tell you what
to do in life but like a pure
understanding or just being heard
understanding is a kind of
it's a lot you know just being heard
feel like you're being heard
like somehow
that's uh alleviation temporarily of the
loneliness
that if somebody
knows you're here
with their body language with the way
they are with the way they look at you
with the way they talk
you feel less alone for a brief moment
yeah very very much agree so i thought
in the past about uh somewhat similar
question to yours which is what is love
uh rather what is connection yes and um
and obviously i think about these things
from ai perspective what would it mean
um
so i said that the you know intelligence
has to do with some compression which is
more or less like i can say almost
understanding of what is going around it
seems to me that uh other aspect is
there seem to be reward functions and
you can have a you know reward for
uh food for maybe human connection for
uh let's say warmth
uh
sex and so on
and um
and it turns out that the various people
might be optimizing slightly different
reward functions they essentially might
care about different things
and um
in case of
love at least the love between two
people you can say that the um you know
boundary between people dissolves to
such extent that
they end up optimizing each other reward
functions
yeah
oh that's interesting um
the success of each other yeah in some
sense i would say love means
uh
helping others to optimize their uh
reward functions not your reward
functions not the things that you think
are important but the things that the
person cares about you try to help them
to optimize it so love is uh
if you think of two reward functions you
just it's a condition yeah you combine
them together yeah pretty much maybe
like with a weight and it depends like
the dynamic of the relationship yeah i
mean you could imagine that if you are
fully uh optimizing someone's reward
function without yours then yeah then
maybe are creating code dependency or
something like that yeah
i'm not sure what's the appropriate
weight but the interesting thing is i
even
i even think that the
individual person
we ourselves we are actually
less of a
unified insight so for instance if you
look at the donut on the one level you
might think oh this like it looks tasty
i would like to eat it on another level
you might tell yourself i shouldn't be
doing it because
i want to gain muscles so and you know
you might do it regardless kind of
against yourself so it seems that even
within ourselves they're almost like a
kind of intertwined personas
and
i believe that the self-love
means that
the love between all these persons which
also means being able to
love love
yourself when we are angry or stressed
or so combining all those reward
functions of the different selves you
have yeah and accepting that they are
there okay you know often people they
have a negative self-talk or they say i
don't like when i'm angry and like i try
to imagine
try to imagine if there would be
like a
small baby alex like a five years old
who's angry angry and then you're like
you shouldn't be angry like stop being
angry yeah but like instead actually you
want the legs to come over give him a
hug and he's like i say it's fine okay
you can't be angry as long as you want
yeah then he would stop
or
or maybe not or maybe not but you cannot
expect it even yeah
but still that doesn't explain the why
of love like why is love part of the
human condition why is it useful
to combine the reward functions
it seems like
that doesn't i mean
i don't think reinforcement learning
frameworks can give us answers to why
even even the hudder
framework has an objective function
that's static so we came to existence as
a consequence of evolutionary process
and in some sense the purpose of
evolution is survival and then the
this
complicated optimization objective
baked into us let's say compression
which might help us
operate in the real world and it bake
into us various reward functions yeah
and then to be clear at the moment we
are operating in the regime which is
somewhat out of distribution where the
event evolution optimized us it's almost
like love is a consequence of
cooperation that we've discovered is
useful correct in some way it's even the
case if you i just love the idea that
love is like the out of distribution
or it's not out of distribution it's
like as you that it evolved for
cooperation
yes and i believe that the cop like a in
some sense cooperation ends up helping
each of us individually so it makes
sense evolutionary and there is a in
some sense and you know love means there
is this dissolution of boundaries that
you have a shared reward function and we
evolve to actually identify ourselves
with larger groups so we we can identify
ourselves you know with a family we can
identify ourselves with a country to
such an extent that people are willing
to give away their life for country
[Music]
so there is we are wired actually even
uh
for love and at the moment i guess
the
maybe
it would be somewhat more beneficial if
you will if we would identify ourselves
with all the humanity as a whole so so
you can clearly see when people travel
around the world when they run into
person from the same country they say oh
which ctr and all this like all of a
sudden they find all these similarities
they they they find some they befriend
those folks earlier than others so there
is like a sense some sense of the
belonging and i would say i think it
would be overall good thing to the word
for people
to
move towards
i think it's even called open
individualism and move toward the
mindset of a larger and larger groups so
the challenge there
that's a beautiful vision and i share it
to expand that circle of empathy that
circle of love towards the entirety of
humanity but then you start to ask well
where do you draw the line
because why not expand it to other
conscious beings and then at the finally
for our discussion
something i think about
is why not expand it to ai systems
like we we start respecting each other
when the other the person the entity on
the other side
has the capacity to suffer because then
we develop a capacity to sort of
empathize
and so
i could see ai systems that are
interacting with humans
more and more having conscious like
displays
so like they display consciousness
through language and through other means
and so then the question is like well is
that consciousness
because they're acting conscious
and so
you know the reason we don't like
torturing animals
is because
they look like they're suffering when
they're tortured
and if ai looks like it's suffering
when it's tortured
how is that not
requiring of the same kind of empathy
from us and respect and rights
that animals do and other humans do i
think it requires empathy as well i mean
i would like
i guess us or humanity or so make a
progress in
understanding what consciousness is
because i don't want just to be speaking
about that the philosophy but rather
actually make a scientific uh to have a
like a you know there was a time that
people thought that
there is a force
of life
and
the
things that have this force they are
alive
and
i think that there is actually a path to
understand exactly what consciousness is
and
um in some sense it might require
essentially putting probes inside of a
human brain
what neuralink
does so the goal there i mean there's
several things with consciousness that
make it a real discipline which is one
is rigorous measurement of consciousness
and then the other is the engineering of
consciousness which may or may not be
related i mean you could also run into
trouble like for example in the united
states
for the department d.o.t department of
transportation and a lot of different
places put a value on human life
i think dot's
uh values nine million dollars per
person
sort of in that same way you can get
into trouble
if you put a number on how conscious a
being is
because then you can start making policy
if a cow
is uh 0.1
or like um
10 as conscious as a human then you can
start making calculations and might get
you into trouble but then again that
might be a very good way to do it
i would like uh
to move to that place that actually we
have scientific understanding what
consciousness is yeah and then we'll be
able to actually assign value and i
believe that there is even the path for
the experimentation in it so uh you know
we said that you know you could put the
probes inside of the brain there is
actually few other things that you could
do with devices like neuralink so you
could imagine that the way even to
measure if ai system is conscious
is by literally just plugging into the
brain
and i mean that that seems that's kind
of easy but the plugging into the brain
and asking person if they feel that
their consciousness expanded
this direction of course has some issues
you can say you know if someone takes a
psychedelic drug they might feel that
their consciousness expanded even though
that drug itself is not conscious
right so like you can't fully trust the
self-report of a person saying their
their consciousness is expanded or not
let me ask you a little bit about
psychedelics because uh there's been a
lot of excellent research on uh
different psychedelics psilocybin mdma
yeah even dmt
drugs in general marijuana too
uh what do you think psychedelics do to
the human mind it seems they take
the human mind to some interesting
places
is that just a little uh hack
a visual hack
or is there some profound expansion of
the mind
so let's see i i don't believe in magic
i believe in that i believe in
in science in
in causality
still let's say and then as i said like
i think that the brain
that the our subjective experience of
reality is uh
we live in the simulation run by our
brain and the simulation that our brain
runs
they can be very pleasant or very
hellish
drugs they are changing some hyper
parameters of the simulation it is
possible thanks to change of these hyper
parameters to actually look back on your
experience and even see that the given
things that we took for granted they are
changeable
so they allow to have a
amazing perspective there is also
for instance the fact that after dmt
people can see the
full movie inside of their head
gives me further belief
that the brain can generate that full
movie that the brain is actually
learning the model of reality to such
extent that it tries to predict what's
going to happen next yeah very high
resolution so it can replay realities
actually extremely high resolution
and it's also kind of interesting to me
that somehow there seems to be some
similarity between
these uh drugs and meditation itself and
i actually started even these days to
think about meditation as a psychedelic
and do you practice meditation
i i practice meditation i mean i once
few times on the
retreats and it feels after like after
second or third day of meditation
there is a there is almost like a sense
of you know tripping
what does the meditation retreat entail
so
i mean you you wake up early in the
morning and you meditate for extended
period of time
and alone
yeah so it's optimized even though there
are other people it's optimized for
isolation so you don't speak with anyone
you don't actually look into other
people's eyes
and
you know you sit on the chair and
say the passage meditation tells you uh
to focus on the breath so you try to put
all the all attention into breathing and
breathing in and breathing out
and the
crazy thing is that as you focus
attention like that
after some time
their stamps starts coming back like
some
memories that you completely forgotten
it almost feels like um that you have a
mailbox and then you
you know you are just like a archiving
email one by one
and at some point at some point there is
like a
amazing feeling
of getting to mailbox zero
zero emails and uh it's very pleasant
it's it's kind of it's it's
it's
crazy to me
that
that once
you resolve these
inner stories or like inner traumas
then once there is nothing
uh left
the default state of human mind is
extremely peaceful and happy extreme
like some sense it it feels that
it feels
at least to me in the way how when i was
a
child that i can look at any object and
it's very beautiful i