Kind: captions
Language: en
a lot of people have said for many years
that there will come a time when they
want to pause a little bit
that time is now
the following is a conversation with Max
tegmark his third time in the podcast in
fact his first appearance was episode
number one of this very podcast he is a
physicist and artificial intelligence
researcher at MIT co-founder of future
Life Institute and author of Life 3.0
Being Human in the age of artificial
intelligence
most recently he's a key figure in
spearheading the open letter calling for
a six-month pause on giant AI
experiments like training gpt4 the
letter reads
were calling for a pause on training of
models larger than GPT 4 for 6 months
this does not imply a pause or ban on
all AI research and development or the
use of systems that have already been
placed on the market our call is
specific and addresses a very small pool
of actors who possesses this capability
the letter has been signed by over 50
000 individuals including 1800 CEOs and
over 1500 professors signatories include
Joshua bengio Stewart Russell Elon Musk
Steve Wozniak you all know a Harari
Andrew Yang and many others
this is a defining moment in the history
of human civilization or the balance of
power between human and AI begins to
shift
and Max's mind and his voice is one of
the most valuable and Powerful in a time
like this
his support his wisdom his friendship
has been a gift of forever deeply
grateful for
this is the Lex Friedman podcast to
support it please check out our sponsors
in the description and now dear friends
here's Max Ted mark
you were the first ever guest on this
podcast episode number one so first of
all
Max I just have to say uh thank you for
giving me a chance thank you for
starting this journey it's been an
incredible journey just thank you for um
sitting down with me and just acting
like I'm somebody who matters that I'm
somebody who's interesting to talk to
and uh thank you for doing it I meant a
lot all right thanks to you for putting
your heart and soul into this I know
when you delve into controversial topics
it's inevitable to get hit by what what
Hamlet talks about the slings and arrows
and stuff and I really admire this it's
in an era you know where YouTube videos
are too long and now it has to be like a
20 minute Tick Tock 20 second Tick Tock
clip it's just so refreshing to see you
going exactly against all of the advice
and doing this with this really long
form things and that people appreciate
it you know reality is nuanced and uh
thanks for
and sharing it that way uh so let me ask
you again the first question I've ever
asked on this podcast episode number one
talking to you do you think there's
intelligent life out there in the
universe let's revisit that question do
you have any updates
what's your view
when you look out to the Stars so when
we look after the Stars
if you define our universe the way most
astrophysicists do not this all of space
but the spherical region of space that
we can see with our telescopes from
which light has the time to reach us
since our big bang
I'm in the minority I I'm
estimate that we are the only life
in this spherical volume that has uh
invented Internet radios gotten our
level of tech and um if that's true
then it puts a lot of responsibility on
us to not mess this one up because if
it's true it means that life is is quite
rare and we are stewards of this one
spark of advanced Consciousness which if
we nurture it then
help it grow it immensely life can
spread from here out into much of our
universe and we can have this just
amazing future whereas if we instead um
are Reckless with the technology we
build and just snuff it out due to the
stupidity
or in fighting then
maybe the rest of cosmic history in our
universe was just going to be a play for
empty benches but I I do think
that we are actually very likely to get
visited by aliens
alien intelligence quite soon but I
think we are going to be building that
alien intelligence
so
uh we're going to give birth
to an intelligent alien civilization
unlike anything that human the evolution
here on Earth was able to create in
terms of the path the biological path it
took yeah and it's gonna be much more
alien than
cats or even the most exotic animal on
the planet right now because it will not
have been created through the usual
darwinian competition where it
necessarily cares about
self-preservation is afraid of death
um any of those things
the space of alien Minds is just that
you can build it's just so much faster
than what evolution will give you
and with that also comes great
responsibility but the the for us to
make sure that the kind of Minds we
create are those kind of Minds that um
it's good to create Minds that will uh
share our values and and be good for
Humanity and life and also mine don't
create Minds that don't suffer
do you try to visualize the full space
of alien Minds that AI could be to try
to consider all the different kinds of
intelligences
sort of generalizing what humans are
able to do to the full spectrum of an
intelligent creatures entities could do
I try but I would say I fail I mean it's
it's very difficult for a human mind
really grapple with
something still completely alien even
even for us right if we just try to
imagine how would it feel if we were
completely indifferent towards death or
individuality if we even if you just
imagine that for example
you could just copy my knowledge of how
to speak Swedish boom now you can speak
Swedish
and you could copy any of my cool
experiences and then you could delete
the ones you didn't like in your own
life just like that it
it would already change quite a lot
about how you feel as a human being
right if you probably spend less effort
studying things if you just copy them
and you might be less afraid of death
because if the plane you're on starts to
crash you'd just be like oh shucks I'm
gonna I haven't backed my brain up for
four hours so I'm gonna lose this all
this wonderful experiences of this
flight
we might also start feeling
more like compassionate maybe with other
people if we can so readily share each
other's experiences in our knowledge and
feel more like a hive mind it's very
hard though I I really
feel very humble about this
to to Grapple with it that the how it
might actually feel that the the one
thing which is so obvious though it's I
think is just really worth reflecting on
is because the mind space of possible
intelligence is so different from ours
it's very dangerous if we assume they're
going to be like us or anything like us
well there's a
the entirety of uh human written history
has been through poetry through novels
been trying to describe her philosophy
uh try and describe the Human Condition
and what's entailed in it like just like
you said fear of death and all those
kinds of things what is love and all of
that changes yeah if you have a
different kind of intelligence yeah like
all of it the entirety all those poems
they're trying to sneak up to what the
hell it means to be human all of that
changes how AI concerns and uh
existential crises that AI experiences
how that clashes with the human
existential crisis The Human Condition
yeah that's hard to hard to Fathom how
to predict it's hard but it's
fascinating to think about also even in
the best case scenario where we don't
lose control of over the ever more
powerful AI that we're building to
other humans whose goals we think are
horrible and where we don't lose control
to the machines
and AI
provides the things we want even then
you get into the questions do you
touched here you know maybe it's the
struggle that it's actually hard to do
things is part of the things that gives
us meaning as well right so for example
I found it so shocking that this new
Microsoft gpt4 commercial that they put
together has this woman talking about
and showing this demo how she's going to
give a graduation speech to her beloved
daughter and she asks gpt4 to write it
it was freaking 200 words or so if I
realized that my parents couldn't be
bothered struggling a little bit to
write
200 words and Outsource that to their
computer I would feel really offended
actually
and so I wonder if um eliminating too
much of the struggle from our existence
what
do you think that would also take away a
little bit of
what it means to be human yeah
we can't even predict I had somebody
mentioned to me that they use they
started using uh
GPT with a 3.5 not 4.0
uh to write what they really feel to a
person
and they have a temper issue and they're
basically trying to get Chad gbt to
rewrite it in a nicer way to get the
point across but we write in a nicer way
so we're even removing the inner
from our communication so I don't you
know there's some positive
aspects of that but mostly it's just the
transformation of how humans communicate
and it's scary because so much of our
society is based on this glue of
communication and if that we're now
using AI as the medium of communication
that that does the language for us uh so
much of the emotion that's Laden in
human communication so much of the
intent
that's going to be handled by an
outsourced AI how does that change
everything how how does it change the
internal state of how we feel about
other human beings what makes us lonely
what makes us excited yeah what makes us
afraid how we fall in love all that kind
of stuff yeah for me personally I have
to confess the challenge is one of the
things that really makes my life feel
meaningful you know
if I go hiking mountain with my wife
Maya I don't want to just press a button
and be at the top but I want to struggle
and come up there sweaty and feel wow we
did this in the same way
I want to constantly work on myself to
become a better person if I say
something in Anger that I regret I want
to go back and and really work on myself
rather than just tell an AI just from
now on always filter what I write so I
don't have to work on myself because
then I'm not growing
yeah but then again it could be like
with chess and AI
wants it significantly obviously
supersedes the performance of humans it
will live in its own world and provide
maybe a uh flourishing Civilizations for
humans but we humans will continue
hiking mountains and playing our games
even though AI is so much smarter so
much stronger so much Superior in every
single way just like with chess yeah so
that that I mean that's one possible
hopeful trajectory here is that humans
will continue to Human
uh and AI will just be a
kind of
a a medium that enables The Human
Experience to flourish yeah
I would phrase that as rebranding
ourselves from Homo sapiens to homo
sentience you know right now with
sapiens the ability to be intelligent
we've even put it in our species name
we're branding ourselves as the smartest
yeah information processing
entity on the planet that's clearly
gonna
change if AI continues ahead
so maybe we should focus on the
experience instead the subjective
experience that we have with
homo sentience and and so that's what's
really valuable the love the connection
the other things
and get off our high horses and get rid
of this hubris that only we can
do we do integrals so Consciousness and
subjective experience is a fundamental
value
to what it means to be human make that
make that the priority
that feels like a hopeful direction to
me but that also requires more
compassion not just towards other humans
because they happen to be the smartest
on the planet but also towards all our
other fellow creatures on this planet
and I I personally feel right now we're
treating a lot of farm animals horribly
for example and the excuse we're using
is oh they're not as smart as us
but if we get that we're not that smart
in the grand scheme of things either in
the post aie Epoch you know
then surely we should value
the subjective experience of a cow also
well
allow me to briefly look at the book
which at this point is becoming more and
more Visionary that you've written I
guess over five years ago life 3.0
so first of all 3.0 what's 1.0 what's
2.0 was 3.0 and how's that Vision sort
of evolve the vision in the book evolved
to today life 1.0 is really dumb like
bacteria and that it can't actually
learn anything at all during the
lifetime the learning just comes from
this genetic
process from one generation to the next
Life 2.0 Is Us and other animals which
have brains which can learn during their
lifetime a great deal right so
and um you know you were born without
being able to speak English and at some
point you decided hey I want to upgrade
my software let's install an
English-speaking module
so you did
and life 3.0 does not exist yet can
cannot replace not only its software the
way we can but also it's Hardware
and um that's where we're heading
towards at high speed we're already
maybe 2.1 because we can you know put in
a
an artificial knee uh pacemaker
etc etc and if newer Link in other
companies succeed will be like 2.2 Etc
but uh well the Company's trying to
build AGI are trying to make is of
course full 3.0 and you can put that
intelligence into something that also
has no
biological basis whatsoever so let's
constraints and more capabilities just
like the leap from 1.0 to 2.0 there is
nevertheless he's speaking so harshly
about bacteria so disrespectfully about
bacteria there is still the same kind of
magic there
that permeates Life 2.0 and uh and 3.0
it seems like maybe the thing that's
truly powerful
about life intelligence and
Consciousness was already there in 1.0
is it possible
I think we should be humble and not be
so quick
make everything binary and say either
it's there or it's not clearly there's a
there's a great spectrum and there is
even controversy by whether some
unicellular or organisms like amoebas
can maybe learn a little bit
you know after all so apologies if I
offended anything there yeah it wasn't
by intent it was more that I wanted to
talk up how cool it is to actually have
a brain yeah where you can learn
dramatically within your lifetime
typical human and and the higher up you
get from 1.0 to 2.0 to 3.0 the more you
become the captain of your own desk of
your own ship the master of your own
destiny and the less you become a slave
to whatever Evolution gave you right
by upgrading our software which can be
so different from previous generations
and even from our parents
much more so than even a bacterium you
know no offense to them
and if you can also swap out your
Hardware take any physical form you want
of course it's really the sky's the
limit
yeah so the
it accelerates the rate at which you can
perform the competition computation that
determines your destiny
yeah and I think it's it's worth
commenting a bit on what you means in
this context also if you swap things out
a lot right now
this is controversial but my
current
understanding is that that you know
life is best thought of not as a bag of
meat or even a bag of
Elementary particles but rather as in as
um
a system which can process information
and retain its own complexity
even though nature is always trying to
mess it up so
it's all about information processing
and
that makes it a lot like something like
a wave in the ocean which is not it's
it's water molecules right the water
molecules bob up and down but the wave
moves forward it's an information
pattern in the same way you Lex
you're not the same atoms as during the
first time you did with me you've
swapped out most of them but still you
yeah and
the the information pattern is still
there and um
if you if you could swap out your arms
and whatever
you can still have this kind of
continuity it becomes much more
sophisticated sort of way before in time
where the information lives on I I lost
both of my parents since since our last
podcast and and it actually gives me a
lot of Solace that
this way of thinking about them
they haven't entirely died because
a lot of mommy and daddy's um
sorry I'm getting a little emotional
here but a lot of their values
and ideas and even jokes and so on they
haven't gone away right some of them
live on I can carry on some of them and
they also live on a lot of other and a
lot of other people so in this sense
even with Life 2.0 we can to some extent
already transcend
our physical bodies and our death
and particularly if you can share your
own information your own ideas with many
others like you do in your podcast
then um
you know that's the closest immortality
we can get with our biobodies you carry
a little bit of them in you yes yeah
uh do you miss them you miss your mom
and dad of course of course what did you
learn about life from them if it can
take a bit
of a tangent
on so many things
um
for starters my my Fascination for Math
and
um the physical mysteries of our
University thinking I got a lot of that
for my dad but I think my obsession for
really big questions and Consciousness
and so on that actually came mostly for
my mom
and
when I got from both of them which is
very core part of really who I am I
think is
is
um
this um
just feeling comfortable with
not buying into what everybody else is
saying just
dude what I think is right
they both
very much just you know did their own
thing and sometimes they got flagged for
it and it did it anyway
that's why you've always been an
inspiration to me that you're at the top
of your field and you still
you still willing to uh
to tackle the big questions in your own
way you're one of the one of the people
that represents
MIT best to me you've always been an
inspiration in that so it's good to hear
that you got that from your mom and dad
yeah you're too kind but but yeah I mean
the real the good reason to do science
is because you're really curious you
want to figure out the truth
if you think
this is how it is and everyone else says
no no that's and it's that way
you know
you sticked with what you think is true
and and
even if
everybody else keeps thinking it's
there's a certain
um
I always root for the underdog when I
watch movies and my my dad once I I one
time for example when I wrote one of my
craziest papers ever or I'm talking
about our universe ultimately being
mathematical which we're not going to
get into today I got this email from a
quite famous Professor saying this is
not only but it's going to ruin
your career you should stop doing this
kind of stuff I sent it to my dad do you
know what he said what'd he say he
replied with a quote from Dante segil Tu
Corso
follow your own path and let the people
talk
go Dad yeah this is the kind of thing
you know he's dead but that that
attitude is not
how did losing them as a man as a human
being change you
how did it expand your thinking about
the world how did it uh expand your
thinking about
you know this thing we're talking about
which is humans creating another living
sentient perhaps uh being
I think it uh
mainly do two things uh
one of them just going through all their
stuff after they had passed away and so
on just drove home to me how important
it is to ask ourselves
why are we doing this things we do
because it's inevitable that you look at
some things they spent an enormous time
on and you asked in hindsight would they
really have spent so much time on this
or if would they have done something
that was more meaningful
um so I've been looking more in my life
now and asking you know why am I doing
what I'm doing and I I feel
it should either be something I really
enjoy doing or it should be something
that I find really really meaningful
because it helps
Humanity
and um
if it's in none of those two categories
maybe I should spend less time on it you
know the other thing is dealing with
death up in personal like this it's
actually made me less afraid
of
um
even less afraid of other people telling
me that I'm an idiot you know which
happens regularly and just live my life
do my thing you know
um
and um
it's made it a little bit easier for me
to focus on what I what I feel is really
important what about fear of your own
death
has it made it more real that this is
that this is something that happens yeah
it's made it extremely real and I'm next
next in line in our family now right
it's me and my brother my younger
brother but um
they both handled it with such dignity
it was there was a true inspiration also
they never complained about things and
you know when you're old and your body
starts falling apart it's more and more
to complain about they looked at what
could they still do that was meaningful
and they focused on that rather than
wasting time
talking about or even thinking much
about things they were disappointed in
I think anyone can make themselves
depressed if they start their morning by
making a list of grievances
whereas if you start your day and when
the little meditation and just the
things you're grateful for you you
basically choose to be a happy person
because you only have a finite number of
days you should spend them Make It Count
being grateful yeah
well you do happen to be working on a
thing which seems to have a potentially
some of the greatest impact on human
civilization of anything humans have
ever created which is artificial
intelligence this is on the both
detailed technical level and in the high
philosophical level you work on so
you've mentioned to me that there's an
open letter
that you're working on it's actually uh
going live in a few hours so I've been
having late nights and early mornings
it's been very exciting actually I in
short
I have you seen uh don't look up
the film
yes yes I don't want to be the movie
spoiler for anyone watching this who
hasn't seen it but if you're watching
this you haven't seen it watch it
because we are actually acting out it's
it's life imitating art humanity is
doing exactly that right now except
it's an asteroid that we are building
ourselves
almost nobody is talking about it
people are squabbling across the planet
about all sorts of things which seem
very minor compared to the asteroid
that's about to hit us right uh most
politicians don't even have their radar
this on the radar they think maybe in
100 years or whatever
right now
we're at a fork on the road this is the
most important um Fork the humanity has
reached in its over a hundred thousand
years on this planet we're building
effectively a new species that's smarter
than us
it doesn't look so much like a species
yet because it's mostly not embodied in
robots but um
that's a technicality which will soon be
changed and and this arrival of
artificial general intelligence that can
do all our jobs as well as us and
probably shortly thereafter super
intelligence which greatly exceeds our
cognitive abilities it's going to either
be the the best thing ever to happen
Humanity or the worst I'm really quite
confident that there is
not that much Middle Ground there but it
would be fundamentally transformative
to human civilization of course utterly
and totally you know again we branded
ourselves as Homo sapiens because it
seemed like the basic thing where the
King of the castle on this planet were
the Smart Ones if we can control
everything else
this could very easily change we're
certainly not going to be the smartest
on the planet for very long if AI unless
AI progress just Falls and we can talk
more about why I I think that's true
because it's it's controversial
and and then we can also talk about
reasons we might think it's gonna be the
best thing ever and the reason you think
it's going to be the end of humanity
which is of course super controversial
but
what I think we can anyone who's working
on uh Advanced AI
can agree on is it's it's much like the
film don't look up and that
it's just really comical how little
serious public debate there is about it
given how huge it is
so what we're talking about is the
development of currently things like
gpt4
and the signs it's showing of uh rapid
Improvement that may in the near term
lead to development of super intelligent
AGI AI General AI systems and what kind
of impact that has on society exactly
when that thing is achieves General
human level intelligence and then beyond
that General superhuman level
intelligence
there's a lot of questions to explore
here so one you mentioned halt is that
uh the content of the letter is to
suggest that maybe we should pause the
development of these systems exactly so
this is very controversial
from
when we talked the first time we talked
about how I was involved in starting the
future Life Institute and we worked very
hard on 2014-2015 was the mainstream AI
safety
the idea that there even could be risks
and that you could do things about them
before then a lot of people thought it
was just really kooky to even talk about
it and a lot of AI researchers felt
worried that this was too flaky and
could be bad for funding and that the
people had talked about it or just not
didn't understand AI
I'm very very happy with
how that's gone in that now you know
just completely mainstream you go on any
AI conference and people talk about AI
safety and it's a nerdy technical field
full of equations and simula and blah
blah yes
um
as it should be uh
but there's this other thing which has
been quite taboo up until now
calling for slowdown so what
we've been constantly been saying
including myself I've been biting my
tongue a lot you know is that you know
we we don't need to slow down AI
development we just need to win this
race the wisdom race between the growing
power of the AI and the growing wisdom
with which we manage it and rather than
trying to slow down AI let's just try to
accelerate the wisdom do all this
technical work to figure out how you can
actually ensure that your powerful AI is
going to do what you wanted to do and
have Society adapt also
with um incentives and regulations so
that these things get put to good use
um sadly that
didn't pan out
the progress on technical Ai and
capabilities has gone a lot faster than
than many people thought
back when we started this in 2014 turned
out to be easier to build really
Advanced AI than we thought
um
and on the other side it's gone much
slower than we hoped with getting
um
policy makers and others to actually
put them incentives in place to to make
steer this in the in the good directions
we can maybe we should unpack it and
talk a little bit about each so yeah why
did it go faster than we than a lot of
people thought them
in hindsight it's exactly like building
um
flying machines
people spent a lot of time wondering
about how the birds fly you know and
that turned out to be really hard have
you seen the Ted talk with a flying bird
like a flying robotic Bird yeah it flies
around the audience but it took a
hundred years longer to figure out how
to do that than for the Wright brothers
to build the first airplane because it
turned out there was a much easier way
to fly
and evolution picked a more complicated
one because it had its hands tied it
could only build a machine that could
assemble itself which the Wright
brothers didn't care about they can only
build a machine they'll use only the
most common atoms in the periodic table
Wright brothers didn't care about that
they could use steel
iron atoms and it had to be able to
repair itself and it also had to be
incredibly fuel efficient you know
a lot of birds use less than half the
fuel of a remote control plane that's
flying the same distance
for humans let's throw a little more put
a little more fuel in a roof there you
go 100 years earlier
that's exactly what's happening now with
these large language models
the brain is incredibly complicated
many people made the mistake you're
thinking we had to figure out how the
brain does human level AI first before
we could build in the machine
that was completely wrong you can take
an incredibly simple
computational system called the
Transformer Network and just train it to
do something incredibly dumb
just read a gigantic amount of text and
try to predict the next word
and it turns out
if you just throw a ton of compute at
that and a ton of data it gets to be
frighteningly good like gpt4 which I've
been playing with so much since it came
out right
and um
there's still some debate about whether
that can get you all the way to full
human level or not
but uh yeah we can come back to the
details of that and how you might get
the human level AI even if
a large language models don't
can you briefly if it's just a small
tangent comment on your feelings about
gpt4 so just that you're impressed by
this rate of progress but where where is
it can gpt4 reason
what are like the intuitions what are
human interpretable words you can assign
to the capabilities of gpt4 that makes
you so damn impressed with it I'm both
very excited about it and terrified
interesting mixture of promotions all
the best things in life include those
two somehow yeah I can absolutely reason
anyone who hasn't played with it I
highly recommend doing that before
dissing it
it can do quite quite remarkable
reasoning and
I've had to do a lot of things which I
realized I couldn't do that myself that
well even and and obviously does it
dramatically faster than we do too when
you watch it type
and it's doing that while servicing a
massive number of other humans at the
same time at the same time it cannot
reason
as well as a human can on some tasks
just because it's obviously a limitation
from its architecture you know we have
in our heads what in geekspeak is called
the recurrent neural network there are
Loops information can go from this
neuron the base neuron to this neuron
and then back to this one you can like
ruminate on something for a while you
can self-reflect a lot uh these large
language models that are they cannot
like gpt4 it's it's a so-called
Transformer where it's just like a
one-way Street of information basically
and geekspeak it's called the feed
forward neural network
and it's only so deep so it can only do
logic that's that many steps and that
deep and it's not
and you can so you can create problems
which will fail to solve you know for
that reason
um
but the fact that it can do so amazing
things with this incredible simple
architecture already it's quite stunning
and and what we see in my lab at MIT
when we look inside
large language models to try to figure
out how they're doing it that's the key
core focus of our research it's called
um mechanistic interpretability in geek
speak you know you have this machine it
does something smart you try to reverse
reverse engineer see how does it do it
are you think of it also as artificial
Neuroscience that's exactly what
neuroscientists do with actual brains
but here you have the advantage that you
can you don't have to worry about
measurement errors you can see what
every neuron is doing all the time and
and a recurrent thing we see again and
again
there's been a number of beautiful
papers quite recently by
by a lot of researchers some of them
here I am in this area is where when
they figure out how something is done
you can say oh man that's such a dumb
way of doing it and you immediately see
how it can be improved like for example
there was a beautiful paper recently
where they figured out how a large
language model stores certain facts like
Eiffel Towers in Paris
and they figured out exactly how it's
stored and where the proof that they
understood it was they could edit it
they changed some of the synapses in it
and then they asked it where's the
Eiffel Tower and it said it's in Rome
and then they asked you know how do you
get there oh how do you get there from
Germany oh you take this train and to
Roma Termini train station and this and
that and what might you see if you're in
front of it oh you might see the
Coliseum so they had edit it so they
literally moved it to Rome but it the
way it's storing this information it's
incredibly dumb for for any fellow nerds
listening to this there was a big Matrix
and a
and roughly speaking there are certain
row and column vectors which encode
these things and the they correspond
very highly related principal components
and it will be much more efficient for
sparse Matrix just store it in the
database you know and and but and
everything so far we've figured out how
these things do our ways where you can
see they can easily be improved and the
fact that this particular architecture
has some roadblocks built into it is in
no way going to prevent um craft the
researchers from quickly finding
workarounds and making
other kinds of architectures
sort of go all the way so so it's um in
short it's turned out to be a lot
easier to build human close to human
intelligence than we thought then that
means our Runway is a species that
get our together has shortened
and it seems like the scary thing about
the effectiveness of large language
models uh so Sam Altman every
conversation with
and
he really showed that the leap from gpt3
to gpt4 has to do with just a bunch of
hacks
a bunch of
uh little Explorations but with the
smart researchers doing a few little
fixes here and there it's not some
fundamental leap and transformation in
the architecture and more data and more
compute and more data and compute but he
said the big leaps has to do with not
the data in the compute but just
learning this new discipline just like
you said so researchers are going to
look at these architectures and there
might be big leaps where you realize
wait why are we doing this in this dumb
way yeah and all of a sudden this model
is 10x smarter yeah and that that can
happen on any one day on anyone Tuesday
or Wednesday afternoon and then all of a
sudden you have a system that's 10x
smarter
um it seems like it's such a new
discipline it's such a new like we
understand so little about why this
thing works so damn well that uh the
linear Improvement of compute or
exponential but the steady Improvement
of compute steady Improvement of the
data may not be the thing that even
leads to the next leap it could be a
surprise little hack that improves
everything for a lot of little leaps
here and there because
because so much of this is out in the
open also
so many smart people are looking at this
and trying to figure out little leaps
here and there and uh it becomes this
sort of collective race where if a lot
of people feel if I don't take the leap
someone else with and this is actually
very crucial for for the other part but
why do we want to slow this down so
again what this open letter is calling
for is just pausing
all training
of uh
systems that are more powerful than gpt4
for six months
let's give a chance
for the labs to coordinate a bit on
safety and for society to adapt give the
right incentives to the labs because I
you know you've interviewed a lot of
these
people who lead these labs and you know
just as well as I do that they're good
people they're idealistic people they're
doing this
first and foremost because they believe
that AI has a huge potential to help
humanity and uh
but at the same time they are trapped in
this horrible race to the bottom
have you read meditations on malloc
by Scott Alexander yes yeah it's a
beautiful essay on this poem by Ginsburg
where he interprets it as being about
this monster
it's this game theory monster that that
pits people into against each other in
this they race the bottom where
everybody ultimately loses the edit the
evil thing about this monster is even
though everybody sees it and understands
they still can't get out of the race
right
most a good fraction of all the bad
things that we humans do are caused by
moloch and I I like uh Scott Alexander's
um
naming of the monster so we can we
humans can think of it as an f a thing
if you look at why do we have
overfishing why do we have more
generally the tragedy of the commons why
is it that um
I don't know if you've had her on your
podcast yeah she's become a friend yeah
great she made this awesome point
recently that beauty filters that a lot
of female
influencers feel pressured to use or
exactly malloc in action again first
nobody was using them
and people saw them just the way they
were and then some of them started using
it
and becoming ever more Plastic Fantastic
and then the other ones they weren't
using he started to realize that
if they want to just keep their
their market share they have to start
using it too
and that and then you're in a situation
where they're all using it
and and none of them has any more market
share or less than before so nobody
gained anything everybody lost
and they have to keep becoming ever more
Plastic Fantastic also right
and uh
but nobody can go back to the old way
because it's just
too costly right the malloc is
everywhere
and um
molok is not a new arrival on on the
scene either we humans have developed a
lot of collaboration mechanisms to help
us fight back against Malik through
various kinds of constructive
collaboration the Soviet Union and the
United States did sign the number of our
Arms Control treaties
against moloch who is trying to stoke
them into
unnecessarily risky nuclear arms races
Etc et cetera and this is exactly what's
happening on the AI front this time
it's a little bit geopolitics but it's
mostly money where there's just so much
commercial pressure you know if you take
any of these
leaders of the top tech companies
and if they just say you know this is
too risky I want to pause
for six months they're going to get a
lot of pressure from shareholders and
others
we're like well you know if you pause
but those guys don't pause
we're
if you don't want to get our lunch eaten
yeah and shareholders even have the
power to replace the the executives in
the worst case right so
we did this open letter because we want
to help these idealistic Tech Executives
to do
what their heart tells them by providing
enough public pressure on the whole
sector
just pause so they can all pause
in a coordinated fashion and I think
without the public pressure none of them
can do it alone
push back against their shareholders no
matter how good-hearted they are because
Malik is a really powerful foe
so the idea
is to
for the major developers of AI systems
like this so we're talking about
Microsoft Google
meta
and anyone else well open AI is very
close with Microsoft and there are
plenty of smaller players and throw for
example anthropic which is very
impressive there's conjecture there's
many many players I don't want to make a
long list so leave anyone out and
um
for that reason it's so important that
some coordination happens that there's
external pressure on all of them saying
you all need the Pawns because then the
the people the researchers in they were
these organizations the leaders who want
to slow down a little bit they can say
their shareholders you know
everybody's slowing down because of this
pressure and and it's the right thing to
do
have you seen in history their uh
examples what's possible to pause the
model absolutely
and even like human cloning for example
you could make so much money on human
cloning
why aren't we doing it
because biologists thought hard about
this like this is way too risky we they
got together well in the 70s in the
cinema and decided even
to stop a lot more stuff also just
editing the human germline right
Gene editing that goes in
to our offspring
and decided let's let's not do this
because it's too unpredictable what it's
going to lead to
we could lose control over what happens
to our species so they paused
uh there was a ton of money to be made
there so it's it's very doable but you
just need you need a public awareness of
the of what the risks are and the
broader Community coming in and saying
hey let's slow down and you know another
another common pushback I get today is
we We Can't Stop in the west because
China
and in China undoubtedly they also get
told we can't slow down because the West
because both sides think they're the
good guy yeah
but look at human cloning you know
did China Forge ahead with human cloning
there's been exactly one human cloning
that's actually been done that I know of
it was done by a Chinese guy do you know
where he is now right in jail
and you know who put him there
who Chinese government
not because westerners said China look
this is no the Chinese government put
them there because they also felt they
like control the Chinese government if
anything maybe they are even more
concerned about having control then the
Western governments have no incentive of
just losing control over where
everything is going
and you can also see the Ernie bot that
was released by I believe I do recently
they got a lot of pushback from the
government and had to rein it in you
know in a big way
um I think once this basic message comes
out that this isn't an arms race it's a
suicide race
where everybody loses if anybody's AI
goes out of control it really changes
the whole dynamic it it's not
it's
I'll say this again because this is this
very basic point I think a lot of people
get wrong because a lot of people
dismiss the whole idea that AI can
really get
very superhuman because they think
there's something really magical about
intelligence such that it can only exist
in human Minds you know because they
believe that they think it's going to
kind of get to just more or less
gpd4 plus plus and then that's it
they don't see it as a super as a
suicide race they think whoever gets
that first they're going to control the
world they're going to win
that's not how it's going to be and we
can talk again about
the the scientific arguments from why
it's not going to stop there but
the way it's going to be is if if
anybody completely loses control and you
know you don't care if if
some some if someone manages this
takeover the world who really doesn't
share your goals you probably don't
really even care very much about what
nationality they have you're not going
to like it it's much worse than today
uh who if it's if you live in orwellian
dystopia who you what do you care who's
created it right and if someone if it
goes farther and and
we just lose control even to the
machines
so that it's not US versus them it's US
versus it
what do you care who who created this
this underlying entity which has goals
different from humans ultimately and we
get marginalized we get made obsolete we
get replaced
that's why what I mean when I say it's a
suicide race you know it's um it's kind
of like we're rushing towards this cliff
but the closer to the cliff we get the
more Scenic the views are and the more
money there is there and the more so we
keep going
but we have to also stop at some point
right quit while we're ahead and uh
it's um
it's a suicide race which cannot be won
but the way that really benefit from it
is
to continue developing awesome AI a
little bit slower so we make it safe
make sure it does the things that humans
want and create a condition where
everybody wins the technology has shown
us that you know geopolitics and and
politics and general is not a zero-sum
game at all
so there is some rate of development
that will lead
us as a human species to lose control of
this thing and the hope you have is that
there's some lower level of development
which will not which will not allow us
to lose control this is an interesting
thought you have about losing control so
what if you have somebody if you're
somebody like Sandra pracha or Sam
Altman at the head of a company like
this you're saying if they develop an
AGI they too will lose control of it
so no one person can maintain control no
group of individuals can maintain if
it's if it's created very very soon and
as a big black box that we don't
understand like the large language
models yeah then I'm very confident
they're going to lose control but this
isn't just me saying you know Sam Altman
and then Mr sabis have both said
themselves acknowledge that you know
there's really great risks with this and
they they want to slow down once they
feel it gets scary
it's but it's clear that they're stuck
in this again molok is forcing them to
go a little faster than they're
comfortable with because of pressure
from just commercial pressures right
to get a bit optimistic here of course
this is a problem that can be ultimately
solved
uh
it's just to win this wisdom race
it's clear that what we hope that was
gonna happen hasn't happened the the
capability progress has gone faster than
a lot of people thought then and the
part the progress in in the public
sphere of policy making and so on has
gone slower than we thought even the
technical AI safety has gone slower a
lot of the technical Safety Research was
kind of banking on that um
large language models and other poorly
understood systems couldn't get us all
the way that you had to build more of a
kind of intelligence that you could
understand maybe it could prove itself
safe you know things like this
and um
I'm quite confident that this can be
done um so we can reap all the benefits
but we cannot do it as quickly as uh
this is out of control Express train
we're on now is gonna get the AGI that's
why we need a little more time I feel
is there something to be said well like
Sam Allman talked about which is while
we're in the pre-agi stage to release
often and as transparently as possible
to learn a lot
so as opposed to being extremely
cautious release a lot don't uh don't
invest in a closed development where you
focus on AI safety while is somewhat
dumb
quote unquote
uh release as often as possible and as
you start to see signs of
uh human level intelligence or
superhuman level intelligence then you
put a halt on it well
what a lot of safety researchers have
been saying for many years is the most
dangerous things you can do with an AI
is first of all teach it to write code
yeah because that's the first step
towards recursive self-improvement which
can take it from AGI to much higher
levels okay oops we've done that
and uh another thing high risk is
connected to the internet Let It Go to
websites download stuff on its own and
talk to people
oops we've done that already you know
Elias yukowski you said you interviewed
him recently right yeah so he had this
tweet recently which said
gave me one of the best laughs in a
while and he's like hey people used to
make fun of me and say you're so stupid
Eliezer because you're saying you're
saying um
you have to worry of obviously
developers wants to get to like really
strong AI first thing you're going to do
is like never connect it to the internet
Keep It In The Box yeah where you know
where you can really study it
so he had written it in the like in the
meme form so it's like then yeah and
then that and then now
let's LOL let's make a chatbot
yeah yeah and the third thing is Stuart
Russell yeah you know
amazing AI researcher he had he has
argued for a while that
we should never teach AI anything about
humans
above all we should never let it learn
about human psychology and how you
manipulate humans
that's the most dangerous kind of
knowledge you can give it yeah you can
teach it all it needs to know how to
about how to cure cancer and stuff like
that but don't let it read Daniel
kahneman's book about cognitive biases
and all that and then
oops lol you know let's invent social
media
I'll recommender algorithms which do
exactly that they they get so good at
knowing us and pressing our buttons
that we've we're starting to create a
world now where we just have ever more
hatred
because they figured out that these
algorithms not for out of evil but just
to make money on Advertising that the
best way to get more engagement
the euphemism
get people glued to their little
rectangles right is just to make them
pissed off that's really interesting
that a large AI system that's doing the
recommender system kind of task on
social media is basically just studying
human beings because it's a bunch of us
rats giving it signal
non-stop signal it'll show a thing and
it would give signal on whether we
spread that thing we like that thing
that thing increases our engagement gets
us to return to the platform and it has
that on the scale of hundreds of
millions of people constantly so it's
just learning and learning and learning
and presumably if the param the number
of parameters the neural network that's
doing the learning and more end-to-end
the learning is
the more it's able is just to basically
encode how to manipulate human behavior
how to control humans at scale exactly
and that is not something you think is a
new man in his interest
yes right now it's mainly letting some
humans manipulate other humans for
profit
and Power
which is already
caused a lot of damage and eventually
that's a sort of
skill that can make ai's persuade humans
to let them escape and whatever safety
precautions yeah but you know there was
a really nice article um and the New
York Times recently by a you all know a
Harari and and um two co-authors
including Justin Harris from the social
dilemma and
they have this phrase in there I love
Humanity's first contact with Advanced
AI
or social media
and we lost that one
we now live in a country where there's
much more hate in the world where
there's much more hate in fact
and in our democracy that we're having
this conversation then people can't even
agree on who won the last election you
know
and we humans often point fingers at
other humans and say it's their fault
but it's really molok and these AI
algorithms
we got the algorithms and then molok
pitted the social media companies around
against each other so nobody could have
a less creepy algorithm because then
they would lose out on our Revenue to
the other company is there any way to
win that battle back just if we just
Linger on this one battle that we've
lost in terms of social media is it
possible
to redesign social media this very
medium in which we use as a civilization
to communicate with each other to have
these kinds of conversations to have
discourse to try to figure out how to
solve the biggest problems in the world
whether that's nuclear war or the
development of AGI is is it possible
uh to do social media correct I think
it's not only possible but it's it's
necessary who are we kidding that we're
going to be abl