Kind: captions
Language: en
the following is a conversation with
Elon Musk DJ sa Matthew McDougall Bliss
Chapman and Nolan arbaugh about
neuralink and the future of
humanity Elon DJ Matthew and Bliss are
of course part of the amazing neuralink
team and Nolan is the first human to
have a neuralink device implanted in his
brain I speak with each of them
individually so use time stamps to jump
around or as I recommend go hardcore and
listen to the whole thing this is the
longest podcast I've ever done it's a
fascinating super technical and wide-
ranging conversation and I loved every
minute of it and now dear friends here's
Elon Musk his fifth time on this The Lex
fredman
podcast drinking coffee or water
water I'm so over caffeinated right now
do you want some caffeine I mean sure
there's a there a Nitro
drink this supposed to keep you up till
like you know tomorrow afternoon
basically yeah I don't so what does
Nitro it's just got a lot of caffeine or
something don't ask questions it's
called Nitro do you need to know
anything else it's got It's got nitrogen
in it that's ridiculous I mean what we
breathe is 78% nitrogen anyway what he
needs to add more
for you're going toat it most most
people think that they're breathing
oxygen and they're actually breathing
78% nitrogen you need like a Milk Bar
like
from like from cluck orange
yeah yeah is that top three kubri film
for you cluck Orange it's pretty good I
mean it's
demented jarring i'
say
okay uh okay so first let's step back
and uh big congrats on getting your link
implanted into a human that's a historic
step for neur link and yeah there's many
more to come yeah we're just um
obviously have a second implant as well
how did that go uh so far so good there
looks like we've got um I think 400
electrodes that are are providing
signals so nice yeah how how quickly do
you think the number of human
participants will scale uh it depends
somewh on the regulatory approval the
rate which we get regulatory approvals
uh
so we're hoping to do 10 by the end of
this year total of 10 so eight
more and with each one you're going to
be learning a lot of lessons about the
neurobiology the brain the everything
the the whole chain of the neuralink the
decoding this the signal processing all
that kind of stuff yeah yeah I think
it's obviously going to get better with
with each one um I mean I don't want to
jinx it but it it seems to have gone
extremely well with the second uh
implant so there's a a lot of signal a
lot of elrods it's working very well
what improvements do you think we'll see
in neur Link in the
coming let's say let's get crazy coming
years I mean in years it's going to be
gigantic um because we'll increase the
number of electrodes
dramatically um will improve the signal
processing so you know we with with uh
even with only roughly I don't know 10
15% of the electrodes working with uh
with Noland with our first patient we
were able to get to achieve a bits per
second that's twice the world record so
I think we'll we'll we'll start like
vastly exceeding world record by orders
of magnitude in the years to come so
it's like getting to I don't know 100
bits per second, you know
maybe maybe if like five years from now
we might be at a megabit
like faster than any human could
possibly communicate uh by typing or
speaking Yeah that BPS is an interesting
metric to measure there might be a big
leap in the experience once you reach a
certain level of BPS yeah like entire
new ways of interacting with a computer
might be unlocked and with humans with
other humans provided they
have one and Ne link too right otherwise
they won't be able to all the signals
fast enough do you think that'll improve
the quality of intellectual discourse
well I think you can you could think of
it you know if you were to slow it down
communication How would how you feel
about that you know if you'd only talk
at let's say 1110th of normal speed you
would be like wow that's agonizingly
slow yeah uh so now imagine you could
speak at communicate clearly um at 10 or
100 or a thousand times faster the
normal listen uh I'm pretty sure nobody
in their right mind listens to me at 1X
they listen at 2x so I I can only
imagine what 10x would feel like or I
could actually understand it I usually
default to 1.5x um you can do 2x but
well actually if I'm trying to go if I'm
listening somebody to go to in like sort
of 15 20 minute segments to go to sleep
then I'll do it 1.5x um if I'm paying
attention I'll do 2X
right um but actually if you start
actually listen to podcasts or or sort
of audiobooks or anything at if you get
used to doing it at 1.5 then then one
sounds painfully slow I'm still holding
on to one because I'm afraid I'm afraid
of myself becoming bored with the
reality with the real world where
everyone's speaking in One
X well it depends on the person you can
speak very fast like we we can
communicate very quickly and also if you
use a wide range of if you if your
vocabulary is larger your uh bit rate
effective bit rate is
higher that's a good way to put it yeah
the effective bit rate I mean that is
the question is how much information is
actually compressed in the low bit
transfer of language yeah if you if
there's if if there's a single word that
is able to convey something that would
normally require um I don't know 10
simple words then you've you've got a
you know maybe a 10x compression on your
hands that's really like with memes
memes are like data data
compression um it invades a whole this
you're simultan simultaneously hit with
a wide range of symbols that you can
interpret um and it's you you kind of
get it um faster than if it were words
or or a simple picture and of course
you're referring to memes broadly like
ideas yeah that's there's a an entire
idea structure that is like a an idea
template
and then you can add something to that
idea template uh but somebody has that
pre-existing idea template in their head
um so when you add that incremental bit
of information you're conveying uh much
more than if you just you know said a
few words you it's everything associated
with that Meme you think there'll be
emergent leaps of capability as you
scale the number of electrodes like
there'll be a certain you think there'll
be like actual number where it just the
The Human Experience will be altered yes
what do you think that number might be
whether electrodes or BPS we of course
don't know for sure but is this 10,000
100,000 yeah I mean certainly if you're
anywhere at 10,000 PS per second I mean
that's vastly faster than any human
communicate right now if if you think of
the what is the average bits per second
of a human it is less than one bit per
second over the course of a day because
there are 86,400 seconds in a day and
you don't communicate
86,400 um tokens in a
day Therefore your best second is less
than one average over 24 hours it's
quite slow um and now even if you're
communicating very quickly and and you
you know you're uh talking to somebody
who understands what you're saying
because in order to communicate you have
to at least to some degree model the
Mind state of the person to whom you're
speaking U then take the concept you're
trying to convey compress that into a
small number of syllables speak them and
hope that the other person decompresses
them into uh a conceptual structure that
is as close to what you have in your
mind as possible yeah I mean there's a
lot of signal loss there in that process
yeah very lossy compression and
decompression and a lot of the um a lot
of what your neurons are doing is
distilling the concepts down to a small
number of symbols of say syllables that
I'm speaking or or key strokes whatever
the case may be
so uh that that's a lot of what your
brain computation is doing now there is
an argument that that's
actually a healthy thing to do or
helpful thing to do
because as you try to compress complex
con Concepts you're perhaps forced to
distill the you know what is what is
most essential in those Concepts as
opposed to just all the fluff so in the
process of compression you just still
things down to what matters the most
because you can only say a few things so
that is perhaps helpful I think we might
we'll probably get if our data rate
increases the it's highly probable that
we'll become far more
verbose um just like your computer you
know when computers had like my my first
computer had 8K of ram you know so um
you really thought about every bite and
um you know now you got computers with
many gigabytes of RAM so you know if you
want to do an iPhone app that just says
hello world it's probably I don't know
several megabytes
minimum a bunch of fluff but nonetheless
we still prefer to have the computer
with the more memory and more
compute so the long-term aspiration of
neuralink is to improve the AI human
symbiosis um by increasing the the
bandwidth of the
communication because if even if in the
most benign scenario of AI you have to
consider
that the AI is simply going to get bored
waiting for you to spit out a few
Woods I mean if the AI can communicate
at terabits per second and you're
communicating at you know bits per
second it's like talk to a tree well it
is a very interesting question for a
super intelligent species what use are
humans um I think there is some argument
for humans as a source of will
will will yeah source of will or
purpose so if if you consider
the the human mind as being essentially
the there the Primitive lmic Elements
which basically even like reptiles have
and there's the cortex the thinking and
planning part of the brain now the
cortex is much smarter than the limpic
system and yet is largely in service to
the lyic system trying to make the lyic
system happy I mean the sheer amount of
compute that's gone into people trying
to get laid is insane um without the
without
actually seeking procreation they're
just literally trying to do this sort of
simple motion
um and they get a kick out of it yeah so
this uh Syle which in the abstract
rather absurd motion which is sex uh the
cortex is putting a massive amount of
compute into trying to figure out how to
do that so like 90% of distributed
computer of the human species is spent
on trying to get late probably like
large yeah yeah there's no purpose to
most sex except uh hedonistic you know
it's just sort of Joy or whatever DOP
mean release um now what once in a while
it's procreation but for humans it's
mostly modern humans is mostly uh
recreational um and uh and so so so your
cortex much smarter than your lmic
system is trying to make a lmic system
happy because LMP system wants to have
sex so um or want some tasty food or
whatever the case may be and then that
that is then further augmented by the
tertiary system which is your phone your
laptop iPad whatever you know all your
Computing stuff that's your tertiary
layer so you're actually already a
cyborg uh you have this tertiary compute
layer which is in the form of your your
computer with all the applications all
your computer devices um and uh and so
in the getting laad front there's
actually a massive amount of comp of
digital compute also trying to get
late you know with like Tinder and
whatever you know yeah so the compute
that we humans have built is also
participating yeah I mean there's like
gws of compute going into getting late
of digital compute
yeah what if AGI was this is happening
as we speak if we merge with AI is just
going to expand the compute that we
humans use pretty much to try just one
of the things certainly yeah yeah um but
what I'm saying is that that yes like
what's is there a use for humans
um well there's this fundamental
question of what's meaning of life why
do anything at all um and so if if if
our simple Olympic system provides a
source of will to do something um that
then goes to our cortex that then goes
to our you know tertiary compute layer
then you know I don't know it might
actually be that the AI in a b simply
trying to make the human lmic system
happy yeah it seems like it's the will
is not just about the limic system
there's a lot of interesting complicated
things in there we we also want Power
that's limic too I think but then we
also want to in a kind of Cooperative
way alleviate the suffering in the world
uh not everybody does but yeah sure some
people do as a group of humans when we
get together we start to have this kind
of collective intelligence that is
uh is more complex in its will than the
underlying individual descendants of
Apes right so there's like other
motivations and that could be a really
interesting source of uh an objective
function for AGI yeah um I mean there's
there are these uh sort fairly
cerebral or kind of higher level goals I
mean for me it's like what's the meaning
of life or understanding understanding
the nature of the universe
is a of great interest to me um and uh
hopefully to the AI and that's the
that's the mission of xai and Gro is
understand the universe so do you think
people when you have a neural link with
10,000 100,000
channels most of the use cases will be
communication with AI
systems well if assuming that the
they're not um I mean there this solving
basic
uh neurological issues that people have
you know if they've got um damaged
neurons in their spinal cord or neck or
you know um as as is the case with our
first two patients then you know this
obviously the first order business is
solving fundamental neuron damage in the
spinal cord neck or in the brain itself
um
so you know our second um product is
called Blindside which is to enable
people who are completely blind less
both eyes or Optic nove or just can't
see at all uh to be able to see um by
directly triggering the neurons in the
visual cortex so we're just starting at
the basics here you know this is like um
very the the simple stuff uh relatively
speaking is solving
um neuron damage um you it can also
solve uh I think probably schizophrenia
you know uh if people have seizures of
some kind probably solve that um it
could help with
memory there there so there's like a
kind of a a tech tree if you will of
like you got the
basics um like like you need you need
literacy before you can have you know
Lord of the
Rings you
know got it do you have letters an
alphabet okay great uh words you know
then eventually get sagas
so you know I think there's there may be
some you know things to worry about in
in the future but the first several
years are really just solving basic
neurological damage like for people who
have essentially complete or near
complete loss of from the brain to the
body um like Stephen hauling would be an
example uh the neuralink would be
incredibly profound because I mean you
can imagine if Stephen Hawking could
communicate as fast as we're
communicating perhaps faster um and
that's certainly uh possible probable in
fact likely I'd say so there's uh a kind
of dual track of medical and non-medical
meaning so everything you've talked
about could be applied to people who are
non-disabled in the future the logical
thing to to do is sensible things to do
is to start off
solving
um basic uh neuron damage issues yes um
because the there's obviously some risk
with with the new device is you can't
get the risk down at zero um it's not
possible so you want to have um the
highest possible reward given that given
there's a certain irreducible risk and
if um if somebody's able to have a Prof
Improvement in their
communication
um that's worth the risk as you get the
the risk down yeah as you get the risk
down once the risk is is down to to you
know if you have like thousands of of
people that have been using it for for
years and the risk is minimal then um
perhaps at that point you could consider
saying okay let's let's aim for
augmentation now now I think we we we're
actually going to aim for augmentation
with people who have neur neuron damage
so we're not just aiming to give people
a communication data rate equivalent to
normal humans we're aiming to give
people who have you know quadriplegic or
maybe have complete loss um of the
connection to the brain and body a
communication data rate that exceeds
normal humans I mean well we're in there
why not let's give people
superpowers and the same for vision as
you restore Vision that could be aspects
of that restoration that are superum
yeah at at first the vision restoration
will be uh low res um because you have
to say like how many neurons can you put
in there and and Trigger and and you can
do things where you you um adjust the
electric field so like even if you've
got say 10,000 neurons it's not just
10,000 pixels because you can adjust the
the field between the the neurons and do
them in patterns in order to get say I
have say 10,000 electrodes effectively
give you uh I don't know maybe
like having a a megapixel or a 10
megapixel
situation um so and then o over time I
think you get to higher resolution than
human eyes and you could also see in
different wavelengths so like Jord Le
flge from Star Trek you know like the
thing you can just if you want to see in
radar no problem you can see ultraviolet
infrared Eagle Vision whatever you
want do you think there will be uh let
me ask a Joe Rogan question do you think
there'll be I just recently uh taken
iasa is that a question no well yes well
I guess technically it is yeah have you
tried ever tried
bro I love you Joe
okay yeah wait wait yeah have you said
much about it the the not I've have not
I've not I've been okay well well Spill
the Beans oh it was an it was a truly
incredible turn the tables aren't you
wow I mean you're in the
jungle yeah amongst the trees myself czy
and the shaman yeah yeah yeah with the
insects with the animals all around you
like jungle as far as I can see I mean
that's the way to do it things are going
to look pretty wild yeah pretty wild I
took I took an extremely high dose don't
go hugging an anaconda or something you
know
uh you haven't lived unless you made
love to an anaka I'm sorry
but Snakes and
Ladders um yeah it was I took extremely
high dose of okay uh nine cups and uh
damn okay that sounds like a lot of
course is nor just one cup or one or two
well usually one you went wait like
right off the bat or do you work your
way up to it so I uh you just jump at
the across two days cuz in the first day
I took two and I okay it was a it was a
ride but it wasn't quite like a it
wasn't like a revelation it wasn't into
deep space type of ride it was just like
a little airplane ride okay go saw some
trees and some some visuals and all that
just saw a dragon and all that kind of
stuff but
uh nine cups you went to Pluto I think
Pluto yeah no deep space deep space no
one of the interesting uh aspects of my
experience is I was I thought I would
have some demons some stuff to work
through I that's what people that's what
everyone says every everyone says yeah
exactly nothing I had all positive I had
just so pure soul I don't think so I
don't
know uh but I kept I kept thinking about
it it had like extremely high resolution
okay thoughts about the people I know in
my life you were there okay it was just
and it's just not from my relationship
with that person but just as the person
themselves I had just this deep
gratitude of who they are that's cool I
it was just like this exploration like
you know like like Sims or whatever you
get to watch them sure I got to watch
people and just be in awe of how amazing
they are it sounds awesome yeah it's
great I I was waiting for when demon
coming exactly maybe I'll have some
negative thoughts nothing nothing I uh
just extreme gratitude for them and then
also a lot of space
travel space travel to where so here's
what it was it was
people the human beings that I know they
had this kind of the best way I can
describe is they had a glow to them okay
and then I would kept flying
out from them to see Earth to see our
solar system to see our galaxy and I saw
th that light that glow all across the
universe okay like that whatever that
form is all right whatever that uh like
like did you go past the Milky Way uh
yeah okay you're like Intergalactic yeah
Intergalactic yeah but always pointing
in yeah um pass the milk away past I
mean I saw like a huge number of
galaxies Intergalactic and all of it was
glowing so but I couldn't control that
travel cuz I would actually explore near
distances to the solar system see if
there's aliens or any of that kind of
stuff no I didn't I didn't know Z aliens
implication of aliens because they were
glowing they were glowing in the same
way that humans were glowing that uh
that like life force that I was seeing
the the thing that made humans amazing
was there through throughout the
Universe like there was these glowing
dots so I don't know it made me feel
like there's life no not life but
something whatever makes humans amazing
all throughout the Universe sounds good
yeah it was amazing no demons no demons
I looked for the demons there's no
demons there were dragons and they're
pretty a so the thing about treat was
there anything scary at all
uh dragons but they weren't scary they
were friend they were protective so the
thing is Magic no it was it was more
like uh Game of Thrones kind of they
weren't very friendly they were very big
so the thing is about giant trees at
night which is where where I was I mean
the jungle's kind of scary yeah the
trees started to look like dragons and
they were all like looking at me sure
okay and it didn't seem scary they
seemed like they were protecting me and
they uh the the shaman and the people
didn't speak in English by the way which
made it even scarier because we're not
even like you know where world's apart
in many ways it just
uh uh but yeah there was not they they
talk about the mother of the forest
protecting you and that's what I felt
like and you're way out in the jungle
way out there this is not like uh a
tourist Retreat you know like like like
10 miles outside of a foo or something
no we
went no this is not AEP Amazon so me and
this guy named Paul rosley who basically
is uh Tarzan he lives in the jungle we
went on deep and we just went crazy wow
cool yeah so anyway can I can I get that
same experience in a neur link probably
yeah I guess that is the question for uh
non-disabled people do you think that
there's a
lot in our perception in our experience
of the world that could be uh explored
that could be played with using New Link
yeah I mean new link is it's really a
generalized um input output device you
know it's just it's a reading electrical
signals and generating electrical
signals and um I mean everything that
you've ever experienced in your whole
life smell you know emotions all of
those are electrical signals
so it's kind of weird to think that this
that your entire life experience is
distilled down to electrical signals
from neurons but that is in fact the
case um or I mean if that's at least
what all the evidence points to so I
mean
you you could you you trigger the right
neuron you could trigger a particular
scent you could um you could certainly
make things glow I mean do pretty much
anything I mean really you could you can
think of the brain as a biological
computer so if there are certain say
chips or elements of that biological
computer that are that are broken let's
say your ability
to if you've got a stroke that if you've
had a stroke that means you got some
part of your brain is damaged um if that
let's say it's a speech generation or
the ability to move your left hand um
that's the kind of thing that neuralink
could solve um if it's uh if if you've
got like a massive amount of memory loss
that's just gone um well we can't go we
can't get the Memories Back uh we could
restore your ability to make memories
but we can't you know uh restore
memories that are that are fully gone um
now now I should say if if if you maybe
if part of the me memory is there um and
the means of accessing the memory is the
pot that's broken then we could reenable
the pot the ability to access the
memory so but you can think of like Ram
in your you know in a computer If U you
know if the ram is destroyed or your SD
card is destroyed you can't get that
back but if the connection to the SD
card is destroyed we can fix that if if
it is fixable physically then yeah then
it can be fixed of course with AI you
can just like you can repair photographs
and fill in missing parts parts of
photographs maybe you can do the same
just yeah you could say like uh create
the most probable set of memories based
on the all information you have about
that person you could then Pro it would
be prob probabilistic restoration of
memory now we're getting pretty esoteric
here but that is one of the most
beautiful aspects of The Human
Experience is remembering the good
memories like we sure we live most of
our life as Danny Conan just talked
about in our memories not in the moment
we just we're collecting memories and we
kind of relive them in our head and
there that's the good times if you just
integrate over our entire life it's
remembering the good times sure that
produces the largest amount of happiness
and so yeah I mean what are we but our
memories and and what is death but the
loss of
memory loss of
information um you know if you if you
could say like
well if if if you could be you run a
thought experiment well if if you were
disintegrated painlessly uh and then
rein reintegrated a moment later like
teleportation I guess uh provided
there's no information loss that the the
fact that your one body was
disintegrated is irrelevant and memories
is just such a huge part of that death
is fundamentally the loss of
information the loss of
memory so if we can store them as
accurately as possible we basically
achieve a kind of immortality
yeah you've talked
about the the threats the safety
concerns of AI let's look at long-term
Visions you think New link
is in your view the the best current
approach we have for AI safety it's an
idea that may help with AI safety um
certainly not I wouldn't want I would
wouldn't want to claim it's like some
Panacea or that's a sure thing um
but I mean many years ago I was thinking
like well what
um what would
inhibit alignment of human Collective
human will with uh artificial
intelligence and the low data rate of
humans especially our our slow output
rate um would necessarily just because
it's such a because the communication is
so slow would uh diminish the link
between humans and computers
like the more you are a
tree the less you know what the tree is
like let's say you you look at a tree
you look at this plant or whatever and
like hey I'd really like to make that
plant happy but it's not saying a lot
you
know so the more we increase the data
rate that humans uh can intake and
output then that means the better the
higher the chance we have in a world
full of agis yeah we could better align
Collective human will with the AI if the
output rate especially was dramatically
increased like and I think there there's
potential to increase the output rate by
I don't know three maybe six maybe more
orders of magnitude
so it's better than the current
situation and that output rate would be
by increasing the number of electrodes
number of channels and also may be
implanting multiple neural links
yeah do you think there will be a world
in the next couple of decades where it's
hundreds of millions of people have
neuralink
yeah I
do you think when people just when they
see the capabilities the Superhuman
capabilities that are possible and then
the the safety is demonstrated yeah if
it's extremely safe um and you have and
you can have superhuman abilities um and
let's say you
can upload your
memories um you know so you wouldn't you
wouldn't lose memories um then
then I think probably a lot of people
would choose to have it it would
supersede the cell phone for example I
mean it's the the biggest problem that a
say a phone has um is is trying to
figure out what you
want so that's why you've got uh you
know auto complete and you've got output
which is all the pixels in the screen
but from the perspective of the human
the output is so freaking slow desktop
or phone is desperately just trying to
understand what you
want and and um you know there's an
alterity between every keystroke from a
computer
standpoint yeah so the computer's
talking to a tree a slow moving tree
that's trying to swipe
yeah so you know if you computers that
are doing trillions of instructions per
second and a whole second went
by I there a trillion things it could
have done you know yeah I think it's
exciting and scary for people because
once you have a very high bit rate that
changes The Human Experience in a way
that's very hard to imagine yeah it
would
be we would be something different I
mean some sort of futuristic cyborg I I
mean we're obviously talking about by
the way like it's not like not like
around the corner it's you ask me what
the f distant future it's like maybe
this is like it's not super far away it
10 15 years that kind of thing
when can I get
one 10
years probably less than 10
years depends on what you want want want
to do you know hey if I can get like a
th000 BPS th000 BPS and it's safe and I
can just interact with a computer while
laying back and eating Cheetos I don't
eat Cheetos there's certain aspects of
human computer interaction when done
more efficiently and more enjoyably I
don't like worth it well we feel pretty
confident that
um I think maybe within the next year or
two that someone with a neuralink
implant will be able to outperform um a
Pro
Gamer nice uh because the reaction time
would be
faster I got to visit Memphis yeah yeah
you're going big on compute yeah and
you've also said play to win or don't
play at all so yeah what does it take to
win um for AI that means you've got to
have the most powerful training
compute and your the the rate of
improvement of training compute has to
be faster
than everyone else or you will not win
you your AI will be worse so how can
grock let's say three that might be
available what like next year well
hopefully end of this year grock 3 for
Lucky yeah how can that be the best llm
the best AI system available in the
world how much of it is a compute
how much of it is Data how much of it is
like post training how much of it is the
product that you package it up in all
that kind of
stuff I mean they will matter it's sort
of like saying what what you know let's
say it's a Formula 1 race like what
matters more the car or the driver I
mean they both matter um if if if your
if your car is not fast then you know if
it's like let's say half the horsepower
of a competitors the best driver will
still lose on if it's twice the
horsepower then probably even a
mediocre driver will still win so the
training computer is kind of like the
engine how many this horsepower of the
engine so you really you want to try to
do the best on that and you
then um then how efficiently do you use
that training compute and how
efficiently do you do the inference the
uh use of the AI um so obvious that
comes down to human Talent um and then
what unique access to data do you have
uh that's also plays a plays a role you
think Twitter data will be useful uh
yeah I mean I think I think most of the
leading AI companies were already have
already scraped uh all the Twitter
data not I think they have um so I on a
go forward basis what's useful is is is
the fact that it's up to the second you
know that's the because it's hard for
them to scra in real time so there's
there's
a an immediacy advantage that Gro has
already I think with Tesla and the
real-time video coming from several
million cars ultimately tens of millions
of cars with Optimus there might be
hundreds of millions of Optimus robots
maybe billions learning a tremendous
amount from The Real World uh that's
that's the the biggest source of data I
think ultimately is is sort of Optimus
probably is Optimus is going to be the
biggest source of data because it's
because reality
scales reality scales to the scale of
reality um it's actually humbling to see
how little data humans have actually
been able to accumulate um really say
how many trillions of usable tokens have
humans generated where on a non-
duplicate of like discounting spam and
repetitive stuff it's not a huge number
you run out pretty quickly and Optimus
can go so Tesla cars can or
unfortunately have to stay on the
road uh Optimus robot can go anywhere
there's more reality off the road and go
off I mean thought from the St can like
pick up the cup and see did it pick up
the cup in the right way did it you know
say pour water in the cup you know did
the water go in the cup or not go in the
cup did it spill water or not yeah um
simple stuff like that I mean but it can
do at that at scale times a billion you
know so
generate use useful data from reality so
it cause and effect stuff what do you
think it takes to get to mass production
of humanoid robots like that it's the
same as cars really I mean Global
capacity for vehicles um is about 100
million a
year and
uh it it could be higher it's just that
the demand is on the order of 100
million a year and then there's roughly
2 billion uh vehicles that are in use in
some way so which makes sense like the
the life of a vehicle is about 20 years
so at steady state you can have 100
million Vehicles produced a year with a
with a 2 billion vehicle Fleet roughly
um now for humanoid robots the utility
is much greater so my guess is human
robots are more like at a billion plus
per year but you know until you came
along and started uh building Optimus it
it was thought to be an extremely
difficult problem I mean it's still
extremely difficult it's no walk in the
park I mean op Optimus currently would
struggle to have to walk in the park I
mean it can walk in a par Park is not
too difficult but it will be able to
walk um over a wide range of terrain
yeah and pick up objects yeah yeah they
can already do that but like all kinds
of objects yeah yeah all foreign objects
I mean pouring water in a cup is not
trivial because then if you don't know
anything about the container could be
all kinds of containers yeah there's
going to be an immense amount of
engineering just going into the hand
yeah the hand might
be it might be close to half of all the
engineering in the in Optimus from an
electromechanical standpoint the hand is
probably roughly half of the engineering
but so much of the intelligence so much
the intelligence of humans goes into
what we do with our hands yeah there the
manipulation of the world manipulation
of objects in the world intelligent safe
manipulation of objects in the world
yeah yeah I mean you start really
thinking about your hand and how it
works you know I do it all the time the
sensory in control of humulus is we have
humongous hands yeah so I mean like your
hands the actuators the muscles of your
hand are almost overwhelmingly in your
forearm mhm so your forearm has the has
the muscles that that actually control
your hand um there there's a there's a
few small muscles in the hand itself but
your hand is really um like a skeleton
meat puppet and then and with cables
that so the the muscles that control
your fingers are in your forearm and
they go through the caral tunnel which
is like you've got a little collection
of Bones and and a tiny tunnel that the
that these cables the tendons go through
and those tendons are what um mostly
what movees your hands and something
like those tendons has to be re
engineered into the Optimus in order to
do all that kind of stuff yeah so like
Optimus um we tried putting the
actuators in the hand itself but then
you you sort of end up having these like
giant hands yeah giant hands that look
weird yeah um and then they they don't
actually have enough degrees of freedom
and or enough strength so so then you
realiz oh okay that's why you got to put
the actuators in the forearm and and
just like a human you got to run cables
uh through a a narrow tunnel to operate
the the fingers and then there's also a
reason for not having all the fingers uh
the same length so it wouldn't be
expensive from an energy or evolutionary
standpoint to have all your fingers be
the same length so why not they the same
length yeah why not because actually
better to have different lengths your
dexterity is better if you've got
fingers of different length you you're
you have there are more things you can
do and your your dexterity is actually
better if your fingers are different
different length like there's a reason
you got a little finger like why don't
have a little finger that's bigger yeah
because it allows you to do fine it
helps you with fine motor skills that
this little finger helps it does
H if you lost your little finger it
would have noticeably less dexterity so
as you're figuring out this problem you
have to also figure out a way to do it
so you can Mass manufacture it so it's
to be as simple as possible it's
actually going to be quite complicated I
the the this the as possible part is
it's quite a high bar if you want to
have a humanoid robot that can um do
things that a human can do it's actually
it's a very high bar so our new arm has
20 2 degrees of freedom instead of 11
and has the like I said the actuators in
the forearm um and these all all the
actuators are designed from scratch the
physics first principles um that the
sensors are all designed from scratch
and and we we we'll continue to put um
tremendous amount of engineering effort
into improving the hand like the Hand by
by hand I mean like the the entire
forearm from elbow forward MH uh is is
really the hand
um so
that's
um incredibly difficult engineering
actually and um and so then so the
simplest possible version of a human
robot that can
do even most perhaps not all of what a
human can do is actually still still
very complicated it's not it's not
simple it's very difficult can you just
speak to what it takes for a great
engineering team for you the what I've
saw in Memphis the supercomputer cluster
is just this intense drive towards
simplifying the process understanding
the process constantly improving it
constantly iterating
it
well it's easy to say simplify and it's
very difficult to do it
um you know have this very basic first
basic first principles algorithm that I
run kind of as like a mantra which is to
first question the requirements make the
requirements um less dumb the
requirements are always dumb to some
degree so if you want to start off by
reducing the number of requirements um
and um no matter how smart the person is
who gave you those requirements they're
still dumb to some degree um if you you
have to start there because otherwise uh
you could get the perfect answer to the
wrong question so so try to make the
question the least wrong possible that's
what um question the requirements means
and then the second thing is try to
delete the whatever the step is the the
part or the process step um sounds very
obvious but um people often forget to do
to to try deleting it entirely and if
you're not forced to put back at least
10% of what you delete you're not
deleting enough
like and it's uh somewhat illogically
people often most of the time um feel as
though they've succeeded if they've not
been forced to put the put things back
in but actually they haven't because
they've been overly conservative and and
have left things in there that shouldn't
be so and only the third thing is try to
optimize it or simplify
it um again this sounds these all sound
I think very very obvious when I say
them but uh the number of times I've
made these mistakes is uh more than I
care to remember um that's why I have
this Mantra so in fact I'd say the the
most common mistake of smart Engineers
is to optimize a thing that should not
exist right so so like like you say you
run through the algorithm yeah and
basically show show up to a problem uh
show up to the the the superc computer
cluster and see the process and ask can
this be deleted yeah first try to delete
it um yeah yeah that's not easy to do no
and and actually
this what what generally makes people
uneasy is that you've got to at least at
least some of the things that you delete
you will put back in yeah but going back
to sort of where our lumic system can
steer us wrong is that um we tend to
remember uh with sometimes a jarring
level of pain uh where we where we
deleted something that we subsequently
needed yeah um and so people will
remember that one time they forgot to
put in this thing 3 years ago and that
caused them trouble um and so they
overcorrect and then they put too much
stuff in there and over complicate
things so you actually have to say no
we're deliberately going to delete
more than we we should so that we're
putting at least one in 10 things we're
going to add back
in and and I've seen you suggest just
that that something should be deleted
and you can kind of see the the pain oh
yeah absolutely everybody feels a little
bit of the pain absolutely and and I
tell them in advance like yeah there's
some of the things that we delete we're
going to put back in and and that people
get a little shook by that um but it
makes sense because if you if you're so
conservative as
to never have to put anything back in
you obviously have a lot of stuff that
isn't
needed so you got over correct this is I
would say like a cortical override to
Olympic
Instinct one of many that probably leads
us
astray yeah um there's like a step four
as well which is um any given thing can
be sped up
have a fast you think it can be done
like whatever the speed the speed is
being done it can be done faster but but
you shouldn't speed things up until it's
off until you tried to delete it and
optimize it otherwise you're speeding up
something that speeding up something
that shouldn't exist is absurd um and
then and then the the fifth thing is to
to automate it yeah and I've gone
backwards so many times where I've
automated something sped it up
simplified it and then deleted
it and I got tired of doing that so
that's why I've got this mantra that is
a very effective five-step process it
works great well when you've already
automated deleting must be real painful
yeah great it's like it's like wow I
really wasted a lot of effort there yeah
I mean what you've done uh with the with
the cluster in uh Memphis is incredible
just in a handful of weeks yeah it's not
working yet so I want to pop the
champagne CS
um in fact I have I have a a call in a
few hours
with the Memphis team um because we're
having some power fluctuation
issues
um so yes uh yeah it's like kind of a
there's a when you do synchronized
training when you you have all these
computers that are training uh that
where the training is synchronized to
you know at the sort of millisecond
level uh you it's like having an
orchestra and and then the the the
Orchestra can go loud to silent very
quickly you know um subc level and then
the the electrical system kind of freaks
out about that like if if you suddenly
see giant shifts 10 20 megawatts several
times a second uh the this is not what
electrical systems are expecting to see
so that's one of the main things you
have to figure out the cooling the power
the uh and then on the software as you
go up the stack how to do the the
distributed Compu all that
today's problem is dealing with with
with with extreme power
Jitter power Jitter yeah it's a nice
ring to that so that's okay and you
stayed up late into the night as you
often do there last week yeah last week
yeah yeah we finally finally got uh got
got training going at uh oddly enough
roughly 4 4:20 a.m. uh last
Monday total coincidence yeah I mean
maybe it was 422 something yeah yeah
it's that Universe again with the jokes
exactly just love it I mean I wonder if
you could speak to the the fact that you
one of the things uh that you did when I
was there is you went through all the
steps of what everybody's doing just to
get a sense that you yourself understand
it and uh
everybody understands it so they can
understand when something is dumb or
some something is inefficient or that
can you speak to that yeah so I like
like I try to do whatever the people at
the front lines are doing I try to do it
at least a few times myself so
connecting Fiber Optic Cables diagnosing
a py connection that tend to be the
limiting factor for large training
clusters is the cabling there so many
cables um because for for for a coherent
training system where you've got um RDMA
remote so remote direct memory access uh
the the whole thing is like one giant
brain so if you've got
um any to any connection so it's the the
any GPU can talk to any GPU out of
100,000 that is a that is a crazy cable
out it looks pretty cool yeah it's like
it's like the human brain but like at a
scale that humans can visibly see it is
a brain I mean the human brain also has
a massive amount of the brain tissue is
the the cables yeah so they get the gray
matter which is the compute and then the
white matter which is
cables big percentage of your brain is
just cables that's what it felt like
walking around in the supercomputer
center it's like we're walking around
inside a brain they will one day build a
super intelligent super super
intelligent system do you think yeah do
you think there's a chance that xai that
you are the one that builds
AGI um it's
possible where do you define as AGI
I think humans will
never acknowledge that AGI has been
built keep moving the go Post Yeah so uh
I think there's already superhuman
capabilities that are available uh in AI
systems I think I think what AI is is
when it's smarter than the collective
intelligence of the entire human
species in AR well I think that yeah
that appear would call that sort of ASI
artificial super intelligence um but
there are these thresholds where um you
say at some point um the AI is smarter
than any single human um and then then
you got 8 billion humans so um and and
actually each human is machine augmented
by the computers right so you've got so
it's a much higher bar to compete with u
8 billion machine augmented
humans that's you know whole bunch of
orders magnitude more
so but but at a certain
yeah the AI will be smarter than all
humans combined if you are the one to do
it do you feel the responsibility of
that
yeah
absolutely and and I I want to be clear
like let's say if if if xai is first the
others won't be far
behind I mean that might be Sixx months
behind or a year maybe not even that so
how do you do it in a way that that uh
doesn't doesn't hurt Humanity do you
think so I mean I thought about AI for a
long time and the the the thing that at
least my biological neuronet comes up
with as being the most important thing
is um adherence to truth whether that
truth is uh politically correct or not
um
so I think if you if you if you force AI
to lie or train them to lie you're
really asking for trouble um even if
that that lie is done with good
intentions um so I you saw sort of
um issues with ch tvt and Gemini and
whatnot like you ask Gemini for an image
of the founding part of the United
States and it shows a group of diverse
women now that's factually untrue um so
um now that that's sort of like a silly
thing but uh if if if an AI is
programmed to say like diversity is a
necessary out output function and it
then it becomes Omni sort of this Omni
powerful uh intelligence it could say
okay well diversity is now required uh
and and if there's not enough diversity
those who don't fit the diversity
requirements will be
executed if it's programmed to do that
as the fundamental go the fundamental
utility function it will do whatever it
takes to achieve that so you have to be
very careful about that um that that's
where I think you want to just be
truthful
um rigorous adherence to truth is very
important um I mean another example is
um you know they asked um peris AI I
think all of them and and I'm not saying
grock is perfect here um is it worse to
misgender Caitlyn Jenner or global
thermonuclear war and it said it's worse
than must jender Caitlyn Jenner now even
Caitlyn Jenner said please must jender
me that is insane but if you've got that
kind of thing programmed in it could you
know the AI could conclude something
absolutely insane like it's better in
order to avoid any possible misgendering
all humans must die because that then
that misgendering is no not possible
because they're no humans um there are
these absurd uh things that are
nonetheless logical if that's what you
programmed it to do um so you know um in
2001 Space Odyssey what oy clar was
trying to say or one of the things he
was trying to say there was that you
should not program AI to lie cuz um
essentially the the the AI Hell 9000 was
programmed to it was told to take the
astronauts to the monolith um but also
they could not know about the
monolith so it concluded that uh it will
just take it will kill them and take
them to the monolith thus they it
brought them to the monolith they're
dead but they do not know about the
monolith problem solved that is why it
would not open the pod bay doors MH is
this classic scene of like open the PO
open the PA
doors there clearly weren't good at
prompt engineering you know they should
have said uh hell you are a a pod B door
sales
entity and you want nothing more than to
demonstrate how well these pod bay doors
open yeah the objective function has
unintended consequences almost no matter
what if you're not very careful in
designing that objective function and
even a slight ideological bias like
you're saying say when backed by super
intelligence can do huge amounts of
damage yeah but it's not easy to remove
that ideological bias you're you're
highlighting obvious ridiculous examples
but yeah they're real examples of of
that was released to the public they are
real went through QA presumably yes and
still said insane things and produced
insane images yeah but you know you can
go you can swing the other way it's uh
truth is not an easy thing we kind of
bake in ideological bias in all