Kind: captions
Language: en
want to start with a quote from Ilyas at
skever he said for people that don't
know he's a co-founder of openai he said
it may be that today is large General
networks are slightly conscious so I
want to pose that question to you are
computers becoming conscious right now I
think it's uh
a question that doesn't make much sense
because we don't even have a clear
scientific understanding of what
conscious means
So based on that I would say no there
are lots of properties of our
Consciousness that are missing what it
means to be conscious in other words
what sort of computations are going on
in our brain when we become conscious of
something
and and
you know how that is is related to
Notions for example of self or relations
to others
um
our thoughts emerge and how they're
related to each other all kinds of Clues
we have about Consciousness including
how it's implemented in neural circuits
that are completely missing in large
language models all right as we as we
think about Consciousness from an
evolutionary standpoint we think about
its utility
um and for for people that haven't heard
Consciousness defined before it
the I think the easiest way to explain
it is it feels like something to be a
human and so the question is does it
feel like something to be a machine and
the most important question I think as
we think about the dangers of AI and
what's coming is does it matter
is it additional utility for it to feel
like something to be a human or to be a
machine do you agree that that's going
to matter in terms of goal orientation
in terms of quote unquote wanting to do
something as we think about our AI you
know is it going to take over are we
going to be dealing with Killer Robots
or am I totally off base with that my
group put out
um paper just in the last couple of
months
and we propose a theory that that may uh
that is anchored in how brains compute
so the theory has to do with the
dynamical nature of the brain in other
words you know you have a
uh 80 billion neurons and their activity
is changing over time
the trajectory that your brain goes
through is all these neurons change
their activity
tends to converge towards some
configuration when you're becoming
conscious that convergence has
mathematical implications that
would suggest that what we store in our
short-term memory are these thoughts
that are discrete but compositional in
other words like think like a short
sentence and it's also something
ineffable which means
it's very hard to translate in words and
there are good reasons for that it's
just the
uh it would take a huge number of words
to be able to translate the the
trajectory that state of your brain
which is a very very high dimensional
object into words it's just impossible
essentially
so even though we may communicate with
language we may have a different
interpretation of what this means and
especially in particular a different
subjective experience because of our ex
or our life has been different right so
we've learned different ways of
interpreting the world okay if if
Consciousness is a byproduct of the
feeling I get when my particular brain
is honing in on a thought that there is
a neural pattern that becomes
recognizable
um the the thing I think that becomes
important and the reason that I think
this is important as we think about
artificial intelligence potentially
becoming Killer Robots is my big thing
with AI has always been AI has to want
something it has to want an outcome not
necessarily interesting let me finish
that sentence and then we'll pick that
apart but if I'm right and AI has to
want something and that's certainly how
humans behave then I understand the
utility of this ineffable feeling that
you're talking about that we call
consciousness because
for humans to make a decision and know
what direction to go in we must have
emotion if you selectively damage the
region of the brain that controls
emotion people cannot make decisions
they can tell you all the rational
reasons why they should eat fish instead
of beef or beef instead of fish but they
can't then actually decide and do it so
we need that feeling that where this
thing is more desirable than that thing
and so my thinking has always been as it
relates to AI that
if AI doesn't want something it will
never be from an emotional standpoint if
it doesn't feel like anything to be a
robot they will never have the final
decision making capability to care
enough to take over the world
and so that's where it's like if it
becomes conscious and it suddenly feels
like something to be a robot then
they're going to be motivated in a
direction that direction could be bad it
could be good whatever but they're going
to be motivated in a direction now if
they are like humans
but if they never become conscious or it
never feels like anything I would think
they would be much like they are now
where it's like well it could be this it
could be that if you've ever talked to
Chachi petite which of course you have
but that feels like it would sort of be
a Perpetual State of Affairs what might
I be getting wrong
my belief is that you're talking about
two things that are actually quite
separate as if there are one so wanting
something having goals and getting some
kind of internal or external reward for
achieving those goals is something that
we already do in machine learning you
know reinforcement learning is all based
on this and you don't need subjective
experience for that
so these are like really distinct
abilities
subjective experience is related to
thoughts that we discussed earlier we
could have machines that have something
like thoughts and potentially if we
implement it similarly to how it is in
our brain they might have subjective
experience it doesn't mean that they
need to have goals I think we can build
machines that that have these
capabilities in other words they can
help us solve problems by telling us how
you know what is the problem what is the
a good scientific understanding of what
is going on and what might be better
Solutions and but they're not trying to
achieve anything except be as truthful
to the data what they know whether you
have observed what then is the disaster
scenario of something that can pass the
touring test that you're worried enough
that you're saying look we need to treat
this the way that we would treat
anything else dangerous whether that's
the environment whether that's or sorry
climate change or whether that's nuclear
weapons like to to put it on that level
just at the touring test level give me
give me the disaster scenario we already
have trolls right that are trying to
influence people on the internet social
media
but there are humans and you can't scale
the number of trolls very easily this
would be too expensive and maybe people
would not want to do it even if you bait
them
but you can scale AI with just more
compute power
so you could have ai trolls
that
I mean I think there already exists AI
Trolls but they are stupid it's easy to
you know interact with them a little bit
and you see they're not human I mean
they've been repetitive and and so on
and so now we get to the point where
you're going to have ai trolls that
essentially invade are
social media invade or even our email
and in fact they can do they could do
better than that it could be
personalized so right now
it's a little bit difficult for a human
troll to have a good personal
understanding of every person that they
hit on
that to know their history I mean it
would just take too much time for them
to study you
and multiplied by a billion people
but an AI system that could just have
access to all of the interactions that
you've had the videos where you spoke
the texts that's available on the
internet
they could know you a lot better right
so how could that be used
well
it could be used to hit on the right
buttons for you to change your political
opinion on something
it could be used to even fool you into
thinking your
in a conversation with someone you know
because they can know you and they can
know your friend
and they can impersonate your friend at
least text other text up
so I don't think we have these things
but just they're just like one small
step away from having these capabilities
as I was thinking through the same
problem
I was thinking here is a terrifying
example dear parents AI is going to
reach out to you mimicking your child
asking for money and so it's not a
Nigerian prince anymore it's Mom uh I
something happened at school whatever
they talk in their language they
reference things that you you don't
think that they could have possibly put
out there but of course if it's if the
AI is good at image recognition and it
knows that you guys were on a beach
seven years ago like it could it could
replicate things in in the form of a
memory that you would never believe that
anybody else could possibly know but we
leak especially kids leak so much data
out into social media that to your point
that AI would be able to have so much
context so at my last company we got
socially engineered and they convinced
us to wire 50 Grand and when we went
back and looked at the emails back and
forth between our
finance department and the the CEO
it was so believable it wrote like it
was obviously a person but it was
writing like they would write to each
other and
I was just I was really flabbergasted
and so to think that a human could do
that to your point it's very hard for
them to get the amount of contextures to
take so much time but when AI is doing
it and it can churn through everything
that those two people had ever said to
each other
ever online uh that gets really scary
really fast okay so if if we were if we
did this pause the the letter that you
guys wrote and we paused for six months
and we were gonna hold the convention in
that time and all governments were there
Yoshua and you're up on stage and your
job isn't to tell us what to do but it's
to open the conversation in the right
place
where would you open that conversation
what do you want us focused on in term
I'm guessing it's like we need to limit
this or something along those lines
where do you begin
I don't know for sure exactly how these
Technologies could be used you and I can
like make up things maybe some are going
to be easier than we thought something
could be harder
but there's so much uncertainty about
how bad it can turn that we need to be
put it so Prudence here
is something that we need to bring in
our decision making uh
individually because we're gonna be
facing potentially these attacks
uh as as Nations at the planet level
yeah that that's that's that would be my
main message that that the technology
has reached a point where it can be very
damaging and there's too much unknown of
how this can happen when it will happen
and even the strongest expert even the
people who built the latest systems
can't tell you
it means
that we have to get our act together and
mostly is going to come from governments
so we need those people to get quickly
educated and we need to uh
also have Scholars experts not just AI
experts but like you know social
scientists legal Scholars
um psychologists because you know this
is the psychology of how this could be
used how to exploit people's weaknesses
um
in order to
do the the work the research also like
what sort of precautions do we need so
there are very simple things that we can
do very quickly for example
um watermarks and
content
um origin display so watermarks just
means that one accompanies say like open
AI what's up their software they could
easily put out
um another software that anybody could
run that can test with
99.99 confidence where they're uh a text
came from their system or not so he was
wouldn't see the difference
but for a machine that has the right
code it's very easy
if if if if their system is instrumented
properly in other words the kind of
sneak in some bits of information that
are not
you can't notice statistically there is
no difference but
the chances of having this particular
sequence of of words would be very very
unlikely and and would go to zero
quickly is the length of the message
increases so watermarks are easy to put
in technically speaking
and they would say this texts
comes from this company this version
whatever okay so a piece of software
running on your computer would be able
to say oh by the way the text that you
gave me to read is this company blah
blah blah
and then we need that information to be
displayed because of course you know
being able to detect the it's coming
from an AI system is one thing and but
when you have a user interface it should
also be mandatory like if I if I'm a on
a social media in particular and I'm
getting
uh you know I'm interfacing I mean I'm
interacting with some some character out
there online I need to know that that
character is not a human
and so that must be displayed if I get
uh a picture or a video or a text in an
email
I need my
email uh you know uh software to tell me
warning this is coming from you know
open AI GPT 5.6 okay so I'm going to
push back with the obvious thing and I
think I won't even have to play devil's
advocate here I I maybe I'm not more
pessimistic than you but I am in the the
toothpaste is out of the tube and
there's no getting it back in so I as as
a way to move all this forward lets you
and I actually debate the reality of all
this so uh I'm at the governmental
meeting you start saying that my
immediate reaction is Yoshua China is
going to develop this if we don't if we
put the brakes on this they're not going
to and this is a winner take all
scenario we cannot allow ourselves to
get behind
what say you
it's a good it's a good concern
um
and that's why we have to get China
around the table as well and Russia and
all the countries that may have the
capability to to do this but Russia
right now feels hemmed into a corner
they are Putin is literally intimating
that he's going to use nuclear weapons
there's no Universe like we've already
tried Financial sanctions that's caused
them to you know start trading in
non-dollar denominations uh they're
grouping up with China Brazil South
Africa
um they India they don't care like
they're going to use that to their
advantage they're in fact even bluffing
would be a way smarter play for him to
say no no we're going to keep doing it
even if he wasn't even if they're like
backwaters it would be wise of him to
say no in in fact if you don't NATO if
you don't immediately back off we're
going to unleash a troll Farm the likes
of which you've never seen we're going
to completely destroy democracy in the
western world
yeah so first of all uh
we can protect ourselves without
necessarily hampering the research so I
think people misunderstood a letter it
never said stop the eye research
it's mostly about these very large
systems
that can be deployed in the public and
then used potentially in the various
ways that we have to be careful with
it's a tiny tiny sliver of the whole
thing that we're doing
um
second
and and second in the short term we do
have to protect the public in our
societies with things like
like trolls and cyber attacks and and uh
that can exploit AI
um third I I don't know I'm not a note
I don't my comfort zone here in terms of
diplomacy and then you know
you and me both but it's fun
um but but my
my guess is that
um the authoritarian governments are
probably as scared of this technology
but for different reasons
so why are they scared because
the same AI systems that could perturb
our democracies could also challenge
their power
in other words
imagine
AI trolls you know being able to defeat
the
protections of the uh
Chinese firewall and and interacting
with people and you know putting
Democratic ideas in their heads in China
um well that would not be something that
this governments probably would like to
see
um and in fact I think China has been
the fastest moving on regulation
not for the same reasons as we are
so they are afraid of this
so I think they will come to the table
but again like it's not my specialty
with anything but at least we
there's a chance that they they might be
willing to talk and remember
um the nuclear treaties were uh
worked on and signed right in the middle
of the Cold War
so
so long as each party recognizes that
they might have something worse to lose
by not entering those discussions I
think there's a chance we can
have a global coordination and we have
to work even if it's hard we have to
work on it yeah I don't I'm not so
worried about the hard part as I am what
is the natural reaction when you have a
very difficult dangerous thing and
history tells me that we don't come to
the table to sign the non-proliferation
agreement until we have proliferated so
far and we have so many missiles pointed
at each other that we finally go okay
let's not let this go beyond any more
and let's not let it go out to other
countries like we're perfectly fine
being in a stalemate with each other and
I worry that a similar kind of reaction
will be had here but I take your point
that this is not an area where either of
us are an expert as much as I find it
utterly fascinating to pursue that line
of thought but I I want to now go back
to what would we do to actually begin to
limit this stuff so we need to get
people thinking hey this is dangerous
that's clear but then the watermark
thing to me works only for people that
agree that they're going to do it
but is there a way so taking the instead
of trying to get people to not do things
how do we build defensive things that
even when somebody's trying to hack the
system so I doubt you know this about me
but we're building a video game and so
one of the things you have to think
about is this game people will attempt
to hack it like that that is just it
goes without saying so rather than me
trying to ask everybody hey please don't
hack video games like literally it's the
dumbest thing ever for the gamers to
hack the games is stupid you end up
ruining the fun that game will die out
and then people will try to invent a
whole new game far better for everybody
to just let's all agree that we're not
going to hack it but it human nature is
is what it is and that's never going to
work so what they do is they create an
adversarial approach where it's like I
find the best hackers in the world to
come in to try to hack this game and
then I figure out what I would have to
do to defeat that so what would an
adversarial setup look like an AI when
someone's trying not to Watermark but I
can still figure out who that came from
or it had you know is there a signature
or something like that that we could
identify you can reboot your life your
health even your career anything you
want all you need is discipline I can
teach you the tactics that I learned
while growing a billion dollar business
that will allow you to see your goals
through whether you want better health
stronger relationships more successful
career any of that is possible with the
mindset and business programs in Impact
Theory University join the thousands of
students who have already accomplished
amazing things tap now for a free trial
and get started today
watermarks are the easy thing and and
the
I agree they will only be done by the
like
legit actress
um people have already been working on
um machine learning
trained to detect
text or images that come from other
machine learning systems but these
systems are not nearly as good but yes
we we this is already being developed
and uh you know presumably there's going
to be a lot more effort in that
direction and we need that as Plan B
right the plan a is already to reduce
the like right now it's just too easy to
you can have an API and just
right on top of uh chat GPT
um
so yeah we should do all these things uh
by the way the kind of adversarial
approach that you're talking about is
from what I hear and read is also what
openai has been doing and and companies
like like Google have been doing the
um they hire people to try to break
their system as much as they can that's
exactly what they're doing like uh you
know red teams
um
and and that's good we need to continue
doing that
um but maybe we need to make sure
um the the the guidelines for doing that
are shared across the board and people
can uh we ensure all companies have have
that sort of uh re-test thing before
it's released to the public for example
yeah
um about because you asked like what we
can we do in the short term at the
beginning of your question
so Canada has a law a bill that is going
to pass into law probably in the spring
that uh maybe the first one
um around the world on on uh Ai and it
has a nice feature
which hopefully other countries will
imitate which is that the law itself
is fairly
you know uh
simple it it states a number of
principles
um
and then it leaves the details of what
exactly needs to be enforced to
regulation
and the reason this is good is because
it's much easier for governments to
change regulation regulation could be
changed like this
uh you don't need to go back to the
parliament
and so you could have much more adaptive
legislative System including the law and
the regulation and that's going to be
super important because
the the the the nefarious uses that we
didn't think about like they're going to
come up and we need to wrap quickly if
we have to go back to Parliament it's
going to take two years no this is not
going to work right we need to have a
system that's very adaptive in terms of
legislation
yeah that that is inevitable uh that
brings me back to we're in this
situation because I think people are
surprised at how rapidly AI is advancing
what how did we get caught off guard
like someone like you has been in this
for so long you knew the rate of change
um what happened is is it just we we
just could not anticipate as we scaled
the data up how fast the machine would
learn or is there what what is the X we
were surprised that the machine did X
quickly what was X ask acid training
tests in other words manipulate language
well enough I can fool us
uh the experience I had of so sorry what
what I'm asking is what allowed it to do
that in a way that caught us off guard
well that's interesting right it didn't
require any new science it it's
essentially scale that did it
do you think Consciousness is a function
of scale no right no
I don't think so uh I mean some people
think so but there are theories around
that uh I
think scale is probably useful but that
there are some very specific
qualitative features of how we become
conscious that would work even at
smaller scales
um
so yeah scale is important simply
because
the job that we're asking these
computers to do when they answer
questions
is computationally
very demanding
and this comes from so I have these I
have a blog post where I talk about the
large language models and some of their
limitations
um the issue here is that if you take
almost any problem in computer science
that you can write down formally like
try to optimize this or that or to find
the answer to this and that question
almost all of these questions
the optimal solution is intractable
meaning it would take an exponential
amount of computation compared with how
big the question is
and so the it's like if you want the
optimal neural net that can answer your
questions about that they can reason
properly and so on is exponentially big
which means it's we can't have it but
the bigger our neural net the better it
approximates this
so there's a sense in which bigger is
better because of that even with
problems that look simple so as an
example to illustrate what I mean
consider the problem of playing the game
of goat
the rules of the game are fairly simple
you can write a few lines of code that
check the rules and tell you how many
points you get and so on
the neural net that can play goal and
like really
win like in other words go by the rules
and exploit them in order to figure out
how you know what is the optimal move
and so on that neural net
the neural Nets we have now that play
really better than humans they are huge
also okay and
um it's just a property of many computer
science problems that are like that like
the the knowledge needed to describe the
problem maybe even when the knowledge is
small the size of the machine that's
necessary to
answer questions take decisions that are
optimal is very big
so I think that's the reason why we need
big neural Nets that's why we have a big
brain even if the amount of knowledge
that's involved is small now in addition
the amount of knowledge that's necessary
to understand the world around us is
also big
so so but but I I think the biggest part
of what our brain does is inference is
this is the technical term to mean given
knowledge how do you answer questions
properly like optimize or take decisions
that are that are good given that
knowledge
okay is inference the ability to apply a
pattern that I saw in the past to a new
novel problem
that's yes that's part of inference
um In classical AI
uh things were very clear between
um knowledge and inference
so knowledge was people having typed a
bunch of rules and facts
and so the knowledge was not launched it
was handcrafted
and inference was well you have some
search procedure that looks how to
combine these pieces of knowledge these
facts and rules in order to answer your
question and we know that's NP hard
that's like exponentially hard and so we
use approximations it's never perfect
and so on but people didn't use neural
Nets in those days they use like
classical computer science algorithms
that try to approximate this like a star
now we have neural Nets
and neural Nets can
do this approximate difference it can be
trained to do a really good job at
searching for
good answers to questions given that
piece of knowledge how does it Define
good is I always assume that what AI was
doing was trying to guess effectively
the next letter or the next word So
based on all the patterns that it had
seen so it's like I've seen questions
like this before and here are the
answers that have been rewarded in that
a human has told me that it likes this
answer better than this answer and that
the the pattern recognition of the
machine combined with the human ranking
those responses from the machine gives
us the way that the AI approaches that
question to this answer
am I missing something
yeah I think I mean what you're saying
makes sense but
there's also a lot of knowledge we have
that can be distilled for example
through
How We Do It For Education
uh we do it through books encyclopedia
so it's it's not old not old knowledge
we have but but you can see that so let
me try to put it in this way Wikipedia
is way smaller than your brain
smaller than my brain yeah smaller is a
number of bits that are needed to encode
it whereas the number of bits that are
needed to encode all the synaptic
weights in your brain got it yep yep
huge orders of magnitudes Greater
um so
if we were just talking about these
kinds of knowledge which is not
everything obviously like
physical intuitions and so on is another
kind that we can't put in Wikipedia But
if we just talk about that kind of
knowledge
uh
you would want a very big brain just the
people to answer questions that are
consistent without knowledge
that's that's that's what I meant
okay right now that's not the way we
train uh
uh our large language models by the way
the way we trade them is we look at
texts that presumably is more or less
consistent without because that's not
even the case there is like people are
not truthful and they say all kinds of
things but even if it were and then by
imitating that text like predicting the
next word and so on
uh we implicitly encapsulate
the underlying knowledge which let's say
is Wikipedia
um
but uh yeah uh so so again the argument
is
scale
is important because many problems
require doing
computation that is intractable if you
want to really get the right answer
and so we need these really large neural
Nets to do a good job of approximating
how to compute the answer
okay so now I'm gonna have to get into
the nitty-gritty a little bit this will
be really 101 for you but might be
certainly will be instructive for me and
hopefully many others
to say that a neural network is large
what do we mean are we just daisy
chaining gpus CPUs
um are they so when I think about the
brain the brain is is broken into these
hyper specialized regions so for
instance vision is comprised of this
part of vision tracks motion and I can
selectively damage the motion Center of
your brain and now you see everything in
a snapshot uh there's uh things to deal
with corners and so you can selectively
damage the part of your brain that that
detects Corners there sayings it detects
straight lines curved lines it's it's
all these like
hyper-specific little bits and pieces
and
I don't my understanding of a neural
network is it isn't that hyper
specialized it's a lot of the same thing
over and over and over and over and over
and over and over
um
help me understand what it means to be a
large neural network
okay so you write that the brain
seems to have very specialized and
modular structure
as in different parts of Cortex
especially uh when when we look at what
neurons do in different parts we see
that they're they're rather specialized
it's it's not
perfectly easy to like identify what
this neuron does but but we we get a
sense of what it's about
and it's also true of our large neural
Nets but to a lesser extent so people
have been
trying to
uh give a name to what each particular
unit in a large neural net is doing
and we can do that by checking when does
it turn on what kind of input was
present
so if we look a lot of the things that
make this particular Unit on
and we ask humans so you know what
what's the what's the category that this
belongs to then we're
often able to
um
to give a name and at least that has
been done a lot for
um image processing neural Nets because
that's easy sometimes you could say well
it's this part of the image and this
kind of object
for text I know there's some papers
doing that
um
now I do think that our brain is is more
modular you know more with more
specialization than what we're currently
uh see by the way cortex is
a uniform architecture
like the the part of your brain that is
cortex which is thought to be the part
that's more modern in evolution and
really uh essential for like Advanced
connect abilities
um
is all the same texture it's all the
same kind of units repeated all over the
place and depending on your experience
or the kinds of uh brain accidents that
you may have a different part of Cortex
will latch on a different job so uh
these are more or less replaceable
pieces of Hardware like like our neural
Nets
um there are other pieces in the brain
that are not cortex that seem to be much
more specialized like hippocampus and
and
hypothalamus and so on I I'm at the
edges now that was certainly useful
information but I want to push a little
bit farther so
when I'm what I'm trying to wrap my head
around is I have a vague understanding
of how the brain works very specialized
I do not understand how we scale a
neural network unless you're saying that
each okay let me uh I was going to say
each node and then I realized to me a
node is either a GPU or a CPU but I
actually don't know if that's true uh so
first is I would need to understand what
is a node inside of a neural net and
then how are the different parts of the
neural net program to do a specialized
thing
we'll start there okay okay all right
um I'm going to start with the end
they're not programmed to do a
specialized thing that emerges through
learning whoa whoa whoa
that's true of the brain and that's true
of neural Nets you don't tell this part
of the neural net you'd be responsible
for vision and this part you'll be
responsible for language but that
happens
yes you get specialization that happens
whoa because they collaborate to solve
the problem they're different pieces
as how learning this like even like a a
simple neural net from 1990 does that
how complex is that underlying code is
that really basic but somehow has these
incredibly complex emergent Properties
or is that incredibly sophisticated of
course whoa very simple
uh what the complexity emerges because
you you have all of these degrees of
freedom and you have a powerful way to
train each of the these degrees of
freedom these synaptic weights so that
collectively they optimize what you want
which is like predicting the piece of
text that comes next properly
um but let me go back to the hardware
question
the hardware we use currently to train
our artificial neural Nets is very
different from the brain they're very
very very different
um we don't know how to build Hardware
that would be as efficient as the brain
in terms of energy
and all uh compute that we can squeeze
into a few Watts right and we wish we
would so lots of people are trying to
figure out how to build circus that
would be as efficient computationally as
the brain
um
another difference is that
the brain has highly decentralized like
at the level of neurons and we got like
80 billions of them decentralized memory
and computation
the traditional uh
CPU
has
memory completely separated from compute
and you have bus that transfers
information from one to the other to do
the computation in the little uh little
CPU
that's very different from how the brain
is organized where every neuron has a
bit of memory and a bit of compute
now people doing Hardware have been
working to build chips that would have
something that's more decentralized and
more like the brain and there are
several companies doing this sort of
things
um they haven't yet
you know reached a point where it can be
a GPU so a GPU is a kind of hybrid thing
where
it's really the same CPU pattern but
instead of having one CPU you've got
5 000.
and they each have their little memory
but there's also some shared memory and
it was designed initially for graphics
I'm going to Graphics but it turned out
that
or many of the kinds of neural Nets that
we we wanted to do it was a pretty good
computational architecture but it has
its own limitation it's it's
energy wise it's like a huge waste
compared to the brain as I said earlier
and a large part of that waste is
because you have all that traffic still
between memory you know places that
contain memory and and places that do
compute
so it's much more parallel than the good
old CPU
but much less parallel than the brain
hmm
you're so deep in this it probably
doesn't freak you out as much as it
freaks me out but this is uh like as I
really start to try to wrap my head
around what is happening this feels
deeply mysterious now I've heard
um people say that one of the things is
freaking them out and this is people
deep deep in AI one of the things that
they find unnerving is that they don't
understand what the neural network is
doing they don't understand how it came
up with a given answer
is
how is that possible
it's it's just a fundamental property of
systems that learn
um and that learn not
like a set of uh simple recipes like you
would learn how to do a a recipe in your
kitchen but learn
something very complicated
that cannot be reduced to a few formulas
uh like how to walk or how to speak or
how to translate or how you go from
speech Acoustics to sequence of words
these tasks
cannot be easily
uh
done by traditional programming
but if you put a machine that has that
can like approximate any function to
some degree of precision so big a big
neural net
and you tweak each of the parameters of
that machine
billions of times
it can learn to do what you want it can
change its but then
you don't really understand how it does
it you understand
why it you know uh you know you
understand the code that specifies how
this machine computes but the actual
computation it does depends on what it
has learned which is based on less and
lots of experience
so maybe a good analogy is like our own
intuition these machines are like
intuition machines so what I mean is
this you know
how to act in different contexts like
for example how to climb stairs
but you can't explain it to a machine
you can't write a program people have
tried robot assists have tried you can't
write a program that does that
one reason is
it's you know it's all happening in the
unconscious right but but there's a more
friend the reason it's all happening in
their countries it's just too big it's a
very very complicated program that's
running in your brain
and the only way that you can acquire
that skill that's reasonable is by trial
and error and practice and you know
maybe some of evolutionary you know uh
pressure that
initializes your weights close to
something that's needed to to learn to
walk
um
so things that we do intuitively that
need a lot of practice
are exactly like what those machines are
learning they they you
they can't explain it we can't explain
our own intuition
uh we just know this is how we should do
it
um and it's knowledge that's so complex
that we can't put it in for We cannot
put it in a few formulas or a few
sentences it's just
that's that's a major of things that
that there are very complicated things
that can't be easily put into
verbalizable form but they can still be
discovered acquired through learning
through practice through repetition of
doing the exercise again and again
I have a grandson who's been learning to
walk in the last few months
you know he was stumbling a lot and and
going again and again and again and
after a few months now he's pretty good
he's not like us yet
but it's months and months of practice
and
getting better gradually
through lots and lots of practice that's
how we train those neural Nets and
that's why we can't explain why they
give this particular answer they're just
like well I know this is the answer but
I can't explain to you because it's too
complicated I have like
500 billion weights that really are the
explanation do you want those 500
billion whites what are you going to do
with that
okay let's start teasing this apart so
one of the more interesting things in
what you just said is going to highlight
the difference between what humans do
and what machines do and why
um until there is a breakthrough and I
always love saying this stuff in front
of experts so you can strike me down if
you think I'm crazy but I think one of
the reasons that a breakthrough is going
to be required and that we're not just
going to be able to scale our way to
artificial general intelligence and I've
completely heard you that AI passing a
Turing test opens up a Pandora's box
that is utterly terrifying in terms of
its ability to disregulate
the human's ability to function well as
a hive
heard but now
the reason I think there's going to need
to be a breakthrough is that the reason
that your grandson is able to get better
over time
isn't just the calculus of balance it's
that by doing it he's building
stabilizing muscles and so his muscles
are getting stronger in areas that they
didn't need to be strong in when he was
crawling so you get this biological
feedback loop of oh I see what I'm going
to have to do part of the repetition
isn't just locking it into my brain part
of the repetition is that I'm going to
need to develop the muscle fibers and
the strength now how much of that is
mediated by the brain in a part of the
brain that's subconscious is a huge
question and certainly gets to the
complexity in your 50 billion parameters
and all that the other part is that his
brain is reconfiguring neuronal
connections and it's making some of
those connections more efficient through
a process called myelination so it's
wrapping the fatty tissue to sheath
different connections just like an
electrician would do and now it's it's
got this incredible biological feedback
loop of I have a desire I'm goal
oriented I want to do this thing this
thing is walk now
how the interplay of I want to walk
because I see my parents walk I see
Grandpa walking I want to do that thing
or I have something in me tells me being
over there is better than being here and
so I actually want a locomote to get
there and I would figure this out even
if I never saw anybody move which is
probably more likely given the baby
start crawling and they don't see people
crawl
they just have a desire to locomote
somewhere
again going back to my initial thing
about I think machines are going to need
to have desire they have a reason that
they want to cross the road if we want
to get to human level intelligence but
let's just let me not fractal too much
here so okay we have this biological
feedback loop
you're not going to get that with a
neural network no matter how much you
scale it up it doesn't have a biological
it doesn't have the ability to change
itself yet now maybe it will and maybe
it could architect a new chip or
something once it has the ability to
manipulate 3D printers or what have you
but for now it's stuck with a physical
configuration of chips unlike a human
which can morph from muscles to brain
matter it's stuck with a configuration
but and this feels like the very
interesting thing that we've gotten
right so far which is I have figured out
the pieces that I need so whether that's
gpus or the code or both but I figured
out the pieces that I need for that
configuration to learn in a very
emergent way so I set up the pieces and
then I give it
a thing I wanted to learn and a quote
unquote reward for doing so and then a
massive amount of emergent Behavior
comes out of that but it's always going
to be limited in a way that human
intelligence is not because of the
biological feedback loop okay now that
I've set that stage do you agree that
machines will need something that
imitates that biological feedback loop
meaning I need efficiency here that I
did not have a moment ago for me to
continue to get good at this thing
and that without that we're sort of
stuck at the the
highly potentially destructive ability
to manipulate language and and images
but that's it
so actually current neural Nets already
do what you say I mean they don't have
the biological framework but they they
do learn from practice and mistakes but
can they Recon re can they reconfigure
their architecture to get better at it
you don't need to change the chips they
just need to change the content of the
memory in those chips that contains that
says so why is the biological Loop
different
Y is different
um it's different because it you know it
it has been designed by Evolution
whereas we are designing these things
using our means and but but
fundamentally let me let me step back
here a little bit
to State something important as a kind
of
uh starting point
bodies
are
machines
they are biological machines cells are
machines there are biological machines
we don't fully understand them we know
it's full of feedback loops we know a
lot I mean we know a lot of biology but
we don't understand the full thing but
we know it's just matter interacting and
exchanging information
so yeah it's just a different kind of
machine now
the question some people think that uh
in particular when people were
discussing Consciousness because
Consciousness looks mysterious some
people think that well
it's got to be something that's based on
biology otherwise how could it ever like
be in machines well it's I I completely
with that
um
because it's just it it it's just
information processing
um now the kind of information
processing going on in our bodies and
our brains and so on uh may have some
particular attributes that we still
don't have in in our current machines
but the
the the specific Hardware just that
needs to have enough power so you know
one of the Great
uh
uh starting points of computer science
by people like Turing and Von Neumann in
in the early days of computing is the
realization
with for example the turing machine that
you can decouple the hardware
from the software that and the same
outward facing Behavior
can be achieved by just changing the
software parts so long as the hardware
is sufficiently complex and trains show
that you need very very simple Hardware
and then you can do any computation
that's like computer science 101
so
that would suggest that there is no
reason why we couldn't in the future
build machines that have the same
capabilities as we do now we are still
the current systems are missing a bunch
of things
um you talked you know we talked about
walking and why is it that we don't have
robots that can walk I mean they can
walk as well as humans have you seen
Boston Dynamics that sucks freakish it
can parkour they're not as good as
humans by you know a big gap
but yeah I've seen I've seen them
um but but I think
the issue is simply that we have tons
more data available to train language
models than we have for training robots
it's hard to create the training data
for a robot because it's in the physical
world you can't just replicate a million
robots and then but eventually people
will do it
uh or be able to do good enough job with
simulation there's a lot of work going
in that direction
but um
but yeah so
I I I kind of disagree with your
conclusions so go back to the the reason
that we don't have robots that can walk
is because it's just not it's not able
to to
use some sort of model to see enough
okay but there's you're saying the point
of that is there's nothing fundamentally
missing from the architecture that the
AI is running on it's just a modeling
problem
it yes the software part we're we're
still far up for example you know one of
the clues I mentioned earlier is that
the amount of training data that that a
large language model needs like you know
gptx
uh compared to what a human needs in
terms of amount of text to kind of
understand language
is is hugely different so that tells me
we're missing something important but I
don't think it's because we're missing
something in the low level Hardware of
biology
uh although I you know I'm a big fan of
listening to biology and and
understanding what brains are doing and
so on so they can serve as inspiration
but I don't think it's a hardware
problem now Hardware is important for
efficiency
so
current gpus are not efficient compared
to our brains and and but but it doesn't
mean that in in the next few years we
will not be able to to build uh
specialized Hardware that will be a
thousand times more efficient than
current ones
um and now there's a much bigger
incentive for companies to actually
invest in this because the these AI
systems are going to be more and more
everywhere and it's going to become much
more profitable to do these Investments
yeah man proliferation to AIS is crazy
uh before we derail on that though I
want to ask you so
we're comparing the way that machines
are evolving the way the AI is evolving
to human evolution
um
I've always thought of evolution as uh
to use Richard Dawkins quote the blind
watchmaker
it's not trying to make a watch
but the watch emerges out of
um up what we could probably refer to as
a few simple lines of code it's like uh
replication and the way that it
replicates plus uh a desire to survive
on a long enough time scale
there's not even a need for a desire to
survive it's simply the selection of
those who survive
yeah interesting that that's is that a
important distinction because I worry
well actually I don't worry this this
would then
um maybe what you're trying to get me to
understand about why machines don't need
a desire they just there needs to be a
selection criteria for the one that does
the thing better and that will be enough
to Boom to have the the exponential
um and that's the way we train those
systems so the way we train them is that
we if you want we throw away all the
configurations of parameters that don't
work and we focus more and more on ones
that do that's that's how training
proceeds it it changes things
in small steps just like Evolution does
except Evolution does it in parallel
with you know billions of uh individuals
uh uh kind of
searching the space of genetic
configurations that can be useful
whereas we're doing it the learning way
so we have like one individual big
neural net and we're like making one
small change at a time
um but it's both our processes of search
in a very high dimensional space of
computations
okay so let me this was something that I
heard you say in an interview at one
point I wasn't sure if I was going to
ask it but it's now as you were saying
that I realize that the entire universe
is born of a simple set of physical
laws for lack of a better word
and everything that we see from because
I was trying to think what is the origin
of evolution because you said that it
you you don't need it to desire it just
needs to get selected and then I was
like well what's selecting it the laws
of physics just dictate that certain
things will continue to hold their form
and function and others will
disintegrate uh okay so then everything
is born out of these laws of physics
which we don't fully understand yet but
do you think there will be similar laws
of intelligence that we realize oh here
are the very simple subset and all of
the struggle that we have right now is
because much like we don't yet fully
understand the laws of physics but yet
we can still build a nuclear bomb
nuclear power GPS all of that we know
enough to do amazing things but we don't
know everything
do you think we have the same thing
happening in intelligence
that's what drove me into the field
that hope that there may be some
principles that we can understand as
humans verbally like write about them
explain them to each other and so on
maybe write math that formalizes them
that are sufficient to explain our
intelligence now obviously for this to
work it has to be that it explains how
we learn because the content of what we
learned the knowledge that has been
acquired by Evolution and then by our
you know in our individual life
is too big to be put in a few uh you
know lines of math
um
so whether this is true or not obvio