Kind: captions
Language: en
what difference between biological
neural networks and artificial neural
networks is most captivating and
profound to you at the higher
philosophical level let's not get
technical just yet one of the things
very much intrigues me is the fact that
neurons have all kinds of components
properties to them and evolutionary
biology you have some little quirk and
how a molecule works or how a Silbert
and it can make me and use of evolution
will sharpen it up and make it into a
useful feature rather than a glitch and
so you expect in neurobiology for
evolution to have captured all kinds of
possibilities of getting neurons of how
you get neurons to do things for you and
that aspect has been completely
suppressed in artificial neural networks
so the glitches become features in them
in the biological neural network they
they can look let me take one of the
things that I used to do research on if
you take things which oscillate their
rhythms which are sort of close to each
other under some circumstances these
things will have a phase transition and
suddenly is the rhythm wolf everybody
will fall into step there was a
marvelous physical example of that in
the millenium bridge across the Thames
River about Bill about 2001 and
pedestrians walking across pedestrians
don't walk synchronized they don't walk
and lock lockstep but they're all
walking about the same frequency and the
bridge could say at that frequency in
the flights way made pedestrians tend a
little bit to lock in the step natural
well the bridge was oscillating back and
forth the pedestrians were walking in
step to it you could see
we waited at the bridge and the
engineers made a simple-minded a mistake
they had assumed when you walk it's step
step step and it's back at forth motion
but when you walk it's also right foot
left with side to side motion and the
side to side motion for which the bridge
was strong enough but it wasn't it
wasn't stiff enough and as a result you
would feel the motion and you'd fall
under stuff with it and people were very
uncomfortable with it they closed the
bridge for two years really fully built
stiffening for it no nerves so nerve
cells loose action potentials you have a
bunch of cells which are loosely coupled
together producing action potentials the
same rate there'll be some circumstances
under which these things can lock
together other circumstances which they
won't
well they fire together you can be sure
the other cells are going to notice it
so you can make a computational feature
out of this and you're in an evolving
brain most artificial neural networks
don't even have action potentials let
alone have the pathology for
synchronizing them and you mentioned the
evolutionary process so they're the
evolutionary process that builds on top
of biological systems leverage is that
the the weird mess of it somehow so how
do you make sense of that ability to
leverage all the different kinds of
complexities in the biological brain
well look in the bite of the biological
molecule level you have a piece of DNA
which included an encode for a
particular protein you could duplicate
that piece of DNA and now one part of it
encode for that protein but the other
one could itself change a little bit and
the start coding for a molecule was just
slightly different John was that
molecule was just slightly different had
an a function which helped any old
chemical reaction it was important to
the cell
you would go ahead and let that eat fry
and evolutional slowly and improve that
function and so you have the possibility
of duplicating and then having things
drift apart
one of them retaining the old function
the other one do something new for you
and there's evolutionary pressure to
improve look there isn't in computers to
adjust improvement has to do with
closing some companies openings or
others the evolutionary process looks a
little different yeah Oh similar
timescale perhaps well no horse shorter
in time skill company's closed yeah go
bankrupt and are born yeah shorter but
not much shorter some some company lasts
the century couples but yeah you're
right I mean if you think of companies a
single organism that builds and you all
know yeah it's a fascinating dual
correspondence there between biological
and companies have difficulty having a
new product competing with an old fraud
large yeah and when IBM built this first
PC you've probably read the dread the
book they made a little isolated
internal unit to make the PC and for the
first time in IBM's history they didn't
insist that you build it out of I
vehicle components but they understood
that they could get into this market
which is a very different thing by
completely changing their culture and
biology finds other markets in a more
adaptive way he adds better at it it's
better at that kind of integration so
maybe you've already said it but what to
use the most beautiful aspect or
mechanism of the human mind
is it the adaptive the ability to adapt
as you've described there's there some
other little quirk that you particularly
like adaptation is everything when you
get down to it but the difference there
are there differences between adaptation
where your learning goes on on the over
generation that over evolutionary time
as your learning goes on at the
timescale of one individual who must
learn from the environment during that
individuals lifetime and biology has
both kinds of learning in it and the
thing which makes neurobiology hard is
that a mathematical systems that were
built on this other kind of evolutionary
system what do you mean by mathematical
system where where's the math in the
biology well when you talk to a computer
scientist about neural networks it's all
math the fact that biology actually came
about from evolution the thing that and
the fact that biology is about a system
which you can build in three dimensions
if you look at computer chips computer
chips are basically two dimensional
structures they two point one dimensions
but they really have difficulty doing
three-dimensional wiring biology biology
is the neocortex is actually also
sheet-like and insists on top of the
white matter which is about ten times
the volume of the gray matter and
contains all what you might call the
wires but there's a huge the the effect
the effect of computer structure on what
is easy and what is hard is immense
so and biology does makes some things
easy that are very difficult to
understand how to do computationally on
the other hand you can't do simple
floating-point arithmetic or so it's
awfully stupid yeah and you're saying
this kind of three-dimensional
complicated structure makes it's still
math it still doing math the kind of
math is doing enables you to solve
problems of a very different kind that's
right that's right
so you mentioned two kinds of adaptation
the evolutionary adaptation at the end
the adaptation or learning at the scale
of a single human life which do you are
which is particularly beautiful to you
and interesting from a research and from
just a human perspective and which is
more powerful
I find things most interesting that I
begin to see how to get into the edges
edges of them and tease them apart a
little bit and see how they work and
since I can't see the evolutionary
process going on I am in awe of it but I
find it just a black hole as far as
trying to understand what to do and so
in a certain sense I'm in awe but I
couldn't be interested in working on it
the human life timescale is however
thing you can tease apart and study yeah
you can do it there's the vel mental
neurobiology which understands all of
these connections and now the structure
evolves from a combination of what the
genetics is like and the real the fact
that you're building a system in three
dimensions in just days and months those
early early days of a human life are
really interesting they are and of
course there are times of immense cell
multiplication there are also times of
the greatest cell death in the brain is
during infancy it's turnover so what is
what what what is not effective which is
not wired well enough to use the moment
throw it out it's a mysterious process
from let me ask from what field do you
think the biggest breakthroughs in
understanding the mind will come in the
next decades
is it neural science computer science
neurobiology psychology physics maybe
math maybe literature
well of course I see the world always
through a lens of physics I grew up in
physics
and the way I pick problems is very
characteristic of physics and of an
intellectual background which is not
psychology which is not chemistry and so
on and so on
at both the ear parents of physicists
both of our parents were physicists and
the real thing I got or that was a
feeling that the world is an
understandable place and if you do
enough experiments and think about what
they mean and structure things that you
can do the mathematics of the relevant
to the experiments you also be able to
understand how things work but that was
I was a few years ago did you change
your mind at all through many decades of
trying to understand the mind of
studying in different kinds of ways not
even the mises biological systems you
still have hope that physics that you
can understand there's a question of
what do you mean by understand of course
when I taught freshman physics I used to
say I wanted to get physics to
understand the subject to understand new
this laws I didn't want them simply to
memorize a set of examples to which they
knew the the equations are right down to
generate the answers I had this nebulous
idea of understanding so the if you
looked at a situation you could say oh I
expect the bowl don't make that
trajectory all right I expect so I'm
into a notion of understanding and I
don't know how to express that very well
I've never known how to express it well
and you run smack up against it for you
to choose these look at these simple
neural Nets feed-forward neural nets
which do amazing things and yet you know
contain nothing of the essence of what I
would have felt whose understanding
Kittanning is more than just an enormous
lookup table that's linger on that how
sure you are of that what if the table
it's really big so I'm he asks another
way these feed-forward neural networks
do you think they'll ever understand
good answer that in two ways I think if
you look at real systems feedback is an
essential aspect of how these real
systems compute on the other hand if I
have a mathematical system with feedback
I know I can unlaid this and do it he
can't prove it but but I have an
exponential expansion and the amount of
stuff I have to build if I could resolve
the problem that way so feedback is
essential so we can talk even about
recurrent you know I sort of occurrence
but do you think all the pieces are
there to achieve understanding through
these simple mechanisms like back to our
original question what is the
fundamental is there a fundamental
difference in artificial neural networks
and biological or is it just a bunch of
surface stuff suppose you ask a
neurosurgeon when does somebody did yeah
they'll probably go back to saying well
I can look at the brain rhythms and tell
you this is a brain which has never
could have functioned again this phone
isn't but this other one is one mister
if we treat it well is still recoverable
and then just do that probably so many
electrodes looking at simple like
electrical patterns just don't look in
any detail at all or what individual
neurons are doing these rhythms are
already absent from anything which goes
on in Google
yeah but the rhythms but the rhythms
would so well that's like comparing okay
I'll tell you it's like you're comparing
the the greatest classical musician in
the world to child first learning to
play the question I'm at but they're
still both playing the piano I'm asking
is there will it ever go on at Google do
you have a hope because you're one of
the seminal figures in both launching
both disciplines both sides of the of
the river I think it's going to go on
generation after generation the way it
has where what you might call the AI
computer science community says let's
take the following this is our model of
Neurobiology at the moment let's pretend
it's good enough and do everything we
can with it and it does interesting
things and after the while sort of
grinded at the sand to say Oh something
else is needed from neurobiology and
some other grand thing comes in and
enables you to go a lot further
according to the sandakan know
everything it could be generations of
this evolution I don't know how many of
them and each one is going to get you
further into what a brain does whatever
then in some sense passed the Turing
test longer and more broad aspects and
how many of these are good they are
going to have to be before you say I've
made something I've made a human I don't
know but your sense is it might be a
couple my senses might be a couple more
yeah and going back to my brainwaves of
the word yes
from the AI point of view of that they
would say ah maybe these are heavy
phenomena and not important at all the
first car I had will record a 1936 dodge
coupe of 45 miles an hour and the wheels
which Jimmy yeah good good speedometer
that
now don't be designed at the car that
way the cars malfunctioning to have that
but in biology if you if it were useful
to know when are you going more than 45
miles an hour you just capture that and
you wouldn't worry about where it came
from
yeah that'll be a long time before that
kind of thing which can take place in
large complex networks of things is
actually used in the computation look
the how many transistors are there at
your laptop these days actually I don't
know the number it's some a scale of 10
to the 10 I can't remember the W there
yeah and all the transistors are
somewhat similar and most physical
systems with that many parts all of
which are selfs or have collective
properties yes soundly is an error
Earthquakes what have you have
collective properties whether there are
no collective properties used in
artificial neural networks in AI yeah
it's very
if biology uses them it's gonna take us
two more generations of things to be the
perfect people to actually dig in and
see how they are used what they mean see
you're very right might have to return
several times in your biology and try to
make our transistors more messy yo-yo at
the same time the simple ones cool
concert will conquer big aspects
and I think one of the most biggest
surprises to me was how well learning
systems was there manifestly
non-biological how important they can be
actually and you how important it how
useful they can be in a high
you