Kind: captions
Language: en
[Music]
you've written advice saying don't get
fooled by people who claim to have a
solution to artificial general
intelligence who claim to have an AI
system that worked just like the human
brain or who claimed to have figured out
how the brain works ask them what the
error rate they get on em 'no store
imagenet you know this is a little dated
by the way that mean five years who's
counting okay but i think your opinion
is to understand imagenet yes maybe data
there may be new benchmarks right but i
think that philosophy is one you still
and and somewhat hold that benchmarks
and the practical testing the practical
application is where you really get to
test the ideas well it may not be
completely practical like for example
you know it could be a toy data set but
it has to be some sort of task that the
community as a whole as accepted as some
sort of standard you know kind of
benchmark if you want it doesn't need to
be real so for example many years ago
here at fair people you know justin west
on board and a few others proposed the
the babbitt asks which were kind of a
toy problem to test the ability of
machines to reason actually to access
working memory and things like this and
it was very useful even though it wasn't
a real task at least is kind of halfway
a real task so you know toy problems can
be very useful it's just that I was
really struck by the fact that a lot of
people particularly our people with
money to invest would be fooled by
people telling them oh we have you know
the algorithm of the cortex and you
should give us 50 million yes absolutely
so there's a lot of people who who try
to take advantage of the hype for
business reasons and so on but let me
sort of talk to this idea
that new ideas the ideas that push the
field forward
may not yet have a benchmark or it may
be very difficult to establish a
benchmark I agree that's part of the
process it's definition benchmarks as
part of the process so what are your
thoughts about so we have these
benchmarks on around stuff we can do
with images from classification to
captioning to just every kind of
information can pull off from images in
the surface level there's audio datasets
there's some video what can we start
natural language what kind of stuff
what kind of benchmarks do you see they
start creeping on to more something like
intelligence like reasoning like maybe
you don't like the term but AGI echoes
of that kind of yes formulation a lot of
people are working on interactive
environments in which you can you can
train and test intelligent systems so so
there for example you know it's the
classical paradigm of supervised
learning is that you you have a data set
you partition it into a training set
validation set test set and there's a
clear protocol right but what if the
that assumes that the samples are
statistically independent you can
exchange them the order in which you see
them doesn't shouldn't matter you know
things like that
but what if the answer you give
determines the next sample you see which
is the case for example in robotics
right you robot does something and then
it gets exposed to a new room and
depending on where it goes the room
would be different so that's the
decrease the exploration problem the
what if the samples so that creates also
a dependency between samples right you
you if you move if you can only move it
in in space the next sample you're going
to see is going to be probably in the
same building most likely so so so the
all the assumptions about the validity
of this training set test set hypothesis
break whatever a machine can take an
action that has an influence in the in
the world and it's what is going to see
so people are setting up artificial
environments where what that takes place
right the robot runs around
3d model of a house and can interact
with objects and things like this are
you do robotic space simulation you have
those you know opening a gym type thing
or mu Joko kind of simulated robots and
you have games you know things like that
so that that's where the field is going
really this kind of environment now back
to the question of a GI like I don't
like the term a GI because it implies
that human intelligence is general and
human intelligence is nothing like
general it's very very specialized we
think it's general we like to think of
ourselves as having general surgeons we
don't we're very specialized we're only
slightly more general than why does it
feel general so you kind of the term
general I think what's impressive about
humans is ability to learn as we were
talking about learning to learn in just
so many different domains it's perhaps
not arbitrarily general but just you can
learn in many domains and integrate that
knowledge somehow ok the knowledge
persists so let me take a very specific
example yes it's not an example it's
more like a a quasi mathematical
demonstration so you have about 1
million fibers coming out of one of your
eyes ok 2 million total but let's let's
talk about just one of them it's 1
million nerve fibers your optical nerve
let's imagine that they are binary so
they can be active or inactive right so
the input to your visual cortex is 1
million bits
now they connected to your brain in a
particular way on your brain has
connections that are kind of a little
bit like accomplish on that they're kind
of local you know in space and things
like this I imagine I play a trick on
you it's a pretty nasty trick I admit I
I cut your optical nerve and I put a
device that makes a random perturbation
of a permutation of all the nerve fibers
so now what comes to your to your brain
is a fixed but random permutation of all
the pixels there's no way in hell that
your visual cortex even if I do this to
you in infancy will actually learn
vision to the same level of quality that
you can got it and you're saying there's
no way you ever learn that no because
now two pixels that on your body in the
world will end up in very different
places in your visual cortex and your
neurons there have no connections with
each other because they only connect it
locally so this whole our entire the
hardware is built in many ways to
support the locality of the real world
yeah yes that's specialization yep it's
still now really damn impressive so it's
not perfect generalizations not even
close no no it's it's it's it's it's not
that it's not even close it's not at all
yes it's all sighs so how many boolean
functions so let's imagine you want to
train your visual system to you know
recognize particular patterns of those 1
million bits ok so that's a boolean
function right either the pattern is
here or not here is the to to a
classification with 1 million binary
inputs
how many such boolean functions are
there okay if you have 2 to the 1
million combinations of inputs for each
of those you have an output bit and so
you have 2 to the 2 to the 1 million
boolean functions of this type okay
which is an unimaginably large number
how many of those functions can actually
be computed by your visual cortex and
the answer is a tiny tiny tiny tiny tiny
tiny sliver like an enormous be tiny
sliver yeah yeah so we are ridiculously
specialized you know okay but okay
that's an argument against the word
general I think there's there's a I
there's I agree with your intuition but
I'm not sure it's it seems the breath
the the brain is impressively capable of
adjusting to things so it's because we
can't imagine tasks that are outside of
our comprehension right we think we
think we're general because we're
general of all the things that we can
apprehend so yeah but there is a huge
world out there of things that we have
no idea
we call that heat by the way heat heat
so at least physicists call that heat or
they call it entropy which is okay you
have a thing full of gas right call
system for gas right clothes on our
coast it has you know pressure it has
temperature has you know and you can
write the equations PV equal NRT you
know things like that right when you
reduce a volume the temperature goes up
the pressure goes up you know things
like that right for perfect gas at least
those are the things you can know about
that system and it's a tiny tiny number
of bits compared to the complete
information of the state of the entire
system because the state minute our
system will give you the position of
momentum of every every molecule of the
gas
and what you don't know about it is the
entropy and you interpret it as heat the
energy contained in that thing is is
what we call heat now it's very possible
that in fact there is some very strong
structure in how those molecules are
moving is just that they are in a way
that we are just not wired to perceive a
Waggoner to it and there's in your
infinite amount of things we're not
wired to perceive any right that's a
nice way to put it well general to all
the things we can imagine which is a
very tiny a subset of all things that
are possible it was like Colonel Goffe
complexity or the komova charge in
someone of complexity you know every bit
string or every integer is random except
for all the ones that you can actually
write down yeah ok so beautifully put
but you know so we can just call it
artificial intelligence we don't you to
have a general whatever novel human of
all Nutella transmissible oh you know
you'll start anytime you touch human it
gets it gets interesting because you
know is this because we attach ourselves
to human and it's difficult to define
with human intelligences yeah
nevertheless my definition is maybe damn
impressive intelligence ok damn
impressive demonstration of intelligence
whatever
you