Kind: captions
Language: en
[Music]
you said that 2001 Space Odyssey is one
of your favorite movies Hal 9000 decides
to get rid of the astronauts four people
haven't seen the movie spoiler alert
because he it she believes that the
astronauts they will interfere with the
mission do you see how is flawed in some
fundamental way or even evil or did he
do the right thing
neither there's no notion of evil in
that in that context other than the fact
that people died but it was an example
of what people call value misalignment
right you give an objective to a machine
and the machine strives to achieve this
objective and if you don't put any
constraints on this objective like don't
kill people and don't do things like
this
the Machine given the power will do
stupid things just to achieve this dis
objective or damaging things to achieve
its objective it's a little bit like we
are used to this in the context of human
society we we put in place laws to
prevent people from doing bad things
because continuously do we do those bad
things right so we have to shape their
cost function the objective function if
you want through laws to kind of correct
an education obviously to sort of
correct for for those so maybe just
pushing a little further on on that
point how you know there's a mission
there's a this fuzziness around the
ambiguity around what the actual mission
is but you know do you think that there
will be a time from a utilitarian
perspective or an AI system where it is
not misalignment where it is alignment
for the greater good of society that
kneei system will make decisions that
are difficult well that's the trick I
mean eventually we will have to figure
out how to do this and again we're not
starting from scratch because we've been
doing this with humans for four
millennia
so designing objective functions for
people is something that we know how to
do and we don't do it by you know
programming things although the legal
code is called code so that tells you
something and it's actually the design
of an objective function that's really
what legal code is right it tells you
you can do here is what you can't do if
you do it you pay that much that's
that's an objective function so there is
this idea somehow that it's a new thing
for people to try to design objective
functions that are aligned with the
common good but no we've been writing
laws for millennia and that's exactly
what it is so this that's where you know
the science of lawmaking and and
computer science will come together will
come together so it's nothing there's
nothing special about how or AI systems
it's just the continuation of tools used
to make some of these difficult ethical
judgments that laws make yeah and we and
we have systems like this already that
you know
make many decisions for for ourselves in
society that you know need to be
designed in a way that they like you
know rules about things that sometimes
sometimes have bad side effects and we
have to be flexible enough about those
rules so that they can be broken when
it's obvious that they shouldn't be
applied so you don't see this on the
camera here but all the decorations in
this room is all pictures from 2001 a
Space Odyssey Wow
then by accident is there a lot about
accident it's by design Wow
so if you were if you were to build hell
10,000 so an improvement of Hal 9000
what would you improve well first of all
I wouldn't ask you to hold secrets and
tell lies
because that's really what breaks it in
the end that's the the fact that it's
asking itself questions about the
purpose of the mission and it's you know
pieces things together that it's heard
you know all the secrecy of the
preparation of the mission and the fact
that it was discovery and on the lunar
surface that really was kept secret and
and one part of Hal's memory knows this
and the other part is does not know it
and it's supposed to not tell anyone and
that creates a internal conflict do you
think there's never should be a set of
things that night AI system should not
be allowed like a set of facts that
should not be shared with the human
operators well I think no I think the I
think it should be a bit like in the
design of autonomous AI systems there
should be the equivalent of you know the
the the oath that hypocrite look at the
common assault
yeah that doctors sign up to write so
the certain thing certain rules said
that that you have to abide by and we
can sort of hardwire this into into our
into our machines to kind of make sure
they don't go so I'm not you know
advocate of the the three tow three
dollars of Robotics you know the Asimov
kind of thing because I don't think it's
practical but but you know some some
level of of limits but but to be clear
this is not these are not questions that
are
ree worth asking today because we just
don't have the technology to do this we
don't we don't have a ton of internal
machines we have intelligent machines so
my intelligent machines that are very
specialized but they don't they don't
really sort of satisfy an objective
they're just you know kind of trained to
do one thing so until we have some idea
for design of a full-fledged autonomous
intelligent system asking the question
of how we design its objective I think
is a little a little too abstract it's a
little tough rat there's useful elements
to it in that it helps us understand our
own ethical codes humans so even just as
a thought experiment if you imagine that
in a GI system is here today how would
we program it is a kind of nice thought
experiment of constructing how should we
have a law have a system of laws far as
humans
it's just a nice practical tool and I
think there's echoes of that idea too in
the AI systems left today they don't
have to be that intelligent yeah like
autonomous vehicles these things start
creeping in that were thinking about but
certainly they shouldn't be framed as as
hell yeah
you