Kind: captions
Language: en
all right welcome back to 6 s0 9 for
deep learning for self-driving cars
today we have Oliver Cameron is the
co-founder and the CEO of voyage before
that he was the lead of the Udacity
self-driving car program that made ideas
an autonomous vehicle research and
development accessible to the entire
world he has a passion for the topic and
a genuine open nature that makes him one
of my favorite people in general and one
of my favorite people working in this
space and I think thousands of people
agree with that so please give Oliver
warm welcome thank you very much Lex and
thank you all for having me here today
super excited to speak all about voyage
but in reality the the kind of thing I
want to share today is kind of like this
title says how to start a self-driving
car startup rarely do you kind of get an
inside scoop of how a startup is formed
you kind of hear all the the PR all the
kind of very lovey-dovey press releases
out there I want to share kind of the
inside of how at least voyage came to be
which was a little unconventional
compared to your average self-driving
car startup they always tell you that
the path to a startup getting to the
goal you want is kind of a zigzag oz was
kind of a insane zigzag as well we'll go
through all of that stuff let's talk
about my background also a little
unconventional I'm not very good at
learning in a classroom for me I found
learning by doing by building has always
been the thing that's that's worked best
for me so going all the way back to when
I was a teenager software just in
general was my passion this idea that
you can make something out of absolutely
nothing and then all of a sudden
millions and in Facebook's case billions
of people can be using that thing and
after building lots of crazy stuff and
perhaps not being too popular in high
school because that's all I did I
started a company offer I won't bore you
with all the details but learned a lot
during the experience and joined
went through Y Combinator J believe
started right here in Cambridge she's
very cool and then this very pivotal
moment happened to me I heard about this
online class which was generating a
whole bunch of scandal and and lots of
controversy and it was from this guy
called Sebastian Thrun he'd taken this
Stamper class he taught in artificial
intelligence and just said screw it
we're gonna put the whole thing online
and back then and this was a around 2011
this was a very controversial thing to
do today MIT and many others do this all
the time but back then there was a hell
of a lot of controversy around and doing
something like this but this learning
format really just appealed to me being
able to sit in my in front of my laptop
learn at my own pace build build build
was something that really resonated with
me and I took this class in 2013
artificial intelligence for robotics and
this again was just this pivotal moment
my head exploded all the enthusiasm I'd
had the software kind of transferred to
artificial intelligence and and robotics
and just became addicted to the format
of what are now called MOOCs massively
open online courses and I love them so
much I decided hey I want to go do this
and help others learn this stuff so hey
let's go join Udacity and and build more
classes like this so I did that for four
years let our machine learning robotics
and eventually our self driving car
curriculum which was a lot of fun and I
got to learn directly from two great
company builders like truly great
company builders one was Vishal Mackey
Jonny he was the operator extraordinary
Udacity understood how to build a
company how to build a culture how to
incentivize and how to do all those
those things that we don't often talk
about and Sebastian Thrun he of course
founded the Google self-driving car
project and it's early days and right
now I believe he's building flying cars
just in general I learned so much from
him but this idea that you are literally
in control of your destiny you can build
absolutely anything if you if you put
your mind
- it was was always pretty inspirational
today of course I Drive in cars at
voyage and we'll talk more about what
makes us special compared to the other
self-driving car companies you may have
heard of in this class and beyond let's
talk about Udacity you raise your hand
if you've heard of Udacity very curious
there you go that's most of the room
Udacity like I said was founded by
Sebastian Thrun he took this class
online and all just exploded and he
built a company around it you'd ask
these real focuses on increasing the
world's GDP this idea that talent is
everywhere that it isn't now just
constrained to the best schools in the
world that because of this proliferation
of content there are talented students
all over the world and all they need is
the content in which to be able to build
crazy cool world-changing things and
what I see is my job today is to go out
into the world and find these
ridiculously talented people and then
put them to work on the hardest problems
that exist and Udacity to me felt like
the perfect place to do this as a kind
of prelude to this about three years
into Udacity we had had this real focus
like I said on machine learning and
robotics but we really want to take it
to the next step and we came up with
this kind of concept internally that we
called only at Udacity what if we taught
the things that other places weren't
teaching what if people all around the
world could come lay learn from what may
appear to be niche topics but were just
being taught at the right time because
that industry is about to blow up and
the first one we did of this and we've
done some after including flying cars a
much more in-depth curriculum on
artificial intelligence with
self-driving cars so this is a quick
video that introduces it and this is of
course Sebastian's run robotics legend
see this place
if you can build the same apartment will
save you always but on top of its
transformations
just imagine instead of owning a car you
have a default in stock apartment and B
Hawkins link is phenomenal and it's eco
Samara how many don't mean in work it
disappears there is an enormous market
for surviving managed genius
lots and lots of companies that you
wouldn't suspect have entered that
feeling a nasty - I challenge everybody
did we pass the Largey
university in the middle of South Africa
so that everyone in the world can become
a self-taught engineer and why did we
want to do this was our goal it was to
accelerate the deployment of
self-driving cars like Sebastian says in
that video there's a number of reasons
why self-driving cars a transformational
and at the time this was around 2016 it
felt like self-driving cars were just
taking a little bit too long we rewind
to that particular spot in time Google
was the really the only main effort
going on and what we believed is that it
needed to happen faster and that one of
the reasons it wasn't happening fast
enough is because there wasn't enough
talent in the space so what we decided
to do is like I said build something
quite special in want to pair up a
world-class curriculum an actual
self-driving car which we'll talk about
more and what we called our open-source
challenges and all of that would come
together to build this this quite
special curriculum so let's start with
the curriculum one of our beliefs was
that partnering with industry was the
right way to go that was because it felt
and I believe this that the knowledge of
how to build a self-driving car was not
necessary trapped in academia
it was trapped in industry so we had to
go straight to industry work with
engineers that were already you know
challenging themselves with these
problems and get them on camera have
them teach the concepts that they know
and build day in day out
and have that be transplanted to
thousands of mines around the world so
these are just some of those partners
there was many many more but we had a
real focus on finding these engineers
wherever they may be and getting those
folks on on-camera we also built an
incredibly talented team this is just a
small snippet of the curriculum team but
of course Sebastian Thrun was was a big
part of this curriculum when I told
folks that I'd gotten the chance to work
with him on specifically self-driving
cars he likened it to getting basketball
lessons from Michael Jordan which I
thought was pretty fun
and they were probably just as
entertaining but some really truly great
folks working on this curriculum and
still doing that to this day who deserve
all of the credit frankly here's a quick
photo of first lecture recordings with
eventual voyage co-founders Eric and Mac
Eric who's on the left he hates this
picture and here's why there you go he
still isn't at max height but we had to
he still has that box on his desk and we
built a whole 12-month curriculum to
take an intermediate software engineer
who may be in consumer software or just
some other part of the software world
and take them into self-driving cars we
wanted to cover perception prediction
planning localization controls even just
the whole breadth of of the industry and
the reason we want to do that is because
we saw the best fit for Udacity student
not necessarily being a specialist in a
niche for example you know just
perception although there's been a whole
bunch of folks doing that as well but
that the skills of a Udacity student
tend to pair themselves well with being
a generalist someone that can contribute
all across the stack so we tried to give
these folks that breadth of knowledge
the curriculum that we built with some
you're also welcome project to detective
just like real economist people's have
to do in term two you'll learn about
sensor fusion localization in control
this is harmful robotics that every
self-driving car interior needs to know
in order to actually move the vehicle to
space
if the localization module you'll build
a kidnapped vehicle project which takes
a vehicle that's lost and figures out
where it is in the world with the help
of sensor readings in a minute
this is exactly what real self-driving
cars have to do every time they turn on
in order to be on where they are in the
world in the control module you'll build
a model predictive controller which is a
really advanced type of controller
that's actually combos no credit cards
move through the world and use the
steering wheel throttle and brake to
follow the set of waypoints or
trajectory to get from one point X in
turn three you'll learn about pathway
and electives month and you'll learn
about system integration path blending
is really the brains of a self-driving
car
it's how the perfect here's how kind of
it from one point to another as well as
obstacles intimacy I'm gonna give you a
sneak preview how it works and this is
something that nobody's ever seen before
so get ready patenting involves three
parts
there's prediction which is figuring out
what the other vehicles are going to do
around us this goes up for a while so
it'll posit that the impact of this
curriculum was was bigger than we
thought it would be when we pitched as a
small team this idea to Sebastian to
viche at Udacity there was a lot of
skepticism that something like this was
was going to be successful and the
reason that you know there was that
skepticism is that one of the kind of
formulas that Udacity looked at to
determine the impact of building a
certain type of content was the number
of open jobs available if there was you
know for example in web development
mobile development told that good stuff
there was millions of jobs open so it
felt like there was a massive
opportunity to impact the area but if
you were to in 2016 search for you know
self-driving car engineers or the
different disciplines that exist within
it was it was kind of just Google so it
was very interesting just to see the
instantaneous reaction that we had to
launching this curriculum today over
14,000 successful students from all
around the world as you can see where
the most exciting thing is to see what
students have done with this so a
curriculum for example I learned
recently that a set of our students here
are building a self-driving truck
stopped in India another set of students
in South Korea are building a perception
engine for self-driving cars just a
whole bunch of folks are building truly
amazing things and only that they've
gotten jobs at cruise zukes way mo agro
all the big names and actively impacting
those companies today now for the fun
stuff so we also decided to make a
curriculum extra special and we decided
to do that by building an actual
self-driving car
and whenever I talked about this
internally Udacity people asked me why
why do we need to do this right isn't
the curriculum just enough why go to the
you know the length of building an
actual self-driving car and selfishly
some of it was just a personal want to
you know build a self-driving car but
the the reasoning that I use is that
what better way to prove to these
students that putting their faith in us
that we know what we're doing than to
build our own self-driving car and also
what better way to collaborate with
these students on an area that is really
infantile then again by having this
platform that students could actually
run code on a car so we decide to buy a
car and we'll talk more about that in a
second but we set ourselves a milestone
for our self-driving car it was to drive
from Mountain View to San Francisco 32
miles of driving with zero dis
engagements it should be repeatable it
won't be zero disengagement so every
single time because otherwise we've got
an actual self-driving car but in a
short period of time how much progress
can we make towards this this stated
goal raise your hand if you've been on
El Camino Rail in in that sort of region
okay so you pray understand it's got a
lot of traffic lights in fact on our
route about 130 traffic lights it's
multi Lane three lanes speed limit of
about 4045 something like that so it's
you know fairly complex but it's also
got some constraining factors which is
what we're looking for - so focused our
tech efforts this is the car we bought
you're probably very familiar if you
follow self-driving cars with the
Lincoln MKZ of the world they're
everywhere and there's a reason for that
in terms of the drive-by-wire nature of
the vehicle and other stuff and we
outfitted a whole bunch of sensors some
cameras some lighters all that good
stuff we also try to build our own mount
we affectionately call this the
periscope I don't know why it's in slow
motion but this was not our final design
build all this from parts at Home Depot
Trulia MVP and then we got to work the
goal was to accomplish that milestone
within six months so we've course had to
work fast assembled a dream team of
folks that I worked with on different
projects at Udacity they also wanted to
- come dabble in this folks that worked
on the machine learning curriculum
robotics curriculum etc so this was one
of our first days testing and we did
this at the shoreline amphitheater
parking lot Jessie now is a very popular
place to test self-driving cars in the
Bay Area because Google used to do it in
the past we saw a lot of weird stuff for
example you'll see here we saw what I
believe to be a motorcycle gang and we
made progress we kept iterating kept
building and it started to come together
in fact some stuff that we thought
wouldn't work surprisingly just start to
work this is on El Camino Real
I'm in the backseat here so Mack
discovered that we shouldn't have
stopped at that traffic light but we did
we resolve the mystery later let's go to
the next video and of course we learned
a lot by going in this route the
different behaviors of drivers one of
the things that we were worried about is
vehicles cutting us off and when we say
cutting us off it means a vehicle
pulling out in front of us even a few
hundred feet in front you'll see here
we drove a little slow 25
[Music]
said that was fine and pretty soon it
got quite boring car was doing very well
driving itself we built some cool
algorithms to change lanes when
necessary similar to you see with Tesla
autopilot these days
we collaborate with some students on a
traffic light classifier which was
integrated into Ross there and yep
pretty boring stuff so you can tell Eric
was surprised that it was just fine and
we also had a penchant for building of
recording themed videos like you saw
maybe from Elon Musk and the Tesla team
with painted black we've got our own
version of that eventually we became
pretty confident but we always you know
wanted to test most of the day just to
get the most learnings out of everything
this video was made at 2:30 a.m. driving
from Mountain View to San Francisco all
32 miles cost as a backing track
[Music]
maybe want to tow it down so it's easier
because there's less traffic right this
is kind of cheating and didn't count as
the milestone just to be clear you'll
see that we eventually hit the 32 miles
and machers in the driver seat is pretty
excited about that
[Music]
and they hit it but of course that
didn't count because it's in the middle
of the night and that's not gonna be a
very useful route but it was awesome
accomplishment just to even make it 32
miles with no dis engagements when this
traffic lights lane changes all that
good stuff but after four months this is
in the daytime
this began I believe it like six sorry 7
a.m. we accomplished it that small team
had come together and build something
pretty cool they could handle again
multi Lane roadways varying speed limits
traffic lights objects all that good
stuff and the thing that really brought
this home to me
is that the industry was now ready right
it felt like this feeling I had in
software where someone in their bedroom
can go and build something and launch it
you know almost feeling overnight could
now not quite the same but close to the
same happen in self-driving cars but
well we'll talk more about what this led
to in a little bit let's talk about open
source challenges we also got the same
question why do this and it was clear to
me that for something like self-driving
cars which was so you know formative we
had to collaborate with students to
figure out the best stuff because you
know even the folks that were Udacity
were not necessarily the world's leading
experts in these topics who want to use
this hivemind of activity from around
the world to teach the best stuff so
just through a period of a year these
are all the different challenges we
launched there was prizes and leaders
leaderboards and all this sort of fun
stuff the one that I'll focus most on
today is using deep learning to predict
steering angles and the challenge was
clear it was that given a single camera
frame you have to predict the
appropriate steering angle of the
vehicle if anyone had read in videos and
and papers in 2016 this stuff was all
the rage
and it felt like one of those areas that
was just begging for more exploration
again let's use this all these students
from around the world to do it and we
did have students from all around the
world there was over a hundred teams
people self organized into these little
groups to go and build this and over the
course of about four months we had a
whole bunch of submissions all taking
incredibly different approaches to the
problem we released a two sets
validation sets all that good stuff here
you'll see are the winning model and I
later found out that the author of this
model actually went on to lead the
self-driving car team at Yandex which if
you've been following CSS is doing some
pretty cool stuff and self-driving cars
today but you'll see this is on a route
from the Bay Area to Half Moon Bay a
very windy road and you'll see that the
prediction matches pretty closely to the
actual which is nice and if you read his
description of his solution it's a
pretty cool solution and it was I think
the most exciting thing was just the
number of different approaches to the
problem all resulting in some some
awesome stuff and again in true voyage
fashion we recorded a video of what this
model perform like on our car
it wasn't perfect as any first model and
just that the general approach of camera
only you know driving had it's had its
faults one of the main ones that you
know we realized after trying all this
stuff out is that of course a car when
steered by such an input performs
differently in a car than it does you
know on your desk in a you know
simulator through pre-recorded camera
frames so adjusting for those
Corrections that might need to be made
is something that students after the
fact added which was pretty cool so
after all of these things building that
curriculum building a self-driving car
launching these challenges it felt like
it was time for something new
it was awesome to go and collaborate
with all these students and it felt like
you know I had to go build something so
gathered that same team that had built
this curriculum and we said we're gonna
go build a self-driving car this is from
my pitch at Khosla Ventures you can kind
of see the pitch deck they're a little
bit voyage is a new kind of taxi service
our pitch has changed somewhat through
time but that's still pretty accurate
and we started what is now called voyage
and our goal really was that we wanted
to again build a self-driving car but we
wanted to do it differently we didn't
want to follow the same formula that we
felt we'd seen from some of the other
folks in the field and the reason is
that those folks have real advantages
right when you think of it about
Google's project of which I'm a big fan
they have this massive engineering
pipeline of folks that want to go build
a self-driving car at today way mo but
they also have a cash bank balance of
billions of dollars that is hard to
match they also have the brand
recognition of getting to work with
Google and all that good stuff so we
just knew we had to think about this
problem quite differently and what
motivated me is that today as we all
know we have this incredibly broken
transportation system you step outside
onto the roads today and I don't know
about you guys but I don't feel
particularly safe when I jump into my
car over we all know the stats over 1
million people have
sefa fatalities on the roads today
doesn't include folks that break necks
that inja break bones all that horrific
stuff it's also incredibly inefficient
we've again all observe this as we go
about our day just the number of lanes
that exist on a road today to account
for peak traffic the number of vehicles
which have enough room for eight people
have usually one person in that front
seat I read a stat recently that only 7%
of the average vehicles energy usage is
going towards moving the things that are
actually in the car
the rest is waste so an incredibly
inefficient system it's also expensive
the reason you know we see a lot of old
cars on the road today is because that's
at least today the most optimal and
affordable way to lots of folks to get
around and inaccessible and you'll see
why this matters to us in particular our
goal is to introduce a new way to
explore our communities this is a video
of one of our cars at a particularly
cool place which we'll talk more about
and this is kind of our mission and why
now why is it possible to build a
self-driving car now a number of factors
that we learned during that you'd ask
the experience but some new as well it
feels from everything we see that
sensors are now in this position which
these sensors are now capable of level
for self-driving cars the resolution the
range the reliability all those things
that were necessary for an elf or sub
revving car are today ready that didn't
used to be the case if you rewind you
know to 2007 and look at the cars that
were participating in the DARPA
challenges you'll see a lot of single
channel lasers you'll see the relic of
the valid Iron Age TL 64 the the
spinning bucket as it's called today and
no one would have claimed those sensors
already but today you've got this
enormous breadth of sensors that can
take you that way
compute is there when we you know think
about the recent rise in in GPUs and
whatnot finally you know being able to
have enough performance in the back of a
car with the power constraints that you
have it's it's there and talent
you know again this is not just Google
today you've got all of these great
minds from all around the world building
this technology so you're able to
recruit those folks put them to work on
on the problems they've solved in many
cases beforehand the reason I have
yellow for computer vision which is not
a knock against computer vision is
because it's not quite there yet for
self a fully driverless self-driving car
if you again rewound you know three four
five years this would have been a red
but today with all the community and
whatnot around computer vision this is
steadily getting to a green state so
pretty soon there'll be green and of
course then you'll have that perfect
formula for level for driving what we
run after is ride-sharing we believe
that the optimal way for people to move
around is to be able to summon a car but
the thing that's suboptimal today is
that you have to have a human driving
you whenever you want to move around
prevents the cost from being lower
prevents some safety issues prevent some
quality issues we think solving that
will will mean these next-generation way
of moving around will will come to
fruition but what we also see is that if
you let's say we never remove the driver
from the car that a ride hailing network
always had a human driver you are
inherently limited by the number of
miles you can drive which means that it
will never replace personal car
ownership will never fix that fatality
number I talked about all of those those
things we must solve so we think by
having a self-driving car that these
next-generation transportation networks
will come to fruition a lead VC is a guy
called Vinod Khosla a the founder of
Khosla Ventures an awesome guy who's
done some truly world-changing things he
has this quote which I'm a big fan of
your market entry strategy is often
different from your market disruption
start where you find a gap in the market
and push your way through and this
better communicated what I mentioned at
the very beginning which is that we
should build a self-driving car but do
it in a different way because if we
don't do that we're gonna fall into the
same traps as many of the others that
have died along the way
we have to find a way to do something
different that we own and that we are
really really good at and for us that
was retirement communities hands up if
you've ever visited a retirement
community and see way less there you go
surprise Lex you've got to get you out
to one but these are just amazing places
and the reasons we choose retirement
communities first to deploy our
self-driving technology and is for these
four reasons they are slower the speed
limits in these communities tend to be
far slower than you'd see on public road
much calmer roadway when you visit these
locations I liken it to listening to a
podcast at 0.75 X just very constrained
very slow and a little boring from time
to time but you've also got these HUD
felt transportation challenges we hear
from these residents all the time about
how transportation is a pain point and
that their only option is a personally
owned vehicle these folks know in many
cases they shouldn't be driving but
because they don't have an alternative
they still drive we hear from folks that
put off much-needed surgeries hip
replacements things like that because
they don't have a friend in town who's
gonna be able to move them around we
hear from folks with vision degeneration
that they just don't see a way that
they'll be able to move around and keep
that quality of life that they've been
able to have folks gripping steering
wheels for extended period of times all
these challenges that felt like the best
first place for a self-driving car to
begin and a clear path to customers we
see that on you know the roads today
ride-sharing on you know Public Citizen
mantra is a particularly brutal battle a
race to the bottom in terms of cost if
we owned every retirement community in
the country meaning the transportation
networks there that would in and itself
be a very valuable very valuable
business one of my favorite passengers
is on ahed she came to visit as recently
and gave this quick speech about why
self-driving cars matter to her in her
community
let's talk about our first community
this is the villages whenever I show
this slide people are astounded by the
number of residents in a community like
this over 125,000 and growing over 750
miles of road and what we have in this
location is an exclusive license to
operate an autonomous vehicle service
this is one of our other beliefs which
is that by partnering very deeply with
the community it means that we're able
to deliver a better service and that
we're able to grow a more reliable
business we won't have you know entrance
and competitors from all of the other
self-driving car companies in our
communities what we actually do in
exchange for that exclusive license is
grant these communities equity because
if we win it's probably in fact highly
likely as a result of those communities
and the addressable market
transportation in these regions is
massive these residents tend to be as a
lot of seniors tend to be quite affluent
which means that they have some
disposable income when it comes to being
able to pay for ride-sharing services
and other things like that so we find
that that recipies is absolutely perfect
here and we're launching and have
launched passenger services to these
these residents I've got a love awesome
feedback learned a lot about the needs
of providing ride-sharing for senior
citizens just some quick stats this is
from my series x fundraising dec just
about the size of the senior market
again this is the first place we go but
you can get a feel for just how large
this transportation market is today
there are 4 to 7 million seniors that's
growing by 2060 to over a hundred
million seniors in the US the total
addressable market for just seniors is
incredibly large 2,500 plus communities
all that good stuff and this is how we
see the world the landscape of potential
deployments
you've kind of got a lot of the big guys
focusing on that bottom left quadrant
they're focusing on large cities and it
makes sense because it's playing to
their unique strengths it's playing to
their ability to deploy thousands of
cars tens of thousands of cars it plays
the strengths that they have at least
some patience or ability to have more
extended time lines when it comes to
building this technology but first up
like us that you know fights for
survival every single day it means that
we have to do things differently so we
focus on that top right quadrant there
what we've kind of coined is
self-contained communities these places
are simplest slower but they also have
this ability for us to have that
exclusivity that I talked about and
there's some others of course that we
play in whether it's a senior market or
maybe even small citizen and things like
that let's talk about autonomous
technology so just to reiterate why do
we deploy in retirement communities
slower speeds simpler roadway there is a
central authority these places tend to
be run by private companies which makes
for a quite unique relationship in a
very positive way means we can deploy
faster it means we have the potential to
have more impact in these regions I also
turns out that retirement communities
tend to be located by this ideal weather
for self-driving cars think about
Arizona Florida etc we have a
world-class team building this a voyage
from all the major programs out there
and that makes our lives infinitely
easier one thing that also makes our
lives easier is the sensor configuration
of our car we've intentionally made this
decision that we're not going to focus
on optimizing for costs today but to
optimize for performance we want to get
to truly drive us sooner than most and
one of the easiest ways you can again
make your life easier is by optimizing
for high resolution sensors at the very
top of the vehicle we have the VLS 128
which is a 128 channel lidar that's
capable of seeing in three hundred three
hundred meters in 360 degrees many other
different light hours on the vehicle to
cover different certain blind spots all
together we
says twelve point six million points per
second and then just looks incredibly
high-resolution you'll see our car at
the bottom there and that's the the raw
point cloud output that we see in the
world
we run towards level four and for us
what that means is that if you're
building a demo self-driving car kind of
like we did at the Udacity project you
may focus on just the top four items
that top row you may focus on perception
prediction planning and controls and it
turns out you can build a very
impressive demo quite quickly by just
focusing on those things but of course
those things fall apart whenever edge
cases are introduced which happen all
the time so we've spent a ton of time on
all the items here because again our
goal is to build not a demo but a truly
driverless vehicle we also have a
emphasis on partnerships because what
we've noticed in the self-driving
ecosystem is that there's not just more
self-driving car companies building the
full stack there's now folks going into
simulation to mapping to middlewares to
tell the operations to routing to senses
of course and and ton more so we make
our lives again easier by partnering
with companies like this so that we
don't have to spin up a simulation team
or we don't have to spin up an
Operations team to go map the world we
can just work with these these very cool
companies let's talk about one unsolved
problem which fascinates me it's to do
with perception and you probably won't
be able to notice this unsolved problem
from just this picture but maybe if I
add some annotations you might foliage
trees bushes whatever you want to call
them you may have seen some quotes in
the media about some popular AV programs
struggling with such such foliage for
example cruise cars sometimes slow down
or stop if they see a bush on the side
of a street or a lane dividing pole that
was in the information wrong way this
one boobers self-driving car software
has routinely been fooled by the shadows
of tree brain
which it would sometimes mistake for
real objects insiders say that's
Business Insider and even voyage there's
only one hard stuff on the way the
culprit is a bush two feet high that
protrudes into a lane from a street
median which voyage considers possible
threat voyage mate remit and we did but
we don't think that's scalable and well
maybe it is I don't know but we at the
beginning of 2018 decide to solve this
problem so of course all of this resides
in the world of perception area of
particular fascination for me we're
sharing these slides but these are just
some of the the papers and research that
we see going on that intends to solve
those sorts of issues one of the reasons
you've seen those programs including
ours be particularly sensitive to
foliage is because from a perception
perspective one of the most well known
way to detect objects is to utilize the
map if you have this map and you
effectively simplifying to a certain
extent but subtract objects that aren't
in the map and then use that as a way to
you know understand what's in and around
you that's dynamic then of course you'll
end up with you know decent
representations of cars and pedestrians
and whatnot but if you know foliage
grows which it does trees then that's
gonna you know extend out from the map
and mean that that particular bush is
now an object in your path these
networks here which these all neural
networks don't use that same technique
they don't use the map as a prior
instead what they do is take of course
this 3d scan of the world and then take
em all learned approach to the problem
you'll have you know tens of thousands
hundreds of thousands of labels of cars
humans etc and then these next networks
will be able to pick these ones out were
particularly fascinated by pixel which
came from some great researchers at
Wilbur ATG voxel net came from Apple SPG
I've heard our engineers talking a lot
about fast and furious recently which
merges together perception
and prediction and tracking into a
single network which is pretty cool and
point pellets which I think came from
the new tana me team recently I think
Kyle is speaking soon right so just in
general we see a whole bunch of work
going out there to solve these issues
the other one that this these sorts of
networks solve which I also find
particularly fascinating is that if you
use traditional clustering algorithms
what you might see is that if two people
have stood next to each other
traditional algorithm will clusters as
one object which when you're trying to
you know move away from those edge cases
and build a truly self-driving car
that's a non-starter
right because pedestrians are the most
important thing you can probably detect
and detecting two things as one thing is
it's not going to cut it and of course
it does that because it's it's a dumb
algorithm it's not trained on any sort
of information but these networks again
are very very good at understanding the
features and perspectives of humans even
if they are in crowds and and whatnot
and that then helps all your stack
downstream because if you have accurate
perception information about objects in
and around you your predictions are much
better your tracking is much better and
ultimately how you navigate the world is
much safer I'm also particularly
fascinated by reinforcement learning
which I know Alexis as well if you've
read our way most recent work on
imitation learning I think that's
particularly cool another company we
track quite closely just because they do
amazing stuff is wave trying to build an
entirely self-driving a self-driving car
powered by reinforcement learning
I think about disengagement saz rewards
and things like that tool to to to net
to better performance also just errors
of learned behavior planning ultimately
fusing rules of the road with more
learned behaviors the ecosystem I think
it's this area that is thriving today
seeing just how many folks are diving
into not just the full stack but
building tools and building other really
important parts of the the stack the
maturation of sensors not just higher
resolution lidar but things like 3d
radar we get pitched all the time from
from these companies and it's clear to
see there's been a rise in volume from
from all these these great
great efforts lessons learned now that
I've been building voyage for two years
and prior to that four years that you
asked me what things have I personally
learned they're not technical in nature
so many things so these all may look
like cliches but I promise you they'll
came from lessons which were really
really painful in the moment don't be
intimidated so the thing that I feel you
know happens a lot in self-driving cars
is that because it started in this very
academic sense meaning you know Stanford
Carnegie Mellon and and whatnot that it
felt like to break into the industry you
had to also go through that same path
you had to get a PhD in something and
and really go that the path that was you
know well trodden but I think that you
know only takes the industry so far I
think it's really important that we get
folks from all different backgrounds all
different industries to come contribute
to this field because if we don't that
there is no driverless it can't happen
in that isolated bubble it needs to be
extended out so don't be intimidated by
those things understand your limitations
this is perhaps more of a kind of CEO
lesson for myself but I think when
you're building out a company from you
know one person or five people to today
with forty four folks you cannot do
everything and it's really important you
build a team around you that is able to
do what you used to do but do it 10
times better
I pray didn't spend enough time building
out that team until we had some
challenges our way when it comes to that
stuff
be proactive versus reactive I think
it's really crucial again when you're
building a company to try and predict
what's gonna happen next because if your
reactive you're constantly you know two
steps behind what other folks are doing
gay love the way I think a lot of folks
again perhaps overstay their welcome in
certain areas of the company when they
should just say okay I've got experts
now I can just step aside and let those
folks do what they do best and speaking
of which hire the best it's really easy
when all this pressures on when you're
building a company to kind of sacrifice
when it comes to your culture when it
comes to high
it's really crucial that you find folks
that are not just the best in their
field but are the best match for your
company and always be curious I think
it's always one of the the things we
believe in a voyage is that it's
important that knowledge is not isolated
just one person that that knowledge
should be spread throughout the company
because even though it may feel like
over sharing or over communicating what
that knowledge may mean for someone that
has a particularly unique background is
they may do something incredibly cool
with it they may build something that
totally transforms our company that's
about it can jump to questions if that's
helpful
that was great please give a big hand
[Applause]
how did you identify retired communities
as the target market to prioritize yes
so retirement communities for us was
actually there's a really long story
about I'll turn me down a little bit so
when we were starting voyage
sebastien theorem was very helpful in
starting helping us start this company
and of course as kind of naive you know
founders of a company we're like well
let's just take this El Camino thing and
like put it on everywhere else it looks
like El Camino and just do that over and
over again but he cautioned against that
and very wisely so because again you're
nothing special compared to the other
self-driving car companies out there by
doing so and in 2009 he had really
advocated to Google you know leadership
etc Larry Page that retirement
communities for self-driving cars might
just be the best way for Google to go
about deploying their self-driving cars
but and I can understand why I think the
Google folks were you know what Google
right well we're not just about
retirement communities were about the
world like level-5 or nothing right so
he got some pushback but he did some
research in that process met some folks
so when you know we were starting he's
like you got to check out these
retirement communities so we did we went
to visit and eventually we got there so
we want to got to that point without
sebastian pushing for that follow-up on
the quest from the retirement
communities the question is do you ever
think about the other collateral issues
especially the retirement community
would have to get into a car yep and how
exactly would they interface like
somebody wants make a call to have a car
come to their wherever they are and they
have to move from a point A to point B
so how did you ever think about all
these issues that are very germane it's
not just a vehicle moving on its own yep
but these are all collateral issues how
do you plan to address this it's good
question so the way we think about this
is that today we've intentionally
focused it on a segment of the market
which is called ad the active adult
communities these folks tend to be able
to you know go into their own cars or
into a taxi open the door sit down
without the need for any you know
assistance when it comes to that but
they may have vision issues they may
have other issues that prevent them from
driving perhaps for example we hear a
lot that folks feel really uncomfortable
driving in the evenings they feel
comfortable driving in the daytime
because their vision supports it but
when it comes to evening time they have
this mad rush to get home but there is
that other market which you're talking
about right which is folks that just
need that helping hand towards getting
to the car and one of our beliefs as a
company is that the senior market like I
had in that slide is surprisingly large
and what that means to us is that we
think we can own it we think we can be
that company that any senior citizen in
that situation thinks oh I should call
voyage because I need to get you know
from point A to point B instead of
thinking I should call way mo or cruise
or any of the folks that gonna go after
the general big market they'll think
about voyage and the reason they'll
think about is is because we'll deliver
a product to them that is meant for
those folks that's designed for their
use cases it may be that actually you
know if they're going on a long trip
let's say they're traveling 50 miles the
first mile of that trip and the last
mile of that trip may involve a human
like you know helping them into the car
and then dropping that human off
somewhere else to go do that all over
again it may involve crazy robots that
help people from their cars we've heard
from you know folk folks at Toyota
they're building these back carrying
robots and other things that may assist
seniors from getting to the
and whatnot so I think that's why that
market fries is particularly exciting
because it feels like you can deliver
these tailored products that would
enable us to be the the market leader
but today we focus on active adult but
who knows where you go next can you talk
a little bit about how you determined
your final sensor sweet hmm yeah so that
the truth is is never final so we think
about generations of vehicles so we have
a first generation vehicle which was a
Ford Fusion had a single valid eye
naitch TL 64 and a bunch of cameras
radar and we you know set some
milestones based on that vehicle and we
accomplished those milestones and then
once we reached kind of that the max in
which were able to take that vehicle we
then say oh we need you know to bring on
a g2 vehicle a second-generation vehicle
so we did that and we said okay we have
these certain goals in mind which are
pretty lofty and pretty ambitious
we need incredible range incredible
resolution for these things and actually
what we've discovered is that in our
particular communities going at the
speeds that we're going at radar isn't
particularly useful so we don't have
radar on our second generation vehicle
for example but I'm sure that when we go
to that third generation vehicle
there'll be other driving factors that
you know we work backwards from the
milestone to say what do we need on this
vehicle maybe cost in the third
generation vehicle right we may say that
hey we need a more affordable sensor
suite than what exists in our second
generation vehicle but they're driven by
technical requirements and that means
that you know we are able to really
marry the two with the vehicle
I was curious when you showed the
student LED content or when you showed
one of the students in your first
practice car had developed a traffic
light sensor and then you showed later
on that you know you were getting
student input for deep learning models
for steering wheel turns
I was wondering how what your system
architecture kind of looks like in terms
of the kinds of perception that you take
in I have modular it is and to what
extent deep learning algorithms have
played a part in those different parts
of that system yeah it's a good question
so I really encourage folks to get
familiar with Ross so
Ross has always been this kind of
playground for robot assists of all
different types of robots to be able to
try things out on robots and Ross one is
particularly notorious for kind of
hockey and hobbyist types of projects
but it's not meant for production Ross -
though which is in kind of an alpha
release state is definitely meant for
more production oriented things and the
reason I mentioned Ross is because it
has this awesome architecture which lets
you plug and play what they call nodes
and be able to experiment with different
approaches to the problem so for example
what you know is running that deep
learning model predicting steering
angles effectively replaced are more
rules based planner and perception
engine and we just plug the output of
that to of the steering angle straight
to our controller to just actuate the
vehicle and Ross is particularly good at
those sorts of architectures and it's
all open source so you can do some cool
stuff with it can you tell like how you
handle the liability insurance rear for
passengers for your vehicles also how we
handle insurance that question so we
have a pretty cool deal with a company
called intact insurance and the idea is
that insurance in the autonomous age is
gonna be very different than insurance
you know today right for human drivers
because there's different risk
assessments and whatnot and one of the
ways that we're able to prove to these
insurers that you know we're good at
what we do is actually sending them data
right we send them data from our cars as
we drive showing that as we move through
the world we accurately detected things
and planned around things and all that
good stuff and then they use that data
to inform our rates of insurance I think
that the future achieve insurance will
be on a similar lines but perhaps more
extreme where for example the rates will
change depending on the complexity of
the environment if we're just driving
down a straight road completely straight
and a zero vehicles around us our
insurance rate should be super low right
but if we enter a city center and as
thousands of people and cars and all
that crazy stuff our insurance rates
should just rise almost instantaneously
so what pond would someone today that
insures the passenger the car senses all
that stuff but I think there's a lot of
room for innovation there too
did you have any problems like
onboarding the people initially
when they were like you know skeptical
scared and then the other question is
what are the like major missing pieces
you computer vision to achieve out for
what was that last maze it missing
pieces between command in computer
vision to achieve my level four
self-driving gotcha so one of the more
interesting insights I think we had
about retirees is that again in my kind
of naive state back in 2016 my general
feeling was retirement communities might
not be the first to adopt this
technology right because they may be
slower to adopt new technology might be
scared of the technology all those sorts
of things and to kind of validate that I
went to talk to some senior citizens
because I talked to my own grandma she
hates self-driving cars sounds like
that's not a good sign but when to talk
to these folks in these sorts of
locations and the really interesting
thing we learned is that traditional
consumer software or devices yes there
is definitely a lag in adoption with
senior citizens and that's proven in
many studies many stats that senior
citizens are slower to adopt the
Facebook's of the world or the
instagrams or the whatsapp's all those
sorts of things cryptocurrency had Anna
but that's because they have these very
well-defined
processes that they've had for most of
their lives right instead of using
facebook they call someone up and they
have a chat you know a conversation with
someone about their day or all the stuff
that's going on or they you know don't
share a picture on Instagram they
physically mail a picture or something
like that so to change that behavior is
tough right because that's a behavior
that is fundamentally different than
what they used to they have to log on to
a computer go to this weird Facebook
thing and like share pictures with
thousands of people that's weird but the
difference between that and a
self-driving car is that our experience
is no different than the car they used
to it just turns out it's being driven
differently right like they see a car
it's the same
you know similar form factor to what
they used to they open it all they sit
in the back seat okay there's a button I
have to press to say go but it's pretty
similar to what I'm used to in my past I
want to learn a new behavior I have to
change something that I'm used to so
that was our first learning and then
also they actually really don't care too
much that it's autonomous they have very
when I'm in the car quite curious and
enthusiastic about the technology and
want to tell them about I don't know
lidar and deep learning and perception
and that they you know don't want to
hear any of that stuff and it kind of
dawned on me that the reason that is is
because what they senior citizens have
witness over their lifetimes it's far
more dramatic than I have right like
we've oldest passenger was 93 and she
told me a story about how when she was
very young she remembers literally
moving on an almost daily basis in a
horse and cart so when you talk about
like self-driving cars to those folks
like they just you know that they
couldn't care less because between that
period and today they've seen the birth
of like flight planes everywhere they've
seen car proliferation they've seen
scooters now they've seen like all of
this crazy subway systems so a
self-driving car to them is like oh
that's cool what I just wanted to move
me that's our biggest learning bet the
question was computer vision what needs
to happen between now and level-4 yeah
so I think the the Holy Grail right so
if you had perfect perception
self-driving cars are solved if we knew
every object that was on the road in and
around is within a reasonable distance
self-driving cars are solved false
positives are accepted today which i
think is good but you really want to
minimize false negatives right you want
to zero false negatives in the world and
I think that's why we still have a tiny
bit of work to do because when you think
about the reason for a test driver being
in the vehicle
well perception feeds everything
downstream right so if you miss an
object miss identifying object any of
that sort of stuff then that effect
causes the whole stack downstream to to
become quite chaotic that's why I'm
excited about all those networks I
talked about one of the other things we
believe that helps us minimize false
negatives to non-existent kind of state
as for us is that we band together
multiple networks so we don't just rely
on a single layer of perception we say
different networks have different
strengths for example voxel net is
particularly good at pedestrians but
Pixar is not so great as pedestrians
because it's from a bird's eye view
where pedestrians are quite thin and
whatnot so let's ban those two networks
together and let's also band together
some more traditional computer vision
algorithms that may not be processed on
the entire you know 360 scan but may be
processed on a small sample maybe at the
front of the vehicle for example so
there's just lots of little bits and
pieces like that to go through to
minimize the worst case scenario which
is a false negative but it's clear when
you see you know way mo and whatnot that
they feel very very very close to that
so state you mentioned that weather was
one of the main reasons this was a great
place to start can you talk about
hurricanes yes it was funny I got a
question recently from Alex Roy I mean
Lex is just talking about about okay in
the event of a hurricane right let's not
talk about the technology second but in
the event of a hurricane like we've all
seen those pictures of people you know
getting on the freeways and trying to
get out of the path of the hurricane
right how is that going to work in a
world where self-driving cars are
everywhere and personally driven
vehicles there may be more of the the
smaller set the smaller size I don't
quite have an answer to that yet but I
think it's an interesting kind of
thought problem from a technology
perspective the really important part
the really important part of whether
fuzz is remote operation so inside
everyone about sorry all of our vehicles
have a cellular connection right and
each of those vehicles is connected to a
remote operator that sat in somewhat
close proximity to that vehicle and that
remote operator has a few jobs one is to
just ensure the safe operation of the
vehicle make sure that vehicle is doing
as it's intended to do all those good
things but another is to make sure that
the operational domain that we are
currently operating in is the one that's
designed for so all these different
camera feeds are being you know live
stream to this remote operator and if
there is sudden downpour of rain that
remote operator has the ability to bring
that vehicle to
safe stop until that you know rain
shower disappears or whatever or hurric
el hurricane whatever it may be but
there are companies I was pitch rescind
by companies building weather
forecasting on a scale that is not
really used today but really
microclimates so thinking about just
like this small subsection of the
villages predicting and understanding
exact weather within those regions and
then having web hooks to tell you or as
voyage that that's about to happen so
there's a lot of cool stuff happening
there but remote operates is currently
kind of the eyes and ears of all cows to
prevent that sort of issue so please
give all or a big hand thank you guys
you