Kind: captions
Language: en
the following is a conversation with
Mark Zuckerberg his second time in this
podcast he's the CEO of meta that owns
Facebook Instagram and WhatsApp all
services used by billions of people to
connect with each other we talk about
his vision for the future of meta and
the future of AI in our human world this
is Alex Freedman podcast and now dear
friends here's Mark
Zuckerberg so you competed in your first
e just turn
and me as a fellow Jiu-Jitsu
practitioner and competitor I think
that's really inspiring given all the
things you have going on so I gotta ask
what was that experience like oh it was
fun I know yeah I mean well look I'm I'm
a pretty competitive person yeah um
doing sports that basically require your
full attention I think is really
important to my like mental health and
and the way I just stay focused at doing
everything I'm doing it's like I decid
to to get into martial arts and it's um
it's awesome I got like a ton of my
friends into it we all train together um
we have like a mini Academy in my garage
um and I guess um one of my friends was
like hey we should go do a tournament I
like okay yeah let's do it I'm not gonna
shy away from a challenge like that so
yeah it was but it was it was awesome it
was it was just a lot of fun you weren't
scared there was no fear I don't know I
I was I was pretty sure that I'd that
I'd do okay I like the confidence um
well so for people who don't know
Jiu-Jitsu is a martial art where you're
trying to break your opponent's limbs or
choke them uh to sleep uh and do so with
Grace and uh elegance and efficiency and
all that kind of stuff it's a uh it's a
kind of art form I think that you can do
for your whole life and it's a basically
a game a sport of human chess you can
think of there's a lot of strategy
there's a lot of sort of interesting
human dynamics of using leverage and all
that kind of stuff and uh it's kind of
incredible what you could do you can you
could do things like a small opponent
could defeat a much larger opponent and
you get to understand like the way the
mechanics of the human body works
because of that but you certainly can't
be distracted no you it's it's 100%
Focus sport to to compete I I you know I
needed to get around the fact that I
didn't want it to be like this this big
thing so I basically just I I rolled up
with a hat and sunglasses and I was
wearing a CO mask and I registered under
my first and middle name so Mark Elliot
and um and it wasn't until I pulled all
that stuff off right before I got on the
map that I think people knew as me so it
was it was pretty lowkey but you're
still a public figure yeah I mean I
didn't want to lose right the thing
you're partially afraid of is not just
the losing but being almost like
embarrassed it's so raw the sport in
that like it's just you and another
human being there's a primal aspect
there oh yeah it's great for a lot of
people it can be terrifying especially
the first time you're doing the comp
competing and it wasn't for you I see
the look of excitement in your face it
was Fe I just think part of learning is
failing okay right so I mean the main
thing like people who who train
Jiu-Jitsu it's like you need to not have
pride because I mean all the stuff that
you were talking about before about you
know getting choked or getting you know
a joint lock it's
um you only get into a bad situation if
you're not willing to tap once you
you've already lost right and but
obviously when you're getting started
with something you're not going to be an
expert at it immediately so you you just
need to to be willing to go with that
but I think this is like
I I don't know I mean maybe I've just
been embarrassed enough times in my life
yeah I I I do think there's a thing
where like you know as people grow up
maybe they don't want to be embarrassed
or anything they've built their adult
identity and they they kind of have have
a sense of of who
they they are and and what they want to
project and I don't know I think maybe
to some
degree you know your ability to keep
doing interesting things is your
willingness to be embarrassed again and
go back to Step One and start as a
beginner and get your ass kicked and you
know look stupid doing things and you
know I think so many of the things that
we're doing whether it's whether it's
this I mean this is just like a kind of
a physical part of my life but um but at
running the company it's like we we just
take on new adventures and
um you know all the big things that
we're doing I think of his like 10 plus
year missions that we're on where you
know often early on you know people
doubt that we're going to be able to do
it and the initial work seems kind of
silly and our whole ethos says we don't
want to wait until something is perfect
to put it out there we want to get it
out quickly and get feedback on it and
so I don't know I mean there's probably
just something about how I approach
things in there but I I just kind of
think that the moment that you decide
that you're going to be too embarrassed
to try something new then you're not
going to learn anything anymore but uh
like I mentioned that fear that anxiety
could be there it could creep up every
once in a while do you do you feel that
in especially stressful moments sort of
outside of the jism at just in
work stressful moments big decision days
big decision moments how do you deal
with that fear how do you deal with that
anxiety the thing that stresses me out
the most is always is always the people
challenges you know I I kind of think
that um you know strategy questions you
know I tend to have enough conviction
around the values of what we're trying
to do and what I think matters and what
I want our company to stand for that
those don't really keep me up at night
that much I mean I I kind of you know
it's not that I I get everything right
of course I don't right I mean make we
make a lot of mistakes but um but I at
least have a pretty strong sense of
where I want us to go on that the the
thing in in in running a company for you
know almost 20 years now one of the
things that's been pretty clear is when
you have a team that's
cohesive you can get almost anything
done and you know you can you can run
through super hard
challenges um you can make hard
decisions and push really hard to to do
the best work even you know and kind of
optimize something super well but when
when there's that tension I mean that's
that's when when things get really tough
and you know when I talk to other
friends who run other companies and
things like that I think one of the
things that I actually spend a
disproportionate amount of time on in
running this company is just fostering a
pretty tight Core Group of of people who
are running the company uh with me and
that to me is is kind of the thing that
both makes it fun right having having
you know friends and people you've
worked with for a while and new people
and New Perspectives but like a pretty
tight group who can who you can go work
on some of these crazy things with um
but to me that's also the most stressful
thing is is when when there when there's
tension um you know that's that that
weighs on me I I think the you know just
it's it's it's maybe not surprising I
mean we're like a very people focused
company and it's the the people is the
the part of it that that um you know
weighs on me the most to make sure that
we get right but yeah that that that I'd
say across everything that we do is
probably the the big thing so when
there's tension in in that inner circle
of of close folks so when you trust
those folks to help you make difficult
decisions
about uh Facebook WhatsApp Instagram
the future of the company and the
metaverse or the AI uh how do you build
that close nck group of folks uh to make
those difficult decisions is there
people that you have to have critical
voices very different perspectives on
focusing on the past versus the future
all that kind of stuff yeah I mean I
think for one thing it's just spending a
lot of time with whatever the group is
that you want to be that Core
Group grappling with all of the biggest
challenges and that requires a fair
amount of openness and you know so I
mean a lot of how I I run the company is
you know it's like every Monday morning
we get our it's about the top 30 people
together and we and this is a group that
just worked together for a long period
of time and I mean people people rotate
in I mean new people join people leave
the company people go to other roles in
the company so it's it's not the the
same group over time but and we spend
you know a lot of times a couple of
hours a lot of the time it's you know it
can be somewhat unstructured we like
I'll come with maybe a few topics that I
that are top of mind for me but I'll
I'll ask other people to bring things
and people you know raise questions
whether it's okay there's an issue
happening in some country um with with
some policy issue there's like a new
technology that's developing here we're
having an issue with this partner um you
know there's a design trade-off and
WhatsApp between two things that that
end up um being values that we care
about deeply and we need to kind of
decide where we want to be on that I
just think over time when um you know by
working through a lot of issues with
people and and doing it openly people
develop an intuition for each other and
a bond in
camaraderie um and to me developing that
is is like a lot of the fun part of
running a company or doing anything
right I think it's like having having
people who are kind of along on the
journey that you're that you feel like
you're doing it with nothing is ever
just one person doing it are there
people that disagree often within that
group it's a fairly combative group okay
so combat is part of it so this is
making decisions on design
engineering uh policy everything
everything everything yeah I have to ask
just back to jiujitsu for a little bit
what's your favorite submission now that
you've been doing it what's uh H how do
you like to submit your opponent Mark
Zuckerberg I
mean well first of all um do you prefer
noi or G Jiu-Jitsu so G is this outfit
you wear that uh is maybe mimics
clothing so you can choke look like a
kimono it's like the traditional martial
arts or come on pajamas um
pajamas that you could choke people with
yes well it's got the lapels yes yeah um
so I I like jiujitsu I also really like
MMA and so I think nogei more closely
approximates MMA and I think my style is
um is maybe a little closer to an MMA
style so like a lot of Jiu-Jitsu players
are fine being on their back right and
obviously having a good guard is is is a
critical part of of of Jiu-Jitsu but but
in MMA you don't want to be on your back
right because even if you have control
you're just taking punches while you're
on your back so um so that's no good so
you like being on top my my style is I'm
I'm probably more pressure and um and
yeah and and i' I'd probably rather be
the top player but um but I'm also
smaller right I'm not I'm not like a a
heavyweight guy right so from that
perspective I think like you know it's
especially because you know if I'm doing
a competition I'll compete with people
who are my size but a lot of my friends
are bigger than me so um so back takes
probably pretty important right because
that's where you have the most leverage
Advantage right where where um you know
people you know their arms your arms are
very weak behind you right so um so
being able to get to the back and and
and take that pretty important but I
don't know I feel like the right
strategy is to not be too committed to
any single submission that said I don't
like hurting people so um so I always
think that chokes are are a somewhat
more humane way to go than than joint
locks yeah and it's more about control
it's less Dynamic so you're basically
like a khabib norov type of fighter so
so let's go yeah back take to a rear
naked choke I think is like the clean
the clean way to go straightforward
answer right there what advice would you
give to um to people looking to start
learning jiu-jitsu given how busy you
are given where you are in life that
you're able to do this you're able to
train you're able to compete and get uh
uh to learn something from this
interesting art I just think you have to
be willing to
um to just get beaten up a lot yeah I
mean it's but but I mean over time I
think that there's there's a flow to all
these things and there's um you know one
of
the one of I don't know my my
experiences that I think kind of
transcends you know running a company
and the different different activities
that I like doing are I I really believe
that like if you're going to accomplish
whatever anything a lot of it is just
being willing to push through right and
having the grit and determination to to
to push through difficult situations um
and I think for a lot of people that um
that ends up being sort of a differen
maker between the people you know who
who who kind of get the most done and
and not I mean there's all these
questions about like um you know how how
many days people want to work and things
like that I think almost all the people
who like start successful companies or
things like that are just are working
extremely hard but I think one of the
things that you learn both by know doing
this over time or you know very acutely
with things like Jiu-Jitsu or or surfing
is um you can't push through
everything
and I that
that's you you learn this stuff very
acutely run doing Sports compared to
running a company because running a
company the cycle times are so long
right it's like you start a project and
then you know it's like months later or
you know if you're You're Building
Hardware it could be years later before
you're actually getting feedback and
able to you know make the next set of
decisions for the next version of the
thing that you're doing whereas you one
of the things that I just think is
mentally so nice about these very high
turnaround conditioning Sports things
like that is you get feedback very
quickly right it's like okay like I I
don't counter something correctly you
get punched in the face right so not in
Jiu-Jitsu you don't you don't get
punched in Jiu-Jitsu but in MMA um there
are all these analogies between all
these things that I think actually hold
that are that are
like important life lessons right it's
like okay you're surfing a wave it's
like you know sometimes you're like you
can't go in the other direction on it
right it's like there are limits to kind
of what you know it's like foil you can
you can pump the foil and and push
pretty hard in a bunch of directions but
like yeah you you know at some level
like the momentum against you is is
strong enough you're that's not going to
work and and I do think that um that's
sort of a a humbling but also an
important lesson for I think people who
are running things or building things
it's like yeah you you um you know a lot
of the game is just being able to kind
of push and and and and work through
complicated things but you also need to
kind of have enough of an understanding
of like which things you you just can't
push through and where where um um the
Finesse is more important yeah what are
your Jiu-Jitsu life
lessons well I think you did
it you made it sound so simple and we so
eloquent that it's easy to miss but
basically being okay and accepting the
wisdom and the joy in the uh getting
your ass kicked in the full range of
what that means I think that's a big
gift of the being humbled somehow being
humbled especially physically opens your
mind to the the full process of learning
what it means to learn which is being
willing to suck at something I think jiu
just very repetitively efficiently
humbles you over and over and over and
over to where you can carry that lessons
to places where you you don't get
humbled as much whether it's research or
running a company or building stuff the
the cycle is longer and just so you can
just get humbled in as period of an hour
over and over and over and over
especially when you're a beginner you
have a little person just you know
somebody much smaller than you just kick
your ass uh
repeatedly uh definitively where there's
no argument oh yeah and then you you
literally tap because if you don't tap
you're going to die so this is an
agreement you could have killed me just
now but we're friends so we're going to
agree that you're not going to to and
that kind of humbling process it just
does something to your psyche to Your
Ego that puts it in its proper context
to realize that you know everything in
this life is like a journey from sucking
through a hard process of improving o
rigorously day after day after day after
day like any kind of success requires
hard work um yeah g just more than a lot
of sports I would say cuz I've done a
lot of them really teaches you that and
you made it sound so simple like I'm I'm
you know it's it's okay it's part of the
process you just get humble get your
just I've just failed and been
embarrassed so many times in my life
that like you know I'm I'm it's a core
competence at this point it's a core
competence well yes and there's a deep
truth to that being able to and you said
it in the very beginning which is that's
the thing that stops US especially as
you get older especially to develop
expertise in certain areas the not being
willing to be a beginner in a new area
yeah uh that because that's where the
growth happens is being willing to be a
beginner being willing to be embarrassed
saying something stupid doing something
stupid um a lot of us that get good at
one thing you want to show that off and
it
sucks uh being a beginner but it's it's
where growth happens yeah well speaking
of which let me ask you about AI it
seems like this year for the entirety of
the human civilization is an inter
interesting year for the development of
artificial intelligence a lot of
interesting stuff is happening So Meta
is a big part of that uh meta has
developed llama which is a 65 billion
parameter
model uh there's a lot of interesting
questions they can ask here one of which
has to do with open source but first can
you tell the story of developing of this
model and uh making the complicated
decision of how to release it
yeah sure I think you're right first of
all that in the last year there have
been a bunch of advances on scaling up
these large Transformer models so
there's the language equivalent of it
with large language models um there sort
of the image generation equivalent with
these large diffusion models um there's
a lot of fundamental research that's
gone into this and meta has taken the
approach of being quite open
an academic in in in our development um
of of AI part of this is we want to have
the best people in the world researching
this and um and a lot of the best people
want to know that they're going to be
able to share their work so that's part
of the deal that we that we have is that
you know we can get you know if if
you're one of the top AI researchers in
the world you can come here you can get
access to kind of industry scale um
infrastructure and and and part of our
ethos is that we we want to share what's
what's invented um broadly we do that
with a lot of the the different AI tools
that we create and llama is the language
model that that our research team made
and you know we we did a limited um a
limited open source release for it right
where which was intended for researchers
to be able to use it um but you know the
responsibility and and getting safety
right on these is um is very important
so we didn't think that for the first
one there there were a bunch of
questions around whether we should be
releasing this commercially so we kind
of punted on that for for V1 of of llama
and and just released it from research
now obviously by releasing it for
research um you know it's out there but
but companies know that that they're
that they're not supposed to kind of put
it into commercial releases and um you
know we're we're working on the
follow-up models for this and and
thinking through how how um what what
the the how exactly this should work for
for follow on now that we've had time to
to work on a lot more of the the safety
and um and the pieces around that but
but overall I mean this
is I I just kind of think
that that it would be good if there were
a lot of different folks who had the
ability to build state-of-the-art
technology here you know it's and not
just a small number of of big companies
but to train one of these AI models the
state-of-the-art models is um just takes
you know hundreds of millions of dollars
of infrastructure right so there are not
that many organizations in the world um
that can do that at the biggest scale
today and now it gets it gets more
efficient every day so um so I I I do
think that that will be available to
more folks over time but but I just
think like there's there's all this
Innovation out there that people can
create and um and and I I just think
that will also learn a lot by by seeing
what the whole community of students and
um and hackers and startups and and
different folks um build with this and
that's kind of that's kind of been how
we've approached this and it's also
we've done a lot of our infrastructure
and we took our whole data center design
and our server design and we we built
this open compute project where we just
made that public and um part of the
theory was like all right if we make it
so that more people can use the server
design then um then that'll enable more
Innovation it'll also make the server
design more efficient and that'll
that'll make our business more efficient
too so that's worked and we've we've
just done this with a lot of our our
infrastructure so for people who don't
know you did the limited release I think
in February of of this year of llama and
it got quote unquote
leaked meaning like it uh
escaped the uh the the limited release
aspect but it was you know that
something you probably anticipated given
that it's just released to research we
shared it with researchers right so it's
just trying to make sure that there's
like a slow release yeah uh but from
there I just would love to get your
comment on what happened next which is
like this is a very vibrant open source
community that just build stuff on top
of it there's uh llama CPP basically
stuff that makes it more efficient to
run on smaller computers yep um there's
combining with uh uh reinforcement
learning with human feedback so some of
the different interesting fine tuning
mechanisms there's then also like
fine-tuning and a gpt3 Generations
there's a lot of uh GPT for all alpaka
uh colossal AI all these kinds of models
you just kind of spring up like run on
top of like what do you think about that
no I think it's been really neat to see
I mean there's been folks who are
getting it to run on local devices right
so if you're an individual who just you
know wants to experiment you know with
this at home you probably don't have a
large budget to get access to like a L
amount of cloud computes so getting it
to run on your local laptop um you know
is is uh is pretty good right and pretty
relevant um and then there are things
like yeah llama
CPP um reimplemented it more efficiently
so you know now even when we run our own
versions of it um we can do it on way
less compute and it just way more
efficient save a lot of money um for
everyone who who uses this so that that
is is is good um I do think it's worth
calling out that
because this was a relatively early
release um llama isn't quite as on the
frontier as for example the biggest open
AI models or the biggest um Google
models right I mean you mentioned that
the largest llama model that we released
had 65 billion parameters and no one
knows you know I guess outside of open
AI um exactly what the specs are for um
for for gp4 but but I think the you know
my understanding is it's like 10 times
bigger um and I think Google's Palm
model is is also I think has about 10
times as many parameters now the Llama
models are very efficient so they they
perform well for for something that's
around 65 billion parameters so for me
that was also part of this because
there's this whole debate around you
know is it good for everyone in the
world to have access to um to the most
Frontier AI models and I I I think as
the IM models start approaching
something that's like a super human
intelligence I that's a bigger question
that we'll have to Grapple with but
right now I mean these are still you
know very basic tools they're um you
know they're they're powerful in the
sense that you know a lot of Open Source
software like databases or web servers
can enable a lot of pretty important
things um but I don't think anyone looks
at the the you know the current
generation of llama and thinks it's um
you know anywhere near a super
intelligence so I I think that a bunch
of those questions around like is it is
it good to to kind of get out there I I
think at this stage surely you you want
more researchers working on it for all
the reasons that um that open source
software has a lot of advantages and we
talked about efficiency before but
another one is just open source software
tends to be more secure because you have
more people looking at it openly and
scrutinizing it um and finding holes in
it um and that makes it more safe so I
think at this point it's more
I think it's generally agreed upon that
open source software is generally more
secure and safer um than things that are
kind of developed in a silo where people
try to get through security through
obscurity so I think that for the scale
of of of what we're seeing now with AI I
think we're more likely to get to you
know good alignment and good um
understanding of of of kind of what
needs to do to make this work well by
having it be open source and and that's
something that I think is is quite good
to have out there and and and happening
publicly at this point meta released a
lot of models as open source so uh the
mass multi lingual speech model theage
model that's I mean I'll ask you
questions about those but the point
is uh you've open sourced quite a lot
you've been spearheading the open source
movement where's uh that's really
positive inspiring to see from one angle
from the research angle of course
there's folks who are really terrified
about the existential threat of
artificial intelligence and those folks
will say that you you know um you have
to be careful about the open sourcing uh
step but what where do you see the
future of Open Source here uh as part of
meta the tension here is do you want to
release the magic sauce that's one
tension and the other one is uh do you
want to put a powerful tool in the hands
of uh Bad actors even though it probably
has a huge amount of positive impact
also
yeah I mean again I think for the stage
that we're at in the development of AI I
don't think anyone looks at the current
state of things and thinks that this is
super intelligence um and you know the
models that we're talking
about the Llama models here are you know
generally an order of magnitude smaller
than what open AI or Google are doing so
I I think that at least for the stage
that we're at now the equities Balan
strongly in my view towards doing this
more openly um I I think if you got
something that was closer to Super
intelligence then I think you'd have to
discuss that more and and think through
that um a lot more and we haven't made a
decision yet as to what we would do if
we were in that position but I don't
think I I think there's a good chance
that we're pretty far off from that
position so um so I I'm I'm
not I'm certainly not saying that the
position that we're taking on this
now applies to every single thing that
we would ever do and you know certainly
inside the company and we probably do
more open source work than you know most
of the other big tech companies but we
also don't open source everything right
a lot of our the core kind of app code
for WhatsApp or Instagram or something I
me we're we're not open sourcing that
it's not like a a general enough piece
of software that would be useful for a
lot of people to do different things um
you know whereas the software that we do
whether it's like a an open source
server design or um or basically you
know things like mcash right like a a
good you know it was was probably our
earliest project um that that I worked
on it was probably one of the last
things that I that I coded and and led
directly for the company um but but
basically this like caching tool um for
for quick dat data retrieval um these
are things that are just broadly useful
across like anything that you want to
build and and I think that some of the
language models now have that feel as
well as some of the other things that
we're building like the translation tool
that that you just referenced so text to
speech and speech to text you've
expanded it from around 100 languages to
more than 1,00 languages and you can
identify more than the model can
identify more than 4,000 spoken
languages which is 40 times more than
any known previous technology to me
that's really really really exciting in
terms of connecting the world breaking
down barriers that language creates yeah
I think being able to translate between
all of these different pieces in real
time this has been
a kind of common sci-fi idea that we'd
all have you know whether it's I know an
earbud or glasses or something that can
help translate in real time um between
all these different languages and that's
one that I think technology is basically
delivering now so I think yeah I think
that's pretty pretty exciting uh you
mentioned the next version of llama what
can you say about the next version of
llama
what what can you say about like what uh
what were you working on in terms of
release in terms of the vision for that
well a lot of what we're doing is taking
the first version which was primarily
you know this research version and
trying to now build a version that
has all of the latest state-of-the-art
safety precautions built in um and and
we're um we're using some more data to
train it um from across our services
but but a lot of the the work that we're
doing internally is really just focused
on making sure that this is um you know
as aligned and responsible as as
possible and you know we're building a
lot of our own you know we're talking
about kind of the open source
infrastructure but you know the the main
thing that we focus on building here you
know a lot of product experiences to
help people connect and express
themselves so you know we're going to
I've I've talked about a bunch of this
stuff but um then you'll have you know
an assistant that you can talk to in
WhatsApp um you know I think I I I think
in the future every Creator will will
have kind of an AI agent that can kind
of act on their behalf that their fans
can talk to I I I want to get to the
point where every small business
basically has an AI agent that people
can talk to for you know to do Commerce
and customer support and things like
that so they're going to be all these
different things
and llama or the language model
underlying this is is basically going to
be the engine that powers that the
reason to open source it is that um as
as we did with um with the the first
version is that it uh you know basically
it unlocks a lot of innovation in the
ecosystem we will make our products
better as well um and also gives us a
lot of valuable feedback on security and
safety which is important for making
this good but yeah I mean the the the
work that we're doing to advance the
infrastructure it's um it's basically at
this point taking it Beyond a research
project into something which is ready to
be kind of core infrastructure not only
for our own products but um you know
hopefully for for a lot of other things
out there too do you think the Llama Or
the language model underlying that
version too will be open
sourced you're do you have internal
debate around that the pros and cons and
so on this is I mean we were talking
about the debates that we have
internally and I think um I think the
question is how to do it right I mean
it's I think we you know we did the
research license for V1 and and I think
the the big thing that we're that we're
thinking about is is basically like
what's the what's the right the right
way so there was a leak that happened I
don't know if you can comment on it for
V1 you know we released it as a research
project um for researchers to be able to
use but in doing so we put it out there
so um you know we were very clear that
anyone who uses the the code and the
weights doesn't have a commercial
license to put into products and we've
we've generally seen people respect that
right it's like you don't have you any
reputable companies that are basically
trying to put this into into um their
commercial products but but yeah but by
sharing it with you know so many
researchers it's it's you know it did
leave the building but uh what have you
learned from that process that you might
be able to apply to V2 about how to
release it safely
effectively uh if if you release it yeah
well I mean I think a lot of the
feedback like I said is just around you
know different things around you know
how do you fine-tune models to make them
more aligned and safer and you see all
the different data recipes that um you
you mentioned a lot of different
projects that are based on this I me
there's one at Berkeley there's you know
there just like all over and um
and people have tried a lot of different
things and we've tried a bunch of stuff
internally so kind of we're we're we're
making progress here but also were able
to learn from some of the best ideas in
the community and you I think it you
know we want to just continue continue
pushing that forward but I don't have
any news to announce on on this if
that's if that's what you're you're
asking I mean this is a a thing that
we're uh we're still we're still kind of
you know actively working through the
the the right way to move forward here
the details of the secret sauce are
still being developed I see uh you
comment on what do you think of uh the
thing that worked for GPT which is the
reinforcement learning with human
feedback so doing this alignment process
do you find it interesting and as part
of that let me ask because I talked to
Yan laon before talking to you today he
asked me to ask or suggested that I ask
do you think llm fine-tuning will need
to be crowdsourced Wikipedia style so
crowd sourcing so this kind of idea of
how to inte the human in the fine-tuning
of these Foundation models yeah I think
that's a really interesting idea that
I've talked to Yan about a bunch
um and you we were talking about how do
you basically train these models to be
as as safe and and aligned and
responsible as possible and you know
different groups out there who doing
development test different data recipes
and fine-tuning but th this idea that
you you just mentioned
is that at the end of the day instead of
having kind of one group fine tune some
stuff and then another group you know
produce a different fine tuning recipe
and then us trying to figure out which
one we think works best to produce the
most aligned model
um I I do think that it would be nice if
you could get to a point where you had a
Wikipedia style collaborative
way for a a kind of a broader Community
to um to to find tune it as well now
there's a lot of challenges in that both
from an
infrastructure and like a community
management and product perspective about
how you do that so I I haven't worked
that out yet um but but as an idea I
think it's it's quite compelling and I
think it it goes well with the ethos of
open sourcing the technology is also
finding a way to have a a kind of
community-driven um
a community-driven training of it um but
I think that there are a lot of
questions on this in general these this
these questions around what's the the
best way to produce aligned AI models
it's very much a research area and it's
one that I think we will need to make as
much progress on as the kind of core
intelligence capability of the of the um
the models themselves well I just did a
conversation with Jimmy Wales the
founder of Wikipedia and to me
Wikipedia is one of the greatest
websites ever created and is a kind of a
miracle that it works and I think it has
to do with something that you mentioned
which is community you have a small
community of editors that somehow work
together well and they uh they handle
very
controversial topics and they handle it
with balance and with Grace despite sort
of the attacks that will often happen a
lot of the time I mean it's not it's it
has issues just like any other human
system but yes I mean the balance is I
mean it's a it's amazing what they've
been able to achieve but it's it's also
not perfect and I think that that's um
there's still a lot of
challenges right it's uh the more
controversial the topic the more the
more difficult uh the um the journey
towards quote unquote truth or knowledge
or wisdom that wikip beia address to
capture in the same way AI models will
need to be able to generate those same
things truth knowledge and wisdom and
how do you align those models that
they
generate um something that uh is closest
to truth there's these concerns about
misinformation all this kind of stuff
that nobody can
Define and that's a it's something that
we together as a human species have to
Define like what is truth and how to
help AI systems generate that is one of
the things language models do really
well is generate convincing sounding
things that can be completely wrong
and so how do you align
it uh to be less
wrong and part of that is the training
and part of that is the alignment and
however you do the alignment stage and
just like you said it's a very new and a
very open research problem yeah and I
think that there's also a lot of
questions about whether the current
architecture for
llms as you continue scaling it what
happens um I mean a lot of the a lot of
what's been exciting in the last year is
that there was there's clearly a
qualitative breakthrough where you know
with with some of the GPT models um that
open I put out and and that others have
been able to do as well I think it
reached a kind of level of quality where
people like wow this is this feels
different and um and like it's going to
be able to be the foundation for
building a lot of awesome products and
experiences and value but I think the
other realization that people have is
wow we just made a breakthrough
um if there are other breakthroughs
quickly then I think that there's the
sense that maybe we're we're closer to
general intelligence but I think that
that idea is predicated on the idea that
I think people believe that there's
still generally a bunch of additional
breakthroughs to make and that it's um
we just don't know how long it's going
to take to get there and you know one
view that some people have um this
doesn't tend to be my view as much is
that simply scaling the current llms and
you know getting to higher parameter
count models by itself we we'll get to
something that is closer to um to to
general intelligence but um I don't know
I tend to think that there's probably
more more
um fundamental steps that need to be
taken along the way there but still the
leaves
taken with this extra alignment step is
quite incredible quite surprising to to
a lot of folks and on top of that when
you start to have hundreds of millions
of people potentially using a product
that integrates that you can start to
see civilization transforming effects
before you achieve super quote unquote
super intelligence it could be super
transformative without being a super
intelligence oh yeah I mean I think that
there are going to be a lot of amazing
products and value that can be created
with the current level of techn ology um
to some degree you I'm excited to work
on a lot of those products over the next
few years and I think it would just
create a tremendous amount of whiplash
if the number of breakthroughs keeps
like if if they're keep on being stacked
breakthroughs because I think to some
degree industry in the world needs some
time to kind of build these
breakthroughs into the products and
experiences that we all use so we can
actually benefit from them um but
I don't know I think that there's just a
a a like an awesome amount of stuff to
do and I think about like all of the I
don't know small businesses or
individual entrepreneurs out there who
um you know now we're going to be able
to you know get help coding the things
that they need to go build things or
designing the things that they need or
um we'll be able to you know use these
models to be able to do customer support
for the people that they're that they're
serving you over WhatsApp without having
to you know it's I I think that's that's
just going to be I just think that this
is all going to be you know super
exciting it's going to create better
better experiences for people and just
unlock a ton of innovation and value so
I don't know if you know but uh you know
what is it over three billion people use
WhatsApp Facebook and
Instagram uh so any kind of AI fueled
products that go into that like we're
talking about anything with llms will
have a tremendous amount of impact d do
you have ideas and thoughts
about possible
products that might start being
integrated into uh into these platforms
used by so many people yeah I I think
there's three main categories of things
that we're working on
um the first that that I think is
probably the most
interesting is
um you know there's this notion of like
you're going to have an assistant or or
an agent who you can talk to and I think
probably the biggest thing that's
different about my view of how this
plays out from what I see with um with
open Ai and Google and others is you
know everyone else is building like the
One Singular AI right it's like okay you
talk to chat GPT or you talk to Bard or
you talk to Bing and my view is
that that there are going to be a lot of
different AIS that people are going to
want to engage with just like you want
to use um you know a number of different
apps for different things and you have
relationships with different people in
your life who fill different emotional
roles for you um and I um so I think
that they're going to be people have a
reason that they that I think you don't
just want like a singular Ai and that
that I think is probably the biggest
distinction in in in terms of how how I
think about this and a bunch of these
things I I think you'll you'll want an
assistant um I I me I mentioned a couple
of these before I think like every
Creator who you interact with will
ultimately want some kind of AI that can
proxy them and be something that their
fans can interact with or that allows
them to interact with their fans um this
is like the common Creator prise
everyone's trying to build a community
and engage with people and they want
tools to be able to amplify themselves
more and be able to do that um but but
you only have 24 hours in a a day so um
so I think having the ability to
basically like bottle up your
personality and um or or you know like
give your fans information about when
you're performing a concert or or
something like that I mean that's that I
think is going to be something that's
super valuable but it's not just that
you know again it's not this idea that I
think people are going to want Just One
Singular AI I think you're going to you
know you're going to want to interact
with a lot of different entities and
then I think there's the business
version of this too which we've touched
on a couple of times which is um
I think every business in the world is
going to want basically an AI that um
that you know it's like you have your
page on Instagram or Facebook or
Whatsapp or whatever and you want to you
want to point people to an AI that
people can interact with but you want to
know that that AI is only going to sell
your products you don't want it you know
recommending your competitor stuff right
so so it's not like there can be like
just uh you know One Singular AI that
that can answer all the questions for a
person because you know that qu like
that AI might not actually be aligned
with you as a business to um to to
really just do the best job providing
support for for your product so I think
that there's going to be a clear need um
in the market and in people's lives for
there to be a bunch of these part of
that is figuring out the research the
technology that enables the
personalization that you're talking
about so not one centralized Godlike llm
but one just a huge diversity of them
that's fine-tuned to particular needs
particular Styles particular businesses
particular Brands all that kind of stuff
and also enabling just enabling people
to create them really easily for the you
know for to for your own business or if
you're a Creator to to be able to help
you engage with your fans and I I think
that's um so yeah I think that there
there's a clear kind of interesting
product Direction here that I think is
fairly unique from from what you I any
of the other big companies are are
taking um it also aligns well with this
sort of Open Source approach because
again we we sort of believe in this more
Community oriented uh more democratic
approach to building out the products
and Technology around this we don't
think that there's going to be the one
true thing we think that there there
should be kind of a lot of development
so that part of things I think is going
to be really interesting and we could we
could go price spent a lot of time
talking about that and the the kind of
implications of um of that approach
being different from what others are
taking um but then there's a bunch of
other simpler things that I think we're
also going to do just going back to your
your question around how this finds its
way into like what what do we build um
there going to be a lot of simpler
things around
um okay you you post photos on Instagram
and Facebook and you know in WhatsApp
and messenger and like you want the
photos to look as good as possible so
like having an AI that you can just like
take a photo and then just tell it like
okay I want to edit this thing or
describe this it's like I think we're
we're going to have tools that are just
way better than than what we've
historically had on this um and that's
more in the image and media generation
side than the large language model side
but but it's it all kind of you know
plays off of advances in the same space
um so there are a lot of tools that I
think are just going to get built into
every one of our products I think every
single thing that we do is going to
basically get evolved in in this
direction right it's like in the future
if you're advertising on our services
like do you need to make your own kind
of AD creative it's no you'll just you
know you just tell us okay I'm I'm a dog
walker and I I'm willing to walk
people's dogs and help me find the right
people and like create the ad unit that
will perform the best and like give an
objective to to the system and it just
kind of like connects you with the right
people well that's a super powerful idea
of generating the language almost like
uh rigorous AB testing for you that
works to find the the best customer for
your thing I mean to me advertisement
when done well just finds a good match
between a human being and a thing that
will make that human being
happy yeah totally and do that as
efficiently as possible when it's done
well people actually like it you know
it's um I think that there's a lot of
examples where it's not done well it's
annoying and I think that that's what
kind of gives it a bad rap but um but
yeah a lot of the stuff is possible
today I mean obviously AB testing stuff
is built into a lot of these Frameworks
the thing that's new is having
technology that can generate the ideas
for you about what to AB test something
that that's exciting so this will just
be across like everything that we're
doing right all the metaverse stuff that
we're doing right it's like you want to
create worlds in the future you'll just
describe them and then it'll create the
code for you so so natural language
becomes the the inter face we use for
all the ways we interact with the
computer with with the digital more of
them yeah yeah totally yeah which is
what everyone can do using natural
language and with translation you can do
it in any kind of
language um I I mean for the
personalization is really really really
interesting yeah it unlocks so many
possible things I mean I for one look
forward to creating a copy of myself I
know we talked about this last time but
this has since last time this becomes
how we're
closer much closer like I could
literally just having interact with some
of these language models I can see the
Absurd situation where I'll have a uh
large uh or a Lex language model and
I'll have to have a conversation with
him about like Hey listen like you're
just getting out of line and having a
conversation where you fine-tune that
thing to be a little bit more respectful
or something like this I mean that's
that's going to be the that seems like
an amazing
product for businesses for humans just
not not just the assistant that's facing
the individual but the assistant that
represents the individual to the public
both of both
directions there's basically a a layer
that is the AI system through which you
interact with the outside world with the
outside world that has humans in it
that's really interesting and you that
have social networks that connect
billions of people it seems like a heck
of a large scale place to test some of
this stuff out yeah I mean I think part
of the reason why creators will want to
do this is because they already have the
communities on our
services yeah and and and a lot of the
interface fo