Simulation: the new Scaling Law — Joon Sung Park, Simile AI ·

00:02

Today we have June in the podcast excited to kick this one off very exciting company

今天我们播客请到了June，很激动能开始这一期。非常令人兴奋的公司。

00:08

I want to kick off and I see the question, you know talk us through the story of your life. How have you gotten here?

我想开场，然后我看到了这个问题，你知道，跟我们讲讲你的人生故事吧。你是怎么走到今天的？

00:14

Yeah, for sure

嗯，当然。

00:15

So we're excited to be here a story from above my life

那么，我们很高兴来到这里。从高处讲讲我的人生故事。

00:19

So I was born in Korea

所以我出生在韩国。

00:21

And I lived there for good 11 years or so of my life and then my family moved to Boston

我在那里生活了大约11年，然后我们家搬到了波士顿。

00:28

so we moved when I was 11

所以我们在我11岁时搬了家。

00:31

And my parents were doctors so they were basically going through their postdoctoral studies

我父母是医生，所以他们基本上都是在做博士后研究。

00:35

My dad was a surgeon so he was doing his

我爸是外科医生，所以他在Boston Children's Hospital做休假年，我就是在那里长大的。其实和tech没什么关系，我那时候很喜欢，你知道，音乐、绘画，就是那种类型的人。我其实到高中才比较认真接触绘画，但那是以前的事了。后来在韩国之后，我主要在东海岸长大，在New Hampshire住了好多年，然后去Pennsylvania上大学，在大学里更多地进入了tech圈。所以我原本是学艺术出身的，我当时真的以为那会是我的职业生涯。

00:38

sabbatical years actually at the Boston Children's Hospital. So I grew up there

其实是在波士顿儿童医院度过的休假年。所以我是在那里长大的。

00:43

Not too close to tech actually. I was very much like, you know

其实离 tech 并不近。我特别像那种，你懂的。

00:48

Music, painting, painting, like that kind of guy. I actually got into painting a little bit later in high school

音乐、绘画、画画，就是那种人。我其实是在高中稍晚一点的时候才开始对绘画感兴趣。

00:55

But that's what I used to do and then I grew up mostly in the east coast after Korea

但那是我以前做的事，然后我主要是从韩国过来之后在东海岸长大的。

01:00

So I lived good number of years in New Hampshire and then I went to college in Pennsylvania

所以我在新罕布什尔州住了好几年，然后去宾夕法尼亚州上了大学。

01:06

And I got into more of this tech scene in college

然后在大学里我更多地接触到了 tech 圈。

01:11

So I was originally trained to be an artist. I actually thought that would be my actually professional career

所以我原本接受的是艺术家的训练。我其实以为那会是我真正的职业生涯。

01:16

So I wasn't a hobby was actually like hey, let's make living out of this

所以这不是爱好，其实是那种“嘿，咱们靠这个谋生吧”的感觉。

01:20

And then gradually I got really interested in this idea of hey the greatest artist often creates their own medium

然后我逐渐对这样一个想法产生浓厚兴趣：最伟大的艺术家常常会创造自己的媒介。

01:27

And the best medium that we had available today was actually in computation

而我们今天能利用的最好的媒介，其实就是计算。

01:31

So I decided to go deeper into that and

所以我决定深入钻研这个，然后

01:34

One things led to another and obviously we can go deeper into this

事情一件接一件，显然我们可以更深入地聊聊这个话题。

01:37

But I decided that research was something that gradually that I got interested in and here I am

但我是逐渐对研究产生兴趣的，于是就到了现在。

01:46

So there's obviously a lot that you packed into the research components

所以很明显，你在研究这块涵盖了大量内容。

01:50

You had one of the best papers in 2023 which was the generative agents paper

你2023年有篇论文是当年最好的之一，就是generative agents那篇。

01:55

Commonly known as the smallville paper. Yeah, we feel free to call back to anything else that you mentioned

通常被称为 Smallville 论文。

02:01

But most people would have heard of you from this obviously

对，我们可以随时回顾你提到的其他内容。

02:04

Do you have any statistics of how many people

但大多数人显然是因为这个才知道你的。

02:07

Have like read it they are kind of gives you something right some some stats. Yeah, it's a good question

你有没有关于多少人的统计数据，

02:11

How many people have read it? I'm actually not sure

比如读过它的人？这些数据会给你一些概念，对吧？一些统计数据。

02:14

I know that I mean we do keep track of the number of citations which I know is going up

是的，这是个好问题。

02:20

Quite fast

有多少人读过？我其实不太确定。

02:21

But

我知道，我是说我们确实会跟踪引用次数，我知道这个数字在上升。

02:26

It made a bigger

它引起了更大的

02:28

Hit and it was actually a pretty instrumental paper

轰动，而且它其实是一篇相当有影响力的论文。

02:31

It was like one that got cited so many times. It is frequently like when people ask what is the best paper of the year

它就像那种被引用超多次的论文，经常是别人问年度最佳论文时会提到的。

02:37

Like basically you read recently. It's this one. I thought the the memory component was pretty underrated

基本上你最近读到的就是这篇。我觉得它的 memory component 被低估了。

02:42

You know like very good early memory system, but yeah, well one of the biggest papers

你知道，非常好的早期 memory system，不过对，它是最重要的论文之一。

02:48

Yeah, so maybe I can talk a little bit about how this particular paper came together

是啊，所以也许我可以简单聊聊这篇论文是怎么来的。

02:53

So when I got into research it was back in 2020 when I started my PhD program at Stanford

所以我开始做研究是在2020年，那时我刚在 Stanford 开始我的 PhD 项目。

02:59

And that was the year when we were about to get GPD 3.4 I and GPD 3 to be available

而那一年，我们即将迎来 GPD 3.4 I 和 GPD 3 的问世。

03:05

So we already had GPD 2 and you could sense that there's this new class of models that was just becoming available in the market

所以当时我们已经有 GPD 2，而且你能感觉到有一类新模型刚刚开始在市场上出现。

03:13

And the thing got very intrigued and the general consensus was what is this model actually going to be useful for anything

而且这件事让人非常好奇，大家的共识是：这个模型到底能有什么用？

03:21

It's really strange that these models are not trying to do any particular task

真的很奇怪，这些模型并不试图去完成任何特定的任务。

03:25

But we decided to take a bath. So a large group of scholars at Stanford

但我们决定赌一把。于是斯坦福的一大群学者，

03:31

And it was actually led by one of my co-founders personally on came together

实际上是由我的一位联合创始人亲自带头，大家聚到了一起。

03:35

Proquined foundation models who coined the term foundation model

提出了 foundation model，也创造了“foundation model”这个术语。

03:37

We wrote this paper where that term came from called opportunities and risks of foundation model

我们写了那篇论文，也就是这个术语的出处，标题是《Opportunities and Risks of Foundation Model》。

03:43

And during that process really the thing that I started to think deeply about was here is a model that is fundamentally new in our ecosystem

而在那个过程中，我真正开始深入思考的是：这是一个在我们的生态系统中根本就是全新的模型。

03:52

The reason why this was new was it wasn't again trained to do anything in particular

这件事之所以新鲜，是因为我并没有受过什么特定训练。

03:57

But it was its premise was it could do anything and everything was like a stem cell if you were to take a biology analogy

但它的前提是它什么都能做，就像 stem cell 一样，如果用生物学来类比的话。

04:04

And I got really interested in this idea that well if we were to really think about what are the killer applications that this particular technology would enable what would that be

我对这个想法很感兴趣：如果我们真的去思考，这项特定技术会催生出哪些 killer applications，那会是什么？

04:14

Many of my colleagues were using this for a simple classification, simple generations

我的很多同事都用它来做简单的 classification、简单的 generation。

04:18

Interesting that these models can do that but from an intractive perspective not that interesting

这些 models 能做这些确实有意思，但从 interactive 的角度来看，就没那么有趣了。

04:23

We've known how to do that for many decades

我们几十年前就知道怎么做了。

04:25

And what we came down to was these models are actually trained on this very broad data from the web

最后我们得出的结论是，这些 models 实际上是在来自 web 的非常广泛的 data 上训练的。

04:30

Right, so these are human behavior data. It's the it's social media Wikipedia all these kind of data

对，所以这些是 human behavior data，就是 social media、Wikipedia 这些 data。

04:36

So if you poke at the right angle

所以如果你从正确的角度去戳它。

04:39

Then you could see human behavior that would just pop out that's actually quite realistic and we've never seen that before

然后你会看到一些人类行为突然冒出来，实际上非常逼真，是我们前所未见的。

04:45

So that that is really interested the exercise that we decided to do and this is something that we

所以，我们决定做的那个练习真的很有意思，而这是我们...

04:49

And this particular group of colleagues that I have

而我有这样一群特别的同事。

04:52

Myself micro Bernstein personally young who ended up becoming a my co-founder. Similarly

我自己 micro Bernstein，还有 Young，他后来成了我的联合创始人。同样地，

04:58

We sat down and we played this game that we call the time machine game

我们坐下来，玩了一个我们称之为 time machine game 的游戏。

05:03

Imagine we were to get on a time machine and fast forward 10 years and look back

想象一下，我们坐上时间机器，快进十年，然后回头看看。

05:10

What would have been the single application that will have matter that would be the most interesting and inspiring

那个唯一重要、最有趣也最激励人心的应用会是什么？

05:16

And what we thought well, what if we can just recreate the world that we live in I mean

然后我们想，嗯，如果我们能重新创造我们所生活的这个世界呢？我是说，

05:19

It's really hard to get more ambitious and that let's just create a world

真的很难再有更宏大的想法了，就是“咱们直接创造一个世界吧”。

05:24

And that's where we started and

而那就是我们的起点。

05:26

Initially we had this paper that was a precursor to the gender of the agents paper called social similar

最开始我们有一篇论文，是“gender of the agents”那篇的前身，叫作“social similar”。

05:32

Before you go further. This was there like a other candidates for the most ambitious thing in the time time machine

在你继续之前——这算是 time machine 里最宏大事情的候选之一。

05:38

Exercise exercise. Yeah, what I'm just yeah, what what what were the next, you know, it was number three number three

练习，练习。对，我刚才是说……对，接下来是什么？你知道吗，是 number three，number three。

05:43

Okay, if you remember so there is a closed second that we were considering which basically ended up becoming more of these

好吧，如果你记得的话，有一个非常接近的第二名，我们当时也在考虑，它基本上最后变成了更多这些……

05:51

Automation tools, but especially the the vision around really personalized agents. They would actually do things for you

Automation tools，但尤其是那个关于真正的 personalized agents 的愿景——它们会真的为你做事。

05:59

And that's also happening. It's also happening

而这也正在发生。确实正在发生。

06:01

But it was sort of interesting for us right in that the reason why we decided to go with the idea of simulation one

但对我们来说还挺有意思的一点是，我们之所以决定采用 simulation one 这个想法，是因为

06:09

I mean, I was a huge science you know science fiction nerd

我是说，我是个超级科幻迷，你懂的。

06:13

And this idea of creating simulation. I was personally really just fascinated. I love the idea

而这个创造 simulation 的想法，我个人真的非常着迷。我很喜欢这个想法。

06:19

It's it's really cool to see like a game time like this and just see these agents live in it

看到像这样的游戏，看到这些 agents 在里面生活，真的非常酷。

06:24

But at the same time my bad was if you were to create a really amazing personal assistant out of this technology

但与此同时，我的想法是，如果你想用这项技术打造一个真正出色的 personal assistant，

06:33

What you actually need first is an amazing motor of your users

你实际上首先需要的，是一个关于用户的惊人的 model。

06:37

So for instance, I taught a model. Hey, can you go by

比如说，我教一个 model：“嘿，你能去帮我买

06:41

They dinner for me and it orders how I am pizza and I do not like pineapple on my pizza

晚餐吗？它订了一个夏威夷披萨，但我不喜欢披萨上有菠萝。”

06:46

Then it totally failed. The way for it to not make the mistake is only by having a deep understanding of why I am and I gave a very simple and

然后它完全失败了。要让它不犯错误，唯一的方法就是深刻理解我为什么是我，而我给了一个非常简单的...

06:54

In a dumb example here, but you can imagine how this core understanding of people is instrumental

在这个很傻的例子中，但你可以想象，这种对人的核心理解是多么关键。

07:00

This is how for instance if we have our family closes friend

这就是为什么，比如，如果我们有家人最亲密的朋友，

07:03

They have a good mental motor of who we are. That's the basis of our social connection

他们对“我们是谁”有一个很好的 mental model。这是我们社会联系的基础。

07:08

So our bad also was this technology around simulation creating accurate representation of people

所以我们的错误也在于这种围绕 simulation 的技术，它创建了对人的准确 representation。

07:17

Out to proceed

于是我们继续推进。

07:19

The more complex agents that would automate the world that we live in

那些更复杂的 agents 将自动化我们所生活的世界。

07:23

So that was the bad

所以那就是糟糕之处。

07:24

So for so but that was a very close second and I'm still very much fascinated by it

所以，但那是一个非常接近的第二名，而且我仍然对它非常着迷。

07:28

I think there's a lot of interesting work that's going around

我觉得现在有很多有趣的工作在进行中。

07:31

My hot take actually here though is I don't think we've actually seen a true personal assistant

不过，说真的，我的爆论是，我认为我们实际上还没有见过一个真正的个人助理。

07:36

That's actually useful

这实际上是很有用的。

07:38

In ways that actually meets the ambition of that particular lineup work

以真正符合那个特定系列工作雄心的方式。

07:42

I think there are early applications that are obviously interesting and if you talk to even chat you pity nowadays or claw

我认为有些早期应用显然很有趣，而且哪怕只是和 chat you pity 或者 claw 聊一聊。

07:49

They obviously know a lot about us

他们显然非常了解我们。

07:51

So a lot of the generation that's doing I do think it's a much more tailored

所以很多正在做这件事的新一代，我确实认为它更加定制化。

07:55

But I think the ambition is quite large in that field and I don't think we quite have the all the right ingredients just yet

但我认为这个领域的野心很大，而且我觉得我们还没完全具备合适的要素。

08:01

So like open claw all these

所以就像 open claw 所有这些——

08:03

Climes of the first origins like what do you want to see from them that you're that they don't currently have they do think it's slowly getting there

Climes 的最初起源，你想从它们身上看到什么它们目前还没有的东西？他们确实觉得正在慢慢接近。

08:11

But I do generally want them to have much deeper understanding of the person

但我确实普遍希望它们对人有更深入的理解。

08:16

Right now you look at the models I mean open claw and what's it's basically leveraging is basically mark down file

现在你看这些模型，我说的是 open claw，它基本上利用的就是 markdown 文件。

08:23

And I think it's quite clever right so if you look at the gender of the agents paper

而且我觉得这挺聪明的，对吧？所以如果你看 Generative Agents 那篇论文。

08:26

It is actually was the same intuition that we had

它实际上和我们当时的直觉是一样的。

08:29

Where initially when we were creating the memory architecture for the gen of the pageants and like this is like back in

当初我们为 Generative Agents 创建记忆架构的时候，就好像是在……

08:34

222 so we didn't really quite have the idea of even agent of architecture were the term agent

222，所以我们当时甚至还没有 agent 架构的想法，或者说 agent 这个词。但我们和其他今天发布的工作共有的直觉是，我们一开始想，嗯，我们是要把记忆做成，比如说，knowledge graph 吗？还是要训练一个 bespoke model？所有这些事情。然后我们决定：不不不，忘掉这些。这些 LLM 其实很擅长建模文本、理解文本和推理文本。所以就把所有东西都放进 markdown 文件里，文本就完事了。我觉得我们能这么做挺有意思的，而且这样做有很多优势。

08:41

But the intuition that we shared with other work that's coming out today

但我们的直觉，和其他今天发布的工作一样，

08:45

Was we initially thought well do we want to make the memory

就是我们一开始想，嗯，我们要不要把 memory 做成，

08:49

Into let's say knowledge graph

比如说 knowledge graph，

08:52

Do we want to train a bespoke model all these kind of things and what we decided to do was no no no

要不要训练一个定制的 model，所有这些之类的事，而我们决定的，就是不不不。

08:58

Just forget about all this these links models are actually quite good at modeling

忘掉这一切，这些 model 其实非常擅长 modeling

09:03

Text and understanding and reasoning about text so just put everything in the markdown file where text while you're done

对文本的理解和 reasoning，所以只要把所有东西放到 markdown file 里，写文本，就搞定了。

09:10

I thought that was quite interesting that we could do that and there's a lot of strength in doing that

我觉得这很有意思，我们能做到这一点，而且这样做有很多优势。

09:14

But also there is limitation

但是呢，也有局限性。

09:16

It's the way you retrieve and make sense of data that's extremely large. It takes a lot of work

就是你去 retrieve 和理解极其庞大的 data 的那种方式，这很费功夫。

09:23

So I think that technology is getting better. I also do however think and there are certain things

所以我觉得技术在变得更好，不过我也确实认为，有些事情

09:27

You just cannot shape just like prompting the model

你没法仅仅通过 prompting model 来改变它。

09:30

So some to some degree you do need to touch the parameters of the model itself

所以某种程度上，你确实需要去调整 model 本身的 parameters。

09:35

So there's these kind of work that I do think does need to happen and obviously it is happening

所以我觉得这类工作确实需要去做，而且显然也正在做。

09:40

The question is how far can we take it? How do we source data and how do you also create an ecosystem where the people are

问题是我们能把它推到多远？我们怎么获取 data，又怎么创建一个 ecosystem，让里面的人一直

09:46

Continuously feeding data to this model so it's learning about you

持续地给这个 model 喂 data，让它学习关于你的事情。

09:50

What's the intuition between what you need to do in the model?

你在model里面需要做什么，这个intuition是什么？我背后的intuition其实是，什么时候该train甚至post-trained这个model，而不仅仅是prompt它——就是如果model必须学习它所在世界的底层physics，那它就得学新的social physics；而那些它不需要train的地方，就是它已经有了我们信任的physics，它已经有了base statistics，但只是试图react to an environment。然后我觉得你就完全可以靠prompt得到你想要的，you know，actions。

09:53

My intuition behind that actually when do you train or even post-trained the model versus just prompt the model is if

我背后的直觉是，什么时候该训练，甚至 post-train 这个 model，而不是仅仅 prompt 这个 model，那就是如果

10:00

The model has to learn the underlying physics

模型必须学习 underlying physics，

10:04

Of the world that it's operating in so it has to learn new social physics

也就是它运行所在世界的物理规律，所以它必须学习新的 social physics。

10:09

The places where it doesn't have to train is it already has the physics we trust the physics

那些不需要训练的地方在于，它已经有了这些物理，我们信任这些物理。

10:14

It already has the base statistics

它已经有了 base statistics。

10:17

But it's just trying to react to an environment

但它只是试图对一个环境做出反应。

10:20

And then I think you can just prompt your way into getting the you know actions out of it

然后我觉得你完全可以通过 prompt 来得到你想要的 actions。

10:25

I don't think the model has yet at least the models that are out in the open has yet learned the complete mapping of social physics of humanity

我不认为模型已经——至少目前公开的模型还没有——学会人类 social physics 的完整映射。

10:36

This actually is one of the core thesis of similarly right and one of the core reason why that is the case is if you look at the

这实际上也是 similarly 的核心论点之一，而之所以会这样，其中一个核心原因就是，如果你看看——

10:42

Data that the model was trained on these models were trained on the web data and whatever was available on the web and these are

这些 models 训练所用的数据，来自 web data 以及网上所有能获取到的东西。从数据意义上讲，这些算是遗物，但本质上它们是自我暴露的 editorial data，再加上一些零散的 behavior data。它还没有学会人类真正深层的行为本质——不只是人们嘴上说不在意什么，而是他们在现实生活中实际做了什么。这其实是我认为的人类 dark knowledge 的一部分，至少能让我们一窥用户的行为。如今当人们使用我们的 models 时……

10:50

Relics interesting data sense, but they are fundamentally

Relics有趣的数据感，但它们从根本上

10:54

the self exposed

自我暴露

10:56

Editorial data with some behavior data that sprung around here and there and

编辑数据，还有一些行为数据散落各处，

11:01

It has yet to learn really deep behavioral nature of people not just what people say they don't mind

它还没有学会人们真正深层的行为本质，不仅仅是人们说他们不介意

11:08

But they what they actually doing real life and this is actually one of the sort of what I would consider to be the dark knowledge of humanity

而是他们在现实生活中的实际行为，这实际上是我认为的人类黑暗知识之一

11:15

That we haven't quite captured and it's this kind of data that would also need to get factored into the model creation

我们还没完全捕捉到的是，这类数据同样需要纳入模型创建。

11:21

You call it behavior foundation model. Yeah, there's a good one minor here

你叫它 behavior foundation model。对，这名字不错，不过这里有个小问题。

11:25

But outside of that what type of data do you need what are you changing on the model level? How do you go about

但除此之外，你需要什么样的数据？在模型层面你会改动什么？具体怎么入手？

11:31

Actually bottling you know doing a behavior foundation model

实际上就是——你懂的——做一个 behavior foundation model。

11:35

We think about that in three buckets

我们考虑这个问题时分三个大类。

11:38

So one bucket is actually we

其中一类呢，实际上我们...

11:41

Interview data for instance. It's quite interesting. It qualitative rich qualitative data is interesting

比如说访谈数据。这很有意思。它是定性丰富的——定性数据很有意思。

11:47

It's not behavioral, but we would literally ask people hey tell me the story of your life

它不是行为数据，但我们会直接问人们：“嘿，给我讲讲你的人生故事。”

11:52

Yeah, just what we're doing here exactly

没错，我们在这儿做的就是这件事。

11:54

The question that you all ask at the beginning of this interview literally is the question we also ask

你们在采访一开始问的那个问题，其实也正是我们会问的问题。

12:00

And obviously you know we ask our participants to go a little bit deeper and then how far I went

当然，你也知道，我们让参与者再深入一点，然后看看我有多深入。

12:06

Maybe I can actually keep more of my life story and leave of this

也许我其实可以保留更多自己的人生故事，然后就此打住。

12:09

But the reason why that data is interesting is by learning about this very long-tailed information about people

但那些 data 之所以有意思，是因为能了解到关于人的这种非常 long-tailed 的信息。

12:17

You actually get a lot of texture around this model like this this person is a model

你实际上会得到很多关于这个 model 的层次感，比如说，这个人本身就是一个 model。

12:23

So even understanding their childhood memory or even their trauma their first love

所以，即使了解他们的童年记忆，甚至是他们的创伤、他们的初恋，

12:29

These kind of things quite informative in ways that's really hard to predict

这类东西信息量很大，但具体会体现在哪些方面真的很难预料。

12:33

So that's one

所以这是一种

12:34

Then there are sort of two tranches of what I would consider to be the behavioral data

然后，我所说的 behavioral data 大概有两批

12:39

So one behavior data actually is observational

所以其中一种 behavior data 实际上是 observational 的

12:41

So this might actually be like transaction data or this might be data that you can get by scrapping the web right

所以这可能像是 transaction data，也可能是你通过抓取 web 得到的数据，对吧

12:46

So you can imagine why this data would be these data sets would be interesting right because they give you the base statistics

所以你可以想象为什么这些 data，或者说这些 data sets，会很有趣，对吧，因为它们给你提供了 base statistics

12:53

Of people's behavior

也就是关于人们行为的

12:55

But then there's the last category of data

但是还有最后一类 data

12:58

And then I personally think is perhaps the most important which is

然后我个人认为这可能是最重要的，那就是

13:02

The the data that basically describes the cause and mechanism the wise of people

基本上，就是那些描述人们行为的原因和机制的数据

13:07

And some of this is covered by the interview data the qualitative because people talk about why they made certain decisions

其中一些由qualitative interview data覆盖，因为人们会谈论他们为什么做出某些决定

13:13

But really where you get to see the most behavioral aspect of this actually is in randomize in randomize control trials like RCTs

但真正能看到行为层面最多的，其实是在randomized control trials里，比如RCTs

13:21

Imagine you basically have the same setup

想象一下，你基本上有相同的setup

13:23

But you have a few different variables that you are trying to tweak

但你有几个不同的variables，是你想要去tweak的

13:27

Can you actually get realistic human behavior out of it in ways where oh, imagine you had to make

你能否真的从中得到真实的人类行为——比如，哦，想象你必须做出选择

13:34

Imagine you had this particular

想象你有这样一个特定的

13:36

Option imagine you're even trying to choose whether you're going to drink coffee or not

选项——想象你甚至是在决定要不要喝咖啡

13:40

The day you drink coffee versus the day you didn't drink coffee does your behavior change

你喝咖啡那天和你没喝咖啡那天，行为会改变吗？

13:45

That's a data set that describes the cause or mechanism

那就是一个描述 cause 或 mechanism 的 data set。

13:48

This actually is quite important in actually modeling people

这其实对 modeling 人来说非常重要。

13:51

The reason why this is important is

为什么这很重要呢？

13:53

Often times when people come to us or not just to us

很多时候人们来找我们，或者说不仅仅来找我们。

13:57

But the reason why people are interested in simulation

但人们对 simulation 感兴趣的原因，

14:00

Actually isn't because they want to predict the future

其实并不是因为他们想预测未来。

14:02

If you're when against if you're trying to win against a stock market predict the future is interesting

如果你是与之对抗，如果你想战胜股市，那预测未来就很有意思。

14:08

But most people most decision makers what they want to know is how can we shake the future

但大多数人，大多数决策者，他们想知道的是我们如何能改变未来。

14:15

It doesn't really help you to hear that your sales is going to tank into quarters

光听到你的销售额会在两个季度内暴跌，其实没什么帮助。

14:20

They're just going to say wow that sucks

他们只会说，哇，那太糟了。

14:21

What they want to know is well what do we need to do now to avoid that future

他们想知道的是，我们现在需要做什么来避免那个未来。

14:26

That's cause or mechanism and this is also very hard data to come by right because the world is our ground truth

这就是原因或机制，而且这种数据也很难获得，对吧，因为世界就是我们的 ground truth。

14:32

But it happens once

但世界只有一个。

14:34

So in a very controlled setup where everything is equal except for one variable this kind of data set almost rarely happens

所以在一种非常受控的环境中，除了一个变量之外其他一切都相同，这种 data set 几乎从未出现过。

14:41

So this is the reason why this data sets data set is both hard to come by but also quite important if you're trying to model human behavior

所以这就是为什么这种 data set 既难以获得，又非常重要，特别是如果你试图 model 人类行为。

14:49

So the behavior I think is the hardest

所以我觉得行为是最难的

14:53

Data set to acquire what is out there? What is possible right even because like you're not going to know a lot of details about my life

要获取什么样的数据集？外面有什么？有什么是可能的？对吧，因为比如你根本不会知道我生活里的很多细节

15:02

I don't even have data for myself

我连自己的数据都没有

15:04

On like I want to analyze my own

比如说我想分析我自己的

15:07

Health or habits. Yeah, and I just don't log everything so how can you have that data

健康或者习惯。是啊，而且我并不会记录所有东西，所以你怎么可能有那些数据

15:13

So we actually run a lot of rent ways control trials. Yeah, but you put people in the lab they watch them sleep or what

所以我们确实做了很多 randomized controlled trials。是啊，但你把人们放在实验室里，他们看着他们睡觉还是什么？

15:20

So we do actually care a lot about the consent process so people know that we are like we invite them to be a member of this community

所以我们其实非常在意同意流程，让大家都知道我们像是邀请他们成为这个社区的一员

15:27

To both share their data and also have their selves represented in different forms

既能分享他们的数据，也能以不同的形式呈现他们自己

15:32

um

嗯

15:33

But

但是

15:34

We bring a lot of people to the lab or virtual lab where we design experiments

我们会带很多人到实验室或虚拟实验室，在那里设计实验

15:40

That would actually pose them real behavioral decisions

这些实验实际上会让他们面对真实的 behavioral 决策

15:43

And often in these kind of experimental setup what makes

而通常在这种实验设置中，区分

15:48

The difference between what is additional versus behavioral is if the stake in your decision is real

所谓的“额外的”和 behavioral 的，就是你的决策中的 stake 是不是真实

15:54

That's ultimately what makes a behavioral

那最终才是让它成为 behavioral 的关键

15:57

So in these kind of setups we are inspired by our colleagues and social sciences, psychology and so forth

所以在这些设置中，我们受到同事们以及社会科学、心理学等领域的启发

16:02

So when they run studies what the kind of techniques they

所以当他们做研究时，他们使用的技巧是

16:05

utilize is

想象一下有一个线上商店，你邀请人们来逛

16:07

Imagine there is a online store that you're inviting people to come by

然后在这个实验里，他们买了什么东西，实际上就会收到那个商品，比如

16:11

Then whatever they purchase in this experiment, they actually get that item delivered for instance

就是这类事情让实验变得有真实感，所以我们做了很多这样的实验

16:16

Like these are the kind of things that makes the stakes real so we run a lot of these experiments

而且我们也会和公司合作

16:21

And we also do partner with firms

嗯，而且现在也有一些客户非常兴奋地想要

16:23

Um and also right now we also have customers who are quite excited to

至少让我们瞥见他们用户的行为类型

16:29

At least give us a glimpse of the kind of behaviors that their users

至少让我们得以一窥他们的用户的某种行为

16:34

Accept it so that we can get a little bit deeper in their sending up help people behave in these different platforms

接受这一点，这样我们就能更深入地了解他们如何在这些不同的 platforms 上帮助人们表现。

16:39

I think on the customer side they have a lot of data about their users who has bought

我认为在客户这边，他们有很多关于已购买用户的数据。

16:45

They have the action data

他们有 action data。

16:46

Can you kind of walk us through an example of what does someone come to you for what questions

你能给我们举个例子吗？比如人们带着什么问题来找你们？

16:51

Would they want solved in the process of do you customize a model for them? Do you have something off the shelf?

他们想解决什么问题？在这个过程中，你们会为他们 customize 一个 model 吗？还是有 off-the-shelf 的东西？

16:57

What is all look like

整个过程是什么样的？

16:59

Today when people leverage our models

今天，当人们利用我们的模型时

17:02

It's often to better understand the population of their interest

这通常是为了更好地理解他们感兴趣的人群。

17:05

So usually the start of the relationship we basically come together and hear about what population they want us to model

所以通常在一段合作关系的开始，我们基本上会一起沟通，了解他们想让我们对什么样的人群建模。

17:12

Right

对。

17:13

So it might be that if you're a CPG company that's selling to all of the US

比如，如果你是一家 CPG 公司，面向全美销售产品。

17:16

It might be fairly straightforward. You want to model the gem pop of the US

那可能很简单，你想对美国的总人口建模。

17:20

But at the same time there is a vertical or if there's a market that they're trying to go into him as you know

但与此同时，如果有一个 vertical，或者有一个他们想进入的市场，你知道，

17:25

They want to better understand

他们想更好地了解

17:27

Let's say people in their 20s and 30s living in California that's a much more specific population

比如说，住在加利福尼亚的 20 多岁和 30 多岁的人群，那是一个更特定的群体。

17:33

So we hear about these population and we go recruit these people

所以我们了解到这些人群后，就去招募这些人。

17:38

With consent and with incentives and we basically collect some other data and create a model of these people

在征得同意并给予激励的情况下，我们基本上会收集一些其他数据，然后为这些人创建一个 model。

17:46

Then what our protocol allows you to do is basically query them

然后我们的 protocol 允许你做的，基本上就是 query 他们。

17:49

So it can take us input a filter that is a description of the population that you want to talk to just like the one I just mentioned

所以你可以输入一个 filter，这个 filter 就是你想交流的人群的描述，就像我刚才说的那样。

17:57

And an environment

还有一个 environment。

17:59

Environment can literally be a survey questions. It can be behavior experiments

Environment 说白了可以就是调查问卷，也可以是行为实验。

18:03

It can be a bit testing oftentimes the core use cases or things like concept testing to start with

可以是 A/B testing。通常情况下，初期的核心用例是 concept testing 这类东西。

18:09

But also, you know people sometimes want to do focus group or one of the

但是，你也知道，人们有时候会想做 focus group 之类的。

18:13

So fun use cases that we also serve is actually even modeling things like the earnings call for public companies

所以我们还提供的一个有趣用例，实际上是对上市公司的 earnings call 做 modeling。

18:20

So these are the use cases that we often start with concept testing. Is that a established term?

所以这些是我们经常从 concept testing 开始的使用场景。那是个已确立的术语吗？

18:25

I've never heard of concept testing

我没听说过 concept testing。

18:27

Yeah, so it basically has to deal with they have let's say different messaging different products different

对，它基本上是关于，比如说，不同的 messaging、不同的产品、不同的……

18:32

It's like a marketing exercise. Yeah, okay. Got it. Got it politics

这就像是一种营销活动。嗯，好的，明白了。明白了，politics。

18:36

We do

我们确实

18:37

Have a strategic partnership with Gallup and of course Gallup is deep into policy space and so forth

和 Gallup 有战略伙伴关系，当然 Gallup 在政策领域非常深入，等等。

18:43

Right now we have not worked deeply with politics like that area just yet. However

目前我们还没有在政治这类领域深入合作。不过，

18:49

I'm curious if there is demand or if they really

我很好奇是否有需求，或者他们真的……

18:53

Would have different needs that somehow fundamentally don't mix with your existing

会有一些不同的需求，从根本上说和你们现有的东西不太相容。

18:58

Users or people I think they're starting to demand yeah, but we are very much mindful of how this technology gets adopted in the

用户或者说人群，我觉得他们开始有这样的要求了，对，但我们非常关注这项技术是如何被采用的，尤其是在

19:06

Society impact that we'll end up having with this technology

社会影响方面，我们最终会通过这项技术产生的影响。

19:09

And I do see politics as an area where a company has to be particularly thoughtful

而且我确实认为，政治是一个公司必须特别深思熟虑的领域。

19:15

About the way they operate and make impact

关于他们运营和施加影响的方式。

19:18

So this is where we also want to make sure that we form enough of car rail and

所以这就是我们也要确保自己形成足够的护栏和

19:23

Perspective on how to leverage this technology before we go on to serve markets like the politics

视角，知道如何利用这项技术，然后才会去服务像政治这样的市场。

19:29

Give people an example

给大家举个例子

19:31

One of my favorite shows is the West Wing. I don't know if if people have watched one of those key storylines is like the president has

我最喜欢的剧之一是《The West Wing》。我不知道有没有人看过，其中一个关键剧情线就是总统患有

19:38

Multiple sclerosis, but they haven't they need to figure out how to disclose it

多发性硬化症，但他们还没有……他们需要想办法如何公开这件事

19:41

So they run a poll with a fake governor and ask people to respond on the poll and they try to make decisions based on the results of that poll on like

所以他们用一个虚构的州长进行了一次民意调查，让人们回答，然后他们试图根据民意调查的结果来做决定，比如

19:51

How well they'll be received like where how shall we play this?

大家会怎么接受？我们该怎么处理这件事？

19:54

And I'm like well, you know, I think those kind of counterfactual things

然后我就想，嗯，你知道，我觉得那种 counterfactual 的东西

19:58

I would actually use a simulation for this if I could trust it. I was sure. Yeah, in that show how did go

如果我能信任 simulation 的话，我其实会用它来做这个。我确定。对，在剧里后来怎么样了？

20:04

In that show it basically was like kind of like a foregone conclusion there

在剧里基本上就是……已经是个定局了

20:08

They were like we know it's bad

他们就说，我们知道这很糟糕。

20:09

We just don't know how bad and then the poking back it was like it's really bad and then they just did it anyway

我们就是不知道有多糟，然后反过来戳了一下，发现真的很糟，但他们还是照做了。

20:13

Part of it is it's a show, right? So you're you're

部分原因在于这是一场秀，对吧？所以你在，你就在

20:16

You're at the amazing drama. How that could it be? Oh, it's horrible

你身处这出惊人的戏剧中。怎么会那样？哦，太糟糕了。

20:20

And to some extent I think that is part of the the trick of the

在某种程度上，我觉得这就是那个把戏的一部分——

20:25

Challenge or with being a customer of yours

挑战，或者说作为你的客户的那种情况：

20:29

Which is that if I know it's if I roughly know

也就是说，如果我知道——如果我大致知道

20:33

And can into it what the effect is going to be do I need you?

而且能看出效果会是什么，那我还需要你吗？

20:37

What sensitivity of it of

它的敏感度呢？

20:40

Effect do I need in order to make a decision, right? So for example if I

我需要多大的效果才能做决定，对吧？所以比如如果我

20:45

My approval approval rating is 50% yeah, and I have this negative piece news item comes out and it drops the 30

我的支持率支持率是50%，然后有一条负面新闻出来，降到了30%

20:52

Yeah, if it drops the 20 if it drops the 40 do I care no it I know it drops. It's negative. So

对，如果降到20%，如果降到40%，我在意吗？不，我知道它会降。是负面的。所以

20:58

When do I care about simulations? You do something that's clearly bad. That's not popular and people don't like you like

我什么时候在乎模拟？你做了明显糟糕的事，不受欢迎，人们不喜欢你，比如

21:04

Yeah, I mean

对，我的意思是

21:06

Well, so there are a couple of things one actually obviously is

嗯，所以有几件事，其中一件很明显是

21:12

And there are

还有

21:14

Use cases where like every day for instance developers, designers,

有些用例，比如每天，比如开发者、设计师，

21:19

policymakers

政策制定者

21:20

Marketers every single day they create assets they create new products and turns out it's actually

营销人员每天都在创造资产，创造新产品，结果发现实际上……

21:27

Maybe the decisions in hindsight is sort of obvious. Yes, of course. This is bad

也许事后看来这些决策挺明显的。是的，当然，这很糟糕。

21:32

But

但是

21:33

We still run those studies

我们仍然会做那些研究

21:35

Because understanding the magnitude and understanding how it could something is is actually quite difficult

因为理解其严重程度，以及理解它究竟是如何变成这样的，实际上相当困难。

21:41

Even if we feel like of course like this makes sense

即使我们觉得这当然有道理。

21:44

I mean, this is the reason why we make so many mistakes like every time somebody goes online and say something that has huge backlash

我的意思是，这就是为什么我们会犯这么多错误，就像每次有人上网说点什么，就会引发巨大的反弹。

21:50

You look at that and like what an idiot. However, it's tough

你看着那个，就会觉得真是个白痴。不过，这很难。

21:55

That's one there's also another aspect here, which is again, this is a reason why simulation is actually different from prediction

这是一个方面，这里还有另一个方面，也就是再次说明，为什么 simulation 实际上不同于 prediction。

22:01

In simulation

在 simulation 中。

22:03

In the ideal case scenario is so what simulation is trying to show is it's trying to show

在理想情况下，simulation 想要展示的是，它试图展示。

22:08

Each step of the way or each step that we need to take to get to a certain outcome

过程中的每一步，或者我们需要采取的每一步，以达到某个特定的结果。

22:13

Right

对。

22:14

So in the most advanced simulations

所以在最先进的 simulation 中。

22:18

Sometimes

有时候。

22:19

The next step that we're suggesting might actually be quite counterintuitive

我们建议的下一步可能实际上相当反直觉。

22:23

The analogy that I sometimes give and I grounded in a more realistic example

我有时会给出一个类比，而且这个类比基于一个更现实的例子。

22:27

But you know as I mentioned I'm a huge friend of science fiction

但你知道，正如我提到的，我是科幻小说的超级粉丝。

22:31

And I don't know how many of the audience members have read like things like the foundation series really has small

而且我不知道观众中有多少人读过像Foundation系列这样的东西，它真的有很小的...

22:37

We've mentioned psycho history number terms

我们提到了psychohistory的一些术语。

22:39

Okay fantastic

好的，太棒了。

22:40

So I might actually be talking to the to the right crew

所以我可能真的在对合适的听众说话。

22:43

If you read foundation series literally the first act is there's a group of scientists who have found out that

如果你读过Foundation系列，第一幕就是有一群科学家发现了...

22:50

Oh our galactic empire is going to collapse and we're going to have 30,000 years of unrest

哦，我们的 Galactic Empire 要崩塌了，而且我们会有三万年的动荡不安

22:56

And they basically run psycho history the simulator that tries to

他们基本上跑一个 psycho history 的 simulator，试图

23:00

Teach them okay, how can we keep this unrest two thousand years

教导他们，好吧，我们怎么才能把这动荡控制在两千年之内

23:04

And they plan this out and the first step of the plan is to get the scientists to say okay, this is coming

然后他们计划好了一切，计划的第一步就是让科学家们说，好吧，这要来了

23:13

Exiled into this random place and this you know galaxy terminus exactly

然后被流放到某个随机的地方，就是那个，你懂的，Galaxy Terminus，准确来说

23:20

And that's so kind of intuitive like what a strange move that you literally sent

这还挺直观的，就像——多么奇怪的一步棋，你竟然真的把

23:26

The group of scientists who was raising voice around the potential collapse of galactic empire into nowhere

那群对 Galactic Empire 可能崩溃而大声疾呼的科学家们，送到了一个虚无之地

23:33

How is that the right first move what turns out in this particular simulation that actually was the move

这怎么会是正确的第一步？结果在这个特定的 simulation 里，这还真是那一步棋

23:39

It's these kind of things right and the reason why this kind of reasoning is possible

就是这些类似的东西，对吧，而且这种推理之所以成为可能

23:43

Is because you're showing the step function or each step that results in a particular outcome

是因为你展示了 step function，或者说产生特定结果的每一步

23:50

So really what simulation allows you to do in its highest form is you give it not a problem or question

所以 simulation 在最高形式上真正让你做的是，你给它的不是一个问题或疑问

23:56

What would people answer to the survey that's not what we do what we tell it is

人们会怎么回答调查——那不是我们做的事情，我们告诉它的是

24:00

Here is a goal that we have in the context of foundation

在 Foundation 的背景下，我们有这样一个目标

24:04

We want to keep the unrest to a thousand years

我们想把动荡限制在一千年之内

24:07

What is the path that we need to take now to get to that particular future

我们现在需要走哪条路，才能到达那个特定的未来

24:12

That's what simulation allows you to do now translating that into real market

这就是 simulation 让你能做的事情，现在把它转化到真实市场

24:16

Imagine you're a automobile company and you're about to release a

想象你是一家汽车公司，你即将发布一款

24:22

Evie and you're trying to understand well, how do you market Evie?

Evie，然后你想搞清楚，嗯，怎么给 Evie 做营销？

24:26

To make sure that our stock price goes up

以确保我们的股价上涨

24:30

But what if the answer comes down that well you can market your Evie in xyz way

但万一得出的答案是，嗯，你可以用 xyz 方式营销你的 Evie

24:36

But that might change people's perception around the cars that's not Evie and actually make your overall sales to go down

但这可能会改变人们对非 Evie 车型的看法，实际上让你的整体销量下降

24:43

Not very intuitive especially all you're trying to optimize is Evie's sale and that's the only thing you're

这不是很直观，尤其是你试图优化的只是 Evie 的销量，而且那是你唯一正在

24:48

Tracking then that might actually result in a completely wrong solution or at least different solution than what you would have expected

追踪的，那么这实际上可能会导致一个完全错误的解决方案，或者至少是一个与你预期不同的解决方案

24:55

Whether it's right or wrong. Yeah, that's the power simulation for listeners

无论对错。是的，这就是模拟的力量，各位听众。

24:59

We covered a similar topic with Mikal Parakin from Shopify where they are working in simgim

我们和Shopify的Mikal Parakin聊过一个类似的话题，他们正在做simgim相关的工作。

25:04

I don't know if you ever talked to you about it. It's very similar

不知道我们有没有聊过这个，非常相似。

25:07

The goal is increase conversion, but then the the journey is very unusual journey is unusual

目标是提高conversion，但这个旅程非常不寻常。

25:12

Yeah, he's actually trying to look for interventions on a shopping trajectory

对，他实际上是在shopping trajectory上寻找interventions。

25:17

Which is similar to what you say like it's not about the

这跟你说的很相似，就是不是关于...

25:21

Attitude and always your your your word for it. Yeah, it's about behavior

态度，而且总是——你用的那个词怎么说来着。对，是关于行为。

25:25

It's about being and that's exactly the difference right. It's like not about the

是关于存在，这正是区别所在，对吧。就像不是关于...

25:28

Near-term direction about but it's more about like how do you affect multiple turns of interactions right?

短期方向，而是更多关于如何影响多轮的interactions，对吧？

25:35

You're the good quote at the start about this as well

你开头关于这个也有一句很好的引用。

25:37

It's not about people wanting to know the outcome. It's about how they can change it change the way to get there something other

重点不是人们想知道结果。重点是他们能怎么改变它，改变到达那里的方式，或者别的什么。

25:42

But I want to take it back to how do we know this is grounded? Like yes, how do you run evels? How do you test that simulations come through

但我想把话题拉回来：我们怎么知道这是 grounded 的？对，你怎么跑 evals？你怎么测试 simulations 真的能成立？

25:50

Basically if I was to do the same thing that you described with say your favorite lm opus

基本上，如果我要做你描述的那种事情，比如说用你最喜欢的 LM Opus，

25:57

GPT-56 have some agent to map out these things

或者用 GPT-56，让某个 agent 去梳理这些事情。

26:01

Yeah, how different are the answers we would get if I give it the same goal the same objective make a decent system

是啊，如果我给它同样的目标、同样的目的，让它做一个像样的系统，我们得到的答案会有多不一样？

26:08

You're saying that you need to change the model weight you have your own solution to this

你是说你需要改变 model weights，你对此有自己的解决方案。

26:11

But how far off are we and how do you check if it's grounded?

但我们现在还差多远？你又怎么检查它是不是 grounded 的？

26:15

You have some interesting stuff on your site that actually points to how you run evels

你网站上有一些有趣的内容，实际上指出了你是怎么跑 evals 的。

26:19

But if you could take us through that side, you know, I think that's one of the big concerns that people have they're like

但如果你能带我们了解一下那方面，你知道，我觉得那是人们的一大担忧，他们会说

26:25

LLM solution 8 you're just solutionating layer after layer right

LLM 解决方案，你就是在逐层加解决方案，对吧？

26:30

The way we do this and this is actually the paper that we worked on after the gender of the agents paper

我们做这个的方式——这其实是我们在 gender of the agents 论文之后做的那篇论文。

26:35

That really became the at least for similarly and also the field of simulation and synthetic panels

那真的成为了，至少对于类似的东西以及 simulation 和 synthetic panels 领域，

26:40

We really became the foundation

我们真的成了基础。

26:42

Yeah, this is the paper the papers called gender observation simulations of thousand people

对，就是这篇论文，叫做 Gender Observation Simulations of Thousand People。

26:46

Here's what we've done for this paper

这就是我们在这篇论文里做的事情。

26:49

We actually brought

我们其实带来了

26:51

Thousand people that's representatively sampled from the u.s to a virtual app

一千名从美国代表性抽样的人到一个虚拟应用

26:56

And what we basically have done was we spent two hours collecting fairly wide ranging data in this particular study

而我们基本上做的就是，在这个特定研究中，我们花了两个小时收集相当广泛的数据

27:02

We focused a lot on this interview data

我们非常关注这些访谈数据

27:05

That was who script was taken from this project called American Voices project

那是从名为 American Voices project 的项目中取得的脚本

27:09

And then we would also pair that with a lot of behavior data and so forth whatever we can collect within two hours

然后我们也会把它和大量行为数据，以及两个小时内我们能收集到的各种数据结合起来

27:15

And then we would actually send these people away for a couple of weeks

然后我们实际上会让这些人离开几个星期

27:18

And during that time I would use this data to create their digital twins

在那段时间里，我会用这些数据来创建他们的 digital twins

27:23

And I would bring the humans participants back after two weeks and have them complete a battery of surveys experiments behavior studies

然后我会在两周后把人类参与者带回来，让他们完成一整套的问卷调查、实验和行为研究。

27:33

So we actually have the list here

所以我们这里确实有这份清单。

27:35

Which basically include the things like the behavior economic games

基本上包括像behavior economic games这类东西。

27:38

You would run literally like the five personality test

你会跑像five personality test这样的测试。

27:41

General source of survey

还有General source of survey。

27:43

We would also go ahead and run the randomness control trials that were published on PNAS

我们也会继续运行发表在PNAS上的randomness control trials。

27:49

And we would have their digital twins predict

然后我们会让他们的digital twins去预测。

27:53

How the source individuals would have

这些source individuals原本会怎样。

27:55

acted in these studies and surveys

在这些研究和调查中的行为表现

27:57

And this is where we basically could replicate people's behaviors and attitudes

而这就是我们基本上能够重现人们行为和态度的地方

28:01

85% as accurately as people would replicate their own

准确度能达到人们自我复制的85%

28:05

So that actually was the first really paper that gave this

所以那确实是最早真正提供这个的论文

28:09

validated results that we can actually model individuals in an accurate way

验证了我们确实能够准确地对个体进行建模

28:14

And what we ended up finding now of course

当然，我们现在最终发现的是

28:17

In AI space so this paper came out at the end of 2024

在AI领域，所以这篇论文是2024年底出现的

28:21

AI space a year and a half two years. That's a lifetime

AI领域，一年半两年，那就是一辈子

28:25

Yeah, I just uh for listeners who are not seeing the YouTube

对，我刚刚，呃，给那些没在看YouTube的听众

28:27

I just want to say like the headline figure is 85% accuracy

我只想说，那个关键数字是85% accuracy

28:31

Like which is a big improvement over all the other

就是，这比其他所有方法

28:34

Yeah, methods that you showed

对，你展示的那些方法，提升很大

28:36

But the part that was actually particularly striking to us

但真正让我们特别印象深刻的部分

28:39

Especially as we

尤其是当我们

28:41

Improved this technology even further

进一步改进这项技术时

28:44

Was the gender of the agents model

是agents model的性别

28:46

A gender of the AI model is like change a PT cloud that's coming out

一个通用 AI 模型就像是即将出现的 PT cloud

28:49

It does give you the right foundation

它确实给了你正确的基础

28:51

However

不过

28:53

What they do not consider is

他们没有考虑到的是

28:56

the

这个

28:57

True additional and behavioral aspect of people

人们真正的额外和行为方面

29:00

Especially in the population that you care about

尤其是在你关心的群体中

29:03

So what these models are really really good at today

所以这些模型如今真正真正擅长的是

29:05

Is they're trying to basically become the super rational objective machines right

他们就是想基本上变成那种超级理性客观的机器，对吧？

29:09

So you go get their data from places like my core scale

所以你去像 my core scale 这样的地方获取他们的 data

29:14

You talk to professional programmers, scientists

你和专业的程序员、科学家交流

29:18

To create model that's amazing at reasoning

来创建在 reasoning 方面非常出色的 model

29:21

That's what they do

他们就是这么做的

29:22

Similarly actually doesn't care about any of this

同样，实际上并不关心这些

29:24

The models that we're talking about here

我们在这里讨论的这些 models

29:25

What we're trying to create are models that are as dumb as I am

我们试图创建的是和我一样笨的 models

29:29

Right so if I make some mistakes

对，所以如果我犯了一些错误

29:31

The model has to make the same kind of mistake

model 也必须犯同样的错误

29:34

Oh, that's very hard

哦，那很难

29:35

That's very hard

那非常难

29:36

You're solving more of expert docs

你更多是在解决 expert docs

29:37

That's exactly

正是如此

29:38

And this is actually completely different kind of data and training objective

而且这实际上是完全不同的 data 和 training objective

29:42

And this is also where we actually see quite a bit of discrepancy in the performance

而这也是我们实际上看到 performance 有不少差异的地方

29:47

In human behavior prediction between the frontier models and similarly model

在人类行为预测中，frontier models和类似模型之间

29:51

And the models that are creating getting created in the space

以及在该领域中正在被创建的模型

29:54

Where in some cases the model performance of frontier models

在某些情况下，frontier models的model performance

29:57

Go all the way down to 20-30%

会一路降到20-30%

30:00

Especially if you go into that more niche population

特别是如果你进入更小众的人群

30:02

On topics that are customers who actually care about

关于客户真正关心的主题

30:05

On more gem pop it might be around 50-60%

在更广泛的人群上可能大约50-60%

30:08

So it's not very robust

所以它并不太robust

30:10

Like you wouldn't want to make your decision of off of these kind of more and these kind of findings

就像你不会想根据这类研究发现来做决定

30:14

If you can bring that up to 85%

如果你能把它提高到85%

30:16

That is ultimately what people end up getting very excited about

这最终就是人们感到非常兴奋的地方

30:19

Yeah

对

30:20

Do we want to keep going on the paper routes

我们要继续沿着论文这条路走吗？

30:23

Yeah, for sure

是的，当然

30:24

So the last one was sort of an interesting one

所以最后一个挺有意思的

30:26

So this paper was the follow-up paper that we had to the 1000 agents paper

所以这篇论文是我们在那篇1000 agents论文之后的后续

30:32

Where basically the idea was now

基本上当时的想法是，现在

30:36

Can we augment the models even further

我们能不能进一步增强模型？

30:39

And actually post-trained the model based on a lot of range control trials

而且实际上基于大量的 range control 试验对模型做了 post-training

30:42

So this was an interesting one

这个挺有意思的

30:44

The data is always the most interesting part of modeling in many ways

在很多方面，数据永远是建模中最有趣的部分

30:47

The data that we got here was there's this

我们得到的数据是这样的，有一个

30:50

And there's this platform called Open Science Foundation

有一个叫做 Open Science Foundation 的平台

30:54

So some the audience might be familiar with this

所以有些听众可能对这个比较熟悉

30:57

And there has been especially in the social sciences

尤其是在社会科学领域

31:00

Over the past five years or so

在过去五年左右

31:02

There has been this concern around replicability of studies

一直有这种关于研究 replicability 的担忧

31:06

And so it was a bit about the crisis

所以这有点像是危机

31:07

The scientists acknowledged where we rerun the study

科学家们承认，当我们重新运行研究

31:10

And we don't actually see the same findings

我们实际上看不到相同的结果

31:12

It's tough

这很棘手

31:14

And the reason why it's dealt with those often the case

而之所以要处理它，是因为那些情况经常如此

31:17

Was there's basically the survival bias

基本上就是 survival bias。

31:20

Where the papers that get published

发表出来的论文，

31:23

Often need to maintain what we call the p-value of lessons 0.05

通常需要维持我们所说的 p-value 小于 0.05。

31:27

In the experiments that we ran

在我们跑的实验里，

31:29

That basically suggests that only there's only five percent chance

这基本上意味着只有 5% 的几率，

31:33

That the results that we saw is false positive

我们看到的结果是 false positive。

31:36

But the tricky part was all the papers that were not published

但麻烦的是，所有那些没发表的论文，

31:39

And there's still five percent chance

仍然有 5% 的几率。

31:41

That whatever we publish is actually totally just randomly generated

我们发表的任何东西实际上都完全是随机生成的

31:45

Like there's like five percent chance that hey

就像，有大概百分之五的几率，嘿

31:47

This effect is not real

这个效应并不是真的

31:48

But it just happened to be real because of the sampling bias

但就是由于sampling bias，它恰好成了真的

31:52

So because of that what scientists started to do

所以正因为这样，科学家们开始做的事

31:54

Was they started to pre-register their studies

就是他们开始pre-register自己的研究

31:57

So before running an experiment

所以在进行实验之前

31:59

They would go to this platform and say

他们会去这个平台说

32:01

Here is the data

这是data

32:03

Here is the population that we're collecting

这是我们在收集的population

32:04

And here's the hypotheses

然后这是hypotheses

32:07

And they would just say here is our hypothesis

然后他们只会说这是我们的hypothesis

32:09

Like this is what we believe

就好比说，这就是我们相信的

32:11

And you cannot retroactively change those hypotheses

而且你不能事后改变这些hypotheses

32:15

This is what actually gives us more scientific statistical confidence

这才是真正给我们带来更多科学上的statistical confidence

32:19

That whatever effect that you ended up seeing is actually true

也就是说，你最终看到的effect实际上是真实的

32:22

So that ended up creating this really interesting platform

所以这最终创造了一个非常有趣的平台

32:24

Where there's one platform that has now

这个平台现在已经

32:27

Contains tens of thousands of real-world experiments

包含了数万个真实世界的实验

32:31

And hypotheses

以及各种假设

32:32

And a lot of these are actually really high quality

而且其中很多确实质量非常高

32:35

Like professionally designed behavior studies

比如专业设计的行为研究

32:37

And randomized control trials

还有 randomized control trials

32:39

So we actually got the data

所以我们确实拿到了数据

32:42

And the studies from this platform

然后这个平台上的研究

32:45

And basically used that to make a point

基本上就是用那个来证明一个观点

32:46

And obviously this particular model is not

然后很明显这个特定的 model 并不是

32:49

Something that we're serving commercially

我们商业化提供服务的东西

32:51

Because this obviously was a part of the open science

因为很明显这是 open science 的一部分

32:54

But this particular data set

但是这个特定的 data set

32:56

And helped us make a point

然后帮我们说明了一个观点

32:58

That by collecting a lot of these randomized control trials

就是通过收集大量这样的 randomized control trials

33:01

And that are really well designed

而且那些真的设计得很好。

33:03

We can make significant improvement

我们可以做出显著的改进。

33:05

In models capability to predict human behaviors

在 model 预测人类行为的能力方面。

33:08

So that's what this paper was about

所以这就是那篇论文的内容。

33:09

Is this stuff done on an individual level

这种工作是在个体层面上做的吗？

33:12

Like we need to tune the model

比如我们需要 tune model。

33:14

For individual, for companies

针对个人，针对公司。

33:15

Our foundation model changes

我们的 foundation model 会改变。

33:18

And then some slight post-training

然后还有一些轻微的 post-training。

33:19

Anything you can share there

这方面有什么可以分享的吗？

33:21

So this particular model actually was trained

所以这个特定的 model 实际上是经过训练的。

33:23

The data we actually had at the level of individuals

我们实际拥有的数据是在个体层面的。

33:26

But this particular model actually was trained

但是这个特定的 model 实际上是经过训练的。

33:29

We experimented with both

我们两种都试过。

33:30

And this is actually what we end up doing as similarly to

而这实际上就是我们的最终做法，类似于……

33:32

We always train two distinct models

我们总是训练两个不同的 models。

33:35

One is what we call the population level model

一个是所谓的population level model。

33:38

The other is what we call the individual level model

另一个是我们所说的individual level model。

33:41

And both actually take very similar input

实际上两者接收的input非常相似。

33:43

Which is the description of a self-population or individual

也就是对self-population或individual的描述。

33:47

And a stimuli

以及一个stimuli。

33:49

In this particular work we've done the same

在这项特定的工作中我们也做了同样的事。

33:52

Here the results that we are reporting are much more geared towards individuals

这里我们报告的结果更偏向于individuals。

33:56

Because we do actually think that is harder task in many ways

因为我们确实认为这在很多方面是更难的任务。

34:00

But that's what we have done

但我们已经这么做了。

34:02

You've seen anything on the questions

你在这些问题上见过什么吗？

34:04

That humans can solve that models can't solve

人类能解决而 models 解决不了的问题。

34:07

So like the

所以就像

34:09

Currently you know I live five minutes walk away from a car wash

目前你知道，我住的地方离一家洗车店走路五分钟。

34:13

It's a ten minute drive, should I walk or drive

开车要十分钟，我该走路还是开车？

34:16

The model will say, oh, walk to the car wash

The model 会说，哦，走路去洗车店。

34:18

You know, you don't have your car

你知道，你的车不在你身边。

34:21

Is anything like this a problem in simulation

类似这种情况在 simulation 里是个问题吗？

34:23

You would assume like very simple for human to think about

你会觉得这对人类来说很简单，想想就明白。

34:27

But if the model is saying you should walk to the car wash

但如果 model 说你应该走去洗车店，

34:29

You know, anything here

你知道，就是这些东西。

34:32

It's less what can we solve

与其说我们能解决什么，

34:34

But I think it's more about what biases

我觉得更关乎的是哪些 bias，

34:37

Or mistakes do people make

或者人们犯了哪些错误，

34:39

The models miss

而 model 会忽略掉。

34:41

Like for instance

比如说啊

34:42

Imagine that you are

想象一下你正在

34:45

You know, like when I was still a Stanford

你知道，就像我还在斯坦福的时候

34:48

I lived in Palo Alto

我住在Palo Alto

34:49

So it's about, I would say, 40 minute walk from the campus

所以大概是，我觉得，从校园走过去40分钟

34:53

You ask the model, okay, let's go home

你问model，好，我们回家吧

34:55

What can I do?

我能做什么？

34:55

It would likely call a nooper

它很可能会叫一个nooper

34:57

Or, you know, give me the bus time

或者，你知道的，直接给我公交时间就行

35:00

But for the longest time I actually really liked walking back

但有很长一段时间，我其实特别喜欢走路回去

35:04

And the reason why I wanted to do that

而我之所以想那样做

35:06

Was not for efficiency

并不是为了效率

35:07

It actually really helped me think

它其实真的能帮我思考

35:09

And I like to walk for half an hour, 40 minutes or so a day

而且我喜欢每天走上半小时、四十分钟左右

35:15

Where I just get to, you know, just think about ideas, research, just get lost in my thoughts

在那儿我可以，你懂的，就想想法、做做研究，完全沉浸在自己的思绪里

35:21

That's very human activity

那是很有人味儿的活动

35:24

Unless the model has seen that

除非 model 已经见过那个

35:25

And actually understands the importance of that activity

并且真正理解那个活动的重要性

35:28

It would actually miss these kinds of features

它实际上会错过这类 features

35:31

So that actually I think is fundamentally what we're trying to model

所以实际上我认为这从根本上就是我们试图 model 的东西

35:34

Like what is fundamentally human

比如什么才是人类的本质

35:37

Might not be the most efficient thing to do

可能不是最有效率的做法

35:39

Might not be the right thing to do

可能也不是正确的事

35:41

But things that make us who we are

但正是那些让我们成为我们自己的东西

35:43

I'm curious if there are some data sets that you really want

我很好奇，有没有一些 data sets 是你特别想要的

35:47

That would materially help you

能对你产生实质性帮助的那种

35:50

One version of this may be interesting

这个问题的另一个版本可能挺有意思

35:52

Which is more valuable to you to acquire the data set

对你来说，获取哪个 data set 更有价值

35:54

All of LinkedIn, all of Twitter, all of Facebook

所有 LinkedIn 的数据，所有 Twitter 的，所有 Facebook 的

35:57

You know, to be honest, it's a little hard to rank

你知道，说实话，这有点难排

36:00

In part because, you know, there's this product saying where

部分原因是，你知道，有这么一个产品理念

36:05

No feedback is wrong

没有反馈是错的

36:07

Because it teaches you something about your users

因为它能让你了解你的用户

36:09

Doesn't matter what kind of feedback

什么类型的反馈都无所谓

36:11

I think it's a little bit like that

我觉得有点像那样

36:12

So just whatever is bigger

所以就选更大的那个

36:13

Yeah, what I think is a different domain

对，我觉得那是一个不同的领域

36:14

It was what about all of Amazon data

那所有Amazon的数据呢

36:17

Shopping data

购物数据

36:18

Shopping data

购物数据

36:19

So Amazon data is interesting

所以 Amazon 的数据很有意思

36:21

In that it's very much behavioral

因为它很大程度上是行为数据

36:23

Although like what people do on social media

虽然比如说，人们在社交媒体上的行为

36:24

You could sort of squint and say that is also behavioral

你也可以勉强说那也是行为数据

36:27

But the transaction data is always interesting

但交易数据总是很有意思

36:30

It is also most commonly available, however

不过，它也是最普遍可用的

36:33

If we were to look at purely social media

如果我们只看纯粹的社交媒体

36:35

If you really, you know, if I were, if I had to really pick

如果你真的，你知道，我是说，如果我必须真的要选的话

36:39

Facebook likely is interesting

Facebook 可能挺有意思的

36:41

Because I actually do think it is most sort of a

因为我确实觉得它在某种程度上算是

36:45

Default version of people

人的一种默认版本

36:47

Because you go to LinkedIn, it's very much professional environment

因为你去 LinkedIn，那里是非常专业的职场环境

36:51

So people put up there, you know, they have their cards up

所以人们会在上面，你知道的，把自己的履历亮出来

36:55

And that still is interesting

这其实也挺有意思的

36:56

Because that is true human attitude and behavior

因为那是人类真实的态度和行为

36:59

But it is not your base state

但它不是你的基础状态

37:02

You go to Twitter

你上Twitter

37:03

Twitter people have their own crazy personas

Twitter上的人都有自己的疯狂人设

37:06

Or depending on who you are

或者说，取决于你是谁

37:07

My Twitter profile and you know, persona is very much

我的Twitter简介和人设，你懂的，非常

37:11

Initially it was very much academic

一开始非常学术

37:12

I'm here to share my studies

我来这里分享我的研究

37:15

Now I share things that's related to similarly

现在我分享的东西也是类似的

37:18

But Facebook is one of those more private space

但Facebook是那种更私人的空间

37:21

Where people just connect with their friends

就是人们单纯和朋友交流的地方

37:23

And that way I actually do think it shows you a little bit more about who that person is

而且那样的话，我确实觉得它能多一点展现那个人的真实面貌

37:28

So if I had to pick, I likely picked Facebook

所以如果非要选的话，我很可能会选Facebook

37:30

Yeah, and you are interested in like the whole person and their background and philosophy

是的，而且你感兴趣的是那个人的整体，包括他的背景和哲学

37:35

I guess is it too clinical or too machine learning oriented to just say

我觉得直接这么说是不是太冷冰冰或者太machine learning导向了？

37:40

This is just ways to inject variants and biases

这只是注入variants和biases的一种方式

37:45

The bar question I guess is like, is this any better than a randomized

我想说关键问题是，这比一个randomized的更好吗？

37:49

like combinatorial explosion version

比如combinatorial explosion的版本？

37:51

So we have a link to the 10 cent

所以我们有一个链接，指向那个10 cent的

37:55

Billion persona paper where they basically did not do any of the groundwork that you are doing

Billion persona论文，他们基本上没有做你在做的那些基础工作

37:59

Yeah, they just sort of did like a cross matrix

对，他们就像是搞了个cross matrix

38:02

So here's all the professions in the world

所以，这里是世界上所有的职业

38:04

Here's all the people possible backgrounds in the world

这里是世界上所有人可能有的背景

38:06

Do a dot product across all of them and that's it

对所有这些做一个dot product，然后就这样了

38:09

That's your prompt for a billion people

那就是你给十亿人准备的prompt

38:12

This will do something

这样也能搞出点东西来

38:13

I don't know if it will do what you do

我不知道它会不会做你做的事

38:15

But it gets you somewhere some percent of the way there

但它能让你在一定程度上接近目标

38:18

So this actually was an interesting paper

所以这其实是一篇有意思的论文

38:19

I mean like what I admired about this paper when it came out was the scale

我是说，这篇论文刚出来的时候，我欣赏的是它的 scale

38:24

And obviously you do gradually want to be able to simulate really large societies and

而且显然你确实逐渐希望能模拟真正大规模的社会和互动，所以这个 scale 确实很值得佩服

38:30

interaction so the scale is definitely admirable

它很大程度上依赖于那些用于 training the model 的已知统计数据

38:34

It is relying heavily on the known statistics that went into training the model

所以，只要你觉得这些统计数据是正确的

38:40

So to the extent that you believe that statistics is correct

所以，如果你相信统计是正确的话

38:43

This is actually not a bad way to go about this

这其实不是个糟糕的做法

38:45

But the thesis here and this is something that we also have seen in the market

但这里的核心论点，也是我们在市场上已经看到的

38:50

Like if this works then we actually have solved simulation

比如如果这能成，那我们就真的解决了 simulation

38:53

Right

对

38:55

Because I survey like okay

因为我大概看了一下，比如

38:57

5% of the US population is in construction

US人口中有5%在建筑

39:01

Yeah

嗯

39:01

The other 5% is in medicine whatever

另外5%在医学，差不多

39:03

Right and then you just keep going down the list and then you do the other side

对，然后你就顺着列表往下走，再处理另一边。

39:05

5% has like you know the big 5 personality

5% 的人有那种，你懂的，Big 5 人格。

39:08

Have like neurotic or whatever

就是有神经质之类的。

39:10

That's it

就这样。

39:11

That's it

就这样。

39:12

So if you believe that the underlying data set

所以如果你相信底层的 data set

39:15

And the platform that we're leveraging has all the right statistics

以及我们正在 leverage 的 platform 有所有正确的 statistics

39:18

Then this actually will have solved it

那么这实际上就已经解决了。

39:20

You're at that point merely retrieving a knowledge that is already embedded in the model in the model parameters

你那时只是在检索已经嵌入在 model parameters 里的知识。

39:26

That's not unfortunately what we see

不幸的是，这并不是我们看到的。

39:28

Where there is such detailed and also niche knowledge about people

那就是有关于人的非常详细且小众的知识。

39:33

That if you just take one example it might feel very mundane

如果你只取一个例子，可能会觉得很平淡无奇。

39:37

But it's actually quite rich when you put together

但当你把它们放在一起时，实际上相当丰富。

39:40

That you actually do need to do a lot of bespoke data collection to better understand people

你确实需要做大量的定制化数据收集来更好地理解人们。

39:44

And this is also you know, I think what makes this particular

而这也是，你知道，我认为正是让这个特定的...

39:48

Job fun which is you want to

工作变得有趣的地方，也就是你想...

39:51

deeply understand people and the process of deeply understanding that actually requires a lot of

深入理解人们，而深入理解的过程实际上需要很多

39:57

Attention to the details and you do need to pay attention to and pay respect to

关注细节，而且你确实需要关注并尊重

40:02

the daily lives that people lead

人们所过的日常生活。

40:03

I want to

我想

40:05

Talk about

谈谈

40:06

Scaling simulation so what can we stimulate

Scaling simulation，那么我们能够模拟什么？

40:10

What can we simulate and how does scaling affect this so how big are the models

我们能模拟什么？Scaling 如何影响这一点？也就是说 models 有多大？

40:14

What if we go from you know it be like couple hundred million like hundred billion parameters trillion

如果我们从，你知道，比如几亿 parameters，到千亿，甚至万亿呢？

40:20

Do we get scaling any interesting emergence like at a certain scale at a certain amount of training

在某个scale和某个training量下，我们会不会从scaling中得到有趣的emergence？

40:26

You uncover anything unusual and any learnings from that

你有没有发现什么不寻常的东西，或者从中得到什么经验？

40:31

What we are seeing is that similarly so we do post train our own model

我们看到的是，同样地，我们也会post train我们自己的model。

40:36

The thing that we're actually seeing is the early clips of scaling law in simulations

我们真正看到的是simulation中scaling law的早期迹象。

40:41

The more data about humans and more compute you and just you actually sort to get predictive

关于人类的数据越多，compute越多，你实际上就会得到某种predictive效果。

40:47

Predictible gains of the model performance in simulating and

在simulating中，model performance的可预测收益，以及...

40:50

We need scaling locker

我们需要scaling law。

40:52

You know is scaling law whenever you find that it's it's a beautiful thing

你知道，scaling law，每当你发现它的时候，它就是一个美好的东西。

40:56

And we're starting to see the glimpse of it which is quite exciting

我们开始看到它的雏形了，这相当令人兴奋。

40:59

But if you talk about the ambition of simulation as as a whole it's not really about building a model

但如果你谈论 simulation 作为一个整体的野心，它其实不是关于构建一个 model。

41:06

It's about

而是关于

41:07

Building a model then creating the agents that become the individuals in a much larger ecosystem

构建一个 model，然后创建 agents，让它们成为更大生态系统中的个体。

41:12

So you're basically creating this multi-agent simulation

所以你基本上是在创建这个 multi-agent simulation。

41:16

Down the line you want these multi-agent simulation to also live in a very rich environment, right

长远来看，你希望这些 multi-agent simulation 也生活在一个非常丰富的 environment 中，对吧？

41:21

What we're really trying to get to at that point is hey can we actually create

那一刻我们真正想要达到的是——嘿，我们能不能真的创造出

41:25

I let's do a time machine game again and five years ten years into the future

我是说，让我们再玩一次时间机器游戏，去往未来五年或十年。

41:30

Can we create a simulation of eight billion people living on earth?

我们能创建一个八十亿人生活在地球上的模拟吗？

41:33

I think that's quite interesting

我觉得那挺有意思的。

41:35

And that really is the vision and once you get to that kind of state

而这确实是那个愿景，一旦你达到那种状态，

41:40

The kind of questions that you can help answer for the society also start to change from my perspective

从我的角度看，你能帮助社会回答的那类问题也开始发生变化。

41:45

The answers are fundamentally about emergence of the emerging behavior of society and large groups of people

答案根本上讲是关于社会和大型群体中 emergent behavior 的 emergence。

41:53

So for instance the kind of questions that I get excited by

比如，让我兴奋的那类问题——

41:56

And maybe this is a still a bit, you know, I have my you know academic side of me and

而且这可能还带点，你知道，我身上有学术的一面，然后——

42:01

And for me it's questions like can we help solve climate change?

对我来说，就是像“我们能帮助解决气候变化吗？”这样的问题。

42:06

If you look at climate change as a problem space

如果你把气候变化看作一个 problem space

42:08

This is what we like sort of scientists would often call the wicked problems

这有点像我们这些科学家常说的 wicked problems

42:12

Problem where you have many actors with competing incentives

就是那种有很多行动者、激励相互竞争的问题

42:16

For trying to make a very complex decision at coordinating that coordination decision

为了尝试做出一个非常复杂的决策，来协调那个协调决策

42:20

A very difficult to really solve in real life, which is also the reason why we can solve it

在现实中真正解决非常困难，而这也正是我们能够解决它的原因

42:26

Can simulation help us solve that?

Simulation 能帮我们解决这个问题吗？

42:28

Another one is can we actually

另一个问题是，我们能不能真的

42:31

Understand the signals for collapsing democracy

理解民主崩溃的信号

42:34

Or can we understand or can we uncover the origin story of the monetary system

或者我们能不能理解，或者我们能不能揭示货币体系的起源故事

42:40

These are societal questions that we never really had a good way of answering

这些社会问题，我们以前真的没有好的方法去回答

42:44

If we can create simulations of our society you have to believe that these are the kind of problems that we can solve

如果我们能够创建我们社会的 simulations，你必须要相信这些是我们可以解决的问题

42:51

So that's really the ambition of this field

所以这真的是这个领域的抱负

42:53

And you know, I also think yes, I mean I think there's a noble price to be one there

而且你知道，我也觉得是的，我的意思是，我觉得那里有一个Nobel Prize可以拿

42:57

Which wouldn't be surprising and I think there's an amazing societal impact that we can have to help people make better decisions

这不令人意外，而且我认为我们可以产生惊人的社会影响，帮助人们做出更好的决策

43:04

Nobel Prize in economics in economics. I see. I see. We're rooting for you to write that paper

Nobel Prize in economics in economics。我明白，我明白。我们都支持你写那篇论文

43:10

One of these days, but um, you know, one of the scholars that I was deeply inspired by

总有一天，但是，呃，你知道，有一位学者，我深受他的启发

43:17

When I was coming into the space of simulation actually is this scholar named Thomas Schelling

当我刚开始进入 simulation 这个领域的时候，其实有个学者叫 Thomas Schelling

43:23

Schelling point

就是那个 Schelling point

43:24

So he the canonical example of the of the work that he's done was he was one of the creators of agent-based modeling

所以他最典型的工作之一，就是他是 agent-based modeling 的创建者之一

43:32

So this was like in the 1970s and 80s. It's very early days, but this was truly one of the first examples of simulations

那大概是在 1970 到 80 年代，非常早期，但这确实是 simulation 最早的例子之一

43:39

And one of the canonical model from that time and of course many of these simulations are trying to tackle the societal problems

而那个时代的经典模型之一——当然，很多这类 simulation 都在试图解决社会问题

43:47

That's most relevant for their era

那些与当时时代最相关的问题

43:49

I was called the model of segregation. So racial segregation was a big topic

这个模型叫做 segregation 模型，所以种族隔离在当时是个大话题

43:54

And then we carried about and what they've done was they actually created this great world

然后他们所做的，实际上就是创造了一个非常棒的世界

44:00

where they had red dots and blue dots

就是那里有红点和蓝点

44:03

And these dots were back in the day like they were the agents

然后这些点在当时就像是 agents

44:07

And they had a simple rule that governed their behavior

然后它们有一条很简单的规则来支配它们的行为

44:12

if

就是如果

44:13

Certain percentage of your neighbors are of different color and if that goes above certain threshold then you move to a new location at random

你的邻居里面有一定比例是不同颜色的，如果超过某个 threshold，你就随机搬到一个新地方

44:21

One of the striking finding of this paper where this agent-based model was

这篇论文里这个 agent-based model 的一个惊人发现是

44:26

For the longest time people thought the segregation within society was caused by explicit and overt racism

很长一段时间，人们都认为社会中的隔离是由明确而公开的种族歧视造成的

44:34

But if you look at this model people's preference towards living with people of the same color

但如果你看这个模型，人们只是偏好和同颜色的人住在一起

44:39

That preference can be very minute

这种偏好可能非常细微

44:42

But the very small difference actually causes the society to segregate completely

但正是这极小的差异，最终导致社会完全隔离

44:48

Over time

随着时间推移

44:49

This was very counterintuitive for a lot of people and this actually this particular work ended up informing housing policies

这对很多人来说非常反直觉，而且实际上，这项特定的研究最终影响了住房政策

44:56

Mix income housing for instance got really inspired by this kind of work

比如，混合收入住房项目就深受这类研究的启发

45:00

And Thomas Schilling ends up winning the Nobel Prize for having later ground work for very early versions of simulations

托马斯·席林最终获得了诺贝尔奖，因为他为非常早期的模拟版本奠定了基础

45:07

The opportunity that I do see here in the more scientific terms

用更科学的术语来说，我在这里看到的机遇是

45:11

Is agent-based models for the longest

agent-based models，而且这是长期以来一直存在的机遇

45:15

Had impact in the in the

有过影响，在，在

45:17

1980s 90s to some extent early 2000s

80年代、90年代，某种程度上还有2000年代初

45:21

But it has now sort of gotten forgotten by the community a little bit because as you can imagine the red dots and blue dots is not really a rich description of people

但现在有点被社区遗忘了，因为你可以想象，红点和蓝点并不是对人的丰富描述。

45:31

But with the emergence of things like generative aia and in particular generative agents

但随着像 generative AI 这类东西的出现，特别是 generative agents，

45:36

we do have an opportunity to create

我们确实有机会去创造

45:39

These kind of agent-based models that are high fidelity

这种 agent-based models，high fidelity

45:42

enough to help us make really complex decisions

到足以帮助我们做出非常复杂的决策。

45:45

And that's the opportunity that I see if

而这就是我看到的机遇，如果

45:48

That truly works than yes, and that is the kind of work that will result in a Nobel Prize. Yeah

45:53

For what it's worth and I grew up in Singapore 80% of Singapore

45:57

Is in public housing and then public housing has

46:01

Enforced racial quotas for exactly that reason which has been interesting

46:05

Okay, so we talk about scaling we talk about all these

46:10

The sort of agent possible

46:12

Applications. I'm scared about the cost

46:16

If you even let's just keep it to the US about a billion people yeah, but

46:22

How much does it cost to model so many hundreds of millions of people oftentimes today?

今天，对数亿人进行 modeling 通常要花多少钱？

46:27

Obviously we don't start at that scale the stage of the

显然，我们不会从那种规模开始，这个阶段的

46:31

Of industry and simulation as technology

行业和 simulation 技术的

46:33

But we can actually get to our users extremely rich and meaningful

但事实上，我们确实能带给用户极其丰富、有意义的

46:37

Insights even by modeling thousands tens of thousands of people and today what we do is every week

insights，甚至通过 modeling 几千到几万人就能做到。而今天我们每周所做的是

46:43

We are collecting data on the scale of tens of thousands people's data

我们正在收集数万人规模的数据。

46:48

And we actually have panel partnerships that gets us to tens of millions of people globally

而且我们确实有 panel partnerships，让我们能触达全球数千万人。

46:54

So that's what we do today and just as a side note once you've collected one person for one study

所以这就是我们今天所做的。顺便提一句，一旦你为某项研究收集了一个人的数据

46:59

You reuse that same person for all the subsequent studies. That's exactly right

你在后续的所有研究里都复用同一个人。没错，就是这样。

47:04

The beauty of this model and these agents is the fact that they are domain diagnostic

这个模型和这些 agents 的美妙之处就在于它们是 domain diagnostic 的。

47:08

That what you're really trying to understand is what is the fundamental nature of these people what's their social physics

你真正想理解的是这些人的根本本质是什么，他们的 social physics 是什么。

47:15

And obviously there are a lot of a lot of people that does change over time like even like even things like

而且显然，有很多人身上有很多东西是会随着时间变的，就像，就像甚至连一些事情——

47:21

How many times have you gone to have you into like CBS the past week obviously that will change

比如你过去一周去了多少次 CBS，这显然会变。

47:26

But there's so many traits about people that are also known to never change

但是人也有很多特质是众所周知的从来不会变的。

47:30

And your risk tolerance doesn't really change over time. It's very consistent

而且你的风险承受能力并不会随着时间真正改变，它非常稳定。

47:35

Um, so it's these kind of things that we're trying to learn

嗯，所以我们想要去学习的就是这类东西。

47:38

But the scale we are operating is right now hundreds or

但我们目前运营的规模是几百或

47:42

tens of thousands to hundreds of thousands and in many of the core use cases that we

几万到几十万，而且在很多我们核心的 use case 中，我们

47:48

We are deployed in and this is more than enough population to cover those

部署在这些场景里，这个 population 量级已经足够覆盖那些了。

47:52

Really at that point what you care about is less than number of people

真的，到那个点上，你关心的不再是人数

47:55

But more do you have the right self population of interest covered and this is also the reason why people want a larger sample

而是你是否覆盖了正确的、你感兴趣的 self population，这也是人们想要更大 sample 的原因。

48:04

It's not because they actually want a stronger statistical guarantees

不是因为他们真的想要更强的统计保证。

48:08

It's more that can they actually settle down to any population of their interest

而是他们能否真正落到任何一个他们感兴趣的 population 上。不过

48:12

However

不过

48:14

You can also imagine in 10 years if we truly believe that the compute

你也可以想象，在十年后，如果我们真的相信compute

48:19

Is going to scale that will have much more availability for compute

会持续scale，那compute的可用性会大大增加

48:23

And our ambition for simulation is also going to scale accordingly

而我们对于simulation的野心也会相应地scale

48:29

And there's definitely a reason for us to create

而且我们绝对有理由去创建

48:32

An entire data center worth of simulations

整整一个data center规模的simulations

48:35

or

或者

48:36

My

我的

48:37

Hunch here is I do think in the next some number of years we will start creating simulations

直觉是，我确实认为在接下来几年里，我们会开始创建simulations

48:43

That will actually cost as much as training a foundation model

那实际上会花费和训练一个 foundation model 一样多的钱。

48:48

But perhaps it's going to be so valuable to the society that it would be a no-brainer

但也许它对社会太有价值了，所以根本不用想，直接做。

48:53

I mean right now even today like we are training

我是说，就在现在，哪怕是今天，我们都在训练

48:55

bunch of new foundation model just so we can say we trained one and we spend tens of millions

一堆新的 foundation model，只是为了能说我们也训练了一个，然后花掉几千万。

49:00

But if we can create a simulation at the level of society

但如果我们能创造一个社会层面的 simulation，

49:04

That would actually solve climate change

那就能真正解决气候变化。

49:06

I'll run that today. I'll raise the money right now just to run that

我今天就会跑这个。我现在就去筹钱来跑这个。

49:10

Amazing

太棒了。

49:12

I guess the follow-up question is

那我想追问的是

49:14

Does it also compound if you let the simulations talk to each other

如果让这些 simulation 互相交流，会不会也产生叠加效应？

49:18

Or do they already do that today they don't right as far as I understand

还是说现在已经这么做了？据我所知，还没有，对吧？

49:21

It depends on what kind of simulation you're trying to run. Yeah

这取决于你想跑哪种 simulation。嗯。

49:24

In the multi-agent simulation setup the agents do talk to each other right which is exactly small

在 multi-agent simulation 的 setup 里，agents 确实会互相交流，对吧，这正好是小规模。

49:29

Right, that's right, but a lot of times for example in e-commerce you're just by yourself

对，没错，但很多时候，比如在 e-commerce 里，你就是一个人。

49:34

So there's no point talking

所以没必要交流。

49:36

Which is always levels right like you

这总是有层次的，对吧，就像你…

49:39

Decide what you will buy based on what other people around you buy and talk about right

根据周围人买什么、聊什么来决定你自己买什么，对吧？

49:43

No, it depends

不，这要看情况

49:45

Again, I'm coming at this from a costment of you

再说，我是从你的成本角度出发的

49:47

I'm like oh my god like I think if there is like some combinatorial thing of like thousands of people talking to thousands of people then

我当时就想，天哪，我觉得如果有某种 combinatorial 的东西，就是成千上万的人和成千上万的人交谈，那么

49:53

That one million access my costs have a very different views the cost point aside like running these studies

那一百万次的访问成本看起来非常不同，撇开成本不谈，比如做这些研究

50:00

In reality is actually a lot more expensive right running any study like this is you got to have people do you got to sign people up

实际上，做这样的研究真的贵很多，对吧。你得有人，你得招募人。

50:07

It's it's very expensive and sometimes

这非常贵，而且有时候

50:10

Like

比如

50:11

Not feasible to actually run the study

实际上，真正去跑这个研究是不可行的。

50:14

But the outcome or the decisions you make are very expensive on them, right? So

但结果或者你做的决定对他们来说代价很高，对吧？所以

50:20

spend x million on something that you know the overall process costs

在一个你明知道整体流程要花多少钱的事情上，花个几百万，对吧？

50:25

A hundred million might as well, right? There's there's a lot of value to be had there. It's a small cost

一个亿也无所谓，对吧？这里面的价值很大，成本很小。

50:30

But I'm excited on the cost side actually to some extent you know

但某种程度上，其实我对成本这块还挺兴奋的，你知道吧。

50:34

And obviously when you deploy technology you often want to deploy in a way where you can replace existing budget or you can

而且显然，当你部署技术时，你通常希望部署得能替换现有预算，或者你可以

50:42

Basically make things more efficient and that is the best way to deploy

基本上就是提高效率，那才是最好的部署方式。

50:46

However, the way you capture the long-term value of the technology actually is making an argument

然而，你获取技术长期价值的方式，其实是提出一个论点。

50:52

Then now it's actually the upside that by making this better decision using simulation

那么现在呢，实际上好处是，通过用 simulation 做出更好的决策

50:57

You have saved yourself or made yourself hundreds of millions or even billions of others and that's a case to be made

你已经为自己省下或者赚到了数亿甚至数十亿，这个说法是成立的

51:06

Random tangent question. So if you're doing a lot of inference a lot of model multi-agent stuff

顺便跑题问一下。所以如果你做大量 inference，大量 model multi-agent 这类东西

51:11

Are you at the point where it makes sense to you know train a model that's you know

你是否已经到了那个阶段，你知道，train 一个 model 是有意义的，你懂吧

51:17

Very sparse you're expecting to do multi-million dollar

非常 sparse，你预期要做 multi-million dollar

51:21

Runs are you thinking about this in model architecture standpoint or inference efficiency or you know

runs，你是从 model architecture 的角度，还是 inference efficiency 的角度，或者你知道，来考虑这个问题？

51:29

You're still at the research phase of

你还处于 research 阶段，就是

51:32

It works it works. We're not super there yet efficiency. We actually do think quite a bit about

能跑，能跑。在 efficiency 方面我们还没完全到位。其实我们确实想了很多关于

51:36

I mean this is technology that is deployed now in some of the largest enterprise companies in the world and we do process

我的意思是，这项技术现在已经部署在全球一些最大的企业公司中，我们确实处理大量的 queries，试图覆盖全球的用户群体。所以效率是一个持续关注的重点，显然我们不想过早地过度优化。所以我不觉得现在到了过度优化的时候，但这绝对是我们会非常认真考虑的事情，是的。还有其他案例研究，我们之前谈到过 CVS、Gallaud 和 Deloitte。

51:46

Significant number of queries and that are trying to you know similarly the populations in the world

大量的 queries，而且这些都是在试图，你懂的，类似地覆盖全球的人口。

51:51

So efficiency is a consistent thing obviously we don't want to

所以 efficiency 一直是一个持续考虑的东西，显然我们不想

51:56

over optimize too early

过早地 over-optimize

51:57

So I wouldn't say like this is the the higher a bit right now

所以我还不会说，这现在是更优先一点的事情

52:02

But this is definitely something that we we think pretty carefully about yeah

但这绝对是我们会非常认真考虑的事情，嗯。

52:06

And the other case studies so we talked about CVS about Gallaud

还有其他案例，我们聊过CVS和Gallaud

52:10

Deloitte

52:11

Well-front what front is interesting one

Well-front这个案例很有意思

52:14

Because one of the things that are trying to do they were one of the first

因为他们试图做的事情之一是，他们是最早的一批

52:17

Customers and that wanted to actually do product testing

客户，而且他们想要真正做产品测试

52:21

That goes beyond just asking people what they think about let's say behavior experiments and so forth

这不止于单纯地问人们，比如说，对行为实验等等有什么看法

52:26

So there really what we had to do was reason about multi-modal input

所以我们真正要做的是对multi-modal input进行推理

52:32

So images but also you can also imagine like these agents traversing through figma mockups or websites

比如图像，但也可以想象这些agents在figma mockups或者网站中穿梭

52:38

So some of the things that our agents can also do is you can be given

所以，我们的 agents 还能做的一件事就是，你可能会得到一个

52:42

Outdomain like not really a website URL and actually go use it for a while

outdomain，比如不是一个真正的网站 URL，然后实际去用它一段时间

52:46

It's these kind of things and both for instance was one of the first

正是这类事情，例如 both 就是最早的那批

52:50

Customers that was very excited about this possibility. Well have people been

对此非常兴奋的客户之一。那么人们是否一直在

52:55

Asking like is there any demand that we have not covered like UI testing right?

问类似这样的问题：有没有我们未覆盖到的需求，比如 UI 测试，对吧？

52:59

I want to try a new

我想尝试一个新的

53:01

I want to ship a new feature test the UI

我想发布一个新功能，测试 UI

53:03

Similarly, I hope people will do it any any interesting things that you're seeing demand for

类似地，我希望人们会去做任何，任何你们看到有需求的有趣事情

53:08

Today a lot of the demand does come from

今天，大量的需求确实来自于

53:14

Basically like the places where people have historically used human panels

基本上，就是那些人们历来使用 human panels 的场景。

53:19

We can basically now replace with agents and synthetic populations

我们现在基本上可以用agents和synthetic populations来替代了。

53:24

And this is obviously not replacing human panel in many ways the simulation that similarly is building is grounded

这显然在很多方面并没有取代human panel，同样地，正在构建的simulation是grounded的。

53:30

So the way that I think about this is we are trying to represent humanity at scale

所以我对此的看法是，我们试图大规模地代表人类。

53:36

And in that way the use cases are what we would expect but it's the scale of deployment that surprises me

在这方面，use cases是我们所预期的，但deployment的规模让我惊讶。

53:44

Turns out there's so many decisions that people make every day in these organizations groups

事实证明，人们每天在这些组织和群体中做出的决策太多了。

53:49

And we want to be able to say we listen to people we have consulted our users

我们希望能够说我们倾听人们的声音，我们咨询了我们的用户。

53:55

But in reality that is rarely the case because getting to people and actually asking them many questions

但现实中很少如此，因为接触人们并真正询问他们许多问题

54:00

It's it's difficult. It's both costly

这很困难，而且成本很高。

54:03

Time consuming but most importantly people are just not available

非常耗时，但最重要的是根本找不到人

54:07

if I had to

如果非要我

54:09

Answer

回答

54:10

Thousand survey questions for this one particular vendor

关于这一家特定供应商的一千个调查问题

54:14

Even if I wanted to do that like I would never do it and that's very much the case

即使我想这么做，我也绝不会去做，事实就是这样

54:20

What simulation can do is

Simulation 能做的是

54:23

Ensure that the voices of people is always represented in rooms where the decisions for them is made

确保在为他们做决定的房间里，人们的声音始终被代表

54:32

So all the stakeholders of this particular product launch ideally they are consulted

所以，这次特定产品发布的所有利益相关者，理想情况下都应该被咨询

54:37

That's what this technology really is trying to enable

这就是这项技术真正想要实现的目标。

54:39

In my mind that means it's used for as more consumer focus right like anything with a wide enough

在我看来，这意味着它更多地面向消费者，对吧？就像任何拥有足够大客户群的东西一样，你会从你所代表的多样性中获益。

54:45

customer base where you do benefit from the diversity that you represent

对于那些不太熟悉这个市场的人来说，大概有哪些统计数据？

54:50

What are some rough statistics just for people who are not familiar with this market in general

市场规模是多少？我相信你显然有一些大概的数字。

54:54

What's the market size that I'm sure you have some like rough numbers obviously

市场规模是个模糊的问题。对，但人们大概会花多少钱？

54:59

Market size is like a vague question. Yeah, but like how much do people spend

所以市场研究是一个一千亿美元的行业。对。

55:03

So market research is a hundred billion dollar industry. Yeah

但关于simulation，重点是simulation不是市场研究的工具。

55:06

But the thing about simulation is simulation is not a tool for market research

但关于 simulation 的问题是，simulation 不是做市场调研的工具

55:12

simulation is a tool for human decision making

simulation 就是人类决策的工具。

55:15

So the the question around what is a 10 here is actually quite tricky right because it's easy to say what market research

所以这里“什么是10”这个问题其实很棘手，对吧？因为要说 market research 是什么很容易。

55:21

10 is roughly a hundred million or a hundred billion

10 大概就是一亿或者一千亿。

55:24

So is that a 10 and not really right because in many ways

所以那算是10吗？其实不太对，因为在很多方面……

55:29

You're trying to inform all human decision making

你想要为所有人类决策提供信息。

55:32

You're trying to basically inform every decisions that are made about human for humans

你基本上是在为所有关于人类、为了人类而做的决策提供信息。

55:38

What is a 10 for that really unclear

对那个来说，什么是10就真的很不清楚了。

55:41

And I'll be honest, you know, I have a scientific background. I have a research background

说实话，你知道，我有科学背景，也有研究背景。

55:46

So I didn't come into the field actually calculating. Oh, what is a 10 for human decision making

所以我进入这个领域时并没有真的在计算。哦，人类决策的10分是什么？

55:51

But I just had to assume well if we can inform every decision that is made about human for human that has to be baked

但我只能假设，好吧，如果我们能给每一个关于人、由人做出的决策提供信息，那一定得融入...

55:58

Something valuable exactly. I mean some extent, you know, you are a unicorn founder now and you have to care as a CEO

某种有价值的东西，没错。我的意思是，在某种程度上，你知道，你现在是独角兽创始人，作为CEO你不得不关心。

56:05

But like I do think like yeah, we go into these boardrooms with people that you're quoting millions of dollars the contracts for like

但我觉得，是啊，我们走进这些董事会，和那些你给他们报价数百万美元合同的人打交道，比如...

56:12

You have to say well, well, here's what you spend on humans

你得说，好吧，好吧，这是你在人力上的花费。

56:15

And here's what we save you and it's 85% similar

而这是我们为你节省的，而且有85%的相似度。

56:18

And certainly the the value case is something that we care deeply about like what is the value that we actually

当然，价值主张是我们非常关心的事，比如我们真正提供的价值是什么。

56:23

Provide to the users and the decision makers

提供给用户和决策者。

56:26

But this is also where like you know as a

但这也就是，像你懂的，作为一个——

56:29

I found it

我觉得吧，

56:31

I think valuation only tells us one very superficial aspect of the story

我觉得估值只告诉我们故事非常表面的一个方面。

56:36

And I try not to think too much about valuation in general because that's not what also motivates the team or something doesn't

而且我一般尽量不去想估值，因为估值也不是驱动团队的东西，或者说至少不是真正激励我们的东西。

56:43

Yeah, I'm I again the interesting thing about researchers is we're happy living in academia getting paid next to I mean

对，我——再说研究者的有趣之处在于，我们愿意在学术界过清贫的生活，拿着接近——我的意思是，

56:52

Be kept paid okay. I mean we don't get paid that much. I mean it's a researcher in academia

我们收入还行吧。我是说，我们其实赚得不多。我是说，作为一个学术界的研究者。

56:56

But it's the impact and it's the

但真正重要的是影响力，是那种——

56:59

It's the value that we can provide to the individuals and the society that really drives us

是我们能给个人和整个社会带来的价值，那才是真正驱动我们的东西。

57:04

And in that way ultimately what drives us is the impact

所以归根结底，驱动我们的是影响力。

57:08

Does the simulation we provide have a real impacting people's decision-making in ways that progresses our society for

我们提供的simulation是否真的影响了人们的决策，从而推动社会进步？

57:16

If the answer is yes

如果答案是肯定的，

57:18

Then yes, I mean that has to be a great business and we see that in numbers and we do care deeply about the upside story

那当然，我是说那肯定是一门好生意，我们在数字上也看到了，而且我们确实非常关心增长的故事。

57:25

But that's the higher a bit

但那就有点太高远了。

57:27

Do you have any timeline prediction? So we talked about scaling laws of simulations your product

你有时间线上的预测吗？我们聊到了simulation的scaling laws，以及你的产品。

57:33

Okay, maybe one day we can simulate how to solve climate change

好吧，也许有一天我们可以simulate如何解决气候变化。

57:38

Where are we now

我们现在处在什么阶段？

57:40

If that's not the end state what is an end state and what does progress look like you know

如果那不是最终状态，那什么是最终状态？进步又是什么样子，你知道吧。

57:45

So what I sometimes tell people is simulation is industry it feels a lot like where jib-t is 3.5 jib-t 4 was

所以我有时候跟人说，simulation这个行业的感觉，很像GPT-3.5和GPT-4当时所处的阶段。

57:53

For the agi saga which basically is we have now technology that is powerfully enough to do real

对于AGI的历程，基本上就是我们现在已经有了足够强大的技术，能够实实在在地

58:01

Damage on the verticals that we are tackling at the same time. There's a lot of progress that is yet to come

对我们同时在攻克的verticals造成真正的冲击。同时，还有很多进步是尚未到来的。

58:07

That's I think where this is

我觉得这就是我们现在的处境。

58:09

So the way I see it I do think there will continue to be breakthroughs both in data

所以在我看来，我确实认为在data方面会持续有突破，

58:16

And obviously in algorithm and there will be much more aggressive scaling

显然在algorithm方面也是，而且会有更加激进的scaling。

58:21

They will also happen over the next few years

这些也都会在未来几年内发生。

58:24

But I think that's roughly sort of where we are

但我觉得大致上我们就处在那个位置

58:27

I think that was about

我想那大概是

58:29

The rough set of topics anything else that we should have asked you you wish people asked you more about

大致的话题范围，还有什么我们应该问你的，或者你希望人们更多问你的？

58:35

Well, similarly

嗯，类似地

58:36

You know, I think the what's

你知道，我想，那个什么

58:40

For me what's actually quite fascinating fascinating about simulation

对我来说，simulation 其实非常非常迷人的地方在于

58:44

It is very impactful technology, but actually

它是一项非常有影响力的技术，但实际上

58:49

It's also very interesting technology both in terms of like what it means for human society our philosophy

它也是一项非常有趣的技术，无论是对人类社会而言，还是对我们的哲学来说

58:56

And the way I sometimes interpret simulation is so

我有时对 simulation 的理解是这样的

59:01

Going back to my background

回到我的背景

59:03

I actually as I mentioned earlier I started my career as a painter

实际上，如我之前提到的，我的职业生涯始于一名画家

59:08

It was a professional pursuit and I actually did ore painting for figures

那是一种专业追求，我确实画了更多的人物画

59:13

So I got my training originally in sort of the realism studios and that's what I

所以我最初在某种现实主义工作室接受训练，而这就是我

59:19

Spend a lot of my years doing

花费了我很多年时间所做的事情

59:23

simulation is a lot like painting

simulation 很像绘画

59:25

right in the best paintings

正是在最好的绘画作品中

59:28

Teach you something deep about the subject that you are trying to represent

教你一些关于你想呈现的那个主题的深层东西

59:33

And it is always not a perfect representation. It no painting is perfect

而且它永远不是一个完美的呈现。没有哪幅画是完美的

59:37

There's always some small differences and discrepancy

总会有一些小差异和出入

59:40

But what it does is it tries to highlight the thing that matters the most about the subject

但它的作用是，试着突出那个主题最核心的东西

59:48

Decential essential essence you he's brought up some of your work just nice to put it up

本质精华。你提到了你的一些作品，正好可以放出来

59:54

Yes, these are some of the works as so this is actually from my personal website that I maintain one

是的，这些就是一些作品。其实这是我个人网站上的，我一直维护着

59:59

I was still a researcher. I think a lot of people will say like you know like a Picasso

我当时还是研究员。我觉得很多人会说，你知道，像毕加索

60:04

Like anything more postmodern is like very much focused on the essence. Yes, all right

就像更后现代的东西，非常专注于本质。对，没错

60:10

Yeah, I don't know if any any one of these evokes something that you like to tell the story of

嗯，我不知道这些画里面有没有哪一幅会让你想讲述它的故事

60:15

No, it's it's one of those things where you know each of these paintings drawings whatever maybe

不，这是那种，你知道，这些画啊素描啊什么的，也许

60:20

It is trying to surface something about the subject that you feel deeply about

它是在试图把某个你深有感触的主题浮现出来，

60:26

onto the surface

到表面上。

60:27

You know when I was a painter and artist the topic that I

你知道，当我还是个画家和艺术家的时候，那个主题，

60:31

Carried really deeply about actually was the mode more

我真正深切关心的，其实是那种更

60:35

mundane aspect of human lives

平凡的人类生活面向。

60:38

This actually shows up in some of the some of the work that I've done

这实际上在我做过的一些作品中有所体现。

60:41

Where I did this entire sort of study of a

我在一个小镇做了这么一番研究，

60:45

ruler town where

基本上就是到处逛逛，给人们拍照，

60:48

I basically went around and took photos of people

他们没做什么特别的事，就只是过着日常的生活。

60:52

For not really doing anything special but just living there everyday lives

我觉得那才是最有意思的部分。

60:57

I thought that was the most interesting thing. I'm somebody who has this

我是那种有这种视角的人——你知道，世界是围绕一个fractal结构运作的。

61:01

perspective where you know the world is oriented around this fractal shape

而你可以选择去理解这个fractal结构：

61:06

And you have to choice to understand the fractal shape

要么向外走，尽可能多去探索，去理解fractal更宏观的形状；要么向内看。

61:09

You either go outward and try to explore as much as you can to understand the broader shape of the fractal or you look inward

你要么向外走，尽可能去探索，去理解 fractal 更广阔的形状，要么向内看

61:16

because you know the outward resembles the inward

因为你知道，外在反映内在

61:19

Shapes and

形态，以及

61:21

Understanding the mundane aspect of it was very much that

理解它平凡的一面，很大程度上就是在做这件事

61:24

simulation has a lot of this right?

simulation 有很多这方面的东西，对吧？

61:26

You're trying to understand even the most mundane aspect of people when put together

你试图理解人们放在一起时甚至最平凡的面向

61:32

Teach you see something really deep about that individual and the society

然后你会看到关于那个个体和社会的一些非常深刻的东西

61:36

So I think that's what's interesting about simulation sort of the way the same way that AGI

所以我觉得这就是 simulation 有趣的地方，就像 AGI 一样

61:41

Helped us better understand or really think critically about humanity and human intelligence

帮助我们更好地理解，或者说真正批判性地思考人性和人类智能

61:46

Simulation is really an exercise of understanding more about human society and our collective lives

Simulation 真的是一项理解人类社会和集体生活的练习。

61:54

So that I find to be a particularly interesting yeah now you're reminding me that some of the best biographers

所以我觉得这特别有意思，对，你提醒了我，一些最优秀的传记作家

62:00

documentarians

纪录片创作者

62:02

And even photographers they're taking a photo of you

甚至摄影师，他们要给你拍照

62:05

But before I take a photo of you, I must spend I must like follow you for a week

但在给你拍照之前，我必须花时间，我得先像那样跟你一个星期

62:09

Just to understand you you know which some artists some do part of your work

就是为了了解你，你知道，有些艺术家确实会把这作为他们工作的一部分

62:14

There's a very famous book called working. I don't know if you've been referred to it before yeah

有一本非常著名的书叫 Working。我不知道你以前有没有被人提过这个，嗯。

62:19

It's very very famous like you know to the point of having a week of pd page

它非常非常有名，你知道，甚至到了有 Wikipedia 页面的程度。

62:23

About this kind of like really in-depth understanding and interview of people as they about that about their lives

关于这种对人们生活的深入了解和访谈

62:28

Which seems mundane but is told in a very

这看起来平淡无奇，但讲述方式却非常

62:31

Compiling way yeah 1970s as well

引人入胜，对，1970年代也是。

62:34

Okay, it was an amazing decade

好吧，那是一个了不起的十年。

62:39

Actually before closing question

实际上，在结束问题之前

62:41

You said that you started similarly with your 10-year question right if we do that now

你说过你也是从你的十年之问开始的，对吧？如果我们现在也那样做的话

62:47

10 years down what what can we simulate what would you simulate if like if you've made significant process are there any questions

十年之后，我们能模拟什么？你会模拟什么？比如说，如果你已经取得了重大进展，有什么问题吗？

62:55

Outside of the ones that we brought up and anything that you think is most important anything that you would go

除了我们提到的那些，还有没有你认为最重要的？你还会去做什么？

63:01

Vision 10 years old in many ways as I imagine I I am somebody who's very much impact-driven

这个愿景已经有十年了，在很多方面，我觉得自己是一个非常注重影响力的人。

63:07

So the what would actually inspire me is I would want to ask

所以真正激励我的是，我想问一个问题。

63:11

10 years later what would actually be the most important societal question that we as a society have to ask

十年后，我们整个社会真正需要提出的最重要的问题是什么？

63:17

I would love to tackle that like for instance

我很想解决这个问题，比如说——

63:20

Do we need ubi that could be an interesting one?

我们需要UBI吗？这可能是个有趣的问题。

63:23

Oh has anyone done that? Well, I mean, you know, we're thinking about it. I can get access

哦，有人做过这个研究吗？嗯，我是说，你知道，我们也在考虑。我可以去获取相关信息。

63:28

Just openly. I just this is like you just trivia now like openly. I or I think Sam Altman actually funded a study on this in Africa

就公开说吧，我觉得这有点像闲聊了。不过我记得Sam Altman确实在非洲资助了一项关于这个的研究。

63:36

And the answer was no the answer was no

而答案是否定的，答案是否定的。

63:38

But what was it something about the implementation? Yeah, I always give this you

但问题是不是出在 implementation 上？是啊，我老是跟你说这个。

63:43

But this is a thing

但这就是问题所在。

63:44

See when Sam

你看，当 Sam

63:47

funded this particular

资助这个特定项目的时候，

63:50

He spent 14 million dollars only

他其实只花了1400万美元。

63:53

But this is the thing this is the reason why you want to run simulation

但关键在于，这就是为什么你要跑 simulation。

63:56

You spend five years 40 million dollars on this one study and have one finding

你花五年时间、四千万美元做一个研究，就得到一个发现。

64:01

But if you can run simulation many many times instantly then that's the value

但如果你能瞬间无数次地跑 simulation，那才是价值所在。

64:07

I feel like that one could you could have done in a simulation like if you can do the housing study

我觉得那个，你完全可以在 simulation 里做，比如如果你能做那个住房研究的话。

64:11

You can do the ubi one like it I mean come on

你也能做 UBI 那个啊，我是说，拜托。

64:13

I think sometimes people will spend the money because they

我觉得有时候人们会花这笔钱，因为他们

64:16

Want to verify what you think right like sometimes you just want to is it actually is it actually right like you got tested

想验证你的想法，对吧？就像有时候你只是想，这到底是不是真的？就像你被测试了一样。

64:23

Okay, closing question. What are the chances? We are in a simulation right now

好，最后一个问题。我们此刻在 simulation 里的几率有多大？

64:27

It's a fun question and I I started at some point. I just answered here with definitely an assimilation

这是个有趣的问题，我不知从什么时候开始，就直接回答：绝对是 simulation。

64:33

What I do feel however is whether we are in a simulation or not

然而我确实觉得，无论我们是否在 simulation 里，

64:39

That I don't think that makes our experience any less real

我不认为那会让我们的体验变得不那么真实。

64:43

And I think that's fundamentally like what I believe in maybe we live in a simulation maybe not but

我觉得这基本上就是我相信的，也许我们活在 simulation 里，也许不是，但

64:48

It's for us. Yeah for me. I don't really care. Yeah unless you die and you wake up in like

这跟我们有关。对，对我来说，我真的不在乎。对，除非你死了，然后你在一个像是...中醒来

64:53

The level higher or no interest. I feel like you wouldn't care

更高一层，或者就是没兴趣了。我觉得你不会在乎

64:57

Once you die then you find out

一旦你死了，你就会发现

65:02

I worry about the one I die. Yeah, I think the other thing that I'm okay

我担心的是我死的那一刻。对，我觉得另外一件事是我可以接受

65:06

So I like the mathematical answer to this which is like the

所以我喜欢这个问题的数学答案，就像是

65:09

sheer number of possibilities that you are in a simulation far away the sheer number of possibilities that you're not yes, except for the

你处于一个遥远的 simulation 中的可能性数量巨大，你不在其中的可能性数量也巨大，是的，除了这个

65:17

simplest answer which is

最简单的答案，就是

65:19

It is computationally very expensive to have you be a simulation

让你成为一个 simulation 的计算成本非常高。

65:24

um

嗯

65:25

Okay, great. You've been very generous. If you're a time congrats on your success

好的，太好了。你真是太慷慨了，付出这么多时间。恭喜你的成功。

65:28

uh, you know

呃，你知道的。

65:29

I met you just after your small will paper and had no idea that you could build like such an enormous company

我就是在你发表那篇 small-world 论文之后见到你的，完全没想到你能建立起这么庞大的一家公司。

65:34

And then now you're like well, it's a hundred billion dollar market, but that's just where we're starting

然后现在你就像，好吧，这是一个千亿美元的市场，但这才只是我们的起点。

65:39

So

所以

65:40

This is

这是

65:41

Uh, very exciting 100 million dollar market was not the time. That was only yeah, exactly

呃，那个令人兴奋的1亿美元市场并不是目标。那只是……对，没错。

65:46

It's if you're thinking too small

那就是你想得太小了。

65:48

But I do believe that I made you my final note here might be

但我确实相信，我最后想说的一点可能是这样。

65:52

Again, I love science fiction. You look at any advanced civilization in science fiction

再说，我喜欢科幻。你看科幻里的任何先进文明。

65:57

There's two twin pillar of technology

技术有两大支柱。

66:01

Once a GI in some form and the other is simulation

一个是某种形式的AGI，另一个是simulation。

66:05

So I think the the market's pretty clear. Yeah, tell us about the company. You guys just raised a lot your half a research lab

所以我觉得市场已经很清楚了。对，给我们介绍一下公司吧。你们刚融了一大笔钱，你们一半是研究实验室。

66:12

Half a company. You guess you're hiring re-based

一半是公司。你们应该在招人吧？

66:15

Yeah, so we're based on mission rock. So not too far away from where we are right now. So we're in sf

嗯，我们就在Mission Rock。离我们现在这儿不远。对，我们在SF。

66:21

Uh, but we are also by coastal. So we have our uh team

呃，不过我们也是东西海岸都有。所以我们的，呃，团队

66:24

I would say our headquarters in sf

可以说总部是在SF。

66:26

And we have a lot of our technical talent in sf

而且我们很多技术人才都在SF。

66:29

And we do have a smaller office that just opened up actually in New York

另外我们确实有个小办公室，是最近刚在New York开的。

66:33

We are as a company an interesting one in that

我们作为一家公司，有意思的一点是

66:36

Today obviously there are AI new labs and then there are AI product companies

现在很明显有AI实验室，也有AI产品公司。

66:40

Similarly, it truly is both

同样地，我们确实两者都是。

66:42

So this is a company that was founded by four co-founders myself micro-brands team personally young lady Ellen

所以这家公司是由四位联合创始人创立的：我自己、micro-brands 团队、年轻女士 Ellen

66:48

Uh micro person I are all researchers

呃，micro person 和我都是研究人员

66:51

So of course Michael was one of the co-authors of the image net kicks her the AI revolution back in 2013

所以当然，Michael 是 ImageNet 的合著者之一，那篇论文在 2013 年开启了 AI 革命

66:56

Has been instrumental in humans and our AI

他在人类和我们的 AI 中一直发挥着重要作用

66:58

Percy coined the term foundation model and obviously say no one of the the grades of the AI researchers today

Percy 创造了 foundation model 这个术语，显然他是当今 AI 研究者中的顶尖人物之一。

67:05

And Laney is my business counterpart where she lends on the fastest growing AI native companies from their seat to AMB

Laney 是我的业务搭档，她在 AMB 负责投资增长最快的 AI native 公司。

67:11

But we have this DNA at the company where

但我们公司有这样的 DNA，

67:15

The vision of the technology that we're creating

那就是我们正在创造的技术的愿景。

67:17

Is continuously developing that we are

在持续发展的就是

67:21

Getting people who were basically my lab mates. We are right now

招入那些基本上是我实验室同门的人。我们现在

67:25

About 60 or so people

60人左右

67:28

15% almost 20% of the company

公司里15%，几乎20%的

67:32

population actually are just my lab mates

员工其实都是我的实验室同门

67:35

And we're it's actually quite fun because many of them then had gone on to open AI

而且这其实很有意思，因为他们很多人后来去了OpenAI

67:40

Uh Google Gemini and these places and so it's been a few years since we really got together and had a chance to work together

呃，Google Gemini这些地方。所以已经好几年没有真正聚在一起共事了

67:47

But now they're coming back and really building out this vision that I find to be quite exciting and that excitement is share

但现在他们回来了，真的在构建这个我觉得非常令人兴奋的愿景，而且这份兴奋感是共享的

67:54

So there isn't that motion that similarly where we are group of researchers trying to do something that no one is working on that

所以没有那种，类似我们是一群研究人员，想做一个还没人在做的东西

68:00

We find to be the most impactful potentially

我们觉得那可能是最有影响力的

68:03

But at the same time this is again technology that can make impact today

但与此同时，这又是一种今天就能产生影响的技术

68:07

So we have an amazing group of engineers

所以我们有一支很棒的工程师团队

68:10

Product people and designers

产品人员和设计师

68:12

Uh who are sitting here with us basically trying to imagine

呃，他们基本就是和我们坐在一起，试图想象

68:17

What does it look like to help people understand what simulation can do and make real world decisions with this

帮助人们理解simulation能做什么，并以此做出真实世界的决策，那会是什么样子

68:23

Having both and then deploying it to some of the largest customers in the world today

两者兼得，然后部署到今天世界上一些最大的客户那里

68:28

It feels quite unique

这感觉挺独特的。

68:30

Yeah, it's very compelling one part of it was this is the culture action like who are you hiring

是的，这很有吸引力。其中一部分就是这种文化层面的东西，比如你招什么样的人。

68:35

You've done part of it, which is you know, you've got a very talented group. Who are you hiring?

你已经在某种程度上做到了，你知道，你有一个非常有才华的团队。那你现在在招什么样的人？

68:39

Like what what roles? So honestly at this point we're hiring up a small yeah section

比如哪些职位？所以老实说，目前我们在招一个小团队，嗯，差不多。

68:44

We are always excited to bring on amazing research talent

我们一直很期待能引进优秀的研究人才。

68:50

So if you're interested in working with you know

所以如果你有兴趣和我们共事，嗯，

68:53

Our lab mates we're always saying we're coming up amazing researchers

我们的实验室伙伴们，我们总是说我们在培养出色的研究人员。

68:57

But also we hire amazing engineers and that some of whom I like I respect

但我们也招很棒的工程师，其中有些人我真的很欣赏和尊重。

69:04

The most many of them actually come from places where we have personal connections with so many of the members are from

他们中的大多数其实来自我们有私人关系的地方，很多成员都来自

69:11

Thickma notion Harvey and so forth

Thickma、Notion、Harvey 等等

69:14

And we're also more broadly from the companies that we as a team have really admired

而且更广泛地说，也来自我们团队一直很欣赏的那些公司

69:19

So engineers both on the product side in fresh side

所以产品端和 fresh 端的工程师们

69:22

We're all looking for those hires. Oh lots of people I think you make a really good case

我们都在找这些人才。哦，很多人，我觉得你说得很有道理

69:27

So um thanks and uh we'll see you in the simulation

所以，嗯，谢谢，我们到时候在 simulation 里见

69:30

Amazing see you all there

太好了，大家到时候见

Simulation: the new Scaling Law — Joon Sung Park, Simile AI

Play Queue