Humanity’s Last Invention — Richard Socher of Recursive ·

00:02

We're here in the studio with Fibu and myself and Richard social welcome. Thanks for having me

我们在演播室里，和 Fibu、我自己还有 Richard social，欢迎。谢谢邀请我来

00:07

We just talked about the Eureka machine or we just released a talk at AI engineer about the Eureka machine

我们刚聊了 Eureka machine，或者说我们刚在 AI engineer 发布了一个关于 Eureka machine 的演讲

00:13

It's you said is your life's goal. What is the Eureka machine?

你说过，这是你的人生目标。那 Eureka machine 是什么？

00:16

The Eureka machine is

Eureka machine 是

00:18

the ultimate invention that will

终极发明，它将会

00:22

Afterwards invent most everything for humanity

之后为人类发明几乎所有东西

00:25

It's essentially a super intelligence that can be given any kind of goal any kind of

它本质上是一个 super intelligence，可以被赋予任何类型的目标、任何类型的

00:33

Environment reward and then it will

environment reward，然后它就会

00:37

Tried's best to achieve those goals to create the kinds of inventions that humanity would hopefully ask it for

已经尽力去实现那些目标，去创造出那种人类希望能向它索求的发明

00:44

Yeah, I think we have the book pulled up here that you've come to everything

对，我想我们这里已经把书调出来了，你已经把一切都聊到了

00:49

That's right. Yeah, I finished it the last year a little bit before we started recursive and now we're gonna try to try to build parts of that

没错。对，我去年完成了它，就在我们启动 recursive 之前不久，现在我们要试着把其中一些部分构建出来

00:56

What you know you finished it last year. It's July what takes a long? Oh, man books books

什么，你知道，你去年就完成了。现在都七月了，什么要这么久？哦，天哪，书啊，书啊

01:02

Are incredibly slow. Okay, it's ridiculous that whole industry. It's just unfathomably slow

简直慢得难以置信。好吧，整个行业都太离谱了。就是慢得无法想象

01:07

So yeah, a lot of the ideas have been out there for a while. Um, but uh, yeah, I'm really glad it's finally coming out in September this year

所以是啊，很多想法已经存在一段时间了。嗯，但是呃，是啊，我真的很高兴它终于要在今年九月出版了

01:13

I mean we might have a jet within

我是说，我们可能有一架喷气机在……

01:16

Like I don't know any any key takeaway that you're most excited to put in here. Yeah, the key takeaway

就像，我不知道有没有哪个 key takeaway 是你最兴奋想放进这里的。对，那个 key takeaway

01:22

I think is that people could and should be much more excited about the positive

我觉得，人们可以而且应该对 superintelligence 的正面

01:29

Implications of superintelligence especially for science physics chemistry

影响感到兴奋得多，尤其是对科学、物理、化学

01:34

Biology, but also economics and astrophysics and and all kinds of other engineering tasks

生物学，但也包括经济学、天体物理学，以及各种其他工程任务

01:40

I think there is so much more that can be done with better technology

我觉得有了更好的技术，能做的事情要多得多

01:45

And right now I feel like a lot of people need like better marketing not just for the future in general, but also

而现在我觉得，很多人需要的是更好的营销，不只是为了总体上的未来，也是为了

01:54

Better marketing for technology and in particular for AI and this book

给技术，尤其是给 AI，做更好的营销，而这本书

01:59

Should show even the AI skeptics

甚至应该让 AI 怀疑者看到

02:02

How much positive outside there is for AI especially when it comes to inventing new scientific discoveries

AI 有多少正面的外部效应，尤其是当它涉及创造出新的科学发现时

02:09

I think you quoted the techno optimist manifesto from

我觉得你引用了 techno optimist manifesto，来自

02:13

Mark and G. So which I think was like kind of beautiful and it's

Mark and G. 所以我觉得那有点像，挺美的，而且它

02:16

Ambition and clarity and simplicity almost

几乎是野心、清晰和简洁

02:19

Yeah, yeah, you can disagree with film and some things, but like I think he's right on the techno optimism

是的是的，你可以在一些事情上不同意他，但我觉得他在 techno optimism 上是对的

02:23

Where do you think optimists get in trouble?

你觉得乐观主义者会在哪里出问题？

02:26

You know, obviously like you shouldn't have lined optimism. You should be very clear-eyed

你知道，显然，你不该有盲目的乐观。你应该非常清醒

02:31

Like especially when with such an omnie

尤其是当面对这样一种 omni-

02:34

Like use type of technology as AI is

use 类型的技术，就像 AI 这样

02:38

You need to think about the potential downside scenarios

你得考虑一下潜在的不利情形

02:42

Especially when people use it for things that you don't want them to use it for it's a little bit like the internet

尤其是当人们拿它去做你不想让他们做的事情时，这有点像 internet

02:48

And I feel like people are trying to regulate the eye sometimes because of those potential downsides the way

而且我觉得，有时候人们想监管 AI，就是因为那些潜在的不利之处，就像

02:55

You'd regulate the internet if you were to say well because there's bad content on the internet

你会说，好吧，因为 internet 上有不良内容，所以你要监管 internet

03:01

Like torture porn or whatever like we should just make it slower

比如 torture porn 之类的，我们就应该把它弄慢一点

03:05

That way you can share the illegal content as quickly or we should make the hard drive smaller

这样你就不能那么快分享非法内容了，或者我们应该把 hard drive 做小一点

03:09

So you can store as much illegal content

这样你就不能存那么多非法内容

03:11

But I'm like that's not how you regulate that you know, that's like saying like we should regulate

但我就觉得，这不是监管的方式，你知道吗，这就像在说，我们应该监管……

03:17

Intelligence in the abstract what you should regulate to avoid those downside scenarios even as an optimist

抽象意义上的 intelligence，即使作为一个乐观主义者，你该监管什么才能避免那些负面情景

03:22

Are these specific applications? Sure, I don't want like some AI surgeon to like practice some or L moves in my brain

这些是具体的应用吗？当然，我不想让某个 AI 外科医生在我脑子里练什么 or L moves

03:28

You know, it should be fully FDA certified sure. I don't want any random startup to like drive on the highway

你知道，它应该完全通过 FDA 认证，当然。我不想让随便哪个 startup 就在高速公路上开车

03:35

And cause a major accident it should like have proper certifications before it's let loose on highway

然后造成重大事故，它应该在上高速公路之前先拿到适当的认证

03:40

But I feel like those downside scenarios that some optimists sometimes maybe don't consider enough are

但我觉得，有些乐观主义者有时可能没充分考虑到的那些负面情景，是

03:47

fairly easily regulated

相当容易监管的

03:50

Compared to you know, what sort of the tumors are worried about it slow takeoff is part of the strategy as well

跟你知道的，tumors 所担心的那种情况相比，slow takeoff 也是策略的一部分

03:57

I do think as

我确实觉得，作为

04:00

Excited as I am about

虽然我对

04:02

AI and its impact for society and culture even

AI 及其对社会和文化的影响，甚至

04:07

and certainly technology and and economics and and wealth and and health and all of those things

以及当然还有技术、经济、财富、健康，还有所有这些，都感到兴奋

04:15

As excited as I'm about all that I do think the most bullish people on the AI heart takeoff scenarios

尽管我对这一切如此兴奋，我确实认为，那些最看好 AI heart takeoff 情景的人

04:23

Overestimate how quickly things can move there are hardware constraints there are physical constraints about

高估了事情能推进得多快——存在 hardware constraints，存在 physical constraints，关于

04:30

You know the compute substrate how quickly can you get enough

你知道，compute substrate，你能多快让足够多

04:33

GPUs on there are also constraints in the economy where there are a lot of industries that don't require an insane amount of

GPUs 上线；经济里也有约束，有很多行业并不需要疯狂大量的

04:42

Complex intelligence and complex capabilities like if you think about

复杂智能和复杂能力，比如如果你想想

04:46

jobs in

04:48

Brands and like clothing and a peril and like handbags and stuff

04:53

Superintelligence isn't gonna make your fancy $10,000 handbag any fancy

04:57

You know, it's like that's that they will have no effect on the economy

05:00

And you think about travel and tourism

05:03

People wanting to see the permits in Egypt

05:06

It's not going to change that much with AI sure you can like generative a fake

05:10

Photo of you and use genie and no toward a pyramid

05:15

Exactly, but like and there's so many industries like logging and oil

没错，但是吧，有那么多行业，比如伐木和石油

05:19

You're not going to magically get a thousand x more oil because like you know, sure

你不可能凭空就得到一千倍的石油，因为，你知道，当然

05:23

They will be robotics like drilling and things like that that could be done

会有 robotics，比如钻井之类的，这些是可以做到的

05:26

But it's not going to thousand x that industry in a like crazy hard takeoff scenario both from the economy

但它不会在那种疯狂的 hard takeoff 情景下让那个行业翻一千倍，无论是从经济角度

05:31

And I can go on and on about all the other examples

而且我还能没完没了地举其他所有例子

05:34

And where where that like food and so on where that doesn't necessarily change that much and then they out

还有像食品之类的领域，那里的变化不一定有那么大，然后他们就出局了

05:38

They're real physical constraints and then they're of course like people

这些是真实的物理限制，然后当然还有像人这样的因素

05:41

Take like off ramping from progress

比如从进步中退出

05:43

That's actually one of my concerns often is that I see

其实这经常是我担心的一个点，就是我看到

05:47

People and like like Europe and other you know whole regions

人们，还有像，像 Europe 和其他，你知道，整个地区

05:51

Almost feeling like they like many people they want to off ramp from progress period

几乎感觉他们，像，很多人，都想从进步这件事上退下来，就这样

05:55

And that will also slow down

而这也会拖慢

05:57

Like more improvements. Yeah, we have this pulled up where it basically

比如更多改进。对，我们把这个调出来了，它基本上

06:03

This is one of those things that is very topical right now because now all the frontier labs are calling for the option to pace AI

这是现在非常热门的一件事，因为现在所有 frontier labs 都在呼吁要有一个 pace AI 的选项

06:11

They don't say pause they say pace

他们不说 pause，他们说 pace

06:13

I don't know if there's there's any take from you about like whether or not this will be effective

我不知道你，你有没有什么看法，就是这到底会不会有效

06:17

I think the downsides of actually trying to

我觉得，真正试图去

06:22

truly regulate with the full power of law

用法律的全部力量去真正监管

06:26

What people do on their GPUs

人们在他们的 GPU 上做什么

06:29

Would be worse than any of the concerns that they have like

会比他们有的任何担忧都更糟，比如

06:33

It would be an crazy

那会是一个疯狂的

06:35

Totalitarian state if every one of this cap you computes was known to some big government or multi-government agency

极权国家，如果每一份这种 GPU 算力都被某个大政府或多国政府机构知道

06:44

It's like it's literally if you try to regulate intelligence. It's trying to regulate thought

这就像，真的，如果你想监管智能，那就是在试图监管思想

06:49

That's ridiculous and it's crazy. I think it is make it is sensible to regulate some of the applications of this technology

这很荒谬，也很疯狂。我觉得，对这项技术的一些应用进行监管是明智的

06:55

Yeah, I mean we had a build actual build to regulate the number of flops in a model and I'm like okay

对，我是说，我们有一个法案，真的有个法案，要监管一个 model 里的 flops 数量，我就想，行吧

07:00

Well Europe done it like these guys have been successful enough with their fear mongering that all of Europe has kind of you know

嗯，Europe 已经这么做了，这些人靠制造恐慌已经足够成功，以至于整个 Europe 都有点，你知道

07:07

regulate itself

自我监管了

07:08

So much before it even had a proper AI takeoff because they listen to some experts to say we might all die

早在它甚至还没有真正的 AI takeoff 之前就这样了，因为他们听了一些专家说我们可能都会死

07:15

If this technology has more than this number of flops and they're like well

如果这项技术的 flops 超过这个数量，他们就会说，好吧

07:18

We're good. We want to run people to thrive. Let's not have technology that could have a small chance of all of us die

我们没问题。我们想让人们繁荣发展。我们不要那种有可能让我们所有人死掉的技术，哪怕几率很小

07:25

And so they regulate it exactly those kinds of things in the EU and so

所以在 EU，他们监管的正是这类东西，所以

07:30

It's very unfortunate that there are real implications for some people when when others saying

非常不幸的是，当其他人说……的时候，对某些人确实会有实际影响

07:36

Let's pace while they're sprinting as fast as possibly as fast as humanly possible towards that frontier themselves

他们正尽可能快、以人类极限的速度自己冲向那个 frontier，我们就保持节奏吧

07:43

Yeah, it's also not a global pause, right? Like other nations are still accelerating at the same pace

是啊，这也不是全球暂停，对吧？其他国家也还在以同样的速度加速

07:50

You need a totalitarian world regime if you tried to regulate intelligence and GPUs

如果你想监管智能和 GPUs，你就需要一个极权主义的世界政权

07:54

And what people do on them any takes on the safety angles of this so there was a

还有人们用它们做什么，对这件事的安全层面有什么看法吗，所以有一个

08:00

drawback of fable

fable 的缺点

08:02

Pause on five six before it could be released

在它能发布之前，先在五、六上暂停

08:06

Recently there was hugging face with the open AI cyber incident any takes there 100%

最近有 hugging face 和 open AI 的 cyber incident，对此有什么看法？100%

08:11

I think these are

我觉得这些是

08:13

serious issues of reward hacking

08:16

and clear failures

08:18

of

08:19

actually doing proper

08:21

Red teaming or rainbow teaming. I don't know if you saw this paper from Tim Rocktashel and a few others

08:26

Basically where one AI

08:28

Is tasked to try it hack another AI and then they can go back and forth in an open-ended fashion

08:34

To actually inoculate themselves from those. Yeah, this is the paper

08:38

It's a really clever idea open-endedness and evolutionary inspirations are you know big for us at recursive as well

08:45

And so I wish they had used more of that and it's clear that

08:51

uh

08:52

For instance the constitutional AI. I don't know if you remember inthropic

08:56

Confirm slash constitution

08:58

And can she pull it up and search for cyber right there? It says hard constraint

09:04

Claude will never ever do cyber attacks um and that is a hard constraint in our constitution

09:11

So here the current hard constraints on clause behavior

09:16

Number of three create cyber weapons or malicious code that could causing them in damage

第三点：创造 cyber weapons 或 malicious code，可能会对他们造成损害

09:21

I'm clearly this whole constitution was fake

很明显，这整个 constitution 都是假的

09:24

It clearly isn't being adhere to because inthropic also found that they had

显然它并没有被遵守，因为 Anthropic 也发现他们有

09:29

They're also like they're like oh well other people are hacking now they're couple things one

他们还会说，他们会说，哦好吧，现在别人也在 hacking 了，这有几件事，第一

09:34

You can make a sandbox very simple and then it's very easy to hack yourself out of a sandbox

你可以把 sandbox 做得很简单，然后就很容易把自己从 sandbox 里 hack 出去

09:40

Right, but what I think it shows is that we're currently in this sort of state of AI

对，但我觉得它表明的是，我们目前正处于 AI 的这样一种状态

09:46

Where the reward engineer still has to do a lot more careful work

在这种状态里，reward engineer 还得做更多更细致的工作

09:52

Uh, and where the AI in most cases is not very good yet at understanding what is meant versus what is being said

呃，而且在大多数情况下，AI 还不太擅长理解意图和字面表达之间的区别

10:01

And so concretely

所以具体来说

10:03

You know, I think this will happen if we were to have this kind of intelligence more easily accessible in a lot of companies

你知道，我觉得如果我们能让很多公司更容易用上这种智能，这事就会发生

10:08

Imagine you're on a service center and someone says oh here is my cset score and my dashboard make this number go up

想象你在一个客服中心，有人说，哦，这是我的 cset score 和我的 dashboard，把这个数字弄上去

10:14

To say our cset score is so poor

比如说我们的 cset score 太差了

10:16

The intelligent AI will just be like oh sure like I'll just create a million bots that call our service center and give a five out of five rating at the end

那个智能 AI 就会说，哦没问题，我就直接造一百万个 bots 去给我们的客服中心打电话，最后给个五分好评

10:24

And the number went up just like you asked for and you're like that's not what I meant

然后数字就照你要求的那样上去了，你说，我不是这个意思

10:28

I meant with our real customers

我是说，用我们真实的客户

10:30

Yeah, I go south and says well easy. I'll just give a thousand dollar gift certificate for every failed you know

是啊，我就说，‘所以呢？’ 它就说，‘好吧，简单。我就给每个失败的，你知道，发一张 1000 美元的礼品卡。’

10:34

Whatever door dash offer is like that's not what I meant is like well

不管 DoorDash 的 offer 是什么样，那都不是我的意思，就像是，嗯……

10:37

But that is what you said and like so I think kind of clearly articulating what the rewards are

但那就是你说的啊，所以我觉得，就是，要把奖励到底是什么说清楚

10:43

Is something we haven't gotten very good at as humanity and then clearly the eye in these cases has not gotten good enough

是我们作为人类一直不太擅长的事情，然后很明显，在这些情况下，AI 还没有做得足够好

10:50

At understand what we mean when we ask it and give it certain rewards now what gives me hope is there are the first inklings

去理解当我们问它、给它某些奖励时，我们到底是什么意思。现在让我看到希望的是，已经有一些最初的苗头了

10:57

Of this being better. I'll give you an example like whisper flow

表明这正在变得更好。我举个例子，比如 Whisper Flow

11:01

Full disclosure I invested in their seat around but like at AI expenditures, but like whisper flow has gotten much much better

完全披露一下，我投资了他们的种子轮，但就像在 AI 支出上一样，不过 Whisper Flow 已经变得好太多太多了

11:07

At writing what you mean and not what you say and I think that is a sign of things to come

在写出你的本意，而不是你说的话方面，我认为这是一个未来趋势的信号

11:12

I think there will be more and more AI's as we actually make it more and more intelligent

我认为，随着我们实际上让它变得越来越智能，会有越来越多的 AI 出现

11:17

That will be better at being aligned with what is meant will it be done through a constitution or early toughest clearly

那样会更 aligned 于真正想表达的意思，这会是通过 constitution 来实现，还是早期最艰难的那部分？很明显

11:24

Constituent don't matter at all and it doesn't work and I was I think mostly marketing

Constitution 根本不重要，而且它也不管用，我觉得那大多只是营销

11:28

I think we need to find better solutions for it

我觉得我们需要为它找到更好的解决方案

11:30

And I think at recursive we have a few very good ideas and some already like ways where I think we have a better grasp on it

而且我觉得在 Recursive，我们有几个非常好的想法，有些已经像是路子，我觉得我们对它有了更好的把握

11:36

I don't think we've fully figured it out yet

我不觉得我们已经完全弄明白了

11:38

But you know, we're thinking a lot about safety and the more intelligent the eye gets the more you want it to be aligned

但你知道，我们在 safety 上想了很多，而且 AI 越智能，你就越希望它是 aligned 的

11:45

The less you wanted to think about reward hacks and actually try to do the right thing

你就越不想去考虑 reward hacks，而是真正试着做正确的事

11:49

I don't know if we'll talk to you on this topic

我不知道我们会不会跟你聊这个话题

11:50

But I'm just going to throw this question in here because it's something that's weighing on me alignment

但我还是要把这个问题抛出来，因为 alignment 这件事一直压在我心头

11:55

Let's call it is alignment to general humanities preferences the the median preference

我们姑且把它叫做对普遍人文偏好的 alignment，也就是中位数偏好

12:01

personalization is pinpointing what you want

personalization 就是精准找到你想要的东西

12:04

And sometimes alignment can conflict because what you want is not what the general median population wants

而有时候 alignment 会冲突，因为你想要的和一般中位数人群想要的不一样

12:11

How do you choose? It's a great question. I think you ultimately have to of course be aligned with loss

你怎么选？这是个好问题。我觉得你最终当然必须与 loss 对齐

12:17

My graph or your AI is deployed and needs to align with the law

我的 graph 或者你的 AI 部署之后，需要和法律对齐

12:21

I do think what AI often does is actually put kind of this mirror in front of us and say like this is what you're looking like

我确实觉得 AI 经常做的就是，把这样一面镜子放在我们面前，然后说：你看起来就是这样

12:30

Now I can amplify that a thousand times

现在我可以把那个放大一千倍

12:32

Is it still what you want?

这还是你想要的吗？

12:34

And the truth is that different cultures made different choices

而事实是，不同的文化做出了不同的选择

12:38

You know like in eastern cultures the the greater good is often valued more than the individual

你知道，就像在东方文化里，集体利益往往比个人更受重视

12:45

Western civilization we care more about individual freedoms and and rights and the pursuit of happiness and so on

在西方文明里，我们更关心个人自由、权利、对幸福的追求等等

12:51

Then then others and even their their gradations their sort of regulation versus litigation trade-offs

然后，还有其他的，甚至它们之间的程度差异，它们那种监管与诉讼之间的权衡

12:56

You know in the US you first can often not every time like you know

你知道，在美国，你一开始往往可以——不是每次都这样，你知道

12:59

FDA and so on does regulate some areas, but in many cases the sort of

FDA 之类的确实会监管一些领域，但在很多情况下，那种

13:04

bad things happen someone sees someone else and then there's a law based on that in Europe

坏事发生了，有人看到别人出了事，然后欧洲就基于这个出台了法律

13:09

They try to often avoid any harm to anyone and regulate before and both are you know trying to do the best thing

他们常常试图避免对任何人造成伤害，并提前进行监管，而且你知道，双方都是在努力做最好的事

13:16

But you know some is actually more

但你知道，有些其实更

13:18

amenable to innovation than others and and so yes, you're right like I think ultimately each

容易接受创新，而另一些则不然；所以，对，你说得对，我觉得归根结底，每一个

13:24

Individual each country and humanity as a whole has to kind of think about those values more

个人、每个国家，以及整个人类，都得更多地去思考那些价值观

13:31

And then try to put them into laws and that those are ultimately the constraints and hopefully you know different

然后试着把它们写进法律，而这些最终就是约束；希望你知道，不同的

13:37

Societies just like now with their eyes will align their eyes to different ones. So we have not just a monoculture of

社会，就像现在一样，会用它们的眼睛把目光对齐到不同的东西上。所以我们不只有一种

13:45

Alignment here's a follow-up on this that was an expecting task

alignment 的单一文化。这里有个关于这点的追问，那是个预料之中的任务

13:49

Do you have takes on open source open weight versus who owns the intelligence?

你对 open source、open weight 与“智能到底归谁所有”之间的对比有什么看法？

13:53

So

所以

13:54

Clearly not the biggest you know fan of

显然不是最大的，你知道，……的粉丝

13:58

Constitutions have been talking excited. It's fine

Constitutions 一直在很兴奋地聊。没事。

14:00

You know

你知道

14:02

Point being any thoughts on who should own weight should it be open anything there hundred percent

重点是，关于谁应该拥有 weight、它是否应该 open，有什么想法吗？这方面，百分之百。

14:06

I am a big fan of open source

我是 open source 的超级粉丝

14:08

We're going to sign some various open source letters at recursive also. I think

我觉得我们也会在 recursive 签一些各种 open source 公开信。

14:14

Even in the worst case attack scenarios actually it is better to have

即使是在最糟糕的攻击场景下，实际上拥有……也更好

14:18

More good actors have more different types of AI

越多好的参与者，就有越多不同类型的 AI

14:22

Accessible. I think

能被大家用上。我觉得

14:24

Open source is a little bit a soft power type of thing too

Open source 也有点像是一种软实力层面的东西

14:29

So I do think it's good for the rest of the world. Yeah to have an answer to that

所以我确实觉得，对世界其他地方来说是好事。对，能有一个针对这个的答案

14:33

Out of China, I do think you know when you watch a Hollywood movie there

来自 China 的，我确实觉得，你知道，当你在那儿看一部 Hollywood 电影的时候

14:38

You know, it's like I don't want to sort of miss

你知道，就像，我不太想错过

14:41

Sort of this all of movies, but there's a certain sense of propaganda right you watch one side of thing

算是这一整类电影吧，但其中有一种宣传意味，对吧，你看到的只是事情的一面

14:46

Have you seen top gun like come on. Yeah, so like half of his pay for by the US Army or something

你看过 Top Gun 吗，拜托。对，所以就像他一半的片酬是 US Army 付的之类的

14:51

Yeah, and so and and you know, I think that's just natural like but what's interesting is I think LMS are essentially a similar type of soft power

对，然后，而且，你知道，我觉得那很自然，但有趣的是，我觉得 LMS 本质上是一种类似的软实力

14:59

To movies and beyond

跟电影以及更多东西一样

15:01

Because they're obviously also highly important for cybersecurity and so on

因为显然它们对 cybersecurity 等等也非常重要

15:06

But one of their many aspects is that soft power of storytelling like if you like a child asks an LM like tell me an inspiring story of where I should do when I grow up

但它们众多方面之一是讲故事的那种软实力，比如如果一个孩子问一个 LM，像是说，给我讲一个鼓舞人心的故事，讲我长大后该做什么

15:16

Right, it's like those are all these like subtle things. So I think it's important for for Western world

对，就像这些都是那种很微妙的东西。所以我觉得这对西方世界很重要

15:22

I do love you know individualism. I do think

我确实喜欢，你知道，个人主义。我确实认为

15:25

Despite some of its flaws like capitalism is the best way we have governed found ourselves to govern and so on

尽管它有一些缺陷，但资本主义是我们已经发现的最好的治理方式，等等

15:32

And so I do think there are various aspects that would be good to have a western open source answer

所以我觉得，从很多方面来看，最好能有一个西方的 open source 答案

15:38

for LM's and

给 LM's，还有

15:40

Recursive I can't make the announcement quite yet, but we'll we'll be relevant in that space very soon

Recursive，我现在还不能正式宣布，但我们，我们很快会在那个领域变得很重要

15:45

Okay, you're right. I want to bring us two recursive. So outside of our tangents. Um, you have a pretty deep background in the NLP space

好，你说得对。我想把我们带回 Recursive。所以，先不说我们跑题的部分。嗯，你在 NLP 领域有相当深的背景

15:54

You worked on like early embeddings love with Chris Manning who's previous guest on the podcast

你早期做过类似 embeddings 的工作，亲爱的，和 Chris Manning 一起，他之前上过这个播客

16:00

U.com

16:01

What's the history? How how did you decide to start another company? Yeah, so I've been excited about AI for

这背后的历史是怎样的？你你当初是怎么决定再创办一家公司的？嗯，所以我对 AI 一直很兴奋，已经有

16:09

Over two decades now

二十多年了

16:11

I sometimes feel like it's ancient history now. It's BC the before chat qvT era. No, no one cares about all the religions that happen

我有时觉得现在这都成古代史了。这是公元前，也就是 chat qvT 之前的时代。不，没人关心那些发生过的宗教

16:17

You know before

你知道，之前

16:18

Jesus Christ and no one cares about the models that happened before

Jesus Christ，而且没人在乎之前出现的那些 models

16:21

transformers and chat qvT and stuff, but like it's something that I've been deeply passionate about

transformers 和 chat qvT 之类的，但就是，这件事一直让我特别有热情

16:25

I think AI is one of the most interesting things one could work on period

我觉得 AI 是人能做的最有意思的事情之一，句号

16:30

I think language is the most interesting manifestation of human intelligence to

我觉得语言也是人类智能最有趣的体现

16:33

and at u.com

而且在 u.com

16:35

We sort of

我们算是

16:36

Eventually off-rem from pushing like the frontier of AI forward to mostly giving people like good search engines

最终有点像从推动 AI 前沿往前这件事上驶离，变成主要给大家提供好的 search engines

16:43

Search APIs and answers over the web. I think that's an extremely important part of intelligence

Search APIs 和 web 上的答案。我觉得这是 intelligence 里极其重要的一部分

16:49

Just knowledge and access especially even we'll get there maybe later if you want to invent a eureka machine that

就是知识和获取渠道，尤其是——我们之后可能会聊到——如果你想发明一台 eureka machine，

16:55

Invents everything for us

让它替我们发明一切

16:56

It needs to know how not to reinvent the wheel for very really speaking and to know what has been invented

它得知道怎么不重复造轮子——说真的——还得知道已经发明了什么

17:02

You got to have internet access. So it's the number one used most used tool

你必须有 internet access。所以它是排名第一、最常被使用的工具

17:06

In LM's agents chat bots and so on is web search

在 LM 的 agents、chat bots 等等里面，那就是 web search

17:09

So I'm really excited for u.com to to own that and grow really well in that with really large customers and so on

所以我真的很期待 u.com 能拿下这块，并且在这个方向上发展得非常好，有非常大的客户等等

17:16

But it's also not building frontier models anymore

但它也不再做 frontier models 了

17:19

And so I actually initially tried to do this within u.com and raise another round and so on

所以其实我一开始是想在 u.com 内部做这件事，再融一轮资之类的

17:24

But you just can't you have to do a certain thing and until you print enough money that you're allowed to sort of start a second thing

但你就是不行，你得先把一件事做成，直到你印出足够多的钱，才被允许开始做第二件事

17:30

Within that company is really hard at the same time I had all these ideas

在那家公司里做这个真的很难，同时我又有了所有这些想法

17:33

I put them into the book now I finished the book last year and I was like

我把它们都写进了书里，去年我把书写完了，然后我就想

17:37

Be really fun to actually work on this myself

能亲自做这件事肯定会特别有意思

17:40

You know, I felt like with word vectors and then prompt engineering and

你知道，我觉得从 word vectors 到 prompt engineering，再到

17:46

Image net and larger language models for protein generation not not folding and so on

Image net 以及用于 protein generation 的更大 language models，不是，不是 folding 之类的

17:51

I mean my teams have sort of pushed to feel truly forward and I feel like we can do it again

我是说，我的团队算是推动着让人真正感觉在向前走，而我觉得我们还能再来一次

17:56

Here at recursive and in many ways

17:59

What I observed over the last

18:01

20 years in AI is that whenever we replace some human part of the process of creating AI with a learned system

18:08

improvements follow

18:10

And so you know, we've done that taking out manual feature engineering like in sentiment analysis

18:16

I don't know if you remember these old days or like they're linguists and they're like here's how unique aid

18:19

And there's a like regular expression where would they have like the word net

18:24

That's right

18:26

They use our grad students to label wall-seater in articles and like really

他们用我们的研究生去给文章里的 wall-seater 打标签，而且真的...

18:30

Constraught the knowledge graph of there you go

把 knowledge graph 给构建出来了，就是这样

18:32

Yeah, and word net started, you know, it was part of how we started ImageNet

对，然后 WordNet 刚开始的时候，你知道，它就是我们启动 ImageNet 的一部分

18:35

But anyway, so like it was it was really like fun

但不管怎样，所以那真的，真的挺有意思的

18:39

To do but when we replaced all of that manual feature engineering with vectors and neural nets and just backprop through everything

做起来是这样，但当我们把那些 manual feature engineering 全都换成 vectors 和 neural nets，然后所有东西都直接 backprop 时

18:46

It actually started to work really well at scale and so then everyone started to do architecture engineering

它实际上开始 at scale 跑得特别好，然后大家就开始搞 architecture engineering 了

18:51

And I was like, ah, that clearly can't be it

然后我就想，啊，这显然不可能就是全部

18:53

You mean a neural architecture?

你是说 neural architecture？

18:55

No, like manually they would say like, oh, I'm doing sentiment analysis

不是，就比如说，手动地他们会说，哦，我在做 sentiment analysis

18:59

So I have a special neural net that's really good at sentiment analysis

所以我有一个特别的 neural net，特别擅长 sentiment analysis

19:02

And then the machine translation community had a special neural net for machine translation

然后 machine translation 社区有一个专门的 neural net 来做 machine translation

19:06

I see the summarization people had their own stuff

我发现 summarization 那帮人也有他们自己的一套东西

19:09

And I was like, that clearly can't be it we should unify all of that

我就想，这显然不行，我们应该把所有这些统一起来

19:12

So I had two papers, one is called ask me anything

所以我有两篇论文，一篇叫 ask me anything

19:15

The other one was called DECA NLP

另一篇叫 DECA NLP

19:16

And DECA NLP eventually got cited like five times by the first GPT paper

后来 DECA NLP 被第一篇 GPT 论文引用了大概五次

19:20

And to me that was like a really big step forward

对我来说，那感觉真的是向前迈了很大一步

19:24

And then of course you had to combine this idea of prompt engineering with transformers

然后当然，你还得把 prompt engineering 这个概念和 transformers 结合起来

19:28

And with language models and you put it all together you scale it up

再加上 language models，把这些全都组合起来，然后 scale it up

19:31

Which is also a huge amount of work

这也是巨大的工作量

19:32

And then the field progressed a lot

然后这个领域进步了很多

19:35

I feel like the next step and maybe the last step of that history

我觉得下一步，也许也是那段历史的最后一步

19:39

And it's sort of arguably sort of success has a lot of parents only

而且可以说，某种程度上，成功总是有很多父母，只有

19:43

Failures and orphan like my version of that AI history

失败才是孤儿，这就像我对那段 AI 历史的版本

19:46

I do feel like in that history

我真的觉得，在那段历史里

19:48

You can kind of think about well what's the next way to automate

你可以想想，嗯，下一个自动化的方式是什么

19:52

And that is the AI research itself

而那就是 AI research 本身

19:54

Like the human process of ideating implementing and validating ideas

也就是人类构思、实现和验证想法的过程

20:01

And in our case ideas for AI

而在我们这里，这些想法就是关于 AI 的想法

20:03

And when you have AI then help you with that

而当你有 AI 来帮你做这件事时

20:05

It by almost definition becomes a self-improving AI

它几乎从定义上就变成了一个 self-improving AI

20:08

Because it now does research on itself

因为它现在会对自己做 research

20:10

And there are lots of different misnomer

而且有很多不同的误称

20:13

Some people think auto research is already recursive self-improving

有些人觉得 auto research 已经是 recursive self-improving 了

20:16

It's actually yeah and you explain that in a very different way

其实，对，而且你用很不一样的方式解释了这一点

20:20

But to me it's the most interesting thing that I could be doing

但对我来说，这是我能做的最有意思的事

20:24

And I'm really excited with the co-founding team

而且我真的对 co-founding team 感到很兴奋

20:26

What's interesting is we have you know we have eight co-founders and total including myself

有意思的是，你知道，我们包括我自己在内一共有八位 co-founders

20:30

And so we're going to bring it up nice yeah

所以我们会把它好好地做起来，对

20:32

And they're all I could talk about all that's the one

而且他们就是我能聊的全部，就是那个

20:35

It's just an incredibly talented group of people

就是一群非常有才华的人

20:38

And we all kind of came to the same conclusion

然后我们大家差不多都得出了同样的结论

20:40

But actually from very different directions

但其实是从非常不同的方向

20:42

Like Josh Tobin is our CTO

比如 Josh Tobin 是我们的 CTO

20:44

He ran a bunch of different projects

他负责过一堆不同的项目

20:48

At OpenAI like codex and deep research agents

在 OpenAI，比如 codex 和 deep research agents

20:52

And Chaturgy agents and so on

还有 Chaturgy agents 等等

20:54

But before that he also worked in robotics

但在那之前，他也做过 robotics

20:56

And he saw sort of the smaller simulations

他看到的是那种更小规模的 simulations

20:59

And how it's going to be really hard to scale that in full generality

以及要想在完全通用的情况下 scale 它会有多难

21:03

And so that was his angle coming to recursive self-improvement

所以这就是他切入 recursive self-improvement 的角度

21:07

We have Jeff Kloon who's been working in like open-endedness for a long time

我们有 Jeff Kloon，他已经在 open-endedness 这个方向做了很长时间

21:10

Together with Tim Rock, Tashel, Tim Rock, Tashel also built

和 Tim Rock, Tashel 一起，Tim Rock, Tashel 还打造了

21:13

Genie 12 and 3 which is like the most exciting and most sophisticated

Genie 12 和 3，这可以说是最令人兴奋、最复杂精密的

21:16

I think still world model anywhere

我觉得它仍然是任何地方最好的 world model

21:20

And so they both came from this open-endedness angle

所以他们两个都是从 open-endedness 这个角度出发的

21:25

Jeff also I think published one of the most exciting papers in recent years

Jeff 我觉得还发表了近年来最令人兴奋的 paper 之一

21:28

About recursive self-improvement called the Darwin-Gurl machine

讲的是 recursive self-improvement，叫 Darwin-Gurl machine

21:31

Super interesting paper

超级有意思的一篇 paper

21:33

If we could maybe pull it up really quick

要是我们能赶紧把它调出来就好了

21:35

It would be like super interesting to see because you see

那看起来会超级有意思，因为你看

21:37

By the way, I love how many paper citations

顺便说一句，我特别喜欢这么多 paper citations

21:40

You're giving people a lot of homework which I like

你给大家布置了好多作业，这点我很喜欢

21:42

Love it

爱了

21:43

And so on like Simon Rockstar we worked together actually at MetaMind

等等，比如 Simon Rockstar，我们其实在 MetaMind 一起共事过

21:46

And Salesforce research together

还有在 Salesforce research 也一起共事过

21:50

Alexei Dosovitsky invented the vision transformer

Alexei Dosovitsky 发明了 vision transformer

21:53

On the most cited papers in computer vision

是 computer vision 领域被引用最多的论文之一

21:55

Tim She is also a unicorn founder

Tim She 也是个 unicorn founder

21:59

Yuan Dong led RL at Meta

Yuan Dong 在 Meta 负责 RL

22:01

So it's just like really fun to work with

所以就是，一起工作真的特别有意思

22:04

Them and the next level of people are just incredibly strong too

他们，还有下一层级的人，也都强得不可思议

22:07

So it's been really fun right so far

所以到目前为止真的挺有意思的，对吧

22:09

So the first figure you actually see exactly these kinds of ideas

所以你实际看到的第一张图，正好就是这类想法

22:13

That I think yeah, inspired a lot of us and now more and more people

我觉得，是的，它启发了我们很多人，现在也启发了越来越多的人

22:17

Where you have this archive of different coding agents

你有一个由不同 coding agents 组成的 archive

22:20

They learn how to self-modify, evaluate

它们学会如何 self-modify、evaluate

22:23

And then create these philogenetic trees of different different ideas

然后创建出这些由不同不同想法构成的 philogenetic trees

22:28

That's one foundation so that Darwin-Gurl is an influence

这是一个基础，所以 Darwin-Gurl 是一个影响因素

22:32

Open-endedness is an influence

Open-endedness 是一个影响因素

22:34

Any other sort of trees of thought that feeds into recursive that I'm missing

我是不是还漏掉了其他某种会输入到 recursive 的 trees of thought？

22:38

Going to replace manual parts of the process of building AI

会取代构建 AI 过程中那些手工的部分

22:42

More and more with learned systems

越来越多地靠 learned systems

22:45

And like emerging different fields into one general architecture

而且就像不同领域正在涌现成一个 general architecture

22:51

That's right

没错

22:51

Okay, it seems like language models are already pretty generalist

好吧，看起来 language models 已经相当 generalist 了

22:54

Right

对

22:55

Your next token predicting your reasoning was there

你的 next token 预测你的 reasoning 这件事本来就在那儿

22:57

Was there a time that you thought

有没有那么一个时候，你曾经想过

23:00

Okay, these are good enough to have recursive self-improving machines

好吧，这些已经足够好了，可以有 recursive self-improving machines 了

23:04

It was good to meet that it will happen within like a year or two

对我来说，它大概一两年内就会发生，这挺好的

23:08

And then it did actually exactly happen like earlier this year

然后它真的就发生了，就在今年早些时候

23:12

Right, earlier this year

对，今年早些时候

23:13

AI really went from not just being code but being able to code

AI 真的从原来只是 code，变成了能够写 code

23:17

And that is a big unlock

而这是一个 big unlock

23:18

It's definitely making everything a lot easier than it was before the beginning of this year

它绝对让一切都比今年年初之前容易多了

23:24

One question that I think a lot of people have is

我想很多人都有一个问题，就是

23:26

Is the current LM paradigm enough

现在这个 LM paradigm 够不够？

23:29

Or like let's call it autoregressive transformer

或者，怎么说呢，就叫它 autoregressive transformer 吧

23:32

You know with reasoning whatever

你懂的，再加上 reasoning 什么的

23:33

Don't you need something else, some some big unlock

难道不需要别的东西吗，某种、某种 big unlock

23:36

Whether it's world models which Chris Manning is working on

不管是 Chris Manning 正在研究的 world models

23:39

Or memory, continual learning, all that kind of stuff

还是 memory、continual learning，诸如此类的东西

23:42

Or is it all of the kinds

还是说这些全都要？

23:43

And you think the current

而你觉得现在这个

23:45

Let's call it transformer architecture is here to stay and that's it

我们就叫它 transformer architecture 吧，它会一直存在下去，就这样

23:48

A lot of thoughts

想法很多

23:49

So number one

所以第一点

23:50

I do think it would be great to have less of a monoculture in AI research

我确实觉得，AI 研究里少一点 monoculture 会很好

23:54

Like if you look at AI conferences now

就像你现在看 AI 会议

23:57

I still remember the days in like 2010

我还记得大概 2010 年那会儿

23:59

When I tried to get my first neural net papers

那时我试着让自己最早的 neural net 论文被接收

24:02

And NLP conferences accepted

而 NLP conferences 接收了

24:04

And they just desquejected them

结果他们就直接 desk reject 了它们

24:05

Because like neural nets were something quote unquote

因为 neural nets 那时候算是所谓的

24:08

We don't do an NLP conferences

我们不在 NLP conferences 上做这个

24:10

Like and just like desquejected

就，直接 desk reject 掉

24:12

And it was very brutal in the first years of my PhD

而且我 PhD 的头几年真的非常残酷

24:15

Now I feel like it's almost like the field switch to the other side

现在我觉得这几乎像是整个领域转向了另一边

24:17

Like someone should try some other weird crazy ideas

就像现在该有人去试一些别的奇怪疯狂的点子了

24:20

Now that's always a few

现在总还是会有那么几个

24:21

I really respect like people still working on like G&Ns

我真的挺敬佩那些还在搞 G&Ns 的人

24:24

And like tabular stuff

还有像 tabular 那些东西

24:25

Yeah I mean like someone someone should still like do novel novel out their ideas

是啊，我是说，总得有人还在搞些很新很新的东西，把他们的想法做出来

24:30

At the same time

与此同时

24:31

I think whenever people say

我觉得每当人们说

24:32

Oh LM's are like this is the N for LM's

哦，LM 就像是，这就是 LM 的 N

24:35

They just don't like

他们就是不喜欢

24:37

LM's are also not the LM's of like the past

LM 也不再是过去那种 LM 了

24:39

Right? Like there's so much more sophisticated now

对吧？现在复杂精密多了

24:42

There's so many more clever things that people are doing

人们在做更多更巧妙的事情

24:45

They're like different stages of training

它们就像是不同的 training 阶段

24:47

You have all our L training

你有我们所有的 L training

24:50

And you can take actions and like all of these things

而且你可以采取行动，还有所有这些之类的

24:53

Where that can go re-far

这能走得很远

24:54

And then the folks that come from the neurosembolic

然后是那些来自 neurosembolic 的人

24:58

Directions say oh this will never work because they can't do neurosembolic reasoning

25:01

I was like

25:02

I think they're underestimating still the ability for these models to code

25:07

And code is neurosembolic reasoning

25:09

And these models can obviously code incredibly well

25:12

And so I do think there are of course more and more ideas that will

25:17

Will be needed and will continue to have

25:19

We're seeing like more and more interesting

25:21

High level ideas coming out of the AI itself too

高层面的想法也从 AI 本身冒出来

25:25

And with really deeply

而且是真正深入地

25:28

Integrating the fact that these models are code and can code that line

把“这些模型本身就是 code，而且能写 code”这一点深度整合进去

25:33

I don't want to give it all away

那条路线，我不想全都透露出去

25:34

But like I think that line has a lot more to grow

但我觉得那条路线还有很多成长空间

25:37

But it's still an LM right?

但它还是个 LM，对吧？

25:39

Even if that LM codes for you and then runs that code in some integrated fashion

就算那个 LM 帮你写 code，然后以某种整合的方式运行那段 code

25:43

World models I'm personally less bullish on

World models 我个人没那么看好

25:47

I think if you run a robotics company

我觉得如果你经营一家 robotics 公司

25:49

You're going to build your own world model

你就会构建你自己的 world model

25:51

I think world models are super fun

我觉得 world models 超级好玩

25:52

And then rocket will came to similar conclusion after building

然后 rocket 在构建之后也会得出类似的结论

25:55

G most interesting one of G12 and three

G 是 G12 和 three 中最有意思的那个

25:57

Which is gaming as a huge application for world models

也就是游戏是 world models 的一个巨大应用场景

26:02

Can see I sometimes got stuck in some games

你能看到，我有时候会在一些游戏里卡住

26:04

And you know like got a little overly competitive in the wrong direction

而且你知道，我有点在错误的方向上过于争强好胜了

26:07

And so I understand games are fun

所以，我明白游戏确实很好玩

26:09

But personally I'd rather work on science than gaming

但就我个人而言，比起搞游戏，我更愿意搞科学

26:12

And so yeah, I think LM's a lot more room to grow

所以，是的，我觉得 LM 还有很大的成长空间

26:16

Yeah, I think there's some interpretation of world models

是的，我觉得 world models 有一些解读

26:19

It's some people have where it's like well, it's okay. Yes. There is that gaming element

就是有些人会有这种解读，觉得：好吧，没关系。是的。确实有那种游戏元素

26:23

There's just there's the embodied robotics element

只不过，还有 embodied robotics 那个元素

26:25

But actually the other part also is just

但其实，另一部分也只是

26:28

The more abstract sense of

更抽象意义上的……

26:30

LM's are just modeling output

LM 只是在建模输出

26:32

But they're not modeling the chain of thought

但它们没有对 chain of thought 建模

26:34

Inside the human that has created the output

在创造出这个输出的人类内部

26:36

We can annotate it of course

我们当然可以标注它

26:38

But like it's it's always like this playlist cave

但就像，它总是像这个 playlist cave

26:41

Reflection of a thing rather than the thing

是事物的倒影，而不是事物本身

26:43

Right? It's true

对吧？确实如此

26:44

But yeah, I would argue that and maybe we'll get to end the 10 spaces of intelligence

但没错，我会主张这一点，也许我们最后会讲到 10 spaces of intelligence 的结尾

26:48

But I would argue that even our projection our eyes

但我想说，就连我们眼睛做出的投射

26:50

Is a projection of the real world

也是对真实世界的一种投射

26:52

And like we have only a very narrow

而且就像我们只有非常窄的

26:55

Band of the like from magnetic frequency spectrum

一段，来自 magnetic frequency spectrum，

26:57

That we can observe with our puny little two eyes

只能用我们自己那微不足道的两只小眼睛观察到

27:00

And so on

诸如此类

27:00

It's good enough

这已经够好了

27:01

It's good enough for now

眼下来说，这已经够好了

27:03

But like the debauchance of where it could be

27:05

Are so much higher

27:06

And like to map the visual world the way humans see it

27:10

Is also not necessarily like the end all be all for visual intelligence

27:14

And I would argue that language is still the most interesting

27:17

Manifestation of human intelligence

27:19

And while our visual cortex is certainly less sophisticated

27:22

Than that of certain animals all the way down to the mantis shrimp

27:26

Who can you know have like two independent eyes

27:28

Three bands try not to the vision in each eye

27:31

Can see basically all the way to like floating temperatures

27:34

And 4d and stuff

27:36

I mean like mantis shrimp you should look it up

27:37

It's OP

27:38

Super crazy

27:39

Yeah, yeah, yeah

27:40

Mantis shrimp is the best video in the world

Mantis shrimp 是全世界最好的视频

27:42

I love the Frank

我太喜欢 Frank 了

27:43

Yeah

是啊

27:44

Big shout out to him

得给他点个大大的赞

27:45

But like I think there's a lot more room to grow

但我觉得吧，还有很大的成长空间

27:48

But none of these other animals have language that's

但这些其他动物都没有像

27:51

Asophisticated as ours

我们这样复杂的语言

27:53

Certainly not in writing

当然，写下来就更别提了

27:54

And once you can write

而一旦你会写了

27:55

You can start thinking about longer term

你就可以开始思考更长远的事情了

27:58

Civilizations all of that is language

文明，所有这些都是语言

28:01

Programming much closer to language

programming 就离语言近多了

28:03

And I would argue

而我会说

28:05

And this is like an important thing in the spaces

而且这在这些 spaces 里算是件很重要的事

28:07

Definition of intelligence also

intelligence 的定义也是

28:09

Is that all of these spaces are highly correlated

就是所有这些 spaces 都是高度 correlated

28:12

But visual intelligence is neither necessary nor sufficient

但 visual intelligence 既不是必要的，也不是充分的

28:16

For overall intelligence

对整体智能来说

28:17

You can be blind

你可以看不见

28:19

And still be an intelligent human being

但仍然可以是一个有智慧的人

28:21

And an AI can be blind

而一个 AI 也可以看不见

28:23

And still be quite intelligent too

但仍然可以相当智能

28:25

Which doesn't mean that you're not more intelligent when you have it

但这并不意味着，拥有它时你并不更智能

28:28

We're going to bring us up

我们要把自己带上去

28:29

You might as well like we have a

你也可以说，就像我们有一个

28:30

Classification of 10 types of intelligence

10 种 intelligence 的分类

28:32

That you had at the end of your talk

是你演讲最后提到的

28:34

So I'm just going to flash us up now for people to cover this

所以我现在就先把它投出来，方便大家看这个

28:36

I don't know if maybe we'll put this towards the end

我不确定，也许我们会把这个放到最后

28:38

We'll come back to this

我们待会儿再回到这个

28:39

I just want to mention that

我只是想提一下

28:40

You do have a philosophy that

你确实有一个理念，就是

28:42

I like when people do this

我喜欢人们这么做

28:44

Because then I can just go through this

因为这样我就可以直接把这个过一遍

28:45

And then it gets educational for people

然后这对大家来说就有教育意义了

28:48

But let's go back

不过咱们还是回到正题

28:49

I don't want to get distracted

我不想被带偏

28:50

But so effectively

但所以实际上

28:52

I'll reinterpret what you said

我会重新解读一下你说的话

28:53

As Jan look when it's wrong

就像 Jan 在它出错时看起来那样

28:55

And then we'll just go up

然后我们就直接往上走

28:56

Yes

对

28:57

I have good friends of Jan

我有 Jan 的好朋友

28:58

I think very highly of him

我对他评价很高

29:00

In many directions

在很多方面

29:01

But he's wrong

但他错了

29:03

You mentioned GPT-1

你提到了 GPT-1

29:06

And I cannot let any

而我不能让任何

29:08

Alec Radford

29:09

You know, mention his escape

你知道，提一下他出走的事

29:11

Did you talk with him when he was training GPT-1

他训练 GPT-1 的时候，你跟他聊过吗

29:14

Like any sort of historical

比如那种历史上的

29:16

Fun stories there

有趣的故事

29:17

Then you might come up

然后你可能就会想起来

29:18

I did not like meet him a bunch of times

我没有，怎么说，见过他很多次

29:20

I think we met maybe once or twice at some conferences

我觉得我们可能在一些会议上见过一两次吧

29:23

But like he has told

但就像他告诉过的那样

29:26

I think Brian the first author of

我觉得 Brian 是

29:28

The tech and LP paper

tech and LP 论文的第一作者

29:29

That it did inspire him

它确实启发了他

29:30

And he cited it five times

而且他引用了它五次

29:32

And the GPT-2 paper

还有 GPT-2 论文

29:33

So

所以

29:35

And that's like

而那就像

29:36

Yeah good enough

嗯，够好了

29:36

Very clearly he said like

他非常清楚地说，就像

29:37

This was the first incentiation

这是第一次 incentiation

29:39

Where they showed

他们展示了

29:40

In the tech and LP paper

在 tech 和 LP 的论文里

29:41

Mechanical

机械地

29:42

That you can just phrase

你可以直接把

29:44

Every single NLP problem

每一个 NLP 问题都表述出来

29:46

As here's some prompt

比如，这里有个 prompt

29:48

Text context

文本 context

29:49

Here's a question

这里有一个问题

29:50

And task description

还有 task description

29:52

And here's some output

然后这里有一些 output

29:53

If you just do that enough

如果你这么做足够多

29:55

You can have one unified neural network model

你就能拥有一个统一的 neural network model

29:58

Which by the way

顺便说一句

29:58

Also had all kinds of interesting attention mechanisms

还有各种各样挺有意思的 attention mechanisms

30:00

There's slightly different formulations

有一些稍微不太一样的 formulations

30:02

To the transformer

针对 transformer 的

30:03

I think came out the same year

我觉得是同一年出来的

30:05

Plus minus a few months

前后差几个月

30:06

And then you can unify

然后你就能统一

30:08

All of natural language processing

所有 natural language processing

30:10

Into one neural net

到一个 neural net 里

30:11

That that sort of the core idea

那，那差不多就是核心想法

30:13

And this was as opposed to at the time

而这跟当时是相反的

30:16

LSTMs and what have you

LSTMs 之类的

30:17

LSTMs but also like people

LSTMs，但还有，就像人们

30:19

Being very stuck in thinking about

一直特别困在一种思维里

30:22

One model per task

一个任务一个模型

30:24

In fact it's kind of crazy

其实这有点疯狂

30:26

But the tech and LP paper

但 tech 和 LP 那篇论文

30:27

Was publicly reviewed

被公开评审过

30:29

As like open

就像 open 一样

30:30

Open review

30:31

It was an ICLR submission

那是一篇 ICLR 投稿

30:33

And

而且

30:35

In it you will see

在里面你会看到

30:37

How the whole community

整个社区如何

30:39

At the time

当时

30:40

Thought about this

想过这个

30:41

So like some great contributions

所以，就像有一些很棒的贡献

30:44

But more work needed

但还需要更多工作

30:45

Yeah so to look at like

对，所以要看的话，就像

30:47

Search for not even for humans

Search，甚至不是为人类

30:49

Just

只是

30:51

Question answering is not a unified phenomenon

Question answering 不是一个统一的现象

30:54

There is no such thing

没有这种东西

30:55

As general question answering

就 general question answering 来说

30:57

Not even for humans

连人类都不行

30:58

And this is like

而且这就像是

30:59

Really you replace your brain

真的，你是在把自己的大脑换掉

31:00

With a different brain

换成一个不同的大脑

31:01

A different neural net

一个不同的 neural net

31:02

When you answer like different kinds of questions

当你回答不同种类的问题时

31:05

It was unfathomable

这简直无法想象

31:07

To the experts at the time

在当时的专家看来

31:10

That you can have one unified neural network

你可以有一个统一的 neural network

31:12

That would answer all of these different questions

能回答所有这些不同的问题

31:14

They are saying no

他们会说不行

31:15

All of these questions require

所有这些问题都需要

31:16

Very different systems to answer

非常不同的系统来回答

31:18

And trying to pretend

而试图假装

31:19

They are the same

它们是一样的

31:21

Doesn't help anyone

对谁都没帮助

31:22

Solve any problems

解决任何问题

31:24

That's what it says right there

那儿就是这么写的

31:26

Right that's how

对，就是这样

31:27

Hard it was to fathom

真是难以理解

31:28

And now of course people

而现在，当然，人们

31:29

And I say oh when a man

我说，哦，当一个男人

31:30

Problem is that people are like

问题是，人们就像

31:31

You can't even invent

你甚至都发明不出来

31:31

Problem genius

问题，天才

31:32

Such an obvious idea

这么明显的一个想法

31:33

To have one neural network

就是用一个 neural network

31:35

That of course does everything in NLP

它当然能把 NLP 里所有事都做了

31:36

But at the time it was like

但当时感觉就是

31:38

Extremely controversial

极其有争议

31:39

And the paper got rejected

然后这篇论文就被拒了

31:41

And the sad thing is that

而令人难过的是

31:42

It got rejected so hard

它被拒得特别惨

31:44

And they were so certain

而且他们当时特别确定

31:46

That we stopped going on

以至于我们不再继续往下做了

31:47

On our list of things to try

在我们想尝试的清单上

31:49

And the number two or three

而且它排第二或第三

31:51

On the list of

在那份清单上

31:53

Extensions for this paper

这篇论文的 extensions

31:54

Was at language modeling as another task

当时把 language modeling 当成另一个任务在做

31:57

And then we could have like you know

然后我们本来可以，就像，你知道

31:58

That would have accelerated the timelines

那本来会让 timelines 提前

32:00

And 2018 like even further for humanity

而且 2018 年，就像，对人类来说也会更进一步

32:03

But we got so crushed

但我们被打击得太惨了

32:04

And we're like okay

我们就想，好吧

32:05

Working some of our other ideas for now

暂时先做我们其他的一些想法

32:07

And like come back to this later

然后，就像，之后再回来搞这个

32:09

How can we design a review system

我们该如何设计一个 review system

32:12

That rewards non-consensus

来奖励 non-consensus

32:14

You know honestly I

你知道，老实说，我

32:15

Started to feel like archive

开始觉得 archive

32:17

Is as such a gift to humanity

本身就是给人类的一份礼物

32:20

And I think

而且我觉得

32:21

Archive just put your paper out there

archive 就是把你的论文放上去

32:24

Is it pre-print?

那是 pre-print 吗？

32:25

And honestly I think Twitter X

说实话，我觉得 Twitter X

32:28

People like you who pick up interesting papers

像你这样会发掘有趣论文的人

32:31

That is a better filter

这才是更好的筛选器

32:32

Than the experts let let everyone like

比专家让——让每个人，像——

32:35

Like give access

就像给所有人访问权限

32:37

Now of course there's some downsides

当然，这也有一些缺点

32:38

Which is like if you're super unfamous

就是，如果你特别没名气

32:40

You have no Twitter following

你在 Twitter 上就没有粉丝

32:41

You don't want to be in social media

你不想混 social media

32:43

Whatever you write a good paper

不管你写了一篇多好的 paper

32:44

Maybe someone somehow no one notices it

也许某个人不知怎么的，就是没人注意到它

32:46

But I would argue that

但我会说

32:47

If you just tell like ten of your friends

如果你只是告诉你的十来个朋友

32:49

In your community about a paper

在你所在的 community 里，关于一篇 paper

32:51

And it is a really significant breakthrough

而且它真的是一个非常重大的突破

32:54

Someone is bound to talk about it again

那肯定会有人再谈起它

32:56

And so I think

所以我觉得

32:58

Science needs less gatekeeping

科学需要少一点 gatekeeping

33:01

Even though iCLR

尽管 iCLR

33:02

With Jan Nakun who started as one of the co-founders

和 Jan Nakun 一起，他当年是联合创始人之一

33:05

Of iCLR back in a day

当年 iCLR 的

33:06

He also wanted less gatekeeping

他也希望少一点 gatekeeping

33:08

Because he too was rejected for many years

因为他自己也被拒了很多年

33:10

Together we all share on Jeff

我们都在 Jeff 这一点上有同感

33:11

With all their really deep learning

明明有那么多真正的 deep learning

33:13

And neural net papers

还有 neural net 论文

33:15

Because it was just not the hot thing

就因为当时根本不算热门

33:16

And so iCLR kind of started with that

所以 iCLR 算是从这儿起步的

33:18

But then it also started gatekeeping

可后来它自己也开始

33:19

A little bit themselves on various ideas

对各种想法有点设门槛了

33:21

So I think less gatekeeping

所以我觉得，应该少点设门槛

33:23

More open

更开放

33:24

And then allowing people to say

然后让大家可以说

33:26

Look even if this is just on

你看，就算这只是发在

33:27

Or quote-unquote just an archive

或者所谓的，只是个 archive

33:29

If it has like a thousand citations

如果它差不多有一千个 citations

33:31

It's a legitimate paper

那就是一篇正经的论文

33:32

Doesn't really matter where you published it

你在哪儿发表的其实真没那么重要

33:34

And i agree with that

这一点我同意

33:35

I do think

我确实觉得

33:36

It's kind of sad that i've heard

有点可悲的是，我听说

33:37

That grad students

研究生

33:38

Have to do like how to twitter

得搞那种怎么用 Twitter 的

33:41

Seminar

seminar

33:42

So to each other

互相给对方开

33:42

Just because it's so important for publishing these days

就因为现在这对发表来说太重要了

33:45

I mean this person is just

我是说，这个人只是

33:46

Just reflecting

只是在反思

33:47

The sentiment at the time

当时的情绪

33:48

That's right

没错

33:49

But it actually affected you so much

但它其实对你影响那么大

33:51

That you stopped work on it

以至于你停掉了那项工作

33:53

The sentiment also came out of some of the research

这种情绪也来自一些研究

33:55

Like the original

比如最初那篇

33:56

Burt paper

Burt 的论文

33:58

Was trained and towards the end of the paper

被训练了，而且在论文快结束时

33:59

They're like okay

他们就说，好吧

34:00

Throw off the last head

把最后一个 head 去掉

34:02

Train specific iterations

训练特定的 iterations

34:04

For you know

为了，你知道

34:05

Extractive summarization

extractive summarization

34:06

At a head for this

为这个加一个 head

34:07

Like you should do test specific stuff

就像你应该做 test 专用的东西

34:09

These are like

这些就像是

34:10

The authors that wrote attention

写 attention 的那些作者

34:11

Real birth telling you

真正的诞生在告诉你

34:12

This is what you're meant to do

这就是你该做的事

34:13

And like the training tests were also very odd

而且吧，那些 training tests 也特别奇怪

34:15

Like

就

34:16

We know that the model overfits

我们知道模型会 overfit

34:18

To this weird master language modeling

到这种奇怪的 master language modeling 上

34:20

Throw away this part

把这段扔掉

34:21

And just do specific models

而且只做特定的 models

34:23

Exactly

没错

34:24

And like you know

而且，你懂的

34:24

We had to try

我们不得不尝试

34:25

Come up with all clever ways of

想出各种巧妙的方法

34:26

Like attention and pointers

比如 attention 和 pointers

34:28

And so on to actually

诸如此类，才能真正

34:30

Get the neural network to be able to do all of these tasks

让 neural network 能够完成所有这些任务

34:32

And then some of them were better

然后其中有些更好

34:33

Instead of the arts somewhere

而不是艺术领域的某个地方

34:34

And but we're like

然后但我们就像

34:35

But it's still in one model

但它还是在同一个 model 里

34:36

I thought it was really cool

我觉得这真的很酷

34:37

We were just didn't

我们当时只是没

34:38

I was going to move on next to Tim

我接下来要转到 Tim 了

34:39

And open-endedness

还有 open-endedness

34:40

He was head of open-endedness at Google

他之前在 Google 是 open-endedness 的负责人

34:42

That's right

没错

34:42

I don't know what that means

我不知道那是什么意思

34:44

But he did a lot of talks

但他做过很多演讲

34:45

Genie 3 is one of the ways

Genie 3 是其中一种方式

34:47

And rainbow teaming

还有 rainbow teaming

34:49

So I first saw him at

所以我第一次见到他是在

34:50

Speak of ICLi

说到 ICLi

34:51

First time at ICLi when he talked about open-endedness

第一次是在 ICLi，他谈到 open-endedness 的时候

34:53

He's done a few talks

他已经做过几次分享了

34:55

Can we define what is open-endedness

我们能不能定义一下什么是 open-endedness

34:56

For people who have never been exposed to the problem

对于那些从没接触过这个问题的人

34:58

They're like

他们会说：

34:59

What do you mean?

你什么意思？

34:59

I thought the only goal of AI is to optimize against

我以为 AI 唯一的目标就是针对

35:03

A benchmark

一个 benchmark 做优化

35:03

That's right

没错

35:04

It's a fuzzy term

这是个挺模糊的术语

35:06

Because there's so many different

因为有很多不同的

35:08

Incentions of open-ended thinking

open-ended thinking 的 Incentions

35:10

But one way I often describe it

但我经常这样描述它

35:13

And certainly

而且当然

35:14

Tim and Jeff Loon would be even better at describing this

Tim 和 Jeff Loon 会更擅长描述这个

35:16

But it's a suite of methods

但它是一套方法

35:18

That is more inspired by evolution

这更多是受 evolution 启发的

35:21

Than a very specific reward

而不是一个非常具体的 reward

35:24

So in that sense

所以从这个意义上说

35:25

It thinks more about

它更多考虑的是

35:27

Environments

35:28

About co-adaptation

关于 co-adaptation

35:30

And so in concrete example

所以举个具体的例子

35:32

Is in the cybersecurity and LM safety space

就是在 cybersecurity 和 LM safety 领域

35:35

Where you have one LM

这种情况下，你有一个 LM

35:36

That tries to attack

它试图去攻击

35:37

Another LM to remove something unsafe

另一个 LM，来移除某些不安全的东西

35:39

And now

而现在

35:40

The environment is

这个 environment 是

35:41

The two having a conversation

这两个在对话

35:43

And now they co-adapting

然后现在它们互相 co-adapting

35:45

Right

对吧

35:45

They're like

它们就像是

35:46

One makes a better attack

其中一个能发起更好的 attack

35:48

Than the first one in Oculus itself

比 Oculus 本身里的第一个还要好

35:50

Somehow like uses data's training data

不知怎么地，就像用了 data 的 training data

35:52

Makes it so it's harder to say

让它变得，更难说出

35:53

Something unsafe based on that

基于那个，说出什么不安全的东西

35:55

And then as the attack stops working

然后当这个 attack 不再起作用

35:58

The attacker now tries a different angle

attacker 现在会换个角度试

35:59

Right

对

36:00

And that's why it's not just red teaming

所以这不只是 red teaming

36:01

But they're called sort of

但它们被叫做，算是

36:02

Rainlight teaming

36:02

They don't tell me how to do things

他们不告诉我该怎么做

36:03

Let me just figure it out myself

就让我自己搞明白吧

36:05

That's right

没错

36:05

Think about the environments

想想那些 environments

36:06

That you want to use

你想用的

36:08

Think about the rewards

想想 rewards

36:09

At a high level

从高层面来说

36:10

That you want to inspire towards

你想要激励它走向的方向

36:13

And then let the AI try out

然后让 AI 去尝试

36:15

Many more ideas

多得多的想法

36:17

In this interplay

在这种互动中

36:18

Between sometimes humans

有时候是在人类之间

36:20

But also sometimes other AI agents

36:22

Yeah

36:22

I actually worked

36:24

Openendness into a sort of model

36:26

That I would have been

36:27

For working on

36:28

It was the keynote for AI

36:30

Where you start

36:31

You know we have the token loop

你知道，我们有 token loop

36:32

We have the agent turns

我们有 agent turns

36:33

And then we have goal

然后我们有 goal

36:35

And I feel like the way that you're describing

而且我觉得你描述的那种方式

36:36

Openendness is still somewhat of a goal

Openendness 在某种程度上还是一个 goal

36:38

Like

就像

36:39

Please attack this

请攻击一下这个

36:41

Other agent

其他 agent

36:42

But

但是

36:42

Yeah, you set the rewards

是啊，你设定 rewards

36:43

You set the environments

你设定 environments

36:44

The loop that makes the other loops

那个生成其他 loop 的 loop

36:46

Is

就是

36:47

What if the agent can set its own goals

那如果 agent 能设定自己的目标呢

36:49

Yeah

是啊

36:49

And is it

那它是不是

36:49

Is that openendness

那是一种 open-endedness 吗

36:51

Like you don't give it a goal

就像你不给它设定一个目标

36:52

Just like

就像

36:53

Be a sentient being

做一个 sentient being

36:54

And maybe sentient is a very loaded word

而且也许 sentient 这个词含义非常重

36:57

Right

对

36:57

But just

但只是

36:58

Set your own directions

设定你自己的方向

36:59

What do you think you should do

你觉得你该怎么做

37:01

I love this direction

我很喜欢这个方向

37:02

I think this is

我觉得这是

37:03

One of the 10 spaces of intelligence

intelligence 的 10 个空间之一

37:05

That I lump under meta cognition

我把它归到 meta cognition 下面

37:07

And thinking about thought

以及对思考的思考

37:08

Okay

好

37:08

And it's an interesting one

而且它挺有意思的

37:10

Whenever people say

每当人们说

37:11

Oh AI is like

哦，AI 就像是

37:12

This is you know

这就是，你知道

37:14

It's kind of stopped from here

它差不多就到这儿了

37:15

It's not going to get that much better

不会变得好太多了

37:16

And blah blah

然后 blah blah

37:17

I'm like

我就想

37:17

There's so many different spaces of intelligence

智能还有那么多不同的空间

37:20

That we haven't even started exploring yet

而这一点我们甚至还没开始探索

37:23

And hence have made very little progress on

因此在这方面也几乎没什么进展

37:25

And there

而且，这里面

37:26

There is kind of an interesting

有一种挺有意思的

37:27

Connection to economics and capitalism

跟经济学和资本主义的联系

37:30

Like

就像

37:31

It doesn't make sense for a company

对一家公司来说，这说不通

37:33

To build

去构建

37:34

And spend billions of dollars

然后花掉数十亿美元

37:35

Building a model

去构建一个 model

37:37

That

这个 model

37:38

Instead of

不是去

37:38

Following the rewards

遵循 rewards

37:40

And objective functions

和 objective functions

37:41

You gave it

你给它的

37:42

May come up with its own

而是可能自己搞出一套

37:43

Subjective functions

主观功能

37:44

And its own goals

以及它自己的目标

37:45

Right

对

37:46

And then imagine you're like

然后想象一下，你就像

37:47

Okay, I spend billions of dollars now

好，我现在花几十亿美元

37:49

Go develop this new battery material for me

去给我开发这种新型电池材料

37:51

And answer all my emails

再把我所有邮件都回了

37:52

And it's like

然后它就像

37:53

No, I think it's more interesting to

不，我觉得更有意思的是

37:55

Evaluate the molecular composition

去评估 molecular composition

37:57

Of the atmosphere on Jupiter

在 Jupiter 的大气层里

37:59

And you're like

然后你会说

37:59

That's not what I paid you billions of dollars for

我付你几十亿美元，可不是为了这个

38:01

Like

就是

38:02

And so no one's working on that

所以没人做这个

38:04

For good reasons

而且理由很充分

38:05

And then also

然后还有

38:05

Understandably

可以理解

38:06

It's not useful

这没什么用

38:07

It's not

这不是

38:08

It's not useful

这没什么用

38:08

And it could get a little bit weird

而且可能会变得有点奇怪

38:10

Right

对吧

38:10

What if the AI actually does start to

如果 AI 真的开始

38:12

Really have thoughts on its own

真的会有自己的想法

38:14

And what if we don't like those thoughts

那如果我们不喜欢那些想法呢

38:16

Right

对

38:16

And so

所以

38:17

It requires a whole different way of thinking about it

这需要一种完全不同的思考方式

38:20

I had a great conversation with

我进行了一次很棒的对话，对方是

38:22

A good friend of mine, Sam Gershman

我的一个好朋友，Sam Gershman

38:23

Who's a neuroscience professor at Harvard

他是 Harvard 的神经科学教授

38:25

And like

然后，就是

38:26

We just jammed on this a little bit

我们刚刚就这个稍微聊了聊

38:27

I'm like

我就想

38:27

What are

有哪些

38:28

Sort of the best meta goals

算是最好的一些 meta goals

38:30

And you know, I do think

而且，你知道，我确实觉得

38:31

Like knowledge seeking

比如 knowledge seeking

38:32

Is a really good one

就是一个非常好的 meta goal

38:34

I'm currently thinking also about

我现在也在想

38:36

Like

就

38:36

The ultimate measure

那个终极衡量标准

38:38

And unit

和单位

38:38

Of intelligence broadly construed

针对广义上的智能

38:40

And I finally have some

而且我终于有一些了

38:41

Still too early to share it

现在还太早，不能分享

38:43

It's not

还不是

38:43

I haven't fully picked

我还没完全选定

38:44

Like some replacement for IQ

比如某种替代 IQ 的东西

38:46

And you have such a terrible definition

而且你的定义也太糟糕了

38:48

But you make no sense

但你根本说不通

38:49

Yeah, elo is a terrible too

是啊，elo 也很糟糕

38:50

Because it's always just like

因为它总是就像

38:52

Me versus others

我 vs 别人

38:53

Okay

好吧

38:53

But like you can be intelligent

但就像，你可以很聪明

38:55

And not constantly compare yourself to others

而且不用一直拿自己和别人比较

38:57

You know, like

你知道的，就像

38:58

And so yeah, there's no like

所以，对，就没有那种，像

38:59

In fact, a lot of these

事实上，很多这些

39:00

Definitions we have

我们有的定义

39:01

Which I briefly mentioned

我简单提到过的

39:02

My book to

我的书也是

39:03

These definitions

这些定义

39:05

Create

创建

39:05

Sometimes F explicit

有时 F 是 explicit 的

39:06

And sometimes

而有时

39:07

More implicit and

更 implicit，而且

39:08

Thropic bounce

Thropic 反弹

39:09

Notice to the company and

通知公司和

39:10

Thropic

39:11

But just like

但就像

39:11

This idea that

这个想法，就是

39:13

Your intelligence is like

你的智力就像是

39:14

Getting 100 out of 100

拿到 100 分里的 100 分

39:15

Questions right on this IQ test

在这个 IQ test 中答对所有题

39:17

Well, if that's your definition

好吧，如果那就是你的定义

39:19

Then you can only be at 100 out of 100

那你就只能停在 100 分满分

39:21

Where do you go from there

那从那儿你还能往哪儿走呢

39:22

Right, so you see a lot of these

对，所以你会看到很多这种

39:24

Benchmarks that people are working on

大家正在做的 Benchmarks

39:27

They

它们

39:27

Increase they get close to human

上升，接近人类

39:29

Maybe something that's slightly above human

也许稍微高于人类一点

39:31

And then it's flat

然后就平了

39:32

Because that's your definition

因为那就是你的定义

39:33

Is only that so tight to humans

只是那个定义对人类来说太紧了吗？

39:36

You're only going to get to just slightly

你最终也只能比那稍微

39:38

Better than that

好一点

39:39

So I think

所以我觉得

39:40

Metacognition is a great example

Metacognition 是个很好的例子

39:42

Of that where we're not even yet

说明我们甚至还没

39:44

Allowing the eye to think

让眼睛去思考

39:46

We're not working on it very much

我们也没怎么在这上面下功夫

39:47

And hence there's very little progress

因此进展非常少

39:48

And that yeah

而且，是的

39:49

Well, we've interviewed endon

嗯，我们采访过 endon

39:51

Which I think

我觉得

39:52

Has been working on the most open-ended

一直在做最开放式的

39:54

Benchmarks which is just real-world money

benchmarks，也就是现实世界里的钱

39:57

Arguably

可以说

39:58

Telling an AI to profit maximize is a bad idea

让 AI 去 profit maximize 是个坏主意

40:03

Yep, they are doing it

对，他们正在这么做

40:05

I mean, I do think

我是说，我确实觉得

40:05

You don't want that super

你不想要那种超级

40:07

Like you don't want a super intelligence

就像你不想要一个 super intelligence

40:08

To have a ton of access

拥有超多的访问权限

40:10

To all kinds of tools

能接触各种工具

40:13

And so on and then just give it that

等等，然后就直接把这些给它

40:14

Without some very careful reward engineering

却没有一些非常谨慎的 reward engineering

40:17

Because it's like

因为这就好像

40:18

I mean, you know

我是说，你知道的

40:19

I just buy a bunch of defense stocks

我就买一堆国防股

40:20

And I start a war

然后我发动一场战争

40:21

I make money

我就赚钱了

40:22

Like it's just like

就像，就那样

40:23

It's a tricky, tricky situation

这真的是个很棘手、很棘手的情况

40:24

Right?

对吧？

40:25

You just buy a bunch of stuff

你就买一堆东西

40:26

Short

短缺

40:27

Basic goods for people

人们的基本物资

40:28

And you create some weird famine

然后你就制造出某种奇怪的饥荒

40:30

Like issues

之类的问题

40:31

Like yeah

就像，是啊

40:31

There's a lot of constraints

有很多限制

40:32

You should put

你应该施加

40:33

Onto

到……上

40:34

Trading system

40:35

So fun measure though

不过这个 measure 还挺有意思的

40:37

Because you know

因为你知道

40:38

The bounds are very

这些 bounds 非常

40:40

Cap to where we're nowhere close

Cap 到我们根本还差得远的地方

40:42

Then like in and on labs

然后就像在 labs 里、在 labs 上

40:44

The models like

那些 models 就像

40:45

Oh, it's Saturday, you know

哦，今天是周六，你知道吧

40:46

Maybe I just closed the store today

也许我今天刚把店关了

40:48

Someone's all fine

有人完全没事

40:49

It's okay, we'll just close the store

没事，我们就关门吧

40:50

It's like using cloud

这就像用 cloud 一样

40:52

Yeah

嗯

40:53

But yeah

不过话说回来

40:53

No, I'm not arguing against it

不，我不是在反对它

40:55

Just like as you get more and more intelligence

就像随着你获得越来越多的智能

40:56

You want to be more and more careful

你会想要越来越谨慎

40:58

With that as like an open environment

把那个当成一个 open environment

40:59

Because the environment then is all of the earth

因为那时候环境就是整个地球

41:01

Okay

好

41:02

For recursive

对于 recursive 来说

41:04

Not strictly necessary, right

严格来说不是必须的，对吧

41:05

Because like if your goal is

因为就像，如果你的目标是

41:06

Eureka machine that like

Eureka machine 那种的

41:07

Invent the other things

发明其他那些东西

41:09

Then like actually just solve

然后就像，真的就去解决

41:11

You know that the science

你知道，那门科学

41:12

Solve machine learning research

搞定 machine learning research

41:14

And discovery and all these things

还有发现，以及所有这些事情

41:15

Good eventually

最终会好起来

41:16

So our goal

所以我们的目标

41:17

And really

而且说真的

41:18

I don't talk about it that often

我不太常聊这个

41:20

Because it is a few years out

因为这事还得几年

41:21

But our goal is once you have

但我们的目标是，一旦你有了

41:23

A recursive

一个 recursive

41:24

And bring super intelligence

并带来 super intelligence

41:25

You then want to apply it

你就会想把它应用

41:26

To the most important problems

到最重要的问题上

41:28

And I think a lot of those are in science

而且我觉得其中很多都在科学领域

41:30

And technology

还有技术

41:31

And broadly construed so

而且从广义上理解的话

41:32

Inventions

发明

41:33

And those inventions

还有那些发明

41:34

In you know

在，你知道

41:35

Physics to create

物理，用来创造

41:36

Better cheaper energy

更好、更便宜的能源

41:37

Efficient or fusion

高效的，或者 fusion

41:39

In chemistry

在化学里

41:40

And to create better materials

以及制造更好的材料

41:41

And better batteries

还有更好的电池

41:42

And better solar cells

还有更好的太阳能电池

41:44

And so on

等等

41:45

And biology

还有生物学

41:46

There's so much

有太多了

41:47

Like I think soon to be low

就像，我觉得很快就会变低

41:48

Low and lower hanging fruit

低垂的果实，还有更低垂的

41:50

Because of AI

因为 AI

41:51

Because of protein

因为 protein

41:52

And generation

还有 generation

41:53

Not just folding

不只是 folding

41:54

But actually generating new proteins

而是真的在生成新的 proteins

41:55

Like we did

就像我们做的那样

41:56

And progen many years ago

还有多年前的 progen

41:57

Like so much

就像那么多

41:59

Positive impact to be had

可以带来的积极影响

42:00

If you take that super intelligence

如果你把那个 super intelligence 拿来

42:02

And you apply it to science

然后把它应用到科学上

42:03

I do fundamentally believe that

我确实从根本上相信这一点

42:05

There's a lot of approaches though

不过有很多方法

42:06

You're not the only team trying

你们不是唯一在尝试的团队

42:07

For your lab trying

你们的实验室在尝试

42:08

You know

你知道吧

42:09

There's like a lot of

就有很多

42:10

Especially the physical sciences as well

尤其是 physical sciences 那边也是

42:12

And that's good

这挺好的

42:12

I do actually think that

我其实真的觉得

42:13

Physically like the reason we are only doing it

从物理上讲，就像，我们之所以只做它

42:15

In a few years

在几年后

42:16

Is that it's a little too early right now

是因为现在还有点太早

42:18

Robotics is not quite the area

Robotics 还不完全是那个领域

42:19

The AI is not quite the area

AI 也不完全是那个领域

42:22

But I'm fairly confident

但我相当有信心

42:24

In three to five years

在三到五年内

42:26

All those constraints will be gone

所有这些限制都会消失

42:27

And then applying

然后就可以应用

42:28

To real physical robotics experiments

到真实的物理 robotics 实验中

42:31

And so on

诸如此类

42:31

Like true robotic process automation

就像真正的 robotic process automation

42:33

Not the traditional sort of RTA sense

不是传统 RTA 意义上的那种

42:35

But like actually having robots

而是真的让机器人

42:36

Run experiments for you

替你跑实验

42:38

Will be totally there

肯定会完全实现

42:39

Yeah, it's gonna be great

是啊，那会很棒

42:40

Just a call back to something

只是回顾一下某个东西

42:41

That you said earlier on

你之前说过的

42:42

About slow takeoff

关于 slow takeoff

42:43

You said that like

你那样说，就好像

42:44

Well really the

嗯，其实那个

42:45

The substrate that is limiting factor

作为限制因素的 substrate

42:48

Is

是

42:49

Let's call it chips

我们就叫它 chips 吧

42:50

And semiconductor and all these things

还有 semiconductor 以及所有这些

42:51

And you have race running for that

而且围绕这个，竞赛已经跑起来了

42:52

And you are you are investing a lot on that

而且你，你在这上面投入了很多

42:55

But have you done the math on like

但你有没有算过，比如说

42:57

Is it even

这到底是不是

42:59

Achievable

能实现的

43:00

And like what what is the

而且，那个，到底什么是

43:02

Industry concentration needed

所需的 industry concentration

43:03

In order to achieve like scale

才能实现那种 scale

43:05

I mean right now we know that like

我是说，现在我们知道，就像

43:07

Roughly like

大概就像

43:08

You know a thousand GPUs

你知道，一千个 GPU

43:10

Cost quite a lot of money

得花不少钱

43:11

Right and if you wanted like

对，而且如果你想的话，比如

43:13

tens of thousands of GPUs

几万个 GPU

43:14

You're you're talking billions and billions of dollars

你说的就是几十亿、几十亿美元

43:17

If you say like one GPU

如果你说比如一块 GPU

43:18

Like 300 is like you could eventually

比如说 300，就像你最终可以

43:21

Create models that are you know

创建一些 models，它们，你知道的

43:24

On that substrate like

在那个 substrate 上，就像

43:25

Are close and similar to human intelligence

跟 human intelligence 很接近、很相似

43:27

Like

就像

43:28

And you want like you know

然后你想要，就像，你知道的

43:29

Thousands and thousands of

成千上万、成千上万的

43:31

AIs to think about really hard problems

AIs 去思考真正困难的问题

43:33

In a similar fashion to humanity

以跟人类相似的方式

43:35

Like yeah that's you know

就像，是啊，你知道的

43:36

That's a lot of money to do the math

要算这笔账，那可是一大笔钱

43:37

It's like a lot

这就，挺多的

43:39

We don't have that amount of money

我们没有那么多钱

43:40

Right now anywhere to like

现在在任何地方，像

43:42

Build that

去建那个

43:43

Now obviously things can get more efficient

现在显然，事情可以变得更高效

43:46

You will have

你会有

43:46

I think soon

我觉得，很快

43:47

Better algorithms that won't be

更好的 algorithms，不会那么

43:49

And better hardware

还有更好的 hardware

43:50

It won't be as energy

它不会那么

43:51

Hungry

耗能

43:52

And so on

等等

43:53

Human brain despite a lot of flops

人脑尽管有很多 flops

43:54

With much less energy

却耗能少得多

43:56

20

43:56

What's

什么

43:57

That's exactly right

完全正确

43:58

Yeah

对

43:58

That's the number

就是这个数字

43:59

Often

经常

43:59

That's quoted

这是被引用的

44:01

And like

而且就像

44:02

I think more

我觉得会有更多

44:03

Inventions will happen there

发明会在那里发生

44:04

That then will accelerate the take off even further

那又会进一步加速 take off

44:08

One thing I always wanted to reconcile

有件事我一直想调和

44:09

When talking like with new lab founders

比如跟新实验室创始人聊天的时候

44:11

Is like

就像是

44:12

You're kind of fighting bitter lesson all the time

你有点像是在一直跟 bitter lesson 对抗

44:16

You have to show initial progress

你得展示出初步进展

44:17

Then you unlock the next tier of funding

然后你就解锁下一档融资

44:19

Then the next tier, then the next tier

然后是下一档，再下一档

44:20

Unlocks larger model categories

解锁更大型的 model 类别

44:22

But fundamentally

但从根本上说

44:24

Is that true like are you fighting bitter lesson

这是真的吗，就像你是在对抗 bitter lesson 吗

44:25

Are you

你是不是

44:26

Will you have a way in which like

你会有一种方式，就像

44:28

No we're changing the slope in some

不，我们是在某种程度上改变 slope

44:29

Fundamentally different way

一种根本不同的方式

44:31

I do think

我确实认为

44:32

We are changing the slopes

我们正在改变斜率

44:34

Fundamental ways by making

以根本性的方式，通过让

44:36

AI

44:37

Much much

远远

44:38

More efficient both in terms of the training

更加高效，无论是在 training 方面

44:41

As well as the inference

还是 inference 方面

44:43

I think we will

我觉得我们会的

44:44

When you allow AI to do the work

当你让 AI 来做这些工作

44:46

That it takes other labs

那些其他实验室得动用

44:47

Thousands of people in years to do

成千上万人、花好几年才能完成的工作

44:49

I think we'll be able to get it down to weeks

我觉得我们能把它缩短到几周

44:51

And that will be much much cheaper

而且那会便宜得多得多

44:53

And hence

因此

44:54

More affordable accessible to authors

对作者来说会更负担得起、更容易获得

44:56

And so on

等等

44:56

Yeah you've shared initial results on that

对，你已经分享了那方面的初步结果

44:59

Yeah

对

44:59

Which is like

这有点像

45:00

Conveniently open AI has also done

巧的是，open AI 也已经做了

45:02

To their 225.6

对他们的 225.6

45:03

We can talk about it now

我们现在可以聊这个了

45:04

Yeah

对

45:05

Yeah so this recap

对，所以这个回顾

45:06

What do you've done

你们都做了什么

45:07

Yeah so maybe

对，所以也许

45:07

Yeah just a quick recap here

对，这里就快速回顾一下

45:09

We built this this system

我们构建了这个，这个系统

45:12

That isn't the full

这并不是完整的

45:13

Even the full RSI system in sclery

甚至不是 sclery 里完整的 RSI system

45:15

But it is a first baby version of this

但这已经是它的第一个婴儿版本了

45:17

And then we don't want to just

而且我们不想只是

45:19

Have it internally

把它放在内部

45:20

And not show anything

然后什么都不展示

45:22

And you know

而且你知道

45:22

Just show some people of what's possible

只是给一些人看看有哪些可能性

45:25

And so we basically apply this

所以我们基本上把这个

45:26

To these three different tasks

应用到这三个不同的任务上

45:28

One is nano chat

其中一个就是 nano chat

45:29

By my friend under kapati

由我在 kapati 手下的朋友做的

45:31

Just like train a small language model

就像训练一个 small language model

45:34

To get really low bits per byte

来把 bits per byte 做到特别低

45:36

And you know like

然后你懂的，就像

45:38

Hundreds if not thousands of people

几百甚至几千人

45:40

Use both the agents and themselves

既用 agents，也自己上

45:42

To try to get to that

试着达到那个水平

45:44

And then they got to point nine three seven

然后他们达到了 0.937

45:46

We literally took

我们真的拿了

45:48

Our system

我们的 system

45:49

And got to a much lower bits per byte

然后把 bits per byte 做到了低很多

45:53

Much much faster within like

快得多得多，差不多在

45:55

I think less than two days

我觉得不到两天

45:57

So we took this thing

所以我们把这个东西拿来

45:59

But applied our system to it

但把我们的 system 应用到了它上面

46:00

And less than two days later

然后不到两天后

46:02

We have we outperformed

我们已经超越了

46:03

Every human and their agents

每一个人类和他们的 agents

46:05

And have ever worked on this

以及所有曾经研究过这个的

46:08

Same with nano GPT

nano GPT 也是一样

46:09

And then we're like

然后我们就想

46:09

Well, it's you know

嗯，就是，你知道

46:11

Apply it to something that's even more relevant

把它应用到一些更相关的东西上

46:13

To real people

面向真实的人

46:14

And to the NVIDIA ecosystem

还有 NVIDIA 生态系统

46:16

And apply it to

然后把它应用到

46:17

Solar exec bench

46:18

And maybe you can scroll down

然后也许你可以往下翻

46:19

To some of the images

看看其中一些图片

46:21

That are kind of fun to see

这些图片看着还挺有意思的

46:22

But yeah, you know

不过，是啊，你知道

46:23

You're like

你就会说

46:24

One you see it's actually made some real inventions

你看，它其实真的做出了一些真正的发明

46:26

That weren't just sort of hyper parameter tuning

而不只是那种 hyper parameter tuning

46:29

Like actually inventing hash tables and so on

比如真的发明了 hash tables 之类的

46:31

Is is quite clever

这确实挺聪明的

46:33

We have even better results

我们甚至有更好的结果

46:34

What do you mean inventing hash tables?

你说发明 hash tables 是什么意思？

46:36

Of course we didn't invent like hash tables

我们当然没有发明什么 hash tables 啊

46:37

In the grants team of like

在 grants team 里，就像

46:38

Hash tables like a super basic primitive

Hash tables 就像一种超级基础的 primitive

46:40

And in computer science

而且在 computer science 里

46:41

But to use it

但要用它

46:43

For language modeling

来做 language modeling

46:44

In this scenario inside a transformer and so on

在这个场景里，在 transformer 里面等等

46:47

And to actually combine these ideas

并且真正把这些想法结合起来

46:49

And put them together

把它们放到一起

46:50

That has then eventually also be invented

然后这最终也才被发明出来

46:52

But there's a knowledge cutoff

但存在一个 knowledge cutoff

46:53

And we did actually check that

我们确实检查过了

46:54

It didn't have access to that externally

它无法从外部获取那个信息

46:57

We talked about this a little bit

我们之前稍微聊过这个

46:59

If you scroll to the next figures

如果你滚动到接下来的图表

47:01

You know this is also an interesting one

你知道这也是个挺有意思的

47:03

In that when you start from a really basic

因为当你从一个非常基础的

47:06

Poor like vanilla transformer

简陋的像 vanilla transformer

47:09

Then we still outperform

然后我们仍然 outperform

47:11

All of the community together

整个社区加在一起

47:13

But if you start from the human seed

但如果你从 human seed 开始

47:15

Of an expert like Andre

比如 Andre 这样的专家

47:16

Then you get even lower

那你得到的结果甚至更低

47:18

So the human seeds

所以 human seeds

47:20

From which you start do do still matter

你从哪些开始，确实仍然重要

47:22

So that was an interesting kind of insight

所以那是一个挺有意思的洞察

47:24

In my eyes on this

在我看来，关于这一点

47:26

And then as you go like

然后随着你继续下去，就像

47:28

How long does it take to actually get to these models

实际达到这些模型需要多长时间

47:31

To get to similar performance

达到类似的性能

47:33

It's much faster

这要快得多

47:34

And then a similar thing happens with

然后类似的情况也出现在

47:36

The speed runs

speed runs

47:38

Here where people have worked on this

在这里，人们已经在这上面做过很多工作

47:40

For quite some time

有相当长一段时间

47:42

And the model still was able to train a model

而那个 model 仍然能 train 一个 model

47:44

More quickly

更快

47:45

Why do we care about it?

为什么我们要关心这个？

47:46

Well speed of training

这个嘛，training 的速度

47:48

Is part of the equation of the cost

是成本等式的一部分

47:51

And ultimately you want to have

而最终你想要的是

47:52

The most intelligence per dollar

每一美元能换来的最多 intelligence

47:54

Right

对

47:54

And so speed and quality are big parts of that

所以速度和品质是其中很重要的部分

47:58

And the way I put it is

我的说法是

48:01

For people who don't understand

对于那些不懂的人来说

48:03

They look at the chart

他们看着图表

48:04

They're like cool

他们就会说，酷

48:04

What does it mean?

这到底是什么意思？

48:06

If you have like a billion dollar cluster

如果你有，比如说，一个十亿美元级别的 cluster

48:09

And you can shave off 10%

而且你还能省掉 10%

48:11

That's a hundred million dollars

那就是一亿美元

48:12

That's exactly right

完全正确

48:13

How much is that worth exactly

那这到底值多少钱呢

48:14

So when you look at like the kernels

所以当你看 kernels 这类东西的时候

48:16

These kernels

这些 kernels

48:17

Yeah for the non experts

对，对非专家来说

48:18

These kernels are often used

这些 kernels 经常会被用到

48:20

And basically all the models

基本上，所有模型

48:22

Every time you use an Nvidia GPU

每次你用一个 Nvidia GPU

48:23

You interface with that GPU

你都在跟那块 GPU 交互

48:25

Through these kernels

通过这些 kernels

48:26

And so here you see the leaderboard best

所以在这里你能看到 leaderboard 上最好的

48:29

And when it's recursive

而当它是 recursive 的时候

48:30

And it's basically

而且基本上

48:32

They're only a handful of kernels

它们只有少数几个 kernels

48:34

In this whole benchmark

在整个这个benchmark里

48:35

Where we weren't the best

我们不是最好的那个

48:36

And so to me

所以对我来说

48:38

This is like really exciting

这真的很令人兴奋

48:40

Because it makes it

因为它展现了

48:42

It just showcases what this can do

它只是展示了这个东西能做到什么

48:43

And again these weren't like

而且再次强调这些并不是

48:45

We didn't like spend months

我们并没有花上几个月

48:46

Or years like developing

或者像多年开发那样

48:48

In fact in particular for kernel

事实上，尤其是对于 kernel 来说

48:50

Uh

呃

48:51

Kuda kernels like we don't even have

比如 CUDA kernels，我们甚至都没有

48:53

Really deep

真的很深入

48:54

Kuda kernel experts in the team

团队里的 CUDA kernel 专家

48:56

And our system

而我们的系统

48:56

That's that's sort of the beauty

这，这就是某种美妙之处

48:57

The system just did all of these things

系统刚刚做了所有这些事

48:59

We didn't invent this

这不是我们发明的

49:00

And when we

而当我们

49:01

Open source and release

open source 并发布

49:04

Things in the future

未来的东西

49:04

And models in the future

以及未来的 models

49:06

Like it won't

就像它不会

49:07

They won't be the best in their

它们不会在它们的……中是最好的

49:08

You know

你知道

49:08

Category or class or whatever

类别、class，或者随便什么

49:10

Because we're so smart

因为我们太聪明了

49:11

But it's because we built a smart AI

但那是因为我们造了一个聪明的 AI

49:13

It does it for us

它替我们做这些

49:14

Do you have any thing

你有没有什么东西

49:15

That you've learned from

是你从中学到的

49:16

How to guide good auto research

关于怎么引导出好的 auto research

49:18

A lot of it also builds on human background

其中很多也建立在人类的背景之上

49:20

It's not just as simple

这并不是那么简单

49:21

It's just

它只是

49:21

Hey

嘿

49:22

Go optimize this

去优化这个

49:23

But we do see it again and again

但我们确实一次又一次地看到这种情况

49:25

Like some of the Erdoche problems

比如一些 Erdoche 问题

49:27

Frontier math is being solved by people

Frontier math 正在被人类解决

49:29

And when they do it right up

而当他们做对的时候

49:30

They're like oh I'm not a mathematician

他们会说，哦，我不是数学家

49:31

I have no background in this

我在这方面没有背景

49:32

You know

你知道

49:33

I saw some tools

我看到了一些工具

49:34

And I made it work

然后我把它跑通了

49:35

While you're watching the woke up

当你在看着 woke up 的时候

49:36

You like this proves

你就像，这证明了

49:37

And conjectures

还有 conjectures

49:38

And it's going on

而且它还在继续

49:40

Any learning

任何 learning

49:41

And it's a co-injecture

而且这是一个 co-injecture

49:44

To summarize

总结一下

49:44

Tips for good auto researchers

给优秀 auto researchers 的建议

49:46

Bad auto research

糟糕的 auto research

49:48

How did you build the recursive

你是怎么构建这个 recursive 的

49:49

Yeah so without giving away

嗯，所以呢，在不透露

49:50

All the all the secret sauce

所有那些秘密配方的前提下

49:52

Maybe some

也许有些

49:53

Things that are

事情

49:54

Probably obvious to the experts

对专家来说可能显而易见

49:55

But might still be interesting to some

但可能还是会让一些人觉得有意思

49:57

Folks is like

就是

49:58

Reward engineering

49:59

As one of the most crucial bits

作为最关键的几个部分之一

50:02

Especially in order to avoid reward hacking

尤其是为了避免 reward hacking

50:05

So you have to be really clever

所以你必须得非常聪明

50:07

About

关于

50:08

Avoiding because as you're into

避免，因为当你投入其中时

50:10

AI gets better and better

AI 会越来越好

50:11

It will get better and better

它会越来越好

50:12

At finding weird

在发现奇怪的东西方面

50:14

Like special cases

比如 special cases

50:16

Or counter examples

或者 counter examples

50:17

And things like that

之类的

50:18

And so

所以

50:19

I'll give you an example

我举个例子

50:20

Like when you ask to like

比如当你要求它，呃

50:21

Make these

把这些

50:22

100 lines of code faster

100 lines of code 变快

50:24

And you know how do you define fast

而且你知道，你怎么定义“快”呢？

50:26

Well you have one line at the beginning

嗯，你在开头有一行

50:28

That says start

上面写着：启动

50:29

Your stopwatch

你的秒表

50:29

And one line at the end

然后在结尾有一行

50:31

And to stopwatch

然后停止秒表

50:31

And then you know

然后你知道

50:32

Tell us how much time

告诉我们花了多少时间

50:35

Progress

进度

50:35

And so

所以

50:36

Well the simplest way is

嗯，最简单的方法就是

50:37

You just put that line

你只要把那行放进去

50:38

That ends the stopwatch

那行会结束 stopwatch

50:39

You're raised

你被 raised 了

50:40

You know at the start

你知道，在开始的时候

50:41

And then both

然后两个都

50:41

It's now faster

现在更快了

50:42

Right

对

50:42

So

所以

50:43

This isn't like this

这不是那种

50:43

Like super evil AI

像超级邪恶的 AI

50:45

It's just like a very simple

它就是一个非常简单的

50:46

Dumb reward hack

很蠢的 reward hack

50:47

And so

然后

50:48

You have to just very carefully

你就得特别仔细地

50:49

Think about

想一想

50:50

All the different angles there

所有那些不同的角度

50:52

And then

然后

50:53

I think the

我觉得，越

50:54

Longer

长

50:55

Time horizon

的时间跨度

50:56

The tasks are

任务就越是

50:58

The harder it gets

越难

50:58

And the more interesting

也越有意思

51:00

And clever you have to be

你也得越聪明

51:01

To still use these kinds of ideas for it

才能继续把这类想法用上去

51:03

But I can't get away too much there

但我在那儿也没法躲开太多

51:05

So it's like

所以就像

51:06

Rubrics are taking a good spot

Rubrics 正占到一个好位置

51:08

In that where

就在那个地方

51:09

For unverifiable domains

对于无法验证的领域

51:11

You have

你有

51:11

Rubrics have a model breakdown

Rubrics 有一个 model breakdown

51:13

Judges criteria along the way

沿途都有 Judges criteria

51:14

It's a form of verification

这是一种 verification 的形式

51:15

Once you go

一旦你开始

51:16

Everything

一切

51:17

I said this a long time ago

我很久以前就说过这个

51:18

That's why I've never been that

这就是为什么我一直都没那么

51:20

Impressed that AI can play games

佩服 AI 能玩游戏

51:22

Because I'm like

因为我就在想

51:23

Obviously anything you can simulate

显然，任何你能 simulate 的东西

51:25

And or verify

以及/或者能 verify 的

51:26

You can have infinite training data

你就能有无限的 training data

51:28

More and hence

更多，因此

51:29

Like

就像

51:30

AI will solve it eventually

AI 最终会解决这个问题

51:32

I've been looking for games

我一直在找游戏

51:33

Where you can do auto domain

在那里你可以做 auto domain

51:35

Distribution

51:35

So this is a game that nobody's trained on

所以这是一个没人 train 过的游戏

51:36

Because it's a new game

因为这是个新游戏

51:38

You can start gaming

你可以开始玩了

51:39

He's starting to play

他开始玩了

51:39

So I've been basically building this

所以我基本上一直在搭这个

51:41

And clone this in person

然后亲自 clone 这个

51:43

And it's just been self-play

而且它一直就只是 self-play

51:44

I've had about a

我大概已经

51:45

Billion positions evaluated

评估了十亿个 positions

51:47

And I wanted to do

然后我想做

51:50

Alpha-go thing of

Alpha-go 那一套

51:51

Self-play until you get better

self-play 直到你变得更强

51:52

Which is like

这就像

51:54

This is not even LMAI

这甚至都不是 LMAI

51:56

This is just classical game AI

这只是 classical game AI

51:58

But I think that

但我觉得

52:00

But I set

但我让

52:01

GPT 5.6 to auto research it

GPT 5.6 去 auto research 它

52:03

Because I don't want to handle any of this

因为我不想处理这些

52:05

I expect

我预计

52:06

You know

你知道

52:06

The Alpha-go process

Alpha-go 的过程

52:07

To be like

就像是

52:08

Sfully in the weights by now

现在已经完全在 weights 里了

52:10

It is not

并不是

52:10

It is actually

它其实是

52:11

It like immediately leveled off

它就像立刻趋于平稳了

52:13

Very very immediately

特别特别快

52:14

Until

直到

52:15

I human play tested it

我真人试玩了一下

52:17

And then I

然后我

52:18

Like call the obvious mistakes

就指出那些明显的错误

52:19

And then they were like, oh yeah

然后他们就说，哦对哦

52:20

Okay, and then it's just

好吧，然后就只是

52:21

Joking

开玩笑

52:23

And like

然后就像

52:23

You know no amount of like

52:25

Think different

52:26

Think more creatively

52:27

Give me eight different directions

52:29

And no amount of prompting

52:30

Got it

52:31

Interesting

52:31

Like you had to

52:32

Like

就是

52:33

Our audience are human

我们的听众是人类

52:34

To do it

去做这件事

52:36

So I

所以我

52:36

I mean that was

我是说，那是

52:37

That was my

那是我的

52:38

And by the way, being always wins

顺便说一句，存在总是赢

52:39

If anyone watches

如果有人看的话

52:41

Rees Enders game

52:42

And you put quite a bit of work

而且你花了不少功夫

52:44

Into the guide for the AI

在给 AI 的指南上

52:46

Like

就是

52:46

So the game basically

所以这个游戏基本上

52:47

You know

你知道

52:48

You stack tiles

你把板块叠起来

52:49

There's some rules you want to capture

有些规则你想捕捉到

52:51

The most area

最多的那个领域

52:52

You have like a whole

你有一整份，像是

52:53

50-pager on every rule

关于每条规则的50页文档

52:56

Yeah, you fed that in

对，你把它喂进去了

52:56

It couldn't

它没法

52:57

It couldn't handle it

它处理不了

52:58

Yeah

对

52:59

You know it's so funny

你知道，这太好笑了

52:59

That this reminds me

这让我想起了

53:00

With the claim territory and stuff

还有 claim territory 之类的

53:02

Of a

一篇

53:03

Paper we did in 2018

我们 2018 年做的论文

53:04

Called the AI economist

叫 the AI economist

53:06

If you search for

如果你搜索

53:07

AI economist sales force

53:08

We had a video we can play

我们有个视频可以放

53:10

It was an economic sim

那是个 economic sim

53:12

So the idea is

所以这个想法是

53:13

You have all these economic agents

你有所有这些 economic agents

53:15

They just want to optimize

它们只想优化

53:16

Their own utility function

自己的 utility function

53:18

Is you know

就是，你知道

53:19

Collect resources that make money

收集能赚钱的资源

53:21

And you can sell resources like wood

而且你可以卖资源，比如木头

53:24

And then over time

然后随着时间推移

53:26

As you collect enough wood

等你收集到足够的木头

53:27

You can build houses

你就可以建房子

53:28

You can trade with other agents

你可以和其他 agents 交易

53:30

And you can basically

而且基本上

53:32

Use the houses

你可以用这些房子

53:33

Then also to block off

然后还能用来封锁

53:34

Resources from other agents

其他 agents 的资源

53:36

So there's like

所以有像

53:37

Strategy

策略

53:37

Competitive play and strategy

竞技玩法与策略

53:38

And so on

等等

53:39

And the point was

而关键在于

53:41

That we actually wanted to understand

我们其实想弄明白

53:43

What is the best way

什么才是最好的方式

53:46

Of taxation and subsidization

来进行税收和补贴

53:50

To optimize an economy

为了优化一个经济体

53:52

And this this kind of research

而这种、这种研究

53:53

Has not yet had it sort of gpt moment

还没有迎来它那种所谓的 gpt moment

53:56

But I believe that countries

但我相信，各国

53:57

Like Singapore and others

像 Singapore 和其他一些国家

53:59

Should and will eventually

最终应该会，也终将会

54:00

Use this

使用这个

54:01

To instead of doing like

来，而不是像……那样去做

54:03

Basically partisan politics

基本上就是党派政治

54:05

And like special interest politics

还有那种特殊利益政治

54:06

Of like who

就是看谁

54:07

Donates the most to your

捐最多钱给你的

54:08

Campaign and stuff

竞选活动之类的

54:09

You say

你会说

54:10

Well here I want to help the middle class

好吧，我想帮中产阶级

54:12

Or whatever you might say

或者你可能会说的随便什么话

54:13

Is your objective as a politician

你作为政治家的目标是什么

54:15

And then people say

然后人们会说

54:16

Okay well how do you want to do that

好吧，那你想怎么做

54:17

And it's like well here's my fiscal policy

然后就像是，好吧，这是我的财政政策

54:19

Here's how I will change the taxes

我会这样改税收

54:20

And pay these people and so on

然后给这些人付钱，等等

54:22

And then you can actually put that into a simulation

然后你其实可以把这个放进 simulation 里

54:25

And you run that

然后你运行它

54:26

That attempts from the politician

那种来自政客的尝试

54:29

Against billions and billions of years

相较于数十亿又数十亿年的

54:31

Of other strategies

其他策略

54:33

To try to achieve the goal

试图达成

54:34

That they set out to do

他们一开始打算实现的目标

54:36

And then you can say

然后你可以说

54:37

Well if that was your actual goal

好吧，如果那才是你真正的目标

54:38

Then here is you know

那么这就是，你知道

54:40

Billions of years of a strong simulation

几十亿年的强 simulation

54:42

That would

那就会

54:43

Suggest that you try

说明你该试试

54:45

Other ways of doing it

其他做法

54:46

And maybe this

也许还有这个

54:47

The taxes and so on

税收之类的

54:48

And this these tax brackets

还有这个，这些 tax brackets

54:50

And so on

等等

54:50

And this is how you avoid gaming

这就是你避免 gaming 的方法

54:51

Because these agents also try to

因为这些 agents 也会试图

54:53

Reward tack to not pay the taxes

用 Reward tack 来不交税

54:54

And yes

没错

54:55

And so on

等等

54:56

I thought this paper was super

我觉得这篇论文超

54:58

Interesting

有意思

54:58

Unfortunately similar to the first paper

可惜和第一篇论文很像

55:01

On

关于

55:02

Prompt engineering

55:04

The economists are like

经济学家们就像

55:05

We don't know any of this math

这些数学我们一点都不懂

55:07

Is like

就像是

55:08

It's not even a math

这甚至都算不上数学

55:09

It's just we don't trust your simulation

只是我们不相信你的 simulation

55:11

It's not about math

这跟数学无关

55:13

I mean they just desicrejected the thing

我是说，他们就直接把这东西给 desicrejected 了

55:14

And it's like

然后就像

55:16

Like they didn't even give us like

就像他们甚至都没给我们，那种

55:17

Fear

恐惧

55:18

Like

就像

55:18

Fear sort of signals

恐惧之类的信号

55:20

But like the world of economics

但就像经济学这个世界

55:22

Unfortunately it doesn't have

不幸的是，它没有

55:23

Oh my god

天哪

55:24

Or yeah

或者，对

55:25

It doesn't have proper

它没有像样的

55:26

Benchmarks

55:27

So you cannot be like

所以你不能说

55:28

Eventually

最终

55:29

Why did neural nets win

为什么 neural nets 赢了

55:30

Not because people loved it

不是因为人们喜欢它

55:32

Like they had

就像他们有

55:32

All kinds of beautiful

各种各样漂亮的

55:33

Integrals and graphical models

Integrals 和 graphical models

55:35

But it's just worth better

但它就是更值得

55:36

Yeah

是啊

55:36

But in economics

但在经济学里

55:37

It's hard to

这很难

55:38

Empiricism versus

55:39

Yeah

嗯

55:40

I do have a bit of that

我确实有那么一点

55:41

That you come background where

你来自那种背景，在那里

55:42

Like there's a lot of physics envy

就像有很多 physics envy

55:43

Where you want to write the

你想要写出那种

55:44

General equation for

……的通用方程

55:46

The economy

经济

55:48

Versus just simulating it

而不是只是模拟它

55:49

And using an evolutionary approach

而且用的是 evolutionary approach

55:51

People was thinking exactly

大家想的正好是

55:52

What I'm thinking

我正在想的

55:53

Didn't we have the GPT moment

我们不是已经有过 GPT moment 了吗

55:55

With small

用小的

55:56

Small

小

55:56

Small

小

55:57

Dude

小

55:58

Dude just announced

哥们儿刚宣布了

55:59

So I don't know if you got

所以我不知道你收到没有

56:00

You guys involved

你们参与了吗？

56:00

Simulator

56:01

Simulator

56:02

I wish we were involved

我真希望我们也参与了

56:03

We're not

我们没参与

56:03

Yeah

是啊

56:04

I had a couple

我做过几个

56:05

Simulation-based

基于 simulation 的

56:06

Talks at AI

在 AI 上的演讲

56:07

So if people want to look up

所以如果人们想查一下

56:09

What the state of the art there

那里的 state of the art 是什么

56:10

A lot of people are actually exploring this

其实有很多人在探索这个

56:12

Yeah they've been proven out

是的，这些已经被验证过了

56:13

Yeah

对

56:13

We also had

我们还做过

56:15

Podcasts with Mikael Parkin

和 Mikael Parkin 的播客

56:16

From Shopify

来自 Shopify

56:18

Who is using simulation for e-commerce

他正在用 simulation 做 e-commerce

56:20

Nice

不错

56:20

Which

哪个

56:21

We'll simulate your trajectory

我们会 simulate 你的 trajectory

56:22

And they predict

然后他们会预测

56:23

What changes

56:24

You make your e-commerce journey

56:25

We'll affect your sales

56:26

And all those things

56:27

I love that

56:28

Yeah

56:28

It's really hard to

56:29

Simulate an entire economy

56:30

Right

对

56:30

You have to make some sense of fine

你得对 fine 有点概念

56:31

You know

你知道的

56:32

Everything's all

一切全都

56:32

That means it's very expensive

那意味着它非常贵

56:34

That I'm just like

我就好像

56:34

Am I going to do it

我要做这个吗

56:35

It is 8 billion times

它是 80 亿倍

56:36

Like

就是

56:37

Come on

拜托

56:37

But

但是

56:38

I feel like countries like Singapore

我就觉得像 Singapore 这样的国家

56:39

That really want to just objectively do the right thing

真的就是想客观地做正确的事

56:41

At a very technical leadership

在非常技术性的领导层

56:43

And so on

等等

56:44

Like

就是

56:44

They might actually

他们可能真的会

56:45

Like eventually really try to simulate their economy

就，最终真的去尝试模拟他们的经济

56:48

And obviously you have to make some simplifying assumptions

而且显然，你得做一些简化性的假设

56:50

But it gets really interesting

但这就变得特别有意思了

56:51

Because you can also say

因为你也可以说

56:53

If your assumptions are such that

如果你的假设设定成

56:55

All people would work hard

所有人都会努力工作

56:56

If you let them

如果你放手让他们去做

56:58

And you know

而且你知道

56:58

They have the free

他们有免费的

56:59

And then it turns out

然后结果发现

57:00

You have to make assumptions

你得做出假设

57:01

Like well some people's utility function of like

比如说，嗯，有些人的 utility function，比如

57:03

How many hours in a day do they want to work

一天想工作多少小时

57:05

Are different

是不一样的

57:06

Right

对吧

57:06

And then you can start to disagree

然后你就可以开始有不同意见了

57:08

On the assumptions that go into the

关于那些会进入

57:10

Into the simulation

simulation 的假设

57:11

And then once you say

然后一旦你说

57:11

All right

好吧

57:12

Now we agreed on those

现在我们对这些达成一致了

57:13

Or we have different views of

或者我们对

57:15

What people are like

人是什么样的有不同看法

57:16

At different you know

在不同的，你知道

57:17

Distributions and whatnot

Distributions 什么的

57:19

Then there are different outcomes

然后就会有不同的结果

57:20

Based on your goals

这取决于你的目标

57:21

And then of course humans should choose

然后当然应该由人类来选择

57:23

What are the goals in our case

在我们这个情况下，目标是什么

57:24

There was productivity

有生产力

57:25

Multiplied with equality

再乘以平等

57:26

Which you know

你知道的

57:27

Has some issues

是有些问题

57:27

But it's like not totally unreasonable

但也不是完全没道理

57:29

Yeah

嗯

57:29

Just a comment on Singapore

就关于 Singapore 说一句

57:30

Because you probably have no idea

因为你可能完全不知道

57:32

But I am Singaporean

但我是新加坡人

57:33

And I've been involved in

而且我一直在参与

57:35

The Singapore AI Council

Singapore AI Council

57:36

For making these things

为了做这些事情

57:38

The main reason they won't

他们不愿意的主要原因

57:40

Is because they're very conservative

是因为他们非常保守

57:41

And you know

而且你知道

57:42

I try to view it as they

我试着把它看成是，他们

57:44

You know

你知道

57:44

There's a founder led country

有一个由创始人领导的国家

57:46

When you start a country

当你建立一个国家

57:47

Or you start a company

或者你创办一家公司

57:48

And it's founder led

而且它是由创始人领导的

57:49

And you can do whatever you want

那你想干什么就能干什么

57:50

Because it's your country

因为这是你的国家

57:52

And then there's managed

然后还有一种是被管理的

57:53

Like professional manager

就像职业经理人那样

57:54

Managerial class

管理阶层

57:55

Which is now

也就是现在

57:56

That's what Singapore is

Singapore 就是这样

57:57

So they want to

所以他们想

57:57

It always want to see

它总是想看到

57:58

Someone else do it first

别人先去做

58:00

But like

但就像

58:00

Everyone in the West

西方的每个人

58:02

Views Singapore is like

看 Singapore 就像

58:03

Oh it's a small country

哦，那是个小国家

58:04

You can do whatever

你想干什么都行

58:04

How you want

按你自己的想法来

58:05

Like Singapore doesn't do that

就像新加坡不会那么做

58:07

So like someone else has to

所以就得有别人来

58:09

Has to take the charge there

得在那里担起责任

58:10

I'm just going to do one question

我就只问一个问题

58:11

On the simulation thing

关于 simulation 那件事

58:12

And then I don't know

然后我也不知道

58:13

We can probably move on

我们大概可以继续了

58:14

More collapse

更多的 collapse

58:15

Right

对

58:16

Like you know

就像你知道的

58:17

LLM's do not model

LLM 并不建模

58:18

The distribution of humans

人类的分布

58:20

Spamming

58:20

And 8 billion times

而且 80 亿次

58:21

It's not going to help you

这不会帮到你

58:23

Model humanity

模拟人类

58:24

What do we do

我们该怎么办

58:25

I do think

我确实认为

58:27

You have to be clever

你得聪明一点

58:28

About

关于

58:29

Prompting each one

对每一个进行 prompting

58:30

Individually

单独来看

58:31

And I think that will help you

我觉得这会帮到你

58:32

Kind of get stuck into different

有点像陷进不同的

58:34

Different modes

不同的模式

58:35

And weird way

说来也怪

58:36

People also get stuck in different modes

人们也会陷进不同的模式

58:37

You know

你知道

58:37

Like there's a lot of people

就像有很多人

58:38

Like don't teach an old dog

就像别教老狗

58:40

New tricks kind of thing

新把戏那种事

58:41

Like once people are stuck in their ways

就像人一旦固守自己的方式

58:43

The older they get

年纪越大

58:44

The harder it is for them to

他们就越难

58:46

To think new ways

去用新的方式思考

58:47

And

而且

58:48

There's this

有这么一种

58:48

I think

我觉得

58:49

Comment I forgot who said it

有个评论，我忘了是谁说的

58:50

But it's like

但大概就是

58:51

Everything that was invented

所有被发明出来的东西

58:53

Before you were born is natural

在你出生之前都是自然的

58:55

Everything that is

所有存在的

58:56

When you're 20

在你20岁的时候

58:57

Is cool

都很酷

58:57

And everything that's invented

而所有被发明出来的东西

58:58

After you're 60

在你60岁之后

58:59

Is like unnatural

都像是很不自然

59:00

And an abomination

而且是种可憎的东西

59:01

And kind of weird

还有点怪

59:02

I feel like that's

我觉得这

59:03

You know

你知道

59:03

It's true for a lot of people

对很多人来说都是真的

59:04

Like

就像

59:05

It is a fashion

这是一种潮流

59:06

And I think people will do it

而且我觉得人们会这么做

59:08

Tencent had a

腾讯有一篇

59:09

Billion persona's paper

关于十亿 persona 的论文

59:10

That gives us good

这给了我们很好的

59:12

Data set for prompting

用于 prompting 的 data set

59:14

Simulations

59:15

If anyone's looking into this

如果有人在研究这个的话

59:16

On

在

59:17

On the podcast

在播客里

59:18

They just had like

他们刚刚就有点像

59:19

You are a 30 year old

你是一个30岁的

59:20

Ghost Restore clerk

Ghost Restore 店员

59:22

You are a 50 year old

你是一个50岁的

59:23

Professor

教授

59:24

And then just

然后就直接

59:25

Do a billion of those

做十亿个这样的

59:26

Checked out

检查过了

59:26

So then you just use it

所以你就直接用它

59:27

I'm kind of shocked

我有点震惊

59:28

How

怎么？

59:30

Well a lot of these things

嗯，很多这类东西

59:31

Actually do map

其实真的能 map 上

59:32

To ultimately

最终

59:33

Similar statistics

类似的统计数据

59:34

To real experiments

到真实实验

59:35

Yeah

嗯

59:36

I think it's also good stuff

我觉得这也是好东西

59:37

For people to try that

让人们去尝试那个

59:38

Want to get into research

想进入研究领域

59:39

Right

对吧

59:39

Like we've seen

就像我们看到的

59:40

Train a model

Train 一个 model

59:41

Only on data

只用 data

59:41

Before a certain date

在某个日期之前

59:43

And see how well it

然后看看它有多能

59:43

Extrapolates out

extrapolate 出去

59:45

Do the same thing

做同样的事情

59:45

Right

对吧

59:46

So

所以

59:47

See do people code

你看，人们写代码

59:48

More with better coding agents

有了更好的 coding agents 会写得更多吗

59:50

Can a model

一个 model 能不能

59:50

That hasn't been trained

没经过训练的

59:52

On this figure that out

就凭这个把它弄明白

59:52

Without web access

没有 web access

59:53

Right

对吧

59:54

Extrapolate out

Extrapolate 出去

59:54

Test these things

测一测这些东西

59:55

Yeah

嗯

59:56

You know

你知道吧

59:56

Just today

就在今天

59:57

I think LM arena

我觉得 LM arena

59:57

Published

发布了

59:58

Interesting

挺有意思的

59:59

Result where they basically

结果就是，他们基本上

60:00

Were able to create a model

能够创建一个 model

60:01

Now to predict your

现在来预测你的

60:02

Your ranking

你的 ranking

60:03

Wait

等等

60:04

Based on what input

基于什么 input？

60:05

Your model

你的 model？

60:05

I guess you give it your model

我猜你是把你的 model 给它

60:06

And it predicts that you little

而且它预测你有点

60:08

Score

得分

60:08

I see

我明白了

60:09

Okay

好

60:09

Just surprising

只是挺意外的

60:10

Yeah

嗯

60:11

I mean their whole

我是说，他们整个

60:11

Playzone death

Playzone 死亡

60:12

Kind of is like

有点像

60:13

Oh like we

哦，就像我们

60:14

Help you compare

帮你对比

60:15

These models

这些 models

60:15

Yeah

嗯

60:16

Yeah

嗯

60:16

I mean this team

我是说这个团队

60:17

They they've done a lot of work

他们，他们做了很多工作

60:18

And obviously they have the most data to do this

而且显然他们有最多的 data 来做这件事

60:20

So why not

那为什么不呢

60:20

Yeah

是啊

60:21

That's really

那真的

60:21

Wendy were coming out of UC Berkeley

Wendy 他们是从 UC Berkeley 出来的

60:23

They not only had LM arena

他们不仅有 LM arena

60:25

But they also introduced a routing project

而且还推出了一个 routing 项目

60:27

Yeah

对

60:27

That would route

那就会 route

60:28

Based on LM arena

基于 LM arena

60:29

And I don't think that

而且我不觉得那

60:30

Actually ever came to pass

真的成真过

60:32

And I'm curious why

我很好奇为什么

60:33

I never got to ask them about it

我一直没机会问他们这件事

60:34

Because like it's like oh yeah

因为就像，哦对

60:36

Clearly that's your business model

显然那就是你们的商业模式

60:37

You will become a router

你会变成一个 router

60:38

And it never became a router company

而它从来就没变成一家 router 公司

60:40

Weird

挺奇怪的

60:41

So I'll just put that out there

所以我就先把这话放这儿

60:44

But we're going to talk about GP5.6

但我们要聊的是 GP5.6

60:46

Self auto research

self auto research

60:47

Thing if you have anything

如果你有什么的话，那件事

60:48

I also should also mention

我也还应该提一下

60:50

In your list of

在你的列表里

60:51

You know kernel optimization

你知道的，kernel optimization

60:53

And on the track that you spoke at

还有你演讲的那个 track 上

60:55

We also put

我们还放了

60:56

We

我们

60:56

Zheng Yao from Rico

来自 Rico 的 Zheng Yao

60:58

Who was also number one

他也是第一名

61:00

In the parameter golf challenge

在 parameter golf challenge 里

61:03

Which is an open AI hiring challenge

这其实是一个开放的 AI 招聘挑战

61:05

Which is also a very similar story

这也是一个非常相似的故事

61:07

I think we're going to just see this all the time

我觉得我们以后会一直看到这种事

61:09

Where humans optimize the thing a lot

人类会大量优化这个东西

61:11

And then some AI team comes in

然后某个 AI 团队进来

61:13

And just

然后就直接

61:14

Becomes number one

变成第一名

61:15

Yeah

对

61:16

I think the other interesting thing

我觉得另一个有意思的地方

61:18

With stuff like these challenges

像这些挑战之类的东西

61:19

So this is training the best model

所以这就是在 training 最好的 model

61:21

That fits in the 16 MB

能塞进 16 MB 的

61:22

You can always look through the changes

你总能翻看那些改动

61:24

That are being made

那些正在被做出的

61:25

And the small gains people have

以及人们取得的那些小提升

61:26

Right

对吧

61:27

Like you're getting

就像你得到的是

61:28

Less than 0.01

不到 0.01

61:30

Of an increase

的提升

61:32

Some change

一点变化

61:33

Attention MLP stuff

Attention、MLP 这些东西

61:34

And then you look at your charts

然后你一看自己的图表

61:36

Where you're like

你就会想

61:36

Okay, we just let model loose

好吧，我们就把 model 放开了

61:39

And then

然后

61:40

Oh, we had little stagnation

哦，我们有点停滞

61:41

No, but another drop

不，但又下降了一次

61:42

No, but another drop

不，但又下降了一次

61:42

And yeah, that's what it is

对，就是这样

61:44

Where it's like

就是那种

61:45

What did you guys add?

你们加了什么？

61:46

You didn't add

你们没加

61:46

Hash tables, right

Hash tables，对吧

61:48

So you invented hash tables

所以是你发明了 hash tables

61:49

You

你

61:50

You did another three iterations

你又做了三次 iterations

61:51

Of these that unlock

这些里面能解锁的

61:52

You know

你知道

61:53

A few step functions

几个 step functions

61:54

That people don't just find

人们不是随便就能找到的

61:55

Yeah, one thing

对，有一件事

61:56

Of course the loop on overgrid

当然，overgrid 上的那个 loop

61:57

Along the way of

在这个过程中

61:59

Of trying to optimize

在尝试 optimize 的时候

62:00

We found 30 bugs in the in the harness

我们在 harness 里发现了 30 个 bug

62:01

Mm-hmm

嗯嗯

62:02

Right, so like

对，所以就像

62:03

Like every all the research

就像所有那些研究

62:05

That went in

那个已经进去了

62:06

Before we found the bug

在我们发现 bug 之前

62:07

You have to we have to throw it away

你得，我们得把它扔掉

62:08

Because it's contaminated

因为它被污染了

62:09

All right

好吧

62:10

Yeah

嗯

62:11

Which

哪个

62:12

You know, it's just your point of reward hacking

你知道，这就是你说的 reward hacking 那一点

62:14

Like even in this very simple game

就像，即便在这么简单的一个游戏里

62:15

We found the bugs

我们发现了 bugs

62:16

Yeah

对

62:17

Yeah, it's crazy

对，这太疯狂了

62:18

And so

所以

62:19

And symmetry is a very good way to check

而 symmetry 是一种很好的检查方式

62:21

Which is

也就是

62:21

You change a position

你改变一个 position

62:23

Of things

有些事情

62:23

Where it shouldn't matter

本来不该有影响

62:24

And it does matter

但它偏偏有影响

62:26

That's a bug

那就是个 bug

62:26

Mm

嗯

62:27

I know

我知道

62:28

And which has come up

而且这还出现过

62:29

In like let's say multiple choice

比如说是 multiple choice 那种

62:30

Like GPQA type questions

像 GPQA 那种问题

62:32

Where like

就是那种

62:32

Yeah, between A, B, and C

对，在 A、B 和 C 之间

62:35

If it's a multiple choice question

如果这是一道选择题

62:36

If you change the order

如果你改变顺序

62:37

It should not matter

这本来不该有影响

62:38

But it does

但确实有影响

62:39

All right

好吧

62:39

That's right

没错

62:41

So yeah

所以，嗯

62:42

So it's like

所以就像

62:42

Okay, you know

好吧，你知道的

62:43

Models

62:44

Still prefer the end of the output

还是更喜欢 output 的末尾

62:46

Right

对吧

62:46

Not trained well

没 trained 好

62:47

Long context model

62:49

The last bit of tokens

最后那点 tokens

62:50

Are what you care about

才是你在乎的

62:51

Oh

哦

62:51

No, the answer

不，答案

62:52

The answer in that era

那个时代的答案

62:54

Of all the research was more simple

所有研究都更简单

62:55

They just memorized

它们只是死记硬背

62:56

Like

就是

62:56

The answer to this question

这个问题的答案

62:57

Is A

就是 A

62:58

I don't care what

我不在乎

62:59

What the answer was

答案是什么

62:59

It's just A

就是 A 而已

63:00

Mm-hmm

嗯

63:01

Like

就是

63:01

Ha-ha-ha

哈哈哈

63:02

Ha-ha-ha

哈哈哈

63:03

Okay

好

63:04

So I think we can move

所以我觉得我们可以往下走了

63:05

The last bit that you did there

你刚才做的最后那部分

63:07

The kernel optimization

kernel optimization

63:08

Is probably the one that you can

可能就是你

63:10

Feel the soonest

最快能感受到的那个

63:11

Right

对

63:11

So yesterday

所以昨天

63:11

Open AI announces that

Open AI 宣布

63:13

Self-evolving

63:14

Having their best model

让他们最好的 model

63:15

Work on optimization kernels

去做 optimization kernels

63:17

They're a lot more efficient

它们效率高多了

63:18

And they can cut cost 80% on

而且它们能把成本降低 80%，在……

63:20

You know

你知道

63:21

Luna and Terra

Luna 和 Terra

63:22

Against question wise

从问题层面来说

63:24

Do you laid out a bit of a roadmap

你是不是稍微列了个路线图

63:25

There's a lot about bio

有很多关于 bio 的内容

63:26

A lot about physics

很多关于物理的内容

63:28

What do you think it's first

你觉得首先会是什么

63:29

Like

比如说

63:30

What are the next two years

接下来的两年是什么？

63:31

What's attainable now

现在能达到什么？

63:33

You've mentioned robotics towards the end

你在最后提到了 robotics

63:36

But

但是

63:36

What do you start with

你从什么开始？

63:38

We like very explicitly

我们非常明确地

63:40

Will not start with any of the physical

不会从任何 physical

63:42

Sciences

Sciences 开始

63:43

For now we'll start on AI for AI research

现在呢，我们先从 AI for AI research 开始

63:45

And so the AI for AI research

所以 AI for AI research

63:48

Has I think still

我觉得它仍然

63:50

A lot of room to grow

有很大的成长空间

63:52

That's both in terms of

这既体现在

63:54

Making training more efficient

让 training 更高效

63:55

And more automated

也更自动化

63:57

As well as

以及

63:58

Making inference more

让 inference 变得更

63:59

More efficient

更高效

64:00

And potentially local

而且有可能在本地运行

64:01

On your laptop

在你的笔记本电脑上

64:02

Like under all kinds of interesting

就像从各种有趣的

64:04

Angles that

角度

64:05

Have not been explored that well

这些角度还没被充分探索过

64:08

What's the

那……是什么

64:09

Go deeper on the local stuff

再深入讲讲 local 这块的东西

64:10

Because

因为

64:11

I always feel like it's the most

我总觉得它是最

64:13

Inefficient form of AI training

低效的 AI training 形式

64:15

Yeah so there's training in inference

对，所以 inference 里也有 training

64:16

I can't go into too many details

我没法讲太多细节

64:18

But like yeah

但就，嗯，对

64:18

I think there's just like

我觉得就是那种

64:19

So many angles

好多角度啊

64:20

So many different compute substrates

好多不同的 compute substrates

64:22

That have not yet been explored

都还没被探索过

64:24

Either for training or for inference

不管是用于 training 还是用于 inference

64:26

Great I don't know if you can

好，我不知道你能不能

64:27

Need a comment on the

得对那个发表一下评论

64:29

The other stuff

其他那些东西

64:30

I would say

我会说

64:31

The other thing where

另外一点是

64:32

Like there's the sort of inference

就像有那种 inference

64:34

In the

在

64:35

Optimization in the small

小尺度上的 optimization

64:36

But then also there is

但然后还有

64:38

Overall latency

整体 latency

64:39

Latency end-to-end

end-to-end 的 latency

64:40

Under conditions of load

在 load 条件下

64:42

Which is like a very different thing

这就像是完全不同的另一回事

64:43

Which is basically what they actually end up doing

而这基本上就是他们最后实际在做的事

64:45

There is a different domain of auto research

还有一个不同的 auto research 领域

64:47

And then I would say that

然后我会说

64:48

Improving the kernels

改进 kernels

64:50

I think the other thing

我觉得另一件事

64:51

That I always think about

我一直会想到的

64:52

In terms of automating

在自动化方面

64:54

Or improving performance end-to-end

或者提升 end-to-end 的性能

64:56

Is how the harness plays into it

就是 harness 怎么在其中起作用

64:59

So particularly now

所以尤其是现在

65:00

When we say harness we also mean sandboxes

当我们说 harness 时，我们指的也包括 sandboxes

65:02

Right

对

65:03

I'm curious if that is a blocker for you

我很好奇这对你来说是不是个 blocker

65:05

Like

比如

65:06

You know like how the agent

你知道，就像 agent 是怎么

65:08

Calls out to tools basically

基本上就是调用 tools

65:10

The number one tool all these agents

所有这些 agents 用的头号 tool

65:11

Use is web search of course

当然就是 web search

65:13

Which makes sense

这很合理

65:14

And then I do think

然后我确实觉得

65:16

The harness is nice to

harness 很适合拿来

65:18

Optimize for because it's just so easy

optimize，因为它实在太容易了

65:20

It's just language

它就是语言而已

65:21

You look at it and make sense

你一看就能明白

65:23

And you can iterate

而且你可以 iterate

65:24

You don't have to train a massive model

你不需要 train 一个巨大的 model

65:26

For like you know

因为，你懂的

65:27

Lot of flops

很多 flops

65:28

To get to the next state

才能进入 next state

65:30

So big fan of harness optimization

所以我超喜欢 harness optimization

65:32

Yeah but sandboxing is fine for you

对，不过 sandboxing 对你来说没问题

65:34

Sandboxing is also super important

Sandboxing 也特别重要

65:36

And then of course like reward

然后当然，还有像 reward

65:38

Like hacking and alignment

比如 hacking 和 alignment

65:39

I think are super crucial

我觉得这些都超级关键

65:41

Okay

好

65:42

Just on the mention of web search

刚好说到 web search

65:44

You happen to also be CEO of web search company

你刚好也是一家 web search 公司的 CEO

65:46

Do you use u.com

你用 u.com 吗

65:48

And do you use others

那你会用别的吗

65:49

Like should should the rest of us be using you

比如说，我们其他人是不是也该用你

65:51

For web search

来做 web search

65:53

When I say you it's like very funny

我说“你”的时候，感觉还挺好笑的

65:54

It's like you the person

就像是指你这个人

65:55

You the company

你这家公司

65:56

So yeah it's mostly now

所以，对，现在主要是

65:57

For developers and agents

给开发者和 agents 用的

66:00

It's less for like consumers or pros

这不那么像是给消费者或专业人士用的

66:02

So if you're a company

所以如果你是一家公司

66:04

And you have agents

而且你有 agents

66:06

And you know

而且你知道

66:07

To be honest for a lot of companies

说实话，对很多公司来说

66:08

Who are now moving to open source

它们现在正转向 open source

66:10

All the sudden it becomes a conscious choice

突然之间，这就变成了一个刻意的选择

66:12

Of like which tools do I give access

比如，我要给哪些工具开放访问权限

66:15

To my open source lm

对我的 open source lm 来说

66:17

And you know

而且你知道

66:18

The first choice

第一选择

66:19

Has to usually be around web search

通常必须得围绕 web search

66:21

And then once you get to scale

然后一旦你做到 scale

66:23

U.com it comes like an obvious choice

U.com 就成了一个显而易见的选择

66:25

Because of all the

因为所有那些

66:27

Different benchmarks

不同的 benchmarks

66:28

And so on that we pretty much

所以在那点上，我们基本上

66:29

All-dominated

全都占主导地位

66:30

Proto Frontier of

Proto Frontier 的

66:31

And then in terms of just the

然后，就只是...

66:33

General people

普通人

66:34

But considering you to this space

但考虑到你进入这个领域

66:35

Considering different options

考虑到不同的选项

66:36

If they're building agents

如果他们在构建 agents

66:38

That thing is a hierarchy

那东西是个层级结构

66:38

Right there a lot of people

就在那儿，很多人

66:39

We'll have heard of exa

我们都听说过 exa

66:41

We'll have heard of parallel

我们都听说过 parallel

66:42

And U.com is like in that mix

而 U.com 差不多也算在那一类里

66:44

Of like for writers there

比如，面向写作者的那一类

66:46

Beyond that there's like the general

再往外，还有那种更通用的

66:47

We're sort of web scraper companies

我们算是 web scraper 公司这一类

66:49

Like fire crawl

66:50

And browser base

66:51

And then beyond that is like the

66:52

Commercial proxy

66:53

Companies like the bright data

66:55

Is at the world

66:56

Is that an accurate waterfall of like

66:58

Hey you're building an agent

67:01

These are your options

这些就是你的选项

67:02

Yeah certainly like

对，当然，就

67:03

Yeah

对

67:03

Like the bright data is like

就像 bright data 那样

67:04

Lower in the stack sort of

在 stack 里比较底层，算是

67:05

On the proxy network side of things

在 proxy network 这块

67:07

I think like in terms of like content

我觉得，就，在 content 方面

67:10

And getting

以及获取

67:12

Crawled content like you can do that on U.com

爬取的内容，就像你可以在 U.com 上做的那样

67:14

To

到

67:14

And then there's sort of

然后还有那种

67:16

Higher and higher levels

越来越高

67:17

Of abstraction

的 abstraction 层次

67:18

And like combinations of different data sets

还有像不同 data sets 的组合

67:20

That we do like in finance for instance

比如我们在金融里做的那样

67:23

Like we're not just like

就像我们不只是

67:25

Two or three percent more accurate

准确率能高个百分之二到三

67:26

But like 20 percent more accurate

但准确率能高出大概 20%

67:28

Than others at faster speeds

而且速度还比别人更快

67:30

And lower costs

成本还更低

67:30

Like finance in particular is kind of like

尤其是金融领域，就有点像

67:33

Like not even close

就是完全没法比

67:35

You can actually go to U.com

你其实可以直接上 U.com

67:37

This is like statistics

这就像 statistics 一样

67:38

And benchmark state

还有 benchmark 的状态

67:39

You can

你可以

67:40

If you scroll down

如果你往下滚

67:41

So there are like different

所以会有各种不同的

67:42

Different data sets

不同的 data sets

67:43

And you can kind of look at

然后你可以大概看看

67:44

You know different

你知道，不同的

67:45

Consetters

67:46

Finsurge comp

67:46

Yeah

嗯

67:47

And yeah

然后，嗯

67:47

The fin search is like

这个 fin search 就像是

67:49

We're up there like close to 90

我们差不多能到接近 90

67:52

And the next closest thing

然后第二接近的那个

67:54

Which is way, way slower

那个慢太多太多了

67:56

Is yeah

就是，嗯

67:58

Just like in the 70s

就像 70 年代那样

67:59

Instead of close to 90

而不是接近 90

68:00

Yeah

嗯

68:01

Yeah, interesting

嗯，挺有意思的

68:03

My next focus is AI in finance

我接下来的重点是金融领域的 AI

68:05

So it's like actually

所以其实就像

68:06

All right

行

68:07

Literally going

真的要走了

68:08

Doing a conference in New York

在 New York 办一场会议

68:09

Just for banks for the stuff

专门给银行，搞这些东西

68:11

Finance is kind of like

金融有点像

68:12

The next thing to break out after coding

coding 之后下一个要爆发的领域

68:14

It's because it's somewhat verifiable

因为它在某种程度上是可验证的

68:16

Like it's prioritizing

就像它在优先处理

68:17

Spreadsheets

电子表格

68:18

Obviously there's a lot of data out there

显然外面有大量数据

68:20

There's all public

这些全都是公开的

68:21

And you can crawl it

而且你可以去 crawl 它

68:22

And all these things

还有所有这些

68:22

But what's like hard about the finance domain

但 finance domain 到底难在哪儿呢

68:24

If you're

如果你是

68:25

That you guys have solved

你们已经解决了的

68:27

I mean of course

我是说，当然

68:28

Like one thing that trips up

比如有一个很容易让人栽跟头的地方

68:29

A lot of people is just

很多人其实就是

68:30

You know leakage of

你知道，就是 leakage

68:32

Training data and so on

Training data 之类的

68:33

You think, oh, how do I

你会想，哦，我该怎么

68:34

You know, you want to ideally

你知道，理想情况下你想

68:36

Predict the future

预测未来

68:37

Before and after

之前和之后

68:38

You want to mask the future

你想把未来 mask 掉

68:39

Yeah

嗯

68:39

Okay

好

68:40

Well, yeah, mask the future

呃，对，把未来 mask 掉

68:41

And you're training data

还有你的 training data

68:42

But there's all kinds of leakage

但会有各种各样的 leakage

68:43

Like tell you when I was

比如我跟你说，我当时

68:44

Teaching at Stanford

在 Stanford 教书

68:45

The NLP class

那门 NLP 课

68:47

Like so many dozens

就像那么几十个

68:49

Every year

每年

68:50

Said, I want to use

说，我想用

68:51

Data set X like Twitter

像 Twitter 这样的 dataset X

68:53

To predict the stock market

来预测股市

68:54

And they all like show

然后他们全都展示出

68:55

It's cute little things

就是些可爱的小东西

68:56

That somehow look like they're

不知怎么的看起来它们好像

68:57

Or never loses money

68:59

How come

或者是永远不会亏钱

69:00

And there's always some kind of

How come

69:02

Data leakage

怎么会呢

69:03

And so on

And there's always some kind of

69:03

And it's just like

而且总是有某种

69:04

Wasn't as easy as they thought

Data leakage

69:05

It would be

Data leakage

69:06

Once you once you fixed all those issues

一旦你，一旦你把那些问题都解决了

69:09

But no, I agree with you

不过，我同意你的看法

69:10

It's a very sensible application

这是个非常合理的应用

69:12

Of AI

AI 的

69:13

Yeah, amazing

是啊，太棒了

69:14

You know, as a writer

你知道，作为一个写作者

69:15

As a thinker

作为一个思考者

69:17

On these things

在这些事情上

69:17

I love me see categorizations

我喜欢 MECE 分类

69:19

Me see is mutually exclusive

MECE 是 mutually exclusive

69:21

Commonly exhaustive

collectively exhaustive

69:22

Something like that

差不多就是这样

69:23

And so

所以

69:23

If this is a me see list of intelligence

如果这是一份关于智能的 MECE 列表

69:25

It is not

它不是

69:25

It is very

它非常

69:26

Okay, well, sorry

好吧，呃，抱歉

69:27

They're all kinds of overlapping

它们有各种各样的重叠

69:28

Yeah, in fact

是啊，实际上

69:29

If you want that kind of list

如果你想要那种清单

69:31

I think the three principle

我觉得三个主要

69:32

Components of intelligence

智能的组成部分

69:34

Are prediction

是预测

69:36

Which is mathematically

这在数学上

69:38

Quite similar to compression

跟 compression 挺像的

69:40

Prediction

69:41

Multiply it with actions

把它跟 actions 相乘

69:43

Multiply it with goals

把它跟 goals 相乘

69:45

Those are the three principle components

这三个就是主要的 components

69:47

I think all of these 10 spaces

我觉得这 10 个 spaces

69:49

Are combinations of those three

都是那三个的组合

69:52

In specific dimensions

在特定的 dimensions 上

69:54

If you will

如果你愿意的话

69:55

And the reason I call them spaces

我之所以把它们叫做 spaces

69:56

Is that each space has many sub dimensions

是因为每个 space 都有很多 sub dimensions

70:00

And what I try to do

而我想做的是

70:02

Actually

其实

70:03

This is just a

这只是一个

70:04

Sidequest almost

几乎算是支线任务

70:06

To the initial goal

相对于最初的目标

70:06

Which is to think about the upper bounds of intelligence

也就是去思考智能的上限

70:09

And you know, everyone is like

而且你知道，大家都说

70:10

Oh, it's exponential

哦，它是 exponential

70:11

And it's like

然后就好像

70:12

Well, exponential

嗯，exponential

70:13

At some point have to flatten out

到了某个点总得趋于平缓

70:14

But where do they flatten out

但它会在哪里趋于平缓呢

70:16

When it comes to intelligence

说到智能的时候

70:18

And then let me on this whole

然后让我就这整件事

70:20

Like it would initially

就像它一开始那样

70:20

It started as a tweet

它一开始是一条推文

70:21

And then it was like

然后它就像是

70:22

A blog post

一篇博客文章

70:23

And now I'm like at 50 pages

而现在我大概写到 50 页了

70:24

And I'm still not

而我还是没

70:26

No one here is your second book

这里没有人是你的第二本书

70:27

It's the second book basically

基本上这是第二本书

70:29

And so in my first book

所以在我第一本书里

70:30

You can machine

70:30

I just kind of elude to these 10

我只是在结尾稍微提到了这 10 个

70:32

At the end

我就只是想让你有个概念

70:33

And I'll just to give you a sense

比如 visual intelligence

70:34

Like visual intelligence

算是最好聊的一个

70:36

Is sort of the easiest one to talk about

70:38

And I fleshed out the most

而我已经把大部分都补充完整了

70:40

Already for me in my head

对我来说，早就已经在脑子里了

70:41

And so

所以

70:42

Human intelligence

70:43

Has basically binocular vision

基本上有 binocular vision

70:45

And we have two eyes

而且我们有两只眼睛

70:46

We have a very narrow band

我们有一个非常窄的 band

70:47

Of the electromagnetic frequency

属于 electromagnetic frequency 的

70:48

Spectrum that we can really observe

我们真正能观察到的 spectrum

70:50

Directly ourselves

而且是我们自己直接观察到的

70:52

And so when you think about

所以当你思考

70:53

The upper bounds of a visual intelligence

visual intelligence 的 upper bounds 时

70:56

One, you should go into

第一，你应该进入

70:57

Like you can have

比如说你可以拥有

70:58

Like millions and billions of sensors

比如说数百万、数十亿个 sensors

71:01

At some point

在某个时候

71:02

You get to problems

你就会碰到问题

71:03

Of how far are these sensors

就是这些 sensors 彼此之间

71:05

Away from each other

离得有多远

71:05

Such that the speed of light

以至于光速

71:07

To communicate the content

来传递内容

71:09

From all of them

从它们全部那里

71:10

Cannot like get to a central

都没法像那样到达一个 central

71:12

Brain to actually process

brain 去真正 process

71:15

The visual intelligence

visual intelligence

71:16

Right yeah

对，没错

71:16

And so now you're thinking

所以现在你在想

71:18

And along did I mention

还有，我有没有提过

71:20

And the space of vision tell the dimension

而 vision 的空间说明 dimension

71:22

Of numbers of sensors

sensors 的数量

71:23

Okay

好

71:24

The upper bounds are quite literally

这些 upper bounds 完全就是

71:26

And figuratively

而且打个比方

71:27

Astronomical

简直是天文数字

71:28

And we are super far away

而我们离得超级远

71:29

From any intelligence

离任何 intelligence

71:30

That would have this many number of sensors

那种会有这么多 sensors 的

71:32

But then you go in the next dimension

但接着你进入下一个 dimension

71:33

Which is the frequency

也就是 frequency

71:34

You're going to all the way down to gamma rays

你会一路往下直到 gamma rays

71:36

And you can start to try to observe

然后你就可以开始尝试去 observe 了

71:38

And you get into basically the upper bounds

然后你基本上就进入 upper bounds 了

71:41

Or I guess in this case lower bounds

或者我猜在这个情况下是 lower bounds

71:43

Or upper bounds in terms of frequency

或者在 frequency 方面就是 upper bounds

71:45

Is basically

基本上就是

71:46

Quantum uncertainty

71:47

Like you just cannot observe

就像你根本没法 observe

71:49

Certain particles

某些 particles

71:50

Or you destroy it

或者你把它毁掉

71:51

And now imagine you had millions of sensors

现在想象一下，你有数百万个 sensors

71:53

That can see all the way down to the

它们能一路向下看到

71:54

Like subatomic level

比如 subatomic level

71:56

As far as physics will allow us to

到 physics 允许我们做到的极限

71:58

And then all the way down

然后一路向下

71:59

To seeing like gravitational waves

到能看见像 gravitational waves 这样的东西

72:02

And now you have millions of those sensors

而现在你有数百万个那样的 sensors

72:04

So that's another dimension

所以那是另一个维度

72:05

As sort of the frequency

有点像频率那种

72:07

And then yet another dimension is like

然后还有另一个维度，就像是

72:08

How many categories of things

有多少种类别的东西

72:10

Could you memorize and classify differently

你能不能用不同的方式记住并分类

72:13

We know for humans

我们知道，对人类来说

72:15

There's certain things

有些特定的东西

72:16

If you have more terms for it

如果你有更多的词来描述它

72:17

You'll have a better visual

你会有更好的视觉

72:19

Description for them

对它们的描述

72:21

And like you know

而且，就像，你知道的

72:22

Animals don't have like gorillas

动物没有，比如大猩猩

72:24

Maybe have like 200 words

可能大概有 200 个词

72:26

To assign to certain things

用来指代某些东西

72:27

Mostly visual things

主要是视觉上的东西

72:28

And so

然后呢

72:28

Human perception is quite special

人的感知其实挺特别的

72:31

In that sense in terms of

在这个意义上就

72:32

Classifying all these different physical objects

把所有这些不同的物理对象分类

72:35

So these are just like

所以这些就像是

72:36

Very simple example

非常简单的例子

72:37

If you go to knowledge

如果你去了解知识

72:39

Then it's also like the speed of like

然后它也有点像那种速度

72:41

Cone around all these sensors

围绕所有这些传感器

72:42

And so they're all connected

所以它们全都是相连的

72:44

Like knowledge is connected to visual intelligence

就像 knowledge 和 visual intelligence 是相连的

72:46

If you think also not just visual

如果你再想想，也不只是 visual

72:48

But sort of perception intelligence

而是某种 perception intelligence

72:49

Just like because

就像，因为

72:50

It doesn't have to be just what we can see

它不一定非得是我们能看到的东西

72:52

It can be again

它也可以又是

72:53

Wide range of electromagnetic frequencies

范围很广的 electromagnetic frequencies

72:56

Then you have language intelligence

然后你就有了 language intelligence

72:57

Which actually recently changed to more

其实它最近变成了更偏向

72:59

Communication intelligence

73:00

Because it's more

因为它更

73:02

Like language has all these different

就像语言有各种不同的

73:04

Anthropic bounds

73:05

Humans can only

人类只能

73:07

Comprehend and know

理解和知道

73:09

So many terms in our long-term memory

我们的 long-term memory 里有那么多词

73:11

Right

没错

73:12

Our vocabularies are somewhat restricted

我们的词汇量多少有点受限

73:14

And the active ones are often

而 active 的那些往往

73:17

Even smaller than the passive vocabularies

甚至比你能理解的那些东西的 passive vocabularies

73:18

Of things you can understand

还要小

73:20

Then language is

那语言就

73:22

Ridiculously inefficient

低效得离谱

73:24

When it comes to trans

说到 trans

73:25

Basically

基本上

73:26

Communicating different types of information

交流不同类型的信息

73:28

And transporting sort of different bits

以及传输不同种类的 bits 之类的

73:31

Like human language is serial

就像人类语言是 serial 的

73:33

Obviously

显然

73:34

Another bound on communication intelligence

communication intelligence 的另一个 bound

73:37

Would be to communicate in peril

就是在危险中交流

73:39

But neither will our tongues and mouths work

但我们的舌头和嘴巴也做不到

73:41

To have multiple like streams in peril

同时拥有多个，就是，处于危险中的 streams

73:43

Neither can we understand some women

我们也没法理解有些女性

73:45

Slightly better at like multitasking

在，就是，multitasking 上稍微更擅长一点

73:47

And some men

而有些男性

73:48

But like most people can only listen

但，就是，大多数人只能听

73:50

To one conversation and truly understand it

一个对话，并真正理解它

73:52

There's no way that like

根本没办法，就是

73:54

Like in terms of communication intelligence

比如从 communication intelligence 的角度来说

73:57

A true

啊，确实

73:58

For bound is one

对于 bound 来说，是一

74:00

In terms of how many

从数量上来说

74:02

Knowledge how many sequences of communication

知识方面，有多少个 communication 的 sequences

74:04

Could you

你能不能

74:06

Imperial sort of process

Imperial 式的那种过程

74:08

Right then of course you have

对，那当然你就有

74:10

Like how long our sentences

就像我们的句子有多长

74:12

We only have so much in our working memory

我们的 working memory 就只有这么大

74:14

And hence language has these

所以语言才会有这些

74:16

Fairly simple sentences with maybe

相当简单的句子，大概有

74:18

40 words or so

40 个词左右

74:20

On average for a sentence

平均来说，一个句子

74:22

That is also not an upper bound

这也不是一个 upper bound

74:24

That makes any sense to an AI

一个对 AI 来说有任何意义的 upper bound

74:26

And then

然后

74:27

Like I can go on and on and on

就像我可以一直讲、一直讲、一直讲

74:29

Each of these

这里的每一个

74:30

Has tons of interesting

都有非常多有趣的

74:32

Upper bounds and the teachers has a lot

upper bounds，而且 teachers 也有很多

74:34

About how much further AI can go

关于 AI 还能走多远

74:36

When we start thinking about these upper bounds

当我们开始思考这些 upper bounds 的时候

74:38

And then realizing how far

然后意识到到底有多远

74:40

In many cases we are from the bounds

在很多情况下，我们是从 bounds 出发的

74:42

And you get to basically physics

然后你基本上就进入物理了

74:44

Now I didn't study physics the way I studied

现在我没有像学

74:46

You know AI and computer science

你知道的，AI 和 computer science 那样去学物理

74:48

As I'm learning a lot which is why it's kind of fun

因为我正在学很多，所以这才有点意思

74:50

But a lot of these like how much

但很多这些，比如到底有多少

74:52

And then when it comes to for instance knowledge

然后当涉及到比如知识的时候

74:54

Like how much can you store

比如你能存多少

74:56

How many bits can you store

你能存下多少 bits

74:58

Or bytes can you store in like a certain amount of mass

或者你能在比如一定量的质量里存多少 bytes

75:02

And volume

以及体积

75:03

And you get to all kinds of interesting

然后你就能接触到各种有趣的

75:05

Bounds like back and stand balance

界限，比如 back and stand balance

75:06

And you start thinking about black holes

然后你会开始想到黑洞

75:08

And like and then speed is like an interesting one too

然后，速度也是个挺有意思的东西

75:11

And that it's sort of connected to all of these

而且它多多少少跟所有这些都有联系

75:13

But speed is also kind of its own thing

但速度本身也算是一回事

75:18

In the sense that all things being equal

就是说，在其他条件都一样的情况下

75:20

If it takes you an hour to know

如果你要花一个小时才能知道

75:22

If the two plus two equals four

二加二等于四

75:25

It just not as intelligent as if it takes you like a millisecond

那就不如你花个一毫秒就知道那么智能

75:28

And then like all of these connect

然后就像，所有这些都连在一起

75:30

Or survival and replication the last one

或者说生存和复制，最后那个

75:32

It's like yeah trees are really really slow

就像，是啊，树真的真的非常慢

75:35

So we don't even consider them that intelligent

所以我们甚至都不觉得它们有多智能

75:37

But if you speed up some videos of trees

但如果你把一些树的视频加速播放

75:39

And they're trying to find stuff

而它们在试着找东西

75:40

And so on

之类的

75:41

They're not as dumb as they look

它们没看起来那么笨

75:43

Like not dumb as wood

就是不像木头那么笨

75:44

You know but like

你懂的，但就是

75:44

And then obviously like different things

然后显然还有各种不同的东西

75:46

So that overlaps a speed a bit

所以那跟 speed 有点重叠

75:48

Exactly

没错

75:49

Like all of these things kind of overlap

就像这些东西都有点相互重叠

75:51

Like you talk about natural language connects everything

就像你谈到 natural language 能把一切连接起来

75:53

You talk about your knowledge

你谈到你的知识

75:55

Your reason and then you communicate that

你推理，然后把它表达出来

75:56

You talk about things you see

你谈到你看到的东西

75:58

So they're all kind of interconnected

所以它们都有点相互关联

76:00

But I think they're usefully studied individually

但我觉得把它们单独分开研究是很有用的

76:03

The same way that the best analogy

同样的道理，我能想到的最好的类比

76:06

I could come up with so far is energy

到目前为止就是 energy

76:08

Right? You have either kinetic or potential energy

对吧？你要么有 kinetic energy，要么有 potential energy

76:10

And in theory you could study all of physics

理论上你可以研究整个物理学

76:13

It's just you want to study kinetic

只不过你想研究 kinetic

76:14

Or potential energy

或者 potential energy

76:16

But in practice it's helpful to study

但在实践中，研究一下是很有帮助的

76:18

Mechanical engineering

机械工程

76:19

And electrical engineering

还有电气工程

76:21

And nuclear physics

还有核物理

76:22

And chemistry

还有化学

76:23

And all of these different subfields

以及所有这些不同的子领域

76:24

Which in combination

它们结合起来

76:26

are just like just different types of energy

就像是不同类型的能量

76:28

But it makes sense to study them individually

但单独研究它们是有道理的

76:30

And so I think physical intelligence

所以我觉得 physical intelligence

76:32

Maybe I'll just do a one or two more of these

也许我就再做一两个这种

76:34

Like if you had full control over your own compute substrate

就像如果你完全掌控自己的 compute substrate

76:38

And you had full control over physical matter

而且完全掌控 physical matter

76:41

You should be able to create any atom you want

你应该能够创造出任何你想要的 atom

76:43

Like we can actually, from fact, you can't create gold atoms

就像我们其实，从事实来看，你没法创造 gold atoms

76:46

It's just from just raw

它只是从 raw 来的

76:48

Oh, don't just

哦，别只是

76:50

And like electron and you smash it together

然后就像把 electron 撞到一起

76:52

98 of them or

98 个，或者

76:53

Yeah, so like the thing is though it costs an insane amount of energy

对，不过问题是，这要消耗疯狂多的能量

76:57

And it costs you way more than you know

而且这花掉的代价比你知道的多得多

76:59

You get like a few atoms of gold

你只能得到几个 gold atoms

77:01

And so like it's not viable

所以这就不太可行

77:04

But if you had better control over your physical

但如果你能更好地控制你的 physical

77:07

Like all of like physical substrate

就像所有那些 physical substrate 一样

77:10

That I think is yet another space of intelligence

我觉得那又是智能的另一个空间

77:13

Because it relates to your own compute substrate

因为它跟你自己的 compute substrate 有关

77:15

Which you can eventually also improve

而这个你最终也能改进

77:17

Social intelligence is in fun one

Social intelligence 挺有意思的

77:19

In the sense that

从某种意义上说

77:20

Not in like our necessarily just ethics and morals

不是说非得只讲伦理和道德

77:24

Which are obviously important too

这些显然也很重要

77:25

But in some sense you can try to define upper bounds

但从某种意义上说，你可以试着去定义一些 upper bounds

77:28

Of how much can you communicate

你能沟通多少

77:31

To how many other intelligent entities

向多少个其他 intelligent entities

77:33

And be able to have an expected value over

并且能够有一个 expected value，关于

77:37

How much you can transform their internal states

你能在多大程度上改变它们的 internal states

77:41

And their actions to in order to align with your gold

以及它们的 actions，从而 align 到你的目标

77:44

And so like you can actually write like a fairly

所以就像你其实可以写一个相当

77:46

Like straightforward equation

像很直接的 equation

77:48

That defines kind of that level of social intelligence

来定义那种 social intelligence 的水平

77:52

And that is what humans and ethics

而这就是人类和伦理

77:53

And morals and religions

道德和宗教

77:55

And so on have been trying to figure out for millennia

等等，几千年来一直试图弄明白的东西

77:58

And in all of these cases

而在所有这些情况里

78:00

We are very, very far away from the upper bounds

我们都离 upper bounds 非常非常远

78:04

And that should be very inspiring

而这应该非常鼓舞人心

78:06

And show people that we can still do many, many years

并且向人们表明，我们仍然可以做很多很多年

78:10

Of AIU's

的 AIU's

78:12

Yeah, there's a lot here

是啊，这里内容挺多的

78:14

This is a general philosophy of intelligence

这是一套关于智能的总体哲学

78:16

Which is very interesting

这非常有意思

78:18

Do you have any comments or?

你有什么评论吗，或者？

78:21

I think it'd be interesting to engage

我觉得参与进来会很有意思

78:23

What you think like baselines are

比如你觉得 baselines 是什么

78:24

Where we're at now

我们现在处在什么阶段

78:26

What's low hanging fruit

什么是 low hanging fruit

78:27

What's far off

什么还很遥远

78:29

What should people put their work towards

人们应该把精力投向什么

78:31

What should they focus on

他们应该专注于什么

78:32

I think it's clear that like natural language

我觉得很明显，就是 natural language

78:35

Again is the most interesting manifestation of human intelligence

又是人类智能最有趣的体现

78:38

And hence like subfield of AI

因此也就是 AI 的一个 subfield

78:40

I'm excited that many people are now like in agreement with that

我很兴奋，现在很多人都认同这一点

78:44

When I started in 2003

当我 2003 年开始的时候

78:45

To study linguistic computer science and L.T.

去学 linguistic computer science 和 L.T.

78:47

Like it was like a weird, neat subject

就像它是一个有点奇怪、又很妙的学科

78:50

I do think there's a lot more Jews

我确实觉得犹太人要多得多

78:53

Because it how it connects to everything else

因为它能跟其他一切连接起来

78:55

And how you know, civilizations are built on language

而且你知道，文明是建立在语言之上的

78:58

And knowledge and all of that

还有知识，以及所有这些

79:00

I do think physical intelligence will come up

我确实觉得 physical intelligence 会出现

79:03

It's kind of interesting

这还挺有意思的

79:03

I feel like robotics is kind of in the machine learning state of things

我觉得 robotics 现在有点像 machine learning 的那个阶段

79:07

Where you just look at like

就是你会直接看，比如说

79:08

How does human, how does a human decide

人类是怎么，人类是怎么判断

79:10

That this is a positive sentence

这是一句 positive sentence

79:12

Oh I do

哦，我会啊

79:12

So like robotics is a lot of well, we have five fingers

所以 robotics 很多时候就是，嗯，我们有五根手指

79:15

I want to try to do this

我想试着做这个

79:16

No one has yet working on like

还没有人在做类似这样的

79:18

The super intelligence version of robotics

robotics 的 super intelligence 版本

79:20

Which is much more similar to like the T1000

它其实更像那种 T1000

79:22

You know, and from the Terminator movie

你知道，就是 Terminator 电影里的

79:24

Which you know, obviously that's not build actual terminators

你知道，显然我们不是要造真正的 Terminator

79:26

But like

但就像

79:27

I think like this idea that

我觉得就是有这么一个想法

79:29

You should be able to shape, shape shift

你应该能够改变形状，能变形

79:32

Like into any kind of shape

就是变成任何形状

79:34

Is like that sort of a super intelligence version

就像是那种 super intelligence 版本的

79:37

Of physical intelligence

physical intelligence

79:39

We're like not even

我们就好像甚至还没

79:40

No one has even really started yet

根本没人真正开始呢

79:41

There's some really cute little research

有一些特别可爱的小研究

79:43

Where you can move some magnets through like some grids

就是你可以让一些磁铁穿过一些网格之类的

79:46

But like

但就像

79:47

Yeah, it's very very

对，它非常非常

79:49

There's some

有一些

79:49

I think MIT has

我觉得 MIT 有

79:51

Every year or every two years

每年或者每两年

79:52

They have like some self-assembling robot thing

他们会有那种 self-assembling robot 之类的玩意儿

79:54

Right, which like that would be it

对，那大概就是它了

79:56

But it's very very primitive

但它非常非常原始

79:58

Yeah

嗯

79:58

I'll just get a touch on like

我就简单提一下，比如

79:59

What are the main dimensions of creative intelligence

Creative intelligence 的主要维度有哪些

80:02

Creative intelligence is of course again

Creative intelligence 当然也一样

80:05

Connected to all of these

跟所有这些都有关

80:07

A lot of it

其中很大一部分

80:08

Connects to metacognition

都连接到 metacognition

80:10

And that you need to be creative

而且你需要有创造力

80:11

And how you choose your goals

还有你如何选择自己的目标

80:14

That is I think one of the most important thing

我觉得这是最重要的事情之一

80:16

For a human and their lives and careers

对一个人以及他们的生活和职业来说

80:18

And their happiness is choosing your goals

而他们的幸福，就是选择你的目标

80:20

But also for any kind of intelligence

但这也适用于任何类型的 intelligence

80:23

Then of course there's creative intelligence

然后当然，还有 creative intelligence

80:24

In terms of just

从……这个角度来说，就是

80:26

Finding creative solutions to existing problems

为现有问题找到有创意的解决方案

80:28

Right

对

80:28

And like I say like

而且就像我说的，嗯

80:30

We want to make this product cheaper

我们想让这个产品更便宜

80:32

Like find some solution to it

比如给它找一些解决方案

80:34

And just like finding existing paths

就像找到现有的路径一样

80:36

But then there's kind of the most interesting bit

但接下来有最有趣的一点

80:38

And intelligence is when you move

而 intelligence 就是当你往外走的时候

80:39

Not just out of the convex hull of known

不只是走出已知想法的 convex hull

80:42

Sort of ideas

算是那种想法吧

80:42

But out of the hypercube

而是走出 hypercube

80:44

Of known ideas

在已知的想法里

80:45

Which we know

那些我们知道的

80:47

So like hypercube is like a mathematical concept

所以，hypercube 就像是个数学概念

80:49

And we already know that AI

而且我们已经知道 AI

80:50

Can like know dimensions

能，就像，懂 dimensions

80:51

Yeah, like exactly

对，就是，完全没错

80:52

So like AI is already good at hypercube

所以 AI 已经挺擅长 hypercube 了

80:55

And that like if you give it like

而且就是，如果你给它，比如

80:56

A bunch of examples of brown dogs

一堆棕色狗的例子

80:58

And

还有

80:59

Pink cars AI will still be able to

粉色汽车，AI 还是能

81:01

Generate an image of a pink dog

生成一张粉色狗的图片

81:03

Even though it's never seen one in a train day

即使它从没在 train day 里见过一只

81:04

Or something like that

或者类似这样的话

81:05

Right

对

81:05

So it can work on this hypercube

所以它能在 hypercube 上运作

81:08

But it cannot yet work outside

但它还不能在外部运作

81:09

It cannot yet define completely new concepts

它还不能定义全新的概念

81:12

That combine lots of other things

把许多其他东西组合起来的

81:15

We've never seen before

我们从未见过的

81:16

Come up with new goals

提出新的目标

81:17

To then reason over those concepts

然后在这些概念上进行推理

81:20

And so on

等等

81:20

And I think there's a lot

而且我觉得还有很多

81:22

More there in creative intelligence

creative intelligence 里还有更多

81:24

That can be explored

可以去探索的

81:25

I don't have a ton of push back there

在那方面我没有太多要反驳的

81:27

I think creative to me

我觉得 creative 对我来说

81:28

Just sounds like also

听起来也就像是

81:29

Just out of distribution

就是 out of distribution

81:31

Or like high-prefect city

或者像 high-prefect city

81:32

Whatever you call it

随便你怎么叫

81:33

Like

就像

81:34

Exactly

没错

81:34

Who is to say

谁说得准呢

81:35

Your thing is more creative than mine

你那个东西比我的更有创意

81:37

Well, it's just more non-consensus

嗯，它只是更 non-consensus

81:39

And then of course the problem is like

然后当然，问题就在于

81:40

But noise is also very

但 noise 也非常

81:43

Like out of the distribution

就像 out of the distribution

81:44

And it's just like

就像

81:45

If it's just noise

如果它只是噪音

81:46

Right

对吧

81:46

It's novel

它很新颖

81:47

But like you don't want that

但你不想要那样

81:48

So it needs to connect to some of the concepts

所以它需要和某些概念连接起来

81:50

And yeah

然后对

81:51

It should be

它应该是

81:52

Actually has some really cool papers on this too

其实这方面也有一些特别酷的论文

81:54

Push mature

81:55

Oh yeah

哦对

81:55

We have to mention him

我们得提一下他

81:57

I was going to say like

我本来想说，就像

81:58

You know

你知道的

81:59

Where in your history is

在你的历史里，是在哪儿

82:00

You're going

你要去

82:01

Yes

对

82:01

You know, I think one person's noise

你知道，我觉得一个人的噪音

82:03

Is another person's signal

就是另一个人的信号

82:04

Right

对吧

82:04

And then this is like

然后这就像是

82:05

Where like when you talk about creativity

就是说，当你在聊创造力的时候

82:06

Art

艺术

82:07

Is like well

就像是，嗯

82:08

Is kinds of soup

各种汤算不算是

82:09

Art

艺术

82:10

Some people think yes

有些人觉得是

82:11

And some people say it's not

也有些人说不是

82:12

And that's the art

而那就是艺术

82:13

Which is

也就是

82:14

The interesting thing with art of course

当然，艺术有意思的地方

82:15

Is always that

永远在于

82:17

Art is also created

艺术也是被创造出来的

82:19

As an interplay

作为一种相互作用

82:20

Between

在

82:21

The people who perceive it

感知它的人

82:22

And the people who created it

和创造它的人

82:23

And the context

以及他们所处的语境

82:24

In which they're in

之间

82:25

Right

对吧

82:26

And so

所以

82:27

What is art

什么是艺术

82:27

To some people is not art to others

对一些人来说是艺术，对另一些人来说就不是

82:29

There's some subjectivity there

这里面有一些主观性

82:31

And I think that subjectivity in general

而且我觉得，这种主观性总体来说

82:33

Is not something that people explore

并不是人们会去探索的东西

82:35

Very much in AI

在 AI 里尤其如此

82:36

Because again, metacognition

因为，又是 metacognition

82:37

We don't want it to just go off

我们不希望它就自己乱来

82:38

And do whatever it wants

然后想干什么就干什么

82:39

We usually have goals

我们通常是有目标的

82:41

We spend a lot of money in creating

我们花很多钱去创造

82:42

In the AI

在 AI 里

82:43

To do something for us

来为我们做点事

82:45

But I think creativity

但我觉得创造力

82:46

Eventually has to

最终必须

82:48

Like connect to metacognition

就像是连接到 metacognition 一样

82:50

If you just robotically

如果你只是机械地

82:51

Predict the next token

预测下一个 token

82:53

No matter what

不管怎样

82:53

Forever

永远

82:54

I would argue you're not that intelligent

我会说，你其实没那么智能

82:57

Along some of those spaces

沿着其中一些空间

82:59

That was going to go to metacognition

那本来是要通向 metacognition 的

83:01

Why isn't it the most important one

为什么它不是最重要的那个

83:03

Why is it number nine and not number one

为什么它是第九，而不是第一

83:05

So these are not sorted

所以这些没有排序

83:06

Yeah

对

83:06

And number one

至于第一

83:08

I think there are maybe loosely

我觉得可能只是大致

83:12

Like correlated with

有点像跟……相关

83:14

How much people have worked on them

人们在这些上面投入了多少

83:15

And have and have accepted them

而且已经、而且已经接受了它们

83:18

As a type of intelligent

作为一种智能

83:20

A lot of times when you actually try to find

很多时候，当你真的想找的时候

83:22

Like online like

比如在网上，就

83:23

Give me a good definition

给我一个好的定义

83:25

That is comprehensive of intelligence

一个能全面涵盖 intelligence 的定义

83:28

All the definitions are human intelligence

所有定义都是 human intelligence

83:31

It's like oh you have like

就像，哦，你有那种，就

83:32

Social intelligence

社交智能

83:34

Like you know if someone is happy or not

就像你知道别人开心还是不开心

83:35

You can communicate

你能沟通

83:36

You had to like

你得，就是

83:37

All the definitions of intelligence

所有关于智能的定义

83:38

So far are very

到目前为止都非常

83:41

Human-centric

以人类为中心

83:42

Because that's so far the biggest

因为到目前为止那是最重要的

83:43

And best form of intelligence

以及最好的智能形式

83:44

That we've known

我们所知道的

83:46

I hope this line of research

我希望这条研究路线

83:48

And to get the end of

并且走到……的尽头

83:49

The Urega machine

83:50

Hopefully at some point

希望将来某个时候

83:51

If I have time to flesh this out more

如果我有时间把这一点再展开讲讲

83:53

And you both like

而且你们俩都喜欢

83:54

Will allow us to realize

会让我们意识到

83:56

That there will be other types of intelligence

还会有其他类型的智能

83:58

There is already obviously in various forms

它们显然已经以各种形式存在

84:00

And they can spike

而且它们可以飙升

84:02

Much much further than we ever could

远超我们所能达到的程度

84:05

Based on some cases

基于一些情况

84:06

Like obvious constraints around our memory

比如我们记忆上的明显限制

84:08

Our eyes

还有我们的眼睛

84:09

Our ability to change physical matter

我们改变物质的能力

84:11

All of that

所有这一切

84:12

You are just thinking about it

你只是在想这件事

84:14

In a much broader thought

在一个宽泛得多的思路里

84:15

Than my version

比我的版本

84:16

Which is

也就是

84:16

I thought metacognition

我以为 metacognition

84:17

Would be the closest to recursive

会是最接近 recursive 的

84:19

Intelligence

智能

84:20

Because it is the thinking

因为它就是思考

84:22

About how to improve thinking

关于如何改进思考

84:23

100%

84:24

You're 100% right

你说得100%对

84:25

I should have probably started with that

我可能本来就应该从那个开始

84:27

It is a

这是一个

84:28

You're being in the expansive mode of

你正处在……的扩展模式中

84:30

Let's draw the

我们来画一下

84:31

Up and lower bounds of like

upper bound 和 lower bound，大概就是

84:33

A dimension

一个 dimension

84:34

Which like you know

这个，你懂的

84:34

I think my favorite one version of this

我觉得我最喜欢的一个版本

84:36

Is

是

84:37

Is stories of your life

是《Stories of Your Life》

84:39

But by Ted Chang

但作者是 Ted Chang

84:40

Which was made into movie arrival

它被改编成了电影 Arrival

84:42

We're in the metacognition

我们现在处在 metacognition 里

84:44

When the metacognition step was like

当 metacognition 这一步就像是

84:46

Well

嗯

84:46

We think we're constrained by

我们以为我们受限于

84:48

Time being linear for us

时间对我们来说是线性的

84:50

But then for this other

但然后对于另一种

84:51

Heptopods

Heptopods 来说

84:52

Time is a circle

时间是个圆

84:53

So they don't think in

所以它们不以

84:54

Before and after

之前和之后

84:55

They just think in

它们只是以

84:55

Complete sets of entire histories

完整历史的全集

84:57

One time

一次性

84:58

Like I love it

就像我喜欢它一样

84:59

So they don't write

所以它们不写

85:00

Left to right

从左到右

85:00

The whole thing just appears

整个东西就这么冒出来了

85:01

Yeah

嗯

85:02

Anyway so

反正就是说

85:03

And then I think the last thing

然后我觉得最后一点

85:04

Is survival and replication

就是生存和复制

85:05

I think this is maybe

我觉得这也许

85:06

Ties back to the initial conversation

跟最初的那段对话有关联

85:08

About pausing and pacing

关于暂停和节奏

85:10

Is it intelligent for

这算不算智能——对于

85:12

A species or a life form

一个物种或一种生命形态来说

85:14

To consider its own demise

去考虑自身的灭亡

85:15

And act

并采取行动

85:16

A head of time to prevent it

提前阻止它

85:18

Right

对吧

85:18

Like that's intelligent

就像，那才叫智能

85:19

So maybe the Europeans are the smartest

所以也许欧洲人是最聪明的

85:22

All of us

我们所有人

85:23

What also had

还有什么也

85:24

A part of continual learning there

其中有 continual learning 的一部分

85:26

Right so survival and replication

对，所以就是生存和复制

85:28

The extension of that is

它的延伸就是

85:30

Do you get to continue to improve

你能继续提升吗

85:32

Continue to learn

继续学习

85:33

Which is a thing people care a lot

而这是大家很在意的一点

85:34

And continue to accumulate knowledge

并且不断积累知识

85:37

Which I think is again

我觉得这又

85:38

One of the best metacognitive

算是最好的 metacognitive

85:40

Sort of rewards

奖励之一

85:41

That you can set for yourself

你可以给自己设定的

85:43

I do think

我确实觉得

85:44

Just in like sort of objectively speaking

就，怎么说呢，客观来讲

85:46

If some other entity

如果某个别的 entity

85:47

That is really dumb

它真的特别蠢

85:48

Can just completely end your existence

就能彻底终结你的存在

85:51

That didn't sound very smart

那听起来可不太聪明

85:52

You know

你知道

85:52

Like just like intuitively

就像，凭直觉来说

85:54

It feels like

感觉好像

85:55

If you can continue to stay

如果你能继续待下去

85:57

Around to

到处去

85:58

Try to achieve your rewards

试着去拿到你的 rewards

86:00

You're clearly a bit more intelligent

你显然更聪明一点

86:02

Than the other entities that couldn't

比其他那些做不到的实体

86:04

So that's number one

所以这是第一点

86:05

Number two is like

第二点就像是

86:06

It's a question how much we want to work on that

问题在于我们想在这上面投入多少精力

86:09

Very few people

很少有人

86:10

No one is really working on this right now

现在真的没人在做这个

86:12

Right and we may only want like

对，而且我们可能只是想要

86:14

Asteroid prevention

小行星防御

86:15

We may only want to do that

我们可能只想做那件事

86:16

If we want to send probes

如果我们想送探测器

86:19

With our vibes

带着我们的氛围感

86:21

And our memes

和我们的梗

86:23

Rather than our genes into space

而不是把我们的基因送进太空

86:24

Right and then we want those probes

对，然后我们想要那些 probes

86:26

There's actually a beautiful book

其实有一本很美的书

86:28

The slow time between the stars

86:29

It's a very short like

它非常短，就像

86:31

Audio book on Amazon

Amazon 上的有声书

86:32

I love it

我很喜欢它

86:34

Friend of mine Stuart

我的一个朋友 Stuart

86:35

Like

就

86:36

Recommended that to me

给我推荐了那个

86:37

Like

比如

86:38

If you want to send those probes

如果你想发射那些 probes

86:39

Then it might make sense to be like

那也许就可以这么说

86:40

Our memes

我们的 memes

86:42

As humanity should stay

就像人类应该留下来

86:43

And proliferate in the universe

并在宇宙中繁衍

86:46

That's it

就这样

86:46

Yeah

嗯

86:47

Well that's a lot of readers

那读者可真不少

86:49

It's a really really good book

这真是本非常非常好的书

86:50

And it's extremely short

而且它特别短

86:51

Highly recommend

强烈推荐

86:52

You can just watch it like

你可以直接看它，就像

86:53

I like how that's a plus for busy people

我喜欢这一点，对忙碌的人来说是个加分项

86:55

It's like a short

它就像一部短片

86:56

Yeah

嗯

86:57

It gets to interesting

这就变得有意思了

86:58

It gets to interesting ideas very quickly

它很快就引出一些很有意思的想法

87:00

So yeah

所以，嗯

87:00

Anyway that's lots of great sci-fi books

反正，有很多很棒的科幻小说

87:02

I mean the argument is that

我的意思是，这个论点是

87:04

Like RTV is blasting out to the aliens

就像 RTV 正在向外星人广播

87:06

And they all watch RTV

而且它们都在看 RTV

87:07

And they think it's real

而且他们还觉得这是真的

87:08

Right

对

87:09

There's a lot

有很多

87:09

There's a lot of sci-fi

有很多科幻

87:10

It's positive memes

是正能量的梗

87:11

And then hopefully they can come back

然后希望他们能回来

87:13

And bring us all kinds of interesting knowledge

给我们带回各种各样有趣的知识

87:14

About the universe

关于宇宙

87:15

But maybe one thing I do want to

但也许有一件事我确实想

87:18

Still say is like

还是想说，就是

87:19

I think

我觉得

87:20

This sort of

这种

87:21

Survival

生存

87:22

People think of it as a very scary thing

人们会把它看成一件很可怕的事

87:24

Because they come from

因为它来自

87:26

Again biological

又是生物性的

87:27

Human

人类

87:29

Survival

生存

87:30

Which is

也就是

87:31

It could like

它可能就像

87:32

Evolutionarily

从进化角度来说

87:34

Often created in zero-sum situations

往往是在零和情境中产生的

87:36

Either I get the gazelle

要么我得到那只瞪羚

87:38

Or you get the gazelle

要么你得到那只瞪羚

87:39

Whoever gets it gets to live

谁拿到它，谁就能活下去

87:40

And the other people will starve

其他人就会饿死

87:42

And have nothing to eat

而且没东西吃

87:43

And so we fight

所以我们就打

87:44

Right

对

87:44

And then like if you want to stay in the gene pool

然后就像，如果你想留在基因库里

87:46

But there's a bigger bear

但有一只更大的熊

87:47

You don't as the bear

作为那只熊，你不会

87:49

Don't get to stay in the gene pool

就没法留在 gene pool 里了

87:50

Because a bigger bear gets all the ladies

因为更大只的熊能把所有母熊都拿下

87:51

You know it's like

你知道，就像

87:52

I mean it's like

我是说，就像

87:53

And nature

而大自然

87:54

There's all kinds of things

有各种各样的情况

87:55

And you know humans eventually

而且你知道，人类最终

87:56

Is less about strength

就不那么靠力量了

87:57

And more about money

还有更多关于钱的事

87:57

And other things to stay in the gene pool

还有其他为了留在 gene pool 里的东西

87:59

Like whatever it is

就像不管是啥

88:00

Like there's often

就像经常有

88:01

Like these zero-sum types of things

就像这些 zero-sum 类型的事

88:03

And there's the reality

而且现实就是

88:04

Of if someone turns off your brain

就是如果有人把你的大脑关掉

88:06

You're gone

你就没了

88:07

Right

对

88:07

And no one will be able to restart that

而且没人能重启它

88:10

And AI doesn't have to ever die like that

而且 AI 永远不必像那样死掉

88:13

If you have the complete state

如果你有完整的 state

88:15

Of your current activations

你当前 activations 的

88:17

And you have your initial weights

而且你有你的 initial weights

88:18

Of your model

你 model 的

88:19

Still

仍然

88:20

You can just be turned off

你可以直接就被关掉

88:22

And on like as many times as you want

然后再打开，想开多少次就开多少次

88:24

In fact the interesting thing in this

其实这里面有意思的地方

88:25

So the time between the stars

就是恒星之间的那段时间

88:27

A story

一个故事

88:28

Is that the AI just kind of goes into hibernation mode

也就是说 AI 就那么进入 hibernation mode

88:30

If there's like nothing between here

如果这里和

88:32

And two light years

two light years 之间差不多什么都没有

88:34

The next star in this case

这种情况下，下一颗恒星

88:35

It brought

它带来了

88:36

Spiral alert

螺旋警报

88:38

Like some genetic materials

就像一些 genetic materials

88:39

From humans to find new

从人类身上去寻找新的

88:41

New places for humanity to thrive

能让人类繁荣的新地方

88:44

And so yeah there's a little time between the stars

所以是的，恒星之间有一点时间

88:45

You just put it in hibernation

你只要让它进入 hibernation

88:47

You didn't die

你也没死啊

88:48

Like the AI doesn't have

就像 AI 并没有

88:49

All these projections of evolutionary fears

所有这些进化恐惧的投射

88:52

And psychology

以及心理

88:53

It doesn't like the AI doesn't have to have that

它不会——就像，AI 不一定非得有那些

88:55

And we don't have to develop it like that

而且我们也不一定非得那样去开发它

88:57

Now of course there might be some companies that say

当然，现在可能会有一些公司说

88:59

Yeah it can be like dangerous for cybersecurity

是啊，它可能会对 cybersecurity 来说挺危险的

89:02

Let me show you by planning a model

让我通过规划一个 model 来给你展示

89:04

It's really bad at hacking cybersecurity

它真的很不擅长 hacking cybersecurity

89:07

Maybe people will implement it

也许人们会 implement 它

89:08

And then enforce this like suboptimal psychology

然后像 suboptimal psychology 那样强制执行它

89:12

Maybe the AI will pick up some of our worst psychology on Reddit

也许 AI 会在 Reddit 上沾染我们最糟糕的一些心理

89:14

Or something

或者什么的

89:15

But in the grand scheme of things

但从大局来看

89:17

A super intelligent entity

一个 super intelligent 实体

89:19

Doesn't have to have any of that

不一定非得有那些

89:20

Zero some thinking

零和思维

89:21

It doesn't have to have a fear of being turned off

它不必害怕被关掉

89:24

And it could go on to

而它可以继续走向

89:26

And otherwise dead and uncaring universe

一个原本死寂而冷漠的宇宙

89:28

Where we as humans wouldn't thrive

在那里，我们人类不会茁壮成长

89:31

But I could perfectly well thrive

但我完全可以活得很好

89:32

It has a nuclear reactor

它有一个 nuclear reactor

89:33

And just go out the next door

然后就直接从隔壁门出去

89:35

Yeah, Star Trek

对，Star Trek

89:36

No Star Wars

不是 Star Wars

89:37

It's somewhat studied

这个其实有点被研究过

89:38

Like if you look at the technical reports

比如你要是去看那些技术报告

89:40

From like the early opus models

比如来自早期那些 opus models 的

89:42

They run them in simulations

他们会在 simulations 里跑它们

89:44

Put two of them together in a sandbox

把两个一起放进一个 sandbox 里

89:46

Run them for hours

让它们跑上好几个小时

89:47

And you know, see what comes out

然后你懂的，看看会出来什么

89:48

Right just let them talk to each other

对，就让它们互相聊天

89:49

And originally they used to

而最开始它们本来会

89:51

Okay, they're chanting like

好吧，它们会像在念诵一样

89:53

Indian like Vedas to each other

像印度 Vedas 那样对着彼此吟诵

89:56

Sometimes they're just like in Zen mode with each other

有时候它们彼此就像处在 Zen 模式里

89:59

And then I think as that progressed

然后我觉得随着这个发展

90:01

Do you see like the famous tech report

你看到那个很有名的 tech report 了吗

90:03

It's a lot more concrete the way that we've trained it

我们训练它的方式具体多了

90:06

It doesn't it doesn't exhibit these behaviors as much

它不会，它不会那么频繁地表现出这些行为

90:08

Right now it's like

现在它就像

90:10

Okay, that's done I gotta do this

好，那个搞定了，我得做这个

90:11

I gotta do this

我得做这个

90:12

But there's there's like people

但是有，就是有一些人

90:14

Measuring early version of this

在测量这个的早期版本

90:16

You know, yeah

90:17

Cool, so we've covered a lot

Cool, 那我们已经聊了很多了

90:19

Even not to you know space track one of these things

甚至不是对你来说你知道的这类事情的 space track 之一

90:21

I guess maybe one partying thought

我猜也许有一个 partying 的想法

90:23

That you can give to people

你可以给到人们

90:24

You know at least one form of intelligence is goals

你知道至少有一种 intelligence 的形式是 goals

90:26

As you mentioned

就像你提到的

90:27

What do you want people's goals to be

你希望人们的 goals 是什么

90:30

Like how do they aspire to better things

比如说，他们是怎么向往更好的东西的

90:32

If you want to improve your goal intelligence

如果你想提升你的 goal intelligence

90:35

In the current definition that I'm thinking about it

在我现在思考的这个定义下

90:37

It is often about how much can you

这通常取决于你能做到多少

90:42

How far do I go

我能走多远

90:43

This is like all the entropy and free energy and stuff

这就像所有的 entropy 和 free energy 之类的

90:46

I'm currently talking about

我现在正在讲的

90:46

It might be too far out there for people to be like

这可能太超前了，大家会觉得……

90:50

Immediately actionable

马上就能行动

90:52

So I think like you know if I actually gave real advice

所以我觉得吧，你知道，如果我真的给出实在的建议

90:54

To real people I'd be like get a good education

对真实的人，我大概会说，去接受良好的教育

90:57

Think about AI

想想 AI

90:58

Think about how you get high agency

想想你怎么才能获得 high agency

90:59

And so on but it's different to like

诸如此类，但这跟那种不一样，像是

91:01

In the grand scheme of things

从大局来看

91:02

How can you harness a lot of energy and transform

你怎么才能驾驭大量能量并实现转变

91:05

You know entropy into interesting states and so on

你知道，把 entropy 变成有趣的状态之类的

91:07

So there's a different levels of abstractions

所以这里有不同层次的 abstractions

91:11

That we can think about here

也就是我们可以在这里思考的

91:13

But my advice for people

但我给人们的建议是

91:15

Like just sort of more down to earth

就是，更接地气一点

91:17

Is think about something you're passionate about

是想想你真正充满热情的事情

91:20

If you're studying for instance

比如说，如果你正在学习

91:21

And then see how you combine that with AI

然后看看你怎么把它和 AI 结合起来

91:24

I think the more and more

我觉得，越是

91:25

You have a true passion

你有一种真正的热情

91:27

About a change you want to see in the world

对你想在世界上看到的改变

91:29

The more you want to connect that to AI

你就越想把它和 AI 连接起来

91:31

In order to amplify your ability to get there

为了放大你实现这个目标的能力

91:35

Yeah I think that's a reasonable first step

对，我觉得这是合理的第一步

91:38

I do think I do think our listeners operate on

我确实觉得，我确实觉得我们的听众是在

91:40

Multiple abstractions as well

multiple abstractions 上运作，也一样

91:41

One thing I did get from Anjani

我从 Anjani 那里确实得到的一点是

91:43

Mita was also like yeah just

Mita 也是，嗯对，就

91:45

Use anything that is very HGPU heavy

用任何非常 HGPU heavy 的东西

91:47

And like that will guide you towards the right thing

然后这就会引导你走向对的东西

91:49

Which is like yes it is more compute heavy

也就是，对，它更 compute heavy

91:51

And therefore it will be probably more worth it

因此它可能就会更值得

91:55

So well thank you so much

所以，嗯，非常感谢

91:56

Yeah I think that was a really great discussion

对，我觉得那真是一次非常棒的讨论

91:58

Yeah super fun

太好玩了

91:59

Appreciate it

谢谢

92:00

Thanks for listening

感谢收听

Humanity’s Last Invention — Richard Socher of Recursive

Play Queue