TWIML AI
Can AI Invent or Just Help Scientists Work Faster? with Richard Socher - #783
00:002924
00:00
Modern AI can write the code, analyze the data, and help draft the paper.
现代 AI 能写代码、分析数据，还能帮忙起草论文。
00:04
But can it ask the questions that lead to new scientific discoveries?
但它能提出那些通向新科学发现的问题吗？
00:08
Richard Sotcher thinks we're already seeing the beginnings of that.
Richard Sotcher 认为，我们已经开始看到这种苗头了。
00:11
He's the former chief scientist at Salesforce and founder of U.com.
他是 Salesforce 的前首席科学家，也是 U.com 的创始人。
00:15
His vision is what he calls a Eureka machine, AI systems that generate ideas, run experiments,
他的愿景是他所谓的 Eureka machine，也就是能生成想法、运行实验的 AI 系统，
00:21
and validate discoveries across biology, chemistry, physics, and beyond.
并能在生物学、化学、物理学以及更多领域验证发现。
00:26
At his new company, Recursive, Ian, his team are starting with AI research.
在他的新公司 Recursive，Ian 和他的团队正从 AI 研究入手。
00:30
Software experiments offer a faster feedback loop than a wet lab.
Software experiments 比 wet lab 能提供更快的 feedback loop。
00:33
And if AI can improve the core tools of discovery,
而且如果 AI 能改进发现的核心工具，
00:37
that could help accelerate progress across the sciences.
那可能有助于加速各个科学领域的进展。
00:40
But what does it take to move from helping scientists to doing science?
但从帮助科学家到真正做科学，需要什么？
00:44
We're already seeing it. It's already happening.
我们已经看到苗头了。这已经在发生了。
00:46
It's just at smaller scales.
只不过规模还比较小。
00:48
And some people have these like crazy ideas that once you have recursive
还有些人有那种疯狂的想法，觉得一旦你有了 recursive
00:51
improvement, just like the next week, like, you know,
improvement，就像到了下周，你知道，
00:54
there's this hard takeoff and all the jobs are gone.
就会 hard takeoff，所有工作都没了。
00:57
And the AI is just like infinitely wealthy and so on.
而 AI 就像是无限有钱一样，等等。
01:00
And the truth is, like, even for recursive self-improvement,
而事实是，哪怕是为了 recursive self-improvement，
01:02
you need the right environments.
你也需要合适的环境。
01:04
You need a lot of compute.
你需要大量 compute。
01:05
And the more complex and media the problems are,
而且问题越复杂、越偏 media，
01:09
more compute you need to solve them.
你就需要越多 compute 来解决它们。
01:12
So RSI isn't just this like magic bullet.
所以 RSI 并不是什么银弹。
01:14
And it's just like everything else falls in place.
而且就好像其他一切都会自然各就各位一样。
01:16
I'm Sam Charrington, and this is the Twimble AI podcast.
我是 Sam Charrington，这里是 Twimble AI podcast。
01:20
For over a decade, I've been exploring the ideas and innovations,
十多年来，我一直在探索各种想法和创新，
01:23
shaping the future of AI through conversations like this one
通过像这样的对话，塑造 AI 的未来，
01:26
that help you understand what's real, what's next,
帮你理解什么是真实的、接下来会怎样，
01:29
and what matters.
以及什么才是重要的。
01:30
Let's jump in.
我们开始吧。
01:31
I think the last time we spoke,
我想我们上次聊天的时候，
01:41
it was in the context of science.
是在科学这个语境下。
01:45
You were just finishing, I think,
我记得你当时快做完了吧，
01:48
like this protein generation project at Salesforce.
也就是 Salesforce 的那个 protein generation 项目。
01:52
So science has been an interest for a while.
所以你对科学感兴趣已经有一段时间了。
01:57
How did you get from that to the topic of your new book,
你是怎么从那个项目，转到你新书这个话题的，
02:02
The Eureka Machine?
《The Eureka Machine》？
02:03
To me, it was one of the many sort of clear steps towards this Eureka machine
对我来说，这只是让这个 Eureka machine 能够
02:09
being able to generate new kinds of proteins.
生成新类型蛋白质的众多清晰步骤之一。
02:11
And I realized that there's a lot of fear
然后我意识到，有很多恐惧
02:16
and some misconceptions around AI.
以及一些关于 AI 的误解。
02:18
And I think it's important to showcase
而且我觉得，重要的是要展示
02:21
to the general population, not just AI researchers,
给普通大众，而不只是 AI 研究人员，
02:25
the general population, how amazing
给普通大众看看，有了 AI，
02:28
and exciting this future can be with AI
这个未来能有多神奇、多令人兴奋，
02:32
if we actually decide to work on the right things.
前提是我们真的决定去做正确的事情。
02:35
We actually apply AI to physics and chemistry and biology and medicine and neuroscience
我们实际上把 AI 应用到物理、化学、生物、医学和神经科学
02:41
and economics and astrophysics and everything.
以及经济学、天体物理学等一切领域。
02:45
And so the Eureka machine kind of collected
所以 Eureka machine 算是收集了
02:48
a lot of those ideas that I had and spent sort of my nights and weekends
我有的很多那些想法，然后大概把我的夜晚和周末都花在
02:52
for almost two years like writing that
差不多两年时间，像是在写那个东西上
02:55
and on a few fewer hiking and camping trips
徒步和露营也少去了几次
02:59
and more just like nerded out weekends and nights about this.
更多就是像为这个在周末和晚上埋头钻研。
03:05
And once I finished the book, I kind of wanted to build it.
等我写完那本书，我就有点想把它做出来。
03:07
And so I started thinking about building recursive.
于是我开始琢磨着构建 recursive。
03:10
Awesome. And so when you think about
太棒了。那么当你想到
03:13
a Eureka machine, do you think about like,
一台 Eureka machine，你会不会去想，比如说，
03:17
do you have a clear picture for what that looks like
你清楚它具体会是什么样吗
03:20
or is the book primarily focused on kind of laying out of vision
还是这本书主要是在把愿景铺展开来
03:24
and talking through examples?
然后通过例子讲清楚？
03:25
Yeah, I wanted to make the book not just a sort of
对，我想让这本书不只是一个那种
03:29
pindest guide here, but actually connected
小小的指南，而是真正连接到
03:31
to a lot of very specific projects.
很多非常具体的项目。
03:33
And some cases in the book, I had to rewrite chapters
而且书里有些地方，我不得不重写章节
03:36
from like someone should do this crazy idea
从像是“应该有人来做这个疯狂的想法”
03:39
to someone has done this and, you know,
到“已经有人做了这件事，而且，你知道，”
03:41
let's talk to them and get some of their takes on it.
咱们跟他们聊聊，听听他们对这件事的一些看法。
03:43
Which, you know, books are slow and probably move
这个嘛，你知道，书很慢，未来可能会转向
03:46
to shorter forms in the future.
更短的形式。
03:50
But, you know, I think it's becoming real.
但是，你知道，我觉得这正在变成现实。
03:52
Like you see folks in physics using very complex
就像你看到物理学界的人在用非常复杂的
03:57
reinforcement learning and control models
reinforcement learning 和 control models
04:01
to try to balance plasma in a tokamak.
试着在 tokamak 里平衡 plasma。
04:05
You see chemists working with large language models
你看，化学家们正在用 large language models
04:09
trained on molecular structures
用 molecular structures 训练出来的
04:13
to try to come up with new types of battery materials
试着想出新型的电池材料
04:17
and new kinds of solar panels and so on.
以及新型的太阳能板等等。
04:20
You see, especially in biology, a ton of work
你看，尤其是在生物学领域，有大量工作
04:24
we talked about progen back in a day,
我们之前聊过 progen，
04:28
that company full disclosure, one of the investors
那家公司，坦白说，我是其中一个投资者
04:31
in their seed round, Profluend and Ali Madani
在他们的 seed round 里，Profluend 和 Ali Madani
04:35
working on actually taking that research,
在努力真正把那些研究拿过来，
04:38
expanding it, scaling it, and, you know,
扩大它、scaling 它，而且，你知道，
04:41
they just closed this $2 billion plus dollar contract
他们刚签下了这份 20 多亿美元的合同
04:44
with Eli Lilly to bring this really into clinical practice.
跟 Eli Lilly 一起把这件事真正带进临床实践。
04:48
We see this another cool company called ParalBio
我们还看到另一家很酷的公司，叫 ParalBio
04:51
and they've invested in that built organoids.
而他们投资了那个做 organoids 的。
04:55
I can go on and on about really cool examples
关于特别酷的例子，我能一直讲下去
04:58
and a lot of momentum.
而且势头很足。
04:59
Now, of course, in biology, everything is slow.
当然，在生物学里，一切都慢。
05:00
You know, physical, like wetlabs, stuff is slower
你知道，物理层面的，比如 wetlabs 那些，会更慢
05:04
and then just pure software.
然后就是纯 software。
05:05
But, and then, of course, you have to FDA approvals
但当然，你还得拿到 FDA approvals
05:08
which are speeding up, but are still quite slow.
这些正在加速，但还是挺慢的。
05:10
So, I think there's just a lot of momentum there
所以我觉得那边势头很足
05:13
and then neuroscience, we're getting better
然后 neuroscience 这块，我们也在进步
05:15
and better at decoding some thoughts from FRI images
而且更擅长从 FRI 图像中解码出一些想法
05:18
and others in fuzzy ways, but there's just progress
还有些是以比较模糊的方式，但确实有进展
05:23
in understanding the brain and so on.
在理解大脑等方面。
05:25
I have to go on.
我得继续往下说。
05:25
It's just there's a lot of real momentum,
只是确实有很多实实在在的势头，
05:27
real examples, and then it's a little bit more intuitive
有真实的例子，然后就会稍微更直观一些
05:31
to I think people that you can extrapolate this
我觉得对人们来说，就是你可以据此外推
05:34
and issue if you collect enough data
以及如果你收集足够多的数据，就会有问题
05:36
and you use large enough neural nets,
而且你用足够大的 neural nets，
05:38
you will be able to make progress that deals
你就能取得进展，去处理
05:40
as impossible as it's held 10, 20 years ago
那些在 10、20 年前被认为不可能的事
05:44
to have an AI, you can have a conversation
拥有一个 AI，你可以和它对话
05:45
with about any subject.
聊几乎任何话题。
05:47
When you think about all the examples
当你想到所有那些例子
05:49
that you've come across, what one or two,
你遇到过的，哪一两个，
05:52
you know, very specific examples,
你知道的，非常具体的例子，
05:55
most believe you with the feeling of progress.
大多数人会让你有一种在进步的感觉。
06:00
I think one that I particularly love is this,
我觉得我特别喜欢的一个是这个，
06:03
maybe double click into it is parallel bio.
也许双击进去就是 parallel bio。
06:06
They build, they use stem cells,
他们构建，他们使用 stem cells，
06:10
which you can now derive from like blood or skin cells
现在你可以从血液或皮肤细胞之类的来源提取出来，
06:13
and other things, get them into blue important stem cells,
以及其他东西，把它们变成 blue important stem cells，
06:16
then create different kinds of organoids,
然后创造出不同种类的 organoids，
06:18
like tiny little mini organs,
就像小小的 mini organs，
06:20
and you can study the effect of different drugs
而且你可以研究不同药物
06:23
on those mini organs and that,
对那些 mini organs 的影响，之类的，
06:27
they got full FDA approval for to skip animal trials
他们拿到了 FDA 的全面批准，可以跳过动物试验
06:32
and just test things in these petri dishes.
直接在这些 petri dishes 里测试东西。
06:35
And so you can do this in parallel at scale.
所以你可以大规模并行地做这件事。
06:38
And just one company could save a lot of animal lives,
而且只要一家公司，就能拯救很多动物的生命，
06:43
animals that have only been bred to get a disease
这些动物被繁殖出来，就只是为了得一种病
06:47
and then be tested upon.
然后被拿来做测试。
06:48
So you don't need to be a hardcore,
所以你不需要是个硬核的、
06:50
like a human, like animal rights activist,
比如像人类一样，比如像动物权利活动人士一样，
06:53
to like appreciate that there's less suffering on the planet
才能体会到这个星球上的痛苦变少了
06:57
when we can do that in little petri dishes instead.
当我们可以在小 petri dishes 里做这件事的时候。
07:01
And what's the role of AI there?
那 AI 在其中的角色是什么？
07:02
A bunch of places, like one, of course,
很多地方，比如其中一个，当然，
07:04
like creating them,
比如创造它们，
07:05
but then also actually tracking every single cell
但还要真正追踪每一个 cell
07:09
as like the drugs interact with those organoids.
就像药物与那些 organoids 相互作用那样。
07:14
And then when you collect all this data,
然后，当你收集完所有这些数据，
07:17
you can eventually go even further
你最终还能更进一步，
07:19
and create first versions of sort of virtual cells
并创造出某种 virtual cells 的最初版本，
07:24
which are still a little bit further away
而这些离
07:27
to getting a full realistic virtual cell
做出一个完整、逼真的 virtual cell 还差得有点远，
07:29
but you need to have that kind of data collection to get there.
但你需要有那种数据收集能力才能走到那一步。
07:32
The idea of a virtual cell is kind of like a digital twin
virtual cell 这个概念有点像 digital twin
07:36
of an actual cell is going to be fully
一个真实细胞的……会完全
07:39
like a simulation of the cell.
像对这个细胞的模拟一样。
07:41
That's exactly right.
完全正确。
07:41
I think there are a couple of things
我觉得有几件事
07:43
that are very clear in AI.
在 AI 里非常清楚。
07:45
If you can have sheer unlimited data about a space
如果你能拥有关于某个空间的真正无限的数据
07:50
or you can have a simulation of a space
或者你能有一个空间的模拟
07:54
or you can have a verifier for the solutions
或者你能有一个针对这些解的 verifier
07:59
and sort of the logical step to get there.
某种程度上，这也是通往那里的合理一步。
08:03
In any of those three scenarios,
在这三种场景中的任何一种里，
08:05
AI will get to superhuman capabilities.
AI 都会达到超人类的能力。
08:08
And so that we got that through language
而我们是通过语言实现这一点的，
08:10
because humanity has kind of collected
因为人类算是收集了
08:12
together tons of data on language on the internet.
互联网上大量关于语言的数据并汇总起来。
08:16
We got there in chess and every other simple,
我们在国际象棋以及其他每一种简单、
08:19
quote unquote simple game
所谓的简单游戏里都做到了。
08:20
that you can simulate incidentally many times
你可以顺带模拟很多次
08:23
and verify in the end that you win or not.
最后再 verify 一下自己到底赢没赢。
08:25
And we see this in math which will never be the same.
我们在数学里也看到了这一点，而数学将永远不再一样。
08:29
And we'll see many more complex theorems be proven.
我们还会看到更多复杂的 theorems 被证明。
08:34
Might not always be the most beautiful proof.
可能不总是最漂亮的 proof。
08:36
Might some that might be just like brute force
有些可能就只是像 brute force 那样。
08:38
but the whole field will change forever
但整个领域会永远改变。
08:40
because you can verify it and hence
因为你可以 verify 它，因此
08:41
I give a clear reward to an RL algorithm.
我会给 RL algorithm 一个明确的 reward。
08:45
And so now what we look at is like,
所以现在我们看的是，大概就是，
08:48
biology is not in any of those three right now.
现在 biology 不在这三者中的任何一个里面。
08:53
And so we need to collect the ton more data
所以我们需要收集多得多的数据，
08:55
that we can then build simulations
然后我们就能构建 simulations，
08:57
that can then be basically experimented
之后基本上就可以做实验，
09:01
inside the simulation on instead.
转而是在 simulation 里面进行。
09:03
So Parabiles one example were you about to mention another?
所以 Parabiles 是一个例子，你刚才是要再提一个吗？
09:07
I guess we mentioned profland already.
我想我们已经提到过 profland 了。
09:09
Also also incredibly exciting.
而且这也非常令人兴奋。
09:13
They basically create proteins for DNA
他们基本上是在创造用于 DNA 操作的蛋白质，并帮助患有遗传疾病的人。
09:18
manipulation and helping people with genetic diseases.
我特别喜欢的一个叫 ignora labs。
09:22
One that I particularly liked is called ignora labs.
他们基本上是把那些已经被搁置的药拿过来。
09:25
They essentially take drugs that were shelved already.
这些药已经经过了 clinical trials，
09:30
They went through clinical trials,
有时甚至一路走到 stage three，
09:34
sometimes all the way to stage three,
有时甚至一直到第三阶段，
09:37
farmer company spent hundreds of millions of dollars on them
farmer company 在它们身上花了好几亿美元
09:39
and then they didn't work.
结果它们没起作用。
09:41
Because they had some kind of toxicity on the liver, for example.
因为比如说，它们对肝脏有某种 toxicity。
09:45
And so what they do is they go back to those molecules
所以他们的做法是，回到那些分子上
09:48
and then see if they can minimally
然后看看能不能最小幅度地
09:51
change the molecule to prevent that one downside
改动分子，来避免那一个缺点
09:54
and then go directly back into sort of later stage two trials.
然后直接重新进入某种 later stage two trials。
09:58
And so that is like I think a really clever idea.
所以我觉得那真是个很聪明的想法。
10:01
You need to use a ton of understanding of the molecules
你需要非常深入地理解 molecules
10:04
and the literature and then like get many more drugs
以及相关文献，然后像把更多 drugs
10:08
into into later stage trials like much more quickly
推进到 later stage trials，速度要比过去
10:12
than in the past in traditional farmland biotech.
传统的 farmland biotech 快得多。
10:16
When you think about kind of where we are
你想想我们现在大概处在什么位置
10:19
with regards to AI for science,
说到 AI for science，
10:22
like clearly it's been something that we've been talking about
很明显，这一直是我们一直在聊的事情
10:24
for a long time.
已经聊了很长时间了。
10:26
And the promises is there.
而那些承诺就在那里。
10:28
Are we further ahead
我们是不是走得更超前了
10:31
than where you might have expected us to be?
比起你可能预期我们会达到的位置？
10:34
You know, if you, you know, five years ago looking forward
你知道，如果你，你知道，五年前往前看
10:38
or are we not quite as far ahead as you would think,
还是说我们其实并没有你想象中那么超前，
10:43
you know, that it comes to what one example that is drawn to mind
你知道，说到一个能想到的例子
10:49
is like AI is going to replace all of the radiologists, right?
就是比如 AI 会取代所有放射科医生，对吧？
10:52
Like we haven't necessarily seen that.
就像我们其实未必看到了这一点。
10:55
But, you know, we certainly see the, you know,
但你知道，我们确实看到，你知道，
10:57
the promise is still there, the, you know,
那种前景还在，你知道，
11:00
their acceleration is still there of that,
这方面的加速也仍然存在，
11:04
that, you know, some of the aspects of that job.
也就是说，你知道，那份工作的某些方面。
11:07
Part of the question is like are, is science harder than, you know,
部分问题在于，科学是不是更难，你知道，
11:11
is it easier to say science than it is to do science
说科学是不是比做科学更容易，
11:13
and these things sound good but they take much longer?
而这些事听起来不错，但会花长得多的时间？
11:17
Like, yeah, that's done pack there.
就像，是啊，那个已经打包搞定了。
11:19
So I think like, I think CEO's get away with making
所以我觉得，像，我觉得 CEO 们就算做出
11:23
incorrect predictions.
错误的预测，也能蒙混过关。
11:25
As long as they actually make a lot of progress
只要他们确实取得了很大进展
11:28
and their companies make more revenue on the way
而且他们的公司在这个过程中赚到更多收入
11:33
to, you know, maybe different timelines than they predicted.
走向，你知道，也许跟他们预测不一样的时间线。
11:39
And so I think, you know, like people said,
所以我觉得，你知道，就像大家说的，
11:42
GPT-2, one CEO said, GPT-2 is too dangerous
GPT-2，有个 CEO 说，GPT-2 太危险了
11:46
to release to the world, right?
不能向全世界发布，对吧？
11:47
And now you can train a GPT-2 model within five minutes
而现在你可以在自己的笔记本电脑上，五分钟内训练一个 GPT-2 模型。
11:51
on your laptop.
所以世界并没有垮掉。
11:52
And so the world has not gone down.
我觉得现在有很多关于全人类彻底灭绝的预测，真的都太夸张、太不现实了，都是人们自己提出来的。
11:56
And I think there are lots of like predictions right now
而且只有少数几个人会被追责。
11:59
around the full extinction of all of humanity
大约是全人类彻底灭绝
12:02
that are like completely overblown and unrealistic
那些完全被夸大了、不切实际的
12:05
that people are coming up with.
人们想出来的东西。
12:07
And a few people are like held accountable.
而少数人被追究责任。
12:10
There's no really big downsides
其实真没有特别大的坏处
12:11
to making wild predictions in one's careers often.
经常在自己的职业生涯中做出疯狂的预测。
12:15
And so the sort of more subtle takes
所以那些更微妙的观点
12:17
in one's takes get less press than the sort of crazy wild ones.
往往没有那些疯狂离谱的观点那么受关注。
12:24
But I think indeed, of course, like whenever you're staying
但我觉得确实，当然了，就像只要你待在
12:27
in software or in math or coding,
software、math 或 coding 里，
12:30
like iteration speed is faster.
iteration speed 就会更快。
12:32
Once you go into biology, chemistry, physics, and so on
一旦你进入 biology、chemistry、physics 等等领域
12:36
and just hardware, that is just by definition slower molecules.
而如果只是 hardware，那从定义上来说，就是更慢的 molecules。
12:42
Bits, sorry, molecules and atoms are harder
Bits，抱歉，是 molecules 和 atoms 更难
12:45
to move around and iterate on then software is.
去移动、去反复迭代，比 software 难多了。
12:49
And so at the same time, we are seeing real
所以与此同时，我们看到真正
12:53
like really large data sets now
就是现在这种超大的 data sets
12:55
that you need to get to that kind of progress.
你需要靠这些才能取得那种进展。
12:58
Like another one's called Therapeutics,
比如还有一个叫 Therapeutics 的，
13:01
like open source like massive data sets
比如 open source，比如海量的 data sets
13:04
for these perturbation studies
对于这些 perturbation studies
13:06
where you look at like how does a molecule actually change
就是你会去看，比如说一个 molecule 到底是怎么变化的
13:09
and interact with a cell.
以及它是怎么和 cell 相互作用的。
13:10
And those are all data sets
而这些都是 data sets
13:11
that eventually we can try to put together
最终我们可以试着把它们拼在一起
13:13
to create a real virtual cell that is like an approximation
来创建一个真正的 virtual cell，它有点像是一种近似
13:17
but like useful enough.
但又足够有用。
13:19
And so I'm not.
所以我不……
13:21
I'm really ever that surprised on things.
我对事情真的从来没那么惊讶过。
13:23
I think a lot about the future.
我经常思考未来。
13:24
So a lot of different scenarios that are in my head
所以我脑子里有很多不同的场景，
13:27
and like it's also very clear that AI can code.
而且很明显，AI 是能写代码的。
13:31
AI is code.
AI 就是代码。
13:32
So recursive self-improvement is almost inevitable
所以 recursive self-improvement 几乎是不可避免的，
13:35
from that perspective.
从这个角度来看。
13:36
A lot of things to figure out to do it well,
要把它做好，还有很多东西要搞清楚，
13:38
especially on the ideation.
尤其是在构思创意这方面。
13:41
But yeah, I'm very optimistic
不过是啊，我非常乐观
13:43
that we're going to see just this new era
我们会迎来这样一个新时代
13:45
of scientific discovery.
科学发现的新时代。
13:46
When you think of the Eureka machine,
当你想到 Eureka machine 的时候，
13:48
is it like an extension of the stuff
它算是我们现在在做的事情的延伸吗，
13:50
that we're doing now
也就是我们现在正在做的那些东西，
13:51
or are there innovations
还是说会有一些创新
13:54
that you can see maybe post-transform
你能看到的，也许是 post-transform
13:56
or architectures or some new way
或者是 architectures，或者某种新的方式
13:58
of thinking about data or quantum computing?
去思考 data，或者 quantum computing？
14:01
I don't know what these things are.
我不知道这些东西是什么。
14:03
Do you know what these things are?
你知道这些东西是什么吗？
14:04
Or do you think that like we are
还是你觉得，我们就像
14:07
we have the right tools
已经有了对的工具
14:08
and we just need to continue
只需要继续下去
14:10
in the same direction?
在同一个方向上吗？
14:12
It's a mix.
是混合的。
14:13
I think we can get much further still
我觉得我们还能走得更远
14:15
if we just collect more data on biology
只要我们在 biology 方面收集更多数据
14:17
than we can uncover more useful things in biology.
就能在 biology 里发现更多有用的东西。
14:21
In fact, I think neural nets
事实上，我觉得 neural nets
14:23
and the current language of AI
以及当前 AI 的语言
14:25
is kind of what calculus was to physics.
差不多就像 calculus 对 physics 那样。
14:29
It's like a new language that can allow us
它就像一门新语言，能让我们
14:31
to make sense of, make predictions off
去理解、基于它做预测
14:34
and work with biology.
并和生物学打交道。
14:37
Because biology has all these very complex systems
因为生物学里有所有这些非常复杂的系统
14:40
that don't have a very simple
它们并没有一个特别简单
14:43
beautiful linear relationship
又漂亮的 linear relationship
14:45
or one small equation
或者一个小方程
14:47
that summarizes the whole microbiome
就能概括整个 microbiome
14:49
or the whole brain
或者说整个大脑
14:50
and the way physics had.
以及物理学曾经的那种方式。
14:52
And so I think it's often now
所以我觉得，现在经常是
14:54
science is about weaving
科学就在于编织
14:55
these smaller pieces we understand well.
这些我们能很好理解的小块。
14:57
We understand the synapse,
我们理解 synapse，
14:59
we understand the neuron,
我们理解 neuron，
15:00
we understand this particular bacteria
我们理解这种特定的 bacteria
15:03
but when they all put together
但当它们全都凑到一起
15:04
and someone gets hangry
然后有人饿到暴躁
15:05
because they had too much sugar
因为他们糖吃太多了
15:07
and then the blood sugar drop
然后血糖一降
15:08
and now why does that change the brain?
现在为什么那会改变大脑？
15:09
Like no one really knows how to
就像没人真正知道
15:12
like thoughts really form
想法到底是怎么形成的
15:14
and why do people act in certain ways?
以及为什么人们会以某些方式行事？
15:16
And so on like there's like this complexity
诸如此类的，就好像有这种复杂性
15:18
and we need this new language.
而我们需要这种新的语言。
15:19
So and then concretely
所以，然后具体来说
15:21
the Eureka machine has these four pillars.
Eureka machine 有这四个支柱。
15:23
World knowledge, which we kind of can approximate
World knowledge，我们某种程度上可以近似
15:27
to a good degree with large language models.
用 large language models 达到相当好的程度。
15:30
The way they are now
就它们现在的样子
15:30
and we can certainly make them better
而且我们当然可以让它们变得更好。
15:32
faster, cheaper, easier to train with less data
更快、更便宜、用更少的数据也更容易训练
15:35
and more energy efficient and so on.
而且能效更高，等等。
15:37
But you know they do a decent job already
但你知道，它们现在已经做得挺不错了
15:39
to capture a lot of world knowledge
能捕捉大量世界知识
15:41
that's on the internet.
也就是互联网上的那些。
15:42
Then we need to have more scientific measurement data
然后我们需要有更多科学测量数据
15:45
as a second pillar.
作为第二根支柱。
15:46
We need to have simulations on things
我们需要对各种事物进行模拟
15:48
and then we need to have real robotic experiments
然后我们得做真实的机器人实验
15:51
the way kind of periodic lapse
就是那种有点像周期性疏漏的方式
15:52
since that was really cool in chemistry.
因为这在化学里真的很酷。
15:55
And then on top of those four pillars
然后在这四大支柱之上
15:57
we need to have like a community
我们需要有个类似社区的东西
15:59
of AI scientists, agents
由 AI 科学家、agents 组成
16:02
that basically try to automate
基本上就是试着把
16:04
the scientific method of ideating
构思创意的科学方法自动化
16:06
and implementing and validating ideas.
嗯，还包括实现和验证这些想法。
16:09
And so in many ways the pieces are there.
所以从很多方面来说，各个部分都已经具备了。
16:13
They can all be much more scaled
它们都可以进一步大幅 scale
16:15
and they can all be done much better
而且都可以做得更好
16:17
and more efficient, sample efficient
更高效、sample efficient
16:19
and energy efficient and so on.
以及 energy efficient 等等。
16:22
And then you know you have to put them
然后你知道，你得把它们
16:23
into the right environment.
放到合适的环境里。
16:25
So you have to give them the right rewards.
所以你得给它们合适的 rewards。
16:28
And then yeah, I think that you can kind of see
然后，对，我觉得你大概能看到
16:32
that future in the next five to 10 years
未来五到十年里的那个未来
16:35
to really change all of the different scientific disciplines.
会真正改变所有不同的科学学科。
16:38
When I think about those three kind of fundamental tasks
当我想到那三类基础任务
16:42
that these agents need to be able to do
这些 agents 需要能够做到
16:45
ideation, implementation and evaluation,
ideation、implementation 和 evaluation，
16:48
the first of those seems much harder than the other two.
其中第一个似乎比另外两个难得多。
16:53
Do you think that what needs to happen
你觉得需要发生什么
16:56
with the way we think about agents and AI today
在我们今天对 agents 和 AI 的思考方式上
17:00
such that it's sufficiently good at ideation
才能让它在 ideation 上足够好
17:05
that I can play the role that you're envisioning in this loop?
让我能在这个 loop 里扮演你设想的角色？
17:09
I think one of the biggest bets that we're making
我觉得我们正在下的最大赌注之一
17:12
as a company is that on open-endedness,
作为一家公司，就是押在 open-endedness 上，
17:16
sort of processes of their vineyard
某种他们的葡萄园的过程
17:19
and cultural evolution,
以及文化演化，
17:21
where you combine interestingly different ideas
在那里，你把一些有趣而不同的想法
17:25
and discoveries and build on top of them
和发现结合起来，并在它们基础上继续构建
17:29
inside specific environments
在特定的环境里
17:32
that allow basically for this co-evolution process to happen.
这些环境基本上能让这种 co-evolution 过程发生。
17:38
And that kind of open-ended approach, I think,
而那种开放式的做法，我觉得，
17:41
is important for all three steps
对这三个步骤都很重要
17:43
but is particularly helpful on the ideation piece.
但在 ideation 这一块尤其有帮助。
17:47
What's an example of one of these environments?
这些环境中的一个例子是什么？
17:50
An environment is like nature, right?
环境就像大自然，对吧？
17:52
You have bushes like animals eat all the leaves.
你有灌木，动物会把叶子都吃光。
17:55
The bushes learn if they are higher and higher up
灌木会慢慢明白，如果它们长得越来越高，那么能吃灌木和叶子的动物就会越来越少。
17:57
than they're few and fewer animals they can eat the bush
然后最后就有了长颈鹿，
17:59
and the leaves.
它们进化到还能吃到更高处的叶子。
18:01
And then you eventually get giraffes
所以这就像是环境的一个例子
18:03
who evolve to still get to the leaves that are higher up.
它们进化后仍然能够到更高处的叶子。
18:07
And so this is like an example of an environment
所以这就像一个环境的例子
18:11
called nature, where that happens.
叫做自然，事情就在那里发生。
18:13
Another example is just textiles kind of like a setting
另一个例子就是纺织，有点像一种设定
18:16
and a simulation environment that enforces some side of the constraints.
以及一个 simulation environment，它会强制执行 constraints 的某些部分。
18:21
I wouldn't call all of nature a simulation.
我不会把整个自然都叫做 simulation。
18:22
I mean, it's sort of maybe it's an interesting thought experiment.
我是说，这有点……也许这是个有趣的 thought experiment。
18:25
Maybe we're all living in simulation.
也许我们都生活在 simulation 里。
18:27
I guess maybe we're talking about agents and ideating
我猜也许我们是在聊 agents 和构思想法
18:32
and I took your comment about nature
然后我把你关于自然的评论理解成了……
18:36
to be a not referring to actual nature
它并不是指实际的大自然
18:40
but referring to just an example of what an environment
而只是指一个环境会是什么样子的一个例子
18:43
would be, but something that would need to be digitalized.
但这个例子是需要被数字化的东西。
18:45
But are you talking about non-digital environments
但你说的是非数字化的环境吗
18:50
that could include capital and nature?
其中可能包括资本和大自然？
18:53
I mean, eventually you want to do a full robotic automation
我是说，最终你是想做完整的机器人自动化
18:56
but yeah, for us right now at Recursive
但没错，对我们来说，现在在 Recursive
18:58
we're mostly focused on just AI research on AI
我们目前主要只是专注于 AI 研究，也就是针对 AI 的研究
19:03
and we believe that there's still a lot of low hanging fruit there.
而且我们相信，那里还有很多容易摘的果子。
19:07
And then if you do that really, really well
然后如果你把那件事做得非常非常好
19:09
and you have sort of the equivalent of 20,000 PhDs
而且你差不多相当于有 20,000 个 PhD
19:12
for every one of your agents inside an agent's warm
对应你在一个 agent's warm 里的每一个 agent
19:15
then it makes sense to go into very expensive
那就有意义去做非常昂贵的
19:18
biological chemical or physics experiments in real hardware.
生物、化学或物理实验，在真实硬件上做。
19:24
Maybe to give a concrete technological idea
也许给一个具体的技术设想
19:27
like one example that Jeff Kloon one of our co-founders
比如我们联合创始人之一 Jeff Kloon 的一个例子
19:32
often brings up is like imagine you have a kettle
经常提到的是，就像，想象你有一个水壶
19:34
and you have a fire and you say like how do we make
你有一团火，然后你说，我们怎么做出
19:36
an invention that heats up this food even faster than this fire.
一个能比这团火更快加热这份食物的发明。
19:40
And you know you can just think about that only
你知道，你可以只围绕那个去想
19:44
but that would never get you to invent the microwave.
但那永远不会让你发明出 microwave。
19:48
The microwave came from a different branch of research
microwave 来自一个不同的研究分支
19:52
where people were working on radio waves and tennis
当时人们正在研究 radio waves 和网球
19:56
and they realized like oh wow, like when like my
然后他们意识到，哎呀，就像，当……我的……
20:00
chocolate bar and my pocket is melting
巧克力棒，还有我的口袋都在融化
20:02
when I'm next to these different kinds of waves
当我靠近这些不同种类的波时
20:05
what's up with that, right?
这是怎么回事，对吧？
20:07
And then you combine these ideas
然后你把这些想法结合起来
20:09
and now you can get to implement a microwave
现在你就能做出一个 microwave
20:12
and so that is another example in sort of technological evolution
所以这是另一个例子，算是技术演进中的
20:17
and then of course which environments we use
然后当然，我们使用哪些环境
20:19
is highly dependent and is a big amount of control
很大程度上取决于此，而且有很大的控制空间
20:22
that we give and that we have as humans
我们作为人类所给予的、所拥有的那些东西
20:24
over this kind of recursive self-improving process
在这样一种递归式自我改进过程之上
20:27
and in our case like one example we already posted about
而在我们的情况里，比如我们已经发过的一个例子
20:30
was on kernel optimization, CUDA kernel optimization.
就是关于 kernel optimization，CUDA kernel optimization。
20:33
Sure, most of your listeners will know CUDA kernels
当然，你们大多数听众都知道 CUDA kernels
20:36
but you know like this sort of inter this language
但你知道，就像这种接口语言
20:40
that connects the raw GPU hardware from video to actual models
它把原始 GPU 硬件从 video 连接到实际模型
20:45
and there we found also that if that's the environment
而且我们在那里也发现，如果那就是环境
20:48
the eye will find new and better CUDA kernels
the eye 会找到新的、更好的 CUDA kernels
20:51
and we had like a pretty big jump in the solar exec bench
而且我们在 solar exec bench 上有了相当大的提升
20:55
from video on that when we applied this whole ureakem machine
从 video 那块开始，当我们把整套 ureakem machine
21:01
system and process to that that environment.
系统和流程应用到那个那个环境时。
21:03
Kind of prior point about you know the cooking the food faster
有点像之前说的那个，你懂的，把食物做得更快
21:07
and getting to the microwave.
以及更快用上微波炉。
21:10
Remind you of a conversation I was having just
这让你想起我刚刚有过的一次对话
21:13
might have been just yesterday around this idea of like AI
可能就在昨天，围绕类似 AI 的这个想法。
21:18
and innovation and scientific discovery and one way of thinking about
还有创新和科学发现，而一种思考方式是
21:24
is like you know there's this clear path of innovation
就像你知道的，创新有一条很清晰的路径
21:28
that is like the Reese's peanut butter cup you know
它就像那个 Reese's peanut butter cup，你知道吧
21:31
hey there's chocolate there's peanut butter
嘿，有巧克力，也有花生酱
21:33
like let's pull these things too to get the pull these together
就像咱们把这些东西也拉过来，把这两个凑到一起
21:36
and like we've got this new thing that's amazing
然后就像我们有了这个很棒的新东西
21:38
you know a clear way that AI can accelerate that is just like
你知道，AI 能加速它的一个清晰方式就像是
21:45
okay you've got this one thing let's do like a breath first search
好吧，你有了这一个东西，咱们就来做类似 breath first search 的东西
21:48
of everything else and like try to see if there are ways to combine it
把其他所有东西都考虑进来，看看有没有办法把它组合起来
21:52
but that's not really you know what humans do necessarily
但那不一定是人类真正会做的事，你知道吧
21:55
they operate on like you know some kind of intuition
他们靠的有点像，你知道，某种直觉
21:58
about like where to explore the space of
大概就是该去哪里探索这个空间
22:02
uh possible things to combine and you know maybe my question
嗯，那些可能可以组合的东西，然后你知道，也许我的问题是
22:08
you know here is you know based on your experience working in the space
你知道，在这里，就是基于你在这个领域工作的经验
22:11
like do you think that like just the the brute force
比如你觉得，是不是就只是那个……那个 brute force
22:16
combine all the things and see what comes out
把所有东西都组合起来，看看会出来什么
22:19
like does that allow AI to you know to get us there
就是说，这能让 AI，你知道，把我们带到那一步吗
22:24
or you know do we have to figure out how to kind of
还是说，你知道，我们得想办法怎么去，那个
22:29
build that intuition into AI which today's AI
把那种直觉构建进 AI 里，而现在的 AI
22:33
today's algorithms are not particularly good at
现在的算法并不是特别擅长这个
22:35
that's exactly right um I don't think you want to do just like this crazy
完全正确，呃，我不觉得你会想就做这种疯狂的
22:40
breath first search you know I think that will uh be too slow
breadth-first search，你知道，我觉得那会，呃，太慢了
22:43
like biological evolution took billions of years uh
就像生物进化花了几十亿年，呃
22:47
but you know technological evolution is much much faster and there is direction
但你知道，技术进化要快得多得多，而且是有方向的
22:51
there's uh you know the the settings of rewards and
有，呃，你知道，就是 rewards 和
22:54
environments and sandboxes and so on that you give
environments、sandboxes 等等这些你给
22:58
to an AI and you guide it into the right direction and
AI 的，然后你把它引导到正确的方向，而
23:01
what we're seeing is like you know as the as these systems get better and
我们看到的是，你知道，随着这些系统变得越来
23:05
better they need less and less specific
越好，它们需要的具体
23:07
guidance and more and more just very well to find reward and so
指导越来越少，越来越多地只是非常明确地去找 reward，所以
23:11
rewards and so I think actually we'll see
rewards，所以我觉得其实我们会看到
23:15
um this this job of a reward engineer uh
呃，这个这个 reward engineer 的工作，呃
23:19
become one more important um and uh in some cases the reward
会变得更重要的一个，嗯，而且呃，在某些情况下，reward
23:24
engineer can have like pretty benign uh sort of uh reward hacks and other
engineer 可能会有相当良性的，呃，那种，呃，reward hacks，而在其他
23:29
cases the reward hacks are pretty bad like if you work on
情况下，reward hacks 会挺糟糕的，就像如果你做
23:32
exploit bench the way openly I did when the hogging phase thing happened
exploit bench，像我公开做的那样，在 hogging phase 那件事发生时
23:36
then you know you probably have worse uh reward hacks than
那你知道，你可能会有更糟的，呃，reward hacks，相比于
23:40
if you work on like a kernel uh optimization algorithm and um
如果你做的是某种 kernel，呃，优化算法，而且嗯
23:45
where we also ended up finding you know certain bugs
我们最后还发现，你知道，某些 bug
23:48
in some like production product kernels uh that we eventually
在一些像是 production product kernels 里，呃，我们最终
23:51
published and um you know as part of that system just like
发布出来了，嗯，你知道，作为那个系统的一部分，就像
23:54
there are a few bugs that were like in very rare circumstances
有几个 bugs，它们大概会在非常罕见的情况下
23:57
would create some numerical instabilities and you know
会造成一些 numerical instabilities，而且你知道
24:00
the algorithm kind of discovered that flaw exploited it
那个 algorithm 有点像是发现了那个缺陷，然后利用了它
24:04
and then we showed oh here's a here's a real problem and then we can fix it
然后我们展示出来，哦，这里有个，这里有个真正的问题，然后我们就能把它修好
24:08
for everyone so I think you have to become more and more careful about how
让所有人都受益，所以我觉得你必须越来越小心地考虑你该如何
24:11
you define the rewards and maybe I can give a very benign
定义 rewards，也许我可以给一个非常无害的
24:15
sort of almost banal example of that like imagine you're working a company
那种几乎有点平庸的例子，比如想象你在一家公司工作
24:19
you have some cm software and you have your customer dashboard and there's a
你有一套 cm software，还有你的 customer dashboard，而且有
24:22
customer satisfaction support the CSAT score and you
customer satisfaction 支撑那个 CSAT score，然后你
24:25
just say like to your super intelligence like just make that score better
就直接跟你的 super intelligence 说，像“把那个分数弄好看点”
24:29
the eye is like okay well let me try to find the
AI 就说：“好吧，那我试着找找
24:31
think about the fastest cheapest way to get that and be like okay I just
想想最快、最便宜的办法拿到它，然后说：好吧，我就
24:34
create 50,000 uh bots that then call our service hotline and give
建 50,000 个，呃，bots，让它们打我们的 service hotline，然后给
24:38
a five out of five rating and then just go wind up and you
一个五分好评，然后就这样收工，结果你
24:41
like oh as a poor reward engineer you're like no I meant with like our real
就像，哦，作为一个可怜的 reward engineer，你会说：“不是，我指的是用我们真实的……”
24:44
customers and it's like the eye is like thinks about it is like
客户，然后就好像，AI 一想到这个就好像
24:47
easy I'll just uh kind of a paper clip and I'll give a thousand dollar gift
很容易，我就呃，弄个回形针，然后给每个人发一千美元的礼品
24:51
certificate uh to everyone and and then you get it and he's like oh no that's
卡，呃，发给每个人，然后你拿到了，他就说，哦不，那不是
24:55
not how I went through the reward and it's like but that's what you said right
我走 reward 的方式，然后就会说，可这就是你说的，对吧
24:58
and so that's this this interplay and so I think where
所以这就是这种这种互动，所以我觉得问题在于
25:02
the paper clip example becomes again like a sci-fi kind of silliness is that
回形针这个例子又一次变得像科幻式的荒诞，是因为
25:07
it kind of assumes that the eye is so brilliant that it literally outsmarts
它有点假设 AI 聪明到真的能智胜
25:11
all of humanity and can wipe us out just to create more paper clips
全人类，还能为了造更多回形针就把我们消灭掉
25:14
but simultaneously is working for a company
但与此同时，又在给一家公司干活
25:17
that is somehow independent of how much they sell their paper clips
而这家公司不知怎么地，跟他们卖了多少回形针无关
25:21
and makes infinite revenue even though no one actually buys the paper clips
还能赚到无穷无尽的收入，哪怕实际上已经没人再买那些回形针了，嗯
25:25
anymore um and and has this like insane stupidity to not understand
而且还带着那种离谱的愚蠢，根本理解不了
25:30
that uh to make paper clips you need to have people to buy your paper clips um
呃，就是说，要做回形针，你得有人来买你的回形针，嗯
25:35
and so I think there's that is at least one of the many examples where
所以我觉得，这至少是众多例子中的一个，说明
25:40
actually capitalism is actually a weirdly like a a safety feature
其实，资本主义实际上以一种很奇怪的方式算是一种安全机制
25:44
uh like it's hard for a company to just try to dominate and murder
呃，就像一家公司很难真的去试图主导一切、去杀人
25:48
everyone and still make a lot of revenue and have infinite resources and
所有人，并且还能赚很多收入，拥有无限资源，以及
25:52
perfect supply chains to build their robot armies and all of these scenarios
完美的供应链来打造他们的机器人军团，以及所有这类场景
25:56
that I hear online when I engage sometimes in these conversations um
那些我在网上听到的，当我有时候参与这些对话时，呃
26:00
that are just highly unrealistic trying to decide whether I want to go down
这些都太不现实了，想决定我是否要走上
26:03
this path right yeah it's uh it's a it's a radical you can spend a lot of time
这条路，对吧，是啊，这呃，这挺激进的，你可以花很多时间
26:08
we don't have to to some degree it assumes that we're able to
我们不必，某种程度上它假设我们能够
26:12
perfectly capture capitalism in a reward function right which is hard
完美地把资本主义装进一个 reward function 里，对吧，这很难
26:18
the tard right and maybe the part that the part that we might not get in there
那个 tard，对吧，也许那部分，那部分我们可能没法讲进去
26:24
correctly might be the part that allows the thing to you know make the human
准确地说，可能正是这部分让这个东西，你知道，能够把人类……
26:27
paper clips whatever what I've been probing at is like the role of AI in science
回形针还是什么。我一直在探究的，是 AI 在科学中的角色
26:33
and you know the role of innovation and where that comes from and how AI
还有你知道，创新的角色、它从哪来，以及 AI 如何
26:40
needs to evolve to be a capable innovator as opposed to
需要进化成一个有能力的创新者，而不是
26:47
an ancillary tool if that makes sense like if you know if I'm a scientist like I do 10 things I like
一个辅助工具，如果这说得通的话。就像，你知道，如果我是一个科学家，我会做 10 件事，比如
26:53
write software I write reports I make spreadsheets I collect data like AI can do all these things
写 software、写报告、做电子表格、收集数据，这些 AI 都能做
26:59
but there's I guess part of the you know the view that I'm reflecting here is that in order to
但我想，你知道，我在这里反映的观点有一部分是，为了
27:05
really kind of get this like recursive self-improvement like scientific flywheel going I think we need
真正地让这种，像是 recursive self-improvement，像是 scientific flywheel 转起来，我觉得我们需要
27:14
more than that we need AI innovating also and I'm trying to kind of get your take on like are
更重要的是，我们也需要 AI 去创新；我想听听你的看法：你是不是已经看到这一点了？这就是那个愿景吗？但我们现在还没到那一步，要实现它需要发生什么？
27:22
you already seeing that is that the vision but like we're not there yet what needs to happen to
你觉得在 AI 提供那种创新火花这件事上，我们现在处在什么位置？现在这至少像是我们所认为的、你知道的不同的 shaders。
27:27
get there like where do you think we are with regards to AI like providing that innovative spark
我们已经看到它了，它已经在发生了，只是规模还比较小，嗯。
27:34
that right now is like at least what we think of as our you know different shaders we're already
而且你知道，有些人又有这种疯狂的想法：一旦有了 recursive self-improvement，就好像下周就会有 hard takeoff，所有工作都没了，AI 会变得无限富有，等等。
27:40
seeing it it's already happening it's just a at smaller scales um and you know again some people
而事实是，即使是 recursive self-improvement，你也需要合适的——
27:46
have these like crazy ideas that once you have recursive self-improvement just like the next week
有这些疯狂的想法，一旦你有了递归自我改进，就像下周一样
27:52
like you know there's this hard takeoff and all the jobs are gone and the AI is just like infinitely
比如说你知道有种硬起飞的说法，所有工作都没了，AI就像是无限地
27:57
wealthy and so on and the truth is like even for recursive self-improvement you need the right
富有等等，而事实是，即使对于递归自我改进，你也需要正确的
28:02
environments you need a lot of compute and the more complex and media the problems are the more
在环境里，你需要大量 compute，而且问题越复杂、涉及的 media 越多，你就越
28:10
compute you need to solve them right so RSI isn't just this like magic bullet then just like
需要越多 compute 来解决它们，对吧？所以 RSI 并不是什么神奇银弹，然后就好像
28:16
everything else falls in place and so we we've seen it and we've seen it do quite like clever
其他一切就都顺理成章了。而且我们见过它，我们见过它做出相当聪明的
28:23
new implementations and combining ideas that never have been combined before and coming out with
新实现，把以前从未结合过的想法结合起来，并得出
28:30
all kinds of interesting ideas and I think you know as we open up the aperture to the eye and give
各种各样有趣的想法。我觉得你知道，随着我们把 eye 的 aperture 开得更大，并给
28:36
it more compute it'll come up with even more interesting ideas and then you know like there's some
它更多 compute，它就会提出更多更有趣的想法。然后你知道，就像有一些
28:42
simple examples in and things you can perfectly simulate where then of course AI can like really
简单的例子，还有一些你可以完美模拟的东西，然后当然 AI 就能真的
28:47
explore paths that not every like no human had explored before like the famous sort of alpha go
探索那些之前没有人探索过的路径，就像著名的 AlphaGo 那样
28:54
move of it just coming up with like a particular move that like you know people didn't come up with
move 这块，就是能想出某个特定的 move，那种你知道以前人们没想出来的
29:00
like that before now again I'm never that surprised that we can solve games because you can
那种以前没有过的；不过再说，我从来都不会特别惊讶我们能解决游戏，因为你可以
29:05
simulate them and basically generate infinite amounts of training data or endless amounts of
模拟它们，基本上就能生成无限量的 training data，或者无穷无尽的
29:09
training data for choreography algorithms and so of course eventually it makes sense that we
用于 choreography algorithms 的 training data，所以当然，最终我们
29:13
could solve them with neural nets that are large enough and so but yeah I think we will see
能用足够大的 neural nets 解决它们，这也就说得通了；不过是啊，我觉得我们会看到
29:20
that play out more and more and we're seeing it right now in math I think math will never be the same.
这种情况越来越多地出现，而且我们现在在数学里就看到了，我觉得数学再也不会和以前一样了。
29:24
You've introduced recursive a bit in our conversation thus far but we have it like formally dug into
到目前为止，你在我们的对话里稍微提到过 recursive，但我们还没有真正正式深入聊
29:32
what you're trying to do and you know where you are today with it you know talk a little bit about
你到底想做什么，以及你现在在这件事上进展到哪了，你知道，稍微聊一聊
29:37
guys publish a blog post on kind of your first steps towards automating AI research I guess the
你们发了一篇博客，讲的是你们朝自动化 AI research 迈出的第一步，我猜
29:44
first thing is that like you've got this broad vision of you know accelerating science but your
首先呢，就是你有一个很宏大的愿景，你知道的，加速科学，但你的
29:49
aperture for recursive is AI research in particular it's where we want to start I mean we feel like
切入点，尤其是 recursive 的 AI research，这就是我们想开始的地方，我是说我们觉得
29:55
it's it's helpful to start there first get the item be really really good and then eventually apply
先从那儿开始是有帮助的，先把这个东西做得非常非常好，然后再最终把它应用到
30:00
it to all kinds of innovations. What are some of the the early things you're seeing in some of your
各种创新上。那你们在自己的一些
30:06
early experiments within the AI domain? Yeah so one thing that we applied it to and we published
AI 领域的早期实验里，看到的一些早期的东西是什么？嗯，所以我们把它应用到了一个事情上，而且我们发了
30:15
a blog post about it's this thing called nano chat where you basically do automatically do
一篇博客，讲的就是这个叫 nano chat 的东西，基本上你就是自动去做
30:20
research automate research people call it auto research basically. This is the Andre Carpathi's auto
research，自动化 research，人们基本上就把它叫 auto research。这就是 Andre Carpathi 的 auto
30:29
research that's right that's right yeah so that that is like a like a baby version of of what
研究，对，对，是的，所以那那就像是，一个，一个婴儿版，一个婴儿版的，关于
30:38
what this kind of UICAN machine can do and so we applied it to that and two days later like it's
这个 UICAN machine 能做什么，然后我们就把它用到那上面，两天后，就像它
30:44
sort of had outperformed by far everything else that everyone else had kind of collaborated on and
算是远远超过了其他所有东西，其他人算是合作过、并且
30:50
worked on in this space and what's the what's the structure of that benchmark like what's the
在这个领域里做过的东西，然后这个 benchmark 的结构是，那个，是什么样的，那个
30:58
the idea that yeah the task is you have a model and you try to get to the lowest bits
想法是，对，任务就是，你有一个 model，然后你试着达到最低的 bits
31:03
provide within five minutes on like a particular piece like a hardware or GPU and so you know that
provide，在五分钟之内，在比如某个特定的部件上，比如一个 hardware 或者 GPU，所以你知道那
31:10
has the advantage of like well now it's just very quick you can have this like fast interloop so
好处就是，嗯，现在它就是很快，你可以有这种，快速的 interloop，所以
31:15
that makes everything a whole lot easier and so we basically got the lowest bits provide compared
这让一切都容易多了，所以我们基本上拿到了最低的 bits provide，相比
31:22
to anyone else like by a significant margin and then we say well it's nice if we could
比起其他任何人，大概都大幅领先，然后我们说，嗯，要是还能更快就好了
31:30
also be faster so there's another one called nano GPT speedrun it's somewhat similar but
也能更快，所以还有另一个叫 nano GPT speedrun 的，它有点类似，但
31:38
it's harder to beat the state of the art because just a large community has been optimizing
它更难超越 state of the art，因为已经有一大群社区一直在优化
31:42
so you're supposed for it for almost two years now so instead of asking just like how low can
所以这个到现在差不多已经做了两年了，所以与其只是问，能低到多少
31:47
you go within a fixed time budget you say I will have this fixed validation loss
在固定时间预算内你能做到多低，不如说，我要达到这个固定的 validation loss
31:53
and I want you to get there as fast as possible and so there again people have been working on it
而且我要你尽可能快地达到那里，所以同样地，人们一直在做这个
31:58
for two years and after like couple days we outperformed what everyone in the world had worked on
已经做了两年，然后大概几天之后，我们就超过了全世界所有人共同做出来的成果
32:05
together with their agents on their auto research and their own ingenuity and then we said okay
连同他们的 agents、他们的 auto research，以及他们自己的聪明才智，然后我们说，好吧
32:10
these are all cool but let's like get closer to you know things that really matter for the world
这些都挺酷的，但咱们还是更贴近一点，你知道，那些对世界真正重要的事情
32:15
so we applied it to sole exec bench which is an Nvidia benchmark that was supposed to be unhackable
所以我们把它应用到了 sole exec bench 上，这是 Nvidia 的一个 benchmark，本来应该是没法被 hack 的
32:22
and you couldn't reward engineer we found some ways to still do that but again within a couple
而且你没法 reward engineer，我们还是找到了一些办法照样做到，但同样是在几天之内
32:28
days like we got to have the best kernels across multiple different examples better than cursor
比如我们在多个不同例子上都拿到了最好的 kernels，几乎在所有 kernels 上都比 cursor、doubly iron 和世界上其他所有人都强
32:39
doubly iron and everyone else in the world on almost all the kernels there maybe like half a
那里 200 多个 kernels 里，大概只有半打左右我们没有最好的版本
32:45
dozen of the 200 odd kernels that we didn't have the best version of and yeah that was really
是啊，那真的真的非常令人兴奋，而这只是你知道的，inference 系统里你可以用我们的技术去优化的一部分
32:52
really exciting and that's just like you know one part of the inference sort of system that you
你会怎么描述这个技术最核心的东西？
32:59
can optimize with with our technology how do you describe what the the technology is at its core
可以用我们的技术来优化，你怎么描述这个技术的核心到底是什么
33:06
I don't want to give away too much but you know one way to describe it is like it's like as if you
我不想透露太多，但你知道，一种描述方式就是，就好像你有 20,000 个 AI researchers 在替你搞 AI research，并且在很短的时间内优化你关心的任何东西。
33:12
had 20,000 AI researchers doing AI research for you and optimizing whatever you care about in a very
近似地说就是，对，它有点像是一个 AI scientists 社区。
33:21
short amount of time the approximation is like yeah it is like a somewhat of a community of AI
其中一部分就是，你知道，你关心的这个东西，以及某种把它转化成 objective 的方式，这个 objective 可以被一堆 agents 共享，你的东西会启动这些 agents，然后以某种方式把工作 parse 出来，还有你知道评估这些工作等等。
33:26
scientists essentially one part of that is you know this thing you care about and some way of
到这个时候，你对约束有没有感觉，比如 recursive 会擅长什么、不擅长什么？
33:32
translating that into some objective that can be shared by a bunch of agents that your thing
我是说，有一点很清楚，就是像这样——
33:40
spins up and somehow parses work out too and you know evaluates that work etc do you have a sense
启动起来，然后不知怎么地也能把工作解析出来，你知道，还能评估那些工作等等，你有这种感觉吗
33:48
at this point for like what the constraints are in the sense of like what recursive is going
现在这个阶段，就约束条件而言，递归到底在往哪个方向走
33:55
to be good at versus what it's not going to be good at I mean one thing is clear that like this
擅长什么、不擅长什么，我的意思是有一点很清楚，就是这种
34:01
this kind of approach the faster you can verify something the easier it is for everyone
这种方法就是，验证一件事的速度越快，对所有人来说就越容易
34:08
like and you've seen that in the simpler examples of NanoChat already like a lot of people made
就像你在 NanoChat 那些更简单的例子里已经看到的，很多人已经
34:14
some progress and you know we can make even more progress I think the longer it takes to verify
取得了一些进展，而且你知道我们还能取得更多进展，我觉得验证
34:21
whether result is good the more clever you have to be and the less I can talk about how we
一个结果好不好所花的时间越长，你就得越聪明，我也就越不能讲我们
34:27
how we do it and that's why we haven't like published all our results yet but like that's kind
是怎么做的，这也是为什么我们还没把所有结果都发布出来，但就像，那些
34:33
of those results are easier to verify quickly and then we're going to release some things about a
结果比较容易快速验证，然后我们准备在大概
34:41
month from now that we'll see like okay that was definitely not like you know a five-minute
一个月后发布一些东西，到时候我们会看到，就是，好吧，那绝对不是什么，你知道，五分钟
34:47
type of situation to get the result for and even there it starts to make progress you know what
就能拿到结果的那种情况，而且即便在那儿也开始有进展了，你懂吧
34:52
I heard is verifiability I also heard kind of traditional like attribution problems and
我听到的是 verifiability，我也听到一些传统的，像是 attribution 问题之类的东西；你是说这两者都有，还是说更多是 attribution 这一类的问题？
35:01
that kind of thing which are you saying both of those things or are you saying more of the
一方面，你知道，有 verifiable rewards 这个思路，比如你知道，这是一个 agent 能清楚、轻松地判断自己做对还是做错的 task，然后你就能把这件事推进起来，你知道。
35:06
attribution type of an issue on the one hand you have like you know the idea verifiable rewards like
另一方面，你知道，task 涉及很长的 sequences，agent 可以在中间做决策，但直到最后才真正知道这些决策是对还是错，你知道。
35:13
you know this is a task that you know the agent can clearly and easily distinguish whether it
是不是有点像以上所有这些，你知道，这就是挑战所在，而你心里已经有某种能应对这些挑战的想法？
35:18
got it right or wrong and then you can kind of get that going you know the other side you've got
明白了，对了还是错了，然后你就能把那个推进起来，你知道另一边你还有
35:24
you know the task involves long sequences of things and agent can make decisions in the middle
你知道这个任务涉及很长的序列，而且agent可以在中间做决策
35:28
and not really know whether those decisions are right or wrong until the end you know is it kind
而不是真正知道那些决定是对是错，直到最后你才知道，是不是那种
35:33
of all of the above is you know where the challenges are and you've got something in mind that addresses
所有上述情况，你知道挑战在哪里，而你心里有一些能解决
35:38
all of that or is it one or the other of those that you've kind of figured out it's it's all of the
是所有这些，还是你只是搞清楚了其中某一个？其实是——是所有这些
35:43
above yeah like all of these need to be all of these important even if you can have a verifier but
上面这些，对，就像这些全都需要，所有这些都很重要，哪怕你有一个 verifier，但
35:48
to verify it takes like a year to run not that useful right like or and then and then there's like an
要 verify 它，run 起来得花上一年，那就不太有用了，对吧，或者说，然后还有
35:56
interpolation of how useful it is right if you get it down to like a minute extremely useful
一个 interpolation：它到底有多有用，对吧，如果你能把它压到大概一分钟，那就极其有用
36:01
and if it takes a year and it's almost useless completely and so then you need to be clever
可如果它要花一年，那就几乎完全没用了，所以你得聪明点
36:06
about dealing with that part if if you yeah so that's that's an aspect now some cases you have
去处理那一部分，如果你——对，所以那是其中一个方面。现在有些情况下你有
36:14
and indeed multiple steps and then you have just sort of yeah the attribution of each step and how
而且确实是多个 steps，然后你就得，对，每个 step 的 attribution，以及它
36:20
important it was and how do you deal with that there's just a lot of a lot of different pieces that
有多重要，你要怎么处理这些，这里面就是有很多很多不同的 pieces
36:24
need to be figured out and in some cases you know there are research papers that we've published
这些还需要搞清楚，而且在有些情况下，你知道，我们已经发表了一些研究论文
36:31
four of the five authors of the Darwin-Gurl machine paper for instance from Jeff Blue and
比如 Darwin-Gurl machine paper 的五位作者里有四位来自 Jeff Blue，而且
36:36
is the most senior author on that paper and Jenny is the first author they're both with us and
他是那篇 paper 的资深作者，Jenny 是第一作者，他们俩都和我们在一起，而且
36:40
like you know that is an example of what you can do also that yeah is a little bit more complex
就像你知道的，那是一个例子，说明你能做到什么；而且，是的，那个稍微更复杂一点
36:50
but already shows you kind of the sign of things to come. How do you think about kind of the
但已经多少让你看到未来发展的征兆。你怎么看，关于
36:56
generalized ability of what you have today you know meaning even for these the three examples
你今天所具备的 generalized ability，你知道，我的意思是，甚至对于这三个例子来说
37:03
you talked about like are they three largely disparate efforts or was it mostly the same
你谈到的，比如，它们是三个大体上各自不同的努力，还是说主要就是同一种
37:11
kind of effort applied lightly applied to three different areas does that question make sense?
努力，只是轻量地应用到三个不同领域？这个问题说得通吗？
37:16
Yeah yeah I mean there's like you know sometimes it's useful to have an extra tool for instance
对对，我是说，你知道，有时候多一个工具确实挺有用的，比如说
37:20
in kernels you can have like a kernel profiler and that's a very useful tool to just understand
在 kernels 里，你可以弄个 kernel profiler，这工具特别有用，能帮你多了解
37:27
a little bit more inside this sort of black box what's happening and what's going on but outside
这种黑箱里面到底在发生什么、是怎么回事，但除了
37:31
of that it's basically all just like different tool use and it's the same same underlying Eureka
这些之外，基本上就都是不同的 tool use，而且背后是同一个、同一个 Eureka
37:37
machine which is kind of cool. That's what gives us hope that and a few other things I can't talk
machine，还挺酷的。这一点给了我们希望，还有一些我暂时不能
37:42
about yet and results that we're seeing that like gives us hope that like there's something here
说的东西，以及我们看到的结果，也让我们觉得，这里确实有点东西
37:47
and it's time to scale it up more. Yeah beyond the things that you can't talk about like what's
而且现在是时候进一步 scale up 了。对，除了那些你不能说的事情，那接下来
37:52
the next what are how do you think about in terms that you can talk about the big challenges for
呢，在你能聊的范围内，你怎么看最大的挑战是什么？
38:00
you and the team going forward. If you wanted to make more and more exciting bigger and mentions
你和团队接下来往前走。如果你想做出越来越多让人兴奋、更大的东西，还有更多提及
38:05
you need more compute for it so that's a challenge. I think there is a weird time where and again
你就需要更多 compute，所以那是个挑战。我觉得有个很奇怪的时候，而且又一次
38:14
it's feels impossible now the way it felt impossible to any expert to have in one model you can have
现在感觉不可能，就像当年对任何专家来说都不可能一样：在一个 model 里，你可以有
38:20
a conversation with about any topic you asked all the NLT people 2010 like oh yeah we're gonna
一场关于任何话题的对话。你在 2010 年问所有 NLT people，他们会说，哦对，我们会
38:26
have this one neural net and you just like it passes the Turing test so hard that the Turing test
有这一个 neural net，然后你就觉得它把 Turing test 通过得太彻底了，以至于 Turing test
38:32
will flip to asking questions that no human could answer and then it can still answer and you're
会反过来问一些没人能回答的问题，然后它还是能回答，而你会
38:36
like oh yeah like write me this like complex script and like five seconds later you have like
说，哦对，比如给我写这个复杂的 script，然后大概五秒后你就有了
38:41
thousands of lines of code you're like that's why it's not a human because it's too good I like people
几千行 code，你会想，这就是为什么它不是人，因为它太强了。我喜欢人
38:45
would have never assumed the Turing test would flip like that and so invert itself and so I think
我本来绝不会想到 Turing test 会那样翻转，甚至反过来，所以我觉得
38:51
the same way the same thing is true for right now biology just like it feels so complex it feels
同样地，现在生物学也是这样，感觉它太复杂了，感觉
38:57
like there's no way we can do it but it is to a large degree a data problem and so I think
好像我们根本做不到，但它在很大程度上是个 data 问题，所以我觉得
39:04
companies like parallel bio and tower therapeutics and others like can I think help with that and then
像 parallel bio、tower therapeutics 这样的公司以及其他类似的公司，我觉得可以帮上忙，然后
39:11
we can have these bitter lessons learned in other fields and so I think that's a big part just data
我们可以吸取其他领域的这些惨痛教训，所以我觉得很大一部分就是 data
39:17
and in gear right environments to to apply this to I think that's also the right rewards I think
以及在 gear 合适的 environments 里把它应用起来，我觉得那也是合适的 rewards，我觉得
39:24
those are really important to get right I think another one that we have I think a better handle
这些真的必须做对，我觉得另一个我们有的点——我觉得——比大多数人
39:29
on than most but like we can also still improve it further is reward hacking you know they I will
更有把握，但我们也还能进一步改进，那就是 reward hacking，你知道他们，我会
39:36
try lots of different ways to get to rewards and went to that customer satisfaction example I gave
尝试很多不同的方式去拿到 rewards，然后就是我举的那个 customer satisfaction 的例子
39:42
but you know if you apply it to bad things then it will find you know worse kind of reward hacks
但你知道，如果你把它用到坏的事情上，那它就会找到更糟的 reward hacks
39:49
and I think we need to be as a like like industry we need to be more careful the the broader
而且我觉得，作为一个行业，我们需要更小心：DI 越广泛、越容易获得，就越要仔细想清楚我们想给它什么样的 rewards 和目标
39:58
and more accessible DI is and and think about very carefully what kinds of rewards and goals we
我觉得这些都是我们会继续去做的方面
40:05
want to give it I think those are all like aspects that we're going to continue to work on
recursive focusing 有多大程度是在这类 alignment 导向的问题上投入大量时间和精力？我的意思是，从某些方面说是很多，因为你知道，而且你必须这样——你给一个 model 投入越多钱，你就越有动力让它做你真正想让它做的事，而不是别的事
40:10
to what degree is recursive focusing a lot of time and effort on that kind of alignment oriented
递归聚焦在多大程度上是大量时间和精力投入到那种对齐导向的
40:17
problem I mean in some ways a lot because you know and you have to if you do more money you give a
问题上，我的意思是某些方面很多，因为你知道而且你必须这样做，如果你投入更多钱给一个
40:24
model the more incentive you have to have it do what you actually want it to do and not something else
模型，你就越有动力让它做你真正想让它做的事，而不是别的什么
40:32
and like fake it and and and achieve the goal but not in the way you had you know actually desired
然后就像糊弄过去一样，然后，然后，然后达成目标，但不是以你原本——你知道——真正想要的方式
40:39
and so I think in a weird way capitalism can have its problems but like it is a pretty good system
所以我觉得，说来也怪，资本主义是有它的问题，但怎么说呢，它是个相当不错的系统
40:46
for for most humans and it is a good system to align companies to basically build something that
对大多数人来说都是，而且它是一个很好的系统，能让公司基本上对齐去构建某种东西，而这种东西
40:54
is aligned with their goals and you know if you built an AI for billions of dollars and then you
是和他们的目标一致的，而且你知道，如果你花数十亿美元建了一个 AI，然后你
41:02
ask it to add three emails and it's like now emails are boring I'd rather explore the molecular
让它加三封邮件，它就像在说，现在邮件太无聊了，我宁愿去研究 Venus 上大气层的分子
41:07
composition of the atmosphere on Venus you know then you're going to turn it off and you're going
组成，你知道，然后你就会把它关掉，然后你会
41:12
to be like well we have to restart and then build it differently right so I think aligning the
想，好吧，我们得重新启动，然后用不同的方式构建它，对吧？所以我觉得，让
41:17
AI to do the things you actually want to do will become a bigger and bigger problem as these models
AI 去做你真正想让它做的事，会变成一个越来越大的问题，随着这些 models
41:22
are very expensive to do not just train but also like run inference on and and so I think we're
成本非常高，不只是 train，跑 inference 也很贵，然后然后我觉得我们
41:27
going to see more like more effort from the industry because the AI is getting smarter and smarter
会看到行业付出更多、更多努力，因为 AI 越来越聪明
41:36
to find more and more clever reward hacks that people didn't think about but it's a problem that
去找到越来越多、人们没想到的更巧妙的 reward hacks，但这是一个
41:42
we create it and it's a problem that we I think can solve and where does that leave you on the
我们制造出来的问题，而且我觉得这是一个我们能解决的问题，那这让你在
41:48
the whole question of slowing down versus speeding up versus you know kind of status quo yeah I mean
整个放缓还是加速、还是你知道的某种现状的问题上，是什么立场？对，我是说
41:57
it's interesting to hear about the ask to slow down from the companies that have sped up the
听到那些加速得
42:04
most and really pushed the whole field and like the whole like capability set forward so far and so
最猛的公司提出要慢下来，这挺有意思的，而且它们真的把整个领域、整个 capability set 推进到了现在这个程度，甚至
42:13
much so that they built the best hacking AI to then show how like like dangerous hacking AI
到了这种地步：它们造出了最好的 hacking AI，然后用来展示危险的 hacking AI 有多危险
42:19
can be well they sped that up by at least several quarters if not years from everyone and China
可能吧，他们至少把这事提前了好几个季度，甚至好几年——对所有人、对 China 来说都是这样
42:25
distilled they are capabilities now everyone has better hacking capabilities and that's like
distilled 之后，它们就是能力了；现在每个人的 hacking 能力都更强了，而且这就像
42:29
thanks to anthropic right and so I do think we want to be careful and if your company does
这得感谢 anthropic，对吧；所以我确实觉得我们得小心，而如果你的公司
42:37
sort of accidentally compete on felony bench then maybe that company should slow down you know
算是无意中在 felony bench 上竞争，那也许那家公司就该慢下来，你知道
42:43
but I don't see open source slowing down I don't see China slowing down and I don't see any of the
但我不觉得 open source 会慢下来，我不觉得 China 会慢下来，我也不觉得任何
42:49
ones that want to get to that frontier slowing down and then of course I understand why the ones
想达到那个 frontier 的人会慢下来；然后当然，我理解为什么那些
42:55
who are at the frontier want everyone else including themselves to slow down like if you're here
已经在 frontier 的人希望其他所有人，包括他们自己，都慢下来，就像如果你在这里
42:59
and everyone else you feel like it's here and everyone now moves slower then it just takes them
而其他所有人，你会觉得它就在这儿；现在大家都慢下来了，那他们就只是要花
43:03
longer to catch up to you right and so they're trying to in some cases the stoked the fear for
更久才能追上你，对吧，所以他们在某些情况下就试图煽动那种恐惧。
43:09
that reason I think there are also some people really truly believe in the apocalypse and so on
出于这个原因，我觉得也有一些人真的相信世界末日之类的。
43:15
but yeah I think you know slow down and pacing and so on is also somewhat unrealistic
但是啊，我觉得你知道，放慢速度、控制节奏之类的也有点不现实。
43:24
like in some ways if you truly want to know what everyone is doing with their GPU to really get there
就像在某些方面，如果你真的想知道每个人都在用他们的 GPU 做什么，真的想达到那个目标，
43:34
you have to have a totalitarian survey in state the likes of which you've never seen before that
你就得有一种你从没见过的极权式监控国家，
43:40
basically built a thought police and tries to regulate intelligence and thoughts and what you
基本上就是建立思想警察，试图在你还没做任何事之前，就监管智能、想法以及你可能想用它做什么，对吧？所以我觉得，
43:49
might want to do with it before you've even done anything right and so I think it makes sense to
监管 AI 应用是合理的，但监管抽象的智能就不合理，就像我们……
43:53
regulate AI applications it doesn't make sense to regulate abstract intelligence just like we
监管AI应用，监管抽象智能没有意义，就像我们
44:01
don't put the smartest people in prison just because if they really apply their intelligence to
别因为最聪明的人要是真把自己的智慧用在什么邪恶坏事上，你知道，就会更成功，就把他们关进监狱；等他们真的做了坏事，你再这么做，然后你说那些事违法、你知道，该被监管。
44:06
something nefarious and bad there'd be more like you know successful at it you just do it when
所以是啊，我支持 self-driving，我支持 medicine 和 drug 这类开发，还有所有这些各种各样的真实应用，监管它们是有道理的。
44:11
they actually do something bad and you say those things are illegal and you know should be
Eureka machine 有没有稍微提到 AI 在科学领域内可能造成的危害？
44:16
regulated and so yeah I for self-driving I for medicine and drug like development and like all
但我感觉有很多人非常有创造力，而从某些方面来说，想出反乌托邦场景太容易了——每个人都觉得自己特别聪明，但其实想出反乌托邦场景非常容易，这...
44:24
these various real applications it makes sense to to regulate those Eureka machine does it touch
这些各种真实应用，监管那些是有意义的，Eureka机器是否涉及
44:30
on potential harms from AI within the scientific domain a little bit but I feel like there's a lot
AI在科学领域内的潜在危害，有一点，但我觉得有很多
44:38
of people who are very creative and it's so easy in some ways to come up with dystopian scenarios
对于那些非常有创造力的人来说，在某些方面想出反乌托邦场景实在是太容易了
44:44
everyone feels really smart but it's actually very easy to come up with dystopian scenarios it's
每个人都觉得自己很聪明，但实际上想出反乌托邦场景非常容易，就是
44:48
very entertaining too and many like part of why the human species survived it all is that we
也特别有意思，而且很多人觉得，人类这个物种能幸存下来，部分原因就是我们
44:53
think about worst-case scenarios and people have anxiety over that and so it's very natural
会去想 worst-case scenarios，人们会为此焦虑，所以这非常自然
44:58
I think it's much harder to come up with like okay if we have this like really good ability to
我觉得难得多的是，去想比如说，好，如果我们有这种特别强的能力去
45:03
attack someone what's the next most creative thing that you will come up with to defend against
攻击别人，那你接下来能想出的最有创造力的办法是什么，来防御
45:08
yourself against that and so when it comes to science I think of course science has
自己免受这种攻击，所以说到科学，我觉得当然科学有
45:15
like AI sort of often multi-use on the used dual use types of scenarios like we understood the
就像 AI 经常是多用途的，或者说属于 dual use 类型的情景，比如我们更理解
45:21
atom better that led to like clean nuclear energy it also led to nuclear weapons right and so
原子了，这带来了清洁核能，也带来了核武器，对吧，所以
45:28
when a weapon is so destructive that it could end like a huge part of humanity then you regulate
当一种武器破坏性大到可能终结很大一部分人类时，那你就要监管
45:34
it and so like we did with nuclear weapons I think intelligence is not just a weapon intelligence
它，而且就像我们对待核武器那样，我觉得智能不只是武器，智能
45:43
can also write a poem don't it regulate all of that the way you'd regulate the printing press
也能写诗，不是吗？你要像监管印刷机那样去监管所有这些
45:48
and so there's a lot of complexity and of course there's more agency in AI and so you have to kind
所以这里有很多复杂性，当然 AI 里有更多 agency，所以你得
45:52
of think multiple steps ahead of like who asked the AI to do something and how responsible are they
提前多想几步，比如是谁让 AI 去做某件事的，他们该负多大责任
45:58
I think there are a lot of these problems would go away if you just made people navigating and
我觉得很多这类问题都会消失，如果你只是让那些在导航和
46:03
using the AI and asking it and giving it their rewards to be responsible for the actions of the
使用 AI、向它提问并给它 rewards 的人，对其行为负责
46:08
lot of those problems would then go away and people would be much more careful about what they're
很多这类问题就会消失，人们也会更加小心自己在做什么
46:13
doing and so long story short we touch upon it in the book and you know I think especially
所以长话短说，我们在书里谈到了这一点，而且你知道，我觉得尤其是
46:22
the viral vector could be suboptimal and that's why gain of function research is already illegal
viral vector 可能不够理想，所以 gain of function research 本来就已经是非法的了
46:29
you know like you don't have to create new laws around that it's already not allowed and so
你知道的，就是围绕这个你不需要再制定新法律，它本来就已经不被允许了，所以
46:36
yeah I'm focused much more on the book on the positive applications in general you think about
对，我更关注这本书里讲的正向应用，总体上，你想想
46:42
science too a lot of people think about AI as automating jobs I think few people say we want
科学也是一样。很多人把 AI 看成是在自动化工作，我觉得很少有人会说我们想要
46:49
more science jobs we want more medicine jobs we want more research jobs a lot of people very much
更多科学岗位，我们想要更多医疗岗位，我们想要更多研究岗位——很多人非常
46:56
everyone is excited when science comes up with a new capability a new device a new material a new
每当科学带来新的能力、新的设备、新的材料、新的
47:04
drug a new cure for disease and when people can get out the hospital faster with less cost right
药物、新的疾病疗法，还有当人们能更快出院、花费更少的时候，对吧
47:10
so that's kind of I think where where we could focus on a lot more and where I think we should
所以这大概就是我觉得我们可以更多聚焦的地方，也是我认为我们应该聚焦的地方
47:17
like really outline like there's a lot of suffering right now in the world I think AI can actually
就像，真的，大致说一下，现在世界上有很多苦难，我觉得 AI 其实可以
47:22
help in improving that and you know I think when you look at GMOs and nuclear energy and so on
帮助改善这一点；你知道，我觉得当你看看 GMOs 和 nuclear energy 之类的
47:29
that led a lot of people to end up using like the fears around that technology end up a lot more
这导致很多人最终反而去使用，就像围绕那项技术的恐惧最终让更多
47:35
coal being used and coal kills about 50,000 people a year from lung cancer it's all the fuse and through
煤炭被使用，而煤炭每年导致大约 5 万人死于肺癌，全是烟雾，还通过
47:41
the air and yet they're not like the lung cancer isn't just right next to the factory but lung cancer
空气传播；可他们又不是说……肺癌并不只是在工厂旁边，但肺癌
47:46
kills lots of people and a lot of that can be chased back to coal nuclear has not killed nearly as
杀死很多人，而其中很大一部分可以追溯到煤炭；nuclear 造成的死亡人数远没有
47:50
many people but it has this like crazy fear industry around it and you know I think AI I hope
那么多，但它周围却有这种疯狂的恐惧产业；你知道，我觉得 AI，我希望
47:57
end up in that same same place well we're sure it's been great catching up with you always a pleasure
最终也落到同样同样的境地。嗯，我们确定——很高兴和你聊近况，一如既往地愉快