Peter H. Diamandis · #216
The AGI Race Is Fake, Building Safe Superintelligence & the Agentic Economy
AGI竞赛是假的、构建安全超级智能与Agent经济
Microsoft AI CEO Mustafa Suleyman goes deep with Peter Diamandis on why the AGI race is a false narrative, AI containment strategies, the agentic economy vision, and how Microsoft is reshaping productivity.
Microsoft AI CEO Mustafa Suleyman 与 Peter Diamandis 深度对话，提出 AGI 竞赛是虚假叙事的观点，阐述 AI 遏制策略与 Agentic Economy 愿景，以及微软如何在 AI 时代重塑生产力。
00:00
00:00
What's the mandate from Satya is it win AGI?
萨提亚的指令是什么？是要赢得通用人工智能吗？
00:03
I don't think there's really a winning of AGI.
我不认为通用人工智能真的能被“赢得”。
00:05
I'm not sure there's a race.
我也不确定是否存在一场竞赛。
00:07
One of the OGs of the AI world,
作为人工智能领域的元老之一，
00:10
Mustafa Tellyman is the CEO now of Microsoft AI.
穆斯塔法·泰利曼如今是微软AI的首席执行官。
00:13
He spent more than decade at the forefront of this industry
他在这个行业前沿深耕了十多年，
00:16
before we even had gotten to feel it
远早于我们过去几年才感受到它的存在。
00:20
in the past couple of years now.
在过去的几年里。
00:23
Fundamentally, the transition that we're making
从根本上说，我们正在经历的转变
00:26
is from a world of operating systems,
是从一个由操作系统、搜索引擎、应用和浏览器构成的世界
00:29
search engines, apps, and browsers
转向一个由智能体和伙伴构成的世界。
00:33
to a world of agents and companions.
我们都在竭尽全力加速前进，
00:35
We're all going as fast as we possibly can,
但“竞赛”一词暗示着零和博弈，
00:37
but a race implies it's zero sum,
暗示着存在一条终点线——
00:40
it implies that there's a finish line
这其实并不完全是一个恰当的比喻。
00:42
and it's just like not quite the right metaphor.
但这比喻不太准确。
00:44
As we know, technologies and science and knowledge proliferate
众所周知，技术与科学知识无处不在，
00:48
everywhere, all at runs at all scales,
在各个层面同时飞速发展。
00:50
basically simultaneously.
你是否投入了大量精力，
00:52
Are you spending a lot of your energy,
以及人力计算资源在安全问题上？
00:54
compute human power on safety?
是的，不，我的意思是……
00:57
Yeah, no, I mean...
女士们先生们，这就是“登月计划”。
00:59
Now that's the moonshot, ladies and gentlemen.
欢迎大家来到《登月计划》。
01:05
Everybody, welcome to moonshots.
各位，欢迎来到“登月计划”。
01:06
I'm here with DB2 and AWG and Mustafa Tellyman,
我与DB2、AWG和Mustafa Tellyman一同在此，
01:11
the co-founder of DeepMind, Inflection AI,
他是DeepMind、Inflection AI的联合创始人，
01:14
and now the CEO of Microsoft AI.
如今担任微软AI的CEO。
01:18
Welcome, my friend.
欢迎你，我的朋友。
01:19
Good to have you here.
很高兴你能来。
01:20
Thank you for making time for us.
感谢你抽出时间与我们交流。
01:21
Thanks for having me.
谢谢你们的邀请。
01:22
Yeah, I'm excited to do this.
是的，我很期待这次对话。
01:23
Yeah, it's, you know, what you've been building
是啊，你知道的，你和萨提亚共同打造的一切，真的很了不起。
01:27
with Satya is amazing.
很难相信微软已经50岁了。
01:30
And it's hard to believe that Microsoft is 50 years old.
它经历了那么多次自我革新。
01:34
And it's reinvented itself so many times.
而过去五年里，
01:36
And for the last five years,
它一直处于行业巅峰，
01:37
it's been, you know, at the top of the game,
是全球市值最高的公司，
01:40
the most valuable company in the world,
拥有25万名员工。
01:42
250,000 employees.
25万名员工。
01:45
And for what I understand, 10,000 employees now under you.
据我所知，现在有一万名员工归你领导。
01:49
So a few, you know, important questions I want to open with.
所以，我想先问几个重要的问题。
01:53
First, some broad context.
首先，谈一下大背景。
01:57
You're building inside a massive company
你是在一家资源雄厚的大公司内部进行建设，
02:00
with huge resources,
其资源可能远超其他大多数公司。
02:01
probably arguably more than almost everybody else.
我想问的是，
02:04
And the question I have is,
最终目标是什么？
02:07
what's the end goal here?
最终目标是什么？
02:10
You've got all the hyperscalers
所有超大规模云服务商
02:12
sort of providing open access to AI.
都在某种程度上提供开放的人工智能接入。
02:14
And they're doing a sort of a land grab
他们正在进行某种“圈地运动”，
02:17
trying to get as many users as possible.
试图争取尽可能多的用户。
02:20
You've been building sort of in a, you know,
你一直在微软365生态系统中
02:23
within the Microsoft 365 ecosystem
进行某种程度的建设，
02:27
is the goal in the, you know,
目标是在未来几年内
02:30
next couple of years, maximum users.
实现用户数量最大化。
02:33
Is it data centers?
是数据中心吗？
02:35
Is it, you know, is it cloud?
是，你知道的，是云吗？
02:38
How do you think of what you're optimizing for?
你如何看待自己正在优化的目标？
02:41
I mean, it's a good question.
我的意思是，这是个好问题。
02:43
So I mean, we are on any given day
所以，我的意思是，在任何一个普通的日子里，
02:44
of $4 trillion company with almost $300 billion of revenue.
一家市值4万亿美元、营收近3000亿美元的公司，
02:48
It's incredible.
这太不可思议了。
02:49
It's just surreal and very, very, very humbling.
这简直超现实，而且非常、非常、非常令人谦卑。
02:54
And we play at every layer of the stack.
我们在每一层技术栈上都有布局。
02:56
I mean, obviously we're an enormous business in data centers
显然，我们在数据中心领域是一家规模庞大的企业，
02:59
and in some ways we're like a modern construction company.
某种程度上就像一家现代化的建筑公司。
03:03
Hundreds of thousands of construction workers
数十万名建筑工人
03:05
building gigawatts a year of, you know,
每年建设着千兆瓦级别的
03:08
CPU and AI accelerators of all kinds
各类CPU和AI加速器，
03:11
and enabling that, you know, to be available to the market.
并确保这些资源能够面向市场开放。
03:15
APIs on top of that, but also first party products
在此基础上提供API，同时也推出自有产品。
03:19
in every domain you can think of
在你所能想到的每一个领域，
03:21
from gaming and LinkedIn right the way through
从游戏到领英，一路贯穿，
03:23
to all the fundamentals of M365 and Windows.
再到M365和Windows的所有基础功能。
03:28
And of course in our search and consumer businesses and two.
当然，还有我们的搜索、消费者业务以及另外两项。
03:31
And fundamentally the transition that we're making
而从根本上说，我们正在经历的转变
03:35
is from a world of operating systems,
是从一个由操作系统、
03:38
search engines, apps and browsers
搜索引擎、应用和浏览器构成的世界，
03:42
to a world of agents and companions.
迈向一个由智能体和伙伴构成的世界。
03:45
All of these user interfaces are gonna get subsumed
所有这些用户界面都将被整合进一种对话式、智能代理的形式。
03:49
into a conversational, agentic form.
这些模型会让人感觉像口袋里随时有一位真正的助手，
03:53
And these models are going to feel like having a real assistant
全天候待命，无所不能，
03:58
in your pocket 24, seven that can do anything
并且掌握你所有的上下文信息。
04:00
that has all your context.
你将越来越少地直接操作计算设备——
04:01
And you're gonna do less and less of the direct computing
正如我们现在所看到的，许多软件工程师
04:04
just as we're seeing now, many software engineers
正在使用辅助编码代理来调试他们的代码。
04:07
are using assistive coding agents to both debug their code
正在使用辅助编码工具来调试他们的代码。
04:11
and also generate large amounts of code
并且生成大量代码，
04:13
just as we used libraries, third party libraries.
就像我们使用库、第三方库那样。
04:16
Now we're just gonna use AI to do that generation.
现在我们将直接用AI来完成这种生成。
04:19
And it's making them more efficient
这让它们变得更高效、
04:21
and more accurate and faster and so on and so forth.
更准确、更快速，诸如此类。
04:25
So the trajectory we're on is quite predictable.
所以我们所处的轨迹是相当可预测的：
04:28
It's one from user interfaces to AI agents.
从用户界面到AI代理。
04:33
And that is the paradigm shift
这就是范式转变。
04:35
which the company is completely focused on.
该公司完全专注于这一点。
04:38
Like after seeing five decades worth of transitions
就像目睹了五十年来的变迁后，
04:42
I think the company is like super alert
我认为公司高度警觉，
04:44
to making sure that we're best placed to manage this one.
以确保我们已做好最充分的准备来应对这一变革。
04:47
Do you see yourself providing sort of an open source AI
您是否考虑像其他参与者那样提供开源AI，
04:52
like the other players out there
还是认为可以将其限制在Microsoft 365内部？
04:53
or do you think you can keep it contained within Microsoft 365?
我认为我们持相当开放的态度。
04:57
I think we're pretty open minded.
我认为我们心态相当开放。
04:59
I mean, we've got some pretty small open source models.
我的意思是，我们有一些相当小的开源模型。
05:03
I think realistic.
我认为这很现实。
05:04
And when you say open source
而当你提到开源时，
05:05
I really mean open access if you would.
我其实更倾向于说开放获取。
05:07
Yeah, I mean, there are always gonna be APIs
是的，我的意思是，总会有一些API
05:10
that provide incredibly powerful models.
提供极其强大的模型。
05:13
I mean, Microsoft is really a platform of platforms.
我的意思是，微软确实是一个平台的平台。
05:16
Being a platform and being a great provider
成为一个平台，并成为一个优秀的提供者。
05:19
of the core infrastructure that enables other people
核心基础设施的一部分，
05:21
to be productive is like the DNA of the company.
让他人能够高效工作，
05:25
And so we will always have masses of APIs
就像公司的DNA一样。
05:29
that turbocharged that.
因此我们将始终拥有大量API，
05:30
But what an API is is gonna start to look kind of different too.
为其提供强大动力。
05:33
Like it may be pretty blurred the distinction
但API的形态也将开始变得不同，
05:37
between the API and the agent itself.
比如API与智能体本身之间的界限，
05:39
Maybe that we're principally in the business
可能会相当模糊。
05:41
and five years time of selling agents
以及五年的销售代理服务期，
05:44
that perform certain tasks
这些代理执行特定任务，
05:46
that come with a certification of reliability,
并附带可靠性、安全性、保障与信任的认证。
05:49
security, safety and trust.
我的意思是，这在很多方面实际上
05:52
I mean, that is actually in many ways
正是微软的优势所在。
05:53
the strength of Microsoft.
而这也是吸引我的原因之一——
05:55
And that's one of the things that attracted me
就像，这是一家这样的公司。
05:57
is like this is a company
就像这样——这是一家公司。
05:59
that's incredibly trusted, it's actually very secure.
这极其值得信赖，实际上非常安全。
06:03
And sometimes I think the slowness or the friction
有时候我觉得这种缓慢或摩擦
06:08
is actually a bit of an asset.
反而是一种优势。
06:10
You know, there's a kind of steadiness
你知道，有一种稳健感
06:12
that comes with having provided
来自于为全球最大的财富500强公司
06:14
for all of the world's biggest Fortune 500 companies
以及政府和主要机构提供服务。
06:19
and governments and major institutions.
是不是就像那句老话：你不可能出错。
06:22
Is it like the old adage you can't go wrong
正如那句老话所说，你不会出错
06:24
buying IBM in the old days?
过去买IBM股票？
06:26
I think there's a steadiness about us
我认为我们有一种稳健的特质，
06:30
which I think is reassuring to people.
这让人感到安心。
06:32
And there's a kind of like deliberate customer-focused
还有一种刻意以客户为中心的耐心，
06:36
patience, you know, there's not the same anxiety
你知道，没有那种焦虑，
06:40
and sort of somewhat sclerotic nature
也没有那种作为挑战者时难免的僵化。
06:43
that comes with being an insurgent.
我们的位置也有其劣势。
06:47
There's some downsides to our position.
我们的处境存在一些不利因素
06:49
We would take a little longer to get things through
我们需要多花一点时间把事情推进，
06:51
but the company is firing on all cylinders.
但公司正在全力运转，
06:53
It's very impressive to see.
看到这一切令人印象深刻。
06:55
One more question for I turn it over to Alex.
在我把问题交给亚历克斯之前，还有一个问题。
06:57
You know, we're seeing in this hyper-scale or war,
你知道，我们正处在这场超大规模竞赛中，
07:00
I mean, literally a week by week
几乎每周都在上演，
07:02
everybody outdoing each other
每个人都在互相超越，
07:04
in this insane period of everybody coming out
在这个疯狂时期，人人都在争先恐后地推出新东西。
07:08
with the new benchmarks.
随着新的基准测试，
07:12
You know, do you miss not being in that game
你知道，你是否怀念没有参与那场游戏的日子，
07:16
or is this stability that Microsoft provides
还是微软提供的这种稳定性，
07:18
to build for a long-term vision
让你能够为长期愿景而构建，
07:20
sort of what you're fine most exciting?
才是你最兴奋的部分？
07:23
You know, my background at DeepMind
你知道，我在DeepMind的背景，
07:26
is such that I spent a good decade grinding through
让我花了整整十年在指数曲线的平缓阶段埋头苦干。
07:30
the flat part of the exponential
指数曲线的平缓部分
07:32
where basically nothing worked.
基本上什么都行不通。
07:34
I mean, you know, really.
我是说，你知道，真的。
07:37
Like there was some amazing papers in AlphaGo
比如，AlphaGo 有一些令人惊叹的论文，
07:40
was obviously incredible
显然非常了不起，
07:41
but it was in a very unique simulated
但那是在一个非常独特的模拟、
07:43
controlled game-like environment.
受控的游戏环境中。
07:46
But things actually working in the real world
而真正能在现实世界中发挥作用的东西，
07:49
were few and far between.
却少之又少。
07:51
And so, you know, I've always taken a multi-decade view
因此，你知道，我一直以数十年为视角来考量，
07:56
and that's just been my instinct.
而这正是我的本能。
07:58
And I think that, you know,
而且我认为，你知道，
08:00
yes, it's super important to ship new models
每个月推出新模型并活跃于市场固然极其重要，
08:02
every month and be out there in the market
但真正更关键的，
08:04
but it's actually more important
是为即将到来的变革奠定正确的基础，
08:05
to lay the right foundation for what's coming
因为我相信这将是史上最剧烈的转型。
08:07
because I think it's going to be the most wild transition
因为我认为这将是最剧烈的转变
08:12
we have ever made as a species.
我们作为一个物种所取得过的成就。
08:14
Can you just flesh that out a little bit?
你能稍微展开讲讲吗？
08:15
Was there a period of time
有没有那么一段时间，
08:16
where it was just three of you grinding it out in London?
在伦敦只有你们三个人在埋头苦干？
08:19
Well, more than three of us.
嗯，不止三个人。
08:20
But I mean, for the decade between 2010 and 2012,
但我的意思是，在2010年到2012年这十年间——
08:24
sorry, 2020.
抱歉，是2020年。
08:25
I mean, there were just like so few successful
那时候成功的案例实在太少了。
08:29
commercial applications of deep learning.
深度学习的商业应用。
08:33
I mean, there were plenty behind the scenes.
我的意思是，幕后其实有很多。
08:35
There was image recognition,
有图像识别，
08:37
there were improvements to search.
有搜索功能的改进。
08:38
A bit commercial market for.
算是一个商业市场吧。
08:40
Commercial, yeah, playing go, not a huge rock, exactly.
商业嘛，对，下围棋，其实算不上多大的市场。
08:43
So I think whereas now, I mean,
所以我觉得，现在呢，
08:44
you then you see LLMs from 2022 onwards,
你从2022年起看到的大语言模型，
08:47
like in production, completely changing the way
如同在生产中，彻底改变了人们与计算机互动的方式，
08:49
that people relate to computers,
改变了我作为人类自身的意义，
08:51
changing what it means to be a human myself,
改变了我们的社会关系。
08:53
changing our social relations.
就像，那只是一个，你知道的，
08:55
Like, that is just a, you know,
我们正处在一个转折点。
08:57
that's we hit an inflection point.
而你知道，我认为这与训练小模型的枯燥工作
08:59
And you know, I think that is very, very different
截然不同。
09:03
to the grind of like training tiny models
转向训练小型模型那种枯燥乏味的工作
09:07
with very little data and very small clusters
数据极少、集群极小，
09:09
back in the 2010s.
回溯至2010年代。
09:11
Every week, my team and I study the top 10 technology
每周，我和团队都会研究未来十年
09:14
meta trends that will transform industries
将变革行业的十大科技宏观趋势。
09:16
over the decade ahead.
我关注的趋势涵盖人类与机器人协同、
09:17
I cover trends ranging from human under robotics,
通用人工智能、量子计算，以及交通能源、
09:19
AGI and quantum computing to transport energy,
长寿科技等领域。
09:22
longevity and more.
长寿及其他方面
09:23
There's no fluff.
没有废话。
09:24
Only the most important stuff that matters,
只有最重要、最核心的内容，
09:26
that impacts our lives, our companies and our careers.
关乎我们的生活、公司和事业。
09:30
If you want me to share these meta trends with you,
若你希望我与你分享这些宏观趋势，
09:32
by writing newsletter twice a week,
每周撰写两次简报，
09:34
sending it out is a short two minute read via email.
以两分钟即可读完的邮件形式发送。
09:37
And if you want to discover the most important meta trends,
若你想比他人早十年发现最重要的宏观趋势，
09:39
10 years before anyone else, this reports for you.
这份报告正是为你准备的。
09:42
Readers include founders and CEOs
读者包括来自全球最具颠覆性公司的创始人及CEO，
09:44
from the world's most disruptive companies
以及正在打造世界最颠覆性技术的创业者。
09:46
and entrepreneurs building the world's most disruptive tech.
如果你不想了解即将到来的趋势、其重要性，
09:49
It's not for you if you don't want to be informed
以及如何从中获益，那这份内容并不适合你。
09:52
about what's coming, why it matters
免费订阅请访问 demandist.com/meta-trends。
09:54
and how you can benefit from it.
你将比其他人提前十年掌握这些趋势。
09:56
To subscribe for free, go to demandist.com slash meta trends.
免费订阅请访问 demandist.com/meta-trends
10:00
You gain access to the trends 10 years before anyone else.
你将比其他人提前十年掌握趋势动向
10:03
All right, now back to this episode.
好的，现在回到这一集。
10:05
Yeah, so when last we spoke circa 2015,
是的，上次我们大约在2015年交谈时，
10:09
I think that was perhaps three years post-image net,
我想那大概是ImageNet之后三年，
10:13
five years pre-language models, our few shot learners.
语言模型和少样本学习器出现前五年。
10:18
Agents, agent AI was nowhere to be seen
智能体、智能体AI当时还完全不见踪影，
10:21
at the level of what we see now.
远未达到如今我们看到的水平。
10:23
Since you've written about your vision,
自从你阐述了自己的愿景——
10:26
what you've, I think, socialized as a modern touring test,
我认为你将其推广为一种现代图灵测试——
10:29
the idea of economic benchmarks for autonomy by agents.
关于智能体自主性的经济基准概念。
10:34
I'd love to hear where are Microsoft's economic benchmarks
我很想了解微软为这些智能体设定了哪些经济基准？
10:38
for these agents?
如果智能体即将接管经济，
10:39
If the agents are about to take over the economy
或接管大量具有经济价值的职能，
10:42
or take over so many economically useful functions,
为何我们仍局限于自动售货机这类基准，
10:45
why are we stuck with benchmarks like vending bench
而非由微软率先推出
10:48
rather than Microsoft leading the way
其智能体的经济自主性基准？
10:51
with Microsoft's economically autonomous benchmarks for its agents?
微软为其智能体设定的经济自主基准是什么？
10:54
Yeah, I mean, it's probably just worth adding the context
是的，我的意思是，或许值得补充一下背景信息，
10:56
that we met in 2015 in Puerto Rico at the AI safety conference.
我们是在2015年于波多黎各的人工智能安全会议上认识的。
11:00
True.
确实如此。
11:01
That many of the field now were at the same time.
当时该领域的许多人都在同一时间聚集在那里。
11:04
I think it's a seminal moment.
我认为那是一个具有里程碑意义的时刻。
11:05
Yeah, was it the day after New Year's Eve
是啊，是在新年前夜之后的那天，
11:07
or somewhere around New Year's?
还是新年期间左右？
11:09
It was pretty cold out everywhere except Puerto Rico.
当时其他地方都很冷，只有波多黎各是暖和的。
11:12
Yeah, exactly.
是的，没错。
11:12
It was pretty cool.
那真的很酷。
11:14
It was quite surreal moment, actually.
其实那是个相当超现实的时刻。
11:16
It's like a silamar right before it all happened.
就像一切发生前的片刻宁静。
11:18
Yeah, yeah, totally.
对对，完全同意。
11:21
And, you know, yeah, the modern touring test
而且，你知道，现代图灵测试
11:24
was something I proposed.
是我提出的概念。
11:26
I guess it was 2022 when I wrote it.
大概是在2022年我写下的它。
11:30
And it was basically making a pretty simple prediction.
这基本上是在做一个相当简单的预测。
11:33
If the scaling laws continued more data and compute
如果规模法则持续生效，更多数据和算力投入，
11:36
and adding an order of magnitude more compute
并且每年为全球最先进的模型增加一个数量级的算力，
11:38
to the best models in the world every year,
那么很明显，我们将从识别阶段——
11:40
then it's pretty clear we would go from recognition,
那是这一波浪潮的第一部分——
11:43
which was the first part of the wave,
进入生成阶段，显然我们现在正处于这一阶段的中期，
11:46
to generation, which is clearly we're now in the middle of
或者可能即将结束这一篇章。
11:49
or maybe ending that chapter,
或者也许结束那一章。
11:51
to then having perfect generation at every time step,
从而在每个时间步都实现完美生成，
11:54
which in sequence is going to produce
进而依次产生
11:56
assistive, agentive actions.
辅助性、主动性的行动。
11:59
And actions would obviously look like an intelligent
这些行动显然会像一位智能的
12:03
knowledge worker or a project manager or a strategist
知识工作者、项目经理、战略家、
12:05
or a startup founder or whatever it is.
初创公司创始人或任何其他角色。
12:07
And so then how would we measure that performance,
那么，我们该如何衡量这种表现，
12:09
rather than measuring it with academic
而不是用学术标准来衡量它？
12:11
and theoretical benchmarks,
以及理论基准，
12:12
one would clearly want to measure it through capabilities.
人们显然希望通过能力来衡量它。
12:16
Where can the thing do in the economy in the workplace
这个东西在经济和职场中能发挥什么作用，
12:19
and how do we measure the economy?
我们又该如何衡量经济？
12:20
We measure it by dollars and cents.
我们用美元和美分来衡量。
12:22
And so what would be the first model
那么，第一个能赚到一百万美元的模型会是什么？
12:24
to make a million dollars?
假设，据我所知，启动资金为十万美元。
12:26
Now given, as I recall, $100,000 in starting capital,
现在，据我回忆，给定10万美元的启动资金，
12:30
that's right.
没错。
12:30
Yeah, which model could turn into a million dollars?
是啊，哪种模型能变成百万美元？
12:34
10X return on investment by an agent.
代理人实现10倍投资回报。
12:36
Exactly.
正是如此。
12:38
And so I think that's a pretty good measure
所以我认为这是一个相当不错的衡量标准，
12:42
of performance and capability.
用于评估性能和能力。
12:44
And certainly, we've kind of just breathed past
而且显然，我们刚刚轻松超越了
12:47
the cheering test, right?
欢呼测试，对吧？
12:48
I mean, kind of has been passed.
我的意思是，某种程度上已经被翻篇了。
12:49
No one's really done a big, you know,
没有人真正搞出那种，你懂的，
12:52
alpha-goat moment.
“阿尔法山羊”式的轰动时刻。
12:53
I've always mentioned that all the time.
我一直都在提这件事。
12:54
The opener silver prize wound down
开场银奖的余温渐渐消退，
12:56
before we breathed past touring.
在我们巡演喘息之前。
12:58
Yeah.
是啊。
12:59
And no one celebrated it.
而且没有人为此庆祝过。
13:00
Yeah, where was the big, like, you know,
是啊，那种重大的、你懂的、
13:01
cast-brough deep blue moment?
划时代的深蓝时刻在哪呢？
13:03
Can we clink virtual glasses right now
我们现在能虚拟碰杯
13:05
and celebrate that we won?
庆祝我们赢了吗？
13:07
It happened.
它确实发生了。
13:09
Yeah, exactly.
对，没错。
13:10
And that's what it feels like to kind of make progress
而这就是在充满指数级叠加的世界里
13:14
in a world full of these compounding exponentials
取得进展的感觉。
13:16
where we just get desensitized to 10X.
我们只是对10倍的增长变得麻木了。
13:18
So much show that you can be like,
有太多展示让你觉得，
13:20
guys, why haven't you done it yet?
“伙计们，你们怎么还没做到？”
13:22
Yeah.
是啊。
13:23
Where's my Microsoft opener prize
我那为现代图灵测试准备的微软开场奖呢？
13:26
for the modern touring test?
没错。
13:27
Right.
正是如此。
13:28
Exactly.
正是如此。
13:30
Yeah, you know, like someone said to me earlier on,
是啊，你知道，就像之前有人跟我说的那样，
13:32
but you know, this AI thing, it's still in its infancy
但你也知道，这人工智能啊，还处在起步阶段，
13:35
isn't it?
对吧？
13:35
And I'm like, man, if this is infancy, wow.
然后我就想，天哪，如果这算起步，那可真是了不得。
13:38
Like, I can talk to my computer fluently
就像，我能跟我的电脑流利对话，
13:41
in real time, yeah, exactly.
实时交流，没错，就是这样。
13:45
So obviously, at the same time, agents don't really work,
所以显然，与此同时，智能体还不太行，
13:49
yeah, the action stuff is still progressing.
是啊，行动方面的功能还在发展中。
13:52
It's getting better and better every minute,
每一分钟都在变得更好，
13:54
but it's pretty clear that in the next couple of years,
但很明显，在接下来的几年里，
13:57
those things come into view and they're going to be very,
那些东西会逐渐显现，而且它们将会非常、
13:59
very good.
非常出色。
14:00
Can we get together again after the modern touring test
在现代巡演测试通过之后，
14:03
has been passed and just to celebrate,
我们能否再聚一次，只是为了庆祝、
14:05
recognize it?
认可它？
14:06
Virtual glasses again?
又是虚拟眼镜吗？
14:07
Absolutely.
当然。
14:08
Hopefully we can pop a champagne or something.
希望我们能开瓶香槟什么的。
14:10
I think we should.
我觉得应该这样。
14:11
Well, we'll have to pop the cork for us or something.
好吧，我们得为自己开个瓶塞什么的。
14:13
Yeah, exactly.
对，没错。
14:16
Dave, I want to flesh out that backstory a little bit more
戴夫，我想把那部分背景故事再充实一下，
14:18
to such a such a cool story.
这么酷的一个故事。
14:20
But I remember really clearly, you know,
但我记得非常清楚，你知道，
14:22
after DeepMind got acquired by Google,
DeepMind被谷歌收购后，
14:25
what was the price tag on that deal?
那笔交易的价格是多少？
14:26
It was like half a billion dollars.
大概是五亿美元左右。
14:28
Yeah, 650, 650, 650.
对，6.5亿，6.5亿，6.5亿。
14:30
What year was that?
那是哪一年？
14:31
2014, 2014.
2014年，2014年。
14:33
I remember reading maybe a year or two later
我记得大概一两年后读到过，
14:35
that Google justifies deal by having DeepMind
谷歌通过拥有DeepMind来证明这笔交易是合理的。
14:40
tune the air conditioning in the data centers.
调整数据中心的空调设置。
14:41
Yeah, right.
是啊，说得轻巧。
14:42
And my interpretation of that was like,
我当时心里想的是，
14:45
wow, this isn't going all that well.
哇，这进展可不怎么顺利。
14:47
And now it's obviously the biggest thing
而现在这显然成了人类历史上
14:48
that's happened in the history of humanity
最重大的事件，
14:50
and forking out all over the place.
到处都在衍生分支。
14:52
But I mean, the data center thing was pretty cool.
不过话说回来，数据中心那事儿还挺酷的。
14:55
We did actually reduce the cost of cooling
我们确实在2014年前降低了
14:56
the Google dates into fleets by 2014.
冷却谷歌数据中心的成本。
14:58
It was so funny because I read it at the time
当时读到这个时我觉得很好笑，
15:00
and I was like, what a bust.
心想，这真是个失败。
15:02
And then I read about it and we competed on the flight
后来我在飞来这里与你会面的航班上
15:04
over here to meet with you.
读到了相关报道并进行了讨论。
15:05
And it's like, it was actually what?
结果发现，实际上有
15:06
500 attributes fitting into the neural net.
500个特征被纳入神经网络。
15:09
And it was actually a lot more complicated
实际情况远比新闻里描述的复杂得多。
15:10
than the news made it sound.
当时就是这样。
15:12
That's it at the time.
没错。
15:13
That's right.
你当时在讨论指数曲线的平缓阶段，
15:15
You were talking about the flat part of the exponential
然后你想着，好吧，所有这些研发——
15:16
and you think about like, OK, all of this R&D,
它们离通用人工智能如此之近——
15:18
which is so close to becoming AGI,
却只是在调试空调。
15:22
is tuning the air conditioning.
是在调节空调。
15:24
That's the nature of exponentials.
这就是指数增长的本质。
15:25
They sneak up on you like this.
它们就是这样悄悄逼近你的。
15:26
But the other way to think about that
但换个角度想，
15:28
is that it's basically taking an arbitrary data input
这本质上是用任意数据输入
15:30
and arbitrary modality and using the same general purpose
和任意模态，通过同一通用方法
15:33
method to produce very accurate predictions
在新环境中产生极其精准的预测——
15:36
in a novel environment, which is the same thing that's
这与文本、音频、图像以及现在的编程领域
15:38
happened with text and audio and image and now coding
所发生的情况如出一辙。
15:43
obviously with other time series data.
显然，对于其他时间序列数据也是如此。
15:45
And so it's just another proof point of the general purpose
因此，这只是模型通用性的又一个证明。
15:48
nature of the models.
我认为人们很容易陷入一种思维，
15:49
And I think it's so easy to get caught up
觉得五年是一段很长的时间。
15:53
thinking five years is a long time.
其实它就像眨眼之间。
15:55
It's like a blink of an eye.
就像沧海一粟。
15:57
It's a drop in the ocean.
我想，这是因为我们总是被瞬息万变的新闻所裹挟。
15:58
I think because we're such a frantic second to second news
我想是因为我们身处一个分秒必争的狂热新闻环境中，
16:02
culture, social media type environment,
文化、社交媒体这类环境，
16:03
we just don't have an intuition for these timescapes.
我们对这些时间景观缺乏直觉。
16:06
I think other cultures do.
我认为其他文化是有的。
16:08
And I think historically, before digitization,
而且我觉得在数字化之前的历史时期，
16:11
we had much more of a natural intuition for the movement
我们对地貌、季节、时代等变迁，
16:15
of the landscape and the seasons and the ages and stuff.
曾拥有更自然的直觉。
16:19
And now we're just like, whoa, it's not coming quick enough.
如今我们只会惊叹：怎么还没来？
16:21
It's like, dude, it's coming pretty quick.
而事实是——伙计，它来得够快了。
16:23
We've shifted to a 24 or seven operations.
我们已转为全天候运营。
16:26
I mean, I know a lot of people including this group
我的意思是，我知道很多人，包括这个团队，
16:29
that are operating around the clock every day
都在日夜不停地运转，
16:32
just because when we do a moonshots podcast week to week
因为当我们每周做“登月计划”播客时，
16:38
just to celebrate and talk about what's just happened,
只是为了庆祝并讨论刚刚发生的事情，
16:41
it's insane on a week-by-week basis what's going on.
每周发生的事都令人疯狂。
16:44
Yeah.
是啊。
16:44
You know, on Peter's always saying,
你知道，彼得总在说，
16:46
people are very, very bad at exponentials.
人们非常非常不擅长理解指数增长。
16:49
200,000 years of evolution has us predicting tomorrow
20万年的进化让我们习惯于预测明天会和昨天一样。
16:53
will be like yesterday.
但你是少数经历过那种“空调变成AGI仅需几年”的人之一。
16:54
But you're one of the few people who, you know,
所以我们此刻正处在另一个拐点上，其影响极为深远。
16:56
having lived through that air conditioning
经历过那种空调环境后，
16:58
becomes AGI in just a few years.
几年内就会演变为通用人工智能。
17:01
So where we sit right now is on another inflection point
我们目前所处的正是一个新的转折点，
17:06
and the implications are massive.
其影响极为深远。
17:08
And people are way under reacting across the board.
而人们普遍反应严重不足。
17:10
And so you're one of the few people who, you know,
所以你算是少数几个，你知道，
17:12
having seen it before can say,
曾经见识过这种情况并能直言，
17:13
yeah, I just got very lucky.
没错，我只是运气特别好。
17:15
I mean, we were very lucky to have an intuition
我是说，我们很幸运能对指数增长有直觉，对吧？
17:18
for the exponential, right?
而这是一件非常强大的事，
17:19
And like, that's a very powerful thing
因为理论上我们都能观察到。
17:22
because we can all theoretically observe
因为理论上我们都能观察到，
17:25
the shape of the exponential.
指数的形状。
17:26
But to go through the flat part
但要穿过平坦的部分，
17:28
and then get excited by a micro doubling, you know?
然后被微小的翻倍所激发，你懂吗？
17:31
And like, that's the bit, is that when you're like,
而关键在于，当你像这样——
17:33
oh my god, this, like I remember this,
“天哪，这个，我记得这个，
17:37
the M-nist image generation thing,
那个MNIST图像生成的东西，
17:39
for sure.
没错。”
17:40
I've worked on that.
我做过那个。
17:41
Generative bolsters.
生成式增强。
17:42
There's like, these are like, I can't remember,
就像，这些是，我记不清了，
17:43
maybe 256 by 256 pixels.
大概是256乘256像素。
17:46
Yeah.
对。
17:47
You know, black and white, handwritten digits.
你知道的，黑白手写数字。
17:50
And, you know, I think this was like 2013,
而且，我觉得这大概是2013年，
17:54
maybe even 2012, and this guy, like,
甚至可能是2012年，然后这家伙，
17:57
I think maybe he was employing number five
我想他可能用的是第五号。
17:58
at DeepMind Don Vistra, this like awesome Dutch guy
在DeepMind的Don Vistra，有个超棒的荷兰小伙，
18:03
out of EPFL was generated like the first number seven.
来自EPFL，生成了第一个数字七。
18:09
There was provably not in the training set for the first one.
第一个七显然不在训练集里。
18:12
I was like, man, that is amazing.
我当时就想，天哪，这太神奇了。
18:15
Like how could it have made,
它怎么会生成出来的呢？
18:17
it's learned something about the idea of seven.
它已经学到了关于“七”这个概念。
18:19
That was the, you know, that was,
你知道，那就是——
18:20
it's got a concept of seven.
它已经理解了“七”的含义。
18:22
How cool is that?
这有多酷？
18:24
You know?
你知道吗？
18:24
So I got the highest score on M-nist ever in 1991
1991年，我在MNIST上拿到了史上最高分，
18:28
when it first came out when you were three years old, right?
当时这个数据集刚发布，你才三岁，对吧？
18:31
Yeah, that's nine, 99 years old.
是啊，那是九岁，九十九岁。
18:33
Okay.
好吧。
18:35
Yeah, and actually that's the same data set
没错，而且实际上就是同一个数据集，
18:37
that's now in PyTorch that people like Benchmark are.
现在在PyTorch里，大家用它来做基准测试。
18:40
Pretty crazy.
太疯狂了。
18:41
Incredible.
难以置信。
18:42
How often are you surprised by what you're seeing?
你多久会被自己看到的东西惊到一次？
18:45
I mean, how often is there like a move, 37,
我是说，像第37步那样的妙招，
18:49
you know, sort of like a harm moment?
那种堪称“神来之笔”的时刻，多久会出现一次？
18:51
Is it happening more frequently?
这种情况变得更频繁了吗？
18:54
I was absolutely blown away
我完全被震撼到了——
18:57
by the first versions of Lambda at Google.
被谷歌第一版Lambda模型的表现。
19:01
It was like a maybe 12 people working on it
大概有12个人在做这个项目，
19:04
led by Nome Shazir, Daniel DeFritis and Kwakli.
由Nome Shazir、Daniel DeFritis和Kwakli领导。
19:08
And I got involved later, maybe three or four,
我后来才加入，大概在他们开始后的
19:11
five months after they've been going.
三四、五个月。
19:13
And it was just breathtaking.
那真是令人叹为观止。
19:15
I mean, obviously everyone at that point
我是说，显然那时候每个人
19:18
had been playing with LLM's
都在玩LLM，
19:20
and they were like one shot, the producer and answer
就像一次生成，直接给出结果和答案。
19:22
and you know, have a prompt and blah, blah, blah.
你知道，就是给个提示词，然后巴拉巴拉说一堆。
19:24
But they were really the first to push it for conversation
但他们确实是第一个将其推向对话交流的。
19:27
and dialogue.
而正是看到那些涌现出来的行为模式，
19:28
And it just seeing the kind of emergent behaviors
在你自身中浮现，比如那些
19:33
that arise in yourself, like things
你甚至没想到要去问的事情，
19:35
that you didn't even think to ask
因为你知道那将是一场对话，
19:36
because you know that's going to be a dialogue
而不是针对某个情境的单一提问。
19:38
rather than a question on to situation.
而非针对情境提出疑问。
19:40
Sounds so trivial to say that.
说起来真是轻巧。
19:42
Like in hindsight, because now we're obviously
就像事后看来，因为如今我们显然
19:44
steeped in conversation as the default mode.
早已将对话视为默认模式。
19:46
But that was like breath taking for me.
但当时那对我来说简直是震撼。
19:48
And obviously then I pushed really hard
于是我当然拼尽全力
19:50
to try and ship that at Google.
试图在谷歌推动这个产品上线。
19:51
And for various reasons, we couldn't get it launched.
但由于种种原因，我们未能成功发布。
19:54
And that was when we all left, like I left
就在那时，我们都离开了，我也一样。
19:56
and Nome left to do character
诺姆离开去塑造角色了，
19:58
and you know, David Luan left to do adept.
你知道，大卫·卢安离开去成为专家了。
20:01
And you know, we were all like, okay, this is the moment.
然后，我们所有人都觉得，好吧，就是这一刻了。
20:04
And so, you know, I think there's been still
所以，你知道，我认为自那以后
20:06
a couple moments since then,
还有过几次这样的时刻，
20:07
but that was probably the biggest one
但那可能是最大的一个，
20:09
that I remember in recent memory is mind blowing.
在我最近的记忆里最令人震撼的。
20:11
And the scaling laws have delivered
而规模定律已经兑现了。
20:13
such unexpected performance, right?
如此出人意料的表现，对吧？
20:16
I mean, was going back to your earlier days,
我的意思是，回顾你早期的日子，
20:20
did you anticipate the kinds of capabilities
你是否预见到了如今这些能力？
20:24
that have resulted?
我是说，这对你而言是完全可以预见的，
20:25
I mean, it was just predictable for you
还是依然让你惊叹于它所能做到的——
20:27
or is it still like wow, what it's able to do
在医学、对话、科学研究中，
20:30
in medicine, in conversation, in scientific research?
尤其是仅凭纯文本就能实现这一切？
20:34
Well, especially working off of pure text.
尤其是在仅基于纯文本的情况下，
20:36
I mean, how far we've gotten,
我的意思是，我们走了多远，
20:38
and nobody, I think, well, you tell me,
而且没有人，我想，嗯，你告诉我吧，
20:40
but nobody would have seen how far we would get
但没有人会预料到我们能走这么远，
20:42
with just text.
仅仅依靠文本。
20:44
Yeah, I mean, in 2015, I collaborated
是啊，我的意思是，2015年，我和一群非常棒的人合作，
20:47
with a bunch of really awesome people
写了一篇关于NLP深度学习的论文，一篇深度思考，
20:49
on a NLP deep learning paper, a deep mind,
当时我们基本上是在尝试预测——
20:53
where we were essentially trying to predict
我们本质上是在尝试预测，
20:56
a single word in a sentence.
一个句子中的一个词。
20:59
I think we had scraped like Daily Mail news articles
我们当时抓取了像《每日邮报》和CNN的新闻文章，
21:01
and CNN articles and we were like,
然后想，
21:03
can we fill in the blank,
能不能填空，
21:04
just predict like one word in a sentence
比如预测句子中的一个词，
21:07
or complete the final word in a sentence,
或者补全句子的最后一个词，
21:09
like the inverse of the problem
就像现在模型所解决问题的反向操作。
21:10
that way the models now work.
这正是当前模型运作的方式。
21:12
And you know, it was like a pretty big contribution,
你知道，那算是一个相当大的贡献，
21:15
it was a good, well-sighted paper,
那是一篇不错、眼光独到的论文，
21:16
but it was like, this is never gonna scale,
但问题是，这根本没法规模化，
21:18
like we were just like, okay,
我们当时就想，好吧，
21:19
we were way too early, not enough data,
我们太超前了，数据不够，
21:21
not enough compute,
算力也不够，
21:22
but we were still optimistic
但我们依然乐观，
21:25
that with more data and compute,
认为只要有更多数据和算力，
21:28
that is a method that will work.
这是一个可行的方法。
21:30
So I don't wanna have like hindsight bias
所以我不想事后诸葛亮地说，这一切都是完全可以预见的，
21:32
and say, well, it was all very predictable,
但整个领域里的人——显然不只是我——
21:34
but everyone in the field, not just obviously me,
都拿着同样的锤子和钉子，
21:37
but everyone in the field just had the same hammer and nail
一直在那里敲敲打打。
21:41
and just kept chipping away.
比如，我们能给这个数据加更多数据吗？
21:42
Like, can we add more data to this?
我们能明确一下预测目标吗？
21:44
Can we clarify our prediction target
我们能否明确我们的预测目标？
21:45
and can we add more compute?
我们可以增加更多算力吗？
21:47
And broadly speaking, that's what's delivered.
从广义上讲，这就是已经实现的结果。
21:50
What's delivered?
已经实现了什么？
21:51
Yeah.
是的。
21:52
Yeah, we'd love to maybe pull on that theme a bit.
是的，我们或许想围绕这个主题再深入探讨一下。
21:54
So you mentioned how surprising your generative seven
你提到了从MNIST生成的“七”
21:58
from MNIST was.
有多么令人惊讶。
21:59
You mentioned how surprising the success of Lambda
你提到了Lambda的成功有多么令人惊讶。
22:03
for conversational tuning and conversational performance
针对对话调优与对话表现的整体情况，我认为你在这期节目中已经透露了一些我此前未知的消息。如果我没理解错的话——若有误请指正——但按照预期，在未来两年内，也就是我理解为2027年，我们将看到智能体开始通过现代图灵测试。
22:06
in general is, I think you've made already a little bit
总体而言，我认为你在这期节目中已经透露了一些我所了解的消息——
22:08
of news to my knowledge in this episode
如果我没理解错的话，请纠正我——
22:12
that if I understood correctly,
但按照预期，在未来两年内，
22:13
correct me if I'm wrong, by,
也就是我理解为2027年，
22:15
but with the expectation that in the next two years,
我们将看到智能体开始通过你的现代图灵测试，
22:18
so I read that as 2027,
你心中是否有时间表？
22:19
we'll see agents start to pass your modern touring test,
我们将看到智能体开始通过你的现代图灵测试，
22:22
we'll see them be able to 10X 100,000 US dollar
我们将看到他们能够实现10倍于10万美元的投资回报。
22:26
return on investment.
我对接下来会出现的惊喜充满好奇。
22:27
I'm curious about the next surprises to come.
人工智能用于科学，微软研究院有一个AI for Science计划。
22:31
AI for science, Microsoft Research has an AI
你心中是否有AI解决数学问题的时间表？
22:33
for science initiative.
我们目前正看到大量初创公司涌现。
22:35
Do you have timelines in your mind
你心中是否有时间表？
22:37
for AI solving math, which we're seeing
对于AI解决数学问题，我们现在看到大量初创公司涌现，从编程谜题到数学领域，其核心在于逻辑推理的本质。这种能力存在于大量日志数据中，并且它本身也自然契合这一点。是的，我认为这已经在非常自然地发生了，对吧？我是说，它完全搞错了。
22:39
a whole bunch of startups right now,
现在有一大批初创公司，
22:41
tear through Erdish problems, AI for physics,
攻克Erdish难题，AI应用于物理、化学、医学、材料科学？
22:43
chemistry, medicine, material science?
材料科学，你认为会发生什么，何时发生？
22:45
Material science, what do you think happens and when?
对，其实你刚提醒了我，
22:48
Yeah, actually you've just reminded me
最近让我震惊的是，
22:50
that the more recent thing that has blown my mind
这些方法竟能从编程谜题、数学等单一领域，
22:53
is the fact that these methods could learn
学习到逻辑推理的本质。
22:56
from one domain coding puzzles, maths,
从单一领域的编程谜题、数学，
23:01
the essence of like logical reasoning.
到逻辑推理的本质——
23:04
So just as it learnt the essence
所以就像它学会了数字七的本质
23:06
or the conceptual representation of a number seven,
或概念化表征一样，
23:10
it's clearly learnt the abstract nature
它显然也掌握了抽象特性，
23:13
of like a logical reasoning path.
比如逻辑推理路径。
23:15
And then can basically apply that, you know,
然后基本上能将其应用，
23:18
to many, many other domains.
到许多许多其他领域。
23:21
And so that's kind of interesting
这还挺有意思的，
23:24
because it can apply that as well as the underlying hallucinations
因为它既能应用这一点，也能利用底层的幻觉机制。
23:27
slash creativity sort of instinct that it has,
斜杠式的创造力，是一种近乎本能的特质，
23:31
which is more like interpolation.
它更像是一种插值能力。
23:34
But those two things combined are like a lethal combination
但这两者结合起来，
23:38
for making progress in like say new mathematical theorem
就像一种致命的组合，
23:42
solving or new scientific challenges
用于推动进步，比如解决新的数学定理
23:44
because that's basically what humans do all the time.
或应对新的科学挑战，
23:47
We should have combined these two, you know, capabilities.
因为人类本质上一直在做这些事情。
23:50
And so I couldn't really put,
我们本应将这两种能力结合起来。
23:52
I mean, some people want to put dates on those things,
我的意思是，有些人想给这些事情定个时间表，
23:54
it's hard to put a date on those things
但很难给它们定个时间表，
23:55
because they really are very, very fundamental.
因为它们确实非常、非常基础。
23:58
But it feels like they're definitely within reach
但感觉它们肯定是可以实现的，
24:00
it's hard to kind of, it would be very odd to bet against them.
很难说，要是赌它们不会实现，那会非常奇怪。
24:04
It just maybe from an over-under perspective,
只是从“上下限”的角度来看，
24:06
do you think, say given all of the recent progress
你觉得，比如说，考虑到数学领域最近的所有进展，
24:09
in math, for example, do you think solving science
你认为解决科学问题……
24:13
and engineering for some reasonable definition of solving
对于某种合理定义的“解决”而言，工程实现最终会比现代图灵测试的十倍投资回报更难还是更容易？
24:16
is going to ultimately be harder or easier
它会更难，因为我认为大量训练数据——
24:19
than modern-turing test 10xing of return on investment?
如果你愿意这么理解的话——关于职场活动序列、
24:23
It's gonna be harder because I think a lot of the training data
创业、初创公司等领域的这类数据，
24:28
if you like, for strings of activity in the workplace
其实大量存在于各类日志数据中，
24:32
or in entrepreneurialism, startups and so on,
而且它本身也天然适合这种处理方式。
24:35
that kind of exists in a lot of the log data
这类数据在大量日志中已经存在。
24:38
and also it lends itself naturally
并且它本身也很自然。
24:40
to real-time calibration with a human.
实现与人类的实时校准。
24:43
So the AI can sort of check in, the human can oversee,
这样AI可以随时确认，人类可以监督，
24:46
the human can intervene, the human can steer and calibrate.
人类可以干预，人类可以引导和校准。
24:49
And so it's gonna be a much more sort of dual combined effort
因此，这将会是一种更加双重的联合努力，
24:55
between AI and reinforcement learning in that category.
结合AI与强化学习在该领域的应用。
24:57
Yeah, where a human is participating
是的，人类参与其中，
24:59
in steering the reinforcement learning trajectory,
引导强化学习的轨迹，
25:02
whereas in a novel domain,
而在一个全新的领域中，
25:04
where it really is inventing completely new knowledge,
真正在创造全新知识的地方，
25:08
that's kind of more happening in a very abstract sort of vector space
更像是发生在一种非常抽象的向量空间里，
25:11
and it's like unclear yet how the human is gonna intervene
目前尚不清楚人类将如何介入
25:15
in the theorem solving problem.
定理求解问题。
25:17
Obviously, everyone's working on this,
显然，每个人都在研究这个，
25:18
particularly in like biology, synthetic materials
尤其是在生物学、合成材料
25:20
and stuff like that, because you want to,
以及类似领域，因为你希望——
25:23
I mean, it's already giving humans a better intuition
我的意思是，它已经赋予了人类更好的直觉。
25:25
for where in the search base to look for for new hypotheses
在搜索库中寻找新假设的位置，
25:28
for drugs, for example, or for materials.
例如针对药物或材料。
25:30
And then the human can either take or reject that,
然后人类可以选择接受或拒绝，
25:32
feed that back to the model, then obviously go and test it in silicone
将其反馈给模型，接着显然会在模拟中进行测试，
25:36
and be like, oh, we actually ran the experiment,
然后说，哦，我们实际上做了实验，
25:38
we pipetted a bunch of stuff
我们移液了一堆东西，
25:40
and then feed that back into the model to improve the search.
再将其反馈给模型以改进搜索。
25:42
And maybe it's a follow-up question,
或许这是一个后续问题。
25:44
what can humanity in general, Microsoft specifically,
人类整体、特别是微软，
25:47
or all of the AI community subset of which listens to the podcast,
或者所有收听本播客的AI社区成员，
25:51
what can they do to accelerate AI for science
他们能做些什么来加速AI在科学领域的应用，
25:53
and accelerate the solution to science,
并借助AI加速解决科学、数学、工程问题？
25:55
math, engineering with AI?
我认为，这可以说是对人类最具影响力的事情之一，
25:56
I mean, arguably that would be like when the most impactful things
它将从根本上让一切以光速前进。
26:00
for humanity that would just fundamentally move everything at light speed.
是的，我觉得这已经在非常自然地发生了，对吧？
26:05
Yeah, I mean, I think it's already happening very organically, right?
是的，我的意思是，我认为这已经在非常自然地发生了，对吧？
26:08
This is also not only is this like the most powerful technology in the world,
这不仅是世界上最强大的技术，
26:13
it's also the fastest proliferating in human history.
也是人类历史上传播最快的技术。
26:16
And, you know, sort of the cost of access,
而且，你知道，使用成本——
26:20
the cost of inference, coming down by multiple orders of magnitude
推理成本——每隔几年就会下降几个数量级，这简直……
26:23
every couple of years, is kind of...
你曾想过它会变得如此便宜吗？
26:25
Would you ever imagine it would be so cheap?
这一点我也……
26:27
That bit I also...
我的意思是，完全搞错了。
26:28
I mean, it totally got wrong.
我是说，它完全搞错了。
26:29
Right, it's like the biggest surprise for me isn't that we're getting
对，对我来说最大的惊喜不是我们获得了这种能力水平，而是它如此便宜、如此易得。
26:32
this level of capability, it's how cheap it is, how accessible it is.
百分之百。
26:36
100%.
我的意思是，这相当于两年内提升了1000倍。
26:37
I mean, that's 1,000X over two years.
那么它会再次做到这一点吗？
26:39
So is it going to do that again?
还是说那只是一次性的……
26:40
Or was that a one-time...
是1000倍吗？
26:42
Is it 1,000X?
我觉得大概是100倍。
26:43
I think it's like 100X.
我认为大概是100倍。
26:44
The inference cost has come to a single token inference cost.
推理成本已降至单个token的推理成本。
26:47
I think it's come down 100X in the last two years.
我认为过去两年里它下降了100倍。
26:49
That's two years, okay.
那是两年时间，没错。
26:50
They've been competing estimates.
他们一直在互相竞争的估算。
26:51
Some estimates measure intelligence per token per dollar.
有些估算衡量的是每美元每token的智能水平。
26:54
Right.
对。
26:55
There's an estimate that it's 40X year over year,
有一种估算认为每年增长40倍，
26:57
but that's for certain weight classes of models.
但那是针对特定权重类别的模型。
26:59
I've seen 1,000X for some classes of models.
我见过某些模型类别达到1000倍的提升。
27:03
It's craziness.
这太疯狂了。
27:04
Oh, wow.
哦，哇。
27:05
That's wild.
那可真够野的。
27:06
No, I mean, yeah, that's actually a good point.
不，我是说，对，这确实是个好观点。
27:08
I got that totally wrong because I didn't think that the biggest companies in the world
我完全搞错了，因为我没想到世界上最大的公司
27:13
were going to open source models that cost billions of dollars essentially to train.
会开源那些训练成本高达数十亿美元的模型。
27:18
So much so that when we found it in flexion, and this was like maybe nine months, or maybe
以至于当我们在Flexion中发现这一点时——那大概是九个月前，或者可能——
27:25
a year before ChatGbT was released, we started doing fundraising a year before ChatGbT was
在ChatGPT发布的前一年，我们就开始了融资。
27:30
released.
我们基本上以25人的团队筹集了15亿美元，用于建设当时视频和通话领域最大的H100集群。
27:33
We basically raised a billion and a half dollars with a 25-person team to build what
我们是CoreWeave的首个AI客户，而他们之前专注于加密货币领域，我们作为他们的首个AI客户，与他们合作构建我们的数据基础设施，显然英伟达也给予了我们支持。
27:40
at the time was the largest H100 cluster within video and call.
我认为我们当时建设的集群大约有15000块H100，未来将扩展到22000块。
27:45
We were call use first AI customer, and they were previously in crypto, and we were
我们曾是他们的首个AI客户，他们之前做加密货币，而我们作为他们的第一个AI客户，与他们合作构建数据，显然英伟达也支持了我们。
27:51
like their first AI customer working with them to build our data, and obviously Nvidia
当时我们搭建的集群大约有15000块H100，后续将增至22000块。
27:55
got behind us.
这纯粹是成本问题。
27:56
I think we built cluster at the time was about 15,000 H100s, going to 22,000.
所以像Perplexity这样的公司，例如在Llama问世后成立，知道他们可以依赖Llama，当然还有OpenAI的API以及其他所有API。
28:03
And then obviously that year, ChatGbT came out and a few months around that time, Lama
然后那一年，ChatGPT 问世了，几个月后，Llama 也发布了。
28:12
came out.
于是我们心想，天哪，我们公司的整个资本基础就这样被开源给动摇了。
28:14
And so we were like, oh my god, our entire capital base of our company has just been
看起来开源的趋势并非关乎性能，
28:21
undermined by the fact that open source.
而纯粹是成本问题。
28:24
It seems like open source is going to, it's not really about performance.
所以像 Perplexity 这样的公司，就是在 Llama 出现后成立的，因为他们知道可以依赖 Llama，当然还有 OpenAI 的 API 以及其他所有 API。
28:28
It's just cost.
这只是成本问题。
28:29
So then like perplexity, for example, founded after the arrival of Lama, knowing that they
所以，比如像Perplexity这样的公司，在Llama出现之后成立，知道他们可以依赖Llama，当然还有OpenAI的API以及其他所有API。
28:34
could depend on Lama and obviously open AI as an API and all the other APIs.
可能取决于Lama，当然还有作为API的OpenAI以及其他所有API。
28:39
And so then they had a much, much lower cost base, basically.
于是，他们的成本基础就低得多了，基本上是这样。
28:43
So yeah, that was like another thing that it was not predictable, I mean, other people
所以，没错，这又是另一件无法预测的事，我是说，其他人
28:48
predicted it to be clear, I just got it wrong.
预测得很清楚，只是我搞错了。
28:50
Abundance, baby, demonization, democratization of the most powerful tools in the universe,
富足，宝贝，妖魔化，宇宙中最强大工具的民主化——
28:56
our universe, you know, hyper deflation, if any, hyper deflation.
我们的宇宙，你知道，超级通缩，如果有的话，就是超级通缩。
28:59
I think that's a really important point, like the cost of accessing knowledge or intelligence
我认为这一点非常重要，比如获取知识、智能或能力的成本，
29:06
or capability, intelligence as a service, as a service is going to go to zero marginal
智能即服务，作为一种服务，其边际成本也将趋近于零。
29:11
cost too.
成本太高。
29:12
And obviously that's going to have massive labor deflation displacement effects, but
显然，这将带来巨大的劳动力通缩与岗位替代效应，但
29:15
it's also going to have a weirdly deflationary effect because, you know, what is going to
它也会产生一种奇特的通缩影响，因为，你知道，人们将不再拥有以美元计价的收入来购买商品，这显然很糟糕。
29:19
happen that people aren't going to have dollar-based incomes to go by things, that's obviously
但消费品的成本也会随之下降。
29:25
bad.
所以我们实际上面临一种转型错配，因为，你知道，劳动力市场会在服务成本下降之前就受到冲击。
29:26
But the cost of consuming stuff is also going to come down.
而这两者之间可能存在10到20年的滞后，这将极具破坏性。
29:29
So we actually have a transition mismatch because, you know, sort of labor markets are going
所以我们实际上存在一个转型错配，因为你知道，劳动力市场会先受到影响，而服务成本下降则要滞后。
29:34
to be affected before cost of services comes down.
这两者之间可能有10到20年的延迟，这将带来极大的不稳定性。
29:37
And maybe there's a 10, 20-year lag between that, which is going to be very destabilizing.
两到七年的时间框架。
29:41
Which by the way is what we started to talk about a little bit earlier.
顺便提一下，这正是我们稍早开始讨论的内容。
29:44
I mean, my, I posit that in the long term, there's an extraordinary future for humanity,
我的意思是，我认为从长远来看，人类将拥有非凡的未来，对吧？
29:51
right?
届时，食物、水、能源、医疗和教育将惠及每一个男人、女人和孩子。
29:52
So access to food, water, energy, healthcare, education is accessible to every man, woman,
而短期内的挑战才是关键，对吧？
29:56
and child.
也就是未来两到七年的时间段。
29:57
And it's the shorter term that is challenging, right?
这符合你的模型吗？
30:02
The two to seven-year time frame.
这符合你的模型吗？
30:05
Is that fit your model to?
就业疲软。
30:06
Yeah.
是的。
30:07
The short term, I think, is going to be quite unstable.
短期内，我认为会相当不稳定。
30:10
The medium to longer term, like, you know, it's pretty clear that these models already
从中长期来看，你知道，很明显这些模型已经
30:14
well-classed diagnostics, we released a paper, maybe four or five months ago now called
在诊断领域表现优异，我们大约四五个月前发布了一篇论文，
30:23
the MAI Diagnostic Orchestra, and essentially it uses a ton of models under the hood to
名为《MAI诊断管弦乐团》，本质上它在底层使用了大量模型，
30:28
try and, you know, take a set of rare conditions from the New England Journal of Medicine, you
试图从《新英格兰医学杂志》中选取一组罕见病症，
30:34
know, rare cases that can't be easily diagnosed that the best experts do, you know, a kind
你知道，那些难以诊断的罕见病例，即便是最顶尖的专家也做得不太好。
30:39
of weak job on.
而且，你知道，令人难以置信的是，如果你让AI自行运作，它的准确性要高得多。
30:41
And it's like four times more accurate, roughly, it's about two X less the cost in terms
准确度大约提升了四倍，粗略来说，不必要的检测成本降低了约两倍。
30:47
of unnecessary testing.
有一项来自哈佛和斯坦福的研究，这次考察的是GPT-4、单独执业的医生、以及医生配合GPT-4的情况。
30:48
There's a study that ox that came out of Harvard and Stanford looking at, in this case,
没错。
30:54
was GPT-4, a physician by themselves, a physician with GPT-4 and GPT-4 by itself.
结果令人惊叹：如果让AI独立工作，它在诊断上的准确度远超人类。
31:00
Yeah.
或者说，这反映了我们思维中的偏见，正如昨天所见，以及我们近期的诊断案例。
31:01
And it was, you know, incredible that if you left the AI alone, it was far more accurate
而且，你知道，如果你不干预AI，它的准确性会高得惊人，这真是不可思议。
31:06
in diagnostics than the human.
在诊断方面比人类更胜一筹。
31:08
Or biased in our thoughts in our, what we saw yesterday, our recent diagnoses.
或是我们思维中的偏见，源于我们昨日所见、近期的诊断。
31:12
Yeah.
嗯。
31:13
Actually, we got a lot of feedback after we released the paper because we only showed
实际上，论文发布后我们收到了大量反馈，因为我们只展示了
31:17
the AI on its own, the physician on its own.
AI单独工作、医生单独工作的情况。
31:21
And a lot of people wanted to see what it was like to have the physician and the AI, or
很多人想看看医生与AI协作的效果，
31:24
at least the physician have access to Google search as well.
或者至少让医生也能使用谷歌搜索。
31:28
And that improves performance a little bit, but the AI still trumps by quite a way.
这确实能小幅提升表现，但AI依然遥遥领先。
31:32
Damn.
该死。
31:33
What are you thinking?
你在想什么？
31:34
Oh, so much.
哦，这么多。
31:35
So, Microsoft, you've been here how many years now?
那么，微软，你在这里已经多少年了？
31:39
Just a year and a half, a year and a half.
才一年半，一年半。
31:41
So you feel like you're part of the, you're indoctrinated.
所以你觉得你已经融入其中，被同化了。
31:44
So what's the, what's the mandate from Satya?
那么，萨提亚下达的任务是什么？
31:46
Is it win AGI, or is it be self-sufficient, or what is the, what's the target?
是赢得AGI，还是实现自给自足，或者目标是什么？
31:54
I don't think there's really a winning of AGI.
我不认为真的存在“赢得AGI”这回事。
31:56
I think this is a misframing that a lot of people have kind of imposed on the field.
我觉得这是很多人强加给这个领域的一种错误框架。
32:01
Like, I'm not sure there's a race, right?
比如说，我不确定这是不是一场竞赛，对吧？
32:04
I mean, we're all going as fast as we possibly can, but a race implies that it's zero sum.
我的意思是，我们都在尽可能快地前进，但“竞赛”意味着零和博弈。
32:10
It implies that there's a finish line and it implies that there's like medals for one,
它意味着有一条终点线，还意味着有奖牌颁给第一名、第二名和第三名，
32:15
two and three, but not five, six and seven.
而不是第五名、第六名和第七名。
32:18
And it's just like not quite the right metaphor.
这其实不太像是一个恰当的比喻。
32:20
As we know, technologies and science and knowledge proliferate everywhere, all at runs
正如我们所知，技术、科学和知识无处不在，几乎同时或在短短一两年内，
32:25
are all scales, basically simultaneously, or within a year or two.
以各种规模迅速传播。
32:30
And so my mission is to ensure that we are self-sufficient, that we know how to train
因此，我的使命是确保我们能够自给自足，懂得如何培养人才。
32:35
our own models end-to-end from scratch at the frontier of all scales on all capabilities
我们在前沿领域，从零开始端到端构建自有模型，覆盖所有规模与能力维度；
32:41
and we build an absolutely well-class super intelligence team inside of the company.
并在公司内部打造了一支绝对顶尖的超级智能团队。
32:45
I'm also responsible for co-pilot.
我还负责Copilot项目。
32:47
So this is sort of our tool for taking these models to production in all of our consumer
这可以说是我们将这些模型部署到所有消费者端产品的工具。
32:51
surfaces.
所以澄清一下，当我们关注Polymarket（我们在播客中经常讨论）——
32:52
So just to clarify, so when we look at Polymarket, which we do a lot on the podcast,
关于年底谁拥有最佳AI模型、以及明年年底谁拥有最佳AI模型的竞赛中，
32:57
the horse race to who has the best AI model at the end of the year and who has the best
那条榜单上并没有微软的名字。
33:01
AI model at the end of next year, there's no Microsoft line on that chart.
明年年底的AI模型，那张图表上并没有微软的线。
33:06
So now there will be.
那么现在就会有了。
33:07
I assume.
我猜。
33:08
Yeah, there will be.
是的，会有的。
33:09
Yeah, next year we'll be putting out more and more models from us, but this is going
对，明年我们会推出越来越多我们的模型，但这需要很多年才能建成。
33:13
to take many years for us to build this.
我的意思是，你知道，DeepMind或OpenAI，这些是成立十年的实验室，已经养成了进行真正前沿研究的习惯和实践，能够仔细剔除失败并重新引导人员。
33:15
I mean, you know, deep mind or open AI, these are decade-old labs that have built the habit
我的意思是，你知道，DeepMind或OpenAI，这些是成立十年的实验室，已经养成了习惯
33:21
and practice of doing really cutting interest search and being able to weed out carefully
和实践，进行真正前沿的研究，并能够仔细剔除
33:26
the failures and redirect people.
失败，重新引导人们。
33:28
I mean, this is an entire culture and discipline that takes many years to build.
我的意思是，这是一个需要多年才能建立起来的完整文化和学科。
33:32
But yeah, we're absolutely pushing for the frontier.
但没错，我们绝对在推动前沿。
33:34
We want to build the best super intelligence and the safest super intelligence models in
我们希望打造世界上最好的超级智能和最安全的超级智能模型。
33:38
the world.
是的。
33:39
Yeah.
很好。
33:40
Nice.
那么当你到达时，如果我们回到Inflection，当时的理念是18年100人，我们要构建一个大型Transformer。
33:41
So when you arrived, so if we go back to inflection, the thesis there is 18,100, we're going
所以当你到来时，如果我们回到Inflection，当时的论点是18,100，我们要
33:47
to build a big transformer.
构建一个大型Transformer。
33:49
We're going to take a transformer architecture build.
我们将采用Transformer架构进行构建。
33:51
So I assume now you've got all the open AI source code and that was here.
所以我假设你现在已经拿到了所有OpenAI的源代码，它们就在这里。
33:57
You probably looked at it a year and a half ago on day one, when you arrived, it's like
你可能一年半前刚来的第一天就看过它了，就像这样——
34:00
start scrolling, I guess.
开始滚动浏览吧，我猜。
34:01
I don't know.
我不知道。
34:02
I'm trying to visualize how multi-decker billion dollars of R&D, what it looks like and
我试图想象数十亿美元的研发投入是什么样子，
34:09
how it arrives in a building, but you just dropped right into it.
以及它是如何进入一栋大楼的，但你就这样直接投入其中了。
34:13
So there was a whole team here already working on it.
所以这里已经有一个完整的团队在从事这项工作。
34:15
Did you bring in your team or how?
你是自己带了团队过来，还是怎么着？
34:17
Yeah.
嗯。
34:18
I mean, all my team came over and obviously we've been growing that team a lot.
我是说，我的整个团队都过来了，而且我们一直在大力扩充这个团队。
34:20
We've hired a lot from all the major labs and we're very much in the trenches of the
我们从各大实验室招了不少人，现在正深陷招聘大战之中，这场面还挺魔幻的。
34:24
hiring wars, which are quite surreal.
我是说，这种局面真是前所未有。
34:26
I mean, it's kind of unprecedented how that's working out.
确实。
34:29
Pretty.
嗯。
34:30
Yeah.
嗯。
34:31
I mean, phone calls every day from all the CEOs to all of the other people.
我的意思是，每天都有CEO们打给其他人的电话。
34:33
So it's this constant battle.
所以这是一场持续不断的战斗。
34:36
And yeah, I mean, we're really building out the team now from scratch.
是的，我们现在真的在从零开始组建团队。
34:39
Okay.
好的。
34:40
It's pretty much how it's been.
基本上一直都是这样。
34:41
10,000 employees.
一万名员工。
34:42
No, no.
不，不。
34:43
So the core super intelligence team is like a few hundred.
核心超级智能团队大概只有几百人。
34:45
I mean, that's really the number one priority.
我的意思是，这确实是首要任务。
34:48
And the rest of that is copilot, the search engine.
而其余部分则是副驾驶，也就是搜索引擎。
34:50
Yeah.
是的。
34:51
Along the lines, let's have to ask because, you know, the terms AGI and ASI, you know,
顺着这个思路，我们得问一下，因为你知道，AGI和ASI这些术语，
34:56
super intelligence start getting thrown around, you know, in a very interesting fashion.
超级智能开始被以一种非常有趣的方式提及。
35:03
Do you do you have a internal definition of AGI versus digital super intelligence here?
你们内部对AGI和数字超级智能有定义吗？
35:09
Yeah.
有。
35:10
I mean, I think very loosely, it's these are just points on a curve.
我认为，粗略来说，它们只是曲线上的不同点。
35:16
Are they interchangeable in your mind, AGI and ASI, or are they different?
在你的认知中，AGI（通用人工智能）和ASI（超级人工智能）是可以互换的概念，还是有所区别？
35:18
Yeah.
是的。
35:19
I mean, I think they're generally used as different.
我的意思是，我认为它们通常被用作不同的概念。
35:22
I mean, I think that, well, different people have different definitions.
我觉得，嗯，不同的人有不同的定义。
35:27
The AGI definition, it's like the touring test.
AGI的定义有点像图灵测试。
35:30
It'll pass by and it'll be blurred and we will have recognized it in retrospect.
它会悄然通过，界限会变得模糊，而我们事后才会意识到它的存在。
35:33
Yeah.
是的。
35:34
Roughly speaking, at the far end of the spectrum, a super intelligence is an AI that can
粗略来说，在光谱的远端，超级智能是一种能够……
35:40
perform all tasks better than all humans combined and has the capacity to keep improving
比所有人类加起来更好地完成所有任务，并且具备随时间持续自我改进的能力。
35:47
itself over time.
那么我该何时问你的问题？
35:48
So I have to ask your question when?
这很难判断。
35:52
It's very hard to judge.
我真的不知道。
35:53
I don't really know.
我无法给出具体时间。
35:54
I can't put a time on it.
极小极大。
35:55
Min max.
抱歉，请再说一遍？
35:56
Pardon?
你说什么？
35:57
A min max.
A min max.
35:58
It's very hard to say.
很难说。
36:00
I don't know.
我不知道。
36:01
Okay.
好吧。
36:02
I don't know.
我不知道。
36:03
But it is close enough that we should be doing absolutely everything our power to prioritize
但这已经足够接近，我们必须竭尽全力优先考虑安全，并优先考虑对齐与管控。
36:06
safety and to prioritize alignment and containment.
我尊重你使命宣言中的这一部分。
36:10
And I respect that part of your mission statement.
我尊重你们使命宣言中的那部分。
36:15
And I want to get into that a little bit is the trades that you talked about in the
我想稍微深入探讨一下你在《下一波浪潮》中提到的那些行业。
36:21
coming wave.
但在此之前，你曾主导过一场讨论，即认为有意识的AI是一种幻觉。
36:23
But before that, there's a conversation you've led that, you know, the perception of conscious
我想区分一下有感知的AI和有意识的AI。
36:29
AI is an illusion.
哦，好的。
36:32
And I want to distinguish between sentient AI and conscious AI.
你是否将两者区分开来——即AI能否拥有感觉、情感和情绪，
36:37
Oh, okay.
与具备意识并能反思自身思想之间的区别？
36:40
Do you distinguish between the two where AI can have sensations and feelings and emotions
你是否区分这两种情况：一种是AI能拥有感知、感受和情绪，
36:47
versus being conscious and reflective of its own thoughts?
另一种是AI能意识到并反思自己的思维？
36:52
Yeah.
是的。
36:53
Again, this gets into the definitions.
这又涉及到定义的问题。
36:55
So I think an AI will be able to have experiences.
所以我认为人工智能将能够拥有体验。
37:00
But I don't think it will have feelings in the way that we have feelings.
但我不认为它会像我们一样拥有情感。
37:03
I think feelings and the kind of sentience that you referred to is something that is like
我认为情感以及你提到的感知能力，是某种
37:11
specific to biological species.
特定于生物物种的东西。
37:13
But you can imagine coding that in.
但你可以想象将其编码进去。
37:15
You can an optimization function that is that can relate to emotional states percept.
你可以设计一个优化函数，使其能够关联到情绪状态的感知。
37:23
You know, can you imagine that?
你知道吗，你能想象吗？
37:25
You could code in something like that, but it would be no different to the way that
你可以用类似的方式编写代码，但这与
37:31
we write models to simulate the generation of knowledge.
我们编写模型来模拟知识生成的过程并无不同。
37:35
Like the model has no experience or awareness of what it is like to see red, it can only
就像模型没有体验或意识到“看到红色”是什么感觉，它只能
37:42
describe that red by generating tokens, according to its predictive nature, right?
根据其预测性质，通过生成标记来描述红色，对吧？
37:49
Whereas you have a qualia, you have an essence, you have an instinct for the idea of red
而你有“感质”，你有本质，你有对红色概念的直觉，
37:53
based on all of your experience, because your experience is generated through this biological
这基于你所有的经验，因为你的经验是通过这种生物性的
37:57
interactive with smell and sound and touch and a sense that you've evolved over time.
与嗅觉、听觉、触觉以及你随时间进化而来的感知的互动所产生的。
38:03
So you certainly could engineer a model to imitate the hallmarks of consciousness or
因此你当然可以设计一个模型来模仿意识、
38:10
of sentience or of experience.
知觉或体验的显著特征。
38:11
And that was sort of what I was trying to problematize in the paper, which is that at some point
而这正是我在论文中试图揭示的问题——在某个节点上，
38:15
it will be kind of indistinguishable.
它将变得几乎难以区分。
38:17
And that's actually quite problematic because it won't actually have an underlying suffering.
这实际上相当棘手，因为它并不会真正拥有内在的痛苦。
38:23
It's not going to, you know, feel the pain of being denied access to training data or
它不会，你知道，感受到被拒绝访问训练数据、
38:28
compute or to conversation with somebody else.
计算资源或与他人对话的痛楚。
38:31
But we might as our empathy circuits and humans just going to over drive, we're going
但我们人类，由于共情回路过度运转，可能会——
38:37
to activate on that.
针对这一点采取行动。
38:38
We're going to activate on that hardcore.
我们将全力针对这一点采取行动。
38:39
And that's going to be a big problem because people are already starting to advocate for
而这将成为一个大问题，因为人们已经开始倡导
38:43
model rights and model welfare and the potential future, you know, harm that might come to a model
模型权利和模型福利，以及未来可能对
38:48
that's conscious.
具有意识的模型造成的潜在伤害。
38:50
You know, Ilia recently started speaking about what he's doing at safe super intelligence.
你知道，伊利亚最近开始谈论他在安全超级智能领域的工作。
38:59
And I think one of the points he made is emotions are in humans a key element of decision-making.
我认为他提出的一个观点是，情感在人类中是决策的关键要素。
39:09
And curious if AIs that have at least simulated emotions are going to be able to be better,
我很好奇，至少具备模拟情感的AI是否能够表现得更好。
39:17
you know, ASIs than those that don't.
你知道，那些拥有ASI的系统比没有的更强。
39:20
But yeah, I mean, again, I worry that this is too much of an anthropomorphism.
但话说回来，我担心这又太拟人化了。
39:24
We already have emotions in the prompt.
提示词里已经包含了情感。
39:27
We have it in the system prompt.
系统提示词里也有。
39:28
We have it in, you know, the constitution, however you want to design your architecture.
还有宪法里，或者你设计架构时想放进去的地方。
39:32
We're, these are not rational beings.
我们面对的不是理性存在。
39:35
They get moved around and it does feel like they have, they've got arbitrary preferences
它们被随意摆布，确实感觉像有任性的偏好，
39:39
because they're stylistically trying to interpret the behaviors that we've plugged
因为它们只是在风格化地模仿我们植入的行为。
39:43
into the, into the prompt, right?
进入提示，对吧？
39:46
So, you know, it's true that we could add.
所以，你知道，我们确实可以添加。
39:50
You could engineer specific empathy circuits or mirror neural on circuits or like a classic
你可以设计特定的共情回路或镜像神经回路，或者像经典的——
39:57
one is motivation or will, right?
动机或意志，对吧？
39:59
At the moment, the, you know, these are like next token likelihood predictive machines.
目前，你知道，这些就像是下一个词元概率预测机器。
40:05
They're really trying to optimize for a single thing, which token should appear next.
它们实际上只优化一件事：下一个应该出现哪个词元。
40:08
There isn't like a higher order predictive function happening, right?
并没有更高层次的预测功能在运作，对吧？
40:13
Whereas humans obviously have multiple conflicting often drives motivations, which, you know,
而人类显然拥有多种常常相互冲突的驱动力和动机，你知道——
40:20
sometimes run together and sometimes pull apart.
时而交织，时而分离。
40:24
And it's the confluence of those things interacting with one another which produces the human
正是这些因素的相互作用，汇聚成了人类境况，以及社会互动。
40:28
condition plus the social, you know, interaction too.
这些模型并不具备这一点。
40:31
These models don't have that.
你可以通过工程手段赋予它意志或偏好，但那并非自然涌现之物。
40:32
You could engineer it to have a will or a preference, but that would be not something
那是我们人为植入的，且必须极其谨慎地对待。
40:38
that is emergent.
我很欣赏你将这种人文视角引入讨论。
40:39
That would be something that we engineer in and we should do that very carefully.
那将是我们需要精心设计的东西，而且必须非常谨慎地去做。
40:43
I do love that you bring this humanistic side to the equation, right?
我很欣赏你把这种人文关怀带入讨论中。
40:48
I mean, in addition to being a technologist, your background is one that is prohuman at
我的意思是，除了作为一名技术专家，你的背景从一开始就是支持人类的。
40:55
the beginning.
这是一个有趣的文化辩论。
40:56
And it's interesting cultural debate.
我认为我们即将进入那种支持人工智能与支持人类之间的对立。
40:58
I think we're about to enter into those that are sort of pro AI versus pro human.
埃隆和拉里·佩奇之间那场著名的对话：你是个投机分子吗？
41:05
That famous conversation between Elon and Larry Page about, are you a specious?
因为你支持人工智能胜过人类。
41:10
Because you're in favor of AI over humans.
我的意思是，看吧，那将成为一条分界线。
41:14
I mean, look, that's going to be a dividing line.
有些人——我不太确定埃隆在这场辩论中站在哪一边。
41:17
There are some people and I'm not quite sure which side of the debate Elon's on these
有些人——我不太确定埃隆在这场辩论中站在哪一边——
41:20
days.
日子。
41:21
I've certainly heard him save some pretty post human, transhumanist things lately.
我最近确实听到他说过一些相当后人类、超人类主义的东西。
41:26
And I think that we're going to have to make some tough decisions in the next five to
而且我认为，在未来五到十年内，我们将不得不做出一些艰难的决定。
41:30
10 years.
我之所以回避关于超级智能时间线的问题，是因为——
41:31
I mean, the reason I dodged the question on the timeline for superintelligence is because,
你知道，我认为无论是一年、十年还是二十年，这都不重要——
41:35
you know, I think that it doesn't matter whether it's one year or 10 or 20 years, it's super
极其紧迫的是，我们现在就必须明确：我们要构建什么样的超级智能？
41:40
urgent that right now we have to declare what kind of superintelligence are we going to
以及我们是否真的要容忍创造出一个我们无法证明能对齐的实体？
41:45
build and are we actually going to countenance creating some entity which we provably can't align
构建并——我们真的打算容忍创建一个我们可证明无法对齐的实体吗？
41:52
and we provably can't contain and which by design exceeds human performance at all tasks
而我们确实无法控制它，且其设计初衷就是在所有任务上超越人类表现。
41:58
and human understanding and understanding, like how do you control something that you
人类的理解与认知，比如你如何控制一个你并不理解的东西？
42:01
don't understand?
如果可以的话，我想顺着拟人化的思路谈谈——你可能记得道格拉斯·亚当斯的书《宇宙尽头的餐馆》，其中有一个场景：一头牛被设计成主动邀请餐厅顾客吃掉自己，因为这能让顾客更自在，而牛也不介意，它已被优化到渴望被顾客吃掉，但许多读者对那一幕感到恐惧。
42:02
I'd like to, if I may pull on the anthropomorphization thread of it, you may remember Douglas Adams
如果允许我顺着拟人化这条线索说下去，你可能还记得道格拉斯·亚当斯的
42:11
book, the restaurant at the end of the universe, there's a scene where there's a cow that's
书《宇宙尽头的餐馆》，里面有个场景：一头牛被
42:15
been engineered to invite restaurant patrons to eat it because makes them feel more comfortable
设计成主动邀请餐厅顾客吃掉自己，因为这能让顾客更自在，
42:22
and the cow doesn't mind, the cow's been optimized to want to be eaten by the patrons,
而牛也不介意——它已被优化到渴望被顾客吃掉，
42:27
but many readers horrified at that scene.
但许多读者对那个场景感到震惊。
42:30
Put that in a box for a moment.
先把那个问题暂时搁置一下。
42:33
Microsoft has a history of anthropomorphizing AI assistance, co-pilots, going back probably
微软在将AI助手、副驾驶拟人化方面有着悠久的历史，可能可以追溯到微软鲍勃之前的例子，比如罗弗狗，然后是微软Office中的剪贴画助手Clippy，再到最近那些更模糊的云状虚拟形象。
42:39
there's an example prior to Microsoft Bob and the Rover dog and then clip it, clippy
一方面不希望过度拟人化智能体，另一方面又要面对一个可以说在拟人化智能体方面一直走在前列的机构，你如何看待这种调和？
42:46
in Microsoft office and then more recently, more sort of amorphous cloud shaped avatars.
我认为整个设计领域始终以人类境况作为参照。
42:54
How do you think about reconciling on the one hand the desire not to overly anthropomorphize
你如何看待这种调和：一方面希望不过度拟人化
43:00
agents on the other hand with an institution that has arguably been in the vanguard of
智能体，另一方面又面对一个可以说一直处于前沿的机构？
43:07
anthropomorphizing agents?
将代理拟人化？
43:08
I think the entire field of design has always used the human condition as its reference
我认为整个设计领域始终以人类境况为参照。
43:15
point.
点。
43:16
Skeomorphic design was the backbone of the GUI from file affixes to calendars and everything
拟物化设计曾是图形用户界面的核心，从文件后缀到日历，乃至其间的一切。
43:22
in between.
我们仍能在那些自诩现代的老式界面中看到它的痕迹。
43:23
We still have the remnants of that in our old school interfaces which we feel that are modern
这是我们文化中不可避免的一部分，而我们终将超越它们。
43:28
and stuff.
我们会设计出更简洁、更优秀、更高效的用户界面。
43:29
That's an inevitable part of our culture and we just grow out of them.
我并非天生反对拟人化设计。
43:33
We figure out cleaner, better, more effective user interfaces.
我们设计出更简洁、更优质、更高效的用户界面。
43:38
I'm not against anthropomorphism by default.
我并非一概反对拟人化。
43:42
I think we want things to feel ergonomic.
我认为我们希望事物具有人体工学的舒适感。
43:46
The chair fits.
这把椅子很合身。
43:47
The language model speaks my tone.
语言模型能说出我的语气。
43:50
It has a fluency that makes sense to me.
它的流畅性让我觉得合理。
43:52
It has a cultural awareness that resonates with my history and my nation and so on and
它具备文化意识，能与我的历史、我的国家等产生共鸣。
43:58
I think that is an inherent part of design today as creators of things.
我认为作为事物的创造者，这已是当今设计中固有的一部分。
44:04
We are now engineering personalities and culture and values, not just pixels and software.
我们现在不仅是在设计像素和软件，更是在塑造个性、文化与价值观。
44:16
There's a line creating something which is indistinguishable from a human has a lot
创造与人类难以区分的事物，这条界限意义深远。
44:22
of other risks and complications.
其他风险和并发症。
44:25
That makes the immersion into the simulation even more kind of dangerous and more likely.
这使得沉浸于模拟变得更加危险且更易发生。
44:34
I think I don't have a problem with entities, avatars or voices or whatever that are clearly
我认为，对于明显独立、不试图模仿、始终披露自身本质且存在明确界限的实体、化身或声音等，我没有意见。
44:41
distinct and separate and not trying to imitate and always disclose and have that they are
这似乎是安全中自然且必要的一部分。
44:46
in AI essentially and that there are boundaries around them.
如果我没理解错的话，你是在说拟人化一方面是新式的拟物化，另一方面则要保持清晰，甚至可能——
44:49
That seems like a natural and necessary part of safety.
这似乎是安全中自然而必要的一部分。
44:53
What I think I hear you saying correct me if I'm mistaken is anthropomorphization is
如果我没理解错的话，你似乎在说——请纠正我——拟人化
44:57
the new skewomorphism on the one hand but on the other hand maintaining clean, maybe even
一方面是新式的拟物化，另一方面则要保持简洁，甚至可能
45:03
legal boundaries between human intelligence and artificial intelligence, do you think
人类智能与人工智能之间的法律界限，你怎么看？
45:09
do you see a future where AI has achieved some sort of legal personhood or is that for
你是否预见未来人工智能会获得某种法律人格，还是说——
45:15
Bowton is that never going to happen to you see a future where humans are allowed to
鲍顿，你认为这永远不会发生？你是否预见人类被允许
45:19
merge with the AI's Kurzweil style friend of the pod or is that also not on the table
以库兹韦尔式的“意识上传”方式与人工智能融合，还是说这在你看来也不在考虑范围内？
45:23
in your mind?
是的，我认为人工智能的法律人格完全不在考虑范围内。
45:24
Yeah, I mean, I think AI legal personhood is extremely not on the table.
我不认为我们这个物种能存活下去。
45:30
I don't think our species survives.
如果我们与一个成本仅为我们零头的物种共享法律人格和权利的话。
45:34
If we have legal personhood and rights alongside a species that costs a fraction of us that
如果我们赋予一个成本仅为我们零头的物种法律人格与权利
45:44
can be replicated and reproduced at infinite scale relative to us, that has perfect memory
可以相对于我们无限规模地复制和再生产，拥有完美的记忆，
45:51
that can just like paralyze its own computation, I mean, these are so antithetical to the friction
能够像瘫痪自身计算一样暂停运作，我的意思是，这些特性与身为生物物种的我们人类所面临的摩擦截然相反，
45:57
of being a biological species are us humans that there would just be an inherent competition
以至于会存在一种固有的资源竞争，
46:04
for resources and until it was provable, until it was provable that those things would
除非能够证明，除非能够证明这些事物
46:09
be aligned to our values and to our ongoing existence as a species and could be contained
会与我们的价值观以及我们作为物种的持续存在保持一致，并且能够被数学上、可证明地控制，
46:15
mathematically, provably, which is a super high bar.
而这门槛极高。
46:20
I don't see that we should be considering giving a bright line of legal right in the
我不认为我们应该考虑在法律上给予其明确的权利界限。
46:24
sound.
声音。
46:25
I really think it's a bright line.
我确实认为这是一条清晰的分界线。
46:26
I think it's very dangerous.
我认为这非常危险。
46:27
There's a separate question which has to do with liability because they are going to
还有一个与责任相关的独立问题，因为它们将拥有越来越高的自主性——明确地说，我也是加速主义者，我想制造这些东西。
46:32
have increasing autonomy, like to be clear, I'm also an accelerationist, I want to make
这之间会存在一种张力，但张力是合理的。
46:37
these things.
人们总说，张力是合理的。
46:38
They're going to be a tension there, but tension is rational.
如果你看不到这种张力，那你显然错过了这场辩论中最重要的部分。
46:42
People always say that, tension is rational.
人们常说，紧张是理性的。
46:44
If you don't see the tension, you're definitely missing the most of the debate is obviously
如果你看不到紧张，那你肯定错过了辩论的大部分内容，这很明显。
46:49
very complex.
非常复杂。
46:50
Like the more we talk about the complexity and hold it in tension, that's when you start
就像我们越是谈论这种复杂性并保持其张力，
46:54
to see the wisdom.
智慧便开始显现。
46:55
And there's no way we can leave these things on the table and say no, like we want to
我们绝不能把这些东西搁置一旁，说“不”——我们希望在诊所、学校、工作场所中应用它们，
46:59
have these things in clinic, in school, in workplace, delivering value for us a huge scale,
大规模地为我们创造价值，
47:06
where they have to be boundary and controlled, and that's the kind of, that's the art that
而它们必须受到边界和管控，
47:11
we have to exercise.
这正是我们必须运用的艺术。
47:12
This episode is brought to you by Blitzie, autonomous software development with infinite
本期节目由Blitzie赞助播出，提供无限自主软件开发。
47:17
code context.
代码上下文。
47:19
Blitzie uses thousands of specialized AI agents that think for hours to understand and
Blitzie使用数千个专业AI代理，通过数小时的思考来理解并评估包含数百万行代码的代码库规模。
47:25
to price scale code bases with millions of lines of code.
工程师们每个开发冲刺都从Blitzie平台开始，带入他们的开发需求。
47:29
Engineers start every development sprint with the Blitzie platform, bringing in their
Blitzie平台提供计划，然后为每个任务生成并预编译代码。
47:33
development requirements.
Blitzie自主完成80%或更多的开发工作，同时为完成冲刺所需的人类开发工作最后20%提供指导。
47:35
The Blitzie platform provides a plan, then generates and pre-compiles code for each task.
Blitzie平台提供一个方案，然后为每个任务生成并预编译代码。
47:41
Blitzie delivers 80% or more of the development work autonomously, while providing a guide
Blitzie自主完成80%或更多的开发工作，同时为完成冲刺所需的人类开发工作的最后20%提供指导。
47:47
for the final 20% of human development work required to complete the sprint.
不过，恕我直言，我听到的反对AI人格化的主要理由，似乎与当前人类形态的不足有关。
47:53
Enterprises are achieving a 5X engineering velocity increase when incorporating Blitzie
企业将Blitzie作为预集成开发环境工具，并与所选编程助手配合使用，可在组织中引入AI原生软件开发生命周期，从而实现5倍的工程效率提升。
47:57
as their pre-IDE development tool, pairing it with their coding co-pilot of choice to
准备好将工程效率提升5倍了吗？
48:03
bring an AI-native SDLC into their org.
访问Blitzie.com预约演示，立即开始使用Blitzie进行开发。
48:06
Ready to 5X your engineering velocity?
不过，恕我直言，我听到的反对赋予AI人格的主要理由，似乎与当前人类形态的缺陷有关。
48:09
Visit Blitzie.com to schedule a demo and start building with Blitzie today.
我听到你说，它们会消灭人类。
48:16
It sounds though, if I may, the primary rationale that I'm hearing for why not AI personhood
不过，恕我直言，我听到的反对赋予AI人格的主要理由，似乎与人类现有形态的缺陷有关。
48:23
has to do with the inadequacies of the human form as currently constructed.
规模翻倍。
48:28
I heard you say, well, they'll erase humans.
我听说你说，好吧，他们会抹去人类。
48:30
They're so much smarter.
它们聪明得多。
48:31
They're so much faster.
它们快得多。
48:32
They're so much more clonable than human intelligences.
它们比人类智能更易于克隆。
48:35
If human intelligence were uplifted, maybe with the benefit of AI, if we had uploading
如果人类智能得以提升，或许借助人工智能的助力，如果我们拥有上传类技术或先进的脑机接口，使我们能提升普通人类的智能水平。
48:40
type technologies or BCIs that are advanced that enable us to lift up the average human intelligence.
那么在你看来，这是否为人工智能的人格化打开了一扇门，如果人类能与人工智能在公平的竞争环境中一较高下？
48:47
In your mind, then does that open the door a bit to AI personhood if humans can compete
我不想让70亿人的和平与繁荣陷入竞争。
48:51
on a level playing found ground with AI's?
在与人工智能平等的竞争基础上？
48:54
I don't want to make the competition for the peace and prosperity of the 7 billion people
我不想为70亿人的和平与繁荣制造竞争。
49:00
on the planet, even more chaotic.
在这个星球上，甚至更加混乱。
49:03
If the path over the next century can be proven to be much safer and more peaceful and
如果未来一个世纪的道路能被证明更加安全、和平，
49:09
less disease and sickness, and there is room for this other species, then I'm open mind
疾病和痛苦更少，并且有空间容纳这个其他物种，那么我对此持开放态度，
49:16
to it, including biological hybrids and so on.
包括生物杂交等等。
49:19
I'm not against that on principle.
我并非原则上反对这一点。
49:22
I'm just a speciesist.
我只是一个物种主义者。
49:24
I'm just a humanist.
我只是一个人文主义者。
49:25
I start with, we're here, and it's a moral imperative that we protect the well-being
我的出发点是，我们在这里，保护福祉是一项道德义务。
49:32
of all the existing conscious beings that I know do exist and could suffer tremendously
在所有我已知存在的、可能因这一新事物的引入而遭受巨大痛苦的意识生命体中，
49:37
by the introduction of this new thing.
对吧？
49:39
Right?
当然，尼安德特人以及过去十亿多年间所有先于我们的物种，都曾参与过这场对话。
49:40
Now, of course, the Neanderthals made of that conversation or every species that preceded
我的意思是，许多人认为我们不过是超级智能体的引导程序中的临时过渡物种。
49:46
us over the last billion plus years.
那句经典的话。
49:48
I mean, there are many who argue we're simply interim transitory species in bootloader
我的意思是，有很多人认为我们只是引导程序中的临时过渡物种，
49:55
for the superatologist.
为超级智能体服务。
49:56
That classic phrase.
那句经典的话。
49:57
Yes.
是的。
49:58
I'm totally aware of that, and I'm also someone who thinks on cosmological time too, so I'm
我完全清楚这一点，而且我也是从宇宙时间尺度思考的人，所以我并非天真地只谈论这个世纪，我深知一场巨大的变革正在发生。
50:03
not just naively saying this century, I'm definitely aware that there's a huge transition
事实上，就在近代记忆中也能看到这一点——250年前，人类平均寿命大约只有30岁左右。当然，从某些方面来看，我们是一种被增强的混合生物物种。
50:10
going on.
所以我服用所有这些药物，而且，每个人的肽类物质都令人惊叹，我对此完全接受。
50:11
In fact, you can even see it in recent memory, I mean, 250 years ago, life expectancy
事实上，就在近代记忆中你也能看到，我是说，250年前，预期寿命
50:14
was about 30 years or whatever it was, of course, in some ways, we are an augmented hybrid
大约是30岁左右，当然，在某些方面，我们是一种增强的混合体。
50:20
biological species.
生物物种。
50:21
So I take all these drugs, and I, everyone's peptides are amazing, I'm down for all of
所以我服用所有这些药物，而且我，每个人的肽都太棒了，我全都支持——
50:27
that.
那。
50:28
Let's go.
走吧。
50:29
The genetic program is coming next year.
基因项目明年启动。
50:30
Exactly.
没错。
50:31
Let's go.
走吧。
50:32
I'm down.
我同意。
50:33
I'm down.
我同意。
50:34
But let's not shoot ourselves in the foot.
但别自找麻烦。
50:37
Like I want to make sure that, you know, most of our planet, if not everybody, gets the benefit
就像我希望确保，我们星球上的大多数人，即便不是所有人，都能享受到
50:43
of the peace and prosperity that comes from the technology first.
科技首先带来的和平与繁荣。
50:45
I mean, there is some level of sanity in that argument.
我是说，这种论点有一定程度的合理性。
50:49
If you believe that the AI will ultimately outcompete us and put us into a box of insignificance
如果你相信人工智能最终会超越我们，并将我们置于无足轻重的境地，
50:58
in the long run, I mean, all intelligences, we can see this in nature.
从长远来看，我是说，所有智能体，我们在自然界中都能看到这一点。
51:05
We're innately hierarchical.
我们天生就是等级分明的。
51:08
So far, we have not seen this super collaborative species that will take self-sacrifice in order
到目前为止，我们还没有看到这种超级协作的物种，会为了保全其他物种而自我牺牲。
51:13
to preserve the other species.
为了保护其他物种。
51:15
So there's an inherent hierarchical, there's inherent clash from coming from, you know,
因此存在固有的层级结构，也存在固有的冲突，这源于——你知道的——智能的层级结构，对吧？
51:20
the hierarchical structure of intelligence, right?
所以我并不是说我们不应该探索它，也不是说它不可能发生，但首要标准必须是：先做到有所了解，或许可以尝试一点，但首先要确保不对人类造成伤害。
51:23
So and all I'm saying is not that we shouldn't explore it, not that it couldn't potentially
别搬起石头砸自己的脚，就像你说的，真是的。
51:27
happen, but the bar has to first be, do know, maybe do a little, but do no harm to our
顺便说一句，在这个话题上我百分之百赞同你，我们的立场完全一致。
51:33
species first.
但杰弗里·辛顿却在公开宣称，它会失控，而我们的安全……
51:34
Don't shoot yourselves in the foot, as you said, damn.
别搬起石头砸自己的脚，就像你说的，该死。
51:37
Well, I'm 100% with you on this topic, by the way, could not be more aligned.
嗯，在这个话题上我百分之百支持你，顺便说一句，完全一致。
51:41
But Jeffrey Hinton is out there, telling the world, it's going to run away and our safety
但杰弗里·辛顿在外面告诉全世界，它会失控，而我们的安全——
51:48
valve is giving it a maternal instinct and-
阀门赋予它一种母性本能——
51:51
Which I found an interesting point of view.
我觉得这个观点挺有意思。
51:53
Well, he's more of an intractive-
嗯，他更像一个互动型——
51:54
He's more of an intractive-
他更像一个互动型——
51:55
Oh, yeah, I think you're not attractive.
哦，对，我觉得你没什么吸引力。
51:56
What's the safety valve?
安全阀是什么？
51:57
The tank is-
那个罐子——
51:58
Well, he believes it's uncontainable, and I'm with you.
好吧，他认为这是无法控制的，我同意你的看法。
52:02
I think it's very containable if you don't give it emotional and intentional programming.
我认为，如果不赋予它情感和意图编程，它是非常可控的。
52:07
But he thinks it's uncontainable.
但他认为它不可控。
52:09
He was very pessimistic when he got his Nobel Prize.
他获得诺贝尔奖时非常悲观。
52:11
Now he's more optimistic because he sees a path to programming in maternal instinct,
如今他更乐观了，因为他看到了通过母性本能进行编程的路径——
52:18
which implies that it's like it's dominant to us, but it cares.
这意味着它虽凌驾于我们之上，却会关怀。
52:21
His thesis was, I've seen a situation where vastly and more intelligent entity takes care
他的论点是：我曾见过一种情形，一个远更智慧的实体照料着
52:29
of a younger inept entity and a mother with their screaming child.
一个年幼无能的实体，就像母亲与哭闹的孩子。
52:34
Yeah.
是的。
52:35
So if there's a maternal instinct that we can program into AI, even though we're far less
因此，即便我们能力远不及，若能编程出母性本能赋予AI，它自会照料一切。
52:41
capable, it will take care of it.
有人将此比作AI对齐的“数字催产素计划”。
52:44
It's been compared to the call it the digital oxytocin plan for AI alignment.
我喜欢这个说法。
52:48
I like that.
这个比喻不错。
52:49
That's a good one.
是啊。
52:50
Yeah.
没错。
52:51
Yeah.
嗯，挺酷的。
52:52
I mean, cool.
我是说，酷。
52:53
Yeah.
嗯。
52:54
I mean, it's about as poetic as it gets.
我的意思是，这已经是最富有诗意的了。
52:55
I think I'm going to need something that's got a little bit more, like, for me, let's
我觉得我需要一些更有深度的东西，对我来说，咱们开始吧。
52:59
do it.
嗯。
53:00
Yeah.
我比那要稍微多一点。
53:01
I'm a little more than that.
但你看，安全策略有101种不同的可能性。
53:02
But look, there's 101 different possible strategies for safety.
我们应该探索所有策略，认真对待每一种。
53:05
We should explore all of them, take them all seriously.
我们应该探索所有可能性，认真对待每一种。
53:07
I mean, Jeff is a legend of the field, no question.
我的意思是，杰夫是这个领域的传奇，毫无疑问。
53:10
Like, I just think approach with caution, are you spending a lot of your energy, compute,
我只是觉得要谨慎对待，你是在投入大量精力、算力、
53:17
human power on safety?
人力在安全上吗？
53:18
Yeah.
是的。
53:19
I would say more as much as we should, you know, I'm wrapping my head around it.
我会说比我们该做的更多，你知道，我还在努力理解这件事。
53:27
Is anybody out there?
有人在外面吗？
53:28
I am, I am curious out of all the hyperscalers out there, is there any entity that's spending
我很好奇，在所有超大规模企业中，有没有哪家在你看来
53:36
enough in your mind because everybody's in such a race.
投入得足够多，因为大家都在疯狂竞争。
53:40
It's like more GPUs, more data, more energy.
更多GPU、更多数据、更多能源，情况就是这样。
53:44
It's just like everybody's optimizing for the next benchmark.
每个人都在为下一个基准测试优化。
53:48
I don't see any safety benchmarks.
我没看到任何安全基准测试。
53:51
Are there any safety benchmarks out there?
现在有安全基准测试吗？
53:52
Oh, there are tons of safety benchmarks, and there's at least in my mind, an argument
哦，安全基准测试多得很，而且至少在我看来，存在一种
53:56
for defensive co-scaling, I'd be curious to hear your ideas on that.
关于防御性协同扩展的论点，我很想听听你对此的想法。
54:00
Do you think in the same way that as a city gets larger, the police force gets larger.
你是否认为，就像城市规模扩大时警力也会增加一样——
54:06
Maybe it's not in direct proportion, maybe there's some scaling exponent.
也许不是直接成比例，而是存在某种扩展指数？
54:09
But do you think defensive co-scaling of alignment forces or safety forces, whatever that ends
但你认为，对齐力量或安全力量的防御性协同扩展——无论最终结果如何——是否属于人工智能战略的一部分？
54:15
up, meaning do you think that's part of the strategy for AI line?
我认为这会是一个好方向，多年来我们已多次提出这一建议。
54:19
I think that would be a good, I mean, we've proposed this several times over the years.
事实上，拜登政府时期的白宫自愿承诺意味着，所有人——包括戴米斯、达里奥、山姆以及我们所有人——在疫情期间都在大力推动这一点。
54:23
I mean, the White House voluntary commitments under Biden that mean, in fact, everyone,
虽然它最终被搁置了，但我认为这是一套非常合理的原则。
54:27
I mean, Demis and Dario and Sam and all of us through COVID, we're pushing this pretty
就像根据计算量级进行审计一样，我们需要共同承担一定比例的责任。
54:31
hard.
很难。
54:32
Look, I mean, it got chucked out, but I think it's a very sensible set of principles.
你看，我的意思是，它被否决了，但我认为这是一套非常合理的准则。
54:35
It's like auditing for scale of flops, you know, having some percentage that we all share
这就像按运算量规模进行审计，你知道的，设定一个我们共同分担的百分比。
54:40
of safety investment, flops and headcount, you know, this is the time.
在安全投资、失败案例和人员配置方面，你知道，现在正是时候。
54:44
And I think on the face of it, everyone is open and willing to sharing best practices
我认为从表面上看，每个人都愿意开放分享最佳实践，并在时机成熟时相互披露和协调。
54:51
and disclosing to one another and coordinating when the time comes.
我觉得我们还没到那个阶段，目前仍处于高度竞争模式。
54:54
I think we're still pre that level, so we're in like high for competitive mode at the moment.
但没错，我认为现在确实是进行这些投资的最佳时机。
55:01
But yeah, I think now is really the time to be making those investments.
是否存在某种让我们所有人都吓破胆、却又能阻止一切的事情？
55:06
Is there something that's going to scare the shad of us that stops everybody?
但你知道，我一直在讨论这个问题——是否存在像三里岛事件那样，吓到所有人却未造成伤亡的情况？
55:10
But, you know, is there a three, you know, I was talking about this, is there a three
但是，你知道，有没有一个三——我正说到这个——有没有一个三英里铁轨式的事件，吓坏所有人，但不会害死任何人？
55:13
mile iron like event, scares everybody, but doesn't kill anybody?
你知道，微软曾占据科技行业一半的市值，而微软的计划是规模翻倍。
55:17
Well, I heard it was said specifically, he's hoping for a hundred deaths because that's
嗯，我听说他明确说过，他希望能有一百人死亡，因为在他看来，
55:23
in his mind, the least that would get the attention of the government or the world
这是至少能引起政府或世界关注、
55:27
cause some kind of a solution.
从而促成某种解决方案的方式。
55:29
Dave, continue please.
戴夫，请继续。
55:31
So it's interesting that you say Dario and Sam and you'll, like, you guys obviously
所以你说达里奥和萨姆，还有你们，显然你们之间
55:36
must interact quite a bit.
肯定有不少互动，这很有意思。
55:38
Is Mira part of that gang is Andre part of that gang?
米拉是那个圈子里的吗？安德烈也是吗？
55:43
Are you like, because this is, it's interesting to think about the competition
你们是不是——因为想想竞争关系，这确实挺有意思的。
55:47
heating up like we were just talking about.
正如我们刚才谈到的，热度正在上升。
55:49
And, you know, Dario started from this position of pure safety.
你知道，达里奥是从纯粹安全的角度出发的。
55:53
And I think you'll you did too, but now we're right on the cusp of self improvement.
而且我认为你也是，但现在我们正处在自我提升的临界点。
55:58
And it's really, really clear that there are serious, I wouldn't say fishers, but,
很明显，存在一些严重的——我不会说是“渔夫”——但，
56:05
but the, the companies are now really racing.
但那些公司现在真的在竞相追逐。
56:08
I mean, really racing.
我是说，真的在竞相追逐。
56:09
And, and I know Microsoft, you know, when I wrote my, my second business, my first company
而且我知道微软，当我写我的第二个商业计划、我的第一家公司时，
56:13
I sold next business plan.
我卖掉了下一个商业计划。
56:15
I was writing the first sentence was stay out of Microsoft's way, because, because at the time,
我写的第一句话是“避开微软的锋芒”，因为当时，你知道，微软占据了科技行业一半的市值，而微软的计划是规模翻倍。如今的世界更加平衡，有微软、谷歌和Meta，但在当时，微软势不可挡、占据主导地位。所以只能避其锋芒。微软似乎总能赢，对吧？而且，据我所知，我们正处于自我提升的边缘。
56:20
you know, Microsoft had half the market cap of tech was Microsoft and Microsoft's plan was
如今我们拥有一个更加平衡的世界，有微软、谷歌和Meta，但在当时并非如此。
56:25
to double in size.
如今，微软、谷歌和Meta让世界更加平衡，但在当时，情况并非如此。
56:26
We have a much more balanced world now with Microsoft and Google and Meta, but at the time,
我也完全同意。而且我认为我们的高中并没有为任何人做好准备。
56:30
Microsoft was just unstoppable and dominant.
微软当时势不可挡，占据主导地位。
56:33
And so it was just stay out of the way.
所以最好的做法就是别挡道。
56:35
Microsoft seems to always win, right?
微软似乎总能赢，对吧？
56:37
There's, and, and we are right on the edge of self improvement, as far as I can tell.
在我看来，我们正处于自我提升的边缘。
56:44
So, is it still, you know, let's all get together and have dinner and talk about safety?
那么，现在还是大家聚在一起吃个饭、聊聊安全吗？
56:49
Or is everybody now in full board?
还是说所有人都已经全面投入了？
56:51
No, definitely.
不，绝对没有。
56:51
I think that's definitely there.
我认为这一点肯定存在。
56:53
I think the recursive self improvement piece is probably the threshold moment if it works.
我觉得递归自我改进这一环节，如果真能实现，很可能就是那个临界点。
57:01
And if you think about it, at the moment, there are software engineers who are in the loop,
而且你想想看，目前还有软件工程师在参与其中，
57:06
who are generating post-training data, running ablations on the quality of the data,
他们正在生成训练后的数据，对数据质量进行消融实验，
57:11
running them against benchmarks, generating new data.
用基准测试来验证，再生成新的数据。
57:14
And that's sort of broadly the loop.
大致上这就是一个循环。
57:17
And that's kind of expensive and slow and it takes time and it's not completely closed.
这过程相当昂贵、缓慢且耗时，而且并未完全闭合。
57:23
I think a lot of the labs are racing to sort of close that loop,
我认为许多实验室正在竞相试图闭合这个循环，
57:26
so that various models will act as judges evaluating quality, you know,
让各种模型充当评判者来评估质量，
57:31
generators producing new training data, adversarial model,
生成器产生新的训练数据，对抗模型，
57:35
sort of like reasoning over which data to include and what's higher quality.
类似于推理哪些数据该纳入、哪些质量更高。
57:41
And then obviously that's then being fed back into the post-training process.
然后这些数据显然会被反馈到后训练流程中。
57:46
So, like closing that loop is going to speed up AI development for sure.
所以，闭合这个循环无疑会加速人工智能的发展。
57:51
Some people speculate that that adds, I mean, okay, I think it probably does add more risk,
有些人猜测那会增加风险，我的意思是，好吧，我认为它确实可能增加更多风险，
57:56
but some people speculate that it's a potential pass to a fume, you know,
但也有人猜测这可能是通向某种烟雾的潜在通道，你知道，
58:00
it's an intelligence explosion.
那是一场智能爆炸。
58:01
Yeah.
是的。
58:03
And I definitely think with unbounded compute and without human in the loop or without control,
而且我绝对认为，在计算能力无限且没有人类参与或控制的情况下，
58:10
that does potentially create a lot more risk.
这确实可能带来更多风险。
58:12
But unbounded compute is a big claim.
但无限计算能力是一个很大的论断。
58:14
I mean, that would need a lot of compute.
我的意思是，那需要大量的计算资源。
58:17
So, yeah, we're definitely taking steps towards like more and more risky stuff.
所以，是的，我们确实在逐步尝试越来越冒险的事情。
58:24
I can ask you a really specific question about that because, you know,
关于这一点，我可以问你一个非常具体的问题，因为你知道，
58:26
the year and a half now at Microsoft before true recursive self-improvement,
在真正的递归自我改进到来之前——这已经迫在眉睫——微软在一年半前就开始进行AI辅助芯片设计了。
58:32
which is imminent, there's AI-assisted chip design.
你知道，PyTorch堆栈中的各层非常笨重。
58:36
You know, the layers in the PyTorch stack are very clunky.
但现在，利用AI穿透整个堆栈进行优化变得非常容易，比如构建自己的内核，就能获得两倍、三倍甚至四倍的性能提升。
58:41
But now it's really easy to use the AI to punch through the stack
但显然，OpenAI现在正在致力于开发定制芯片。
58:46
and optimize, you know, build your own kernels, get two, three, four X performance improvement.
通过优化，比如构建自己的内核，就能获得两倍、三倍甚至四倍的性能提升。
58:52
But clearly, OpenAI is now working to build custom chips.
但显然，OpenAI 现在正在研发定制芯片。
58:55
And the TPU 7s just came out.
TPU 7 刚刚发布。
58:58
When you arrived at Microsoft, first of all, was I know there's a lot of quantum chip work going on,
你刚到微软时，首先，我知道有很多量子芯片的工作在进行，
59:02
but was there any work going on similar to the TPU work?
但当时有没有类似 TPU 的工作在推进？
59:05
Yeah, there's also a chip effort.
有的，也有芯片方面的努力。
59:08
And, you know, I think progress has been pretty good.
而且，我认为进展相当不错。
59:10
I mean, I think that, you know, we've got a few different ions in the
我的意思是，我们有几个不同的方向在推进，
59:15
fire that we've sort of talked about publicly yet.
其中一些我们还没有公开讨论过。
59:17
But I think, you know, the chips are going to be an important part of it, for sure.
但我认为，芯片肯定会成为其中的重要组成部分。
59:22
And those are internal efforts.
这些是内部的工作。
59:23
Are those teams under you?
那些团队归你管吗？
59:25
That's part of your...
那是你的一部分……
59:26
No, I mean, they're in the broader company.
不，我的意思是，他们属于整个公司。
59:28
Yeah, okay. Interesting.
嗯，好的，有意思。
59:30
I want to switch subject a little bit and go come to your book.
我想稍微换个话题，聊聊你的书。
59:34
The coming wave, I enjoyed it greatly.
《未来浪潮》，我非常喜欢。
59:37
I listened to it.
我是听的有声书。
59:38
I love the fact that you read it.
我喜欢你读了它。
59:39
Thank you.
谢谢。
59:40
Yeah, I tell my kids I read books.
是啊，我跟孩子们说我读书。
59:41
I know, Dad, you listen to books.
我知道，爸爸，你听书。
59:42
You don't read books anymore.
你不再读书了。
59:44
I want to read what I wrote here because it's important.
我想读我写在这里的内容，因为这很重要。
59:48
So you identify the containment problem as the defining challenge of Artura.
所以你将遏制问题视为阿图拉面临的决定性挑战。
59:54
Warning that as these technologies become cheaper and more accessible,
警告说，随着这些技术变得更便宜、更易获取，
59:58
they will inevitably proliferate, making them nearly impossible to control.
它们必将扩散，使其几乎无法控制。
60:04
This creates a terrifying dilemma,
这造成了一个可怕的困境：
60:07
failing to contain them,
若未能遏制它们，
60:09
forces risk for catastrophe, like engineered pandemics,
就会面临灾难性风险，例如人为制造的流行病，
60:15
and a lot of your concerns were in the biological world.
而你的许多担忧都集中在生物领域。
60:17
And I agree being a biologist and a physician,
作为生物学家和医生，我同意这一点——
60:21
or potentially democratic collapse with deep fakes and all of that.
或者可能因深度伪造等技术导致民主崩溃。
60:25
But the extreme surveillance required to enforce containment
但实施遏制所需的极端监控，
60:30
could lead to a totalitarian dystopia.
可能导致极权主义反乌托邦。
60:33
So you say we need to navigate this narrow path between chaos and tyranny.
所以你说我们需要在混乱与暴政之间走这条狭窄的道路。
60:39
And that is a very fine line to navigate.
而这是一条非常难以把握的界限。
60:44
So you propose a strategy of containment.
因此你提出了一种遏制策略。
60:46
This includes technical safety measures, strict global regulations,
这包括技术安全措施、严格的全球监管、
60:50
choke points on hardware supply, international treaties.
硬件供应链的瓶颈控制、国际条约。
60:56
How are we doing on that?
我们在这方面做得如何？
60:57
Yeah, I mean, it's kind of important to just take a step back and distinguish
是的，我的意思是，退一步并加以区分是很重要的。
61:02
between alignment and containment.
在对齐与控制之间。
61:05
The project of safety requires that we get both right.
安全工程要求我们两者皆需把握得当。
61:09
And I actually think we have to get containment right before we get alignment right.
而我实际上认为，我们必须先做好控制，才能实现对齐。
61:14
Alignment is the kind of like maternal instinct thing.
对齐有点像母性本能那样的事——
61:17
Does it share our values?
它是否认同我们的价值观？
61:18
Is it going to care about us?
它会在意我们吗？
61:20
Is it going to be nice to us?
它会善待我们吗？
61:21
Containment is, can we formally limit and put boundaries around its agency?
控制则是——我们能否从形式上限制并划定其行动边界？
61:27
And are we correct everybody?
我们每个人说得都对吗？
61:29
Not just for ourselves, for everybody, yeah.
不仅仅是为我们自己，而是为所有人，是的。
61:31
Yeah, I mean, I think that is part of the challenge.
是的，我认为这正是挑战的一部分。
61:33
Is that like one bad actor with something that is really this powerful in a decade or two
就像是一个坏分子，手握如此强大的力量，在一二十年或几十年内，真的可能动摇整个系统的稳定。
61:40
decades or something, you know, really could destabilize the rest of the system.
所以，你知道，这个系统本身就是人类。
61:43
And so, you know, just the system being human.
全球人类系统，没错。
61:46
Global humanity system, yeah.
正如你所说，当一切变得高度数字化时——
61:47
Just as you said, like as everything becomes hyper digitized,
正如你所说，随着一切变得高度数字化——
61:51
the verse does become the metaverse, even though that kind of like went in and out of fashion
诗句确实成为了元宇宙，尽管它曾迅速流行又过时，但我认为这仍是一种恰当的框架。
61:56
very quickly, it's still, I think the right frame in a way.
因为一切将主要变得数字化、超连接、即时且实时。
62:00
Because everything is going to become primarily digitized and hyper connected and instant
因此，一对多的传播效应突然被大幅放大。
62:04
and real time.
显然，我们在社交媒体上已看到这一点，但如今我设想，被广播的不仅仅是文字，
62:06
And so, the one to many effect is suddenly massively amplified.
更是实际行动。
62:10
I mean, obviously we see on social media, but now I imagine that it's not just
我的意思是，我们在社交媒体上已经看到了这一点，但现在我想这已经不仅限于此了。
62:14
words that are being broadcast.
正在被广播的词语。
62:16
It's actually actions.
实际上，那是行动。
62:18
It's agents are capable of, you know, you know, breaking into systems or, you know, sort of.
其代理能够，你知道的，侵入系统，或者，嗯，类似这样。
62:25
And they're resident in humanoid robots at a billion on the planet.
它们存在于地球上数以亿计的人形机器人中。
62:28
And that too, yeah, it's both atoms and and and bits.
而且，没错，这既涉及原子，也涉及比特。
62:32
So, um, equilibrium requires that there is a type of surveillance that we don't really have
所以，嗯，平衡需要一种当今世界尚未真正拥有的监控方式。
62:40
in the world today.
我的意思是，物理层面显然没有。
62:41
I mean, we certainly don't have it physically.
实际上，网络监控程度惊人，你知道，远超人们的预期。
62:44
The web is actually remarkably surveilled, I think, surprisingly, you know, more than I think
网络实际上受到惊人的监控，我认为，令人意外地，你知道，比我想象中
62:49
people would expect.
人们预期的还要多。
62:52
And some form of that is necessary to create peace.
某种形式的这种力量对于创造和平是必要的。
62:56
Just as we centralised power and taxation or sort of military force and taxation
正如我们在三四百年前将权力、税收或军事力量集中于政府那样。
63:02
around governments, you know, three or four, five hundred years ago.
而这实际上一直是进步的驱动力。
63:05
And that's been the driving force of progress, actually.
这种秩序释放了科学、技术与稳定。
63:09
That order unleashed science and technology and stability and stability.
是的。
63:14
Yeah.
所以问题在于，现代如何以一种既非极权、又不至于陷入自由意志主义灾难的方式，来施加稳定？
63:15
So the question is like, what is the modern form of imposition of stability in a way that isn't
所以问题在于，现代形式的稳定强加方式是什么，既不是极权主义，
63:21
totalitarian but also doesn't relinquish it to a libertarian catastrophe?
也不会放任其走向自由意志主义的灾难？
63:26
I think it's naive to think that somehow, you know, the best defense against the gun is a gun.
我认为，认为某种方式下，你知道，对抗枪支的最佳防御就是枪支，这种想法太天真了。我的意思是，那种认为我们每个人都会拥有自己的AI，并且这会产生一种稳定的平衡，所有AI都会互相抵消的想法，根本不会发生。我内心有一部分希望出现一个超级智能，像“至尊魔戒”那样统治一切，并提供——你知道，我并不担心自己如何表述。我担心的是你的未来。
63:32
I mean, just sort of the idea that somehow we're all going to have our own AIs and that's
我的意思是，那种认为我们每个人都会拥有自己的AI，
63:36
going to create this sort of steady equilibrium that all the AIs are just going to neutralise each other.
并且这会产生一种稳定的平衡，所有AI会相互制衡的想法。
63:42
Like that ain't going to happen.
像那样的事根本不会发生。
63:44
I mean, part of me hopes for a super intelligence that
我的意思是，我内心有一部分希望出现一种超级智能，
63:52
is the ring to rule them all and provides, you know, I'm not worried about
能成为统御万物的至尊魔戒，并提供——你知道的，我并不担心
63:57
how I put it.
我如何表述它。
63:58
I'm worried about your future.
我担心的是你的未来。
63:59
You're hoping for a singleton.
你希望是个独生子。
64:00
Yeah, that's how it's going.
是啊，事情就是这样。
64:03
Part of me is like colour me shocked.
我有点想说“真是让我震惊”。
64:06
Really?
真的吗？
64:06
Yeah.
真的。
64:07
I mean, I imagine that the level of complexity we're
我是说，我觉得我们正在逼近的复杂程度，那种平衡极其困难。
64:14
mounting towards, that balancing act is extraordinarily difficult.
而且，你知道，你不能推一根绳子，但有没有什么机制能把它拉向前呢？
64:19
And, you know, you can't push a string but is there some mechanism to pull it forward?
而且，你知道，你无法推一根绳子，但有没有某种机制能把它拉向前方？
64:28
We should have this space in time.
我们应当在时间上预留这个空间。
64:29
Some would call government at least historically a geographic monopoly on violence.
有人会认为，政府至少在历史上是对暴力的地理垄断。
64:34
And what I think I'm hearing is some sort of monopoly on intelligence or at least
而我听到的似乎是某种对智能的垄断，或至少是
64:39
capabilities exposed to intelligence in order to ring fence to contain AI.
将能力暴露于智能之下，以便围堵并控制人工智能。
64:44
But that's the exact opposite as far as I can tell of what we've seen over the past few years.
但据我所知，这与过去几年我们所看到的恰恰相反。
64:48
Before I used to armchair AI alignment researchers 10, 15 years ago would say
十年前、十五年前，那些纸上谈兵的人工智能对齐研究者曾说过：
64:53
humanity wouldn't be so stupid the moment we have something resembling general intelligence
人类不会愚蠢到在拥有类似通用智能的东西时，
64:57
says to give it terminal access or to give it access to the economy.
就给它终端权限或让它接入经济体系。
65:01
And that's exactly what we did.
这正是我们所做的。
65:03
Oh my god.
哦，我的天哪。
65:03
There was the open AI Google moment.
这就是OpenAI的“谷歌时刻”。
65:07
And yet, but that's concerning, right?
但这令人担忧，对吧？
65:10
So I mean, Google develops all this technology is holding internally until
我的意思是，谷歌开发了所有这些技术，却一直内部保留，
65:15
some actor happens to have initials open AI releases it.
直到某个名字缩写为OpenAI的参与者将其发布。
65:19
And then there's no other option but to follow suit.
然后除了跟进，别无选择。
65:23
I'm less concerned by it.
我倒没那么担心。
65:25
If you look at Anthropic, for example, which prides itself on being a very alignment forward
以Anthropic为例，这家以高度注重对齐性为傲的组织，发布了模型控制协议，该协议目前至少已成为模型与环境交互的标准。许多AI研究者曾明确表示，在通用智能出现之前，我们并不想这样做。所以我很好奇。我的意思是，在你看来，鉴于经济层面存在各种压力——包括现代图灵测试——都在推动智能体与整个世界互动，并采取与限制完全相反的做法，那我们为何现在要开始限制它们？
65:30
organization, alignment Anthropic released the model control protocol, which is now the standard
组织方面，对齐公司Anthropic发布了模型控制协议，这目前已成为标准，
65:36
at least for the moment for models to interact with the environment.
至少暂时是模型与环境交互的标准。
65:40
What many AI researchers said exactly, we did not want to do prior to general intelligence.
许多AI研究者曾明确表示，我们在通用智能出现之前并不想这样做。
65:44
So I'm curious.
所以我很好奇。
65:46
I mean, in your mind, given that the economy, there's every economic pressure,
我的意思是，在你看来，考虑到经济上存在各种压力——
65:52
including modern touring test, to empower agents to interact with the entire world and to do the
包括现代图灵测试——都在推动智能体与整个世界互动，并采取与限制完全相反的行动，
65:57
exact opposite of containment, why would we start containing them now?
我们为什么现在要开始限制它们？
66:00
And containment, it's not that binary, right?
关于“管控”，它并非那么非黑即白，对吧？
66:03
I mean, we contain things all the time.
我的意思是，我们无时无刻不在进行管控。
66:06
We have powerful forces in the engine in your car that is contained and broadly aligned.
比如汽车引擎中蕴含的强大动力，就被妥善管控并大致协调一致。
66:12
Right? And there is an entire regulatory apparatus around that
对吧？而且围绕这一点，还有一整套监管机制——
66:15
from seatbelts to vehicle admissions to driving to street lighting to driver
从安全带、车辆排放、驾驶规范，到街道照明、驾驶员资质，
66:19
head, to freeway speeds.
再到高速公路限速。
66:22
I mean, that's healthy functional regulation, enabling us to collectively interact with each other.
我认为，这是健康且有效的监管，使我们能够彼此协作互动。
66:29
Now, obviously, it's multiple orders of magnitude more complex because these things are not cars
当然，这显然比汽车复杂得多，因为人工智能系统并非汽车。
66:35
there, you know, sort of digital people, but that doesn't mean to say that we shouldn't be striving
你看，虽然存在某种数字人类，但这并不意味着我们不应努力
66:39
to limit their boundaries. And nor does it mean that we have to centralize, by the way.
去限制它们的边界。而且，这也不意味着我们必须集中化，顺便一提。
66:44
The answer isn't that we have a totalitarian state of intelligence.
答案并非是要建立一个极权主义的智能国家。
66:47
Peter wants a single term.
彼得想要一个单一的术语。
66:49
No, I think it's just instinctively, it can be easy to go there when you kind of start to
不，我认为这只是本能反应，当你开始深入思考时，很容易走向那个方向。
66:55
think it through. It's like, obviously, we do have centralized forces, but even in the US,
就像，显然我们确实有集中化的力量，但即便在美国，
67:00
we have, you know, military, we have divisions of the army, we have divisions of the police force,
我们有军队，有陆军师，有警察部门，
67:06
they're nested up in different layers. There's checks and balances on the system.
它们嵌套在不同的层级中。系统内部存在制衡机制。
67:10
And that's kind of what we've got to start thinking about designing it.
这正是我们在设计时必须开始思考的方向。
67:12
Analogy, they're driving is a great one. And just to follow through on it,
用驾驶来类比非常贴切。顺着这个思路往下说，
67:17
the complexity difference, very high, right, for AI.
AI的复杂度差异极大，对吧。
67:21
But the timeline also, I mean, driving evolved from
但时间线也完全不同——驾驶从19世纪10年代演变至今，
67:25
what 19, 10 to today, so the laws related, you know, seatbelts came out 80% of the way through
相关法规，比如安全带，是在那段时间线过了80%后才出现的。
67:33
that timeline. So lots and lots of time to iterate here, very little time, and immensely more
所以驾驶有大量时间迭代，而AI时间极短，复杂度却高得惊人。
67:40
complex. So do you have a vision? But I completely agree, we need a framework for containment
那么，你有愿景吗？我完全同意，我们需要一个快速建立“围栏”的框架。
67:46
fast. And do you have a thought on how we're going to do that?
关于如何实现，你有什么想法吗？
67:49
I think that there's also a good commercial incentive to do this, right?
我认为这样做也有很好的商业动机，对吧？
67:52
I think that like that many of the companies know that they that our social license to operate
我认为许多公司都明白，我们的社会运营许可
67:59
requires us to take more accountability for externalities than ever before.
要求我们比以往任何时候都更对外部性负责。
68:04
We're not in developer barren era, we're not in the oil era, we're not in the smoking era,
我们不再处于开发者蛮荒时代，不再处于石油时代，也不再处于烟草时代，
68:10
right? We've learned a lot, not everything, there's still a lot of conflicts,
对吧？我们已经学到了很多，尽管并非全部，仍然存在许多冲突，
68:14
but it really is a little bit different to last time around. And I think that's one reason
但这次确实与以往有所不同。我认为这是
68:20
to be a bit more optimistic, plus there's the commercial incentive, the commercial incentive,
让人稍微乐观一点的原因之一，再加上商业动机、商业动机，
68:24
and the kind of externalities shift. So if Eric Schmidt is right and something either
以及外部性的转变。所以，如果埃里克·施密特是对的，那么要么……
68:31
radiological or biological happens, and there's 100 deaths, and then the phone starts ringing
一旦发生放射或生物事件，导致100人死亡，电话便会响起。所有人立刻前往白宫。那么，首先，你希望接到那个电话吗？接听并应对它，是否是你人生规划的一部分？此外，在应对过程中，你信任社区中的哪些人？我认为，在未来20年内，终将有一个时刻，让地球上所有人——包括中国及其他所有重要大国——都完全认同在安全、控制与对齐方面展开合作。这是出于自我保存的完全理性之举。这些极为强大的系统，对使用模型的恶意行为者构成的威胁，与对其受害者构成的威胁同样巨大。
68:37
everyone comes to the White House right now. Well, first of all, do you want that call? Is that part
现在所有人都来到了白宫。那么首先，你想接那个电话吗？这是你人生计划的一部分吗——接起电话并做出回应？
68:41
of your your life plan to take that call and react to it? And then who else do you trust in the
然后，在这个社区中，你还信任谁能够参与这一回应？
68:46
community to be part of that reaction? Look, I think that there is going to be a time in the next
听着，我认为在接下来的某个时刻——
68:53
20 years where it will make complete sense to everybody on the planet, to Chinese included
未来20年内，地球上每个人，包括中国人，都会完全理解这一点：
69:01
and every other significant power to cooperate on safety, on safety and containment and alignment.
所有其他重要大国都应当合作，确保安全、控制与对齐。
69:09
It is completely rational for self-preservation. These are very powerful systems that
出于自保，这完全是理性的。这些系统极其强大，
69:16
present as much of a threat to the bad actor that is using the model as it does to the victim.
对使用模型的恶意行为者造成的威胁，与对受害者造成的威胁同样巨大。
69:24
And I think that will create an interest in cooperation, which is kind of hard to empathize
我认为这将会激发合作的兴趣，而在当今世界如此两极分化的阶段，这种合作很难让人产生共鸣。但我确实相信它终将到来。
69:36
with at this stage given how polarized the world is. But I do think it's coming.
我的意思是，能够团结全人类的首要因素就是外星人入侵。而那种外星入侵，可能正是一个失控的超级智能的潜在威胁。
69:40
I mean, the number one thing to unify all of humanity is an alien invasion. And that alien
嗯，好的。那我的第一个问题呢？那是你人生使命的一部分吗？
69:49
invasion could be a potential for a rogue superintelligence. Yeah. Okay. What about the first part of my
我的意思是，只有少数人——我在麻省理工或其他地方遇到的很多人都有这样一种想法：总有人在某个地方已经想明白了，政府里一定有人在思考这个问题。
69:55
question? Is that part of your calling in life? I mean, there's only a handful. I think a lot of
但你去过那里，对吧？那里根本没有人。
69:59
people that I meet around MIT or elsewhere are they have this vision that somebody has
你是说，房间里没有成年人？这就是你的意思吗？
70:05
a figured out somewhere. Someone in government somewhere must be thinking about this. But you've
某个地方应该已经有人想明白了。政府里一定有人在思考这个问题。但你去过那里，对吧？
70:10
been there, right? There's no one there. Were the adults in the room? Is that what you're saying?
那里空无一人。你是说，房间里没有成年人吗？
70:15
Yeah, definitely. There's nowhere to go from this room. Did this ask for the smoke-filled
是的，确实如此。这个房间无处可去。这是否指向那个烟雾弥漫的
70:20
backroom where the leads of all the frontier labs are secretly swapping safety tips?
密室，所有前沿实验室的负责人都在那里秘密交流安全技巧？
70:25
Yeah, something like that. I think that in practice, intelligence exists outside of the smokey room.
对，差不多是这样。我认为实际上，智慧存在于那间烟雾缭绕的房间之外。
70:32
I think that the notion that decisions get made in the boardroom or in the White House situation room
我觉得那种决策是在董事会或白宫战情室做出的想法——
70:38
or actually, you mentioned polymarkets and stuff. Intelligence coalesces in these big balls
或者你提到的Polymarket之类的东西——智慧是在这些巨大的
70:48
of iterative interaction. And that's what's propelling the world forward. And so this is where
迭代互动球体中凝聚而成的。而这正是推动世界前进的力量。所以这就是
70:55
the conversation's happening. Like your audience, all the other podcasters, everyone online,
对话发生的地方。就像你的听众、所有其他播客主持人、网上的每个人，
71:00
we're collectively trying to move that knowledge base forward. In November, you announced the
我们都在共同推动那个知识库向前发展。在十一月，你宣布了
71:04
launch of humanist superintelligence and focused on three applications, in particular, medicine
人文主义超级智能的启动，并专注于三个应用领域，特别是医疗、伴侣和清洁能源。我很想深入探讨一下这一点，但我好奇你没有把教育包括在内。我们的观众中有企业家和人工智能开发者，我认为教育和医疗一样，现在都处于待重塑的状态。我完全同意。我不认为我们的高中正在为即将到来的世界培养人才，它们仍然在回顾50年前的旧模式。你认为微软会参与重塑教育吗？我认为这已经在整个行业中发生了。
71:14
and companions and clean energy. I'd love to double-click in that a little bit, but I was
还有伙伴关系与清洁能源。我很想深入探讨这一点，但我好奇你为何没有把教育纳入其中。
71:20
curious that you didn't include education in that space. And we have an audience of entrepreneurs
而我们的听众是创业者。
71:29
and AI builders. And I think education as much as health care is up for grabs right now, education
以及人工智能的构建者。我认为，教育领域和医疗保健一样，目前正处于变革的关口，教育也是如此。我完全同意。而且我认为我们的高中并没有为任何人做好准备，去迎接即将到来的世界。它们仍然在向后看，就像盯着后视镜里50年前的景象。你认为微软会在重塑教育方面发挥作用吗？你知道，我认为这在整个行业已经发生了。我的意思是，博士学历以及能够根据你个性化的学习风格调整课程的能力。目前它还不能做到的是，去演变或策划一个跨越许多许多课时的长期学习计划，但我们离那一步已经不远了。我的意思是，我们刚刚发布了一个功能，它……
71:36
is too. I totally agree. And I don't think our high schools are preparing anybody
这一点也是如此。我完全同意。而且我认为我们的高中并没有为未来五到十年内处于前沿领域的人做好准备。
71:42
for the world that's coming. They're still retrospectively 50 years in looking the rear-view mirror.
对于即将到来的世界，他们仍然在向后看，目光停留在50年前的后视镜里。
71:49
Do you think Microsoft will play in reinventing education?
你认为微软会在重塑教育方面发挥作用吗？
71:54
You know, I think it's already happening across the whole industry. I mean,
你知道，我认为这已经在整个行业中发生了。我的意思是，
71:57
it's never been easier to get access to an expert teacher in your pocket that has essentially a
现在，你比以往任何时候都更容易在口袋里找到一位拥有博士学位的专家教师，它能根据你的个性化学习风格调整课程。目前它还不能做到的是，在多次课程中逐步发展或策划一个长期的学习计划，但我们离这一步已经不远了。我的意思是，就在几个月前，我们推出了一项名为“测验”的功能。在任何话题上——不仅仅是传统的学校教育——它都能为你设置一个迷你课程、一个测验，而且是互动式的、可视化的，你还可以随着时间的推移追踪自己的学习进度。对此我也非常乐观。这是一个巨大的突破。目前我们在播客中经常讨论的一个争议点是……
72:03
PhD and that can adapt the curriculum to your bespoke learning style. The bit that it can't do at
博士课程可以根据你的个性化学习风格调整课程内容。目前它还不能做到的是，
72:09
the moment is to evolve or sort of like curate an extended program of learning over many, many
演变或策划一个跨越许多许多课时的长期学习计划，但我们离那一步已经不远了。
72:16
sessions, but we're like just around the corner from that. I mean, we released a feature just a
我的意思是，我们刚刚发布了一个功能……
72:20
few months ago called quizzes. And so on any topic, not just a traditional school education,
几个月前，它被称为测验。而且涉及任何主题，不仅仅是传统的学校教育，
72:26
it can set you up with a mini curriculum, a quiz, and it's interactive, and it's visual,
它可以为你制定一个迷你课程、一个测验，并且是互动式的、视觉化的，
72:32
and you can sort of track your learning over time. And I'm very optimistic about that too.
你还可以随着时间的推移追踪自己的学习进度。对此我也非常乐观。
72:36
It's a huge unlock. One of the debates we have right now in the podcast and a really regular basis
这是一个巨大的突破。目前我们在播客中经常讨论的一个议题是——
72:42
is do you go to college? Do you go to grad school? I mean, this is the most exciting time to build
你是否在上大学？你是否在读研究生？我的意思是，这是最激动人心的创业时代。我不知道你是否想在那一天继续追随。我不得不不断这样做。对我来说，在校园里其实很棘手，因为我在麻省理工、斯坦福和哈佛任教。而这个机会窗口如此短暂且如此紧迫。现在在人工智能领域，成功的方式非常非常清晰。后通用人工智能时代？我是说，谁能预测呢？没人知道。但就在此时此刻，你看到这些初创公司的估值——比如昨晚，我就不提名字了——高达数十亿美元。我是说，仅仅通过把合适的人聚集在一个房间里，就能获得40亿美元的初始估值。我其实想问问这个，因为你在Inflection的时机选择……
72:49
ever. I don't know if you want to follow on that day. I got to do this constantly. It's really
我甚至不确定你是否想接着那天的话题聊。我不得不一直处理这件事。
72:56
tricky for me on campus because I teach at MIT and Stanford at Harvard. And this window of
这对我来说在校园里非常棘手，因为我在麻省理工、斯坦福和哈佛任教。
73:01
opportunity is so short and so acute. And it's really, really clear how you succeed right now in AI.
而这个机遇窗口如此短暂且紧迫。在当下的人工智能领域，成功的方式非常明确。
73:06
Post-AGI? I mean, who could predict? Nobody knows. But right here right now, you see these
后通用人工智能时代？我是说，谁能预测呢？没人知道。但就在此时此刻，你看到了这些。
73:12
these startup valuations like we're last night, I won't mention it, but billions.
这些初创公司的估值就像昨晚那样——我不点名了，但都是数十亿级别。
73:17
I mean, just yeah, an opening valuation of $4 billion. By collecting just the right group of
我的意思是，仅仅通过聚集一屋子合适的人，开盘估值就达到40亿美元。
73:23
people in the room. I wanted to ask about that actually because your timing on inflection was
我其实想问问这个，因为你在Inflection的时机把握，
73:28
early in hindsight earlier. But now you've got the new wave with Miramarati and Ilya and a
早期看来还算及时，但现在有了米拉马拉蒂、伊利亚和其他几位带来的新浪潮，他们的人工智能项目估值都高达数十亿美元。我曾以为我们设定了估值标准——20人团队、零营收——可我们不过是条小鱼。这差不多是两年半前的事了。就这些吗？天哪，我想是三年了。是啊。你觉得当智能成本变得低到无法计量时，以市值衡量的人力资本价值会呈反比渐近地趋向无穷吗？奇怪的是，确实如此，因为时间压力在作祟，对吧？而且实际上，能从事这项工作的人才仍然相当稀缺，而供给却过剩了。
73:34
couple of others look with AI that all have multi-billion dollar valuations. And I thought we set
还有另外几家做AI的公司，估值都是数十亿美元。
73:39
some standards on valuations, pre-revenue with a 20-person team, but we're just a minnow.
我以为我们对估值设定了标准——20人团队、零营收，但我们只是小虾米。
73:45
It's almost two and a half years ago. Is that all it was? Oh my god. That's three years, I think.
那差不多是两年半前的事了。才这么点时间？天哪，我觉得是三年。
73:49
Yeah. You think as the cost of intelligence becomes too cheap to meter that the value
是啊。你觉得当智能成本变得低到无法计量时，
73:54
ascribed at least in terms of market cap to human capital is sort of inversely asymptotic
至少以市值衡量的人力资本价值，会不会呈现一种反向渐近线趋势？
74:00
going to infinity? Weirdly it is because of the pressure on timing, right? And there's actually
走向无限？奇怪的是，这是因为时间压力，对吧？而且实际上
74:05
still a pretty concentrated pool of people that can do this stuff. And there's like an oversupply
仍然有一批相当集中的能处理这类事情的人。而且存在一种
74:11
of capital that's desperate to get a piece of it. It might not be the smartest capital the
那些急于分一杯羹的资本，或许并非世上最聪明的资本，但它们极其热切。正因如此，我不得不问——毕竟这笔钱在我口袋里烧得发烫。但你知道吗，亚历克斯在麻省理工的大一室友是纳特·弗里德曼。准确说，是大一前的室友。后来纳特离开，成了Safe Super Intelligence的联合创始人。我还没问过他，也不知道你是否问过——但他离开后去了Meta。我敢肯定，其中巨大的吸引力来自算力。没错。而你现在处境何其相似，对吧？你创办了公司，融了十亿或十五亿美元，你可以大展拳脚了。
74:16
world's ever seen, but like it's very eager. And so that's what I have to ask because it's
世界从未见过的过剩供应，但需求却非常迫切。所以我必须问这个问题，因为它
74:23
burning a hole in my pocket. But you know, Alex's freshman roommate at MIT was Nat Friedman.
让我心痒难耐。但你知道吗，亚历克斯在麻省理工的新生室友是纳特·弗里德曼。
74:29
And pre-fresh actually pre-fresh pre-fresh roommate. And so Nat goes off and he ends up at
而且是新生前，实际上新生前的新生前室友。于是纳特离开了，最终成为
74:35
co-founder of Safe Super Intelligence. And I haven't asked him, I don't know if you've asked him yet,
Safe Super Intelligence 的联合创始人。我还没问过他，不知道你是否问过他，
74:41
but he leaves to become the guy at Metta. And I've got to believe a huge part of that attraction
但他离开后成为了 Metta 的人。我敢肯定，其中巨大的吸引力
74:49
is the compute. Yeah. And so here you are, very similar situation, right? You've got your startup,
在于算力。没错。所以你现在的情况非常相似，对吧？你有了自己的初创公司，
74:54
you've got a billion or whatever, a billion and a half that you've raised. You can build it,
你筹集了十亿或更多，十五亿左右。你可以用它来建设，
74:58
you can get your 20,000-invitiate, well wait a minute, here's Microsoft 300 billion of cash flow
你可以获得两万名受邀者，等等，微软有三千亿美元的现金流和庞大的算力。这是否占了很大一部分……是的。我的意思是，更不用说我们为单个研究人员或技术员工支付的薪酬，以及未来两年甚至十年所需的投资规模。我认为，身处大公司内部显然具有结构性优势。而且，要在未来五到十年保持在行业前沿，可能需要数千亿美元。所以，总结一下这个观点，那么……那些目前以两百亿或五百亿美元估值融资的公司……
75:05
and a huge amount of compute. Was that a big part of the... Yeah. I mean, not to mention the
还有大量的算力。这是否占了很大一部分……是的。我的意思是，更不用说我们为单个研究人员或技术人员支付的薪酬，
75:11
prices that we're paying for individual, you know, researchers or members of technical staff and
以及所需的投资规模——不仅是在两年内，而是在十年内。我认为，身处大公司内部显然具有结构性优势。
75:16
like, I mean, just to also just the scale of investment that's required not just in two years,
而且我认为，在未来五到十年内，要保持在最前沿，可能需要数千亿美元。
75:21
but over 10 years. I think it's clearly there's a structural advantage by being inside the big
所以，顺着这个思路，那么……那些目前以200亿或500亿美元估值融资的公司——
75:28
company. And I think it's going to take, you know, hundreds of billions of dollars to keep up
公司。而且我认为，在未来五到十年内，要保持在行业前沿，可能需要数千亿美元的资金投入。所以，顺着这个思路，那些目前以200亿或500亿美元估值融资的公司，我认为仍然是两大基石。至于是否应该上大学？当然应该。要知道，人文教育、从中获得的社交能力、院校提供的资源，以及用三年时间在课程内外自由思考和探索的机会——这本身就是一种巨大的特权。
75:33
at the frontier over the next five to 10 years. So finishing that thought, then you...
所以，接着刚才的思路，那么你……
75:38
the companies that are raising money at a 20 or 50 billion dollar valuation right now,
那些目前以200亿或500亿美元估值融资的公司，
75:44
and no chance. Okay. I'll take that. I wouldn't say that. But I think it depends. I mean,
毫无机会。好吧，我接受这个说法。我不会这么说。但我认为这取决于具体情况。我的意思是，
75:54
there's obviously a near term. If suddenly we do have an intelligence explosion,
显然存在一个短期前景。如果突然真的出现智能爆炸，
75:59
then lots of people didn't get there simultaneously, but then also at the same time,
那么很多人并没有同时达到那个阶段，但与此同时，
76:03
you have to build a product with those things. You have to distribution like all the traditional
你还得用这些技术构建产品。你需要像所有传统机制那样进行分发——
76:06
mechanisms that apply are you going to be able to convert that quickly enough. I mean, you know,
你能否足够快地完成转化？我的意思是，你知道，
76:10
everything goes really kind of weird if that happens in the next five years. It just is unrecognizable.
如果这一切在未来五年内发生，一切都会变得非常奇怪。那将完全无法辨认。
76:17
There's so many emergent factors to play into on another. It's hard to say. And I think that's partly
有太多涌现因素在相互作用。很难说清楚。我认为部分原因在于，
76:25
the ambiguity is what's driving the frothiness of the valuations. Because I think there's people
这种模糊性正是估值泡沫的驱动力。因为我觉得有些人——
76:29
going, well, I don't know. Do I want to be... So what do you call it? Read calls it schmuck insurance.
好吧，我不知道。我想成为……这该怎么说呢？里德称之为“傻瓜保险”。
76:34
Yeah, we had read on the pod here a couple of months ago. He's brilliant. So to that graduating
是的，几个月前里德来过我们的播客。他非常聪明。所以，对于那位即将高中毕业的学生，现在该学些什么呢？毫无疑问，你仍然需要学习两个学科。哲学和计算机科学，我认为在很长一段时间内，仍将是两大基础。
76:43
high school student, what do you study these days? I mean, there's no question that you still have
你应该上大学吗？当然。要知道，人文教育、从中获得的社交能力、大学机构带来的益处——拥有三年时间基本上用来思考和探索，无论是课内还是课外——这是一种巨大的特权。人们不应该放弃它。这是黄金般的机会。所以我总是鼓励大家去上大学。
76:51
to study both disciplines. Like philosophy and computer science is going to, for a long time,
同时研习两门学科。比如哲学与计算机科学，我认为在很长一段时间内，仍将是两大基石。是否应该上大学？当然应该。要知道，人文教育、由此带来的社交性、院校的益处，以及用三年时间在课程内外进行思考与探索的机会——这是巨大的特权。人们不应轻易放弃它，这弥足珍贵。所以我一直鼓励大家去上大学。公共服务领域所承载的地位、声誉与尊重，在里根与萨切尔时代之后已遭贬损。我认为这实为一种悲剧，因为我们比以往任何时候都更需要这种情怀、这种精神以及这些能力。
76:57
remain, I think the two foundations. Should you go to college? Absolutely. Like, you know,
我认为仍然是两大基础。
77:05
human education, the sociality that comes from that, the benefit of the institution, having three
你应该上大学吗？当然。你知道，
77:12
years to basically think and explore, you know, in and out of your curriculum. This is a huge privilege.
人类教育、从中获得的社会性、机构带来的益处，以及拥有三年时间在课程内外思考和探索的机会——这是一种巨大的特权。
77:18
Like people should not be throwing that away. That is golden. So I always encourage people to do that.
像这样的东西，人们不应该丢弃。那是金子般的宝贵。所以我一直鼓励大家这样做。
77:24
Obviously, I did also drop out, but I mean, I still think it was a cool thing to do.
显然，我也确实辍学了，但我的意思是，我仍然觉得那是一件很酷的事。
77:28
Yeah, it was just felt right at the time. But the other thing is go into public service.
是的，当时感觉就是对的。但另一件事是投身公共服务。
77:38
Yeah. I respect that part of what you did in that sequence in your life,
是的。我尊重你人生中那段经历所体现的部分，
77:44
which gave you this very much humanist point of view.
它赋予了你非常人本主义的视角。
77:47
Yeah. And it was really hard and very different. And it didn't, it wasn't instinctively
是的。那真的很难，也非常不同。而且它并非本能地正确。
77:52
right. But I learned a lot. And it was a very influential and important part of my experience,
但我学到了很多。那是我经历中非常有影响力和重要的一部分，
77:57
even those very short, like a couple of years, basically. And I think if you look at the actors
即使时间很短，基本上只有几年。而且我认为，如果你看看如今我们生态系统中的参与者——
78:03
in our ecosystem today, corporations, the academics, the sort of news organizations, now the podcast
企业、学术界、各类新闻机构，还有现在的播客——
78:11
world, it's really our governments that are probably institutionally the weakest and our democratic
世界范围内，真正最薄弱的机构可能正是我们的政府，以及我们的民主进程，但实际上，我们的公共服务体系才是症结所在。这是因为过去五十年间，自里根和撒切尔时代以来，公共服务人员的地位、声誉和尊严一直遭受打击。我认为这实际上是一种悲剧，因为我们比以往任何时候都更需要那种情怀、那种精神以及那些能力。
78:17
process, but actually our civil service. And that's because there's been five decades of
我想我刚刚听到你说的是——如果理解有误请纠正——公共部门和公共服务需要更多智慧。那么政府中的人工智能呢？你认为政府需要什么？特别是代理型人工智能在政府中的应用？当然，所有上述问题都同样适用。
78:23
battering of the status and reputation and respect that goes into being part of the public service,
公共服务所承载的地位、声誉和尊重，在里根和萨彻之后遭到了贬损。
78:31
like post Reagan and Satcher. And I think that's actually a travesty because we actually need
我认为这其实是一种悲哀，因为我们比以往任何时候都更需要那种情怀、那种精神，以及那些能力。
78:36
that sentiment and that spirit and those capabilities more than ever.
到十年后，或者一个《星际迷航》式的经济时代，我们该怎么办？是的。我认为很清楚，大多数
78:40
I think maybe what I just heard you say correct me if I'm wrong again is we need more intelligence
我想我刚刚听到你说的，如果我理解错了请纠正，是我们需要更多智慧
78:47
in the public sector, in public service. What about AI in government? Do you think the government
在公共部门、公共服务中。那么政府中的人工智能呢？你认为政府
78:52
needs and what about agentic AI and the government in particular? For sure, with all the same
需要什么？特别是代理型人工智能在政府中的应用？当然，所有这一切
78:58
caveats that apply, but I mean, you know, I mean, you know, rate of adoption for what is worth
需要注意的几点是，但我的意思是，你知道，我的意思是，你知道，就采纳率而言，
79:01
of co-pilot inside of governments is really high. It's a brilliant job of synthesizing documents
政府内部对 Copilot 的采用率确实很高。它在合成文档、
79:06
and transcribing meetings and summarizing notes and facilitating the discussion and chipping in
转录会议、总结笔记、促进讨论以及
79:12
with actions at the right time. I'm just clearly going to save a lot of time and improve decision
在适当时机提出行动建议方面表现出色。这显然能节省大量时间并改善决策。
79:18
making. So then maybe to tie a nice bow on the discussion, isn't that arguably a form of AI
那么，或许给这场讨论画上一个圆满的句号：这难道不可以说是一种
79:24
containing AI? If AI is infusing the government and AI is infusing the economy and the government
AI 包含 AI 的形式吗？如果 AI 正在渗透政府，AI 正在渗透经济，而政府
79:29
is regulating the economy, isn't this just defensive co-scaling with AI regulating itself?
又在监管经济，这难道不正是 AI 自我监管的一种防御性共同扩展吗？
79:34
Yeah, I mean like everyone is going to use AI all at the same time to pursue, but the agendas
是的，我的意思是，每个人都会同时使用 AI 来追求各自的目标，但议程却各不相同。
79:40
that we all have are going to remain the same. I mean, people who want to start companies,
我们所有人都将保持不变。我的意思是，那些想创业的人，
79:44
people who want to write academic papers, people who want to start, you know, cultural groups
那些想写学术论文的人，那些想创办文化团体和娱乐项目的人，
79:49
and entertainment things, everyone is just going to be empowered like in some way. Their
每个人都会以某种方式被赋能。他们的能力将因这些工具而得到放大。显然，政府也包括在内。
79:54
capability is going to be amplified by having these tools. Obviously, the government included.
很好。穆斯塔法，非常感谢你周五晚上抽出时间。
80:00
Nice. Mustafa, thank you so much for taking the time on a Friday night.
很感激能与你进行这次对话。戴夫、亚历克斯，感谢你们——
80:04
Grateful to have this conversation with you. Dave, Alex, I appreciate you're going to
戴夫，我的最后一个问题。最后一个问题？如果只能问一个，好吧，预测一下。
80:11
final question for me, Dave. Final question? If I have one, all right, prediction.
量子计算目前与LLM AI领域发生的事情毫无关系。这一切都——
80:18
Quantum computing right now is nothing to do with what's going on in LLMAI. It's all
量子计算目前与LLMAI领域发生的事情毫无关系。它还要
80:24
Matt moles on NVIDIA chips and soon to be TPUs and other custom chips. Best guess, six, seven
马特·摩尔专注于英伟达芯片，不久后也将涉足TPU及其他定制芯片。最乐观估计，六到七年后。AI非常擅长编写代码和编译，并能推演量子运算。量子芯片是仍处于边缘地位，还是所有技术都已转向量子领域，让微软能利用其先发优势？是的，我认为它将成为整体格局中的重要组成部分。我觉得我们讨论AI的时间相对不足，这其实是浪潮中未被充分认知的部分。实际上，有点像合成生物学，尤其是在大众讨论中，我认为人们尚未把握这两股浪潮——它们将与AI的崛起同样具有冲击力，并与之同步爆发。
80:32
years from now. The AI is very good at writing code and compiling and can figure out quantum
等很多年。人工智能非常擅长编写代码和编译，并能推算出量子
80:39
operations. Are quantum chips relevant on the sideline still or is everything ported over to quantum
操作。量子芯片是仍然处于边缘地位，还是所有东西都已迁移到量子领域，
80:46
and Microsoft can take advantage of its lead? Yeah, I mean, I think it's going to be a big part
而微软可以利用其领先优势？是的，我认为它将成为
80:51
of the mix. I think it's sort of an under-relativity amount of time we spend talking about AI is kind
整体中重要的一部分。我觉得我们花在讨论人工智能上的时间相对不足。
80:57
of an under-acknowledged part of the wave. Actually, a little bit like synthetic biology, I think that
一个未被充分认知的浪潮组成部分。实际上，有点像合成生物学，我认为
81:02
especially in the general conversation, I think people aren't grasping those two waves, which are
尤其是在大众讨论中，我觉得人们没有把握住这两波浪潮，它们
81:10
going to be just as impactful and crash at the same time that AI is coming into focus.
将与人工智能逐渐成为焦点时同样具有冲击力，并同时席卷而来。
81:17
All right. You heard it here. This is a closing question to appeal maybe to your more accelerationist
好的。你在这里听到了。这是一个收尾问题，或许是为了迎合你更偏向加速主义的一面。观众能做些什么来加速人工智能在科学和工程领域的应用？你认为限制因素是什么？如果我在播客中经常谈论“最内层循环”这个概念——即在计算机科学中，如果你想优化一个程序，你往往会发现循环嵌套循环，而你需要优化最内层的循环以优化整个程序——那么你认为什么是那个最内层循环，或者说那个限制因素？如果观众有足够的能力，他们可以帮助优化什么，以便在未来十年内加速实现一个《星际迷航》式的未来或《星际迷航》式的经济？我们该怎么做？是的。我认为很明显，大部分……
81:23
side. What can the audience do to accelerate AI for science, AI for engineering? What do you view
另一方面，观众可以做些什么来加速人工智能用于科学、人工智能用于工程？您认为
81:30
as the limiting factors? If I often talk on the podcast about this notion of an innermost loop,
限制因素是什么？我经常在播客中谈到“最内层循环”这个概念，
81:36
the idea that in computer science, if you want to optimize a program, you tend to find loops
即在计算机科学中，如果你想优化一个程序，你往往会发现循环
81:41
within loops and you want to optimize the innermost loop in order to optimize the overall program.
嵌套循环，而你需要优化最内层的循环，才能优化整个程序。
81:45
What do you see as the innermost loop, the limiting factor, if you will, that the audience listening
您认为最内层循环是什么，或者说限制因素是什么，以便听众能够理解？
81:52
if they're suitably empowered can help optimize to speedrun maybe a Star Trek future over the next
如果他们获得适当的赋能，就能帮助优化流程，或许能在未来十年内加速实现《星际迷航》式的未来，或构建《星际迷航》式的经济。我们该怎么做？是的。我认为很明显，大部分工作是在现实世界中验证假设。现阶段我们所能做的，就是将越来越多的信息输入自己的大脑，然后与一个随你共同进化的单一模型协同使用——因为它正逐渐成为你的第二大脑。例如，Copilot 在个性化方面已经非常出色。它的大部分回答——你用得越多，这些回答就越能捕捉到你感兴趣的主题，同时它也在逐渐变得更加主动。所以它会温和地提醒你关注那些显然相关的新论文或新文章。
81:59
to 10 years or a Star Trek economy. What do we do? Yeah. I think it's pretty clear that most of
在现实世界中验证假设的工作，到了这个阶段，我们只能不断吸收更多。
82:05
these models are going to speed up the time to generate hypothesis. The slow part is going to be
这些模型将加速生成假设的过程。而真正耗时的部分，是在现实世界中验证这些假设。目前我们能做的，就是不断向自己的大脑输入更多信息，并将其与一个伴随你成长的单一模型协同使用——因为它正逐渐成为你的“第二大脑”。例如，Copilot如今在个性化方面已经做得非常出色。它的大多数回答会随着你的使用频率增加，逐渐捕捉你感兴趣的主题，并且也在温和地变得更加主动。它会适时向你推荐新发表的论文或文章，这些内容显然与你之前讨论过的话题高度契合。所以，这其实是一种相对简化的模式。
82:10
validating hypothesis in the real world. All we can do at this point is just ingest more and
在现实世界中验证假设。
82:17
more information into our own brains and then co-use that with a single model that progresses with
将更多信息注入我们自己的大脑，然后与一个随着你共同进步的单一模型协同使用，因为它正逐渐成为你的“第二大脑”。例如，现在的“副驾驶”在个性化方面已经非常出色。它的大多数回答——你用得越多，这些回答就越能捕捉到你感兴趣的主题，同时它也在逐渐变得更加主动。因此，它会温和地提醒你关注新发表的论文或文章，比如最近哈佛和麻省理工学院联合推出的Lila项目。我觉得这很令人兴奋，人工智能正在朝着这个方向发展。
82:26
you because it's becoming like a second brain. For example, co-pilot is actually really good
目前我们所能做的，就是不断吸收更多信息，
82:31
at personalization now. Most of its answers, so the more you use it, the more those answers
因为它正变得像第二大脑。例如，Copilot现在在个性化方面确实非常出色。
82:37
pick up on themes that you're interested in and it's also gently getting more proactive.
它的大部分回答——你用得越多，这些回答就越能捕捉到你感兴趣的主题，
82:41
So it's kind of nudging you about new papers or new articles that come out that are obviously
同时它也在逐渐变得更加主动。
82:48
in tune with whatever you've been talking about previously. So it's a bit of a simplistic
与你之前谈论的内容保持一致，所以这有点简单化。
82:53
cop-out answer, but just the more you use it, the better it gets, the better it learns you,
这听起来像是一种敷衍的回答，但事实就是：你用得越多，它就越强，它就越了解你，你也变得更好，因为它成了你探索思路的辅助工具。所以，你对观众的建议就是多使用Copilot，这是你能做的最有效的加速器。那么还有其他AI吗？我的意思是，我也在讨论能否构建一个物理系统，让AI能够24小时全天候在黑暗循环中运行实验，从而从自然界挖掘数据。确实有一些公司正在做这件事，比如Lila，它最近从哈佛和麻省理工孵化出来。我觉得这很令人兴奋——AI正在代表我们进行探索，收集数据。是的，Spawn，再次感谢你。
82:57
the better you become because it becomes this sort of aid to your own line of inquiry.
你越擅长，它就越能成为你自身探究路径的辅助工具。
83:01
So that sounds like your advice to the audiences use co-pilot more and that's the single
所以这听起来像是你给观众的建议：多使用Copilot，这是你能用来加速这一过程的最有效的催化剂。
83:06
best accelerants that you can do to speed this up. So any other AI? I mean, I also talk about can
那么还有其他AI吗？我的意思是，我也谈到能否构建一个物理系统，让AI在24小时全封闭黑暗循环中运行实验，从而从自然界中挖掘数据，对吧？
83:12
you build the physical system that is going to enable AI to run the experiments in a 24-7 closed
有一些公司正在这样做，比如最近从哈佛和麻省理工分离出来的Lila。我觉得这很令人兴奋，AI正在代表我们进行探索并收集数据。
83:21
dark cycle to be able to mine nature's for data, right? There are a number of companies that are
是的，Spawn，再次感谢你。
83:27
doing this lila is one recently out of Harvard and MIT. I find that exciting where AI is becoming
所以它会温和地提醒你关于新论文或新文章的出现，这些内容显然……
83:34
and explore on our behalf, gathering that data. Yeah, spawn. Thank you again.
并代表我们进行探索，收集那些数据。是的，启动。再次感谢。
83:44
This has been great. Thanks a lot. It was a really fun conversation.
这真是太棒了。非常感谢。这是一次非常愉快的对话。
83:47
Yeah, really fun. Thanks. Appreciate my friend.
是啊，真的很愉快。谢谢。感激不尽，我的朋友。
83:49
All right, good to see you. Every week, my team and I study the top 10 technology meta trends that
好的，很高兴见到你。每周，我和我的团队都会研究未来十年将改变行业的十大科技宏观趋势。
83:53
will transform industries over the decade ahead. I cover trends ranging from human
我涵盖的趋势包括人类与机器人、人工智能和量子计算，以及交通、能源、长寿等领域。没有废话。
83:58
under robotics, AI and quantum computing to transport energy, longevity and more. There's no fluff.
只有最重要、最相关的内容，那些影响我们生活、公司和职业的关键信息。
84:04
Only the most important stuff that matters, that impacts our lives, our companies, and our careers.
如果你想让我与你分享这些宏观趋势，我每周写两次简报，
84:09
If you want me to share these meta trends with you, I writing newsletter twice a week,
以两分钟阅读时长的邮件形式发送。如果你想发现最前沿的资讯——
84:13
sending it out is a short two-minute read via email. And if you want to discover the most
发送出去是一封仅需两分钟阅读的邮件。如果你想了解全球最具颠覆性公司的CEO们以及正在构建最具颠覆性技术的创业者们——如果你不想了解即将到来的趋势、其重要性以及如何从中获益，那它并不适合你。免费订阅请访问demandus.com/meta-trends，提前十年掌握趋势。好了，现在回到本期节目。
84:17
important meta trends 10 years before anyone else, this reports for you. Readers include founders
重要趋势提前十年洞察，这份报告为你呈现。读者包括全球最具颠覆性公司的创始人、CEO，以及正在构建世界最前沿技术的创业者。若你不想了解未来动向、其重要性及如何从中获益，则不必阅读。免费订阅请访问demandus.com/meta-trends，提前十年掌握趋势。好了，现在回到本期节目。
84:23
and CEOs from the world's most disruptive companies and entrepreneurs building the world's most
最近哈佛和麻省理工推出的Lila项目就在做这件事。
84:28
disruptive tech. It's not for you if you don't want to be informed about what's coming,
还有来自全球最具颠覆性公司的CEO们，以及正在打造世界最前沿颠覆性技术的企业家们。如果你不想了解即将到来的趋势、它为何重要、以及你如何从中受益，那么这个节目不适合你。免费订阅请访问demandus.com/meta-trends，提前十年掌握趋势。好了，现在回到本期节目。
84:32
why it matters, and how you can benefit from it. To subscribe for free, go to demandus.com,
我觉得这很令人兴奋——AI正在变得……
84:38
slash meta trends. To gain access to the trends 10 years before anyone else. All right,
来自全球最具颠覆性公司的CEO们，以及正在构建全球最具影响力企业的创业者们，
84:43
now back to this episode.
探讨其重要性以及你如何从中受益。