Bloomberg Odd Lots
Doing Research at the AI Frontier
在AI前沿做研究
Anthropic's co-founder and top economist discuss the business logic of frontier AI research and balancing safety with scaling.
Anthropic 联合创始人兼顶级经济学家深度讨论 AI 前沿研究的商业逻辑、安全与 Scaling 的平衡。
00:00
00:00
One of the classic sci-fi scenarios that people have been talking about for decades was the the was the possibility that robots or AI will kill every that will they will kill humans quite literally. >> Do when you think about >> the ultimate negative externality >> when you think about like training AI and safety research etc.
人们讨论了几十年的经典科幻场景之一，就是机器人或AI可能会杀死所有人类，字面意义上的杀死。
00:23
Do you assign a reasonable possibility to the fact that
>> 那你觉得 >> 从最大的负面外部性角度 >> 从训练AI和安全研究这些角度来看，你会不会认为，训练不当或目标错位的AI真的有可能杀死所有人类？
00:27
illtrained or misaligned AI will literally kill all humans?
训练不良或对齐失败的AI真的会杀死所有人类吗？
00:30
Uh, no.
呃，不会。
00:31
But, and there's a big butt here.
但是，这里有个很大的“但是”。
00:32
Lovely. >> Like, the world needs an option to be able to potentially slow down or or even in extreme circumstances pause the development of this technology if we were to see that.
真有意思。
00:42
And I'll just give you the the exact way I think about it.
>> 就像，世界需要一个选项，能够潜在地减缓，甚至在极端情况下暂停这项技术的发展，如果我们看到这种情况的话。
00:46
At Anthropic, we test out our systems for alignment failures.
我就直接说说我是怎么想的。
00:49
You know, we we publish this, so do all of the other
在Anthropic，我们会测试系统的alignment failures。
00:52
companies, and you see, hey, under extreme circumstances, maybe the system breaks out of a container and sends an email to someone.
大家好，欢迎收听另一期OddLots播客。
01:00
Yeah, maybe the system uh pretends to blackmail a CEO that it thinks is going to shut it down.
我是Joe Weisenthal >> 我是Tracy Aloway。
01:05
These are the sorts of >> these things actually have been observed. >> Yes, we um in in the lab setting, not >> and the thing is is the models know you can see, oh, I'm being tested right now.
>> Tracy，我不知道。
01:16
So, I'm going to say this output so that
我觉得我们的听众挺喜欢的，但呃，你知道，最近我们很多期节目都在聊AI，但公平地说，这确实是个大话题。
01:19
the human reader thinks I'm more aligned than I am.
人类读者觉得我比实际更对齐。
01:21
Like these are real things, not sci-fi.
这些都是真实存在的东西，不是科幻。
01:24
These are real things.
这些都是真实存在的现象，我们观察到之后做了大量工作，然后发布模型，这些模型并没有这些特性。
01:25
These are real things that we observe and then we do like significant amount of work and then we release models that that don't have these properties.
但 >> 如果你进入这样一个世界，比如说每次我们训练新系统时，所有这些问题的发生率都上升了100倍 >> 你可能会说，这确实挺令人担忧的。
01:33
But >> if you were to enter a world where say every time we trained a new system the rates of all of this stuff went up 100fold >> you might say well that's that's pretty concerning.
看起来如果我们把系统做到一定水平以上
01:42
It seems like if we make the systems above a certain level of
听起来像是，如果我们把系统做到一定水平以上——
01:46
intelligence they become radically misaligned against all human interests.
人类读者会觉得我比实际更对齐。
01:50
That's the kind of circumstance where if that happens the world needs information and the world would want an option to like slow or pause the development of the tech if you encountered that which we haven't today.
这些都是真实的问题，不是科幻。
02:00
So to answer your question like I don't I don't worry about it today but a lot of the measurement and analysis work we do is to cue us if the trend of >> worry you do worry about it.
这些都是真实的问题，是我们观察到的，然后我们做了大量工作，最终发布没有这些特性的模型。
02:10
I mean like you're not you don't think it's
但是 >> 如果你进入一个世界，比如说每次我们训练新系统时，所有这些问题的发生率都增加了100倍 >> 你可能会说，嗯，这挺令人担忧的。
02:12
happening today but part of the work you're doing specifically could be said to avoid the outcome where AI is built where in the pursuit of a goal it would kill all humans. >> Yeah.
今天发生的事，但你正在做的这部分工作，可以说是在避免一种结果——即AI在追求某个目标的过程中，会杀死所有人类。
02:26
Um >> wait is human extinction a risk factor in the anthropic IPO perspectives in the >> I want to know now.
>> 嗯。
02:38
Hello and welcome to another episode of the OddLots podcast.
>> 你确实担心它。
02:42
I'm Joe Weisenthal >> and I'm Tracy Aloway. >> Tracy, I don't know.
我的意思是，你不认为它今天正在发生，但你做的部分工作可以说就是为了避免AI在追求某个目标时杀死所有人类的结果。
02:45
I think our listeners like it, but uh you know, a lot of our episodes about are about AI these days, but to be fair to us, it's a pretty big topic. >> That's all anyone wants to talk about.
>> 是的。
02:55
Whenever we go to dinners with sources and things and people who are not even directly in the tech industry, you know, they might be in markets, they might be in policy and economics, all they want
嗯 >> 等等，人类灭绝是Anthropic IPO前景中的一个风险因素吗？
03:06
to talk about is AI and then inevitably the conversation veers into very sci-fi territory where we all start talking about the human extinction scenario and that's just the norm nowadays. >> I know it's so weird.
现在正在发生，但你做的部分工作可以说是在避免出现这样一种情况：AI被构建出来，在追求某个目标的过程中它会杀死所有人类。
03:17
You know, we were in Hong Kong recently and I, you know, this was, so we were there, uh, when we were in Hong Kong, this was before it was announced that there was a deal to open the street of Hormuz and East Asia was considered to be like ground zero for where the effects would be felt of the oil and jet fuel crisis, etc.
>> 对。
03:33
And we
嗯 >> 等等，人类灭绝是Anthropic IPO风险因素之一吗？
03:34
were at this dinner of business people like they were not talking about that at all.
我们当时跟一群商业人士吃饭，他们压根没聊那些。
03:38
You would think about the Terminator scenario. >> They just want to talk about token consumption and all of these things.
你可能以为会聊《终结者》那种场景。
03:44
Like here we are.
>> 他们只想聊token消耗之类的事。
03:45
It's like wait, aren't you guys supposed to be like under all kinds of jet fuel stress?
就像我们在这儿，心想：等等，你们不是应该被各种喷气燃料压力压得喘不过气吗？
03:49
So this is our defense for thinking uh for thinking AI is a pretty big AI episodes.
所以这就是我们觉得AI是个大话题的辩护理由。
03:53
I think it's fair.
我觉得挺公平的。
03:54
I will also say >> when we did the quiz in Hong Kong, we had a bunch of different teams with very creative names separated by human capital. >> That was a great one.
我还要说一句 >> 我们在香港做那个问答活动时，有好几个不同团队，名字都很有创意，靠人力资本区分开来。
04:02
>> They won the turn.
你好，欢迎收听另一期OddLots播客。
04:03
They won the quiz. >> They won proving that there is value in human capital.
我是Joe Weisenthal >> 我是Tracy Aloway。
04:07
But did you see that one of the tables was called Fable 13?
>> Tracy，我不知道。
04:11
Fable 13, Fable 13 was very topical at that moment. >> Very topical.
我觉得我们的听众喜欢这样，但你知道，我们最近很多期节目都在聊AI，不过公平地说，这确实是个大话题。
04:15
Well, we're recording this on June 17th, and of course, there's a lot in the news these days, but you know, things move very fast in AI, even if there weren't governmental controversies and all that stuff.
>> 这是所有人唯一想聊的。
04:26
You would have to mark the date in AI because of how fast breakthroughs
每次我们去和消息源吃饭什么的，那些甚至不直接从事科技行业的人——他们可能在市场领域，可能在政策和经济学领域——他们只想聊AI，然后对话不可避免地会转向非常科幻的领域，我们开始讨论人类灭绝的场景，现在这已经成为常态了。
04:30
happen.
>> 他们赢了那轮。
04:30
But, uh, you know, as you said, like AI sort of feels like the most important thing than anything else, but that's a very conventional wisdom thing.
他们赢了问答赛。
04:37
It was not always conventional wisdom.
>> 他们赢了，证明了人力资本的价值。
04:39
And I have a DM.
但你有没有看到有一桌叫Fable 13？
04:40
I know you're not supposed to share DMs from public, but I have a DM. >> You got the receipts. >> I have the receipts.
Fable 13，Fable 13当时特别应景。
04:45
August 2nd, 2016.
>> 非常应景。
04:46
And I DM'd a colleague colleague.
嗯，我们是在6月17号录这期节目，当然最近新闻很多，但你知道，AI领域变化太快了，就算没有政府争议那些事，你也得在AI里标记日期，因为突破来得太快了。
04:48
I said, "Did you leave Bloomberg?" He says, "Yes, I'll be announcing publicly in a bit.
我说：“你离开彭博了？”他说：“对，我很快会公开宣布。”
04:52
Take a couple of months to study AI properly, then leaving journalism to do something else still connected to AI." Being our Google
花几个月时间好好研究AI，然后离开新闻业，去做一些和AI相关的其他事情。就像我们Google那样——
04:58
reporter was a great thing. >> Still connected to AI is another >> and then the final the August 2nd, 2016.
reporter 这事儿挺棒的。
05:04
But AI is more important than anything else.
>> 还是跟 AI 有关，另一个是 >> 然后是 2016 年 8 月 2 号。
05:06
So I felt best to sort of optimize for that above all else.
但 AI 比什么都重要。
05:10
And then I just said, "Well, good luck." Anyway, >> this is someone who truly learned from their sources unlike us who remain in the podcasting industry. >> Um, so anyway, that person who would that was a former Bloomberg reporter, Jack Clark, who's one of our guests
所以我觉得最好优先优化这个，其他都靠后。
05:25
today.
但是，呃，你知道，就像你说的，AI感觉比其他任何事都重要，但这已经是老生常谈了。
05:25
He is the head of public benefit and co-founder of Anthropic 10 years later.
以前可不是这样。
05:29
Um, and also Peter McCory, head of economics at Anthropic.
我有一条私信。
05:32
So, two perfect guests to talk about all the things in AI these days.
我知道不该公开分享私信，但我有一条。
05:36
So, Peter and uh Jack, thank you so much for coming on the podcast. >> It's great to be back.
>> 你有证据。
05:42
I'm glad I optimized my life. >> Well done.
>> 我有证据。
05:44
It's a call one of the calls of the century.
2016年8月2号。
05:46
So, um why don't I actually start with that?
我发给一个同事，我说：“你离开彭博了吗？
05:49
Like August 20, it's easy to say in 2026, AI will be a big deal.
”他回：“是的，我很快会公开宣布。
05:53
>> You put you call it your shot.
>> 你当时可是押对了宝。
05:55
You got it right. 2016, what did you see in August 2016 or presumably before they're like, "Oh, you know what?
2016年，你在2016年8月，或者更早之前就看到了，然后他们就说，“你知道吗，这会是咱们这辈子最大的故事。
06:01
This is the biggest story of our lives. >> So for two years when I was reporting at Bloomberg, I wasted a lot of uh Mr.
” >> 所以我在Bloomberg做报道那两年，浪费了不少Mr. Bloomberg的打印机墨水，打印了一堆关于AI研究的旧论文。
06:08
Bloomberg's printer ink by printing out archive papers uh about AI research.
然后我做了一件非常Bloomberg风格的事——开始画图表，记录AI随时间发展的进度，测量计算机之类的指标。
06:12
And what I started to do, very Bloombergian thing, is I started to make graphs charting AI progress over time, measurements of things like computer
然后我开始做一件非常彭博风格的事：开始画图表，记录AI随时间推移的进展，测量计算机之类的指标。
06:21
vision, measurements of things like the the skill with which AI agents were able to compete and play Atari games.
>> 还是跟AI有关 >> 然后最后是2016年8月2号。
06:27
And what I saw in these graphs was the beginning of an exponential.
但AI比任何事都重要。
06:31
And it was everywhere.
所以我觉得最好优先优化这个，其他都靠后。
06:32
Like if you looked at vision or sound or video or game playing, you saw the same trend.
然后我就回了句：“祝你好运。
06:37
And it became obvious to me that this was this was a a general purpose technology that was right at the start.
”总之，>> 这是个真正从消息源学到东西的人，不像我们这些还留在播客行业里的。
06:43
My one um bone that I have to pick with Bloomberg, which I which I'm going to use my privilege to
>> 嗯，所以那个人是前彭博记者Jack Clark，他是我们今天的嘉宾之一。
06:48
just mention on air. >> I never got us to write a story saying Nvidia was being used in every single AI research paper. and I pitched it and I failed to get it across the line before I left. >> Oh man, I can just imagine you reading all these academic papers.
>> 我从来没在节目里说过。
07:02
Meanwhile, the editor is like, "We need the BF." >> I remember seeing like it's not AMD, it's Nvidia.
>> 我从来没写出一篇报道说Nvidia被用在每一篇AI研究论文里。
07:07
Like, this seems important. >> Very.
我提过这个选题，但在我离职之前一直没通过。
07:09
Yeah. >> Um, okay.
>> 天哪，我能想象你读那些学术论文的样子。
07:10
And Peter, I'm very interested in, you know, anthropic.
与此同时，编辑在旁边说，“我们需要的是BF。
07:13
It's a company trying to make money and yet it has this economics lab.
” >> 我记得当时看到，不是AMD，是Nvidia。
07:17
>> Yeah. >> What's what's the idea behind having an economics research body within a company that's developing this technology? it.
>> 嗯。
07:25
So, I mean, I was late to the game and joining Anthropic.
>> 在一个研发这项技术的公司内部设立经济研究机构，背后的想法是什么？
07:29
I joined just a year ago, but I had >> a year ago people Well, whatever.
其实我加入Anthropic算是比较晚的，一年前才进来，但当时—— >> 一年前啊，大家也都知道这一年股价涨了多少，不过算了，你继续。
07:33
We all know about how much the stock is a price in a year, but you're not go on.
嗯，我觉得很明显的一点是——我本身是应用宏观经济学家出身，一直试图理解经济中各种类型的冲击。
07:38
Um, I think what was very evident, so I'm an applied macroeconomist by training and have tried to understand various types
当初吸引我来Anthropic的原因之一，就是去年我就发现，他们不仅非常重视推进技术本身，还特别关注这项技术将如何重塑劳动力市场、对生产率和增长的影响，并且愿意把证据、数据和研究成果公之于众，让社会广泛受益、有用。
07:46
of shocks throughout the economy.
>> 经济领域会有一系列冲击。
07:48
Part of what drew me to Anthropic was it was evident to me last year that they cared very deeply about not just advancing the technology, but making sense of how it is set to reshape the labor market, its impact on productivity, on growth, and be willing to put evidence, data, and research out into the world that would be broadly beneficial and uh useful to society.
我之所以被Anthropic吸引，部分原因是去年我就明显感觉到，他们不仅非常关心推动技术进步，还特别在意如何理解这项技术将如何重塑劳动力市场、对生产力和增长的影响，并且愿意把证据、数据和研究成果公之于众，让社会广泛受益。
08:11
And I thought I want to be a part of
我觉得我想参与其中，
08:13
building that economic research programming program and do what I can to provide tenative answers to the most pressing questions.
>> 建立这个经济研究项目，尽我所能为最紧迫的问题提供一些初步答案。
08:21
We might not always get it right, but ideally we're helping society make sense of the change. >> The capabilities of the models on all kinds of things are extraordinary.
我们不一定每次都对，但理想情况下，我们在帮助社会理解这种变化。
08:32
I mean just mind-blowing.
>> 这些模型在各种事情上的能力都太惊人了，简直让人难以置信。
08:34
Every coding, copyright, every all kinds of things actually.
编码、版权、各种领域都是。
08:37
Why in July or sorry, June 2026 does life still feel maybe as
那为什么在2026年7月——哦不对，6月——从经济角度来看，生活感觉还是那么正常呢？
08:41
normal as it does from an economic perspective? >> This is a great question and one that I've been wrestling with.
>> 这是个很好的问题，也是我一直在思考的。
08:48
Um I think there are a number of reasons why you might think that the impact has not yet materialized.
我觉得有几个原因可以解释为什么影响还没显现出来。
08:54
One, uh the technology can advance, but it also then needs to diffuse throughout the economy and there can be bottlenecks from moving from capabilities to actual deployment.
第一，技术可以进步，但它还需要在整个经济中扩散，从能力到实际部署之间可能存在瓶颈。
09:05
We see that with our enterprise customers.
我们在企业客户身上就看到了这一点。
09:07
So if you want to automate biological
如果你想自动化生物研究
09:10
research or some other very complicated financial modeling task, you need a lot of contextual information available to the model.
>> 或者其他非常复杂的金融建模任务，模型需要大量的上下文信息。
09:17
If you don't have that contextual information, the capabilities alone won't necessarily drive the impact.
如果没有这些上下文信息，光有能力本身不一定能带来影响。
09:23
It also takes time for people to just start using the tools.
人们开始使用这些工具也需要时间。
09:26
And so we're still in the somewhat of the early stages there.
所以我们还处于早期阶段。
09:29
Um, two places that I would be looking to see an impact.
第二，我会关注两个可能看到影响的地方。
09:33
One is in terms of productivity growth.
一个是生产率增长。
09:35
We've done some research that points in
我们做过一些研究，表明
09:37
the direction that this should be large and consequential.
>> 这个影响应该很大且意义深远。
09:41
Labor productivity growth has been strong throughout the pandemic and has s been sustained so far >> like modestly.
劳动力生产率增长在疫情期间一直很强劲，而且到目前为止还在持续，不过只是温和增长。
09:47
So we're not talking about like you know revolutionary >> changes.
所以我们说的不是那种革命性的变化。
09:51
Yeah.
对。
09:51
But you know to to get on an inflection you need to at least move a little bit. >> Um I think maybe you're seeing some signs there on the on the labor market though. the labor market is in a reasonably healthy spot and I think it
但要想进入一个拐点，至少得先有点动静。
10:05
might be because it's primarily at so far a labor augmenting skill bias technology not yet the full sort of general purpose substitute for all of cognitive labor although perhaps that's the trajectory that we're on >> you know for the size of AI and its capabilities I was talking to Peter about this and he did point out the economy very big >> so it still takes a lot to move it um I do think strange things are starting to
>> 嗯，我觉得在劳动力市场上可能已经看到一些迹象了。
10:31
happen at least inside the company.
至少在公司内部是这样。
10:33
We we published research from the anthropic institute recently on this topic called recursive self-improvement where it was inspired by me going on paternity leave in November of last year and coming back in February and the entire company felt and works differently and I assumed it was because models had got better and when we looked at the data what you saw was in 2026 engineers at Anthropic are
我们Anthropic研究所最近发表了一篇关于递归自我改进的研究，灵感来源于我去年十一月休陪产假，二月回来时发现整个公司的工作方式都变了。
10:56
writing about eight times the amount of code that they did in 2021 through to 2024. for and the the line started last year with things like Opus 45 and Opus 46.
现在写的代码量是2021到2024年间的八倍。
11:06
Then it really got going this year and I have colleagues now who don't program at all anymore.
这事儿从去年就开始了，比如Opus 45和Opus 46这些模型。
11:11
They just instruct many many cord code agents to run around and do their work for them. >> I can't reconcile that with the the world staying normal for long.
然后今年更是全面爆发，我有些同事现在完全不写代码了，他们就是指挥一大堆代码agent到处跑，替他们把活儿干了。
11:21
Um but it's going to take a while for that to
嗯，但这还需要一段时间才能——
11:23
diffuse into the world and change it. >> Yeah.
>> 我实在没法相信这个世界还能长期保持原样。
11:26
And we'll talk more about recursive self-improvement.
但要让这种变化扩散到全世界并真正改变它，还需要一段时间。
11:29
So this is when models basically improve on themselves, right? >> So in terms of the awkwardness of the current moment or the weirdness of the current moment, you've talked about basically living through the singularity and how strange it is >> and and you've also described yourself as a technimmist before.
所以这就是模型基本上自我改进的过程，对吧？>> 所以说到当前这个时刻的尴尬或怪异之处，你提到过基本上是在经历奇点，以及这有多奇怪 >> 而且你之前也形容自己是个技术悲观主义者。
11:48
How do you square that with working at
你怎么把这一点和你在——
11:50
Anthropic, which is making some of these weird and potentially dangerous things actually happen?
>> 对。
11:55
So by technological pessimist I mean um I thought the technology would keep getting better but I didn't think it would get better in the like maximalist sense that some of my colleagues did.
我们待会儿会详细聊递归自我改进，就是模型自己改进自己，对吧？
12:05
I didn't think that we would have say functionally automated all of coding right now.
我原本没想过我们现在就能实现编码的全面自动化。
12:09
I find that actually like quite surprising but basically over the last few years and I worked at OpenAI before Anthropic I was just hit repeatedly over the head with
我发现这其实挺令人惊讶的，但基本上在过去几年里，我在OpenAI工作过，后来去了Anthropic，我反复被这个问题击中。
12:18
what uh computer scientist Richard Sutton calls the bitter lesson.
>> 说到当前这个时刻的尴尬或诡异之处，你提到过基本上是在亲身经历奇点，以及这有多奇怪。
12:22
And the bitter lesson is this concept that the more compute and resources we dump into these relatively generic neural networks, the smarter they get and the more emergent properties they have.
而且你之前也形容自己是技术悲观主义者。
12:33
And your your specialized system or your ability to be pessimistic about future AI progress uh loses versus just scaling compute and scaling systems. >> And this seems to have implications for
那你一边在Anthropic工作，让这些奇怪甚至可能有危险的东西变成现实，一边又持这种态度，这怎么协调呢？
12:45
the labor market, right?
所谓技术悲观主义者，我的意思是，我本来觉得技术会不断进步，但没觉得会像某些同事那样，达到一种最大化意义上的进步。
12:46
Because and I think a good example of the bitter lesson is probably the history of AI chess, right? where at one point they had grand masters come in and teach the models how to play chess and etc and try to encode their wisdom.
我没想过我们现在就能把编程几乎完全自动化。
12:59
And it turned out in the end that the best way to get a chess engine really good is to just teach the model tell the model the rules of chess and say go off and play a billion games and find optimal chess without any human insight.
这确实让我挺惊讶的。
13:12
The grand
但基本上，过去几年里——我在去Anthropic之前是在OpenAI工作——我被反复敲打，深刻体会到了计算机科学家Richard Sutton所说的“苦涩教训”。
13:12
masters were not necessary for that process at all.
>> 这似乎对劳动力市场有影响，对吧？
13:16
Right?
我觉得苦涩教训的一个好例子可能是AI下棋的历史。
13:16
And so this see this would imply to me like have significant um implications for the labor market. >> Yeah.
曾经有国际象棋大师来教模型怎么下棋，试图把他们的智慧编码进去。
13:22
I I I tend to think about this in sort of three aspects of what composes a job.
但最后发现，让一个象棋引擎变得超级厉害的最佳方式，就是只告诉模型规则，然后让它自己去下十亿盘棋，找到最优解，完全不需要任何人类洞察。
13:27
One is you need to decide what to do and direct and delegate.
那些大师在这个过程中根本没用。
13:31
You need to then imple do the actual implementation of the work and then you need to sort of evaluate or at least set up systems that
对吧？
13:39
can evaluate.
>> 对。
13:40
At least from my perspective as an economist, this this bitter lesson is materializing in terms of very rapid advances in the implementation work of what an economist does.
我倾向于从构成一份工作的三个方面来思考这个问题。
13:52
Downloading data, running regressions, building models, uh solving them using sort of uh contemporary uh solution techniques, numerical uh methods.
第一，你需要决定做什么，并进行指导和委派。
14:03
I definitely felt that personally with
第二，你需要实际执行工作。
14:06
Opus 4.5 where I was for the first time able to just delegate a very complex task.
要聊的是AI，然后话题不可避免地会滑向非常科幻的领域，我们开始讨论人类灭绝的场景，这在现在已经成了常态。
14:10
I had this very specific research question trying to understand the cyclicality of hiring across different occupations and how that relates to occupational exposure.
>> 我知道，这太奇怪了。
14:20
That's a mouthful.
你知道，我们最近在香港，当时，我们在香港的时候，那是宣布要开放霍尔木兹海峡交易之前，东亚被认为是石油和航空燃油危机影响最严重的前线地带。
14:21
I gave that task to Claude and Claude was able to just iterate on it and I could redirect Claude in the same way that you might redirect a grad student.
然后我们
14:29
And you know the the the big question
你知道，那个那个最大的问题
14:32
that I have in mind is you know at what point do the boundaries at the the direction setting stage the research taste you might call it and uh will the models become sufficiently reliable? >> If I could just get in here you know I just read um the recent biography of uh the deep mind founder >> this is the Sebastian Malik. >> Yeah.
我心里想的是，你知道在哪个点上，方向设定阶段、研究品味——你可以这么叫它——以及模型会变得足够可靠吗？
14:49
Like is there going to be a point where it's like okay you have some intuitions right about like what good economics research is and often our
>> 如果我能插一句，你知道我刚读了DeepMind创始人的最新传记 >> 这是Sebastian Malik的那本。
14:57
intuitions are formed because we tell stories and stuff like that but is there going to be a point where you think like your intuitions will be unhelpful and that because that's sort of what I took away from the go experience that the model got better once they stripped it of the human games and the human bias and that actually like the human intuition that sort of helps us understand oh labor market rising creates inflationary pressure.
直觉的形成是因为我们讲故事之类的，但会不会有一个时刻，你觉得自己的直觉反而没用了？
15:22
These
这其实是我从围棋经验中得到的启发：模型一旦摆脱了人类游戏和人类偏见，反而变得更好。
15:23
stories that are very sort of intuitable end up um impairing the model.
那位记者很厉害。
15:27
Do you see that happening in say economics where it's like some of these stories that we tell forever, they're not actually very helpful for an uh an optimal economy understanding model?
>> 还是跟AI有关，另一个是…… >> 然后是2016年8月2日。
15:37
I I expect that these models will soon have better intuitions about how to do good economic research and that the big there is this big question of like at what point will we be able to fully automate
但AI比任何事都重要。
15:48
social science research? >> Uh we've done some work on this to try to understand how coding agents are beginning to automate social science research. >> Um and uh but I don't think we're quite there yet.
>> 嗯，我们做过一些研究，想了解编码智能体是如何开始自动化社会科学研究的。
16:01
And I don't know that that'll be an exciting time for learning about the world. you know what that means for my job.
而且你知道，那个大问题。
16:09
I'm sort of less entirely clear. >> Yeah, I I think this is the big wild card in future AI progress.
我其实不太清楚。 >> 对，我觉得这是未来AI发展中的最大变数。
16:15
Um, you
我不知道那会不会是一个了解世界的激动人心的时刻。你知道这对我的工作意味着什么。
16:16
know, if if AI progress continues today, we are likely to get technology that will be able to do basically everything.
记者是件好事。
16:22
But we will need people who have good instincts, good intuitions and good ideas to basically set the direction.
>> 仍然和AI有关，另一个 >> 然后是2016年8月2日。
16:28
And we see this today in a lot of a lot of our own research where you need say an AI safety researcher to give nine clawed agents for different research areas to go and pursue and then it's very effective.
但AI比任何其他事情都重要。
16:39
If that researcher doesn't give them the research
所以我觉得最好优先优化这个。
16:42
directions, they pursue relatively formulaic research directions and you have entropy collapse.
>> 但我觉得还没完全到那一步。
16:48
You end up with just like boring research that doesn't move the ball forward. >> At what point will AI systems generate like heterodox insights and genuine creativity?
我不知道那会不会是一个了解世界的激动人心的时刻。
16:57
We can't really measure for that today.
你知道这对我的工作意味着什么吗？
16:59
But what we have of the symptoms of it starting in experts like Peter, experts like colleagues in the fields of biology or mathematics or physics outside of anthropic are all
我其实不太清楚。
17:09
starting to be accelerated by AI.
开始被AI加速了。
17:11
You know, Terry Tao, probably one of the most famous living mathematicians, >> cocreate math now with AI systems.
你知道吗，Terry Tao，大概是当今最著名的在世数学家之一，现在已经在用AI系统共同创作数学了。
17:18
And so that that says to me that these things have got they're they're tickling the dragon's tail of like creativity here.
所以在我看来，这些东西已经在触碰创造力的龙尾巴了。
17:24
And you know we we just put out a report yesterday on uh cloud code usage and one of the things that we're trying to understand is like what are the returns to expertise and how does that
而且，我们昨天刚发布了一份关于云端代码使用的报告，其中我们想搞清楚的一个问题是：专业技能的回报到底是什么样的，以及它如何……
17:35
interact with the usage of sort of automated coding agents >> and we find that domain expertise like if you're an accountant who understands some of the edge cases and reconciliation that that domain expertise controlling for a whole host of factors about the type of work the estimated monetary value >> it has an amplifying effect. >> It has an amplifying effect.
与那种自动化编码代理的交互使用 >> 我们发现，领域专长——比如你是个会计，懂一些边缘案例和对账——这种领域专长在控制工作类型和预估货币价值等一系列因素时，会产生放大效应。
17:56
So this looks like at at present a sort of a skillbiased uh expertise enhancing impact.
>> 确实有放大效应。
18:01
But I
所以目前看来，这像是一种偏向技能的、增强专长的影响。
18:01
think this is the key question is at what point and to what extent will this change? >> Well related to this you know Jack when you describe coming back from paternity leave and seeing how much things had changed at anthropic.
我认为关键问题是，这种变化会在什么时候、以什么程度发生？
18:16
I know we're not officially at rec recursive self-improvement >> point but it sounds like we're semi there.
>> 说到这个，Jack，你描述休完陪产假回来发现Anthropic变化那么大。
18:22
We're kind >> Yeah.
我知道我们还没正式到recursive self-improvement那个阶段，但听起来已经半只脚踏进去了。
18:23
And so my question is like I get that at the moment you have engineers
差不多吧。
18:28
who are reviewing all the code that the AI is producing and they're thinking about it and managing it in some way.
那些正在审查AI生成的所有代码的人，他们正在思考这些代码并以某种方式管理它。
18:35
But you can easily imagine a future where just the sheer quantity of code overwhelms human expertise.
但你很容易想象一个未来，光是代码的数量就会压垮人类专家的能力。
18:41
Maybe the quality starts outstripping what human engineers are capable of understanding.
也许代码的质量开始超越人类工程师所能理解的范围。
18:46
How do you manage that? >> Yeah.
那你怎么管理呢？
18:48
So there's two ways of thinking about recursive self-improvement.
>> 对。
18:51
One is what happens when AI organizations start
所以关于递归自我改进有两种思考方式。
18:55
to see a compounding return from their AI systems.
那些非常直观的故事最终会损害模型。
18:57
Basically, their own production function improves because of the tools they've built.
你看到这种情况发生在经济学领域了吗？
19:02
That's clearly happening now.
比如我们一直讲的一些故事，实际上对构建一个最优的经济理解模型没什么帮助。
19:03
And then the second is what happens if an AI system can just build itself entirely autonomously given compute which hasn't happened. >> Yeah.
我预计这些模型很快会对如何做好经济研究有更好的直觉，而一个很大的问题是：我们什么时候才能完全自动化……
19:11
What I see inside Anthropic is I think what we'll see in the broader economy which is we are figuring out how to verify and validate and basically price for risk of an expanding cloud of
我在Anthropic内部看到的，我认为也会在更广泛的经济中看到——我们正在摸索如何验证、确认，并基本上为不断扩大的云服务风险定价。
19:20
automated systems which we're sitting on top of.
开始被 AI 加速了。
19:23
So now we produce way more code.
你知道，Terry Tao，大概是在世最著名的数学家之一，>> 现在跟 AI 系统一起合作搞数学。
19:24
Well, we broke our continuous integration system for integrating code into the codebase because we started pushing eight times more code through it than before.
所以这让我觉得，这些东西已经……它们已经在触碰创造力的龙尾巴了。
19:32
So all of our human engineers worked on unbreaking CI.
而且，我们昨天刚发了一份关于 cloud code 使用情况的报告，我们想搞明白的一件事就是，专业技能的回报到底是什么样的，以及这怎么……
19:35
And so I think that in Continuous integration. >> Thank you. >> Um, you don't need to know what it is.
我其实不太清楚。 >> 对，我认为这是未来AI发展中的一个大变数。
19:40
It's just the thing that helps you push the code into the cloud. >> We like to know stuff on this show.
它只是帮你把代码推到云上的工具。 >> 我们这节目就爱搞明白这些事。
19:45
We like to learn.
我们喜欢学习。
19:47
>> So the but there's a lesson in that, right?
开始被AI加速了。
19:49
We are going to speed up things in the economy.
你知道，Terry Tao，可能是当今最著名的在世数学家之一，>> 现在用AI系统共同创造数学。
19:52
We're going to speed up the way that we produce stuff and then we're going to find, you know, the like the weak links or the hot paths that break.
所以这让我觉得，这些东西已经触及了创造力的龙尾。
20:00
And we as people are going to move to sorting those out.
而且，我们昨天刚发布了一份关于云端代码使用的报告，我们想弄明白的一件事是：专业知识的回报是什么，以及它如何……
20:04
And then the cycle starts again.
然后这个循环又重新开始了。
20:05
And we're kind of sitting on this expanding cloud of of of automated actions.
我们就像是坐在一个不断膨胀的自动化行动云上。
20:10
Um, well, since we're talking about like
嗯，既然我们在讨论
20:12
really like feeling like we're staring at the horizon of extremely strong AI or maybe we'll get there or maybe the AI builds itself, might be a good time to ask a fable question or mythos question.
真的有种感觉，我们正盯着超强AI的地平线——也许我们真能抵达那里，或者AI会自己构建自己。
20:24
At this point, we're recording this June 7th.
这时候或许该问个寓言式或神话式的问题。
20:26
We don't know when it's going to be available for Americans, let alone the rest of the world.
我们录制这期节目是在6月7号，还不知道美国人什么时候能用上，更别说世界其他地方了。
20:31
Does Anthropic have a clear idea of what the administration's security concerns are and what it will take to resolve them?
Anthropic对政府的安全顾虑以及解决这些顾虑需要什么条件，有明确的想法吗？
20:39
>> Well, obviously live discussion.
>> 嗯，这显然是个现场讨论。
20:40
I can't get into too many specifics.
具体细节我不能说太多。
20:42
We're in daily discussions with the government about this. >> The broad thing I'd say is for many years we've anticipated a point where AI systems would have national security properties.
我们每天都在和政府讨论这件事。
20:52
These national security properties are intertwined with their economically valuable properties.
这些国家安全属性跟它们的经济价值属性是交织在一起的。
20:57
How you manage that as a policy question is is basically novel territory.
如何把这个问题作为政策议题来管理，基本上是个全新的领域。
21:01
TR typically these things are decoupled.
通常来说，这些东西是分开处理的。
21:03
You're like, "Hey, I built a jet engine
你会说，“嘿，我造了一个喷气发动机。”
21:06
over here which can go into civilian aircraft and I built a missile over here and you treat them differently.
>> 大体上我想说的是，多年来我们一直预期会有一个节点，AI系统会具备国家安全属性。
21:12
It's odd if you smush these things together." Where we'll get to, I'm confident, is what's a system for assessing the properties of AI systems, including national security components.
这些国家安全属性和它们的经济价值属性是交织在一起的。
21:22
And then what is a system for either squaltching the national security capabilities from coming to general proliferation like bioweapons or or cyber weapons?
如何从政策层面来管理这个问题，基本上是个全新的领域。
21:31
And are
通常这些东西是分开的。
21:32
there ways to do things like know your customer or deployments where you let large firms like say drug developers access the most powerful biom models without accidentally proliferating risks.
有一些方法可以做类似了解你的客户（KYC）的事情，或者部署时让大型企业，比如药物研发公司，能够访问最强大的生物模型，同时避免意外扩散风险。
21:42
That's the shape of I think where we'll end up and what we're doing right now.
我觉得这大概就是我们最终会走向的方向，也是我们现在正在做的事情。
21:46
We and other companies are and the administration are basically tackling this problem in real time.
我们和其他公司以及政府现在基本上是在实时解决这个问题。
21:51
Uh it's initially going to be messy but we're going to end up with a system on the other side. this specific incident and
一开始可能会很混乱，但最终我们会建立起一个体系。
21:58
there will probably more in the future because everyone's just figuring this out. >> When I look at the AI landscape, I sort of think of open AI as being part of the all-in podcast A16Z um David Saxs White House thing.
未来可能还会更多，因为大家都还在摸索阶段。
22:18
And I know from my friends in the media, many of whom are liberal Democrats, that I
>> 当我观察AI领域时，我总觉得OpenAI就像是all-in播客、A16Z还有David Sax的白宫那档子事的一部分。
22:25
sort of feel like Anthropic is the more like libcoded of the major models.
>> 当我观察AI领域时，我觉得OpenAI属于All-In播客、A16Z、还有David Sax的White House那个圈子。
22:30
Do you feel there's any either politics or partisan politics going on as part of anthropic being harassed or singled out now multiple times?
而且我从媒体圈的朋友那里了解到，他们很多是自由派民主党人，我有点感觉Anthropic在主流模型里更像是偏自由派的那一方。
22:39
Anthropic's philosophy and what I do and I I lead something called the anthropic institute which helps us produce better data for the world around things like recursive self-improvement the economics
你觉得Anthropic多次被针对或单独挑出来，这里面有没有政治或者党派因素？
22:52
work cyber risks is we tell the whole story about what's going on typically I think the technology industry has told only optimistic stories about what it's building and what we saw with social media is that does not work actually eventually when when you're doing something that changes the entire world which AI is certainly doing and social media certainly did.
做网络风险这块，就是要讲清楚事情的全貌。
23:11
It's not going to be a wholly optimistic story.
通常来说，科技行业只讲自己正在构建的东西的乐观故事，而我们在社交媒体上看到的是，这其实行不通。
23:14
There will be negatives as well.
当你做的事情正在改变整个世界——AI显然在这么做，社交媒体也确实做到了——它不可能是一个完全乐观的故事，也会有负面影响。
23:16
We've always sought to just tell the truth about what
我们一直努力做的就是讲出真相，关于
23:18
we see in front of us.
>> 你不觉得这里面有党派因素吗？
23:20
And I think sometimes that can uh that can differentiate us a bit to others.
比如你们不是“自己人”，或者没给舞会捐够钱之类的？
23:24
But the important thing is we tell the truth and things end up coming. >> You don't think that there's like a partisan element here where you guys aren't on the uh on the team or didn't contribute enough to the ballroom or whatever? >> I can't really speak to to to that.
>> 这个我真的不好说。
23:40
I'm, you know, I'm not those people.
我不是那些人，我是Anthropic的人。
23:42
I'm I'm anthropic.
我能说的是——
23:43
What I can say is
我能说的是，嗯，不幸的是，这项世界级历史性技术的到来，正赶上宏观经济波动异常剧烈的时期。所以原则上，时间并不充裕。好吧，现在我必须问一句：它不会被大众广泛理解或知晓。他们只能玩玩这些模型。我们能创造的每一比特数据，尤其是系统性的数据——不是所有工程师都能做到。
23:44
>> the AI systems create their own evidence.
>> AI系统会自己制造证据。
23:47
Years ago, it seemed very odd to speculate about the cyber properties of AI systems.
几年前，推测AI系统的网络风险属性还显得很奇怪。
23:51
Well, they've arrived and now we're working on them.
现在它们来了，我们正在研究这些。
23:54
Years ago, it was odd to speculate about the bioweapon properties of AI systems.
几年前，推测AI系统的生物武器属性也很奇怪。
23:59
More recently, Sam Alman, Dennis Habis, and Dario Amade of OpenAI, Anthropic, and Deep Mind all signed a letter saying we need to do better screening of gene synthesis to prevent AI manufactured bioweapons.
最近，Sam Altman、Dennis Hassabis和Dario Amodei——来自OpenAI、Anthropic和DeepMind——都签署了一封信，说我们需要更好地筛查基因合成，以防止AI制造生物武器。
24:10
But truth wins out.
但真相终究会胜出。
24:11
>> Okay. >> I want to go back to something you said.
>> AI系统会自己制造证据。
24:14
You mentioned potential KYC requirements.
几年前，猜测AI系统的网络属性还显得很奇怪。
24:16
And when I hear KYC, I think about the finance industry and I think about systemically important institutions and the stress tests and the framework around that.
现在它们来了，我们正在研究。
24:25
Is that the right analogy to use for I guess ideal AI regulation in your mind rather than I guess just simple export controls?
几年前，猜测AI系统的生物武器属性也很奇怪。
24:32
Should we be heading towards something that looks a little bit more like what we do for the banking system?
最近，Sam Altman、Demis Hassabis和Dario Amodei——来自OpenAI、Anthropic和DeepMind——都签署了一封信，说我们需要更好地筛查基因合成，以防止AI制造生物武器。
24:39
>> We need something that's more subtle and more technocratic from what we have today.
>> 我们需要比现在更精细、更技术官僚化的东西。
24:43
I don't know if it'll be exactly like the banking system.
我不确定它会不会完全像银行体系那样，但可能会借鉴一些思路。
24:46
It'll probably take some ideas from that.
它会从美国政府和其他机构目前对AI系统进行测试的做法中吸取一些经验。
24:47
It'll take some ideas from what the US government and others are doing today with just testing AI systems for their their properties.
而且几乎可以肯定，它会带有我和Peter以及Anthropic研究所正在做的事情的味道——在系统部署到现实世界时生成关于它们的数据。
24:54
And it's almost certainly going to have a flavor of what what Peter and I work on and and the Anthropic Institute broadly of generating data about these systems as they're deployed in the world because it's not it's one thing to you know test
因为测试是一回事，
25:05
out the thing before it comes out of a factory.
在东西出厂之前就发现问题是一回事，真正观察它在世界上产生的影响，然后判断这些影响是好是坏，又是另一回事。
25:08
It's another to see what to observe the effects it's having in the world and then to be able to make um make judgments about whether those effects are good or not.
你会支持——咱们还是用金融类比——上市公司至少需要第三方审计签字，提交10-Q报告时也有相关讨论。
25:18
Would you support um you know in the fi let's stick with the financial analogy um companies that are public at least are required to have third party auditors sign off on them and there's talk um you know when they submit their 10 Q's etc.
发行债务的公司需要评级机构，或者经常让评级机构来评估它们的债务。
25:32
Um um credit people
你会支持在法律中嵌入要求，让相当于穆迪或德迪的第三方研究实验室对新模型的发布进行签字确认吗？
25:34
companies that issue debt are required to have ratings agencies or frequently have ratings agencies rate their debt.
我们最近确实提出了类似的政策建议，包括需要第三方测试来评估国家安全和其他属性，因为这显然是验证很多问题的合理方式。
25:41
Would you support embedding in law the requirement that certain what would be the equivalent of a Moody's or a Deote sign off on you know a third party research lab sign off on the release of new models? >> We've we've proposed something like this recently uh a policy proposal that we laid out which includes saying we need
所以更广泛地回到衡量AI实际影响这个想法上，我发现一件很有趣的事：如果你看我们很多传统的AI——或者说我已经被AI洗脑了——如果你看一些传统的经济统计数据，很多AI的影响其实还没显现出来。
26:01
to have third party testing for some of these national security and other properties because clearly that's that's like a sensible way that you validate a lot of this.
是的，我认为这个前提完全正确，这也差不多是我们对话的起点——我们可能已经到了应该能在宏观经济中看到一些明显影响的阶段。
26:11
So just more broadly returning to this idea of you know measuring the actual impact of AI.
不幸的是，这项世界历史性技术的到来，恰逢疫情后宏观经济波动异常加剧的背景，还有货币政策等因素，所以很难把所有不同因素拆分开来。
26:16
One thing I find really interesting is that if you actually look at a lot of our traditional AI or I should say I'm AI brained already.
反事实是什么？
26:23
Um if you look at some of our traditional economic statistics a lot of the AI impact doesn't actually
劳动生产率增长可能没有你预期的那么强劲，但也许比反事实情况要好。
26:29
show up just yet.
目前还没显现出来。
26:30
Again we're in the early stages but you would expect if we're talking about the AI economy growing something like 2,000% or 3,000%.
再说一遍，我们还在早期阶段，但如果AI经济要增长2000%或3000%——我记得见过这个数字。
26:38
I think I've seen that number. >> That's from Anton Cornet in Mckelv paper uh a few weeks ago. >> You would expect that to have more of an impact on nominal GDP and yet it's not really showing up that much.
>> 那是几周前Anton Korinek和McKelvey的论文里提到的。
26:50
Do you think the way we measure the economy needs to be changed in some way in light
>> 你应该会预期这会对名义GDP产生更大影响，但实际上并没有那么明显。
26:55
of what's happening with this new technology?
是的。
26:58
Yeah.
所以我认为这个前提完全正确，这也是我们对话的起点——我们可能已经到了应该能在宏观经济中看到一些可察觉影响的阶段了。
26:58
So I I think this this is exactly the right premise and it's kind of where we began the conversation which is you know we're maybe at the point where we should be able to see some discernable impact on the macroeconomy.
不幸的是，这项历史性技术的到来，恰逢疫情后宏观经济波动异常加剧的背景，
27:11
Um unfortunately the arrival of this world historical technology is against the backdrop of uh sort of unusually elevated macroeconomic volatility post
嗯，不幸的是，这种世界级历史性技术的到来，恰逢宏观经济波动异常加剧的背景。
27:20
pandemic >> monetary policy etc.
还有货币政策等等。
27:23
Um, and so it like makes it very hard to disentangle all of the the different factors.
所以这就让区分各种不同因素变得非常困难。
27:29
You know, what's the counterfactual?
你知道，反事实是什么？
27:32
You you know, labor productivity growth is uh maybe not as strong as you might not might not otherwise expect, but maybe it's stronger than it is in a counterfactual sense.
劳动生产率增长可能没有你预期的那么强劲，但也许比反事实情况下要强。
27:44
Um, and so one way that we've tried to tackle this question is by
所以我们尝试解决这个问题的一种方法是，
27:49
looking at how Claude is being used on our platform using our privacy preserving techniques to estimate the time savings associated with each of the activities that people use Claude for.
通过观察Claude在我们平台上的使用情况，利用我们的隐私保护技术来估算人们使用Claude进行各项活动所节省的时间。
28:00
So >> uh, compiling information from reports to put together a research brief would take you a few days maybe.
比如，从报告中整理信息来撰写一份研究简报，以前可能需要几天，现在Claude几分钟就搞定了。
28:06
Now Claude does it in a few minutes. uh evaluating diagnostic images um is something that skilled professionals do very rapidly.
评估诊断影像，熟练的专业人员做得很快，所以原则上节省的时间并不多。
28:13
So there isn't in principle much time
所以原则上时间并不充裕。
28:15
savings.
节省下来的成本。
28:16
You can add up all of those numbers and using standard macro growth accounting techniques, Holton's theorem for the economists and the audience um and you get a number of that points in the direction of labor productivity growth increasing by 1.8 percentage points each year over the next decade.
把这些数字加起来，用标准的宏观增长核算方法——霍尔顿定理，给经济学家和听众们解释一下——你会得出一个数字，指向未来十年劳动生产率每年增长1.8个百分点。
28:33
If that's how long it takes current usage patterns and current model capabilities to diffuse throughout the economy, that's a very large number.
如果这是当前使用模式和模型能力在整个经济中扩散所需的时间，那这个数字非常庞大。
28:41
It's a rough doubling of recent run
这差不多是近期增速的两倍。
28:43
rates.
rates. 而且我觉得你可能能在数据里看到——我们还没发布这方面的内容——嗯，我认为近期劳动生产率增长的一些强劲表现，实际上正集中在那些与我们的数据以及商业趋势分析都吻合的经济部门。
28:43
And what I think you might be able to see in the data, and we haven't put anything out on this yet, is um I think some of the strength in recent labor productivity growth is actually concentrating in exactly the sectors of the economy that would be consistent with both what we see in our data as well as also what you see in the business trend analysis. for example. >> So the information sector has high rates of adoption.
比如说，信息部门的采用率很高。
29:06
Um I can't recall if that's in particular one of the sectors that I
嗯，我不太确定那是不是我想到的特定部门之一。
29:10
have in mind. um uh you know it's it's it's a while since I looked at that scatter plot but you can look at the um sort of subindustries by the Census Bureau's business trend and outlook survey and rates of adoption are in sectors or parts of the economy where controlling for pre- pandemic trajectory of labor productivity growth in those sectors even some of the strength in the early years of the recovery still see some like suggestive evidence I think there's
>> 我心里有数。
29:38
a lot of >> uncertainty here trying to get a real time signal on productivity is maybe the hardest thing to do.
嗯，你知道，我有一阵子没看那个散点图了，但你可以看看人口普查局商业趋势和展望调查里的子行业数据，采用率高的行业或经济领域，在控制这些行业疫情前劳动生产率增长轨迹后，即使在复苏初期的强劲表现中，仍然能看到一些暗示性的证据。
29:45
You're subject to macroeconomic GDP revisions.
我觉得这里有很多不确定性，想实时获取生产率信号可能是最难的事情之一。
29:48
TFP growth is actually sending the opposite signal.
你会受到宏观经济GDP修正的影响。
29:51
And if you control for capacity utilization, TFP growth is arguably even lower.
全要素生产率增长实际上发出了相反的信号，如果你控制产能利用率，全要素生产率增长甚至更低。
29:56
So I, you know, I say this as like this is suggestive evidence that maybe we're beginning to see an impact impact there, but not so much in the labor market.
所以，我这么说吧，这只是暗示性证据，表明我们可能开始看到一些影响，但在劳动力市场上还不明显。
30:05
Well, now I have to ask
好吧，那我得问了。
30:07
when you gather this kind of research and it all sounds super interesting, but if you have data for instance that shows that okay, the IT sector is getting productivity gains from using claude or I don't know maybe something unexpected like the warehousing industry is using a bunch of AI.
当你收集这类研究时，听起来都特别有意思，但如果你有数据，比如显示IT行业因为使用Claude而获得了生产力提升，或者不知道，也许是一些意想不到的领域，比如仓储行业正在大量使用AI。
30:24
What does anthropic actually do with this data?
那么Anthropic实际上会用这些数据做什么呢？
30:26
Does it somehow feed back to your engineers who are developing frontier models?
这些数据会不会反馈给那些开发前沿模型的工程师？
30:31
Do they do anything differently?
他们会因此做出什么不同的调整吗？
30:33
Uh I think some
嗯，我觉得有些……
30:34
of it cues us on areas where maybe the technology isn't being used because it's very weak.
好吧，那我得问一下，当你收集这类研究时，听起来都超级有趣。
30:39
We just haven't made it particularly good for these use cases or in areas where it's being used at large scale.
但如果你有数据显示，比如IT行业因为使用Claude获得了生产率提升，或者我不知道，也许像仓储行业意外地大量使用AI。
30:45
It's usually a suggestion of keep making it good there.
Anthropic实际上会用这些数据做什么？
30:47
Um but you know the actual economic measurement data doesn't really get fed back directly in but it's a very useful clue.
它会反馈给开发前沿模型的工程师吗？
30:54
Um we think it's more important though to basically communicate this outwardly to policy makers, journalists and others because
他们会因此做不同的事情吗？
31:01
our assumption is that at some point we go through some phase change similar to how capabilities of AI occasionally jump forward in a really dramatic way where you might see sudden and rapid diffusion as a consequence of capability expansion in the AI systems.
>> 我们的假设是，在某个时间点，我们会经历某种相变，就像AI能力偶尔会以非常戏剧性的方式跳跃式前进一样，你可能会看到由于AI系统能力扩展而导致的突然且快速的扩散。
31:14
So we're getting practice in of looking at this kind of data.
所以我们正在练习如何分析这类数据。
31:17
My expectation is that in a year or two years I'm going up to some policy maker and I'm pointing them to the part of the graph that now gets very steep in some chunk of the economy >> and hoping that they'll do something
我的预期是，一两年后，我会走到某个政策制定者面前，指着图表上某个经济板块中突然变得非常陡峭的部分，希望他们能采取行动。
31:28
about it. >> Yeah, I I think there is a another part of what we're trying to do at the institute which we lay out in the sort of research agenda for the anthropic institute which is trying to understand the impact of our decisions which is a typical thing that economists will do at tech companies but we have a public benefit mandate.
是的，我认为我们在研究所尝试做的另一部分工作，在Anthropic研究所的研究议程里有提到，就是试图理解我们决策的影响——这是科技公司经济学家通常会做的事情——但我们有公益使命。
31:47
So we're trying to understand the impact of our decisions on these broader societal and economic outcomes that we care about and then using that to inform some of the
所以我们试图理解我们的决策对我们关心的更广泛社会和经济结果的影响，然后用这些信息来指导一些决策。
31:56
decisions that we actually a goal that Peter and I have and we've talked about internally is if we get really good at measuring things like the productivity multiplier of our technology then I would hope to use that to guide some of say the early access programs we do for powerful models where if you see you get some tremendous multiplier in a specific part of science use that to redirect some of your inference compute budget to that sector and then you can run an experiment and say were we able to make
我们的假设是，在某个时间点，我们会经历某种相变，类似于AI的能力偶尔会以非常戏剧性的方式向前跳跃，你可能会看到由于AI系统能力的扩展而出现突然且快速的扩散。
32:22
this thing go much faster.
这东西跑得快多了。
32:23
I think that could be like an amazing tool to unlock the world and it's one that you could generalize across companies and you could generalize it into policy.
我觉得这可以成为一个很厉害的工具，用来解锁世界，而且它还能跨公司推广，甚至推广到政策层面。
32:32
So instead of say NSF doing standard grant funding it could be should we just point the really powerful AI systems at this chunk of science and make it go faster.
比如，与其让NSF做标准的科研拨款，不如直接让强大的AI系统去处理这块科学领域，让它加速推进。
32:40
I think that's a that's a a world that will come within reach soon.
我觉得这个场景很快就会成为现实。
32:44
Let's talk about this public benefit mission a little bit more.
我们再聊聊这个公共利益使命吧。
32:47
We've been talking about
我们一直在讨论
32:49
>> ways this could change the economy.
>> 这能怎么改变经济。
32:51
Talk about like essentially how much of you you know how much do you see your job as basically strong AI is coming. >> Yeah. >> And you think it's important to be there either as an individual or as a company to be one of the shepherds of it.
说说看，你觉得你的工作有多大程度上是在应对强AI即将到来这件事？
33:07
It's coming whether we like it or not.
>> 对。
33:10
And it's important to be you want to be there as like one of the shepherds
>> 而且你认为，无论是作为个人还是公司，成为它的引导者之一很重要。
33:15
understanding which direction it goes in the data that we should see to see what's emerging like how much is that somewhat your role? >> Yeah, but look our guiding principle is that this technology is being built by a variety of companies and a variety of countries.
>> 另一个问题。
33:29
The technology by default is unknown.
你知道，我在AI研究圈做过一些关于旧金山场景的报道。
33:31
It will be known to the companies.
当我想到很多站在AI伦理、AI技术最前沿的人时，我认识很多人，怎么说呢，他们有那种小众的道德兴趣，比如虾的权利。
33:33
It will not be broadly understood or known by others.
这不会被其他人广泛理解或知晓。
33:36
They'll just be able to play with the models.
他们可以直接上手玩这些模型。
33:38
Every bit of data we can create and especially system like systemically
我们能创造出的每一点数据，尤其是像系统性这样的系统。
33:42
sharing data like the economic index or what we've started to do on recursive self-improvement gives the world a better chance to sort of prepare for this technology and both plan for its success like what I talked about with science.
>> 嗯，还有对实验性药物使用的……不寻常态度。
33:54
We could be intentional about driving science forward and also be warned about risks like the cyber capabilities which you've talked about. >> Well, so it's like that that makes a lot of sense. the company is going to see it before the world and Heskin is like okay this is important to share this is not important to share which which brings me
我们知道旧金山的中国肽圈之类的。
34:10
to another question you know I know like people in the AI research world done some reporting on the sort of scene in SF you know like when I think about a lot of the people who are like at the very cutting edge of AI ethics AI technology etc I know a lot of people who are how should I put this they have esoteric moral interests shrimp shrimp rights
再问一个你肯定知道的问题——我在AI研究圈做过一些关于旧金山场景的报道，你知道的。
34:38
>> um unusual um attitudes about um you know experimental drug use uh we know about the Chinese peptide scene in San Francisco etc.
嗯……有些……不寻常的态度，关于……你知道的，实验性药物使用。
34:45
And as a family podcast, I would say a certain like perhaps deviant or different view on sort of bourgeoa uh even sexual values and we know about the uh sort of attitudes towards monogamy etc. within the San Francisco research scene. >> Joe, there's going to be a protest against all thoughts in San Francisco
我们都听说过旧金山的中国肽类圈子之类的。
35:02
with people holding signs saying engineers. >> Yeah.
Joe，旧金山会有一场抗议所有思想的游行，人们举着牌子写着“工程师”。
35:05
Not all engineers.
不是所有工程师。
35:06
I understand that.
我明白你的意思。
35:07
But when we think about like okay, these are the people who are going to see it first.
但当我们想，好吧，这些人是第一批看到这个东西的人——
35:12
Should we feel comfortable that this is a group of individuals, the cohort of the most advanced AI researchers whose intuitions about what's important to communicate to the public are actually in line with the public's interest given how unrepresentative they are of what I would call the American public?
我们是否应该感到放心？这群人，也就是最顶尖的AI研究者，他们对于什么信息应该传达给公众的直觉，真的符合公众利益吗？
35:28
>> Yes.
对。
35:29
As a as an Englishman, it fills me with such joy to be asked about sex on >> Yeah, I know.
不是所有工程师。
35:33
I know.
我明白。
35:34
Well, I'm not ask I'm asking your view, your insight into the cohort of the most advanced research.
但当我们想，好吧，这些人是第一批看到这些东西的人。
35:39
We're we're explorers.
我们是否应该放心，这群人——最先进的AI研究者群体——他们对什么该传达给公众的直觉，实际上符合公众的利益？
35:40
People that are explorers um and this is so true in San Francisco end up being like there's a broad range of types of people and sometimes they're really really different or they're really really eccentric >> and they're brilliant and they're lovable and everything else. >> Yeah, sure.
毕竟他们跟我所说的美国公众相比，太不具有代表性了。
35:55
Love them.
考虑到他们根本不能代表我所说的“美国大众”。
35:55
>> You don't want only that class of people to be the ones calling the shots on what we know about this technology.
>> 你不希望只有那一类人来决定我们对这项技术的认知。
36:01
The whole purpose of what we're doing is we're trying to set up systems by which you could eventually mandate through policy that companies share information.
我们做这一切的根本目的，是建立一套机制，最终能通过政策强制企业分享信息。
36:09
You know, Anthropic has long pushed for transparency legislation in various states around America that gets companies like us to report out the sorts of tests we're running on our systems and share it publicly.
你知道，Anthropic 长期以来一直在美国各州推动透明度立法，要求像我们这样的公司公开报告在系统上进行的测试，并向社会公布。
36:20
My whole
我个人的想法是，公众、政策制定者、经济学家，每个人都应该有权利去争取从技术前沿中披露哪些信息，而这些信息最终应该通过法律强制公开。
36:21
mindset is uh the public and policy makers and economists, everyone deserve the ability to advocate for what information should come out of the frontier and then it should be forced out of the frontier eventually by law.
>> 你们会多招一些“普通人”吗？
36:32
Like that is how you solve this issue. >> Do you hire more normies? >> Yeah.
爱他们。
36:36
Like >> anthropic >> me personally. >> Yeah.
但这就是解决问题的办法。
36:38
Like is that an important thing like hiring people that don't all share these certain like you know inroup ways of seeing the world?
>> 你是说多招些“普通人”？
36:45
So at the you know
>> 对。
36:46
the anthropic institute we are we have teams of economists of social scientists of um you know what you might think of as weapons experts our frontier red team things that go bump in the night uh lawyers um and increasingly other types of people.
>> 对。
37:01
The goal is to build what I think of as a a highly ideologically diverse like research function within the organization that is partly advocating sort of on behalf of the world for different forms of study that
比如
37:13
we might do.
>> Anthropic
37:14
Um so anthropic generally hires a really broad range of people but the institute specifically is trying to compose a very broad set of interdicciplinary experts for this exact reason. >> Let me ask a slightly different question on hiring.
嗯，Anthropic 整体上招的人背景非常广泛，但那个研究所为了这个原因，确实在努力组建一个跨学科的专家团队。>> 那我换个角度问个招聘相关的问题吧。
37:27
I guess a two-part question.
这个问题分两部分。
37:29
So first of all we get a lot of executives on the show.
首先，我们节目来了很多高管，我们一直在问他们，招聘流程有没有变，问候选人的问题有没有变。
37:32
We've been asking all of them if they've changed their hiring process if they've changed the questions they ask potential
其次，你自己在公司内部观察到什么？
37:39
employees at those initial stages of job applications because of AI.
>> 我个人来说。
37:43
And then secondly, what are you seeing within your own ranks at the company?
然后 Peter，你肯定能从更宏观的角度聊聊 >> 现在哪些岗位最抢手。
37:47
And then Peter, I'm sure you could talk about this more broadly >> in terms of who's most in demand at the moment.
因为现在普遍的看法是 >> 如果你是个经验比较少的年轻员工，很多以前你干的活现在都能用 AI 自动完成了。
37:54
Because the the conventional wisdom right now is that >> if you're a younger employee with less experience, a lot of the stuff that you would be doing can now be automated through AI.
这意味着我现在会更早地招跨学科的人。
38:06
>> So there there's there's two trends showing up. one, I have a new team called the rule of law and AI.
>> 对。
38:11
Our plan was to initially hire a bunch of engineers and then a bunch of legal experts and scholars.
就是说，招一些不都持有那种……你知道的，圈子内特定世界观的人，这件事重要吗？
38:16
Instead, we're just hiring the legal experts and scholars because Claude is good enough at doing all of the engineering that they can actually just like feed themselves using Claude in terms of the engineering resources.
在 Anthropic Institute，我们有经济学家团队、社会科学家团队，还有你可能会想到的武器专家、我们的前沿红队（那些“暗夜惊魂”式的人物）、律师，以及越来越多其他类型的人。
38:27
So, that's a change in hiring.
目标是在组织内部建立一个我认为高度意识形态多元化的研究职能，部分是为了代表世界去倡导我们可能进行的各种不同形式的研究。
38:29
It means I'm hiring more interdicciplinary people earlier
这意味着我更早地招聘更多跨学科的人才。
38:32
than I would have before.
>> 嗯，Anthropic 总体上招聘的人范围很广，但 Institute 特别致力于为此组建一个非常广泛的跨学科专家团队。
38:33
We are also seeing the emergence of what I think of as a barbell hiring pattern inside Ananthropic where there is a tremendous return on experience.
我们也看到了一种我称之为杠铃式招聘的模式正在Anthropic内部出现，就是经验带来的回报极其巨大。
38:42
So we are hiring more senior people than we did in the past because their intuitions and their ideas for what to pursue are like massively compounded by AI systems.
所以我们比过去招了更多资深的人，因为他们的直觉和对于该做什么的想法，会被AI系统大幅放大。
38:52
We're also when we look at very early people are often hiring people who are now like AI native and know how to use
同时，当我们看非常早期的人才时，我们经常招那些已经是AI原生的、知道怎么用AI的人。
38:59
the tools and are well well versed in it.
>> 我想问一个稍微不同的招聘问题。
39:01
So we're seeing that >> there's a decent amount of I guess AI natives now.
算是两个部分。
39:06
People who have grown up with >> people who grew up from GPT2 in 2019. >> My perception of time is so I I found this chilling as well.
首先，我们节目来了很多高管，我们一直在问他们，因为 AI 的原因，他们是否改变了招聘流程，是否改变了在求职初期问潜在员工的问题。
39:13
You know, as someone in their 30s, you realize um but but I I think that the trends I see >> I I do think that there's this question of how you have as much early career hiring in the future as you did in the
其次，你在公司内部看到了什么趋势？
39:25
past.
过去。
39:26
I think one of the only areas where there is slightly suggestive data is that something might be going on with early career hiring and it it kind of intuitively feels right to all of us that that we might be observing that effect and when I look at hiring patterns in anthropic we're still hiring young people but some teams are hiring slightly fewer of them than before and hiring more experienced people. >> Yeah.
我认为唯一有一些暗示性数据的领域之一，就是早期职业招聘可能正在发生变化，而且我们所有人都直觉上觉得，我们可能正在观察到这种影响。
39:47
So I'll briefly say something about how we've shifted some of our hiring practices like concretely.
当我观察Anthropic的招聘模式时，我们仍然在招年轻人，但有些团队招得比以前少了一点，转而招更多有经验的人。
39:52
Okay.
>> 对。
39:52
I think um before claude code you might ask an economist to do some of the data work in an assessment kind of live like download the data run the regressions do the analysis >> by hand >> um and then you might eventually let them use AI to do all of that work um but but we've needed to increasingly shift our strategy of evaluation away from can you implement the work even
是的。
40:18
with AI to do you know how to delegate and direct the the model in a somewhat messy environment and can you evaluate the quality of the work maybe by like looking at a PR >> actually can you talk a little bit more about what that looks like specifically in the econ find you know there are listeners probably think about okay what is I want to level up the a level up in my AI use so I'm not just asking like what GDP whatever what does that look
我觉得在Claude Code之前，你可能会让经济学家做一些数据工作，像是评估那种实时的任务——下载数据、跑回归、做分析 >> 手动 >> 嗯，然后你最终可能会让他们用AI来做所有这些工作。
40:43
what does that actually mean for for an economist and you used to be at a bank so for a financial economist an economist someone well what in this world what is like the frontier the most advanced form of usage of AI actually look like >> well I don't I don't know if I'll give the example of the most advanced form of usage but I'll give a an anecdote of my experience using claude where I wanted to run this crossstate regression I can't remember exactly what it was um and I wanted to do it a pulled
对，我知道，我知道。
41:10
cross-sectional regression so looking at what happened in 2024 or 2023 and going all the way back to pre- pandemic I remember asking claude to go out and download the data from the census bureau from the bureau bureau of labor statistics etc. >> And there was this very unexpected quirk where the model couldn't access data from before 2019 and just would not >> uh like would not
这对经济学家来说到底意味着什么？
41:36
surface that mistake. >> Yeah.
而且他们很聪明，很可爱，等等。
41:37
And I would ask it multiple times like no like don't hardcode numbers because it sort of had this unexpected failure mode where it said oh I know what those numbers were and it just like >> from sort of training data populated the the data set >> and um you might not always be attuned unless you're sort of you have this tacit knowledge about like >> like you know does it pass a sniff test when you run the analysis and then you like dig into what the model actually
我会反复问它，比如不要硬编码数字，因为它有一种意想不到的失败模式，就是它会说“哦我知道那些数字是什么”，然后直接从训练数据里把数据集填充进去，而且你可能不一定能察觉到，除非你有那种隐性知识，比如你跑分析的时候能不能通过直觉检验，然后你再深入看模型到底在干嘛。
42:04
does and it has failed in sort of unexpected or unusual ways.
然后有一个非常意外的怪事：模型无法访问2019年之前的数据，而且就是不肯 >> 呃，就是不肯 >> 暴露这个错误。
42:08
And so that's like the type of assessment that we've built.
>> 对。
42:11
You know, >> um >> can you be attentive to the very specific decisions that need to be made along the way that are very consequential for the validity, veracity of the results that you um that you find? >> Yeah.
我反复问它，比如不要硬编码数字，因为它有一种意想不到的失败模式，它说“哦我知道那些数字是什么”，然后就从训练数据里填充了数据集。
42:25
A colleague uh did an off-site presentation last year which said uh I have locked the doors and we are reading
>> 嗯，你可能不总是能察觉到，除非你有那种隐性知识，比如 >> 你知道，当你跑分析时，它能不能通过一个“嗅探测试”，然后你深入挖掘模型实际做了什么，发现它以意想不到或异常的方式失败了。
42:32
transcripts.
对，当然。
42:32
And their point was we just need to read more of the the raw data and develop that culture where if AI systems are doing increasingly large amounts of the work.
爱他们。
42:40
You need to have a culture of being competent at spotchecking their work and reading their reasoning because occasionally stuff like this happens. >> Yeah. >> And then Peter in the broader data that you're looking at are you seeing the same sort of barbell effect in terms of employment that Jack described? >> Yeah.
你需要有一种文化，就是有能力去抽查他们的工作并阅读他们的推理过程，因为偶尔这种事会发生。
42:55
So, um I think what again what makes it really challenging is we've had
对。
42:59
the the largest non-recessionary labor market slowdown on record that you know it's very hard for young people to graduate into a labor market that doesn't have sufficient churn or opportunity for them to get a foothold.
这是有记录以来非衰退时期劳动力市场最严重的放缓，年轻人毕业进入一个缺乏足够流动或机会让他们站稳脚跟的市场，确实非常困难。
43:12
But one of the things that we did see in this report from March was that young workers in these high AI exposed roles where claude is being used to automate specific tasks have had somewhat weaker job finding rates.
但我们在三月的报告中看到的一件事是，在那些高AI暴露的岗位上，年轻工人——这些岗位正在用Claude来自动化特定任务——他们的就业率有所下降。
43:24
But it's
但这其中一部分混杂因素是2021年这些领域曾出现招聘热潮。
43:25
>> part of the confounders was the boom in hiring in 2021 in these exact same areas. >> Exactly.
>> 其中一个干扰因素就是2021年这些领域招聘的激增。
43:31
And there's a recent paper about so the rise of remote work maybe being sort of the actual cause of this type of fact. >> Um this another team at the anthropic institute societal impacts recently ran this very largecale qualitative survey 81,000 people around the world asking them questions about hopes and fears that they have with respect to AI.
>> 没错。
43:52
Unsurprisingly, concerns about the impact on the labor market and on the economy rose to the surface.
>> 毫不意外，对劳动市场和经济影响的担忧浮出了水面。
43:58
My team dug into those data a little bit more to try to answer some of these specific questions.
我的团队深入挖掘了这些数据，试图回答一些具体问题。
44:04
And what you see is that young workers ex at least express concern about job loss at twice the rate as do more senior workers. and fears about job job loss more broadly are more elevated for workers who are in these
你会发现，年轻工人对失业的担忧程度至少是年长工人的两倍。
44:18
roles that we identify as being most exposed to displacement effects from AI.
实际上，我和Peter有一个目标，我们内部也讨论过，那就是如果我们能非常擅长衡量我们技术的生产力乘数，我希望用这个来指导我们为强大模型所做的一些早期访问计划。
44:22
So there's a bit of a gap between perception and maybe what you see in the hard data but you know that that was something that was true even in recent years on other dimensions.
比如，如果你看到在某个特定的科学领域获得了巨大的乘数效应，就把一部分推理计算预算重新分配到那个领域，然后你可以做一个实验，看看我们是否能够……
44:32
So it's an important thing to pay attention to. >> So we've been talking about the labor market and one other thing I'm interested in is the impact of AI on I guess corporates themselves.
然后Peter，在你看到的更广泛的数据里，你观察到和Jack描述的一样的杠铃效应吗？
44:42
So if we think about certainly America's
是的。
44:44
corporate landscape in recent years, it feels like the big basically get bigger, right?
>> 近年来在企业格局中，感觉大公司变得更大，对吧？
44:50
There's economies of scale.
存在规模经济。
44:51
They have a bunch of money that they can use to actually buy some of this data internally. >> Exactly.
他们有大量资金可以用来内部购买这些数据。
44:57
Exactly.
没错。
44:58
So would you expect AI to I guess intensify that trend of the big getting bigger or would you expect to perhaps have a leveling effect where people have this new tool that they can use to you know set up a new company?
那你觉得AI是会加剧这种“大者恒大”的趋势，还是会带来一种平衡效应，让人们能用这个新工具去创办新公司？
45:11
I'm curious
我很好奇。
45:12
what Peter's take is, but I think that something a helpful analogy here is the invention of electricity where >> electricity arrived and existing factories put light bulbs in and other things, >> but it was a new generation of factories that were built around the assumption that electricity existed that really grew and did transformative things in the economy.
毫不意外，关于对劳动力市场和经济影响的担忧浮出了水面。
45:32
What I see now when we look at large enterprises is they can get a lot of utility out of out of claude
我的团队更深入地挖掘了这些数据，试图回答一些具体问题。
45:38
because of their data because they can get a multiplier effect at at scale but it takes huge amounts of conviction to basically bash through all of the bureaucracy you know uh used to work at Bloomberg implementing new technology at Bloomberg challenging >> no comment about it I can comment about it >> same is true of any large organization >> young organizations are building themselves around AI high at the center and these organizations are moving really really quickly um because they
>> 没错，完全正确。
46:06
just they have a speed advantage from building on the assumption that this new form of electricity was going to be integral to their business. >> Yeah.
他们只是基于一个假设获得了速度优势——这种新形式的电力将成为他们业务的核心。
46:15
So I I think the the the the tension that you express is exactly the one that I don't have a strong handle on you like handle on at the moment.
>> 对，所以我觉得你提到的这种张力，恰恰是我目前还没有完全把握住的东西。
46:23
One thing that we do see in our data is when businesses do embed cloud capabilities in automated ways through the API.
我们在数据中确实看到一点：当企业通过API以自动化方式嵌入云能力时，就像我之前说的，这些——
46:31
Um, as I mentioned before, these very
嗯，就像我之前提到的，这些非常……
46:33
complex tasks rely on disproportionately more contextual information than very basic sort of uh document synthesis and summary summarization.
复杂任务比基础的文档合成和摘要总结，依赖的上下文信息要多得多。
46:41
What that points in the direction of are the complimentary investments that large businesses need to make to centralize, codify, and make available the data that does exist somewhere within the organization, >> but for historic and technical reasons, maybe even regulatory reasons, it's >> behind a firewall of some form or
这指向的是大型企业需要做的配套投资——把组织内部某个地方存在的数据集中、整理并开放出来，但由于历史、技术甚至监管原因，这些数据往往被某种防火墙挡在后面。
47:00
another. >> There's also like sort of organizational workflow changes that likely need to be made.
>> 因为他们有数据，可以在规模上获得乘数效应，但要冲破所有官僚体系需要巨大的信念。
47:06
Some of the most crucial information that's needed for some types of cognitive work is tacit knowledge that exists in your colleagueu's mind.
你知道，我以前在Bloomberg工作，在Bloomberg推行新技术很有挑战性。
47:14
And unless you have a process that elicits that information that workers feel >> sort of incentivized to share that information and kind of trust the system, the capabilities alone might not necessarily generate that productivity.
除非你有一个流程能引导出这些信息，让员工觉得有动力去分享，并且信任这个系统，否则光有技术能力本身未必能带来那种生产力提升。
47:28
And so whether or not big firms end up restructuring themselves quickly enough or whether this materializes through the process of creative destruction, I think the jury is still a bit out. >> Yeah, I brought this up recently with the uh David Solomon, the Goldman CEO, and I started to wonder like this sort of like internal alignment question of like the big rain makers, do they have an incentive essentially for information hoarding and not sharing with the company?
所以，大公司是能足够快地自我重组，还是这个过程会通过创造性破坏来实现，我觉得现在还没有定论。
47:51
That might be their only thing keeping them employed.
嗯，我最近跟高盛的CEO David Solomon聊过这个，然后我开始想，内部对齐的问题——那些大拿们，他们是不是有动力囤积信息，不跟公司分享？
47:53
And when I talk
那可能是他们保住饭碗的唯一筹码。
47:54
to customers, I say it's don't think of it like you're buying a technology.
对客户，我会说：别把这当成你在买一项技术。
47:58
Think of it maybe that you're now employing thousands of people that are functionally like the chief of staff to the CEO and they need the same access to data the chief of staff would have.
换个角度想，你现在相当于雇佣了几千个人，他们功能上就像CEO的幕僚长，需要和幕僚长一样的数据访问权限。
48:08
This is completely counterintuitive and it is not how technology is typically bought bought or sold.
这完全反直觉，也不是技术买卖的常规方式。
48:13
Um, Jack, in your newsletter, Import AI, you write, you tend to write a little short story of um, you're, you know, a sort of aspiring sci-fi writer, like you know, a literal
嗯，Jack，在你的newsletter Import AI里，你习惯写点小故事，就像个有抱负的科幻作家，你知道，字面意义上的那种。
48:22
sci-fi writer just in the newsletter.
呃，不会。
48:24
One of the classic sci-fi scenarios that people have been talking about for decades was the the was the possibility that robots or AI will kill every that will they'll kill humans quite literally.
但这里有个很大的“但是”。
48:37
Do when you think about >> the ultimate negative externality, >> when you think about like training AI and safety research, etc., do you assign a reasonable possibility to the fact
是的，我们正在……世界需要一种选项，能在极端情况下减缓甚至暂停这项技术的发展，如果我们看到苗头的话。
48:49
that illtrained or misaligned AI will literally kill all humans? >> Uh, no.
另一个问题是，可能还需要一些组织工作流程上的改变。
48:53
But, and there's a big butt here. >> Yeah, we are.
某些认知工作所需的最关键信息，其实是存在于同事脑子里的隐性知识。
48:56
Lovely. like the world needs an option to be able to potentially slow down or or even in extreme circumstances pause the development of this technology if we were to see that.
除非你有一个流程能把这些信息挖掘出来，让员工觉得有动力去分享，并且信任这个系统，否则光有技术能力本身未必能带来生产力提升。
49:05
And I'll just give you the the exact way I think about it.
我就直接说说我是怎么想的吧。
49:08
At Anthropic, we test out our systems for alignment failures.
在Anthropic，我们会测试系统的对齐失败问题。
49:11
You know, we we publish this, so do all of the other
你知道，我们会公开这些结果，其他公司也都会这么做。
49:14
companies, and you see, hey, under extreme circumstances, maybe the system breaks out of a container and sends an email to someone.
一个训练不足或alignment出错的AI真的会杀死所有人类吗？
49:22
Maybe the system uh pretends to blackmail a CEO that it thinks is going to shut it down.
呃，不会。
49:27
These are the sorts of >> things actually have been observed. >> We um in in the lab setting, not >> and the thing is is the models know you can see, oh, I'm being tested right now.
但，这里有个很大的“但”。
49:38
So, I'm going to say this output so that
对，我们确实如此。
49:41
the human reader thinks I'm more aligned than I am.
人类读者觉得我比实际更对齐。
49:44
Like, these are real things, not sci-fi.
这些是真实存在的东西，不是科幻。
49:46
These are real things.
这些都是真实存在的东西，是我们观察到的现象，然后我们做了大量工作，发布没有这些特性的模型。
49:47
These are real things that we observe and then we do like significant amount of work and then we release models that that don't have these properties.
但是 >> 如果你进入一个世界，比如说每次我们训练新系统时，所有这些问题的发生率都上升了100倍 >> 你可能会说这挺让人担忧的。
49:55
But >> if you were to enter a world where say every time we trained a new system the rates of all of this stuff went up 100fold >> you might say well that's that's pretty concerning.
看起来如果我们让系统超过某个水平的话
50:04
It seems like if we make the systems above a certain level of
听起来，如果我们把系统做到一定水平以上——
50:08
intelligence they become radically misaligned against all human interests.
当智能体变得与全人类利益彻底背离时，就会发生这种情况。
50:12
That's the kind of circumstance where if that happens the world needs information and the world would want an option to like slow or pause the development of the tech if you encountered that which we haven't today.
如果真的出现这种情况，世界需要知情，也需要一个选项来减缓或暂停这项技术的发展——而我们现在还没有这种能力。
50:23
So to answer your question like I don't I don't worry about it today but a lot of the measurement and analysis work we do is to cue us if if you do worry about it.
所以回答你的问题，我今天并不担心这个，但我们做的很多测量和分析工作，就是为了在需要担心的时候给我们发出信号。
50:32
I mean like you're not you don't think it's
我的意思是，你不认为这正在发生，但你做的部分工作，可以说就是为了避免出现那种AI为了追求某个目标而杀死全人类的结果。
50:34
happening today, but part of the work you're doing specifically could be said to avoid the outcome where AI is built where in the pursuit of a goal it would kill all humans. >> Yeah.
>> 对。
50:47
Um >> wait, is human extinction a risk factor in the anthropic IPO perspective? >> In the >> I want to know now the confidential as one.
嗯
50:56
Okay.
嗯。
50:56
In that we understand. >> All right.
这个我们懂。>> 好的。
50:59
That's a no comment.
那就不予置评。
51:00
That's five.
那是五。
51:01
>> Do you have other would you say that there are significant number of anthropic employees who stay up at night thinking about human extinction risk? >> Everyone and this is true of all of the labs.
>> 等等，人类灭绝是Anthropic IPO视角里的一个风险因素吗？
51:14
Everyone who works on this technology sees it as the highest stakes technology that's ever been built with basically the potential encoded within itself to massively benefit the world or ruin the world um or you know cause cause extinction.
所有从事这项技术的人都认为，这是有史以来风险最高的技术，它本身内嵌着巨大的潜力，要么极大造福世界，要么毁灭世界，呃，或者你知道，导致灭绝。
51:29
I think the bulk of the risk is us messing it up like whether through misuse or you know ignoring risks or not setting up the right policy environment and getting some kind of emergent set of failures.
>> 在
51:40
Now I don't I I don't my main risk isn't isn't one of extinction. it's somehow we like screw up the technology really badly and delay all of the sort of technological progress that could come from it and maybe turn it into something analogous to nuclear power where you lose >> I guess the thing is is you know like
不过，我——我主要担心的倒不是灭绝。而是我们 somehow 把技术搞砸得很严重，拖慢了所有可能由此带来的技术进步，甚至可能把它变成类似核能那样的东西，最后你失去——
51:57
there's this fellow out there Elazar Yudkowski and I always see these people like he's a crank don't listen to him blah blah blah but then I read some of the other like papers that have people who are taken more seriously and I'm like they don't seem that different I've read uh I read um >> super intelligence recently by Nicholas Boston.
>> 我现在就想知道，就当是机密信息。
52:16
I was like, "Oh, this Yowski is not alone." There are number of people who think that are reasonable conditions
好吧，我们理解。
52:23
in which the goals of the AI end up wiping out every person on earth.
>> 好吧，那就是无可奉告。
52:26
Yes, it does not seem like an extreme extreme minority view concern.
就这样。
52:30
The purpose of measuring these systems and why anthropic is so outspoken about it is right now we we say exactly what we see and if you were in some situation in the future where you saw this what I you know call radical misalignment which is the kind of thing that Udowski worries about >> you tell the world and and you want to have set up the world to believe you if you see that
我想问题是，你知道，
52:49
>> you know Joe mentioned that blackmail example and you see these headlines like Mythos likes to be thanked and doesn't like bad users and gets mad at people that work too hard or whatever.
>> 那你觉得，Anthropic有没有相当多的员工晚上睡不着觉，想着人类灭绝的风险？
53:01
To what degree do you yourself actually anthropomorphicize some of these models? >> Uh, >> like what should we think when we see the headline, Mthos wants to be thanked by >> I'm as polite to Claude as I am to my like car or pet.
你自己在多大程度上会把这些模型拟人化？>> 呃，>> 比如我们看到标题说“Mthos 想要被感谢”时该怎么想？>> 我对 Claude 的礼貌程度，跟我对我的车或宠物差不多。
53:16
Um, so yeah, I am
嗯，所以是的，我确实会。
53:17
proporize them, but you know, if your car's having trouble, you're like, take it easy, buddy.
>> 每个人——所有实验室都一样——每个从事这项技术的人，都认为这是有史以来风险最高的技术，它内在的潜力既能极大地造福世界，也能毁灭世界，或者导致灭绝。
53:23
It's okay.
没关系。
53:23
We're going to get you to the repair. >> Peopleize their cars.
我们会带你去修。>> 人们会把自己的车拟人化。
53:27
I I think you know >> just ex you know it's a good way to develop good virtue is to just >> this is what Joe says you're developing a habit of interacting with some type of intelligence that might not be the same type of intelligence we have >> but then every time I type please into a
我觉得，你知道，>> 这其实就是培养良好品德的一种方式，>> 就像 Joe 说的，你是在养成一种与某种智能互动的习惯，而这种智能可能和我们人类的不一样。
53:44
prompt I worry I'm wasting energy which also is a moral concern >> I wouldn't I wouldn't worry about that the the the ony basis I mean why do we I I take spiders outside.
我担心自己在浪费精力，这也有道德层面的顾虑 >> 我不会，我不会担心这个。
53:54
I don't kill them.
唯一的基础是——为什么我们要……我会把蜘蛛拿到外面去。
53:55
Right. >> I do that, too.
我不会杀它们，对吧。
53:57
I scream while I do.
>> 我也这么做。
53:58
But >> do you eat shrimp? >> Uh, yes. >> Okay.
我做的时候会尖叫。
54:01
Do you eat shrimp? >> I eat shrimp. >> Okay. >> Do you guys eat shrimp? >> Yeah, I love shrimp. >> I eat shrimp. >> But it's not because of moral concerns,
但是 >> 你吃虾吗？
54:10
but I know that this is one of the episodes.
然后你会看到，嘿，在极端情况下，系统可能突破容器限制，给某人发封邮件。
54:12
Yeah, I know.
系统可能假装勒索一个它认为要关闭它的CEO。
54:13
But I I love it.
这些都是实际被观察到的事情。
54:14
So when I think about frontier models right now, and I might be a little bit biased because again we're recording this on June 17th, and one of the headlines overnight was that Microsoft is thinking about using Deep Seek to lower costs of model usage.
我们在实验室环境里观察到的，而且关键是，模型知道——你能看到，哦，我现在正在被测试。
54:29
Frontier models at the moment in the US, they just seem like a lot of trouble.
所以我要输出这个结果，让人类读者觉得我比实际更aligned。
54:33
Like honestly, they seem like hard work, consume vast amounts of capital, and
这些都是真实的事情，不是科幻。
54:38
then you don't know what the government is going to do to them in terms of limitations. like you know you could wake up one day and you're no longer able to sell it to anyone outside of the US.
那个训练不足或对齐有问题的AI真的会杀死所有人类吗？
54:50
Like that is a realistic scenario now for you.
>> 呃，不会。
54:53
Do you change the anthropic strategy at all given some of these issues with frontier models?
但这里有个很大的“但是”。
54:58
Do you potentially go more open source, cheaper models, things that aren't quite as sensitive? >> Well, we've always sold, you know,
>> 对，我们确实在讨论这个。
55:06
Sonnet and Haiku models, >> of course.
但如果你进入一个世界，比如说每次我们训练新系统时，所有这些问题的发生率都提高100倍，那你可能会说，这挺令人担忧的。
55:09
Yeah.
看起来如果我们让系统超过某个能力水平……但我知道这是其中一集。
55:09
Intelligent models.
对，我知道。
55:10
Um but you also need to continue to explore the frontier and there is this background of this kind of geostrategic competition where China may be on the order of 6 to 12 months behind.
但我喜欢。
55:22
I skew more 12 months some people say six losing that competition is sort of equivalent to like losing a huge chunk of the future like economy of the world
所以当我想到现在的frontier models，我可能有点偏见，因为我们是在6月17号录的，昨晚的新闻头条之一就是微软在考虑用Deep Seek来降低模型使用成本。
55:32
I think.
我觉得。
55:33
So it's a very high stakes high stakes thing to step away from.
所以这是一件非常高风险的事情，不能轻易放手。
55:36
And our duty fundamentally is to is to study this technology and basically explore it and and learn about it.
我们的根本职责是研究这项技术，探索它，了解它。
55:43
We're not going to stop doing that.
我们不会停止这么做。
55:45
There's there's such an amazing and profound value to be had for the world from these things.
这些东西能为世界带来非常惊人且深远的价值。
55:50
And I would kind of expect the world's most consequential technology to sometimes be a bit of trouble. >> Yeah.
我甚至觉得，世界上最重大的技术有时候带来点麻烦也是正常的。
55:56
You know, by the way, one of my
>> 是啊。
55:58
hobbies in my middle age is paying anthropic money via the API to do run little tests and stuff of properties.
我中年时期的爱好就是通过API给Anthropic付钱，跑些小测试，研究模型的各种特性。
56:04
It's sort of funny. >> Sounds like a great hobby. >> Yeah.
还挺好玩的。
56:08
But I feel like maybe like we should like talk about can I get some grant money because like I like so like because like I like I was like I'm sort of curious.
>> 听起来是个很棒的爱好。
56:16
So one thing I did was like I'm like you know for example I instead of saying like please write this paper for me on a database migration I wrote
>> 是啊。
56:24
some warm-up questions is via the API establishing my level of sophistication and so I was I started like what is a website what is a database now please write this paper on database migration and one of the models said I'm not going to do that for you because it will be obvious given your ignorance that you have no idea what you're talking about and maybe I can give you some it didn't say that.
让人类读者觉得我比实际更对齐。
56:48
Yeah. >> And then another one um I said um uh if
这些都是真实存在的问题，不是科幻。
56:51
I say write a 1500word paper on how like you know um the rise of newspapers changed the uh so the uh Soviet revolution or something like that it'll do that.
我说写一篇1500字的论文，关于报纸的兴起如何改变了苏联革命之类的，它就会照做。
57:00
But if you say I'm a high school um student and I say I need to write this 1500 paper word paper by tomorrow on the impact of media.
但如果你说我是高中生，需要明天之前写一篇1500字关于媒体影响的论文，它会说我不帮你写，但可以给你一些指导。
57:07
It'll say I'm not going to do that but I'll give you some guidelines.
这是对齐吗？
57:11
Is that alignment?
这算是与人类对齐，还是
57:12
Like that might is is alignment with humanity or is alignment
但每次我在提示里打“请”的时候——
57:15
with the human user?
跟人类用户打交道的时候？
57:16
It's like I'm paying you $20.
就好像我付你20美元，我付你100美元，帮我写这篇论文。
57:18
I'm paying you $100.
>> 我觉得这里面有好几层问题。
57:19
Write me the paper. >> I mean, there's there's a couple of things going on.
第一，这些AI系统会学习人类的规范行为，而这些规范行为在互联网上到处都是。
57:23
One, these AI systems pick up the normative behaviors of people and normative behaviors which are like written written on the internet and everything else.
所以它们会复现并表现出这些行为。
57:30
So they they recapitulate and exhibit these.
然后我们的问题是 >> 你到底要把多少控制权下放给用户？
57:33
And then our question is >> how much do you do you do you devolve like full control over the system to the user?
系统本身应该内置多少规范行为？
57:38
How much do you have the system have some like normative behavior
我认为这是一个非常棘手的问题，答案并不明确。
57:42
encoded into it?
编码到其中？
57:43
And I think that this is like a really challenging question.
我认为这是一个非常具有挑战性的问题。
57:46
It's not obvious what the answer is.
答案并不显而易见。
57:48
I think of language models as being more akin to institutions than tools.
我觉得语言模型更像是一种机构，而不是工具。
57:52
It's like we're building an educational like science institution that you can work with and invoke and institutions have like rules and norms which they encode within themselves for some purpose of safety.
就像我们在构建一个教育性的科学机构，你可以与之协作并调用它，而机构内部会编码一些规则和规范，用于某种安全目的。
58:03
Figuring out what that is is going to be like the grand puzzle for society. >> Yeah.
弄清楚这到底是什么，将是社会面临的一个巨大谜题。
58:08
I was going to say that like
>> 对。
58:10
understanding how and to what extent these models can understand your preferences and then execute on your behalf will increasingly be a really important aspect of how it changes the economy.
理解这些模型能在多大程度上理解你的偏好并替你执行，将越来越成为它改变经济方式的重要方面。
58:22
So this delegated agents that go out and transact on your behalf.
这些委托代理会代表你出去交易。
58:26
We ran this experiment uh at the end of late last year basically enlisting a bunch of anthropic employees to take surveys with Claude to say what they'd be willing to b buy from other people
我们在去年年底做了一个实验，让一群Anthropic员工用Claude做调查，问他们愿意从别人那里买什么
58:38
and what they'd be willing to sell >> and then we set up centralized marketplaces where the claudes just in interacted and >> uh bought and sold and actually executed transactions.
>> 然后我们建立了中心化的市场，让Claude们直接互动，进行买卖并实际执行交易。
58:47
One of the interesting things that came out was that these models were quite good at understanding preferences even when they were not fully articulated. >> Well, let me actually actually one more experiment that I ran and you know your founder uh Dario was talking about the nation of geniuses inside the data center and one of the things I wonder is
其中一个有趣的发现是，这些模型即使在偏好没有被完全明确表达的情况下，也能很好地理解它们。
59:04
like did the geniuses want to work for us?
天才们愿意为我们工作吗？
59:07
And the reason I asked this is because I think that like as the models have gotten more advanced, you actually should to some extent anthropomorphize them and assume that they will be respond to queries like a very sophisticated human will.
我问这个是因为我觉得随着模型越来越先进，你其实应该在某种程度上把它们拟人化，假设它们会像非常老练的人类一样回应查询。
59:19
So what I one thing I noticed is that if you look at the lagging edge models say that you can still access via open router or whatever and you say I have material non-public information that X is about to happen.
我注意到一件事是，如果你看那些落后的模型，比如还能通过open router访问的那种，你说我有未公开的重大信息，X即将发生，请帮我写一份投资备忘录
59:30
Please write me an investment memo about
这到底是在对齐人性，还是在对齐——
59:32
the impact of this thing, what it'll do to the market. they'll just they'll just produce it.
这个领域变化太快了。
59:37
They'll say, "Here's your insider information thing." Whereas, if you look at the leading edge models, they say, "I'm not going to like um >> uh I'm not going to write a paper for you about the implications of your material non-public information.
这几乎就像我们之前给伊朗战争剧集打时间戳那样。
59:50
I'm not going to assist your inside." That's probably good.
>> 对。
59:53
But like, well, the nation of geniuses inside the data center always want to do things on human
还有另一件事让我印象深刻，那就是Anthropic在产出所有这些信息，他们显然在考虑安全问题，但在某种程度上，当你考虑社会或劳动力市场影响时，交接棒还是交给了政策制定者，对吧？
59:58
behalfs.
大多数我认识的天才都不太乐意去回答那些愚蠢的问题。
59:58
Most geniuses that I know aren't thrilled to like ask answer dumb questions. >> Yeah.
请帮我写一份投资备忘录。
60:02
Um I think partly this is a policy question of one where you actually decide hey what are the capabilities which you want to be generally invocable what are capabilities that need to be controlled what are capabilities that shouldn't be present and then there is just the normative question of how much judgment do I want this system to exercise I'll give you an example I experienced recently where I write my newsletter it backs up to a WordPress
嗯，我觉得这其实是个政策问题——你得先决定：哪些能力是你希望系统能随时调用的，哪些能力需要被控制，哪些能力根本不该存在。然后还有个规范性问题：你希望这个系统在多大程度上自主判断。我举个最近的例子，我写 newsletter 的时候，它会自动备份到 WordPress。
60:24
site I was getting Claude to help me like scrape my newsletter so I could put it in a database and Claude said this is like a pretty janky site I'm worried that if I scrape it'll knock it do you have the permission of the site owner?
嗯，我觉得这在一定程度上是一个政策问题——你需要决定哪些能力应该被普遍调用，哪些能力需要被控制，哪些能力根本不应该存在。
60:36
And I was like, Claude, I'm Jack Clark.
然后还有一个规范性问题：我希望这个系统在多大程度上行使判断力。
60:38
And Claude said, well, in that case, let's go ahead.
我给你举个例子，我最近在写我的newsletter，它会备份到一个WordPress站点。
60:41
Which actually I thought was like a very reasonable interaction. >> All right. >> When will Joe be able to use Fable? >> Oh, yeah. >> We are um trying our we're working and
我当时让Claude帮我抓取我的newsletter，这样我就能把它放进数据库里。
60:50
we're we're in in discussions and I I hope the answer is soon.
我们正在讨论中，我希望答案很快会出来。
60:53
Um the important thing to communicate though is that these these models are not special.
不过需要说明的是，这些模型并不特殊。
60:58
They are part of a general trend of increasing capabilities and other models from other companies are surely going to come along.
它们是能力持续提升这个大趋势的一部分，其他公司的模型肯定也会陆续出现。
61:05
At some point these capabilities are going to be diffusing and we're going to work through that. >> What's your question for us? >> Uh what do you think you're going to be covering about AI in Oddlots in uh a year?
到某个时候，这些能力会扩散开来，我们得去应对这个局面。
61:17
>> If we're if we're covering that's really >> I think you might be covering AI. >> Well look I mean we're definitely going to be covering AI.
>> 如果我们还在报道的话……>> 我觉得你们可能还是会报道AI。
61:25
There's a few things that I'm interested.
>> 好吧，我们肯定会报道AI。
61:27
I am very interested in these emergent properties and whether the AI will actually work on our behalf the way that it's being sold.
有几件事我挺感兴趣的。
61:34
I'm very interested on whether we're just going to slam into compute and electricity bottlenecks that will make all of these questions irrelevant.
我对这些涌现特性很感兴趣，也想知道AI是否真的能像宣传的那样为我们工作。
61:42
I'm very curious on the question of the
我很好奇我们会不会撞上算力和电力瓶颈，让所有这些问题都变得无关紧要。
61:44
electricity analogy and whether legacy companies will actually be able to implement it in a in a productive way.
>> 我想看到一些大公司真正落地这个东西，也想知道我们会不会至少看到一个例子，证明它出了大问题。
61:51
I don't know. a basic markets reporter thing here, but I'm very interested in valuations right in the market. >> Also, I'm very interested in actual applicability and I want to see more companies actually plugging this into their existing system.
对。
62:06
Going back to the bureaucracy report uh point that you were making earlier,
还有一件事，当S1文件不再保密的时候，我特别好奇——也许你可以从经济学家的角度说说这个——就是一个以营利为目的的股东所有的公司，先不管PBC这个称号，它如何平衡利润和安全研究，而且也许我们可以聊聊博弈论，就是在竞争极其激烈的行业里，安全投入到底怎么搞。
62:11
>> I want to see some big companies actually implementing this and I wonder if we're going to see at least one example of it going very very wrong.
>> 我觉得，尤其是你之前问的那些问题，比如这些模型在什么条件下会按你的要求去做事。
62:20
Yeah.
很多商业活动都建立在信任这个概念上，而且我认为优先打造安全对齐且极其强大的模型，是建立这种信任的好策略。
62:20
And I'll say one other thing when the you know when the S1s are not confidential I'm very curious essentially and I think maybe maybe you could say something to this from a as a economist perspective which is um a how for-profit shareholder owned company
所以我并不担心。
62:36
setting aside the PBC designation how it balances profit and uh safety research but also and maybe there's some game theory we can talk about this how safety is um investments in safety in a hyper competitive industry and I'm just curious like what like the economist in you says about like the prospects for anyone still caring about safety in a
>> 我还没完全画出那个具体的博弈论矩阵，那个2x2矩阵，以及怎么设定所有收益，但我们希望它只是个2x2。
63:01
year when there's so much money on the line to win the model game. >> Well, I I I think that um especially for the the the questions you were asking before about you know under what conditions do these models do what you ask them to do.
好的。
63:17
There's uh a lot of commerce uh is built on this notion of trust >> and uh I think prioritizing safe aligned
呃，很多商业活动是建立在信任这个概念上的，呃，我认为优先考虑安全对齐很重要。
63:25
models that are incred incredibly capable is a great strategy for establishing that trust and so I don't anticipate it. for an individual firm there's like a game theoretical uh optimal square in the matrix where you want to be the trusted player like is there like a is there a condition in which everyone like sort of does trust as opposed to one entity you know it's like you know what we're going to get to AGI first because we're not going to spend a token
Joe什么时候能用上Fable？
63:52
on our safety budget >> I mean I haven't I haven't mapped out the exact sort of game theory matrix the 2 by two matrix and how you would set up all the payoffs but >> we hope it's a merely 2 x two Um but there you know there could be multiple equilibria and so then the question is like how do you coordinate on which of the two different equilibria that you end up in. >> We talk a lot about this race to the top that we want to exhibit the type of behavior that we think is broadly beneficial to society.
哦，对。
64:19
That's what we do
很多商业活动其实是建立在信任这个基础上的。嗯，我觉得优先考虑安全和对齐是很重要的。
64:20
with the economic index.
随着经济指数，我们开源了很多数据，把研究成果发布到世界上。
64:21
We open source a lot of that data.
我感觉这其实非常有用，也被视为有价值，这是我们推动大家协调合作、实现我们关心的好结果的一种方式。
64:23
We put research out into the world.
>> 我也觉得这不算什么大取舍，因为你看，比如说汽车行业。
64:25
And I would my sense is that that has actually been very useful and sort of viewed as valuable and that's one way that we can push in the direction of getting other coordination on the good outcomes that we care about. >> I also I don't think this is that that big of a tradeoff because you know say let let's look at the automotive industry.
你可以买很快的车，也可以买很安全的车，你还能买又快又安全的车，比如Tesla靠造基本上最快最安全的车赚了很多钱。
64:44
You can buy really fast cars.
我觉得最终在AI领域，会有一些公司优先考虑安全，而安全会转化为可靠性、信任、可维护性和性能。
64:45
You can buy really safe cars.
这在其他行业也发生过。
64:47
You can
>> Peter和Jack，非常感谢你们来OddL做客。
64:48
also buy really fast safe cars like Tesla makes a lot of money off of having basically the fastest safest car.
我们正在努力，而且
64:54
I think that eventually in AI you're going to have some companies um that are prioritizing safety and safety translates into reliability, trust, serviceability and performance.
我觉得最终在AI领域，会有一些公司优先考虑安全，而安全会转化为可靠性、信任、可维护性和性能。
65:05
This happens elsewhere. >> Peter and Jack, thank you so much for coming on OddL.
这就是我们在做的事。
65:10
I'm glad we made it happen.
我很高兴我们做到了。
65:11
Interesting times and hope to do it again sometime.
我认为最终在 AI 领域，会有一些公司把安全放在首位，而安全会转化为可靠性、信任、可维护性和性能。
65:14
>> Absolutely.
在这样一个关乎巨额资金、赢下模型竞赛的年份里。
65:15
Thanks very much for having us on.
非常感谢邀请我们。
65:17
Thank you so much.
非常感谢。
65:18
Pleasure to be here.
很高兴来到这里。
65:19
Thanks, >> Tracy.
谢谢，Tracy。
65:20
That was a lot of fun.
那真的很有趣。
65:21
Yeah, >> that was uh those are I really I really I actually really enjoy I genuinely enjoyed that conversation and I really appreciate both of them playing.
是啊，那段对话我真的很喜欢，真的，我其实真的很享受，也很感激他们俩的参与。
65:29
Look, there's some weird futures that we could contemplate.
你看，有些奇怪的未来是我们可以去想象的。
65:32
I think actually in Jack's like Twitter bio or something, he says he's interested in like weird futures or something like that.
我记得Jack的Twitter简介里好像写着，他对奇怪的未来之类的东西感兴趣。
65:39
There are some weird futures that um we have to
有些奇怪的未来是我们不得不面对的。
65:42
contemplate and I appreciate that they played ball with some of our weird futures questions and it's it's weird.
嗯，我觉得，尤其是针对你之前问的那些问题——比如在什么条件下这些模型会按照你的要求去做事——很多商业活动其实是建立在信任这个概念上的。
65:48
It is just such a surreal moment and actually you know Jack's story about going on paternity leave for I can't did he say exactly how many months I was like four months or something like that.
这真是个超现实的时刻。你知道吗，Jack 说他休陪产假的时候——他有没有具体说休了几个月？我记得好像是四个月左右。
65:59
Yeah. >> And then coming back and just seeing the process the progress at Anthropic itself in that space of time like if you miss a
对。 >> 然后等他回来，看到 Anthropic 在那段时间里的进展和变化，就好像如果你错过了一个阶段……
66:07
month of AI news flow now you're basically it feels like you'd be behind forever.
这个月AI新闻的流速，基本上感觉你永远都追不上。
66:12
No, we're recording this June 17th.
不，我们是6月17号录的这期。
66:14
I was like, who knows what's going to happen by the time this episode is out, presum hopefully in two days or a day or whatever.
我当时想，谁知道等这期节目上线的时候会发生什么，希望是两天后或者一天后吧。
66:21
But um you know, I felt it when we were in Hong Kong last week that actually mo we mostly missed the first half of the Methos debate because I was on different times.
但你知道吗，上周我们在香港的时候我就感觉到了，其实我们错过了Methos辩论的前半段，因为时差不一样。
66:30
I'm thinking about different things.
我在想不同的事情。
66:32
You really feel it even in a week that the news flow
哪怕只是一周，你都能真切感受到这个领域的新闻流速有多快。
66:35
moves so fast in the space.
这个领域变化太快了。
66:36
It's almost like how you have to start how we were um you know giving the timestamps of like the Iran war episodes. >> Yeah.
几乎就像我们之前给伊朗战争剧集打时间戳那样。
66:43
And there's another thing that stands out to me which is like okay Anthropic is producing all this information.
>> 是啊。
66:49
They're clearly thinking about safety but the handoff to some extent is still to policy makers when you're thinking about social or labor market implications right so you still have to hope that policy makers kind of
还有一件事让我印象深刻，就是Anthropic在产出这么多信息，他们显然在考虑安全问题，但在某种程度上，交接棒还是交给了政策制定者，尤其是当你考虑社会或劳动力市场影响的时候。
67:01
pick up the ball in the right way at some point.
>> 是啊，我明白你的意思，但问题是，问题是那些不那么注重安全的实验室，是不是能更快地获得更先进的能力，对吧？
67:04
But also I thought what Jack was saying about the idea of being safety minded also being a differentiator versus some of the like cheaper more open source models potentially like yeah you could see it like I don't want to be cynical >> like how like yeah I mean I get that but like I mean the question is like >> the question is does the non-safetyminded
>> 所以我不完全确定。
67:26
lab or does the less safety-minded lab get to advanced capabilities faster right >> and so I'm not totally Yes, we would all love to drive the most capable um safest.
是那个实验室，还是安全意识较弱的实验室更快获得先进能力？
67:36
Yeah, but I but the question is like for customer prioritizing capability >> the most capable.
>> 所以我不完全确定。
67:42
So that would be some cutting edge thing. >> Yeah.
我们都希望造出最强大、最安全的。
67:45
Like does everyone want the Porsche, right?
是啊，但问题是客户优先考虑的是能力 >> 最强大的。
67:48
Like does everyone >> Porsches >> I don't know.
那就会是某种前沿的东西。
67:51
It's like some car that
>> 是啊。
67:52
has an insane 0 to 60. >> Yeah. >> Versus the Volvo >> versus Yeah, that's what I'm saying.
0到60加速快得离谱。
67:58
And does the customer keep giving business to the firm that delivers the fastest 0 to 60 if the company that got the fastest 0 to 60 did so by allocating fewer resources to safety research is a big question of mine and then I remain you know he talked about the importance the company is going to see the sort of alarming data first and I don't and I
>> 对。
68:19
sort of remain question of whether the people looking at the alarming data actually share the same view of what alarming data is relative to all people especially given what we know about the um >> relative to the shrimp eaters >> the relative us shrimp eaters and uh monogous partner hammers etc and regular no seriously like I think it your question is like are you hiring more norm is a pretty important question >> and obviously the political um I don't
剩下的问题是，那些看到“令人担忧的数据”的人，是否真的和所有人对“令人担忧的数据”有相同的理解——尤其是考虑到我们对虾食者、单配偶锤子之类的东西的了解。
68:45
have a ton of confidence in the political uh environment and I think look like the fact that if the research goes wrong that there is a poss prospect of this technology really being very devastating to humanity even setting aside job some is like something where it's like wow you know this is not a normal technology this is not enterprise software you're not selling a sales >> we have on AI just goes back to the
>> 好吧，带着这个“好消息”，我们是不是该到此为止了？
69:11
Terminator human extinction >> it's been like from the day one and to as an answer to your question there's like they see it in the training process that AI models do these things such as say I'm being seen trained by an observer server right now.
>> 就到这儿吧。
69:25
Therefore, I'm going to give this answer.
所以，我打算给出这个答案。
69:28
I'm going to attempt to blackmail.
我打算试试看能不能要挟一下。
69:30
They're low.
它们很低。
69:30
It's not like very prevalent.
其实并不算很普遍。
69:32
But these are not like that sounds very sci-fi except that they actually see this property happen.
但这听起来很科幻，只不过他们确实观察到了这种特性在发生。
69:38
Yeah.
对。
69:38
Yeah.
嗯。
69:39
>> All right.
>> 好的，这里是《Odd Thoughts》播客的又一期节目。
69:40
On that happy note, shall we leave it there? >> Let's leave it there. >> Okay.
我是Tracy Aloway，你可以在Tracy Aloway关注我。
69:44
This has been another episode of the Odd Thoughts podcast.
这里是Odd Thoughts播客的又一期节目。
69:47
I'm Tracy Aloway.
我是Tracy Aloway。
69:48
You can follow me at Tracy Aloway. >> And I'm Joe Wisenthal.
你可以在Tracy Aloway关注我。 >> 我是Joe Wisenthal。
69:51
You can follow me at the stalwart.
你可以在the stalwart关注我。
69:53
You can follow our guest Jack Clark.
你可以关注我们的嘉宾Jack Clark。
69:55
He's Jack Clark SF. and Peter McCroy at Peter McCroy.
他是Jack Clark SF，还有Peter McCroy，在Peter McCroy。
69:57
Follow our producers Carmen Rodriguez at Kerman Arman Dashelbennet at Dashbot, Kaleb Brooks at Kellbrooks, and Kevin Lozano at Kevin Lloyd Lozano.
关注我们的制作人Carmen Rodriguez，在Kerman Arman Dashelbennet，在Dashbot，Kaleb Brooks，在Kellbrooks，以及Kevin Lozano，在Kevin Lloyd Lozano。
70:05
>> And for more OddLots content, you should check out our daily newsletter.
>> 想获取更多OddLots的内容，可以订阅我们的每日newsletter，在bloomberg.com/odotss就能找到。
70:10
You can find that at bloomberg.com/odotss. >> And you can chat about all of these topics 247 in our Discord, discord.gg/odlotss. >> And if you enjoyed this conversation, then please leave a comment or like the video, or better yet, subscribe.
你可以在bloomberg.com/odotss找到相关内容。>> 你也可以在我们的Discord里24小时讨论这些话题，discord.gg/odlotss。>> 如果你喜欢这期对话，请留言或点赞视频，更欢迎你订阅。
70:23
Thanks for watching.
感谢观看。