Building Codex with Tibo Sottiaux ·

00:00

Codex is one of the most popular AI coding harnesses today, but how did it all start?

Codex 是如今最流行的 AI coding harness 之一，但这一切是怎么开始的？

00:05

Many of you will know today's guest Tibo from his generous and pretty fru-can-codex usage

你们很多人认识今天的嘉宾 Tibo，是因为他慷慨、而且相当频繁地重置 Codex 使用额度。

00:09

resets.

他也从 Codex 作为一个产品起步时就在那里，并从那以后一直领导更广泛的 Codex 团队。

00:10

He was also there on Codex as a product started and has led the broader codex team since.

今天我们会聊 Codex 是如何起步的，为什么它用 Rust 构建并做成 open source。

00:15

Today we cover how codex started and why it was built in Rust and made open source.

Codex 正在 Codex 团队和 OpenAI 内部发生怎样的变化？

00:19

How codex is changing inside the codex team and open AI?

当维护和 re-architecting 变得便宜到离谱时意味着什么，Codex 合并进 ChatGPT 是什么样子，以及许多被低估的工程挑战。

00:23

What it means when maintenance and re-architecting are getting ridiculously cheap, what the merge

当 maintenance 和 re-architecting 变得便宜得离谱时，这意味着什么，

00:27

of codex into chat GPC looked like, and the many underappreciated engineering challenges

codex 合并进 chat GPC 是什么样子，以及那些被严重低估的 engineering 挑战

00:31

of this project.

这个项目的。

00:32

If you want to understand how teams inside of open AI plan, review and ship software,

如果你想了解 open AI 内部的团队如何规划、评审和发布 software，

00:37

this episode is for you.

这一集就是为你准备的。

00:39

This episode is presented by TurboPuffer, a ridiculously scalable, fast and cheap hybrid

本期节目由 TurboPuffer 呈现，它是一个极其可扩展、快速又便宜的 hybrid

00:43

search engine built on top of object storage by an engineering team that are really going

search engine，构建在 object storage 之上，来自一支你花时间相处后真的会

00:47

to like after spending time with them.

喜欢上的工程团队。

00:48

TurboPuffer is the tool that companies like Antropic, Notion, Cognition and Harvey all use

TurboPuffer 是 Antropic、Notion、Cognition 和 Harvey 这些公司都在用的工具，

00:53

to connect their AI products to massive amounts of unstructured data.

用来把他们的 AI 产品连接到海量 unstructured data。

00:57

When I've talked with the engineers who use TurboPuffer, the theme that always comes

当我跟使用 TurboPuffer 的工程师们交流时，总会出现的主题

01:00

up is reliability and performance at scale.

就是规模化的可靠性和性能。

01:03

The reasons for this have everything to do with TurboPuffer's architecture.

这背后的原因，全都和 TurboPuffer 的架构有关。

01:07

TurboPuffer uses only object storage for state and NVMe SSDs with memory cache for compute.

TurboPuffer 只用 object storage 来存储 state，并用带 memory cache 的 NVMe SSDs 来做 compute。

01:13

Data and TurboPuffer is organized into namespaces.

Data 和 TurboPuffer 被组织成 namespaces。

01:16

You can think of a namespace as a database table or a search index or an S3 prefix depending

你可以把 namespace 看作一个 database table、一个 search index 或一个 S3 prefix，具体取决于

01:21

on the world you come from.

你来自哪个世界。

01:23

When a namespace is not being queried, it stays on cheap object storage with no associated

当一个 namespace 没有被查询时，它会留在便宜的 object storage 上，不带有任何相关的

01:27

compute cost.

compute cost。

01:29

When a namespace is active, TurboPuffer pulls it up into hot-catching tiers so queries

当 namespace 处于活跃状态时，TurboPuffer 会把它拉进 hot-catching tiers，因此查询

01:33

are very fast.

会非常快。

01:35

This design fundamental makes it effortless to scale to hundreds of millions of namespaces.

这种设计从根本上让扩展到数亿个 namespaces 变得轻而易举。

01:40

If you're building a multi-tenant AI product, every user and their agent can have their

如果你在构建一个 multi-tenant AI 产品，那么每个用户和他们的 agent 都可以拥有自己的

01:44

own dedicated search index without any overhead.

专属 search index，而且没有任何 overhead。

01:47

And each namespace can hold hundreds of millions of documents without any special configuration.

而且每个 namespace 都能容纳数亿个 documents，无需任何特殊 configuration。

01:52

You can scale TurboPuffer virtually without limit and the performance, reliability and operating

你可以几乎无限制地扩展 TurboPuffer，而且性能、可靠性和运营

01:56

model all stay the same.

model 全都保持不变。

01:58

If you need to connect AI to lots of data, TurboPuffer should be your first choice.

如果你需要把 AI 连接到大量 data，TurboPuffer 应该是你的首选。

02:02

Check it out at TurboPuffer.com slash pragmatic.

去 TurboPuffer.com/pragmatic 看看吧。

02:05

Tiibo, welcome to the podcast.

Tiibo，欢迎来到播客。

02:07

So good to have you here.

你能来真是太好了。

02:08

Thank you for having me.

谢谢你们邀请我。

02:09

It's so good to see you again.

很高兴再次见到你。

02:10

It's good to do this a lot last time with it in person.

能经常这样做真好，上次还是当面一起做的。

02:13

Now, now we're doing a video.

现在，我们现在在录视频。

02:15

First, I wanted to ask you, how did you get into tech?

首先，我想问问你，你当初是怎么进入 tech 行业的？

02:20

Then did you first know that you want to work with computers?

那你一开始就知道自己想跟电脑打交道吗？

02:23

I think good question.

我觉得这是个好问题。

02:24

It was a long, long time ago.

那是很久很久以前了。

02:29

My parents actually decided to move out of Brussels where I was born.

其实我父母决定搬离我出生的 Brussels。

02:34

And just thought it was great to just buy a small house and refurbish it, but it was

他们只是觉得，买个小房子翻新一下就很不错，但它其实是在

02:41

in the middle of a village with not much going on.

一个没什么活动的小村子正中间。

02:46

I think there was roughly 200 people living there.

我觉得那儿大概住了 200 个人。

02:50

Not many that I felt like I wanted to talk to or could make friends with it.

但没几个是我想去聊天、或者能交上朋友的。

02:55

And so I kind of got stuck.

所以我就有点陷进去了。

02:57

This is like very eight years old.

这大概就是我八岁那会儿。

03:00

I kind of got stuck as computers and early days for me, the internet, and that was my

我有点陷进电脑里了，对我来说，那是互联网的早期，而那就是我

03:06

way to learn about things.

了解各种事情的方式。

03:09

And so just the rest just came from that.

所以剩下的就都是从那儿来的。

03:12

Sort of like I owe it to my parents to have moved into the middle of nowhere and then

有点像是我得感谢我父母搬到了那种前不着村后不着店的地方，然后

03:17

had no choice but to get interested in computers.

没办法，只能对电脑产生兴趣了。

03:20

Once you've finished high school, you went on and went to university, right?

高中毕业后，你就继续去上大学了，对吧？

03:23

It's actually studying it properly.

其实是在正儿八经地学这个。

03:25

Yes.

是的。

03:26

I studied mathematics, applied mathematics at university.

我在大学学的是数学，应用数学。

03:30

I went there quite early and so I graduated early as well.

我去得很早，所以也提前毕业了。

03:36

I thought for a long time that I would actually not make it and I would drop out.

有很长一段时间，我都以为自己其实读不下来，会退学。

03:42

I had small companies and small consulting business like a while I was studying.

我读书那会儿，也搞过一些小公司和小型咨询业务之类的。

03:49

I was working for banks.

我当时在给银行工作。

03:51

I was working for, I was very interested in supply chain and applied mathematics problems

我当时在做，我对 supply chain 和 applied mathematics 问题特别感兴趣

03:57

and I started selling that and learning a lot through that.

然后我就开始卖这个，也从中学会了很多。

04:00

Eventually I ended up in the startup world in Belgium.

后来我进入了 Belgium 的 startup 圈。

04:04

I did that for a little while and then moved to London to work initially at Google and

我做了一段时间，然后搬到了 London，一开始是在 Google 工作，然后

04:09

then DeepMind and then now moved to be here at OpenEI.

然后去了 DeepMind，再后来就搬到这里，来到 OpenEI。

04:13

This is California.

这里是 California。

04:14

I love the California weather.

我特别喜欢 California 的天气。

04:15

We can talk about that.

我们可以聊这个。

04:16

It's been very good.

一直都非常好。

04:18

Right after university you started, you found the startup, right?

大学一毕业你就开始创业了，你创办了那个 startup，对吧？

04:22

You had the startup bug in you or the entrepreneur bug.

你心里有那种 startup 的冲动，或者说创业者的那股劲儿。

04:25

Yeah.

对。

04:26

So this started it was all about pharmaceutical supply chain, looking at the supply chain

所以最开始，它完全是围绕 pharmaceutical supply chain，研究 clinical trials 的 supply chain，

04:33

for clinical trials and like tried to optimize and decide like, hey, should you produce

然后试着去 optimize，去判断，比如说，嘿，你是不是应该生产

04:38

more medicine?

更多药品？

04:39

Where should you send it?

你应该把它送到哪里？

04:40

Where should you dispatch it?

你应该把它调度到哪里？

04:41

How do you avoid waste?

你怎么避免浪费？

04:42

And through that, making clinical trials more efficient.

然后通过这一点，让 clinical trials 变得更高效。

04:46

And this was using traditional non-ML techniques, it's a traditional more optimization, solving

而且这用的是传统的 non-ML 技术，是一种更传统的 optimization，去解决

04:53

Monte Carlo simulations, these kinds of things, stochastic multi-stage optimization problem

Monte Carlo simulations 这类东西，stochastic multi-stage optimization problem

04:58

really.

真的。

04:59

And we also applied it on steel industry and we applied it to electrical grid as well

而且我们还把它应用到了钢铁行业，也用到了 electrical grid 上。

05:04

in Europe.

在欧洲。

05:06

There's like anything that sort of had the shape of like an optimization problem we

基本上，任何那种有点像 optimization problem 的东西，我们

05:09

sort of like get interested in.

就会有点感兴趣。

05:11

And you know, to this day like this, this company still exists and I think they do some

而且你知道，直到今天，这家公司仍然存在，我觉得他们至今还在做一些

05:14

of the most interesting work still, but it's changing a lot, you know, with modern AI

最有趣的工作，但你知道，随着现代 AI，它变化很大

05:19

for sure.

这是肯定的。

05:20

But it's interesting because you kind of said like, oh yeah, that wasn't a mail.

但有意思的是，你有点像在说，哦对，那不是一个 mail。

05:23

It was just a traditional stuff and then you go into like Monte Carlo simulation and optimization

那只是传统的东西，然后你就进入像 Monte Carlo simulation 和 optimization 这类东西

05:27

and this algorithm.

还有这个 algorithm。

05:28

I get a sense that you kind of just went deep, right?

我感觉你就是有点一头扎进去往深里钻了，对吧？

05:31

It was like, okay, like here's a problem space.

就像，好吧，这里有一个 problem space。

05:33

Like how can I use mathematics stuff that I learned stuff that I didn't learn to just

就像，我怎么能用 mathematics 里的那些东西，我学过的、没学过的，来就这么

05:37

go deeper and deeper?

越钻越深？

05:38

Do I sense that correctly?

我这么理解对吗？

05:39

Yeah, that's that's why I was obsessed with applied mathematics is just really this

对，这就是为什么我那么痴迷 applied mathematics，真的就是这个

05:44

idea of that you have theoretical mathematics or you have theoretical science and physics

想法，就是你有 theoretical mathematics，或者你有 theoretical science 和 physics

05:48

and like there, you just you do it because there's something to be discovered and something

而且就像在那儿，你就是去做，因为有些东西值得被发现，还有某种

05:54

beautiful about it and so all about patterns and pushing different here.

它美好的那一面，所以一切都在于 patterns 和在这里推动不同。

05:57

But you don't necessarily always know like how you're going to apply it.

但你不一定总是知道，就像，你要怎么把它应用起来。

06:00

And then there was like the real world, right?

然后就有了，就像，真实世界，对吧？

06:02

This is like, you know, there's all these cool problems that just lie around and I was

这就像，你知道，有所有这些很酷的问题就摆在那儿，而我

06:06

like very interested in seeing like, you know, how can I make the world better?

就像，非常想看看，就像，你知道，我怎样才能让世界变得更好？

06:09

And so like how do I apply like, you know, sophisticated mathematics, you know, to just optimize

所以就像，我该怎么应用，就像，你知道，高深的数学，你知道，去直接 optimize

06:14

the world around me?

我周围的世界？

06:15

And that was like a lot of the thesis behind that startup.

而那差不多就是那家 startup 背后很大一部分的核心思路。

06:18

Yeah.

是啊。

06:19

And then after a startup you ended up at Google and first at Google London, it was in 2015

然后离开 startup 之后，你最后去了 Google，一开始是在 Google London，那是 2015 年

06:23

and I remember if 2015 Google was a really, really competitive place to get into like maybe

我记得 2015 年的时候，Google 真的是一个非常非常难进的地方，可能就像

06:27

as competitive as OpenAI is today in terms of the industry or in terms of prestige, you

在行业地位或声望方面，跟今天的 OpenAI 一样有竞争力，你

06:32

worked on maps initially and then you moved over to DeepMind.

一开始做的是 maps，后来转到了 DeepMind。

06:35

Can you talk a little bit about what you worked on and then why did you move on from a already

你能不能稍微聊聊你当时做了什么，以及后来为什么离开了一个已经

06:40

really interesting space that you'd securely loved, you know, like the optimization logistics

非常有意思、而且你肯定很热爱的领域，你知道，就像 optimization logistics

06:44

and all these things?

还有所有这些事情？

06:45

Yes.

是的。

06:46

I didn't I didn't start on Google Maps.

我并不是，我并不是一开始就在做 Google Maps。

06:48

I started on a project that was meant to make the web faster and make to make websites

我一开始做的是一个项目，本来是想让 web 更快，让网站

06:57

faster especially on mobile.

更快，尤其是在 mobile 上。

06:59

At the time, you know, Google was saying, so like seeing the transition from desktop to

当时，你知道，Google 就在说，就像看到从 desktop 到

07:04

mobile and like more and more traffic going to like mobile phones, so like why don't you

mobile 的转变，越来越多的流量流向 mobile phones，所以就说你为什么不

07:08

get ahead of that?

提前布局这个呢？

07:09

So funded like a number of a number of initiatives and projects.

所以资助了，呃，好几个，好几个倡议和项目。

07:13

I was working on one of them.

我当时在做其中一个。

07:15

This was like really, really fun because it was a small group within actually the ads organization.

这真的真的特别好玩，因为它其实是广告部门里的一个小团队。

07:21

It was meant to sort of like, you know, offset the loss for the ad revenue loss because

它的目的有点像是，你知道，抵消广告收入的损失，因为

07:26

of this shift of traffic to mobile and worked on it for roughly two years.

流量向移动端转移，然后我大概做了两年。

07:31

And then it was canceled.

然后它就被取消了。

07:33

And although it was like the most fun I've had on, you know, solving hard technical challenges,

而且虽然这是我解决困难技术挑战时，你知道，最有意思的一次，

07:38

I learned a lot from not having product market fit, not having the right users, not having

我从没有 product market fit、没有合适的用户、没有

07:42

the right feedback loop, not trusting your product manager when they say the project is

正确的 feedback loop，别相信你的产品经理说项目

07:46

going well, when in fact, it's not going well at all.

进展顺利，而实际上，根本一点都不顺利。

07:48

And then one day just like this VP flew in from California and then it was just like,

然后有一天，这个 VP 就从 California 飞过来了，然后就像，

07:53

oh yeah, it's like, you know, we're canceling this project.

哦对，就像，你知道，我们要取消这个项目了。

07:56

You know, unfortunately, you only have, you know, hundreds of users and this is clearly

你知道，不幸的是，你只有，你知道，几百个用户，而这显然

08:00

not Google scale.

不是 Google scale。

08:01

And then it's unbelievable, but people were surprised.

然后简直难以置信，但大家居然还很惊讶。

08:04

And I think there's a lesson there that I carried me, of course, is, you know, just always

我觉得这里有个教训，当然，我一直记着，就是，你知道，永远要

08:11

always question, always go to like always, you know, deeply think about the impact that

永远要质疑，永远要，怎么说呢，你知道，深入思考自己正在产生的影响，但也要看到你所参与的整个项目的重要性。

08:16

you're having, but also like the importance of the overall project that you're contributing.

后来我转到了 Google Maps，Google Maps 特别有意思，我做的是评论相关的工作。

08:20

And then I moved into Google Maps, Google Maps was super fun, worked on reviews.

然后大概一年后，我没办法再忽视 DeepMind 了。

08:24

And then after roughly a year, I couldn't ignore like deep mind.

它就是，它就是一个特别的地方，总部在 London，当时有很多很棒的事情在发生。

08:27

It was just, it was this special place, headquartered in London, so many great things were happening.

那真的是特别早期的时候，你知道，当时已经有像 AlphaGo 这样的东西开始冒头了。

08:34

This was like really the early days, you know, with rumblings of things like AlphaGo.

而且他们看起来就是在做非常了不起的事情，你知道，真的是在攻克你能攻克的最难最难的问题。

08:39

And they just seemed to be doing extraordinary things and, you know, just really tackling

而且他们看起来就是在做特别了不起的事，你知道，就是真的在攻克

08:43

the very, very hardest problems that you can tackle.

你能去攻克的最最难的那些问题。

08:46

And like with my background, I was obviously drawn to that.

而且吧，以我的背景，我显然会被那个吸引。

08:48

I started there.

我就是从那里开始的。

08:49

Like I worked on a lot of the research infrastructure, research tooling.

比如我做了很多 research infrastructure、research tooling 方面的工作。

08:54

This is a theme that I carried on for almost a decade.

这个主题我差不多延续了十年。

08:56

And it's like this is very much also the, but how I approach things is how can I build tooling

而且这就像，这也很大程度上是那个……但我做事的方式是，我怎么能构建 tooling

09:02

and products that help make others more efficient and bring a lot of utility to them?

和产品，帮助别人变得更高效，并给他们带来很多 utility？

09:06

Initially I was doing this for research and then like over time, you know, I got like

一开始我是为了研究做这些，然后随着时间推移，你知道，我就开始

09:11

into thinking about things in a much more like more general and general and general way.

用一种更、更、更 general 的方式去思考事情。

09:16

You know, eventually like, you know, ending up where I'm now.

你知道，最终，就，你懂的，走到了我现在的位置。

09:19

And a fun story that you recently shared on X as well is how you were part of the team

而且你最近也在 X 上分享过一个有趣的故事，就是你曾经是那个团队的一员。

09:24

that built this internal Google bot that was, you know, if you want to say similar to

他们打造了 Google 内部的那个 bot，它，你知道，可以说类似于

09:29

chat GPT, but a year before chat GPT.

chat GPT，但比 chat GPT 早一年。

09:33

Can you, can you talk about that?

你能不能，你能不能聊聊这个？

09:34

That's that, that's a new story I haven't heard it before.

那，那是个我之前没听过的新故事。

09:37

This was part of deep mind.

这是 deep mind 的一部分。

09:38

There were like multiple efforts as well.

当时好像也有多个不同的尝试。

09:39

There was like brain as well.

当时还有像 Brain 这样的。

09:40

That was separated at the time.

当时那是分开的。

09:42

They had their own efforts on large language models, but it was definitely something that was

他们自己在 large language models 上有自己的投入，但这绝对是当时正在探索的事情。

09:45

being explored.

那不是我关注的重点；我当时脑子非常担心、非常忙，像是在想品牌挑战和游戏，而且，你知道，在想 RL，但不是语言意义上的那种。

09:46

It was not the main thrust of my mind was like very much worried and busy like thinking

所以当时就有这么一个小组在推动 large language models，而且，你知道，

09:53

about brand challenges and games and, you know, thinking about RL, not in the language

关于品牌挑战和游戏，还有，你知道，思考 RL，不是从语言

09:58

sense.

意义上去想。

10:00

And so there was like this group that was pushing on large language models and, you know,

然后就有这么一个小组在推进 large language models，而且，你知道，

10:05

thinking about, you know, hey, what if what if large text corpuses are everything?

就在想，你知道吧，嘿，万一，万一 large text corpuses 就是一切呢？

10:10

What if you just pushed language to its maximum and you just scaled language models?

要是你把语言推到极致，然后把 language models 一路 scale 上去呢？

10:16

Like, you know, would that be enough to get to general intelligence?

就像，你知道吧，那样就够达到 general intelligence 了吗？

10:19

That was like the hot debate at the time.

当时这就是特别热门的争论。

10:21

And then one group decided to just really push on that.

然后有一拨人就决定，真的就往这个方向使劲推。

10:25

And then it felt really natural like, you know, as I was building tooling, you know, with

然后这就感觉特别自然，你知道吧，就像我当时跟别人一起为 research 做 tooling 的时候，你知道吧，很明显你就会想，你知道吧，我们能用这个 model 做点什么呢？

10:29

others for research is like, you know, obviously you're like, you know, what can we do with

其他人做研究时就像，你知道，很明显你会想，你知道，我们能拿它来做什么

10:33

this model?

这个 model？

10:34

Like, how do we present it?

就是，我们该怎么把它展示出来呢？

10:35

You know, to the researcher, like, how can they sort of like, you know, debug the inputs

你知道，对研究员来说，就是，他们怎么才能，你知道，去 debug 那些 inputs

10:38

outputs and eventually you sort of like end up with, you know, like a chat system.

还有 outputs，最后你差不多就会得到，你知道，一个 chat system。

10:42

So we built that internally.

所以我们在内部把它做出来了。

10:44

We had a lot of fun initially.

一开始我们玩得特别开心。

10:45

The models were like, you know, kind of like almost like a little bit absurd, like, you

那些 models 就是，你知道，有点像，几乎有点荒唐，就你

10:48

know, not very coherent, not super useful, but it was a lot of fun to sort of like think

知道，不太连贯，也不是特别有用，但就是，有点像在这儿琢磨

10:54

here with them that caught up like, you know, like wildfires, like, you know, this application

跟它们一起，然后那就像，你知道，像野火一样迅速传开了，你知道，这个 application

10:58

is just sort of like, you know, everyone was kind of like sharing little conversations

就是那种，你知道吧，大家都像是在分享一些小小的对话

11:03

within deep mind.

在 deep mind 内部。

11:04

It felt more than like a research project or like a research project for researchers.

它感觉更像是一个研究项目，或者说像是给研究人员做的研究项目。

11:09

And so then there was this desire over time to like launch it as an external product.

然后慢慢地就有了这种想法，想把它作为一个对外的产品推出来。

11:13

But deep mind was just like not set up, you know, there was like the, the right way to

但 deep mind 就是没准备好，你知道吧，就有点像，那个，正确的方式去

11:17

launch products at Google, there's like, you know, the whole machinery of like, you know,

在 Google 推出产品，有那种，你知道吧，整套机制，就像，你知道吧，

11:20

how you do that, you know, the whole like the last production stack, you obviously very,

你怎么做这件事，你知道吧，整个就像最后那套 production stack，显然非常，

11:25

very optimized over the years to do things well, but also very, very hard as an environment

非常优化过，很多年下来把事做好，但作为一个环境也非常非常难

11:31

to truly innovate.

真正去创新。

11:32

And then I want to ask what made you, you know, look around or maybe consider open AI,

然后我想问，是什么让你，你知道，开始四处看看，或者可能考虑 OpenAI，

11:37

but I feel you partially answered this question, just putting myself back into your shoes.

但我觉得你已经部分回答了这个问题，我只是把自己代入你的处境。

11:41

Like, you're, you know, if it's 2024 or 2023, you're inside of Google who are publishing

就像，你，你知道，如果现在是 2024 或 2023 年，你在 Google 内部，而他们在发表

11:47

amazing papers, doing really good research, you're doing super fun stuff, right?

很棒的论文，做着非常出色的研究，你在做超级有意思的事情，对吧？

11:50

I've pushing the limits of what's been done before.

我一直在突破以前做过的事情的极限。

11:53

It's an inside a company where you are already moved, you know, for people who are feeling

这在一家公司内部，在那里你已经，你知道，被打动了，对于感觉

11:57

kind of comfortable or good about where they are right now, which I imagine you must have

对自己现在所处的位置还算舒服或者感觉不错的人，我想你肯定也有过这种感觉

12:00

been.

一直是这样。

12:01

So what made you still explore all right, like what else might be there?

所以是什么让你还是去探索了，对吧，比如那里可能还有什么别的？

12:06

Yeah, I was, I was very comfortable, you know, it's a good place, but really I had a,

是啊，我，我当时非常舒服，你知道，那是个好地方，但说实话我有一个，

12:14

I had a desire to, you know, meet great people, but also join a mission that I truly believe

我有一种渴望，你知道，去认识很棒的人，但也加入一个我真正相信

12:19

in, and that, you know, I felt like the people were true to the mission and cared deeply

的使命，而且，你知道，我觉得那里的人忠于这个使命，并且非常在乎

12:25

about impacting the world in a very, in a deeply positive way, but also in a direct way,

以非常、以极其积极的方式影响世界，而且是以直接的方式，

12:31

not, not being like, oh, yeah, it's just like, you know, we just do this work over here,

不是，不是那种，哦对，就好像，你知道，我们只是在这边做这个工作，

12:34

and then it's like, it's the job of someone else to figure out, you know, how to, how

然后就好像，那是别人的工作去弄清楚，你知道，怎么去，怎么

12:38

to make this useful.

让这变得有用。

12:39

It's like, I wanted to join a group where, you know, like all the parameters were sort

就像，我想加入一个团队，在那里，你知道，就像所有参数都在某种

12:42

of like considered together where, you know, research and product were like really co-designing.

程度上被放在一起考虑，在那里，你知道，research 和 product 真的在共同设计。

12:46

Opening, I was just like crushing it.

刚开始的时候，我就是，简直杀疯了。

12:49

At that chat, it's like, you know, taking off is like, I was, I met a couple of people

在那个 chat 里，就像，你知道，开始起飞了，就像，我当时，我遇到了几个

12:54

from OpenAI, and then I was like, wait, what, you know, you only have like 20 people working

来自 OpenAI 的人，然后我就想，等等，什么，你知道，你们只有大概 20 个人在做

12:58

on chat to PC, like, that is, that is an insanely small number that must be like extremely

chat to PC，就像，那，那是一个小到离谱的数字，那肯定，就像，极其

13:04

empowering.

有赋能感。

13:05

Like, you know, how does that work?

就是，你知道，这到底是怎么运作的？

13:07

How do you manage to maintain, you know, a product with that level of scale and with

你是怎么做到维护，你知道，一个有着那种规模、并且有

13:12

that level of autonomy with, you know, only, only 20 engineers?

那种自主程度的产品，而且，你知道，只有，只有 20 个工程师？

13:15

And then, you know, I said kind of dug and dug and dug and it's like, it was just an amazing

然后，你知道，我就说，我算是挖了又挖、挖了又挖，然后就像，那真是一个很棒的

13:19

group of people, amazing mission, you know, super talented, super driven, and like, it

一群人，了不起的使命，你知道，超级有才华，超级有干劲，而且就像，它

13:23

was, it was like, drew me in.

就是，它就像，把我拉了进去。

13:25

And then I joined pre-reasoning efforts immediately, like typical OpenAI fashion, like I joined

然后我马上就加入了 pre-reasoning 的工作，就像典型的 OpenAI 作风，就像我一加入

13:31

it was like, oh yeah, you know, like, there's this thing going on, like, you know, we're

就像，哦对，你知道，就像，有这么一件事在进行，就像，你知道，我们正

13:35

going to launch reasoning models, like, you know, six new paradigm, and then, you know,

要推出 reasoning models，就像，你知道，六个新范式，然后，你知道，

13:38

start sprinting on that.

就开始全速冲刺。

13:39

And like, you know, like a month later, like the company launched O1, O1, O1 preview.

然后，就像，你知道，大概一个月后，就像，公司推出了 O1、O1、O1 preview。

13:45

And that was exhilarating to be part of, I wanted to be part of like a place that moves

而且能参与其中真的让人兴奋，我想加入的就是那种节奏很快、

13:49

fast, cares about impact, would be in tune with the world and, you know, just really listen.

在乎影响力、能跟世界同频，而且，你知道，真的会倾听的地方。

13:56

And sort of like, that's also, to me, like, you know, what I've carried with me, like,

而且有点像，对我来说，你知道，这也是我一直带着的东西，就像，

13:59

when, when building codex, when building products, it's like, having a community, listen

当，当在做 codex 的时候，当在做产品的时候，就像，要有一个社区，倾听

14:04

to the community, just really focus on like a really intense feedback loop.

社区，真正专注于一种非常高强度的 feedback loop。

14:09

And then building something that is just like, you know, you just really want to care

然后就是去构建某个东西，就像，你知道，你就是会真的很想在意它

14:13

about it and like, you know, care about the utility of it that it provides to the world.

而且，你知道，在意它给这个世界提供的价值。

14:17

And then of course, you start to work pretty quickly on codex.

然后当然，你很快就投入到 codex 的工作里了。

14:20

So you joined in 2024.

所以你是 2024 年加入的。

14:22

Can you take us back?

能不能带我们回到当时？

14:23

What the thinking back there when you joined was about AI or LM's and code.

当时你加入的时候，对 AI 或者 LM's 和 code 的想法是什么？

14:29

I know there was this ASWE effort back then, the, we talked about it in the deep type as

我知道当时有 ASWE 这个 effort，那个，我们在 deep type 里也

14:36

well that we did in the pragmatic engineer, the autonomous software engineer.

聊过，就是我们在 the pragmatic engineer 做过的那个，autonomous software engineer。

14:39

A3.

14:40

Yeah.

嗯。

14:41

That's what it is.

就是这样。

14:42

It was pronounced internally.

内部就是这么念的。

14:43

We don't, we don't have the ASWE effort anymore like it's codex.

我们，我们已经没有 ASWE effort 了，就像 codex 一样。

14:48

But really for me is, I joined, I started building infrastructure for research.

但对我来说，其实我加入之后，是开始为 research 搭建 infrastructure。

14:53

Like my, a lot of what I did before was large scale data storage analysis and then tools

比如我之前做的很多都是 large scale data storage analysis，然后就是 tools

15:01

to understand training runs.

来理解 training runs。

15:03

I did a lot of different things over my years.

我这些年做过很多不同的事情。

15:06

But it was always about building for others and making them faster and just really caring

但一直都是在为别人构建，让他们更快，而且真的很在意

15:12

about, you know, fundamentally doing that well.

你知道吗，从根本上把这件事做好。

15:15

And then through tooling and infrastructure, making new things possible.

然后通过 tooling 和 infrastructure，让新的东西成为可能。

15:18

And so when I joined OpenAI, it was like with the same idea.

所以当我加入 OpenAI 的时候，也带着同样的想法。

15:22

And then with the, with what I want preview and like, you know, some, some, some, some

然后随着那个，随着 what I want preview，还有，你知道，一些、一些、一些、一些

15:26

of the later models, it was very clear that we had to use the models themselves to help

后来的 models，很明显，我们必须用 models 本身来帮

15:31

us go faster.

我们更快地前进。

15:32

And so I just really got obsessed with the idea of what were the limitations, how are we

所以我就真的特别着迷于这个想法：这些局限到底是什么，我们该怎么

15:37

going to use those models for research itself.

用这些模型来做研究本身。

15:40

So got together with other folks in research, we started training models, we started building

所以我就和研究领域的其他人聚到一起，我们开始训练模型，我们开始搭建

15:46

little agents.

一些小 agents。

15:47

Those were truly the precursor to, to, to codex.

那些真的就是 codex 的前身，前身，前身。

15:50

And like this was like, we were training internal models to be very proficient on the

而且就像这样，我们当时在训练内部模型，让它们非常擅长

15:55

Python code base of OpenAI.

OpenAI 的 Python code base。

15:58

And then very proficient with, you know, having like good, good taste in architecture, good

然后非常擅长，你知道，就是拥有很好的、很好的 architecture 品味，好的

16:04

taste in, you know, like code style, it was like Python only.

在品味上，你知道，比如 code style，就是只用 Python。

16:09

And then the idea was like, you know, we would sort of like use that to build infrastructure

然后那个想法就是，你知道，我们差不多会用那个来搭建 infrastructure，

16:12

very quickly and, you know, help researchers code faster as well.

非常快，而且，你知道，也帮研究人员更快地写代码。

16:16

And then, you know, and then we would move faster.

然后，你知道，然后我们就能推进得更快。

16:18

And then over time, when you just kind of push that and simplify it to its score, you're

然后随着时间推移，当你就是那样推动它，并把它简化到它的 score 时，你就在

16:22

making a lot of, you know, we found like we could make a lot of progress very quickly.

取得很多，你知道，我们发现我们好像能非常快地取得很多进展。

16:27

And then learn very quickly.

然后学得非常快。

16:28

And then Greg and Sam are, you know, people with, with immensely supportive and also Greg was

然后 Greg 和 Sam 是，你知道，那种，那种极其支持别人的人，而且 Greg 也

16:36

very adamant that, you know, we would, we would not just focus on ourselves, but we

非常坚持，你知道，我们不只会关注我们自己，也会关注造福世界。

16:40

would also focus on benefiting the world.

所以他就有点鼓励我们，不要只把这件事看成 OpenAI 自己的工具，而是也要把它真正做成一个产品。

16:43

And so he just sort of encouraged that we would be thinking about this not just as a tool

也就是在这个时候，我们把这个研究项目和这个 A3 effort 合并了，然后开始打造一个东西。

16:48

for OpenAI itself, but also as something that we would actually make it through a product.

然后这就引出了一个 sprint，也就是我们最初推出的那个 cloud codex，但它其实并没有真正的 PMF，因为用起来有点太 high friction。

16:52

And this is when we merged this research effort with this A3 effort and we started building

而这就是我们把这项研究工作跟这个 A3 effort 合并起来、然后开始打造

16:59

one thing.

一个东西的时候。

17:00

And then that led to a sprint, which was like the initial cloud codex that we launched,

然后这就引出了一次 sprint，差不多就是我们发布的那个最初的 cloud codex，

17:04

which didn't really have PMF, because it was like a little bit too high friction.

它其实并没有真正的 PMF，因为它的 friction 有点太高了。

17:08

And then we also launched the codex CLI and then we could push, but it was always this

然后我们还发布了 codex CLI，之后我们就能 push 了，但一直都有这个

17:13

idea of, hey, how do we get models to really help here?

想法：嘿，我们怎么才能让 models 在这里真正帮上忙？

17:17

You mentioned that first you started to build this model to train on the Python code and

你提到过，一开始你开始 build 这个 model，是为了在 Python code 上 train，并且

17:21

actually help me build in for better.

真正帮我更好地 build infra。

17:24

But then you made this interesting decision where for codex, you built it in Rust.

但后来你做了一个很有意思的决定：对 codex 来说，你用 Rust 来 build 它。

17:29

And at the time, the model was not on distribution for Rust, right?

而且当时，这个 model 并不在 Rust 的 distribution 里，对吧？

17:32

It wasn't as good as in Rust, and it wasn't Python or TypeScript.

它在 Rust 上没那么好，而且 Rust 也不是 Python 或 TypeScript。

17:37

Why did you make that kind of a decision?

你为什么会做那样的决定？

17:39

Was it kind of like did you expect that it'll catch up or you figure that performance

这有点像是，你原本就预期它会追上来，还是你觉得 performance 更重要？

17:43

is more important?

因为这非常反直觉。

17:44

Because it was very counterintuitive.

大多数其他被构建的 harnesses 其实都不是用 Rust 构建的。

17:46

Most of the other harnesses built were actually not built in Rust.

它们其实是构建在 distribution 上，用的是 TypeScript 或 Python 或别的什么。

17:49

They were built on distribution on TypeScript or Python or something else.

是的。

17:52

Yes.

从 first principles 出发，我们很早就把 product interface 和 agent 当成不同的东西来思考。

17:53

From first principles, like we very early on, we were thinking about the product interface

从 first principles 出发，我们很早就把 product interface

18:01

and the agent as different things.

和 agent 当成不同的东西来想。

18:04

So it was very important to build the core of the agent in a way that was robust, that was

所以，把 agent 的核心打造成既稳健、

18:14

secure as well, that was engineered for efficiency and skill.

又安全，并且为效率和技能而设计，这一点非常重要。

18:21

And having worked through projects over the years that go from, hey, this is a fun thing

而且这些年来做过各种项目，从“嘿，这挺好玩”

18:28

to, hey, we need to scale this to the scale of the largest data center, the decisions

到“嘿，我们得把它 scale 到最大 data center 的规模”，那些

18:35

early on are really turned out to be quite important, as long as you don't sacrifice too

早期决定最后证明真的相当重要，只要你不牺牲太多

18:41

much of the velocity.

速度。

18:42

And so it's a trade-off, but we had very prolific and amazing Rust developers.

所以这是个 trade-off，但我们有非常高产、非常厉害的 Rust 开发者。

18:47

Our internal models were not bad at Rust.

我们内部的 models 在 Rust 方面并不差。

18:50

And then you get a lot of validation as well at compile time, it's statically verified

然后你在 compile time 也会得到很多 validation，它是 statically verified 的

18:55

in all these things.

在所有这些方面。

18:56

And that is great for agents too.

而这对 agents 来说也很棒。

18:57

So turns out, it was quite clear that Rust, as a language, would actually be quite good

所以结果发现，很明显，Rust 作为一门语言，实际上会相当适合

19:04

for agents fairly quickly if we decided to put some effort into it.

agents，而且会相当快，只要我们决定投入一些精力进去。

19:08

But primarily, we were focused on correctness and we were focused on efficiency as well.

但主要来说，我们关注的是 correctness，同时也关注 efficiency。

19:12

Interesting.

有意思。

19:13

So you're saying, it's worth, in your case, it was worth thinking ahead of where you

所以你是说，这值得，在你的情况下，值得提前思考你

19:18

want this thing to be.

想要这个东西变成什么样。

19:19

And for example, a language choice, obviously with agents, you can rewrite a bunch of stuff

比如说，语言选择，显然有了 agents，你可以重写一大堆东西

19:23

and easier than in the past, but it's still, you can save yourself reworking by putting

比以前更容易，但仍然，你可以通过放

19:28

in the right, I guess, scaffolding or the baseline of what you're building on, right?

对，我想是，scaffolding 或者你所构建的基础的基线，来省去返工的麻烦，对吧？

19:35

I think we could have been successful if we had been in a TypeScript or maybe even Python

我觉得如果我们用 TypeScript 或者甚至 Python 的话，本来可以成功的

19:40

and then it would have been fun and then we would have rewritten it at some point.

然后那会很有趣，然后我们会在某个时候重写它。

19:43

But having a very clean separation between the agent itself, which can exist irrespective

但让 agent 本身之间有非常清晰的分离，agent 可以独立于

19:50

of the product,

产品而存在，

19:52

it was a very important principle.

那是一个非常重要的原则。

19:54

And if you write everything in the same code base, in the same language, it's like inevitably

如果你把所有东西都写在同一个 code base 里、用同一种 language，那就好像不可避免地

19:59

you're going to be a little bit sloppy and you're going to intertwine things more than

你就会有点马虎，而且会把东西互相纠缠得比

20:03

you should.

你应该的程度更厉害。

20:04

And then it's going to prevent further innovation after that.

然后这就会阻碍之后的进一步创新。

20:08

And so that was very important.

所以这一点非常重要。

20:09

The Rust boundary in a sense was very useful for that.

从某种意义上说，Rust boundary 对此非常有用。

20:12

One interesting decision that you made, which is unique across all of the major labs is

你做过的一个很有意思的决定，在所有 major labs 里都是独一无二的，就是

20:17

having this built-in open source, right?

这个是内置的 open source，对吧？

20:19

The CLI is open source, the SDK and the app server are all open source.

CLI 是 open source，SDK 和 app server 也都是 open source。

20:23

When and why did you decide that?

你是什么时候、又为什么决定这么做的？

20:24

It's not a given, especially, you know, there used to be jokes about open AI having things

这可不是理所当然的，尤其是，你知道，以前还有笑话调侃 OpenAI 有些东西

20:29

closed, but this is actually the opposite where like this is open, whereas like some

是闭源的，但这里其实正好相反，就是这个是开放的，而有些

20:34

competitors would ship close source harnesses, which again, I think it's very easy to understand

竞品会推出 closed source 的 harness，而且，我觉得很容易理解

20:39

why you want something close source.

为什么你会想要 closed source 的东西。

20:40

Why did you want it open source?

那你为什么想让它 open source 呢？

20:42

There was something really cool about the idea of having the code open source because

让 code open source 这个想法真的挺酷的，因为

20:49

fundamentally what you're building is you're building a coding agent.

从根本上说，你做的其实就是一个 coding agent。

20:53

And so we were sort of like thinking about, well, if you have that, you know, you're obviously

所以我们当时有点像是在想，嗯，如果你有了这个东西，你知道，你显然

20:58

going to point it at itself and know maybe, you know, you can build a community of, you

会把它指向它自己，然后，你知道，也许，你知道，你可以建立一个由，你知道，

21:04

know, contributors that use it to improve it.

贡献者组成的社区，让他们用它来改进它。

21:06

And then you know, you can learn a lot from that.

然后你知道，你能从中学到很多。

21:08

Also, I felt at the time is like, you know, very clear to us that if we were going to

另外，我当时的感觉是，你知道，对我们来说非常清楚的是，如果我们想要

21:13

be successful, open source itself would change and the role of code itself would change.

成功，open source 本身会改变，code 本身的角色也会改变。

21:18

And so being part of that community seemed important instead of divorced from it.

所以，成为那个社群的一部分，而不是跟它脱节，似乎很重要。

21:22

I think, you know, it's hard to solve problems if you don't sort of like witness them yourself.

我觉得，你知道，如果你自己都不算亲眼看到这些问题，就很难解决它们。

21:28

And then the other thing was just, it still feels like early, but it was very early at the

然后另一件事就是，现在感觉还是很早期，但当时真的

21:32

time.

非常早期。

21:34

It felt like we would have some ideas for how to solve things well.

感觉我们会有一些想法，知道怎么把这些事情解决好。

21:39

And we were co designing these, you know, with the training and the research and it's all

而且我们是在共同设计这些东西，你知道，结合 training 和研究，而这一切都是

21:46

about expressing like the capabilities of the model and like the most flexible in the

关于表达出 model 的能力，而且是以最灵活、

21:50

best way.

最好的方式。

21:51

And then also we didn't have all the answers.

而且当时我们也不是所有问题都有答案。

21:53

And sort of being very open about, hey, this is what a good harness looks like.

而且我们还挺坦诚的，直接说，嘿，一个好的 harness 大概就是这样。

22:00

This is how we think about it.

这就是我们看待它的方式。

22:01

We did like a couple of like very technical like deep dives and blog posts and we talked

我们做了几个非常技术性的 deep dive，也写了博客文章，还聊了很多。

22:05

about it a lot.

而且我们当时想，你知道，嘿，外面的世界就是这么快。

22:06

And we thought, you know, hey, it's just like the world is fast out there.

你知道，外面有那种超级聪明的人，就像，你知道，我们也会从其他 open source project 那里获得灵感。

22:10

There's like, you know, crazy smart people is like, you know, we're going to get inspired

就有那种，你知道，特别聪明的人会说，你知道，我们会被启发

22:13

by other open source project as well.

也被其他 open source 项目。

22:16

And so let's just make this a level playing field and sort of like encourage a lot of

所以咱们就让这变成一个公平的竞争环境，然后有点像鼓励大家多去

22:23

tinkering and exploration at this stage.

在这个阶段折腾、探索。

22:26

Now this has been now, you know, like a year later or a year and a half later, which

现在这已经，你知道，差不多一年或者一年半之后了，而

22:30

is a very long time in right now in this AI timeframe.

在现在这个 AI 时间尺度里，这已经是很长一段时间了。

22:33

But looking back or taking the experience, what are the benefits you've seen that kind

但回头看，或者从经验来说，你看到的好处有哪些，就是那种

22:39

of entering benefits, the entering teams benefits from being open source and just honestly,

加入的团队从 open source 中获得的好处，而且说实话，

22:43

what are things that are kind of hard about being open source, right?

open source 有哪些地方是有点难的，对吧？

22:46

Like there must be downsides, like just trying to get an honest take on both sides.

就像肯定会有一些缺点，就是想要对两边都有一个诚实的看法。

22:51

Yeah, they're definitely downsides and it comes at a cost, right?

是啊，肯定有缺点，而且是有代价的，对吧？

22:57

The benefits are, it's almost time to build in the open.

好处是，差不多是时候 build in the open 了。

23:01

It's awesome to have like a small, a small repo as well, like whenever we hire someone

有个小的、小的 repo 也很棒，就像每当我们招人，

23:07

and they join the critics team is like, they've seen the repo before, they've looked

然后他们加入 critics team 的时候，就好像，他们之前已经看过 repo，看过

23:11

at PR, they're like boarding is done.

PR 了，他们就会觉得 onboarding 已经完成了。

23:13

Yeah, it's it's done.

对，就是，已经完成了。

23:14

Yeah, it's like an onboarding is just like, you use codex to look at the repo, you know,

对，感觉 onboarding 就像是，你用 codex 看一下 repo，你知道，

23:17

with you and you ask some questions.

和你一起，然后问一些问题。

23:18

But it's like, it's not, it's not a secret issue that you can get productive right away.

但这就像是，这不是，这不是什么秘密，你马上就能变得高效。

23:23

We get a lot of good contributions.

我们会收到很多很好的贡献。

23:24

Although we get like, you know, a tsunami of like random stuff as well.

虽然我们也会收到，你知道，像海啸一样的一大堆乱七八糟的东西。

23:29

Obviously you and everyone else.

显然，你和其他所有人也一样。

23:31

Right open source is changing.

对，open source 正在改变。

23:32

I think this is one of the examples.

我觉得这是其中一个例子。

23:34

That's right.

没错。

23:35

And then to me, it just, and to a lot of the team, it just brings a lot of energy to just be

然后对我来说，它就，对团队里很多人来说，它就带来很多能量，就只是……

23:39

part of the community and like be directly contributing.

成为社区的一部分，然后就是直接做贡献。

23:43

Not just saying that we care about the community, but actually doing things that,

不只是说我们关心社区，而是真的去做一些事情，

23:46

you know, you can see it's it's costing us effort, right?

你知道，你能看到这、这其实是在耗费我们的精力，对吧？

23:49

We don't have to do it.

我们并不是非得做这个。

23:51

And the downsides are, you know, it's separate from the rest of our code.

而缺点是，你知道，它和我们其余的 code 是分开的。

23:55

So, you know, sometimes we have to draw like artificial boundaries and,

所以，你知道，有时候我们得划一些像人为的边界，而且，

23:59

you know, work across multiple repos.

你知道，要跨多个 repos 工作。

24:01

When we're working on something particularly exciting and, you know, we're building it

当我们正在做一些特别让人兴奋的事情，而且，你知道，我们正在构建它

24:06

in the open, then, you know, at times we find that, you know, others copy it, you know,

公开出来，然后呢，你知道，有时候我们会发现，你知道，别人会抄它，你知道，

24:10

before we have the time to release it.

在我们还没来得及 release 它之前。

24:12

And it's like, it's just a little bit sad.

而且感觉就是，这还真有点让人难过。

24:16

But also it's like, it's quite a game, you know, it's like, you're building

但另外，感觉又像是，这挺像一场游戏，你知道，就像，你在构建东西

24:20

and you put it open and it's like, you know, that's that's sort of like the contract that you

然后你把它公开出来，感觉就像，你知道，那、那就有点像你

24:23

signed is like, you know, you can copy it.

签的那份合约，就是，你知道，你可以复制它。

24:25

We have a very permissive license as well.

我们也有一个非常 permissive 的 license。

24:27

But it does sting a little bit when you're working on something and you're like, you know,

但当你正在做某个东西，然后你会觉得，你知道，还是会有点扎心。

24:31

and then the third thing is just like everyone else is like, you know,

然后第三件事就是，其他人都这样，你知道，

24:34

we are overwhelmed with, you know, random contributions and, you know, we have to deal with that

我们被，你知道，随机的 contributions 搞得焦头烂额，而且，你知道，我们还得处理那

24:39

additional tax.

额外的税。

24:40

But then that pushes us to, you know, also like try and solve for it, right?

但这又会推着我们，你知道，也去试着解决它，对吧？

24:45

Which I think is good.

我觉得这是好事。

24:46

And on top of the open source, one thing that surprised me about Codex and I didn't even know about

而且除了 open source 之外，Codex 有一点让我很惊讶，我直到最近才知道，

24:50

it until recently, it's not tied to the OpenAI models.

它并不绑定 OpenAI 的 models。

24:54

You can use other models with Codex, you know, like putting myself in a vendor's shoe,

你可以把其他 models 跟 Codex 一起用，你知道，就像把自己放到供应商的角度，

25:00

it might not be very obvious because again, all the other brands that I look at when they do a

这可能不太明显，因为再说一次，我看过的其他所有品牌，他们做一个

25:06

CLI, it's kind of use it with our models.

CLI 的时候，基本上就是拿它跟我们的 models 一起用。

25:08

Again, what made you decide to be this permissive about, you know, using or allowing to use

再说，是什么让你决定在这件事上这么宽松，你知道，允许使用或允许别人使用

25:15

your harness with other models?

你的 harness 和其他 models 一起用？

25:17

It felt quite natural.

这感觉很自然。

25:20

If you are part of this community and building an excellent coding harness,

如果你是这个社区的一员，而且在做一个很棒的 coding harness，

25:28

is like, why would you couple it to your model?

就会觉得，为什么非把它跟你的 model 绑在一起呢？

25:32

That felt like quite disappointing to make that decision.

做出那个决定会让人觉得挺失望的。

25:35

So it didn't feel right.

所以感觉不太对。

25:37

And in general, it's like I think, you know, it's like I kind of tried to make decisions that

而且总的来说，就像我觉得，你知道，就有点像我在试着做一些决定，就是

25:42

I'm like, yes, you know, it's just like I can just like, explain it, you know, it is correct.

我会想，对，你知道，就有点像我能直接解释它，你知道，它是对的。

25:46

It's the same reasoning with, you know, it is open source in the first place.

同样的道理，你知道，它本来就是 open source。

25:50

It would have been trivial for anyone to fork it and then add support for another thing.

任何人去 fork 它，然后再给另一个东西加 support，本来都是很 trivial 的。

25:55

But then you're just encouraging people to just like, you know, go and use that fork.

但那样的话，你就只是在鼓励大家，就有点像，你知道，直接去用那个 fork。

26:00

And then not suddenly you have overhead.

然后，不是突然之间，你就有 overhead 了。

26:01

And the only reason you have a fork is because, you know, you wanted to change like 10 lines of

而你会有一个 fork 的唯一原因，就是，你知道，你想改大概 10 lines 的

26:06

go to add support for like another model provider.

还要去支持另一个 model provider。

26:09

That feels very silly.

这感觉特别傻。

26:10

So like, you know, why not just support it in the first place?

所以说，你知道，为什么不干脆一开始就支持它呢？

26:13

The other thing is we benefit a lot from like being able to just give optionality.

另一件事是，能直接提供 optionality 这一点让我们受益很多。

26:18

So, you know, it's like maybe today, you know, you love using OpenAI models.

所以，你知道，就像可能今天，你知道，你很喜欢用 OpenAI models。

26:25

And you know, you're super productive with them.

而且你知道，你用它们效率超高。

26:27

But like tomorrow, there's a new model that comes out.

但就像明天，又有一个新的 model 出来了。

26:29

You want to try it at.

你想试试它。

26:30

Why force you to go and completely change your setup just to try a new model?

为什么要逼你只为了试一个新的 model，就非得去彻底改掉你的 setup 呢？

26:35

And then we benefit from the feedback that we didn't get, which is like, you know,

然后我们还能从本来拿不到的 feedback 里受益，就有点像，你知道，

26:38

maybe there's something that you liked about that model.

也许那个 model 有你喜欢的某个点。

26:40

Maybe it actually didn't work well.

也许它其实效果并不好。

26:42

But it's sort of like being nice to our users and to the community.

但这有点像是对我们的用户和社区友善一点。

26:46

It's like, you know, feels like the right thing to do here.

就像，你知道，感觉在这儿这才是该做的事。

26:50

And then, you know, we also, we will also try like other models, right?

然后，你知道，我们也会，我们也会去试试其他 models，对吧？

26:53

So, you know, we tried them in the same harness and, you know, it's just all, all good.

所以，你知道，我们在同一个 harness 里试了它们，然后，你知道，一切就是都，都挺好。

26:58

And then this is all also often like this optionality is very important to companies that we work with.

然后，这一切也常常是这样，这种可选性对我们合作的公司来说非常重要。

27:05

Yeah.

嗯。

27:06

And this is something that, you know, we absolutely need into.

而且这是，你知道，我们绝对需要的东西。

27:08

This last point, I think, you know, as any serious company,

最后这一点，我认为，你知道，作为任何一家严肃的公司，

27:11

you want to have optionality and you want to use a tool that gives you that optionality.

你都会想要有可选性，并且你想要使用一个能给你这种可选性的工具。

27:15

But I kind of appreciate it because I feel to me like it's kind of honest, like look,

但我有点欣赏它，因为我觉得对我来说它有点诚实，就像你看，

27:18

like it forces the whole company to beat, to compete the best in everywhere in the model layer

就像它迫使整个公司在 model layer 的各个地方都要做到最好，去竞争，

27:24

and the harness layer with open source with, with chooseable models.

以及在 harness layer 上，用 open source，用，用可选择的模型。

27:27

And it kind of like doesn't, doesn't allow you to like kick back and say like, all right,

而且它有点，就是，不让你能像那样放松下来说，好吧，

27:30

we're done. We can, we can, we can hang back for a little bit for now.

我们搞定了。我们可以，可以，可以暂时先缓一缓。

27:33

Yeah, I want us to win users by having, you know, the best models, the most efficient models,

对，我想让我们赢得用户，是靠有，你知道，最好的 models、最高效的 models，

27:38

the best product. And then, you know, if we do all of these things, it's like we're going to have

最好的产品。然后，你知道，如果我们把这些都做到，那就像我们会有一段

27:42

a good time. If we sort of like force you to use the product because, you know, this one thing,

好时光。如果我们有点像是强迫你用这个产品，因为，你知道，就这一件事，

27:46

it's just like then I don't think that will attract, you know, the, the very best people to work

那就好像，我不觉得那会吸引，你知道，最、最优秀的人来

27:51

on this product either. And so, you know, we're doing our best work here. We care a lot about the

做这个产品。所以，你知道，我们在这里尽最大努力。我们非常在意

27:56

experience. It should feel the light hole, you know, like it doesn't, irrespective of the model

体验。它应该感觉就是那个 light hole，你知道，就像它不，不管 model 是什么

28:01

that powers it, you should feel the light hole. I love the idea of winning based on merit,

支撑它的，你应该感受到 light hole。我喜欢基于实力取胜的想法，

28:05

not based on lock-in. And this is a perfect time to mention our season sponsor,

而不是基于 lock-in。现在正是提到我们本季赞助商的好时机，

28:09

entire who also play by the same rules. Like it or not, Git is becoming a bottleneck for modern

Entire，他们也遵守同样的规则。不管你喜欢与否，Git 正在成为现代

28:14

agent heavy software development. Devs are creating more code with agents. These agents are

agent 密集型软件开发。开发者正在用 agent 创建更多代码。这些 agent

28:18

pushing more code. Many Devs are running more parallel agents. These are pushing even more code.

推送更多代码。许多开发者正在运行更多并行的 agent。这些又在推送更多代码。

28:23

GitHub is clearly struggling to keep up and has frequent outages. So what's the solution?

GitHub 明显难以跟上，并且频繁出现服务中断。那么解决方案是什么？

28:28

Entire was founded by GitHub last CEO Thomas Dumpka and he rebuilt Git hosting for the agent

Entire 由 GitHub 前 CEO Thomas Dumpka 创立，他为 agent 重建了 Git hosting

28:33

to care from scratch. Entire was built to be very fast and to have your repos regionally close

从头开始关心。Entire 被构建得非常快，并且让你的 repos 在地理上离你很近

28:38

to you to reduce latency, allowing for fleets of agents to push in parallel. Some numbers they

给你用来降低 latency，让成群的 agents 能够并行 push。

28:44

published. Entire can handle 418 pushes per second. That's up to 89 times faster than every competitor

他们公布了一些数字。

28:50

on the market. When GitHub is down, you can still keep working and you don't even need to migrate

Entire 每秒能处理 418 次 push。

28:55

away from GitHub. You just sign up to entire and the platform mirrors your repo. And one more

这比市面上所有竞争对手都快了最多 89 倍。

29:00

neat thing. Have you ever wondered what prompt resulted in this specific code being generated?

当 GitHub 宕机时，你仍然可以继续工作，甚至不需要从 GitHub 迁移走。

29:05

I find that the prompt and conversation with the agent carries more information than the peer

你只要注册 Entire，平台就会镜像你的 repo。

29:09

itself, at least for me. Entire captures all the prompt history with your agent right in the

还有一个很酷的功能。

29:14

repo easy to check back. And has a pretty innovative UI to show all of this. If you're looking for

你有没有想过，是什么 prompt 生成了这段具体的代码？

29:19

Git hosting that works even when GitHub is down, head to entire.io-sash-primatic install the CLI

就算 GitHub 挂了也能用的 Git hosting，前往 entire.io-sash-primatic 安装 CLI

29:24

and mirror your repo with a click. I've already done it. Oh, and did I mention that it works with

然后点一下就能 mirror 你的 repo。我已经弄好了。哦，还有，我有没有提过它能配合

29:29

any agent and its open source? I'd also like to mention our season's sponsor, Antisysis.

任何 agent，而且它是 open source？我还想提一下我们这一季的赞助商，Antisysis。

29:35

Tiibo talked about how the experience of the software you use should feel delightful.

Tiibo 谈到，你使用的软件体验应该让人感到愉悦。

29:39

The life full includes no annoying bugs. But when you're using agents to write your code,

The life full 里没有烦人的 bug。但当你用 agents 写 code 时，

29:43

how do you avoid shipping bugs? Reviewing every line of code is becoming a challenge with the

你怎么避免把 bug 带到线上？要 review 每一行 code 正变得越来越难，因为

29:48

amount of code that agents generate, which is why Antisysis goes well beyond code review. Antisysis

agents 生成的 code 量太大了，所以 Antisysis 远不止 code review。Antisysis

29:53

runs your whole system in a hostile simulation. This simulation includes both targeted testing

会在一个 hostile simulation 里跑你的整个 system。这个 simulation 既包括 targeted testing

29:58

and fast testing. By running this simulation, it finds every bug before your users do. And because

以及 fast testing。通过运行这个 simulation，它能在你的用户发现之前找到每一个 bug。而且因为

30:04

the simulation is fully deterministic, it doesn't only find bugs, it gives you a perfect reproduction

这个 simulation 是完全 deterministic 的，它不仅找到 bug，还能给你

30:09

of every issue, which makes it much easier to fix issues. The first thing I talk when I heard about

每一个 issue 的完美 reproduction，这让修复 issue 变得容易得多。我听到

30:15

Antisysisis is that automated bug discovery and fully deterministic testing sounds like science

Antisysis 时，我首先谈到的是 automated bug discovery 和 fully deterministic testing 听起来像科幻

30:20

fiction, but it's actually hardcore engineering under the hood. Jane Street, fly.io on the

小说，但实际上底层是硬核 engineering。Jane Street、fly.io 和

30:25

Etsy, the community-shaped agent-riching code with full confidence because they know it's been

Etsy，社区塑造的 agent-riching code，充满信心，因为他们知道它已经过

30:29

verified by Antisysis. To see more case studies and details, head to Antisysis.com-sash-pragmatic.

Antisysis 的验证。想查看更多 case studies 和细节，请访问 Antisysis.com-sash-pragmatic。

30:35

And with this, let's get back to Tiibo and why competition between tools is great.

那么，让我们回到 Tiibo，以及为什么工具之间的竞争是件好事。

30:40

Yeah, and I think as an engineer, like I always see that whenever there's competition,

是啊，而且我觉得作为一名工程师，我总是看到，只要有竞争，

30:44

as to someone who's using tools, it's always amazing. Like I remember when Microsoft had

对使用工具的人来说，这总是很棒。

30:49

with JetBrains, with the IDE wars, and then there's a cloud battling with each other with all

我记得 Microsoft 和 JetBrains 之间有过 IDE 大战，然后 cloud 之间也在用各种功能互相竞争。

30:54

the features. And now, of course, we have the harnesses. We have the models. And as a user is great,

当然，现在我们有 harnesses，有 models。

30:59

because now we have more choice. They just develop faster, I guess our voice gets heard a bit better,

作为用户来说这很棒，因为我们现在有更多选择。

31:05

so it's great to hear. Speaking of the harness, can you tell me how it works today in the sense of

它们就是开发得更快，我猜我们的声音也能更多地被听到，所以听到这些很棒。

31:10

when I start a codex task? Does it run always on my machine? Does it choose the cloud? Does it use a

说到 harness，你能告诉我它今天是怎么运作的吗？比如当我开始一个 Codex task 的时候？

31:20

sandbox, and how do I control this, or how much should I know about this as an engineer?

它总是运行在我的机器上吗？

31:27

Yes. So by default, it runs sandboxed. Everything that if there is a command that should run

是的。所以默认情况下，它是在 sandbox 中运行的。一切，如果有一个命令需要运行

31:39

with additional permissions outside of the sandbox, it will ask you as a user for permission,

需要在 sandbox 之外有额外的权限，它会向你这个用户请求权限，

31:46

that everything, every tool execution happens within the sandbox by default,

也就是说，所有事情，每次 tool execution 默认都发生在 sandbox 内，

31:50

and it runs entirely on your local machine. And this has been the case for more than a year now.

而且它完全在你的 local machine 上运行。而且这种情况已经持续一年多了。

32:01

But it is something that is evolving and shifting, where you can select to run this

但这是一个不断演变和转变的东西，你可以选择运行这个

32:07

in the cloud, which then runs in a managed VM, where it's the same VM that you get through

在 cloud 中，然后它会在一个 managed VM 中运行，这个 VM 和你通过

32:14

Chad J.P.T. work. You can inspect it, but it runs in a catacontainer. It's a secure environment.

Chad J.P.T. 工作获得的 VM。你可以检查它，但它运行在一个 catacontainer 中。这是一个安全的环境。

32:23

Everything runs inside of that VM, and it doesn't run on your machine. And then the only thing

所有东西都在那个 VM 内运行，不在你的 machine 上运行。然后唯一的一件事

32:28

that happens in your machine is the input and then the streaming back of the output. And so that

在你机器上发生的是 input，然后是 output 的 streaming 回传。所以这

32:35

obviously, that is much nicer on your CPU and your machine, and you can scale much, much more.

显然，这对你的 CPU 和机器来说友好得多，而且你能 scale 得多得多。

32:42

And this is just a step. It's going to be much more seamless in the future,

而这只是一步。未来会无缝得多，

32:49

to use cloud machines and then maybe have a combination of partial execution on your laptop,

去使用 cloud machines，然后也许有一个组合：一部分 execution 在你的 laptop 上，

32:55

partial execution on cloud machines. And really, the thing that we're thinking about that is

一部分 execution 在 cloud machines 上。而真的，我们正在想的那件事是，

33:01

very natural is as models just get better and more capable, they can leverage so much more compute

非常自然的一点是，随着 models 越来越好、能力越来越强，它们可以利用多得多的 compute

33:09

and many more resources than are available on your local machine. And so it would be a constraint

以及比你的 local machine 上可用的多得多的资源。所以这会是一种限制

33:17

at some point to just limit execution on your local machine. But one thing that is great about

到某个时候，只把 execution 限制在你的 local machine 上。但有一件很棒的事是

33:22

running locally, and I think the reason I love it when it runs locally, of course, it's a pain

在本地跑，我觉得我喜欢它在本地跑的原因，当然，这很麻烦

33:26

because if I'm doing some work, it's like, you know, I have several agents, it's eating CPU.

因为如果我在做点工作，就有点像，你知道，我有好几个 agents，它一直在吃 CPU。

33:31

If I want to close my laptop, I cannot kind of leave it like hop open when I was in one of the

如果我想合上笔记本，我没法就那样让它开着，就像我当时在一家

33:37

offices of an AI company. I had it open and they're like, are you running agents? I'm like,

AI 公司的办公室。我让它开着，他们就说，你在跑 agents 吗？我说，

33:41

yeah, I have one running. I get it. But the reason I do it because I have my local tools,

对，我有一个在跑。我明白。但我这么做是因为我有我本地的 tools，

33:46

I have my local Postgres database, I have my this than that. How are you thinking about the

我有我本地的 Postgres database，我有我这个那个。你怎么看

33:52

cloud is amazing, but it doesn't have this setup or it's just a pain to set it up. Are you thinking

cloud？它很棒，但它没有这个 setup，或者要 setup 起来就是很麻烦。你是在想

33:57

or are you experimenting with making these setups? And I'm kind of reminded of a topic that

还是在试验做这些 setups？然后我有点想起一个话题，

34:03

we talked about pre-A, which is cloud development environments in like 2022, 23,

我们之前聊过 pre-A，也就是大概 2022、23 年那会儿的 cloud development environments，

34:07

they're hot and then we talked about AI more. I think outside of large tech companies,

它们当时很火，后来我们更多地聊到了 AI。我觉得在大型科技公司之外，

34:14

like cloud dev boxes really never took off because there's a very big upfront cost.

cloud dev boxes 其实一直没真正起来，因为前期成本非常高。

34:20

And then you need to pay like a maintenance cost as well. And you just don't benefit from it as

而且你还得付 maintenance cost。而且你根本享受不到它带来的好处，作为

34:24

like a solo developer or like a small team. With the level of capabilities that we have in

独立开发者或者小团队。以我们现在在

34:29

agents now is like the setup almost as free, right? So like this setup cost and this maintenance

agents 上具备的能力水平，setup 几乎就跟免费一样，对吧？所以这个 setup cost 和 maintenance

34:34

cost is like if your agent is capable of doing it, you know, you should just do it for you.

cost 就像，如果你的 agent 有能力做这件事，你知道，你就应该直接让它替你做。

34:39

So for example, if you're saying like, hey, you know, I have like I have my local

所以比如说，如果你说，嘿，你知道，我有，就像我有我本地的

34:44

decolite or I have a local server and MCPs and whatnot. It's like, how hard is it to actually

decolite，或者我有一个 local server 和 MCPs 之类的。就像，实际上到底有多难

34:51

configure exactly the same setup and keep it in sync on a cloud dev box?

在 cloud dev box 上配置出完全一样的 setup 并保持同步？

34:56

Well, maybe it's not that hard if the model just does it for you. And so I think we're going to

嗯，也许如果 model 直接帮你做，就没那么难。所以我觉得我们会

34:59

see your resurgence of, you know, fully cloud orchestrated machines, which then frees you

看到，你知道，完全 cloud orchestrated 的机器重新兴起，然后这把你

35:07

from your laptop, right? It's like one thing that we've, you know, ton of success with

从笔记本上解放出来，对吧？就像我们有一件事，你知道，取得了巨大成功，就是

35:11

your chat if you work is like it's just available on your mobile. I start my day just dictating

你的 chat if you work，就像它就在你手机上可用。我每天一开始就是口述

35:16

a bunch of tasks into it next to the coffee and it just does it. It has access to my calendar.

在咖啡旁边把一堆任务丢给它，然后它就做了。它能访问我的日历。

35:21

It has access to my email. It has access to Slack. And it's just so awesome to just be able to

它能访问我的邮件。它能访问 Slack。而且能直接做到这些，真是太棒了

35:26

walk around and you know, get stuff done without having to, you know, carry my laptop everywhere.

到处走走，你知道，把事情搞定，不用，你知道，到哪儿都得带着我的笔记本电脑。

35:32

And I think it's the same things like, you know, we shipped like critics remote where, you know,

而且我觉得也是一回事，你知道，我们上线了 critics remote，在这个里面，你知道，

35:36

execution is like still happening on your laptop. But it would be wonderful if, you know,

execution 其实还是在你笔记本电脑上跑。但如果，你知道，

35:41

you didn't have to keep your laptop open. Can you tell me a bit on how in the past, how did you

你就不用一直开着笔记本电脑了。你能跟我稍微讲讲，过去你们是怎么

35:46

improve codex? Because I remember when I first used codex, this was one of the early versions,

改进 codex 的吗？因为我记得我第一次用 codex 的时候，那是早期版本之一，

35:52

you know, like, you could talk to it. It did stuff. But for example, I said like, all right,

你知道，就像，你可以跟它对话。它会做些事情。但比如说，我说，好吧，

35:56

make this change and it did that change. And I had units test and it didn't run it. And then later,

把这个改一下，它确实做了那个改动。而且我有 unit tests，但它没跑。然后后来，

36:01

a few months later, I don't know exactly when it just started to run it automatically.

几个月后，我不太确定具体是什么时候，它就开始自动跑了。

36:04

Were these things, did you improve the, you know, the script that runs, you know, the instructions?

这些东西，你们有没有改进，那个，你知道，运行那些 instructions 的 script？

36:09

I'm not sure how exactly you call the, you know, the bootstrapping script or whatever that is.

我不太确定你们具体怎么称呼那个，你知道，bootstrapping script 还是什么的。

36:14

Is it improving the model? Like, as a dev, how can I imagine you making each version better

这会让 model 变得更好吗？就像，作为一个 dev，我该怎么想象你们让每个版本变得更好

36:20

between the harness and then between the model and like, what's the connection between the two?

在 harness 这边，然后在 model 那边，就像，这两者之间是什么联系？

36:25

Yeah. This is a good question. So the, the harness in a sense is always a little bit ahead of the model.

对。这是个好问题。所以，那个，harness 在某种意义上总是比 model 稍微领先一点。

36:33

Oh, really? How so?

哦，真的吗？怎么说？

36:35

Oh, what, what I mean by that is that you, you have the model, it's capable of certain things,

哦，我，我的意思是，你有这个 model，它能做某些事情，

36:40

but then you, you started up with like a couple of crutches so that you can actually do the thing

但然后你，你一开始是带着几根拐杖起步的，这样你才能真的把这件事做起来

36:48

to a level of reliability and in a way that is like efficient and also with the behavior that you

达到一种可靠性的水平，而且方式要高效，同时还要有你作为用户所期望的那种行为。所以，这其实就是 harness 的角色，对吧？就是，你知道，提供 guardrails，比如 safety，让它更高效，让它更 steerable、更 controllable。然后 harness 通常也负责，你知道，我们所谓的那个 developer message，它就像在每个 turn 开始的时候被注入到 context 里。所以这显然会影响，嗯，目的就是影响 agent 在整个 turn 里的行为。你得到的大部分东西，其实都是 harness 和 model 的结果。一开始，可能你会说，哦，它不跑 tests。所以，你知道，你就得

36:57

expect as a user. And so like, that's the role of the harness, right? It's like, you know, provide

这是作为用户所期望的。所以就像，那就是 harness 的角色，对吧？就像，你知道，提供

37:01

guardrails, like safety, make it more efficient, make it more like steerable, controllable.

guardrails，比如安全，让它更高效，让它更 steerable、controllable。

37:07

And then the harness usually is also responsible for, you know, what we call like the, the developer

然后 harness 通常也负责，你知道，我们所谓的那个，那个 developer

37:14

message, which is like infected in the context at the start of each turn.

message，它就像在每个 turn 开始时被注入到 context 里。

37:21

And so that affects obviously like the, the purpose is to affect like the behavior of the agent

所以这显然会影响，就像，目的是影响 agent 的行为

37:27

throughout, throughout the turn. A lot of what you have is like the result of the harness and the

在整个，整个 turn 中。你得到的很多东西，就像 harness 和

37:33

model. Initially, like maybe you're like, oh, it doesn't run tests. So, you know, you have to

model 的结果。一开始，就像你可能觉得，哦，它不跑 tests。所以，你知道，你必须

37:37

remind it to run tests. And then, you know, we train a battle model that is just, you know, capable of

提醒它去跑测试。然后，你知道，我们训练一个 battle model，它其实，你知道，能够

37:44

like better reflecting on what is it that you really want when you ask for something. And then,

就是更好地反思，当你提出要求的时候，你真正想要的是什么。然后，

37:50

you know, you don't actually have to tell it anymore. So over time, what we see is like the system,

你知道，你其实不用再告诉它了。所以随着时间推移，我们看到的就是，系统，

37:55

the developer message shrinks. And then the harness also shrinks. Inside of the codex,

developer message 会缩小。然后 harness 也会缩小。在 codex 里面，

38:00

same, do you have specific goals? Do you say like, all right, right now the codex as a harness

同样，你们有具体的目标吗？你会不会说，好吧，现在 codex 作为一个 harness

38:06

and model to combine, it's not very good at this, or it's kind of doing silly mistakes or here.

和 model 结合起来，它在这方面不是很好，或者它有点犯愚蠢的错误，或者在这里。

38:12

How can I imagine that how, as the engineering team, how you're working on the next, you know,

我怎么能想象，作为工程团队，你们是如何在开发下一个，你知道，

38:17

version of codex is the, because the thing that I don't really get as a, as a dev is like, okay,

版本的 codex 是，因为作为，作为开发者，我不太理解的事情就是，好吧，

38:22

there's a model, which to me is this magical thing, which will get better. Of course, I'm sure you

有一个 model，对我来说它就像个神奇的东西，它会变得更好。当然，我确定你

38:26

have some feedback channels, but the also the harness, which is the tools that you're building,

有一些反馈渠道，但还有 harness，也就是你在构建的那些工具，

38:30

like that's probably what the team is responsible for. How do you set even your goals, right? Like,

就，这大概就是团队负责的事情。那你怎么设定自己的目标呢，对吧？就，

38:33

in traditional software, you'll be like, we will build this feature and you build that feature

在传统软件里，你会说，我们要做这个 feature，你做那个 feature

38:36

because you know how to do it, but it feels a bit more fuzzy to me this development process.

因为你知道怎么做，但这个开发过程对我来说感觉更模糊一些。

38:40

Yeah, that's it. And that's why we, we code design, you know, most things. And it's a process where

对，就是这样。这也是为什么我们，我们大部分东西都是 code design，你知道。而且这是一个过程，其中

38:46

it's a collaboration between research and the engineering team, like primarily building the core,

它是研究团队和工程团队之间的协作，主要是构建核心的，

38:52

the core agent harness. It's always a, it's always a question of like, okay, we see today that,

core agent harness。它总是，总是这样一个问题，就像，好吧，我们今天看到，

38:59

you know, we are very good at this, but we're not very good at this. And you know, we have a desire

你知道，我们很擅长这个，但我们又不太擅长这个。而且你知道，我们有一种渴望

39:03

to do like another thing because it would be a very cool product feature. And then, you know,

去做另一件事，因为那会是个很酷的产品功能。然后，你知道，

39:08

we always sort of like look at it. It's like, okay, this should just be like a harness change,

我们总是会有点像这样看它：好吧，这应该只是个 harness 改动，

39:12

or should this be a model change? And if it's a model change, like, how soon can we have it?

还是说这应该是个 model 改动？如果这是 model 改动，那，我们多快能搞定它？

39:16

And we have it in a month, can we have it in, you know, three months, six months? And we sort of like

我们一个月能搞定吗，还是说你知道，要三个月、六个月？然后我们有点像是

39:20

work through that. And then depending on, you know, how soon we can just fix it in the model at

把这事捋一遍。然后取决于，你知道，我们多快能直接在 model 里修好它，在

39:25

which level of training, then we might decide to not even do something in the harness at all.

哪个 training 层面，然后我们可能会决定干脆完全不在 harness 里做任何东西。

39:30

And not, and just wait for for the model to solve it. You know, it's agents all the way, right? So

而不是，不，而是就等 model 来解决它。你知道，一路都是 agents，对吧？所以

39:35

we use agents to analyze like a lot of the feedback to like, you know, come up with themes, you know,

我们会用 agents 来分析很多反馈，就是，你知道，提炼出一些主题，你知道，

39:40

to just help us have these conversations and decide on priorities. But we analyze it across

来帮我们进行这些对话，并决定优先级。但我们会在

39:46

all of coding. We analyze it across like all of like, you know, the other domains like finance,

整个 coding 里分析它。我们也会在，就是，你知道，其他领域，比如金融、

39:51

comms, marketing, you know, all the things where our users are using these agents nowadays. And

传播、市场营销，你知道，所有用户现在用这些 agents 的地方。还有

39:56

there's like, you know, sub categories within those. And then we roughly know like, you know,

就是，你知道，里面还有子分类。然后我们大致知道，就是，你知道，

40:00

how well we perform. And then we're always pushing different here. And there's a thing that is

我们做得怎么样。然后我们总是在这里推动不同的东西。还有一件事，它

40:04

interesting is like, as we make, you know, as our pre-training model gets better, as we make the

很有意思，就是，随着我们让，你知道，随着我们的 pre-training model 变得更好，随着我们让

40:08

overall model better, like the whole thing lifts up. But then there are sometimes things that we pay

overall model 更好，整个东西就会提升。但有时候也会有一些事情，我们会 pay

40:13

a little bit more attention to. You mentioned you in the analyze agents all all the way. Can we talk

再多关注一点。你提到过，你在 analyze agents 里一路走来。我们能聊聊

40:17

about the software development lifecycle on codex in the sense of whenever a new engineer joins

codex 上的 software development lifecycle 吗？也就是每当一个新工程师加入

40:25

a team, any team, it's like, okay, how are things done here? And you know, back pre-AI,

一个团队，任何团队，都会想，好吧，这里的事情是怎么做的？而且你知道，回到 pre-AI 那时候，

40:30

it would have been you joined the company like Uber or Google. And they would tell you that cool

以前会是你加入了像 Uber 或 Google 这样的公司。他们会告诉你，酷，

40:34

the way it works is we have an idea or the PM has an idea, we make a plan, we get together,

运作方式是这样的：我们有个想法，或者 PM 有个想法，我们做个计划，大家聚在一起，

40:38

we do some estimations, we break up the work, we code the work, we do tests, we do code reviews,

我们做一些 estimations，把工作拆开，我们写 code，我们做 tests，我们做 code reviews，

40:43

we release, we do feature flags, and then, you know, we were on call. That's how it used to be.

我们 release，我们做 feature flags，然后，你知道，我们当时要 on call。以前就是这么干的。

40:49

When someone joins the codex team, you know, they've been contributing to the open-source part,

当有人加入 codex 团队时，你知道，他们一直在给 open-source 那部分做贡献，

40:55

but what do you tell them? How do things get done here? If they're like a total newbie?

但你会跟他们说什么？这里的事情是怎么推进的？如果他们完全是个新人呢？

41:00

I introduced them to great people. And then the thing that they hear the most about when they have

我会把他们介绍给很棒的人。然后他们最常听到的，就是当他们有

41:05

a question is like, have you asked codex? And codex is just, by default,

问题的时候，别人会说，你问过 codex 了吗？而 codex 就是，默认情况下，

41:12

the open-source is plugged into everything, so it has access to Slack, it has access to older

这个 open-source 接入了所有东西，所以它能访问 Slack，能访问旧的

41:16

documents, access to older code, and it still surprises new starters that you can basically ask

文档，能访问旧的代码，而且这仍然会让新来的同事很惊讶，因为你基本上可以问

41:23

it anything, and it will very often just come up with a really good response. And so the easiest

它任何东西，而它经常会直接给出一个非常好的回答。所以最简单的

41:31

way to understand the state of a project or who's working on something or why decision was made

了解一个项目现在什么状态、谁在做什么，或者为什么做了某个决定的方法

41:37

is like codex knows about it all internally. So you just use all of that. We do a lot of work

就是，codex 在内部全都知道。所以你直接把这些都用起来就行。我们做很多工作

41:44

in, you know, for that reason, we do a lot of work in public channels. We open up documents with

因为，你知道，出于这个原因，我们在公开频道里做了很多工作。我们会把文档开放，

41:50

like, you know, fairly broad permissions, and so that, you know, everyone has access to this

就是，你知道，给相当广泛的权限，这样一来，你知道，每个人都能接触到这些

41:54

information as well. And so that, you know, your agent can go through things and like, you know,

信息。这样呢，你知道，你的agent就可以去翻阅这些东西，然后就是，你知道，

41:57

reason through things. And then, you know, we have a couple of other things that are just really

对事情进行推理。然后，你知道，我们还有几个其他东西，对团队生产力和团队协作

42:02

very helpful for team productivity and team collaboration that we haven't released yet, but are

真的非常有帮助，我们还没发布，但会推出，其中一些会在DevDay上亮相。

42:08

going to come like some of it at DevDay. Although that just sort of like makes you very grounded

虽然那只是有点像让你非常接地气、跟团队其他人保持同步，让你能够，就是，你知道，

42:12

and in tune with the rest of the team, and allows you to like, you know, just very, very quickly

非常非常快地理解事情的现状，并且自己产出东西。总的建议就像，

42:17

understand the state of things and produce things yourself. The general recommendation is just like,

理解事情的现状，然后自己产出东西。一般的建议就像是，

42:23

care about the user, care about the coherence of the product, care about the models and where they're

关心用户，关心产品的连贯性，关心 model 以及它们要往

42:28

going. If you're doing something and, you know, you're building like this 10,000 lines of code crutch

哪儿走。如果你在做某件事，而且，你知道，你在搭一个像 10,000 行 code 这样的拐杖

42:34

to work around the model flaws, you know, you're probably doing the wrong thing. So we have a set of

来绕开 model 的缺陷，你知道，那你很可能就是在做错事。所以我们有一套

42:37

principles, but it's just really sort of like a team culture and ethos at this point. And, you know,

原则，但到了现在，它其实更像是一种团队文化和精神气质。而且，你知道，

42:45

it's just very much sort of like carries on, you know, when people join, it's just like through

它就非常像是会一直延续下去，你知道，当有人加入时，就像是通过

42:49

the rest of the team just like, you know, sort of like teaching the route.

团队里的其他人，你知道，有点像在教这条路。

42:52

And then when I have an idea, I think it's a good idea. I talk through with codex. Maybe I talk

然后当我有个想法，我觉得这是个好主意。我会跟 codex 把思路过一遍。也许我会聊

42:57

with you with some of my colleagues, like here's a cool new feature I'm going to build as my first

跟你、跟我的一些同事聊，比如，这是我准备作为第一个来做的很酷的新 feature

43:01

first contribution or first major contribution to codex. How do I go about that? Obviously,

第一次给 codex 做贡献，或者第一次做重大贡献。我该怎么做呢？显然，

43:06

I go down with codex. I obviously test it and make sure that it works. From there on, what's the

我会跟着 codex 走。我显然会测试它，确保它能跑。从那之后，流程是什么？

43:11

process? Do you still have the concept of code review or AI code review verification rolling out

你们还有 code review 这个概念吗，还是说会 roll out AI code review verification，

43:17

a verifying of stage rolled out? You know, the thing is because codex itself, it goes out to

会 roll out 一个 verification stage？你知道，事情是这样的，因为 codex 本身，它会面向

43:22

millions of people, like I just crossed a big 20 million active user mark. But if it's

数百万的人，比如我刚刚跨过 2000 万 active user 这个大关。但如果是

43:28

chat GPT, then it also goes out to like even a lot bigger number of people. Yes. But it's surprisingly

chat GPT，那它面向的人甚至还要多得多。对。但令人意外的是

43:35

like a similar process, whether you ship on codex or a chat GPT, even though chat GPT goes out to

流程居然差不多，不管你是 ship 到 codex 还是 chat GPT，尽管 chat GPT 面向的是

43:40

a billion active users and growing, you can ship a PR. You can make a change and get a chip

十亿 active user，而且还在增长，你还是可以 ship 一个 PR。你可以做个改动，然后拿到一个 chip

43:49

like the next day or even the same day. And it just goes out to a billion users and it's fine.

比如第二天，甚至当天。然后它就直接发布给十亿用户，而且也没问题。

43:53

We just really instill a sense of ownership and care. People are very empowered to make changes,

我们真的会培养一种主人翁意识和用心。大家被充分授权去做改动，

44:01

even large changes. The general thing that is being asked is evidence that it's going to be

甚至是很大的改动。通常大家要求的是，有证据表明它会

44:06

well received. Evidence that is like a worthy addition. Evidence that it is worth maintaining

很受欢迎。要有证据证明它是个值得加入的补充。要有证据证明它值得维护

44:13

over time. But also like the cost of maintenance is like just really as you know, gotten done

长期来看。但还有，就像，维护成本，就像，真的，你知道，也已经

44:17

significantly as well. So we think about these things slightly differently than you know, say like

大幅降下来了。所以我们思考这些事情的方式会和你知道的，比如说

44:22

two years ago or three years ago. The other thing as well is you know, we automate as much as

两年前或三年前稍有不同。另外，你知道，我们尽可能

44:26

possible. So like a lot of like the process of like code review and deploys and you know,

自动化。所以像很多，比如 code review 和 deploys 的流程，你知道，

44:30

catching regressions. It's like you know, all of that is like pretty much automated. And so like

抓 regressions。就像，你知道，所有这些基本上都自动化了。然后就像

44:35

you know, you get to just focus on just really the idea and you know, how it's going to help our

你知道，你就可以只专注于真正的那个想法，然后你知道，它要怎么帮助我们的

44:41

users. And you care about you know, the coherence of it all and so like the overall power of the

用户。然后你在乎，你知道，整个东西的连贯性，然后就像，整体的能力，那个

44:46

agent and making things better. And we don't we have a long, long list of things that you know,

agent，还有让事情变得更好。然后我们，我们有一长串，很长很长的清单，你知道，

44:51

we sort of like aspire to do and haven't gotten to yet. And then there's like the sort of like the

我们有点想去做但还没做到的事情。然后还有，就像，那种，就像那个

44:55

North Star direction, which is a delightful simple to use personal AGI that you know knows everything

North Star 方向，就是一个令人愉快的、简单易用的 personal AGI，你知道，它知道所有

45:02

about you, like you know, that it needs to know as access to the right resources can take like

关于你的一切，就像，你知道，它需要知道的，能访问合适的资源，可以采取就像

45:07

you know, sometimes risky actions on your behalf. But then you know, you get like the push notification

你知道，有时候是代表你采取的冒险行动。但然后你知道，你会收到像 push notification 这样的

45:11

and then you know, you can verify that. And it's like a thing that you know, you deeply understand

然后你知道，你可以验证这一点。而且这就像是一个，你知道，你深深理解的东西。

45:15

as a user. But also it knows about your schedule, it knows about your goals, it can be proactive.

作为用户。但它也了解你的日程，了解你的目标，它可以很主动。

45:19

And it should be like extremely natural. It should be something that you can control through like

而且它应该极其自然。它应该是那种你可以通过，比如说，

45:24

natural language, voice, you know, like maybe it should understand, you know, your emotions. Like

natural language、语音，你知道，就像也许它应该理解，你知道，你的情绪。就像

45:28

if it has like a camera, if it should be the most natural thing on earth, it's like it should not be

如果它有像摄像头，如果它应该是世界上最自然的东西，就像它不应该是

45:32

like a thing with 10, you know, different buttons and configurations. It's like AGI should be simple

像是一个有10个，你知道，不同按钮和配置的东西。就像 AGI 应该是简单的

45:38

to use that we kind of mentioned just briefly the review, the code review. But what what

去使用的，我们刚才简单提到了 review，code review。但是什么什么

45:43

when it's a go back to it, you worked at Google on a on a product used by, you know, like hundreds of

当回到它的时候，你在 Google 工作过，在一个被，你知道，像数百个

45:48

millions, which is Google Maps and Google is very well known for their culture of a very strict

数百万，也就是 Google Maps，而且 Google 以非常严格的 code review 文化而闻名。我觉得他们有两层 code review。有一个 language correctness review，而且我认为他们在整个行业里长期以来真的把它完善到了极致，他们确实相信它有效，并且他们在用。你觉得那部分正在如何变化，尤其是 human review？因为很长一段时间，直到大概一两年前，我可能会说，你知道，code review 有所有这些好处。知识分享，多一双眼睛，消除 bus factor，因为现在有别人理解了，当那个人不在时，他们可以顶上来。大家会讨论 architecture，而不仅仅是代码。

45:52

code reviews. They have, I think two layers of code reviews. There's a language, correct

code reviews。我觉得他们有两层 code reviews。有一个是语言正确性上的，

45:56

in this review and they've taken, I think they've really perfected it across the industry for

在这个 review 里，而且他们采用了，我觉得他们真的在整个行业里花了很长时间把它打磨得很完善，已经

46:00

for a long time and they do believe that it works and they use it. How do you think that part is

有很长时间了，而且他们确实相信它有效，并且他们在用。你觉得这部分

46:05

changing specifically the human review? Because for a very long time, until maybe a year or two ago,

具体来说，human review 是怎么变化的？因为很长一段时间里，直到大概一两年前，

46:11

I would have said, you know, code review has all these benefits. Knowledge sharing, second pair of

我会说，你知道，code review 有所有这些好处。知识分享、第二双

46:15

eyes, removing the bus factor because now someone else understands and when that person is out,

眼睛、消除 bus factor，因为现在别人也理解了，而当那个人不在时，

46:19

that they can jump in. Conversations are happening about architecture, not just the code.

他们就能顶上来。大家会聊 architecture，而不只是 code。

46:25

But now there's, you know, there's a lot more code. And what was the value of code review?

但现在，你知道，code 多太多了。那 code review 的价值到底是什么？

46:33

What in what cases? And so on your team, because you guys are so ahead of this,

在什么情况下？还有，在你们团队里，因为你们在这件事上太超前了，

46:39

where do you see humans still being or developers being involved in the review stage of

你觉得在 review 阶段，人类或者开发者仍然参与，是在哪些地方

46:44

valuable and where is it fine? Did you find it fine? It's fine to hand it off to an agent.

有价值，而哪些地方又没问题？你们觉得没问题吗？交给 agent 就行了吗？

46:51

Yeah, the role of code review is changing. One of the early projects that I did on codex was

是啊，code review 的角色正在变化。我在 codex 上做的早期项目之一，就是

46:57

like working with research on developing a code review model that was going to be to a level

和研究团队合作，开发一个 code review model，目标是达到一种水平，

47:06

where it can spot mistakes in logic and reasoning, to a degree where it would require humans like,

让它能发现 logic 和 reasoning 里的错误，到了这种程度：人类得花，

47:13

you know, multiple potentially multiple hours to capture the same level of mistake because

你知道，可能好几个小时，才能捕捉到同样程度的错误，因为

47:18

it requires like really digging like, you know, three four levels deep into like the dependencies

这需要你就像，你知道，真的往 dependencies 里深挖，挖个三四层

47:23

and like, you don't understand that maybe the documentation actually was wrong. And like,

而且就像，你不明白，也许 documentation 其实是错的。然后就像，

47:27

the implementation of like the third party dependencies like different from what you expected. And so

那些 third party dependencies 的 implementation，就像，跟你预期的不一样。然后所以

47:30

therefore your invariants are not upheld. And these things just like, you know, unless you're an

所以你的 invariants 就没法成立。而这些东西就像，你知道，除非你是

47:35

expert in that library, you wouldn't know. And therefore you have a bug. And so we developed like these

那个 library 的专家，否则你根本不会知道。因此你就有了 bug。所以我们开发了就像这些

47:40

code review models and, you know, we released them. And now they're like the same level of like

code review models，而且，你知道，我们发布了它们。而现在它们就像，有着同样水平的，就像

47:45

capability and like ability to spot these mistakes by doing like, you know, deep verification

能力，以及就像，通过，你知道，做 deep verification 来发现这些错误的能力

47:51

are like just part of the mainline models. Like when we benchmark them, it's like there are like

就像已经只是 mainline models 的一部分了。就像我们 benchmark 它们的时候，就像是有

47:54

superhuman in code review. And this is not just true for correctness. This is also true for security,

在 code review 方面简直是超人级。而且这不仅对 correctness 成立，对 security 也一样，

47:59

for example, where they're capable of like reasoning across like, you know, very, very complex things.

比如说，它们能对，你知道，非常非常复杂的东西做 reasoning。

48:04

And then you know, coming up with like, hey, you know, you have a critical security vulnerability here,

然后，你知道，得出类似这样的结论：嘿，你知道吗，你这里有个 critical security vulnerability，

48:08

which is now mandatory across like all of open AI pull requests like we block pull requests from

而现在这已经是所有 open AI pull requests 的强制要求了，就像我们会阻止 pull requests 被

48:13

merging. If you know, we flag them with like a security issue. And this is like all automatic.

merge。你知道，如果我们用 security issue 来 flag 它们的话。而这就像是全自动的。

48:19

And the role of code review now is like, I think it was always about correctness. It was always about,

而现在 code review 的角色呢，就像，我觉得它一直都是在关注 correctness。它一直是为了，

48:24

you know, ensuring that things worked. But it was also sort of like a little ritual for information

你知道，确保东西能正常工作。但它也像是一种小小的仪式，用来做信息

48:28

exchange and, you know, bringing people on the same page and like, you know, encouraging like a

交流，而且，你知道，让大家达成共识，并且，你知道，鼓励一种

48:32

discussion, which ideally would have happened before. But sometimes it just only happens like

讨论，理想情况下本来应该在此之前就发生。但有时候它就偏偏只会在

48:36

around the code because once it merged, it just actually runs in production and is doing stuff.

code 周围发生，因为一旦它 merged，它实际上就会在 production 里跑起来、开始做事。

48:40

And then you have to maintain it. So there's like this social aspect to it as well. And so I think

然后你就得维护它。所以这里面也有这种社交层面的东西。所以我觉得

48:45

all of it is changing like the correctness, the cyber security is like, I think that will be automated.

所有这些都在变，比如 correctness、cyber security，我觉得这些都会被 automated。

48:51

Really what we see and I see is there's this sort of a really

其实我们看到的、我看到的，是真的存在这样一种

48:55

discussion around the intent that takes place around the pull requests. It's like, what are you

围绕 intent 的讨论，发生在 pull requests 周围。就像，你到底

49:00

even trying to do? And is that a right thing to attempt to do? I think you can have that discussion

想做什么？那是不是一件值得尝试去做的事？我觉得你可以在

49:05

outside of the pull requests. It doesn't have to be a run code.

pull requests 之外进行那种讨论。它不一定非得是 run code。

49:07

So maybe this helps crystallize the, you know, like where a discussion needs to happen versus where

所以，也许这能帮助理清，你知道的，就是，哪些地方需要展开讨论，而哪些地方

49:13

we did it because maybe we didn't have the type of tool that we have right now.

我们那样做了，因为也许我们当时没有我们现在拥有的这类工具。

49:18

Yeah, I think this is going to change and it was like a forcing function because, you know,

对，我觉得这会改变，而且它就像一种 forcing function，因为，你知道，

49:21

you have to have that discussion or like it's good to have that discussion before you merge it

你必须进行那个讨论，或者说，在把它 merge 之前进行那个讨论是好的

49:25

and it becomes production code. But I think there are other ways to have these discussions and,

然后它变成 production code。但我认为还有其他方式进行这些讨论，而且，

49:31

you know, design things together and make sure that the intent is good. And then the code doesn't

你知道，一起设计东西，并确保 intent 是好的。然后 code 就没那么

49:36

matter as much. And it's interesting because when I think back of all my code reviews, like,

重要了。而且有意思的是，当我回想我所有的 code review 时，就像，

49:40

of course, I have like memories where like it was great. We had a good discussion or I learned

当然，我有一些记忆，比如当时很棒。我们进行了一次很好的讨论，或者我学到了

49:44

something really interesting. But a bunch of times, honestly, it was such a pain in the ass.

有些东西真的挺有意思。但说实话，很多时候那真是烦死人了。

49:49

Like I was trying to get my stuff. You're paying, hey, could you read my code and like, no,

就像我想推进我的东西。你花了钱，嘿，你能不能看一下我的 code，结果人家说，不行，

49:54

right now I'm busy. No, I really need this to unblock me and then you context switch and then

我现在正忙着呢。不行，我真的需要这个来 unblock 我，然后你就得 context switch，然后

49:59

I feel it's always been like good and bad, right? So I feel whatever we do, there will be always

我觉得这事儿一直都是有好的也有坏的，对吧？所以我觉得不管我们做什么，总会

50:05

upsides and downsides. But they're now they're just moving. So I guess one upside is, as an engineer,

有好处也有坏处。但现在它们就是在往前走。所以我觉得一个好处是，作为 engineer，

50:10

you might have to not give your attention to just kind of basic stuff that doesn't need your input

你可能就不用把注意力放在那些其实不太需要你 input 的基础东西上

50:17

per se. Yes, it's a time. And then progressively what we're going to see is also like you have an

本身。对，这就是个时间问题。然后慢慢地，我们还会看到，就像你得先有一个

50:24

agreement on, you know, the box and the overall contract of what it's supposed to do. And then,

关于，你知道，那个 box 和它整体该做什么的 contract 的共识。然后，

50:29

you know, what is inside the box, as long as you have like strict guarantees in terms of resource

你知道，box 里面是什么，只要你在 resource utilization、data access、security 这些方面有严格的保证，就像，box 里面发生什么，你知道，真的可以是什么都行。就像，其实不需要太在意。而且，真正需要达成一致的是，呃，这个 box 到底做没做到？以及必须满足哪些 invariants？我觉得那才值得，你知道，好好聊一聊，也许让你最喜欢的 agent 来协助。但一旦你有了这个，有了这种理解，那就像，改 box 里的任何东西，你知道，都不需要讨论。而且这就像，你知道，真的能保护你的注意力。

50:34

utilization, data access, security, these kinds of things, is like, what happens inside the box

utilization、data access、security，这些东西，就像是，box 里面发生了什么

50:40

is, you know, it could be literally anything. It's like, don't really need to care. And like,

就是，你知道，它可以是任何东西。就像是，真的不用太在意。而且，

50:45

really what you need to agree on is like, well, does the box actually do it? And what are the

你真正需要达成共识的是，嗯，这个 box 到底做不做这件事？还有哪些

50:48

invariants that must be satisfied? And I think that is then worthy, you know, having like a really

invariants 是必须满足的？然后我觉得这时候就值得，你知道，来一场真的

50:53

good conversation on, you know, maybe assisted by your favorite agent. But then once you have that

很好的对话，你知道，也许可以由你最喜欢的 agent 来协助。但一旦你有了那个

50:59

and you have that understanding, it's just like changing anything within the box is like, you know,

而且你有了那种理解，那就跟改 box 里面的任何东西一样，你知道，

51:02

doesn't require for discussion. And it's like, you know, just really preserves your attention.

不需要讨论。而且就像，你知道，真的能保住你的注意力。

51:06

The cost of maintenance has gone down. You know, maintenance is always such a hot topic whenever we

维护成本已经降下来了。你知道，每当我们在

51:11

build something inside of all these companies like Google Uber or even startups like building was

Google、Uber 这些公司里，甚至像创业公司里，去构建东西的时候，搭建是

51:16

the fun part, but then maintenance was the painful. And that's when we learned like, okay, it was not

有趣的部分，但之后维护就成了痛苦的部分。也就是那时候我们明白了，好吧，并不是

51:20

we're building it, et cetera. Inside of codex and open AI, what do you see maintenance becoming

我们在搭建它，等等。在 codex 和 open AI 里面，你觉得维护正在变得

51:27

cheaper, changing in terms of instead of what you're building, what the ambition is, the, I guess,

更便宜，变化在于，不再是你在搭建什么、你的雄心是什么，那个，我猜，

51:33

custom tooling, those kind of things. Maintenance is really like sort of like a talk that you pay

custom tooling，那些东西。维护其实有点像是你要随着时间

51:38

over time just to keep the exciting. And it's it's it's it's always been necessary. We'll continue

付出的代价，只是为了保持那种兴奋感。而且它一直都是必要的。它还会继续

51:43

to be necessary. But where I think it changes is like a lot of it is just going to be automated. So,

是必要的。但我觉得变化的地方在于，很多维护都会直接自动化。所以，

51:48

you know, it's like, okay, you have you have this third party dependencies, like you need to upgrade

你知道吧，就有点像，好吧，你有一些 third-party dependencies，比如你需要 upgrade 这个 version。我就会说，哦对呀，这完全可以 fully automate，你知道吧，如果你有好的 change log，而且 code 的 documentation 也很好，然后 model 就能自己 reason through 它，就像，你知道吧，我可以直接在你的 code base 里一路冲过去，几个小时就搞定。以前你可能会有点拖着不做，因为这不是最有趣的事。但它其实对你的 business 特别重要。所以，它对你的 project 真的特别重要，你知道吧，尤其是 security vulnerabilities，你想保持最新嘛，对吧？你想把所有这些 patches 都打上。

51:52

the version. I'm just like, oh yeah, you can fully automate this, you know, if you have good

那个版本。我就想，哦对，你完全可以把这 fully automate，你知道，如果你有好的

52:00

change log and you know, and the code is well documented and like, you know, and and the model can

change log，还有，你知道，code 的文档也很完善，而且，像，你知道，而且 model 能

52:04

just like reason through it is like, you know, I can just like blast through your code base, do it

就直接把它 reason 明白，就像，你知道，我可以直接扫一遍你的 code base，搞定它

52:09

in a couple of hours. And you know, previously, you would have like, sort of punted on it because

就几个小时。你知道，以前你可能就会，有点像，不太想碰它，因为

52:13

it's not the most fun thing to do. But it's actually really important for your business. So,

这可不是最好玩的事。但它对你的业务其实真的很重要。所以，

52:17

it's like really important for your project, you know, especially for security vulnerabilities,

它对你的项目真的非常重要，你知道，尤其是 security vulnerabilities，

52:21

you want to stay up to date, right? You want to apply, you know, all these patches.

你得保持最新，对吧？你得把，你知道，所有这些 patches 都打上。

52:26

I think that's just going to be fully automated. So, a large part of like maintenance,

我觉得这以后就会完全自动化。所以，很大一部分 maintenance，

52:30

it just kind of comes for free, right? And then I think it's it's awesome to also think about

它基本上就是白送的，对吧？然后我觉得，去想想这个也很棒

52:36

before like, you know, when you want it to just completely react, you have to do like a new

以前呢，就像，你知道，当你想让它完全做出反应的时候，你就得搞一个全新的

52:40

architecture because you're trying to make space for like a new, you know, different kind of trade-offs

architecture，因为你在试着为一种新的、你知道、不同类型的 trade-offs 腾出空间

52:46

or you have a new understanding of like the workload or you're trying to fit a new feature. And like

或者你对 workload 有了新的理解，或者你想塞进一个新的 feature。然后就像

52:50

suddenly you realize like your current system is just very limiting and you need to completely

突然你意识到，你现在的 system 限制太多了，你需要完全

52:54

architecture it. That was like a really, really costly endeavor, right? So, you know, so sometimes

重新做它的 architecture。那可真是一个非常、非常昂贵的事情，对吧？所以，你知道，有时候

52:58

like multiple years. And I think this should also like super, super accelerate it now. So, you

可能要花好几年。而且我觉得现在这也会超级、超级加速它。所以，你

53:03

like the cost of mistakes, you know, I would say like, you know, is going down. But then at the same

就像犯错的成本，你知道，我会说，就像，你知道，是在下降。但与此同时

53:10

time, the good old rules, I would say of stuff engineer like, you know, having good abstractions

那些老规矩，我会说是 staff engineer 的，就像，你知道，要有好的 abstractions

53:14

like really help, like, you know, is going back to this like having the box with invariance, like,

就真的很有帮助，就，你知道，又回到这个，就像有一个带 invariance 的 box，就，

53:18

you know, if you sort of like draw the right shape, you're going to be able to change things much

你知道，如果你大概画出对的形状，你就能把东西改得

53:22

more quickly within the box and like not affect the rest of the, the services or the rest of your

在 box 里快得多，而且就，不影响其余的那些 services 或者你其余的

53:27

infrastructure. And I think it's important. It's important to design for very quick iteration and

infrastructure。而且我觉得这很重要。为非常快的 iteration 和

53:32

change. I remember when I talked with Peter Tainberger. That was before he joined OpenAI, but about

变更做设计很重要。我记得我和 Peter Tainberger 聊过。那是在他加入 OpenAI 之前，但聊的是

53:38

OpenClaw and how he thinks about it. Like, you know, he told me that he doesn't read the code, but

OpenClaw，以及他是怎么想它的。就，你知道，他告诉我他不读 code，但

53:42

he kept thinking about like, I could see that he's holding the architecture in his head and he was

他一直在想，就像，我能看出来他脑子里装着 architecture，而且他

53:46

telling me how he re-architects a lot and he thinks about how to make it modular, how to allow

跟我说他经常重新 re-architect，他会想怎么让它 modular，怎么让

53:51

100 contributors to each build their thing without stepping on each other's toes. So I'm hearing

100 个 contributors 各自做自己的东西，又不会互相踩到脚。所以我听到的是

53:57

what you're saying that this, this care, this, this planning, this, this structuring has become

你的意思是，这种，这种用心，这种，这种规划，这种，这种结构化，已经变得

54:04

maybe just a lot more important to like which was, which was something back in a day, you know,

也许只是重要得多了，就像，这在当年，当年，你知道，

54:09

it was like the architect or the staff engineer or experienced folks were doing this thing and

就像是 architect 或 staff engineer，或者有经验的人在做这件事，而

54:14

other engineers around the work and building this, you know, smaller parts. But it sounds like

其他工程师在旁边一起做，搭这些东西，你知道，更小的部分。但听起来

54:18

now all engineers need to be aware of when you're building your software right in plan for it.

现在所有工程师在构建自己的 software 时，都得意识到这一点，对吧，并且为它做好规划。

54:23

Yeah. And the GPT models are getting better and better at this as well of like, you know,

是啊。而且 GPT models 在这方面也越来越擅长了，就像，你知道，

54:27

thinking about long-term maintenance and like good architecture and like this is like a natural

考虑长期维护，还有好的 architecture，而且这就像一种自然的

54:31

sort of like next step, right? It's like not just about code quality in the sense of like, oh,

有点像下一步，对吧？它不只是关于 code quality，不是那种，哦，

54:35

is this code clean within this file, but like, you know, it's like, is the architecture actually correct

是这个文件里的 code 干不干净，而是像，你知道，它更像是，architecture 到底对不对

54:40

to reduce maintenance burden over time and like, you know, make space for like the future

来随着时间减少 maintenance burden，而且就像，你知道，给未来的

54:44

product or future extensions or changes and just really this act of like, you know,

product 或未来的 extensions 或变化腾出空间，真的就是这种，就像，你知道，

54:48

engineering over time. That's kind of like something that models are starting to become capable

随着时间进行的 engineering。这有点像是 models 开始变得有能力

54:53

of like thinking about very well. I think it's just kind of fascinating to understand that

去很好地思考的事情。我觉得，能理解这一点就是有点迷人：

54:58

the software that we're building is just going through the life cycle much, much faster,

我们正在构建的 software 就是在经历 life cycle，而且快得多得多，

55:02

right? So, you know, before you had, you know, you were, you were scaling, you were starting it,

对吧？所以，你知道，以前你有的，你知道，你，你在 scaling，你在启动它，

55:06

you know, maybe as like a small team of, you know, yourself, maybe a couple of engineers and then

你知道，可能一开始就像个小团队，你知道，就你自己，可能再加上几个工程师，然后

55:11

you would add engineers like slowly and then, you know, maybe after a year, you know, it's like if

你会慢慢地加工程师，然后，你知道，可能过了一年，你知道，就像如果

55:15

it's very, very successful, you would have 50 engineers on it or like 100 engineers on it,

它非常非常成功，你会有 50 个工程师在这个项目上，或者像 100 个工程师在这个项目上，

55:19

you would have time to see it coming, you would have time to see like, you know, the humans on board

你会有时间预见到它的到来，你会有时间看到，你知道，那些人加入进来

55:24

and you can think about the documentation and all of that stuff, but now it's just sort of like

而且你可以想想 documentation 和所有这些东西，但现在就有点像

55:28

that explosion of like, you know, suddenly you have like a hundred agents contributing to this thing

那种爆发，就像，你知道，突然你有一百个 agents 在给这个东西做贡献

55:33

is like, you know, that can happen like, you know, in a weekend. And so, you know, you're just going

就像，你知道，那可能，你知道，在一个周末就发生了。所以，你知道，你只是

55:37

into it at, you know, major, major speed compared to before. Okay, but how do you and the folks that

以，你知道，跟以前相比非常非常快的速度冲进去。好吧，但你和那些……的人，怎么

55:43

open, hey, I like deal with this. This is not mess with your mind. Like, you know what I mean?

open，嘿，我挺喜欢聊这个的。这不是要搞乱你的脑子。就像，你懂我意思吧？

55:48

In the sense of like, you've been in this business for quite some time now, like, like decades

从某种意义上说，就像，你在这行已经挺久了，怎么说呢，几十年了

55:53

or well over. And there was a pace that we kind of got used to and obviously it's now a lot faster,

或者还不止。以前有一种节奏，我们多少已经习惯了，而现在很明显快多了，

55:59

but how do you get your head around the fact that, hey, it's faster, be the stuff that you've been

但你怎么才能想明白这个事实：嘿，它更快了，但那些你以前一直在

56:05

doing a year ago, right now we're not doing because now the model is good at it. And, you know,

做的东西，一年前还在做，现在不做了，因为现在 model 很擅长它。而且，你知道，

56:10

like, how do you kind of reconcile that? Because I'm sure there's stuff that you've been really good at

就像，你怎么去接受这件事？因为我敢肯定，有些东西你一直特别擅长

56:14

relate to software that now you can hand off to the agent. Do you not get a little bit of sting? You

跟软件相关的，现在你可以交给 agent 去做。你会不会有点扎心？你

56:18

know, we talked about it's stinging for your features to be implemented open source, but it can

知道吧，我们聊过，你的 features 被 open source 实现出来是会有点扎心的，但它也可以

56:23

also sting that I've been really good at like, I don't know, refactoring or, or, or, or right now,

还有一点让我难受的是，我一直很擅长，就，我不知道，refactoring，或者，或者，或者，或者现在，

56:28

it might be architecture, but maybe the model will be really good at that. And now I'm like, oh,

它可能是 architecture，但也许 model 会非常擅长那个。然后我就想，哦，

56:31

okay, Tom, like, I'm glad, but also, like, it would have been nice for me to do that.

好吧，Tom，就，我挺高兴的，但也会，就，要是那个能让我来做就好了。

56:35

Yeah, I think there's like a craft aspect to it, which occasionally I still, you know,

是啊，我觉得这里面有一种手艺活儿的感觉，偶尔我还是会，你知道，

56:42

pull up an editor and like write some code. And it's just like, it feels nice. And it's sort of like

打开个 editor，写点 code。就感觉，挺舒服的。而且有点像

56:48

I have fond memories of like late nights sitting in in them and, you know, just like cranking it out,

我有一些美好的回忆，比如深夜坐在那儿，你知道，就是一直猛敲，

56:56

you know, jicking, uh, zero and, uh, yeah, it's just not having to think about anything else

你知道，jicking，呃，zero，然后，呃，是的，就是不用去想其他任何东西

57:03

other than like the problem in front of me. But really, I think it's, um, it's all about being

除了眼前这个问题之外。但说真的，我觉得，嗯，这全都关乎成为

57:10

in the flow and solving problems. And what I find is like, you know, folks here and also like,

在 flow 里，解决问题。而且我发现，你知道，这里的人，还有，就像，

57:15

everyone I talked to was just like, adapting very quickly. And I think if you, if you have a mindset

我聊过的每个人都是，就像，适应得非常快。而且我觉得，如果你，如果你有一种心态

57:21

where it's all about code is a tool to solve problems. And you can solve so many more problems.

就是，code 只是解决问题的工具。那你就能解决多得多的问题。

57:27

It's like, before you wanted to benchmark something and you weren't quite sure where you were

就像，以前你想 benchmark 某个东西，你不太确定你会

57:31

going to net at. It's like, you can just do it. It's, it's going to take you like no more than 30

最终得到什么结果。就像，你直接做就行了。它，它会花你大概不超过 30

57:36

seconds, you know, to launch something in the background and you know, get proper numbers and be

秒，你知道，在 background 里启动点什么，然后，你知道，拿到准确的数据，并且

57:40

able to do like a better trade off. It makes it, it should make you a better engineer. If you're

能够做出更好的 trade off。这会让你，应该会让你成为更好的工程师。如果你

57:45

just really care about, you know, the outcome and the system working well. And so what it allows

真的在乎，你知道，结果和系统运行得好。所以，它让你能够

57:51

us to do at opening, it allows us to run, you know, our inference much more efficiently. It allows us

我们一开始要做的，它让我们能，你知道，更高效地运行我们的 inference。它让我们

57:57

to, you know, get like much more like effective compute and you know, deploy that to the world. And

能，你知道，拿到多得多，像是 effective compute，然后，你知道，把它部署到全世界。而且

58:01

so like, everyone's just like very focused on that, involving important problems at the speed

所以，就，大家都非常专注在这上面，以那种速度去参与重要问题

58:06

that was not possible before. And like, I haven't yet, you know, encounter someone who's like,

那是以前不可能做到的。而且，就，我还没有，你知道，遇到某个人，他会说，

58:11

oh, that's not, that's not good. That's not fun. Do I understand correctly that it sounds like

哦，那不好，那不好。那不好玩。我理解得对吗，听起来像是

58:16

if you have ambitious problems, if you have way more problems than what you can solve today or

如果你有雄心勃勃的问题，如果你的问题比你今天能解决的多得多，或者

58:20

tomorrow or the next week, sounds like this is not really a problem because when, you know, you get

明天或下周，听起来这其实不是问题，因为当你，你知道，你变得

58:25

more efficient somewhere, you keep going. Which is a lot of startups, right? Like startups always

在某个地方更有效率时，你就继续往前走。很多 startups 就是这样，对吧？就像 startups 总是

58:30

weigh more ambitious than what they're able to do. I don't, we're not, we're not out of problems

他们远比他们能做到的更有野心。我不，我们不是，我们还没摆脱问题

58:35

for sure, right? So and I don't think we'll be for a while. We have a long, long road ahead of us

肯定的，对吧？所以，我也不觉得我们短期内会摆脱。我们前面还有很长、很长的路要走

58:41

in terms of like mathematical breakthroughs, scientific breakthroughs, you know, making the world

就比如数学突破、科学突破，你知道，让这个世界

58:46

a better place, like just really building for humans and solving the most important problems that

变得更好，就真的是为人类而建设，解决那些最重要的难题，

58:52

everyone is facing and just doing it in a deeply human way. That's, that's what we're here for.

每个人都在面对的问题，并且以一种非常人性化的方式去做。这，这就是我们在这里的意义。

58:59

Also just going back to coding and like, you know, these late nights is like, I think there's like,

还有，回到 coding，就比如说，你知道，这些熬夜，我觉得，就有点像，

59:03

it's also like maybe like a glamorous version of it. Just like I also had very a lot of late nights

它也可能像是一个光鲜的版本。就像我也熬过非常多的夜

59:08

where I was trying to refactor something. And you know, it's just like, it would be like three

我在试着 refactor 某个东西。而且你知道，就有点像，可能就三点了

59:13

hours deep into the refactor and then realize like, actually, this is a dead end. And I must restart

refactor 做了好几个小时，然后才意识到，其实，这就是个死胡同。而且我必须从头开始。

59:19

from scratch. And it was like very frustrating. And so it's like, there was like, there are like

而且那真的特别让人沮丧。然后就好像，确实有，就是有

59:24

these very, very fun times that there's also the time where it's like, it doesn't compile in your

那些非常非常开心的时刻，但也有那种时候，就是，它不 compile，按你的

59:28

sense. Like, why is it not compiling yet? Like, I'm sure you had the time where you go, you will have

理解来说。就像，为什么它还没 compile 好？我肯定你也有过那种时候，你会说，之后

59:35

later, it's now super late. You need to go to bed because you need to, you need to get some

再弄吧，现在已经超级晚了。你得去睡觉了，因为你得，你得睡会儿觉。

59:38

sleep. And then you can't really sleep. And you have this thing where like, you have some,

然后你又真的睡不着。你会有这种状况，就是，你有些，

59:43

some tasks that is halfway. And it upsets you sometimes. I remember dreaming about the code as well.

有些任务只做了一半。有时候这会让你很烦躁。我记得我也梦到过 code。

59:50

And I guess one thing I don't really have these days when I'm working on my,

然后我猜，有一件事，这些天我在做我的……的时候，真的不太有了，

59:55

software for my business is I don't really have something that is halfway because I can just tell

我业务里用的 software，我其实没有那种做到一半的东西，因为我可以直接告诉

60:00

it, do this. And then I can leave it as a state where it's kind of like, you know, done either finish

它，做这个。然后我可以把它留在一个状态，就有点像，你知道，已经做完了，要么就完成

60:04

it. It's either working or it's, I have proof that it's failed. But it's interesting because

它。要么它能跑通，要么就是——我有证据证明它失败了。但这很有意思，因为

60:08

you know, everything's sped up, right? Yeah. Maybe. Like, I do have, like what, what a lot of people

你知道，一切都加速了，对吧？是啊。也许吧。就像，我确实有，就像什么，很多人

60:14

do and I do myself is like, you know, I have like sometimes like bigger questions that I'm asking

会做、我自己也会做的是，就像，你知道，我有时候会有一些更大的问题在问

60:18

myself, like, and I, you know, from conversations I've had during the day or like, I haven't yet,

自己，就像，而且我，你知道，从白天聊过的一些对话里，或者就像，我还没，

60:22

you know, just like had the time to just look into it. And so I, you know, I will send off code X to

你知道，就是还没时间真正去研究它。所以我会，你知道，我会把 code X 发出去，让它

60:28

just like look at it overnight. And then I'm very excited to then wake up and look at the results.

就像整晚去看看。然后我就特别兴奋，等醒来后去看结果。

60:33

And so, you know, it's always like an exciting morning. Well, I feel there's an arch to

所以，你知道，这总是像是一个让人兴奋的早晨。嗯，我觉得这里有个 arc，

60:37

doing long running tasks. And of course, you can use the slash goal, which will go and run,

做 long running tasks 的时候。当然，你可以用 slash goal，它会去跑，

60:42

you know, that's also something that was recently added like a few months ago, right? The

你知道，这也是几个月前才刚加进来的东西，对吧？那个

60:46

slash goal command to code X. Yeah. And back to, you know, maybe like the harness is a crutch, right? Is

给 code X 的 slash goal command。对。然后回到，你知道，也许就像 harness 是个拐杖，对吧？是不是

60:53

slash goal was like necessary to allow like, you know, to keep the model like on track on like a

slash goal 当时就像是必要的，用来允许，你知道，让 model 保持在正轨上，在

61:00

singular goal for a very long period of time. And it's like, it allows the model to literally run

单一目标上，持续很长一段时间。而且它就像，让 model 真的能跑

61:05

for days or weeks if it's like a really, really hard problem. But with the new generation of models,

好几天甚至好几周，如果这是个非常非常难的问题的话。但有了新一代的 model，

61:11

like what we're seeing is like, you know, you don't need slash goal anymore. You don't need a harness

就像我们现在看到的，你知道，你不再需要 slash goal 了。你不再需要 harness 了

61:15

around it. You can just tell the model like, you know, hey, go and work for a week. And you know,

围绕它。你可以直接跟 model 说，你知道，嘿，去工作一周吧。而且你知道，

61:19

I will actually do it. Speaking of hard problems and the fact that you're not out of them, one of

我真的会去做。说到难题，以及你还没摆脱它们这件事，其中一个

61:24

the interesting things that you've shipped from the outside it, I would say it was, you know,

你从外部发布出来的有意思的东西，我会说，它算是，你知道，

61:29

as an engineer was moderately interesting is the, what you call the merge, which is code X appeared

作为一个工程师来说还算有点意思的是，那个你称之为 merge 的东西，也就是 code X 出现在了

61:36

inside of chat GPC. And the reason I say that's as engineers, it kind of moderately interesting

chat GPC 里面。而我说，作为工程师，它还算有点意思的原因是

61:40

because we've been using code X like, yeah, it's there. You can now open it in the chat GPC app. Great.

因为我们一直在用 code X，就像，对，它就在那儿。你现在可以在 chat GPC app 里打开它了。挺好。

61:45

Like, I just went there and I just immediately went to code X because I don't, I don't really use

就像，我刚刚进去，然后马上就去了 code X，因为我并不，我并不真的用

61:49

chat GPC in the app per se. But I talked with folks at OpenAI and people in your team. And you know,

app 里的 chat GPC 本身。但我跟 OpenAI 的人聊过，也跟你们团队里的人聊过。而且你知道，

61:55

they were telling me like, there was a lot of preparation going on, a lot of engineering challenges.

他们跟我说，就像，当时有很多准备工作在进行，有很多工程上的挑战。

62:00

Can you give a sense of how big this project was, what you needed to do? And why was it difficult

你能大概讲讲这个项目有多大、你们需要做什么吗？

62:06

to pull off and how, you know, how did code X and other tools help you get it done? In ways that

还有，为什么它那么难做成，以及，你知道，code X 和其他工具是怎么帮你们搞定的？

62:11

would have been hard before. Because since you've launched the merge, the numbers that you keep

用的是以前会很难做到的方式。

62:17

sharing of how many people use code X is like, it's going up way faster than before. So I assume there's

因为自从你们推出 merge 之后，你一直分享的那些使用 code X 的人数，就像，涨得比以前快多了。

62:22

a big scale problem you've solved here. A lot of things were challenging with the merge is,

所以我猜你们在这里解决了一个很大的 scale 问题。

62:30

first of all, completely different stacks. Chat GPC is like fully managed cloud-based. Like,

merge 有很多事情都很有挑战，首先就是完全不同的 stacks。

62:39

you know, you run everything on our systems. We store things like traditional,

Chat GPC 就像 fully managed cloud-based 的那种。

62:46

traditional way of like building things, built for scale, built for efficiency.

传统那种做东西的方式，是为了 scale、为了效率而构建的。

62:52

Code X fully local. And so the merge is just really like how do you get the same

Code X 完全 local。所以这个 merge 其实就真的是，你怎么才能拿到同样的

62:58

the same benefits and the same capabilities from this local coding agent and then build a product

同样的好处、同样的能力，从这个 local coding agent 上，然后再做一个产品，

63:04

around it and build it in a way where it can benefit like a much, much broader pool of people,

围绕它，并且以一种能惠及广得多、广得多的人群的方式来构建它，

63:09

which is just also why all of us joined OpenAI is like to benefit like this very, very broad

这也正是我们所有人都加入 OpenAI 的原因，就是想去惠及这样非常、非常广的

63:17

population across the world. And so it was a very exciting journey of like figuring out like,

全球人群。所以这是一段非常激动人心的旅程，就像是在搞清楚，

63:22

how do we build a cloud version of this that in essence is capable of like very, very much

我们怎么给这个东西做一个 cloud 版本，本质上能具备非常、非常多的

63:29

the same things. But it's also built in a way where, you know, we can serve it through like

同样的东西。但它也是以这样一种方式构建的，你知道，我们可以通过类似

63:34

tens and hundreds of millions of millions of users. And a way that is still like, you know,

数千万、上亿的用户。而且还要有一种方式，仍然就像，你知道，

63:38

efficient so that we can include it all the way into the plus plan. Chat GPC work is essentially like

足够高效，这样我们才能把它一路放进 plus plan 里。Chat GPC work 本质上就像

63:44

running the full codex harness in a cloud like together with like a cloud computer.

在 cloud 里运行完整的 codex harness，就像跟一台 cloud computer 一起。

63:52

It's a very powerful machine actually like people have sort of picked up on it and showed that,

它其实是一台非常强大的机器，就像，人们已经多少注意到它了，也展示出了，

63:57

you know, what you can do. Like, you know, if you are creative with the prompt is that, you know,

你知道，你能做什么。就像，你知道，如果你在 prompt 上很有创意，就是，你知道，

64:00

you can get, you can get Chat GPC to like, you know, train another model in there, you know, you say,

你可以让，你可以让 Chat GPC 在里面，就像，你知道，训练另一个 model，你知道，你说，

64:07

wow, there's like pretty wild things, you know, you can get it to install Blender and, you know,

哇，有些东西真的，就像，挺狂野的，你知道，你能让它安装 Blender，而且，你知道，

64:13

do like 3D modeling. It's like very permissive. It has like internet access. It's like a powerful

做，就像，3D modeling。它就像非常放得开。它就像有 internet access。它就像一台强大的

64:19

machine. And then codex just works on it. And this is like what we shipped through gradually work.

机器。然后 codex 就直接在它上面干活。这大概就是我们一步步推进、慢慢 ship 出来的东西。

64:24

So a lot of system challenges. The team did it very quickly. Obviously, like, codex helped,

所以有很多系统层面的挑战。团队做得非常快。显然，codex 帮了不少忙，

64:30

you know, to make it more efficient, look at and build a lot of the infrastructure.

你知道，就是让它更高效，帮忙审视、搭建了很多 infrastructure。

64:36

And then, you know, help resolve a lot of the little differences as well that, you know,

然后，你知道，还帮忙解决了很多小差异，你知道，

64:40

had been occurring between between codex and Chat GPC, like, merging plugins, architecture,

这些差异一直出现在 codex 和 Chat GPC 之间，比如合并 plugins、architecture，

64:46

you know, merging library. And like, sort of like really, really working towards like,

你知道，合并 library。然后就是，有点像是真的、真的在朝着，

64:50

unified system, which is really the goal is like, you shouldn't feel like, you know, you can do

unified system 努力，而这其实才是目标：你不该觉得，你知道，你能在

64:53

something in codex that you can't do in Chat GPC or vice versa. Like, what we're trying to build

codex 里做某件事，却在 Chat GPC 里做不了，或者反过来。就像，我们想打造的东西

64:58

is like one unified product that gives you access to the same intelligence, but in the way that you

就像是一个统一的产品，让你能用到同样的智能，但以你

65:03

want to use it. And so it was very fun as well, because codex throughout the whole journey also

想要使用它的方式。而且整个过程也特别好玩，因为 codex 在整个历程里也

65:09

acted as a journalist to sort of like document all the steps and the debates and the discussions

扮演了记者的角色，差不多就是把所有步骤、争论和讨论都记录下来

65:14

at the teams we're having. And it was very animated debate, you know, of how we should do it and

在我们团队里发生的。而且那是非常热烈的争论，你知道的，关于我们该怎么做，以及

65:18

how we should name the thing. And you know, when to introduce it in one way, like, what to merge into

该怎么给这个东西命名。还有你知道，什么时候以某种方式推出它，比如什么要合并到

65:23

what? There were like many different permutations considered. And so there's like a very fun

什么里面？当时考虑了很多不同的排列组合。所以这有种很好玩的

65:28

journalistic element to it where we have a full recounting that codex did over time.

新闻式元素，我们有一份 codex 随着时间做的完整回顾。

65:34

And yeah, it's just it's kind of become known as well as like the toggle arc of OpenAI where,

而且，是的，它也就慢慢变得众所周知，像是 OpenAI 的 toggle arc，

65:42

you know, we introduced like the work toggle, which there was also a lot of debate around of like,

你知道，我们引入了像 work toggle 这样的东西，当时关于这个也有很多争论，就是，

65:47

you know, whether this was like the right thing. And then, you know, it's just like, we kind of

你知道，这到底是不是对的做法。然后，你知道，就感觉，我们有点

65:50

grew to just really like it. But over time, we're going to merge things further. So it's like,

慢慢就真的喜欢上它了。但时间一长，我们会把东西进一步合并。所以就像，

65:56

we're really headed into the direction of like full unification. And you know, we kind of view this as

我们真的在朝 full unification 这个方向走。而且你知道，我们有点把这看成

66:02

a temporary state where, you know, you have like, you have better, stronger capabilities when you're

一个暂时状态，就是，你知道，你会有，就像，你会有更好、更强的能力，当你

66:08

in work mode. But over time, we're bringing this, you know, all the way through like, you know,

在 work mode 下。但时间一长，我们会把这个，你知道，一路带过去，就像，你知道，

66:13

everyone that uses chats with you. And how do you personally use codex? Like what's your,

所有跟你聊天的人那里。还有，你个人是怎么用 codex 的？比如你的，

66:19

what's your working setup in terms of agents, intramarist tasks in terms of what you,

你的工作设置是什么样的，在 agents 方面、intramarist tasks 方面，就你，

66:23

what you manage with it? And related to this, I asked Peter Stainberger what I should ask about

你用它管理什么？还有，与此相关，我问了 Peter Stainberger，我该问你什么

66:29

you. And he said like, you need to ask him, how do you deal with the fact that you involved with

他说，你得问他，你怎么处理这个事实：你参与了

66:35

all these projects, your calendar is like Tetris, but usually you show up pretty cheerful.

所有这些项目，你的日历就像 Tetris，但你通常都显得挺开心。

66:42

My calendar is fine. And it's just, I am capable of doing so many more things nowadays because

我的日历没问题。只是，我现在能做多得多的事情，因为

66:53

I have the technology like codex. And I actually shifted a lot of like my, my work on,

我有像 codex 这样的技术。而且我其实把我很多，很多工作转到了

67:00

on mobile, using chats if you work, where whenever I have something that I want to take note of,

手机上，用聊天，如果你在工作的话，就是每当我有想记下来的东西，

67:07

I just like fired it off, I use dictation a lot. Whenever I have a question, instead of like

我就直接发出去，我经常用语音输入。每当我有一个问题，而不是

67:13

writing it down to look into later or delegating to someone, I just like, fire it off in

把它写下来以后再看，或者委派给别人，我就直接把它发到

67:18

a chat if you're working, I get like a report. It has like a whole bunch of like custom skills

在一个 chat 里，如果你在工作，我就会拿到一份报告。它有一大堆 custom skills 和 custom instructions，现在已经非常非常定制化了，能生成那种，你知道，报告、slide decks 和 code explorations，风格是我能有效消化的。所以每次我在会议间隙，或者，你知道，你会看到我，就像，你知道，我刚才就是在对着手机口述。就像我之前说的，我们很多工作都在 public channels 里做，我们在 Slack 里有很多，在 notion 和 Google docs 里也有很多。所以基本上，真的没有什么问题是我觉得不能问的，你知道，codex 至少能先做一版 first pass。

67:23

and custom instructions where it's now like very, very tailored to like, you know, produce the

还有 custom instructions，现在就是非常非常量身定制的，就，你知道，来生成

67:28

kinds of reports and slide decks and code explorations, you know, in the style that I can consume

各种报告、slide decks 和代码探索，你知道，以我能有效消化的

67:34

effectively. And so every time I'm like, between meetings or like, you know, you'll kind of like see

风格。所以每次我，就像，在会议之间，或者，你知道，你会有点像看到

67:38

me like, you know, I was just like dictating to my phone. As I said before, it's just like

我，就像，你知道，我就是在对着手机口述。就像我之前说的，就有点像

67:42

we do a lot of work in public channels, we have like a lot in Slack, we have a lot in, in

我们在公共频道里做很多工作，我们在 Slack 里有很多，我们在，在

67:48

notion and Google docs as well. And so there's pretty much like, there's no question really that I

notion 和 Google docs 里也有很多。所以基本上就是，真的没有什么问题是我

67:53

feel like I cannot ask that, you know, codex will be able to sort of like do at least a first pass

觉得不能问的，你知道，codex 都能，有点像，至少先做一轮

67:57

of thinking through whether it is like public sentiment on a feature, looking at production logs for,

比如去琢磨某个 feature 的公众情绪，去看 production logs，

68:05

you know, how much users we have on a certain thing, making a list of things that we should deprecate

你知道，看某个东西上有多少用户，列一份我们该 deprecate 的东西清单，

68:09

because they're not getting traction, understanding what a certain team is up to. It's like any question

因为它们没什么 traction，搞清楚某个团队在做什么。就像我有的任何问题，

68:13

I have, I can get an answer to like, you know, within 30 minutes. And so that's how I use it. I use it

我都能，你知道，在 30 分钟内得到答案。所以这就是我用它的方式。我用它

68:18

for everything. It's like my personal agent in like all the ways. And then oftentimes on weekend

做所有事。它就像我的 personal agent，方方面面都是。然后经常在周末

68:26

says, well, I do some like code explorations or like I built some prototypes and I have fun,

会说，嗯，我会做一些 code explorations，或者搭一些 prototypes，玩得很开心，

68:31

like sort of like imagining the future of the product in some ways. And I do that with others

有点像在某些方面想象产品的未来。我会和团队里的其他人一起做这些，

68:36

on the teams. It's not always the same team. And it's just like in one day, I can build things that

不总是同一个团队。而且就像一天之内，我就能做出一些东西，

68:43

I sort of like, I had it in my system, right? It's like, it's like, I woke up one day. I'm just like,

我有点像，我脑子里一直有这个念头，对吧？就有点像，有点像，我有一天醒过来，我就想，

68:47

we should explore what it means to build this. And then I can just sort of express all of that and

我们应该去探索一下，构建这个东西到底意味着什么。然后我就能把所有这些都表达出来，并且

68:52

get like something in front of people in a day so that they can think through it and criticize it

在一天之内把某个东西放到大家面前，让他们能去琢磨它、批评它，

68:57

and hopefully get inspired by it. It's like, by no means, you know, we need to ship it, but it's

并且希望能从中获得启发。这绝不是说，你知道，我们就得把它 ship it，而是说，

69:02

more like, okay, I flush it out of my system. And then, you know, I go on and like, you know,

更像是，好吧，我把它从我的 system 里清出去。然后，你知道，我继续，你知道，

69:06

do other things. So it's just like so, I know it's such a magical time and it's like so empowering.

做别的事情。所以就是，我知道这是一个特别神奇的时刻，而且感觉特别有力量。

69:12

And as closing, what would your advice be for a software engineer, such AI engineer, someone who

最后收个尾，你会给一个 software engineer，比如这样的 AI engineer，一个

69:16

built software who would want to get the skill set and the experience to have the opportunity to

做过软件、想获得那套技能组合和经验，从而有机会去……的人什么建议？

69:24

work at a place like the codex team, like OpenAI or like an AI startup. So like, you know, just

在像 codex team 这样的地方工作，比如 OpenAI，或者像一家 AI startup。所以就像，你知道，就

69:28

become this really great builder with these tools. Because the question that comes up often,

成为借助这些工具非常出色的构建者。因为经常会出现的问题是，

69:34

like, hmm, should I start with the theory, how important are the basics? Should I just get really

就像，嗯，我应该从理论开始吗，基础有多重要？我是不是只要真的

69:39

good at using the tools? Yeah. I think there are two things that are important is deep curiosity

擅长使用这些工具就行？对。我觉得有两件重要的事，一个是对

69:45

for how things work and an ability to like, you know, train yourself to understand things very quickly.

事物如何运作的深深好奇心，以及一种，就像，你知道，训练自己很快理解东西的能力。

69:51

And so it's, it's, it is the case that things will continue to change, but people that do

所以，就是，确实是这样，事情会不断变化，但那些

69:56

extraordinarily well at OpenAI are like, you know, people that just sort of like are able to like

在 OpenAI 做得特别出色的人，就像，你知道，就是那种能够

70:01

grok a system quickly and like, you know, also dive into like a new code base and sort of like,

很快 grok 一个系统，而且就像，你知道，也能一头扎进一个新的 code base，然后有点像，

70:07

you know, make sense of it. But obviously like all of that is help with agents nowadays, right? So

你知道，就是把它弄明白。但很明显，现在所有这些都有助于 agents，对吧？所以

70:13

there's like, there's so much information that you need to absorb and like, you know,

就，有太多信息需要你去吸收，而且，你知道，

70:16

being able to understand and reason through it. And a lot of that is asking good questions

能够理解并把它推理清楚。而其中很大一部分就是问出好问题

70:20

really about, you know, how do you things work? And just like going into like the five wise, which I

其实就是，你知道，事情是怎么运作的？然后就像一路进入 five wise，我

70:24

think, you know, you can just kind of keep digging and digging and digging. And you know,

觉得，你知道，你就可以一直深挖、深挖、再深挖。而且你知道，

70:28

you're learning very, very fast through that. The other thing is being in tune with the community

通过这个，你会学得非常非常快。另一件事是要跟社区保持同频

70:33

or, you know, the people that you're trying to solve a problem for is like, not everything is like

或者说，你知道，那些你想帮他们解决问题的人，就，不是所有事情都像

70:37

solving a direct problem. Sometimes you're solving a problem that will be useful, you know,

是在解决一个直接的问题。有时候你是在解决一个以后会有用的问题，你知道，

70:41

to like another group of people in the pursuit of like solving a problem for humans. But just being

去喜欢另一群人，在追求为人类解决某个问题的过程中。但只要能

70:46

crisp about the taste or the needs or the requirements and being able to think clearly and like,

对品味、需求或要求保持清晰，并且能够清楚地思考，还有，

70:52

you know, exercising through this clarity of thought feels really important to me. Like if you can't

你知道，通过这种清晰的思考来锻炼，对我来说真的非常重要。就像如果你不能

70:56

explain what you're trying to achieve, if you can't explain your intent, if you don't have a tie

解释你想实现什么，如果你不能解释你的意图，如果你没有一种联系

71:00

to a community, if you don't have the taste, it's, it's, it's, it's going to be much harder to do

与一个社区的联系，如果你没有这种品味，那，那，那，那就很难去做

71:05

great work. Awesome. Tiibo. Well, thanks a bunch for this conversation. This was awesome. Thanks for

出色的工作。太棒了。Tiibo。嗯，非常感谢这次对话。这太棒了。谢谢你

71:10

having me. I've always wanted to get together with Tiibo. And I'm glad that we finally made it happen.

邀请我。我一直想和 Tiibo 聚一聚。我很高兴我们终于做到了。

71:15

I appreciate it how Tiibo talked about not just the upsides of open source, but also the downsides.

我很欣赏 Tiibo 不仅谈了 open source 的好处，也谈了它的缺点。

71:20

Most notably how competitors can copy features you are just working on in the open right now

最明显的是，竞争对手可以把你现在还在公开开发的功能直接抄过去

71:25

and then ship it right before release and just how much this things. Plus, you get a lot of low quality

然后在发布前就抢先发出来，以及这种事到底有多……另外，你会收到很多低质量

71:30

contributions that you still need to somehow deal with. Another interesting one was Tiibo saying

的贡献，你还是得想办法处理。另一个有意思的点是 Tiibo 说的

71:34

how the harness is always a step ahead of the model. From the inside, the codecs teams see their

harness 总是比 model 领先一步。从内部来看，codecs 团队把自己的

71:39

job as building clutches for the model with the harness, the tools and the setup instruction.

工作看成是用 harness、工具和 setup instruction 给 model 做 clutches。

71:45

And then the next version of the model will be trained to need fewer of these clutches.

然后下一个版本的 model 会被训练成需要更少这些 clutches。

71:49

I'll be honest, as a dev, this sounds a little demotivating that the stuff I build in the next

说实话，作为一个 dev，这听起来有点让人泄气：我在下一个

71:54

version of the model, it'll just know and we can get rid of it. Plus, I do suspect that it's not

版本的 model 里做的东西，它直接就会了，然后我们就能把它扔掉。另外，我确实怀疑这并不

72:00

just about building these clutches, but also building tools that models will use. And it's not

仅仅是关于构建这些 clutches，还有构建 models 会用的工具。而且这不是

72:05

like the next version of the model will reinvent an MCV protocol or scales or plugins, at least I

就像下一个版本的模型会重新发明一个 MCV protocol，或者 scales，或者 plugins，至少我

72:10

hope not. I also enjoyed hearing what the merge merging chat GPC and codecs look like from the inside.

希望不会。我也很喜欢听这次把 chat GPC 和 codecs 合并的 merge，从内部看是什么样。

72:17

It was merging a previously fully local coding agent codecs into a managed cloud-based stack

那是把一个之前完全本地的 coding agent codecs 合并进一个托管的 cloud-based stack，

72:22

and doing it efficient enough so that it can be included in OpenAX $20 per month's plan when

并且做得足够高效，好让它能被放进 OpenAX 每月 $20 的套餐里，而

72:27

$20 is not all that much in terms of compute purchase. It was pretty amusing to hear how

$20 在购买 compute 这方面并不算多。挺有意思的是听到

72:33

codec is self-acted as a journalist of the whole project as it was present in all the slack

codec 如何把自己当成整个项目的记者，因为它出现在所有 Slack

72:37

conversations and all the documents and so it could capture all the important debates and decisions.

对话和所有文档里，所以它能捕捉到所有重要的讨论和决策。

72:42

I'm not gonna lie, this part felt a little bit of a big brother feel to it,

说实话，这一段让我有点 Big Brother 的感觉，

72:46

where the AI is always watching, but it could well become the new normal startups in the future.

就是 AI 一直在看着你，但未来这很可能成为 startups 的新常态。

72:51

I've not yet decided how I feel about this. And finally, I appreciate a table's advice for

我还没决定自己对此是什么感觉。

72:56

engineers to succeed, be curious, understand symptoms quickly, and be in tune with the group you

最后，我很欣赏 table 给工程师的建议：要想成功，就要保持好奇、快速理解症状，还要和你正在为之打造产品的那群人同频。

73:02

are building for. It's reassuring to hear from the table as well how much the fundamentals

从 table 那里也听到基本功仍然这么重要，这让人很安心。

73:06

still matter. Do check out the show notes below for deep dives on how codecs,

一定要看看下面的 show notes，里面有关于 codecs、cloud code encursure 是怎么构建的，以及其他相关话题的深入探讨。

73:09

cloud code encursure were built, and other related topics. If you like what you heard, please hit

如果你喜欢听到的内容，请在你用的 podcast player 上打个评分，这对我、对节目都很重要。

73:14

a rating on a podcast player that you're using, it means a lot to me and to the show. Thanks,

谢谢，

73:19

and I'll see you in the next one.

我们下期见。

Building Codex with Tibo Sottiaux

Play Queue