TWIML AI
Canva Is Tackling AI Design’s Most Frustrating Problem with Stefano Corazza- #784
00:003450
00:00
As good as modern image generation models have become,
现代 image generation models 已经变得这么强了，
00:03
designing with them can still feel like playing games.
但用它们做设计，还是感觉像在玩游戏。
00:06
Put the right magic words in the prompt and you might win at the slot machine.
在 prompt 里放对“咒语”，你可能就能在老虎机前中奖。
00:10
Try to iterate on the result and you're playing whack-a-mole.
想对结果反复迭代，又变成在玩打地鼠。
00:13
That's fine for memes, but when design is your job, the game gets old.
拿来玩梗图没问题，可如果设计是你的工作，这套游戏很快就玩腻了。
00:17
Canva's AI research team wants to make that process more predictable.
Canva 的 AI 研究团队想让这个过程更可预测。
00:21
With greater control over layouts, easier revisions,
对 layouts 有更强的控制、修改更容易，
00:24
and models fast enough to keep up with the creative process.
还有快到能跟得上创作过程的 models。
00:28
To get there, they've trained their own family of models using Canva's design data
为了达到这个目标，他们用 Canva 的设计数据训练了自己的一整套 models
00:32
and integrated those models right into their tools.
并把这些 models 直接整合进了他们的工具里。
00:35
Here's Stefano Coraza, head of AI Research at Canva
下面是 Canva 的 AI Research 负责人 Stefano Coraza
00:38
on what his team is trying to change.
聊聊他的团队正试图改变什么。
00:40
You have to build around iteration AI will get 70%, 80% there.
你必须围绕 iteration 来构建，AI 会做到 70%、80% 的程度。
00:45
I will never get to 100%.
我永远不会达到 100%。
00:47
And so you have to factor in that for the last mile,
所以你必须把这一点考虑进去：在 last mile 上，
00:50
you need a human in the loop and be able to iterate in a way that is fast and also fun.
你需要 human in the loop，并且能够以既快又有趣的方式 iterate。
00:55
I'm Sam Charrington and this is the Twilmal AI podcast.
我是 Sam Charrington，这里是 Twilmal AI podcast。
00:59
For over a decade, I've been exploring the ideas and innovations
十多年来，我一直在探索各种想法和创新，
01:02
shaping the future of AI through conversations like this one
它们通过像这样的对话塑造着 AI 的未来，
01:05
that help you understand what's real, what's next, and what matters.
帮你理解什么是真实的、什么是接下来的、什么才是重要的。
01:09
Let's jump in.
我们开始吧。
01:19
This has been a pretty busy week for you and your team.
这周对你和你的团队来说挺忙的吧。
01:23
Well, first of all, the team was announced.
嗯，首先，团队正式公布了。
01:26
Was this like the debut of AI Research at Canva?
这算是 AI Research at Canva 的首次亮相吗？
01:31
Yeah, that was the big debut last week.
对，那就是上周的重磅亮相。
01:34
I think 12 hours before we actually announced Canva Craft,
我记得在我们正式发布 Canva Craft 之前 12 个小时，
01:38
the model, and we also announced Canva Research.
也就是这个 model，同时我们还发布了 Canva Research。
01:41
We had blog posts, the website, with a lot of information about who we are
我们准备了博客文章和网站，里面有很多关于我们是谁
01:46
and what we do.
以及我们做什么的信息。
01:47
So it was really fun.
所以真的很有意思。
01:48
And then this month has been the busiest month probably of my life.
然后这个月大概是我这辈子最忙的一个月。
01:51
We're like every week we are announcing a new model.
我们基本上每周都在发布一个新的 model。
01:54
And so exactly 40 minutes ago, we had this announcement about Canva Brush.
所以就在整整40分钟前，我们发布了关于 Canva Brush 的公告。
02:01
So very exciting.
非常令人兴奋。
02:03
This visual language model for design, creation, and anything.
这个 visual language model 用于设计、创作以及任何场景。
02:08
Excellent.
太棒了。
02:08
And so have there been more models than just those two
那么这个月里除了那两个之外，还有更多 models 吗？
02:13
over the course of the month?
是的，目前我们大概总共有 100 个 AI models in production。
02:14
Yeah, we have a total of a hundred AI models in production,
02:18
roughly right now.
02:19
And some are smaller and may not be a huge, large language model,
而且有些规模更小，可能并不是那种超大的 large language model，或者类似的东西。
02:23
or anything like that.
但我们开始把 AI 注入并融合到产品的各个方面，
02:24
But we started injecting and fusing AI in all arts of the product
真正以各种可能的方式帮助这个 design workflow。
02:30
and to really help this design workflow in every possible way.
但现在我们正通过这项工艺，也就是用于 image generation 的 financial model，迈出更大的步伐。
02:35
But now we are taking bigger steps with the craft, the financial model
然后借助 Brush，我们正在彻底革新人们创作内容的方式。
02:41
for image generation.
02:42
And then with Brush, we are completely revolutionizing the way
02:46
people can create content.
02:48
And also we are revolutionizing how fast and affordable that is.
而且我们正在彻底改变这件事的速度和成本可负担性。
02:53
Because those two parameters,
因为这两个参数，
02:56
pioneers using AI are okay, waiting a minute for the magic moment
用 AI 的先行者可以接受等一分钟，等那个魔法时刻
03:01
where something magical gets generated.
等某个神奇的东西生成出来。
03:03
But like, if you want to really be using the day to day,
但如果你想真的把它用到日常里，
03:07
like that's not an option, you know, in game development
那根本不是一个选项，你知道，在游戏开发里
03:10
there's a rule, anything that takes more than 10 seconds is
有个规则，任何超过 10 秒的东西
03:13
completely out of the question.
都完全不可接受。
03:15
And we want it to get to like instantaneous AI.
而且我们希望它能达到那种即时 AI 的程度。
03:18
So yeah, I mean, particularly with the way that you need to
所以是啊，我的意思是，尤其是考虑到你在创作时需要
03:24
iterate when you're creating, like those delays really stack up.
不断迭代，那些延迟真的会越积越多。
03:30
Yeah, absolutely.
是啊，完全同意。
03:31
And people, I think there's a little bit of Silicon Valley
而且我觉得，人们身上有一点 Silicon Valley 的
03:34
bios that I've seen myself being far to that, but you think
偏见，我自己也见过自己挺受这个影响的，但你会觉得
03:38
like the new tech comes and it's going to be single shot, perfect.
就像新技术一出来，就会一次搞定、完美无缺。
03:42
And then people would just adopt it.
然后人们就会直接把它用起来。
03:43
But the reality is never the case, especially if the task
但现实从来不是这样，尤其是当任务
03:48
has some creativity in it.
带一点创造性的时候。
03:50
You know, it's not just a purely technical task.
你知道，它不只是一个纯技术任务。
03:53
And so you have to build around iteration.
所以你必须围绕 iteration 来构建。
03:56
AI will get 70%, 80% there.
AI 能做到 70%、80% 的程度。
03:59
I will never get to 100%.
我永远到不了 100%。
04:01
And so you have to factor in the, for the last mile,
所以你得把那个 last mile 考虑进去，
04:04
you need a human in the loop and be able to iterate in a way
你需要 human in the loop，并且能够以某种方式 iterate
04:07
that is fast and also fun.
那很快，也很有趣。
04:09
So that's a, that's a key part to think of what we do.
所以这是——这是理解我们所做的事情的一个关键点。
04:12
Until, you know, the moment when we will have telepathic,
直到，你知道，那个时刻——我们会拥有心灵感应式的，
04:15
you know, helmets that you can wear,
你知道，可以戴上的头盔，
04:17
you can transfer the full amount of your intent and your vision.
你能把自己完整的意图和愿景都传递过去。
04:22
But, you know, that is a little bit far out.
但是，你知道，那还有点太遥远了。
04:24
I think, you know, we're going to dig into the models
我觉得，你知道，我们会深入聊聊这些 models
04:28
and what you've created there.
以及你在那里做出来的东西。
04:31
Talk a little bit about how, but I'm trying to decide
稍微讲讲是怎么做的，不过我在琢磨
04:35
how interesting it is that you even did this.
你居然做了这件事，这到底有多有意思。
04:38
Like, on the one hand, you know,
就是说，一方面，你知道，
04:41
there are lots of, you know,
有很多，你知道，
04:43
there are frontier labs have models
有些 frontier labs 有模型
04:45
and they, a big part of their businesses are allowing companies
而且他们很大一部分业务，就是让公司
04:49
like Canva to use those models as part of broader processes.
比如 Canva，把这些模型作为更大流程的一部分来用。
04:53
There are open weights models that you can build on.
也有 open weights 模型，你可以基于它们来构建。
04:57
And so there's a lot, you know,
所以有很多，你知道，
04:59
that you could have, you Canva could have chosen to build on.
你本来可以，你 Canva 本来可以选择基于这些来构建。
05:02
On the other hand, it's like, you know,
但另一方面，这就像是，你知道，
05:04
maybe it's obvious that a Canva with all of the, you know,
也许很明显，一个 Canva，拥有所有这些，你知道，
05:08
data and user interactions that you have access to
你能接触到的数据和用户交互，
05:12
would build a model.
会去构建一个模型。
05:13
Do you think it's interesting or do you think it's obvious
你觉得这有意思，还是觉得这很明显，
05:15
that you did it?
这件事是你做出来的？
05:16
No, I think it's very interesting
不，我觉得这很有意思
05:17
because I think it also applies to other company,
因为我觉得这也适用于其他公司，
05:19
maybe of similar size when basically you,
也许是规模相近的公司，基本上你
05:22
there's two factors.
有两个因素。
05:24
So first of all, you have unique data
所以首先，你有独特的数据
05:26
that nobody else has.
是别人没有的。
05:28
And then, and then the second part is in deep vertical
然后，然后第二部分是深度垂直整合
05:33
integration of AI models in your stack
把 AI models 整合进你的 stack
05:35
is going to provide incredible performance
会带来惊人的 performance
05:37
and cost reduction.
以及成本降低。
05:39
Those two factors are basically the driving input.
这两个因素基本上就是驱动输入。
05:42
And I'm sure there's a lot of companies that are like
而且我确信有很多公司，就像
05:44
in the thousands of employees that are even smaller
规模在几千名员工，甚至更小
05:48
that will do that and where it makes sense.
会去做这件事，而且在合理的地方去做。
05:51
And so for us, it was kind of the discovery
所以对我们来说，这算是一种发现
05:54
of the fact that you can have,
即这样一个事实：你可以拥有，
05:57
you can find you in our train a model
你会发现，你可以训练一个 model
05:59
that is two orders of magnitude fewer parameters
它的 parameters 少两个数量级
06:03
if you have the data and you can beat the bigger model.
只要你有数据，你就能打赢更大的 model。
06:06
And then because it's smaller,
然后因为它更小，
06:07
you're going to be like another magnitude faster and cheaper.
你大概会再快一个数量级、便宜一个数量级。
06:11
So that is the belief that we had.
所以这就是我们当时的信念。
06:14
And right now, we are in the 40x cheaper
而现在，我们已经便宜 40x
06:18
than like the most expensive frontier models.
比最贵的那些 frontier models 还便宜。
06:21
They can do the same design task,
它们可以完成同样的设计任务，
06:23
but they are using other so many more parameters to do it.
但它们用了多出许多的 parameters 来完成这件事。
06:27
Instead, we have like smaller models
相反，我们有更小的 models，
06:29
that we can fine tune, we can do the first learning
可以对它们做 fine tune，我们可以先做第一轮学习，
06:33
with unique data that we have.
用我们手头独有的数据。
06:35
And we can basically squeeze a lot more,
而且我们基本上能榨出更多，
06:38
same level of quality, sometimes better,
同样的质量水平，有时还更好，
06:41
but most importantly, like incredible speed
但最重要的是，速度快得惊人
06:45
and cost reduction.
以及成本降低。
06:46
So that's one part.
所以这是一部分。
06:47
The second part is that the product itself,
第二部分是，产品本身，
06:50
with AI, is evolving dramatically, right?
有了 AI，正在发生巨大的变化，对吧？
06:53
And the way AI is helping humans generate design
而且 AI 帮助人类生成设计的方式
06:57
is really changing.
真的在变。
06:58
And so there has been a lot of like,
所以有很多那种，
07:01
I would say siloed innovation.
我会说是 siloed innovation。
07:03
We're like, okay, I have an image model.
我们就会说，好吧，我有一个 image model。
07:05
The image model creates an image.
这个 image model 会生成一张图。
07:07
Then I have a next step, an LLAM looks at the,
然后我还有下一步，一个 LLAM 会看看这个，
07:10
and things, oh, we need to change something,
然后想，哦，我们得改点什么，
07:12
goes back to the image model and says,
再回到 image model 那边说，
07:13
hey, can you make something different?
嘿，你能不能生成点不一样的东西？
07:16
And like in the back and forth, you lose control
而且就像这样来回折腾，你会失去控制
07:19
and you're also throwing away information.
同时你也在丢掉信息。
07:22
And so we have, with craft, we have created,
所以，我们通过工艺，创造出了，
07:24
I can image model that most of the work actually
我可以想象，image model 的大部分工作实际上
07:27
is creating a scene graph of what the design should look like.
是创建一个 scene graph，描述设计应该是什么样子。
07:31
And then the conversion to pixel is just the last step
然后，转换成 pixel 只是最后一步
07:34
that we trained from scratch.
而那是我们从零开始训练的。
07:36
But at the layout, let's call it the design layout stage,
但在 layout，我们姑且叫它 design layout 阶段，
07:41
the LLAM can already intervene and make changes
LLAM 已经可以介入并做出修改
07:43
before the pixels are generated.
在 pixels 生成之前。
07:46
And so we are seeing that there's incredible product
所以我们看到，产品层面有非常惊人的
07:50
differentiation by integrating the LLAM word
差异化，靠的就是把 LLAM 这块
07:55
and the diffusion model word if you want.
和 diffusion model 这块整合起来，如果你愿意这么说的话。
07:57
In a more cohesive way,
而且是更连贯、更一体化的方式，
07:59
where like the iterations can happen earlier,
比如迭代可以更早发生，
08:02
and you have all non-destructive modifications
所有修改也都是非破坏性的，
08:06
with humans in the loop.
并且有 humans in the loop。
08:07
Instead of having to bake the result, show it, iterate,
而不是非得先把结果 bake 出来、展示、再迭代，
08:11
and then do basically throw away in that process
然后基本上在那个过程中就丢掉了
08:14
a lot of information every single time.
每次都会丢掉大量信息。
08:17
So those were, I think, the two main factors
所以我觉得，这两个就是主要因素
08:20
that drove our decisions.
推动了我们的决策。
08:22
And also, if you look, the top models
而且，如果你看的话，顶尖模型
08:24
are no open source open weights on the image generation of space.
在 image generation 领域里现在都是 open source、open weights 的。
08:28
I think video is a little bit of a different ecosystem.
我觉得视频这块的生态有点不太一样。
08:31
Actually, it's quite, this will be like probably a net
其实，它挺……这会大概像是一个 net
08:36
is on how the ecosystem for like video model
讲的是 video model 的生态是怎么
08:39
and image generation model is completely different.
跟 image generation model 完全不一样的。
08:42
Sometimes upside down, it's kind of fascinating.
有时候是反过来的，还挺有意思。
08:45
But in what sense, without making it a full episode,
但具体是指哪方面呢，不把它做成一整期的话，
08:48
in what sense?
到底是指哪方面？
08:50
Something happened, like about, I think, four months ago
就大概是四个月前吧，我觉得，发生了一件事
08:53
where like some of the companies developing video model
就是一些开发 video model 的公司
08:59
started releasing open weights versions.
开始发布 open weights 版本了。
09:03
And that had a fantastic domino effect,
而这产生了极好的多米诺骨牌效应，就是它迫使所有竞争对手都这么做。所以现在，你有一个生态系统，其中很多图像模型不是open weights，但视频模型是。然后正因为如此，我觉得也正是因为如此，你看到AI视频生成的每秒成本真的在暴跌，因为有一整个社区
09:06
where like it forced all the competitors to do it.
09:09
And so now, you have an ecosystem
09:10
where a lot of the image models are not open weights,
09:14
but the video models are.
09:16
And then because of that, I think also because of that,
09:19
you see the cost per second of AI video generated,
09:25
like really plummeting, because there's a whole community
09:29
that is trying to figure out how to crack that problem.
也就是在想办法怎么搞定那个问题。
09:31
And because there's a lot of models that are open weights,
而且因为有很多 open weights 的 models，
09:33
more people can apply to that problem.
更多人就能去解决那个问题。
09:37
And so it's a fascinating, different dynamics
所以可以说，这是一种很有意思、很不一样的 dynamics，
09:40
on the two ecosystems, if you want.
就在这两个 ecosystem 里，如果你愿意这么说的话。
09:43
With craft, did you start from an open weights model
那 craft 的话，你们是从一个 open weights model 开始，
09:46
and train it, tune it, or is it a from scratch train?
去 train 它、tune 它，还是 from scratch train 的？
09:52
So for craft, we started, it's like trained from scratch,
所以 craft 的话，我们一开始就是，相当于是 from scratch train 的，
09:57
because we realized that there were some several breakthroughs
因为我们意识到，有几项突破
10:01
that we believe were going to unlock a much more powerful model.
我们相信这些突破会解锁一个强大得多的 model。
10:06
And those were not just there in the existing model.
而这些并不只是现有 model 里本来就有的。
10:08
So that when we trained from scratch,
所以当我们从零开始训练时，
10:11
for Campbell Brash, which is our visual language model,
对于 Campbell Brash，也就是我们的 visual language model，
10:14
we started from an open source in open weights model
我们是从一个 open source、open weights 的 model 起步的
10:18
and then we built on top of that.
然后在此基础上继续构建。
10:20
So you're using whichever of those makes the most sense
所以你们就用其中最有道理的那个。
10:23
given the time or given the particular use case.
考虑到时间，或者考虑到具体的 use case。
10:28
And it sounds like with craft,
而且听起来，用 craft 的话，
10:30
I have covered this idea of a kind of two-stage generation
我之前讲过一种 two-stage generation 的思路，
10:35
with layout and then rendering.
也就是先 layout 再 rendering。
10:38
It was in a conversation with Qualcomm,
那是在和 Qualcomm 的一次对话里，
10:40
who has some papers in that area.
他们在这个领域有一些论文。
10:42
But I don't believe it's not to my knowledge,
但我不觉得——据我所知，
10:45
there are no like popular open weights versions of that.
并没有那种很流行的 open weights 版本。
10:49
Are the two parts of a model like that?
一个 model 的两个部分是这样的吗？
10:53
Are they trained independently or end-to-end
它们是独立训练，还是 end-to-end 训练，
10:56
or what's the system approach to training a model like that?
还是说训练这种 model 的系统性方法是什么？
11:01
Yeah, that's a great question.
对，这个问题问得好。
11:02
So the answer is both.
所以答案是两者兼有。
11:05
So we trained them individually,
所以我们是分别训练它们的，
11:07
but then there's always a time where we're trying to look at the end to end.
但总会有需要我们从 end-to-end 来看的时候。
11:12
But I would say the steps are actually three.
但我会说，其实步骤有三个。
11:14
Now, first, we take all the input
现在，首先，我们把所有 input 都拿过来
11:19
and the prompt or what the context may be.
以及 prompt，或者 context 可能是什么。
11:23
And we try to amplify and make it as powerful as possible.
然后我们会试着把它放大，让它尽可能强大。
11:27
Then there's a process of taking that
接着有一个过程，就是把这些拿过来
11:30
and converting into basically a structured form of data
然后基本上转换成一种结构化的 data 形式
11:34
like more like a syngraph, if you want.
更像是 syngraph，如果你愿意这么说的话。
11:37
And then the last step is generation
然后最后一步就是 generation
11:39
of the T-cells where our model training from scratch
生成 T-cells，也就是我们的 model 从零开始 training 的地方
11:43
has shown really good performance.
已经表现出非常好的性能。
11:47
And by breaking down into these steps,
而且通过拆分成这些步骤，
11:50
you're also allowing for a better integration with agents
你也能更好地和 agents 集成，
11:55
with LLAMs because there could be some components of the pipeline
和 LLAMs 集成，因为 pipeline 里可能有一些组件
12:00
that could be useful for some task.
对某些任务会很有用。
12:01
And if I say, hey, can you just make me ten different layouts
如果我说，嘿，你能不能给我这个单页做十种不同的布局，
12:05
of this single page, right?
对吧？
12:07
I don't need the LLAM to generate pixels.
我不需要 LLAM 去生成 pixels。
12:10
You can use the credit craft design planning
你可以用 credit craft 做设计规划
12:13
can create the different plans,
可以做出不同的方案，
12:16
can have a human in the loop to confirm
可以让 human in the loop 来确认
12:18
and then can do all the generations, for example.
然后比如就能完成所有的 generation。
12:21
So there's actually, we are seeing new workflows coming out
所以实际上，我们看到新的 workflows 正在出现
12:24
from breaking down into these steps.
是通过拆解成这些步骤来的。
12:27
It seems like that would also help with editability.
这看起来也会有助于可编辑性。
12:31
Yeah, historically, you have a scene
对，过去你有一个 scene
12:35
and you say, oh, make the person a little bit bigger
然后你说，哦，把这个人稍微放大一点
12:38
and the thing you get back is totally different.
结果你拿回来的东西完全不一样。
12:40
We're a lot better now.
我们现在好多了。
12:42
Like the GPT 2.5 image models are really, really good
就比如 GPT 2.5 的 image models 真的非常非常擅长
12:46
at consistency between revisions.
在多次修改之间保持一致性。
12:50
But it does strike me that if you could change the scene graph
但让我觉得有意思的是，如果你能改 scene graph
12:56
and move the thing over or make the thing bigger
把那个东西挪过去，或者把它变大
12:58
and not have to re-render,
而且不用重新 re-render，
13:00
that that would help with consistency
那、那会有助于 consistency。
13:03
as you're trying to move things around.
尤其当你想把东西挪来挪去的时候。
13:05
Like 100%, we believe on this source strongly
真的，100%，我们非常相信这个 source。
13:09
that we release magic layers at the beginning of the year
就是我们在年初发布了 magic layers，
13:12
which is basically taking a flat image, decomposing in layers
基本上就是拿一张 flat image，把它拆成 layers
13:17
and elements so that people can really have that fine control.
和 elements，这样大家才能真正有那种精细的控制。
13:21
How many times have you tried to say to any image model generation,
你有多少次试着对任何一个 image model 说，生成的时候，
13:26
oh, can you move that thing a little bit to the right
“哦，能不能把那个东西稍微往右挪一点？”
13:29
or like two feet?
还是说大概两英尺？
13:31
But in that image there is no information
但那张图里并没有信息
13:33
to know what is two feet or two inches or centimeters.
来判断两英尺、两英寸或者厘米到底是多少。
13:38
And so it's a lot easier for our designers to just move
所以对我们的设计师来说，直接去移动
13:41
that element to the right and then regenerate.
那个 element 到右边，然后再 regenerate，会容易得多。
13:44
And so our belief is with craft,
所以我们的理念是，靠 craft，
13:47
basically the humans and the designers can edit elements,
基本上，人类和设计师就可以编辑 element，
13:53
can make them bigger, smaller, move them to the left
把它们变大、变小，移到左边。
13:56
with the right and then regenerate the pixels later.
用正确的，然后再重新生成 pixels。
13:59
And we also are extending the capability to images
我们也在把这项能力扩展到图片上
14:02
that are generated outside Kamba using magic layers.
这些图片是在 Kamba 之外用 magic layers 生成的。
14:05
So actually the majority of images that we process
所以实际上，我们处理的大多数图片
14:09
with magic layers come from outside Kamba
用 magic layers 处理的，都来自 Kamba 之外
14:12
and people generate them somewhere else
人们在别的地方生成它们
14:15
and then they come into Kamba and they want to edit.
然后它们进入 Kamba，人们想编辑。
14:17
And so there we can decompose into layers and elements
所以这样我们就能分解成 layers 和 elements
14:20
and people can start moving things around
人们就可以开始把东西挪来挪去了
14:23
and have that fine level of control.
并且拥有那种细粒度的控制。
14:25
I mean people want consistency and control.
我是说，人们想要的是连贯性和控制感。
14:29
Those are the two things that any creative has asked AI
这两样东西，是任何创作者过去
14:33
for the past, I would say two years.
——我敢说——两年里一直向 AI 要的。
14:35
And we came quite far along,
而且我们已经走了很远，
14:38
but like it's still, you know,
但就像，它还是，你知道，
14:39
we don't want to provide these lot machines, you know,
我们不想提供这些 lot machines，你知道，
14:42
we're like, hey, if you generate 20 times,
我们的意思是，嘿，如果你生成 20 次，
14:45
you may get lucky, right?
说不定就能碰巧得到好结果，对吧？
14:46
We want to actually make sure the time investment
我们其实想确保大家投入的时间
14:49
that people are putting into it is really controlled
真的能被控制住，
14:52
and the result is predictable.
而且结果也是可预测的。
14:54
You kind of referenced earlier two dimensions
你前面大概提到了两个维度
14:58
that were important, quality and efficiency.
它们很重要：质量和效率。
15:01
Talk a little bit about each of those.
那分别聊一点吧。
15:04
We can start with quality.
我们可以先从质量聊起。
15:05
The model performs well like on design arena
这个模型表现很好，比如在 design arena 上
15:11
is one of those number four now.
现在都排到第四了。
15:13
Is that right?
是这样吗？
15:15
What you brought to bear to create a high quality model.
你为了打造高质量模型，都动用了些什么？
15:18
What went into that result?
那个结果背后都有哪些因素？
15:20
I think we underestimated the power of like a decade
我觉得我们低估了差不多十年
15:25
of design generated and design data that we had.
我们积累的设计生成和设计数据的力量。
15:29
I think we were even ourselves a little bit surprised
我觉得我们自己甚至都有点惊讶
15:32
to rank the high because it was also
能排得这么高，因为这也是
15:35
the first submission that we did.
我们做的第一次 submission。
15:36
So we, you know, we have of course internal evils
所以，你知道，我们当然有 internal evals
15:40
and we do a lot of also human evaluation internally,
而且我们在内部也做了很多 human evaluation，
15:44
but you never know until you actually get into
但你永远不知道，直到你真正进入
15:46
the real world arena, but yeah,
真实世界的 arena，不过是的，
15:50
it was basically a lot of work over the last year
这基本上是过去一年的大量工作
15:54
really collecting data that makes sense for designers
真正去收集对设计师来说有意义的数据
15:57
and really harvesting data that Kamba has been created
并且真正去汇集 Kamba 已经创建出来的数据
16:00
for over a decade now, including also data from users
到现在已经十多年了，也包括来自用户的数据
16:07
with their permission or data that we license from the outside.
这些数据是在他们许可下获得的，或者是我们从外部授权来的
16:11
And I think we put together like a pretty good body
而且我觉得我们拼出了一个相当不错的整体
16:15
of a pretty good training set
构成了一个相当不错的 training set
16:18
and I think deliver a lot of value.
而且我觉得这能带来很多价值
16:20
And also we had like several breakthroughs
另外我们也取得了好几项突破
16:22
about how to use the data now to train the model.
关于现在怎么用这些 data 来 train model。
16:25
So I think that was another, you know,
所以我觉得那是另一个，你知道，
16:27
the ability to run, you know, hundreds of experiments
就是能够跑，你知道，几百个 experiments
16:31
and really figure out what worked
然后真正搞清楚什么有用
16:33
and what provided the quality.
以及是什么带来了质量。
16:35
I think that was a great, you know,
我觉得那是一个很棒的，你知道，
16:38
several breakthroughs filed one on top of the other
几个突破一个叠一个地堆叠起来
16:40
that I think made a difference.
我觉得那起了作用。
16:41
Can you give us some examples of the kinds of things
你能给我们举几个例子，说说
16:44
you learned that really moved the needle for you?
你学到的哪些东西真正带来了明显提升吗？
16:46
Without getting too much into the weeds,
不展开太多细节的话，
16:50
we underestimated how detailed
我们低估了你可以给 diffusion model 的
16:54
of a scene description you can give to a diffusion model.
场景描述到底能有多详细。
16:58
Actually, if the diffusion model is above some number
其实，如果 diffusion model 的 parameters 超过一定数量，
17:02
of parameters can handle like incredible detail
它就能处理令人难以置信的
17:07
of the scene.
场景细节。
17:08
So you can, the size of our scene description
所以你看，我们的 scene description 的规模
17:12
really grew in the last six months significantly
在过去六个月里真的显著增长了
17:15
as we realized, oh, actually you can be a lot more precise.
因为我们意识到，哦，其实可以精确得多。
17:20
And so we expanded basically not only the initial prompt,
所以基本上，我们不仅扩充了 initial prompt，
17:23
of course, but like the description of the scene
当然，还有对场景的描述
17:25
is now to a level of detail.
现在已经细到一个程度了。
17:28
I think no human will want to actually go
我觉得没有人会真的愿意去
17:31
and look through and edit, you know,
一条条翻看和编辑，你知道的，
17:33
unless it's like visually editing, right?
除非是像视觉编辑那样，对吧？
17:35
And so there's so much detail.
所以细节真的非常多。
17:38
And by increasing a little bit the size
而只要稍微增大一点
17:42
of our diffusion model we trained from scratch,
我们从零开始训练的 diffusion model 的规模，
17:44
we were actually able to capture all that detail,
我们实际上就能捕捉到所有这些细节，
17:47
all those nuances and really, you know,
所有这些细微差别，而且真的，你知道，
17:50
squeeze the maximum quality out of it.
把它的质量压榨到极致。
17:52
And is that a unique property of your particular
那这是你们这个特定……的独特属性吗？
17:58
diffusion model that you trained
你训练的那个 diffusion model
17:59
or is that just a general observation
还是说那只是一个一般性的观察
18:01
about diffusion models?
关于 diffusion models 的？
18:02
I think it can be generalized for, you know,
我觉得它可以 generalize 到，你知道，
18:06
for more cases.
更多的情况。
18:07
In our case, I think because we understand so well
在我们这个情况里，我觉得是因为我们非常清楚
18:11
the coupling between the generation of the scene
场景的 generation 和
18:15
and the diffusion model, you know, we really work to make sure,
diffusion model 之间的 coupling，你知道，我们真的在努力确保，
18:18
you know, we didn't over generate on the scene description
你知道，我们在 scene description 上并没有 over generate
18:23
or, you know, or under train on the diffusion model.
或者说，你知道，也没有在 diffusion model 上 under train。
18:27
And so the two are like personally calibrated
所以这两者就像是专门校准过的
18:29
to be the input and the ideal input
一个作为 input、一个作为理想 input
18:31
and output model of each other.
并且互为对方的 output model。
18:33
Are the scene descriptions only unstructured texts
scene descriptions 只是 unstructured texts 吗
18:37
or they're also some degree of structure,
还是说它们也有一定程度的结构，
18:41
coordinates, bounding boxes, sizes, things like that?
比如 coordinates、bounding boxes、sizes 之类的？
18:45
Yeah, they are absolutely structured.
对，它们绝对是结构化的。
18:47
Yeah, it's a fully structured,
对，它是完全结构化的，
18:50
it's still text based by it's fully structured
它仍然是 text based，但已经完全结构化了
18:53
with all the details that you just said,
包含你刚才说的所有细节，
18:55
bounding box or a relative position,
bounding box，或者 relative position，
18:57
or position like all that.
或者类似 position 这些。
18:59
We have found that the more detail we can provide
我们发现，我们能给 composition 提供的细节越多
19:03
to a composition, really the better.
真的就越好。
19:06
And we have the ability to basically render
而且我们基本上有能力渲染出
19:09
all that detail in the diffusion model.
diffusion model 里的所有这些细节。
19:12
Also one thing that I want to mention is,
另外我想提一点，
19:15
which is great for the creative industry
这对整个创意行业来说很棒，
19:17
overall, I think there is this race
总的来说，我觉得现在有这么一场竞赛，
19:20
to build the most capable LLM out there, right?
去打造市面上能力最强的 LLM，对吧？
19:24
And now we are already like in the, you know,
而现在我们已经到了，你知道，
19:27
tens or hundreds of trillions or parameters.
几十万亿甚至几百万亿 parameters。
19:30
But for creative tasks, I think we have found
但对于创意类任务，我觉得我们已经发现
19:35
that let's say, you know, a hundred billion,
比如说，你知道，一千亿，
19:37
it's already a pretty large number
已经是个相当大的数字了
19:39
that can, if properly tuned and post-trained
只要好好 tuned 和 post-trained
19:43
and, you know, and fed the right data,
然后，你知道，再喂给它合适的数据，
19:47
a model of that size can already achieve so much.
这个规模的模型就已经能实现很多很多了。
19:51
And we are seeing also a lot of other models,
而且我们也在看到很多其他模型，
19:53
also video models that leverage, you know, LLM
还有利用 LLM 的 video models，你知道。
19:55
as an input to create that structured data
作为输入来创建那种结构化数据
19:59
like the Minimax H3 is another model
比如 Minimax H3 是另一个模型
20:01
that is as a similar approach.
那是一种类似的方法。
20:03
You know, it's a video model.
你知道，它是一个视频模型。
20:05
But we are seeing the size of the basically vision,
但我们看到，基本上 vision
20:10
language model, LLM, there is the input,
language model、LLM 的规模，也就是输入这块，
20:13
you know, not growing 10X every three months anymore,
你知道，不再每三个月增长 10X 了，
20:17
but being kind of stable.
而是变得比较稳定了。
20:18
In an industry where like everything grows,
在一个像所有东西都在增长的行业里，
20:20
you know, 10X every month, like that number,
你知道，每个月 10X，就像那个数字，
20:24
I've shown that like, you know, you don't need
我已经证明了，就像，你知道，你不需要
20:27
more than a hundred billion or a meter
超过一千亿或者一米
20:30
to do most of the creative tasks, I would say.
就能完成大多数创意任务，我会这么说。
20:33
You know, probably like video understanding,
你知道，大概像 video understanding，
20:36
maybe it's just a little bit more
可能只是稍微多一点
20:38
because it's probably the most complex type of task
因为它大概是最复杂的一种任务
20:40
but for everything else, like, you know,
但其他所有情况，就像，你知道的，
20:44
below 100 billion, you can, a parameters,
低于 1000 亿，你可以，呃，parameters，
20:46
you can already achieve wonders.
就已经能做出很了不起的东西了。
20:49
And so to be clear on what you're describing there,
所以，为了说清楚你在这里描述的是什么，
20:52
in this two-stage model where you have an LLM-like thing
在这个两阶段模型里，你有一个类似 LLM 的东西
20:57
or an LLM period, generating a scene description,
或者就是一个 LLM，就这样，在生成场景描述，
21:00
you're saying there's kind of diminishing returns
你是在说，超过 1000 亿之后
21:03
beyond 100 billion.
会有某种边际收益递减。
21:04
And smaller models do just as well as larger models
而且更小的 model 和更大的 model 表现一样好
21:09
and kind of nailing that scene description
还能把那个场景描述拿捏得挺准
21:10
and providing the right input to the diffusion model.
并且给 diffusion model 提供正确的 input。
21:13
That's correct.
没错。
21:14
And while being, you know, a lot faster
而且同时呢，你知道，速度还快很多
21:16
and a lot cheaper, right?
还便宜很多，对吧？
21:18
And so just the economics, they don't work
所以单从经济性来看，这套根本行不通
21:22
if you use like the latest frontier model to do that task.
如果你用比如最新的 frontier model 去做那个任务的话。
21:24
Then, you know, it will be like a dollar
然后，你知道，大概就是一美元
21:26
for image generated or something like that.
每生成一张 image 之类的。
21:29
And so, and so these are, these I think together,
所以，所以这些，我觉得这些合在一起，
21:33
with the ability of people to train those smaller models
再加上人们能够训练那些更小的 models
21:36
in a very efficient ways, I think,
用非常高效的方式，我觉得，
21:38
really unlocked a lot of value
真的释放了很多价值
21:40
and a lot of usefulness of AI in decorative workflows.
以及 AI 在装饰类 workflows 里的很多实用性。
21:45
To what degree did you use synthetic data
你在多大程度上用了 synthetic data？
21:47
in the training process?
在训练过程中吗？
21:49
Is that a big part of your recipe?
这是你们方案里很重要的一部分吗？
21:51
Yeah, it is.
是的，确实是。
21:52
I think like in, you know, every data set of this size,
我觉得，你知道，对于每一个这种规模的数据集来说，
21:57
we also want to make it as diverse as possible.
我们也想让它尽可能多样化。
21:59
And so we have all kind of data coming in
所以我们会引入各种各样的数据，
22:02
from, you know, stock images
比如，你知道，图库图片
22:06
and that kind of stuff to synthetic data,
以及类似的东西，一直到 synthetic data，
22:08
to user generated data, to designs that we created.
到 user generated data，再到我们创作的设计。
22:11
We have, you know, millions of designs
你知道，我们有几百万个设计，
22:13
that can be created or licensed over the last decade.
过去十年里能创作出来或者授权出去的。
22:17
And so that was also, of course, a big part of it.
所以这当然也是其中很大的一部分。
22:20
It kind of helps also with the taste, right?
这对审美也挺有帮助的，对吧？
22:22
Because we have seen AI's law and AI generation
因为我们见过 AI's law 和 AI generation，
22:26
where like everything looks very similar.
就是那种所有东西看起来都很像的情况。
22:28
And so I think we put so much effort on diversity,
所以我觉得我们在多样性上投入了特别多精力，
22:32
which is also aligns with our company value.
这也和我们的公司价值观一致。
22:36
And so we are a company that wants to empower everybody to design.
所以我们是一家希望赋能每个人去设计的公司。
22:40
We are global.
我们是全球化的。
22:42
We serve every market, every language.
我们服务每一个市场、每一种语言。
22:45
And so we wanted to really push the diversity
所以我们真的很想推动多样性
22:48
to make sure that the output also of the model
确保模型的输出也能
22:52
is as diverse as possible.
尽可能多样化。
22:53
Talk a little bit about the way you think about benchmarks
稍微谈谈你们是怎么看待 benchmarks 的
22:56
for image generation, you know,
对于 image generation 来说，你知道，
22:59
there are well-established benchmarks.
已经有一些很成熟的 benchmarks。
23:02
But I always wonder, you know,
但我总是会想，你知道，
23:04
how well they capture kind of the human perspective
它们到底在多大程度上能体现那种人类的视角
23:08
and human judgment and taste as you just referenced.
以及你刚才提到的人类判断和品味。
23:11
And I'm wondering, you know,
然后我在想，你知道，
23:12
if someone works on this constantly,
如果一个人一直做这个，
23:13
like how do you think about, you know,
比如你会怎么思考，你知道，
23:15
the applicability of the popular benchmarks,
热门 benchmarks 的适用性，
23:17
how do you supplement them, that kind of thing?
你们怎么补充它们，之类的？
23:20
That's a great question.
这个问题问得好。
23:21
I would say the benchmarks overall
我觉得总的来说，benchmarks
23:23
are lagging a little bit behind
有点滞后，
23:25
to where like the power of AI in design is going to be.
跟 AI 在设计中的能力未来会达到的水平相比。
23:31
Because they're mostly focused on like single shot,
因为它们大多关注的是类似 single shot，
23:34
the first generation.
也就是第一次生成。
23:37
And you have benchmarks for like image editing,
而且像 image editing 这种，你是有 benchmarks 的，
23:40
which is great, but like design editing
这很不错，但像 design editing
23:42
is different than just image editing.
和单纯的 image editing 是不一样的。
23:45
And so as of today,
所以截至今天，
23:47
there is no established benchmark
还没有一个既定的 benchmark
23:49
for how good a model is at editing a design, for example.
来衡量一个 model 编辑 design 的能力到底有多好，比如说。
23:53
There's some public benchmark for generating,
有一些公开的 benchmark，是用来 generating 的，
23:56
let's say a presentation like this,
比如说像这样的一个 presentation，
23:58
a present bench being one,
present bench 就是其中一个，
24:00
you know, has like 200 examples of like a presentation
你知道，它大概有 200 个示例，像是和演示文稿相关的 slide、相关任务。
24:04
related slide, related tasks.
你可以用它来测试，但这又是新一代的了。
24:06
And you can test it, but again is new generation.
所以并没有很多 benchmarks 能真正评估编辑一个设计的有效性、速度，以及随之而来的成本。
24:10
And so there's not a lot of benchmarks
24:13
that really can assess the effectiveness
24:16
of editing a design, the speed,
24:19
and the cost also to go with that.
24:21
So I hope there's going to be more.
所以，我希望以后会有更多。
24:23
Of course, we have internally,
当然，我们内部，
24:25
we have a lot of those,
我们有很多这类东西，
24:26
but yeah, we haven't found an industry established benchmark
但说实话，我们还没找到行业公认的 benchmark
24:30
for the ability to actually edit a design
来衡量真正编辑一个设计的能力
24:32
because the industry is focused on,
因为行业现在关注的是，
24:34
oh, AI is so magic,
哦，AI 太神奇了，
24:35
it's going to single shot everything,
它会把一切都 single shot 搞定，
24:37
which we know is not the case.
而我们知道事实并非如此。
24:39
And so I think now the understanding is finally landing.
所以我觉得现在这种理解终于开始落地了。
24:44
And I hope we're going to see more benchmarks
我希望我们会看到更多 benchmarks
24:47
that really start from like a given set
真正从一组给定的
24:50
of preexisting designs
已有 designs 出发
24:52
and defines rubrics
并定义 rubrics
24:53
for how those designs can be modified
来说明这些 designs 可以怎么修改
24:57
and then does the comparison
然后再做比较
24:59
and the benchmarking on that.
以及在那方面的 benchmarking。
25:00
That kind of implies that you can use an LLM or VLM
这某种程度上意味着，你可以用 LLM 或 VLM
25:06
as a judge to look at an image
来作为 judge 看一张图
25:09
and assess it visually,
并从视觉上评估它，
25:12
you know, from any number of perspectives.
你知道，从任意多个角度。
25:16
Are we there yet?
我们已经到那一步了吗？
25:17
Or, you know, how good are VLMs
或者说，你知道，VLM 到底有多好
25:21
at kind of articulating the qualities
在表达这些特质方面？
25:25
that you might want to optimize an image for?
你可能会想针对它来优化一张图？
25:27
I think they're really amazing.
我觉得它们真的很厉害。
25:29
And we have also to keep in mind that
而且我们也得记住，
25:31
like if you structure the rubrics really well,
比如说，如果你把 rubrics 结构得特别好，
25:34
they are really good at defining, I don't know,
它们真的很擅长定义，我不知道，
25:36
is this design readable?
这个设计可读吗？
25:39
Like, what's the score of readability?
比如，readability 的分数是多少？
25:41
What's the score of like layout and composition?
那 layout 和 composition 的分数是多少？
25:43
So we have very well articulated rubrics
所以我们有表述得非常清楚的 rubrics
25:46
about the design quality
关于设计质量
25:48
and then we can use VLMs to judge that.
然后我们可以用 VLMs 来评判这个。
25:51
And we get, you know, quite far along.
而且，你知道，我们进展得相当不错。
25:54
Of course, we always do also human evaluation
当然，我们也一直会做 human evaluation
25:57
and some of the arenas are strongly, you know,
而且有些 arenas 会非常，你知道，
25:59
leveraging that because it's the ultimate test.
依赖这一点，因为这是终极测试。
26:02
But I think, yeah, we came a long way.
但我觉得，是啊，我们已经走了很远了。
26:06
It forced us also to really define
这也迫使我们真正去定义
26:09
what a good design means, right?
什么才算好的设计，对吧？
26:11
And so if humans are not able to articulate that,
所以如果人类都没法把这一点讲清楚，
26:14
then VLMs are not going to be able to score it, right?
那 VLMs 也没法给它打分，对吧？
26:17
And so I think we had to really go back
所以我觉得我们不得不真正回过头去
26:19
and really understand what makes the design really good
真正理解到底是什么让设计变得真正出色
26:22
to be able to leverage VLMs to do it.
才能利用 VLMs 来做这件事。
26:25
But I think now we are going to a place
但我觉得现在我们正进入一个阶段
26:28
where it's quite reliable.
在这一点上它相当可靠。
26:31
When you think about this two model approach in craft,
当你想到 Craft 里的这种 two model approach 时，
26:34
what are the qualities that,
那些特质是什么，
26:37
what are the qualities of an image come from,
一张 image 的特质有多少来自，
26:40
you know, having the two separate systems or models
你知道，有两个独立的 systems 或 models，
26:43
and what just come from having better training data?
又有多少只是来自更好的 training data？
26:47
Yeah, that's the interesting question.
对，这确实是个有意思的问题。
26:49
We have seen just different behavior,
我们只是看到了不同的 behavior，
26:51
depending on a different task, for example,
比如，这取决于不同的任务，
26:55
which we are always trying to understand
我们一直想弄明白的是
26:58
is the limitation of the diffusion model
是 diffusion model 的局限性
27:00
or is the, you know, the, the alongside of things.
还是说，你知道，那个，那个，那些并列在一起的东西。
27:04
And sometimes it's hard to know.
而且有时候很难判断。
27:08
We have seen, for example,
比如说，我们见过
27:09
the model performed really well on like portraits
模型在像人像这样的内容上表现得特别好
27:13
and people and also nature landscapes.
以及人物，还有自然风景。
27:17
We have seen the model doing not so great
我们看到过 model 表现没那么好，
27:19
on the cartoon type of content for example.
比如说在卡通类的内容上。
27:24
Is it not so great?
是不太好吗？
27:26
Yeah, I mean, it's not ranking in the top three for that.
对，我是说，它在这个类别里没排进前三。
27:29
And so that's an area there.
所以那是一个方面。
27:30
So like all that is together,
所以这些全都加在一起，
27:33
that is counterintuitive to me
这对我来说挺反直觉的，
27:35
because, you know, a portrait, for example,
因为你知道，比如说人像，
27:40
seems like it would kind of know the value
就感觉它多少会知道
27:43
of like the planning and the scene component,
planning 和 scene component 的价值，
27:46
like the supporter, you know, and a cartoon,
就像那个 supporter，你知道，还有卡通里，
27:49
for example, like, you know,
比如说，就像，你知道，
27:51
you've got all these things in a cartoon,
卡通里这些东西你全都有，
27:53
you know, that would lean a lot on a scene generating component.
你知道，那会很依赖一个 scene generating component。
28:00
But the two results that you're seeing suggest
但你看到的这两个结果说明
28:04
that that model shouldn't be as strong as it is
那个 model 不该有现在这么强
28:07
or that that's not the source of the strength of the model.
或者说，那并不是模型强大之处的来源。
28:10
Am I thinking about that correctly?
我这样理解对吗？
28:12
Yeah, it's interesting.
嗯，挺有意思的。
28:13
So the number one thing for people and portraits
所以对于人物和肖像来说，你最想要的第一件事，基本上就是尽可能 photorealistic，对吧？
28:15
that you want is basically to be as photorealistic
就是你不想要那种 AI skin effect 之类的东西。
28:18
as possible, right?
28:19
Like, you don't want to have that AI skin effect
28:22
and all that stuff.
28:23
And we put so much effort in this
我们在这件事上投入了特别多精力
28:27
because we were like,
因为我们当时就在想，
28:28
we got a nail it should not look like AI generated.
我们必须把它做好，它不能看起来像 AI 生成的。
28:32
I think that effort may have bled into the cartoon side
我觉得那份努力可能也蔓延到了卡通这边
28:36
where instead you want very stylized.
而在卡通这边，你其实想要的是非常风格化的效果。
28:38
And so it turns out that the images,
结果发现，那些图像，
28:40
the cartoon images generated with craft initially
一开始精心制作生成的卡通图像
28:44
were a little bit too realistic and too like 3D looking.
有点太写实了，也太像 3D 了。
28:49
But again, this is something that, you know,
但话说回来，你知道，这其实是那种
28:51
we, you know, we, we discover when we put it on the arena
我们，你知道，我们把它放到 arena 上时才发现的东西
28:55
because probably they had like a different population
因为可能他们那边有一批不一样的
28:58
of designers that would test it.
设计师来测试它。
29:00
But at the same time, it's something that is more about the taste
但与此同时，这更多是关乎 model 的品味
29:04
of the model and that is a lot simpler,
而且这要简单得多，
29:07
a lot easier and a lot faster to tune and correct
tune 和纠正起来也容易得多、快得多
29:10
and so that we are in the process basically doing exactly that
所以我们现在基本上就是在做这件事
29:13
and, you know, tuning the taste of the model
而且，你知道，调模型的品味
29:17
and the style of the model is more of a fine tuning
而模型的风格更多是一种 fine-tuning
29:20
and RL type over task.
以及 RL 类型的任务。
29:23
You don't have to pre-trained, right?
你不需要 pre-trained，对吧？
29:25
In the pre-training, we of course had a lot of cartoon content.
在 pre-training 里，我们当然有很多卡通内容。
29:29
It just now, that capability is didn't surface to the full extent.
只是现在，那个能力还没有完全展现出来。
29:34
And so now we are like teaching the model,
所以现在我们就好像在教模型，
29:37
okay, this is a style that resonates more with humans
好，这是一种更能引起人类共鸣的风格
29:41
and we want, you know, if this cartoon is to be flat-shaded
而且我们希望，你知道，如果这个卡通是 flat-shaded 的话
29:44
and these lines and so on, I mean, it's fascinating, honestly.
还有这些线条之类的，我是说，老实说，这真的很迷人。
29:49
It's a journey of like learning as much as the model is learning.
这就像一段旅程，我们学到的东西和 model 学到的一样多。
29:54
We are learning how to train the model
我们在学习怎么 train 这个 model
29:56
and how the model behaves.
以及这个 model 是怎么表现的。
29:59
It's, you know, sometimes you'll find this maybe a little weird.
你知道，有时候你可能会觉得这有点奇怪。
30:03
But sometimes I have like two small kids
但有时候我有两个小孩
30:05
and sometimes I learn things about learning
然后有时候我会学到一些关于学习本身的东西
30:08
as a high level concept that I learn by training the models.
作为一个我通过训练模型学到的高层概念。
30:14
I'm like, oh, this I'm sure applies also to humans, you know?
我就想，哦，这个我确定也适用于人类，你知道吧？
30:17
And so it's kind of fascinating, especially on pre-training.
所以这挺有意思的，尤其是在 pre-training 上。
30:22
You see pre-training is basically like teaching a human
你看，pre-training 基本上就像是教一个人
30:26
as many words as possible about vocabulary, right?
尽可能多地教他词汇，对吧？
30:30
But has nothing to do with the style of talking performance
但这跟说话表现的风格没关系
30:36
or like if that human becomes, I don't know, a stand-up comedian,
或者比如说那个人成了，我不知道，一个单口喜剧演员，
30:40
the how many words it knows has nothing to do
它知道多少词跟这个一点关系都没有
30:43
with like the style of the delivery, right?
就像那种表达风格，对吧？
30:46
And so I think we, the concept is so true
所以我觉得，这个概念真的很成立
30:48
for pre-training and post-training and AI model
对 pre-training、post-training 和 AI model 来说都是如此
30:51
and is also true for like humans.
对人类来说也一样。
30:53
We're like in the early years you want as much variety
就像人在早年，你会想要尽可能多的多样性
30:57
and as much exposure to different type of activities,
尽可能多地接触不同类型的活动、
31:01
sensory information, experiences, languages and so on, right?
感官信息、经历、语言等等，对吧？
31:07
And so the parallel actually is fascinating.
所以这个类比其实很有意思。
31:09
I think I'm sure this book is really written about it.
我觉得我挺确定，这本书确实就是写这个的。
31:13
Let's talk a little bit about brush.
我们稍微聊聊 brush 吧。
31:15
Is brush, what's the best way to think about it?
那 brush 呢，最好怎么理解它？
31:17
Is it an editing model?
它是一个 editing model 吗？
31:19
Is it agent, a design agent?
它是一个 agent，一个 design agent 吗？
31:23
A little bit of both of those things?
还是这两者都有一点？
31:24
Yeah, we call it like a vision language model,
嗯，我们叫它 vision language model 这类东西，
31:27
but it's basically an agent.
但它基本上就是一个 agent。
31:29
They can create design, it can also edit design.
它们可以创建设计，也可以编辑设计。
31:33
So has a deep understanding of what's going on
所以它能深入理解到底在发生什么
31:36
in the layout of your design
在你设计的 layout 里
31:39
and can really take very fine grain directives
并且真的能接收非常细粒度的指令
31:44
from humans and then operate on the design itself.
来自人类，然后对设计本身进行操作。
31:47
So what is a design in this context?
那在这个语境里，设计是什么？
31:49
It's not an image, it's thinking of it like a slide
它不是一张图片，而是把它想成一张幻灯片
31:53
or a camera document, multiple layered composition of things.
或者一个 camera document，是多个东西的分层组合。
31:58
Yeah, could be multiple layers, could be multiple elements,
嗯，可能有多层，也可能有多个元素，
32:02
could also different type of elements.
也可能是不同类型的元素。
32:03
I can have some vector graphics and some image
我可以有一些 vector graphics，还有一些 image
32:07
mixed up together and then some text as font.
混在一起，然后再加一些作为 font 的 text。
32:11
So it's basically this heterogeneous composition
所以它基本上就是这种 heterogeneous composition
32:14
of different elements.
由不同元素组成。
32:16
They have particular features and particular type of details
它们有特定的 features 和特定类型的 details，
32:22
that humans can tweak and also the agents can tweak.
人类可以 tweak，agents 也可以 tweak。
32:26
And then they all come together to create that unique design.
然后它们全都汇聚到一起，创造出那个独特的设计。
32:29
There is the final output.
这就是最终的输出。
32:31
So it's a little bit like what craft generates
所以这有点像 Craft 生成的东西
32:35
in the first step is basically all the underlying structure
第一步基本上就是一个设计的所有底层结构
32:40
of a design.
一个设计的。
32:42
Brush really understands that very well.
Brush 非常理解这一点。
32:44
It can really act on a design and then relay out,
它真的能对一个设计进行操作，然后重新布局，
32:48
can move parts, can make it more readable,
能移动各个部分，能让它更易读，
32:50
more professional, can take high-level inputs.
更专业，能接收高层次的输入。
32:53
Like, hey, make this thing pop or make it cool.
比如，嘿，让这个东西更抓眼球，或者让它变酷。
32:57
I'm like, you know, make it Gen Z.
我就说，你懂的，让它更 Gen Z。
32:59
Like these are, you know, directives will be extremely hard
比如这些，你知道，这些指令会非常难
33:03
for a designer that is not to maybe a Gen Z designer
对一个不是——也许不是 Gen Z 的设计师来说
33:07
to really understand, you know.
真正理解，你知道吧。
33:09
So those are high-level directives
所以那些都是高层次的指令
33:11
instead can really be applied by brush, you know,
反而真的可以用笔刷来套用，你知道，
33:15
quite successful way.
挺成功的一种方式。
33:17
I'm kind of thinking about it.
我大概是在想这个。
33:18
Like, so with craft you've got this scene creator
就像，用 craft 的话，你有一个 scene creator
33:23
and you've got this generator and the generator,
然后你有一个 generator，而这个 generator，
33:27
you know, generates one kind of one image.
你知道，它会生成一种图像。
33:30
And brush, you have the same scene layout thing,
而 brush 呢，你有同样的 scene layout 这东西，
33:36
but maybe it has different tools
但也许它有不同的工具
33:38
and that, and one of those tools is generating images
而且，其中一个工具就是生成图像
33:41
and it's got, you know, layout tools
而且它还有，你知道，layout 工具
33:44
and the output is like a composition of sorts.
输出像是某种 composition。
33:48
Is that a reasonable way to think about it
这样理解合理吗
33:50
or where does that way of thinking
还是说这种理解方式
33:51
about a breakdown from what's actually happening?
在哪儿和实际情况对不上？
33:53
Yeah, it's not that far.
嗯，也没差那么远。
33:54
Like I would say right now, brush can do more general things
就像我现在会说，brush 能做更通用的事情
33:59
than just operating a layout description of the scene.
而不只是操作场景的 layout 描述。
34:04
But basically it operates a set of tools,
但基本上，它操作的是一组工具，
34:06
like a large set of tools, they're all design tools.
就像一大套工具，而且全是设计工具。
34:09
It can make decisions around design.
它能围绕设计做决策。
34:11
I think it goes beyond the image generation and editing, right?
我觉得它不只是 image generation 和 editing，对吧？
34:15
It can also create like, you know, 30 slides deck
它还能做，你知道，30 页的 slide deck，
34:20
can make sure that deck is cohesive with like branding.
还能确保那个 deck 跟 branding 保持一致。
34:24
So I think you operate like a higher level of abstraction
所以我觉得你操作的是更高层次的 abstraction，
34:28
than craft which is really about image
而不是 craft，craft 其实更偏 image。
34:31
and soon will be also doing more of image editing.
而且很快还会做更多 image editing。
34:36
And so, yeah, brush is basically a full agent
所以，对，brush 基本上就是一个完整的 agent，
34:39
that you can ask for like more complex work
你可以让它做更复杂的工作，
34:42
and more abstracted type of instructions.
以及更抽象的那类指令。
34:46
And then, you know, we are constantly iterating on,
然后，你知道，我们一直在不断迭代，
34:50
I mean, it's a lot is like having basically another lamb
我是说，这很大程度上就像是基本上又有一个 lamb
34:52
in the loop using a bunch of tools
在 loop 里，用一堆工具，
34:54
and be able to see what it generates.
并且能看到它生成什么。
34:56
And that loop is really what has changed everything
而正是这个 loop 真正改变了一切
35:00
because until I don't know a year ago,
因为直到——我也说不清，大概一年前吧
35:02
LLMs would just do like, we're blind
LLMs 就只会那样，好像自己是瞎的
35:04
and we'll just do a single shot
然后只做一次 single shot
35:07
and then be blind to the result, right?
之后对结果就完全看不见了，对吧？
35:10
And instead now, not only that single shot
而现在呢，不仅那个 single shot 变成了一个带 human in the loop 的 iterative process
35:13
becomes an iterative process with human in the loop,
而且 LLMs 现在能看到自己生成的内容，并且能迭代
35:15
but LLMs now can see what they generate and can iterate.
35:19
And the faster you make the iterative loop,
而且你把 iterative loop 做得越快，
35:23
the more effective everything is
一切就越有效，
35:24
because now an LLM can do maybe 10 iterations
因为现在一个 LLM 也许能做 10 次迭代
35:27
before even shows the result to a human.
甚至在把结果展示给人类之前。
35:30
And before every iteration will take, you know, a minute
而以前每次迭代都要花，你知道，一分钟
35:33
and so that was not, you know, feasible.
所以那，你知道，并不可行。
35:36
But now, you know, that's why speed is not just a nice to have.
但现在，你知道，这就是为什么速度不只是个锦上添花的东西。
35:39
Speed is like a complete game changer
速度简直就是一个彻底改变游戏规则的东西。
35:41
because it can really increase the quality.
因为它确实能提升质量。
35:44
It can increase the user experience.
它能提升用户体验。
35:47
It can really make you like an iterative process.
它真的能让你喜欢上迭代的过程。
35:49
Sounds like there's a simpler way to think about a craft
听起来有一种更简单的方式来理解 craft，
35:53
as a model and brushes it kind of an agent.
就是把它当成模型，而 brushes 有点像 agent。
35:57
And so in creating brush, like I'm imagining,
所以在创建 brush 的时候，我就在想，
36:01
like part of the task here is like a custom harness
比如这里的一部分任务好像就是做一个 custom harness。
36:05
and I'm wondering what all that entailed.
我想知道那都涉及了些什么。
36:07
How did you approach that harness development?
你们当时是怎么推进那个 harness 开发的？
36:10
This is a great question because I think people are,
这是个很好的问题，因为我觉得大家，
36:14
like usually they are all focused on the model
通常都只盯着 model
36:17
and not the harness.
而不是 harness。
36:18
But I think the importance of the harness
但我觉得 harness 的重要性
36:22
cannot be overstated.
怎么强调都不为过。
36:24
And for us, this probably was the hardest part
而对我们来说，这可能是最难的部分
36:28
and the ones the part where we had the most iterations.
也是我们 iterations 最多的那部分。
36:31
In fact, we already like our third generation of harness
其实，我们已经在用第三代 harness 了
36:35
for brush.
给 brush 用的。
36:38
And really like the high level goal is basically
而且其实，高层目标基本上就是
36:42
to give as much freedom to the LLM
尽可能给 LLM 更多自由
36:45
to really run free of constraint
让它真的不受约束地运行
36:47
and really be able to like tap into the tools
并且真的能够接入这些工具
36:51
and have as much insight on what is able to generate.
对它能够生成什么有尽可能多的洞察。
36:54
And so we have really focused on making the harness
所以我们一直非常专注于打造这个 harness
36:58
thinner and smaller and more agile and more and simple.
更薄、更小、更敏捷，也更简单。
37:02
And then at the end, you know,
然后到最后，你知道，
37:03
a brush can write code to operate tools
brush 能写代码来操作工具
37:08
and make design operations.
并进行设计操作。
37:10
And so we were at the same time trying to leverage
所以我们同时试图利用
37:12
how good these LLMs are writing code
这些 LLMs 写代码有多强
37:15
and at the same time simplifying the harness
同时简化 harness
37:18
in the most basic possible way
用尽可能最基础的方式
37:20
to really unleash the full power of the LLM.
真正把 LLM 的全部能力释放出来。
37:23
And so that's where we did a lot of iteration.
所以那就是我们做了大量迭代的地方。
37:25
And actually it was more about like removing the human,
而实际上，这更多是关于把人类的
37:29
you know, knowledge and just letting the LLM like run free
你知道，知识给去掉，然后就让 LLM 自由发挥
37:34
to some extent.
在一定程度上。
37:35
And when you talk about the code generation
而当你说到 code generation 时
37:39
what kind of code are we talking about?
我们说的是什么样的 code？
37:41
Like I can imagine a lot of what brush might want to do
就像我能想象 brush 可能想做的很多事情
37:45
is running a Python code, but I could also imagine
它正在跑一段 Python 代码，不过我也能想象
37:49
you creating like some design DSL
你去创建某种 design DSL 之类的
37:51
that's like more natively represents the things
它更像是能更原生地表达那些东西
37:54
that you care about.
你真正在乎的东西。
37:55
Yeah, it can be that or it can be just writing code
对，可以是那种，也可以只是写代码
37:58
also to operate, you know, API calls
也用来操作，你知道，API calls
38:01
and like LLL operations and that kind of stuff.
还有像 LLL operations 那一类的东西。
38:05
So it's really code or a little bit of both or like
所以它其实就是代码，或者两者兼有一点，或者像
38:08
it's definitely both.
这绝对是两者兼有。
38:10
Yeah, like the capabilities are quite broad.
对，能力范围确实挺广的。
38:13
What's an example of something that a user would do
能举个例子吗，说说用户会做、而且
38:16
and brush that uses, you know, either code
brush 会用到的事情，你知道，要么是代码
38:19
or this kind of some kind of DSL or abstraction?
要么是某种 DSL 或 abstraction？
38:22
I mean, a lot of the editing operation
我是说，很多编辑操作
38:24
that you can do on an editor,
在编辑器上能做的，
38:27
they can be expressed as like editing
都可以表达成类似 editing 的形式
38:30
to an editing API, right?
到一个 editing API，对吧？
38:32
And the problem is there is
而且问题在于
38:36
there's like infinite ways of describing
有几乎无限种方式去描述
38:38
that design operation.
那个 design operation。
38:40
Let's say I take an object, I move it to the left
假设我拿一个对象，把它移到左边
38:42
and I rotate and I make it bigger, right?
再旋转一下，再把它放大，对吧？
38:44
Like these three operations can be encoded
就像这三个操作可以被编码成
38:48
as a format change, you know,
一次 format change，你懂吧，
38:51
you can have a format to describe the whole state
你可以用一个 format 来描述整个 state
38:54
or you can describe them as diffs
或者你可以把它们描述成 diffs
38:58
and they can be achieved through like an editing API.
而且它们可以通过类似 editing API 的方式来实现。
39:01
And so that is the complexity that we had to navigate
所以这就是我们当时必须去应对的复杂性
39:05
trying to understand like what was
试着去理解到底什么是
39:07
more natural for an LLM,
对 LLM 来说更自然的，
39:10
which actually has to do with,
而这实际上跟
39:13
what is the most available data out there
现有最唾手可得的 data 是什么有关
39:16
that the LLM are trained on, right?
LLM 就是拿这些来训练的，对吧？
39:18
So this way you kind of leverage a groove
所以这样一来，你多少算是利用了一个现成的路径
39:22
that has been already created by an enormous body
这个路径早就由一个庞大的
39:26
of data instead of trying to teach something new
数据体建立起来了，而不是试图教一些新东西
39:29
or like there is underrepresented
或者说，就像是有东西 underrepresented
39:31
in the training set or is out of distribution
在 training set 里，或者 out of distribution
39:34
in the training set of an LLM.
在 LLM 的 training set 里。
39:36
And so I think that's where, you know,
所以我觉得，这就是那个点，你知道，
39:38
you're trying to align with the wind, right?
你是想顺着风向走，对吧？
39:40
And the capabilities as opposed to trying to like find it.
以及能力，而不是试图去找它。
39:44
Earlier I mentioned kind of these two north stars
之前我提到过类似这两个 north stars
39:47
you had quality and efficiency
一个是质量，一个是效率
39:48
and I think we double clicked on quality
我觉得我们把质量深入展开讲了
39:50
but didn't really double click on efficiency.
但并没有真正把效率深入展开讲。
39:53
Recap your kind of vision for efficiency
再概括一下你对效率的愿景
39:57
and you know, talk through in the case of craft
然后你知道，结合 craft 的情况来讲讲
40:00
and brush like what were the levers
然后，就像，你用了哪些杠杆
40:02
that you manipulated to drive that efficiency?
来推动那种效率？
40:06
Yeah, efficiency in inference specifically
对，尤其是 inference 的效率
40:10
really enables affordability of AI, right?
真的让 AI 变得用得起，对吧？
40:14
And we, the majority of our users
而且我们，大多数用户
40:16
are actually free users of Canva
其实是 Canva 的免费用户
40:18
and we love that like students,
我们很喜欢这一点，比如学生、
40:20
non-profit can use Canva for free.
非营利组织可以免费使用 Canva。
40:23
If we can bring the cost of AI inference low,
如果能把 AI inference 的成本降下来，
40:26
we can also allow them to use more of AI.
我们也能让他们更多地使用 AI。
40:29
And so we really wanted to make sure AI was something
所以我们真的很想确保 AI 是
40:32
that was within reach for everybody.
每个人都能用得上的东西。
40:36
And so the optimizations we have learned
所以我们学到的这些优化，
40:38
is we are overall relatively early
总体来说我们还在
40:42
in the journey of optimizing AI inference.
优化 AI inference 这条路上相对早期的阶段。
40:46
If you can squeeze 10x performance improvement
如果你能挤出 10 倍的性能提升
40:50
in a month means you're very early, right?
一个月的话，说明你还处在非常早期，对吧？
40:53
Like we are so far from the from the plateau right now
就像我们现在离 plateau 还远得很
40:57
and that's very exciting because it's also very satisfying
而且这非常令人兴奋，因为这也特别有成就感
41:01
for our AI engineers to be able to work on a problem
对我们的 AI 工程师来说，能在一个问题上工作
41:04
for like two weeks and be like,
大概两周，然后就会觉得，
41:05
oh, okay, it's five times faster
哦，好吧，它快了五倍
41:07
than it was two weeks ago, right?
比两周前，对吧？
41:08
And so we have, we have put a lot of effort.
所以我们已经，我们已经投入了很多精力。
41:11
And if the only you can only make it really efficient
而且，如果唯一能真正把它做得非常高效的方式
41:16
if you own the full stack, right?
就是你拥有整个 full stack，对吧？
41:18
Because there are optimizations you do
因为有些 optimizations 是你在
41:21
at the like GPU level, at the hardware level,
比如 GPU level、hardware level 做的，
41:24
the optimizations you do at the kernel level,
有些 optimizations 是你在 kernel level 做的，
41:27
the optimization are the reducing the number
这些 optimization 就是减少
41:30
of steps the model does the distillation.
model 做 distillation 的 steps 数量。
41:33
And so the going from basically high level
所以，从基本上 high level 的
41:37
to low level hardware and how you survey,
到 low level hardware，以及你怎么去 survey，
41:40
how you batch it, like, are you cache it?
怎么 batch 它，比如，你会不会 cache 它？
41:43
There's like a pile of things that you can really do.
其实有一堆事情你都能做。
41:46
And then if you tackle all of them,
然后如果你把这些全都搞定，
41:48
you're gonna squeeze like significant gains
你就能榨出相当明显的提升，
41:51
out of each step.
从每一步里。
41:53
And that's how we have seen, you know,
而这就是我们看到的，你知道，
41:55
for example, the cost of generation of a second
比如，generation 一秒钟的成本
41:58
of 720p video in literally over a year
生成 720p 视频，真的在一年多里
42:03
go down two orders of magnitude, right?
降了两个数量级，对吧？
42:05
Now you can generate some cent
现在你花几分钱就能生成一些
42:08
where like a year ago it was like a $1 per second
而大概一年前，还差不多是每秒 1 美元
42:12
of generation.
的生成成本。
42:13
And so there's still a lot to do there.
所以这方面还有很多要做。
42:16
I'm not gonna like forecast it.
我不会去预测这个。
42:19
AI inference is gonna be as cheap as storage.
AI inference 会变得像 storage 一样便宜。
42:22
But we have seen, we have seen what happened with storage
但我们确实看到了，我们已经看到 storage 方面发生的情况
42:24
at the beginning, you know, we were paying like by the gigabytes
一开始，你知道的，我们是按 gigabytes 付费的
42:29
and it was so expensive.
而且特别贵。
42:30
And then you had to like do your math.
然后你还得自己算一算。
42:32
And now, like, I don't even know how many terabytes
而现在，就……我甚至都不知道我的 draw board second 上
42:35
I have of storage of my draw board second.
有多少 terabytes 的 storage。
42:37
You know, like, I don't know.
你知道的，就……我也不知道。
42:38
I don't think I would ever have the problem
我觉得我永远都不会有这个问题
42:40
to think about it.
去想想这件事。
42:41
And so I don't think, you know, AI inference will get to that level
所以我不觉得，你知道，AI inference 会达到那个水平
42:45
but it's getting so much better than it was, you know,
但它比过去，你知道，已经好太多了，
42:49
a year ago.
比一年前。
42:50
And I think given the first derivative is definitely, you know,
而且我觉得，考虑到 first derivative 肯定，你知道，
42:54
still very healthy.
仍然非常健康。
42:55
I think we're gonna see probably another one
我觉得我们可能还会看到另一个
42:57
they will mind me to going down.
他们会介意我往下走。
42:59
And when you get to like point one's tent
当你到了比如 0.1 美分就能生成一张图片，或者生成一秒视频的时候，
43:02
for you generate the image or generate the second of video,
我觉得这是另一个突破，
43:05
I think it's another breakthrough
然后突然之间，你知道，普及化和可负担性真的会远远超出我们今天看到的样子。
43:06
and all of a sudden, you know, the democratization
在构建那个 full stack 并运营那个 full stack 的过程中，你们运营的是整个 full stack 吗？
43:09
and affordability go really beyond what we see today.
你们的 data center 里有 hardware 吗？
43:13
In building that full stack and operating that full stack,
43:16
what are you operating the full stack?
43:18
Do you have hardware in a data center
43:20
or do you delegate part of that to the cloud
或者你们把其中一部分交给 cloud 去做？
43:24
but you're running everything on top of cloud providers,
但你们所有东西都跑在 cloud providers 之上，
43:28
like how far down do you go?
就是，你们会往下做到多深？
43:32
Yeah, we are as far down as we can go
对，我们能往下做多深就做多深，
43:36
other than owning the actual hardware ourselves.
除了实际的 hardware 由我们自己拥有这一点之外。
43:40
So yeah, we run those models on our cloud,
所以对，我们在自己的 cloud 上跑那些 models，
43:43
we optimize them, we manage, you know,
我们会优化它们，会管理，你知道，
43:46
how we optimize the inference, how we serve it, all that.
怎么优化 inference、怎么 serve 它、所有这些。
43:49
So the raw thing that you're getting
所以你拿到的最原始的东西
43:51
from someone else's GPU instances
是从别人的 GPU instances 上来的
43:55
and you're clustering them,
再对它们做 clustering，
43:57
you're loading them with whatever your inference stack is,
不管你用的 inference stack 是什么，都加载进去，
44:00
you're tuning LLVM and all the bits and pieces
你调 LLVM，还有各种零零碎碎的东西
44:06
batch sizes and all the stuff you talked about.
batch sizes，以及你刚才聊到的所有这些东西。
44:09
Yeah, exactly.
对，就是这样。
44:11
Talk about how's your knowledge about that space evolve
聊聊你对这个领域的了解是怎么演变的
44:15
where there are things that you found surprising
有没有哪些事情是你觉得意外的
44:18
once you started to work on these new models
在你开始做这些新 models 之后
44:21
or did it kind of roll out as expected?
还是说基本上就按预期展开了？
44:25
Yeah, it's interesting because there was an industry
嗯，挺有意思的，因为当时有一个行业
44:27
that was completely dominated by AI researchers, if you want,
可以说是完全被 AI 研究人员主导的
44:32
where you had like graphics engineers
里面有像 graphics engineers 这样的人
44:37
or like cloud engineers coming in and being like,
或者像 cloud engineers 进来然后说
44:40
oh my God, there's so much more juice to squeeze here.
天哪，这里还有太多潜力可以榨出来
44:44
And so it's been really like a phenomenal to see how,
所以看到这一切真的挺让人惊叹的，
44:48
and here's some Francisco.
然后这里是 Francisco。
44:49
I saw like a whole wave of startups
我看到了一整波 startups，
44:53
actually focusing on inference optimization.
其实都在专注做 inference optimization。
44:56
I mean, because there is so much money
我的意思是，因为现在有太多钱
44:59
left on the table right now.
都还留在桌上没被拿走。
45:01
And so there's been this massive wave.
所以就出现了这么一大波浪潮。
45:03
I think I can probably name like a dozen
我觉得我大概能说出十几个
45:05
or companies that they do that because, you know,
或者有些公司这么做，是因为，你知道，
45:08
people that have a different background realize,
有不同背景的人会意识到，
45:10
okay, these models are really no run efficiently.
好吧，这些 models 真的跑得并不高效。
45:13
We should rethink about how we feed all these data
我们得重新想想，怎么把所有这些 data
45:16
and these jobs into the GPU.
和这些 jobs 喂进 GPU。
45:18
So there's been like really like a flurry of interest
所以最近真的有一波非常密集的关注，
45:22
because it's also a very simple value proposition
也因为它的价值主张解释起来非常简单，
45:25
to explain, you know, this thing costs you now
你知道，就是这东西现在是要花你钱的。
45:27
like a billion dollar.
比如说十亿美元。
45:28
And if I do this optimization,
而如果我做这个 optimization，
45:30
it's gonna cost you your one tenth of that.
那只会让你付出其中十分之一的成本。
45:33
But everybody gets it, it's very easy to explain.
但大家都懂，这很容易解释。
45:35
And it's a very easy ROI to put in place.
而且这个 ROI 很容易落地。
45:38
And we have seen now, as AI adoption
而现在我们也看到，随着 AI 的采用率
45:43
is still skyrocketing,
仍在飙升，
45:44
we are seeing a lot of pressure on reduction of cost,
我们看到降低成本方面有很大的压力，
45:47
right, and make it stillable.
对，然后让它变得 stillable。
45:49
And so it's been like fascinating to see
所以看着这些真的挺有意思的，
45:51
a lot of those startups and also in big companies
很多这类 startups，还有大公司里也是，
45:54
bringing knowledge there is more about, you know,
把知识带过来，更多是关于，你知道的，
45:58
graphics and you know,
graphics，还有你知道的，
46:00
graphics has been optimizing things for graphics cards
graphics 一直在为 graphics cards 做优化，
46:03
for like decades, right?
大概有几十年了，对吧？
46:04
And now that knowledge is brought to combine
而现在这些知识被拿来结合
46:07
with like cloud scalability knowledge and AI models,
再加上 cloud scalability 这类知识，以及 AI models，
46:11
all those three components together
这三个组成部分合在一起
46:13
are really delivering wonders and it's quite, quite exciting.
真的能创造出很惊艳的效果，而且非常非常令人兴奋。
46:16
And are there prerequisites
那么，有没有什么前提条件
46:19
to being at the steep part of the optimization curve,
才能处在 optimization curve 的陡峭阶段，
46:24
meaning do you have to have a certain degree of scale
也就是说，你是不是必须达到一定程度的 scale
46:28
so that you can manipulate batch sizes
才能调整 batch sizes
46:30
or do you have to build your own models
还是说你必须自己 build models
46:34
such that you can take advantage of, you know,
这样你就能利用，你知道，
46:38
deep levels of control,
深层次的控制，
46:39
you know, where do you think it makes sense
你知道，你觉得哪里比较合理
46:41
to like own that full stack?
去拥有整个 full stack？
46:43
Yeah, it was really like an interdisciplinary type
是啊，那真的像是一种跨学科式的
46:47
of endeavor where we actually put together
尝试，我们实际上把
46:51
different teams that had different skillsets, right?
拥有不同技能组合的不同团队凑到了一起，对吧？
46:54
And when we saw the combination of those come together,
而当我们看到这些组合汇聚在一起时，
46:57
that's where like the best games came.
最好的游戏就是从那儿来的。
47:00
And yeah, absolutely, like if you have to host,
而且对，绝对是这样，就像如果你必须 host，
47:04
you know, you have to self-host your model
你知道，你得 self-host 你的 model
47:06
and then you have to have access to the weights of the model.
然后你还得能拿到这个 model 的 weights。
47:10
You know, if you have, you know,
你知道，如果你有，你知道，
47:12
if you're using like an open weights model,
如果你用的是那种 open weights model，
47:14
that's, you know, that's good.
那，你知道，那挺好的。
47:15
But in some cases, you don't have access to that.
但有些情况下，你拿不到那个。
47:18
So in that case, you build your own model.
所以那样的话，你就自己搭一个 model。
47:20
But when you self-host and when you have access to the weights,
但当你 self-host，而且能拿到 weights 的时候，
47:23
there's a lot more that you can do
你能做的事情就多得多，
47:25
and allow more juicy, you can squeeze out of those models.
也能让你从那些 model 里榨出更多有料的东西。
47:29
And again, we found that by bringing together like a very
而且，我们发现，通过把一群非常
47:33
interdisciplinary team with different backgrounds,
背景不同的跨学科团队聚在一起，
47:37
we were able to really tackle the problem with the right,
我们就能真正用对的、
47:40
you know, with a different perspective.
你知道，用不同的视角去解决问题。
47:42
And again, it goes back to diversity, right?
再说，这又回到 diversity 了，对吧？
47:45
In this case, it was a very, a very diverse mindset
在这个案例里，那是一种非常、非常多元的思维模式
47:48
from the eye researcher that we brought in.
来自我们请来的那位 eye researcher。
47:51
And I think it really delivered.
而且我觉得它真的奏效了。
47:54
You're going back to craft and that kind of two-part model
你又回到 craft 和那种 two-part model 了
47:59
does having a unique model like that
拥有那样一个独特的 model
48:03
get in the way of some types of optimizations like distillation?
会妨碍像 distillation 这样的某些 optimizations 吗？
48:08
No, distillation is something that we do really regularly
不会，distillation 是我们经常做的东西。
48:12
on every model that we develop.
在我们开发的每一个 model 上。
48:14
And so for craft specifically, we have announced that
所以具体到 craft，我们已经宣布
48:18
this is going to be a quick follow on craft fast and craft
这将会是 craft fast 和 craft 的快速跟进，
48:22
ultra fast.
ultra fast。
48:24
And those are basically models where we are doing the
基本上，这些就是那些 model，我们在其中做
48:26
also distillation among other many other optimizations.
也包括 distillation，以及其他很多 optimizations。
48:31
And again, by having all the modern in-house,
另外，通过把所有现代化能力都放在内部，
48:33
we can also make sure that the distillation that we do
我们也能确保我们做的 distillation
48:37
on the, you know, on the lens side and the one that we do
在，你知道，在 lens 这一侧，还有我们做的
48:40
on the diffusion model side are also like tuned to work
在 diffusion model 这一侧的，也都是调校到能
48:43
optimally together.
最优地配合工作。
48:44
So again, it goes back to the concept of like making sure
所以还是那句话，这就回到了那个概念，就是要确保
48:48
that, you know, the input and output are really tuned
你知道，输入和输出真的都调校到
48:50
for the best quality.
最佳质量。
48:52
And so we're going to have more distilled versions of craft
所以我们会推出更多 distilled 版本的 craft
48:57
coming out in the next few weeks because again,
接下来几周就会出来，因为还是那句话，
49:01
we want to make it even more affordable.
我们想让它更实惠。
49:03
And we want also to leave the choice to the user
而且我们也想把选择权留给用户
49:07
whether it is a developer using an API or, you know,
不管是使用 API 的开发者，还是，你知道，
49:11
a designer actually using it to the UI or to MCP.
还是实际把它用到 UI 或 MCP 上的设计师。
49:16
We launch it.
我们发布了它。
49:18
I think now it's also available on MCP as of yesterday craft.
我觉得现在从昨天起，它也能在 MCP 上用了，craft。
49:22
And that's another place where like, you know,
那也是另一个地方，就像，你知道的，
49:24
if I'm building a prototype app, I don't want to run
如果我在做一个原型 app，我不想运行
49:27
on the most expensive model, of course.
当然，是在最贵的 model 上。
49:29
And so the ultra fast and faster version of the model
所以这个 model 的超快版和更快版
49:34
are also designed for that.
也都是为此设计的。
49:36
They may become, you know, kind of the default.
它们可能会变成，你知道，某种 default。
49:39
But then when you go, when you want quality
但然后当你，当你想要质量的时候
49:40
then you switch to the higher fidelity model.
你就会切换到更高 fidelity 的 model。
49:44
Sorry.
抱歉。
49:44
And so the MCP is a canva MCP or craft MCP.
所以这个 MCP 是 canva MCP 还是 craft MCP。
49:50
Through the canva MCP, you will be able to also access craft.
通过 canva MCP，你也能访问 craft。
49:54
Okay.
好。
49:55
There's basically a bunch of like AI capabilities
基本上就是有一大堆 AI 能力
49:58
that we are exposing to MCP and craft is being added.
我们正在把这些开放给 MCP，而且 craft 也正在被加进来。
50:03
And then as we add more variants of craft,
然后随着我们加入更多 craft 的变体，
50:07
we're going to keep adding to MCP, which by the way,
我们会继续往 MCP 里加，顺便说一下，
50:10
we are seeing like the users of MCP on canva
我们看到 canva 上 MCP 的用户
50:14
like a really styrofoch it.
好像真的对它很兴奋。
50:16
I think people are finding a lot of value.
我觉得人们从中发现了很大的价值。
50:18
So we want to give more capabilities, you know,
所以我们想给它更多能力，你知道，
50:21
access to more coverage to MCP going forward.
未来让 MCP 覆盖更多、访问更多。
50:24
Do I understand correctly that beyond the MCP access to craft,
我理解得对吗：除了通过 MCP 访问 craft 之外，
50:30
you are also making craft and some of the other models
你们也让 craft 和其他一些模型
50:34
directly available to folks kind of, I don't know
直接提供给用户，类似这样，我不太确定
50:38
the specifics, but I'm envisioning like via an open router
具体细节，但我设想的是比如通过 open router
50:41
or via canva directly, but just per use access to the model.
或者直接通过 canva，但只是按使用量访问模型。
50:46
Yeah, absolutely.
对，完全没错。
50:46
So we released it through our API called Canva Connect.
我们是通过一个叫 Canva Connect 的 API 发布它的。
50:52
And so now it's available.
所以现在它已经可用了。
50:54
You can access through the API as well as MCP.
你可以通过 API 以及 MCP 来访问。
50:58
So for the developers that want to do that.
所以对那些想这么做的开发者来说。
51:00
That feels like a very different business like running,
那感觉像是很不一样的业务，就像运营，
51:03
you know, super low latency, very narrow,
你知道，超低延迟、范围非常窄的，
51:06
you know, model inference service as opposed to,
你知道，model inference 服务，而不是，
51:10
you know, this is an API for our product.
你知道，这是我们产品的一个 API。
51:12
Is this the first, the first model that you're hosting
这是你们以这种方式 hosting 的第一个模型吗？
51:16
in this way?
我们已经开放了各种能力的 API。
51:17
We have exposed the APIs for various capabilities.
我觉得你也可以访问 magic layers，还有一些我们之前就有的 image generation models，比如 lucid origin。
51:20
I think also you can access magic layers
但我觉得现在大家表现出的兴趣程度
51:22
and some of the image generation models that we had before,
51:27
like lucid origin.
51:28
But I think now the level of interest that we are getting
51:31
is a lot higher.
这要高得多。
51:32
And yes, you're right.
是的，你说得对。
51:34
It is a type of different business if you want.
如果你愿意这么说，它算是另一种业务。
51:37
But we are getting so much demand coming from mostly like
但我们收到的需求非常多，主要来自
51:41
two categories.
两类。
51:42
So enterprise customer, our enterprise business
所以 enterprise customer，我们的 enterprise business
51:45
is the fastest growing business that we have in the company.
是公司里增长最快的业务。
51:50
Really, you know, triple digit growth year year.
真的，你知道，同比三位数增长。
51:54
And we are getting a lot of demand also from those enterprise
而且很多企业客户也在跟我们提需求，想通过 API 访问这些能力，因为他们可能会搭建自己的 custom workflow。
51:57
customers to access the capabilities to, you know, an API
现在人们可以 wide code 任何东西了，对吧？
52:02
because they may build their own custom workflow.
这块的需求会越来越多。
52:05
Now the people can wide code anything, right?
然后我们也看到很多需求，来自那些通过 MCP 使用，呃，怎么说，cloud 或者 charge GPT 的人。
52:08
There's going to be more demand for that.
他们现在想访问更多能力，对吧？
52:10
And then we are getting also a lot of demand for people
52:12
using, I don't know, cloud or charge GPT through MCP.
52:16
They now want to access more capabilities, right?
52:18
So the agents can go there and say, oh, you want to generate
所以 agents 可以去那里说，哦，你想生成
52:21
an image.
一张 image。
52:22
I can use Kamba and then they can always then open
我可以用 Kamba，然后他们总是可以再打开
52:25
in Kamba for further editing.
在 Kamba 里做进一步编辑。
52:27
And so we are seeing these two type of use cases
所以我们看到这两类 use case
52:31
really growing.
真的在增长。
52:32
And sometimes those users are already using Kamba and
而且有时候这些用户已经在用 Kamba，而
52:35
they just want to use it in a different way or they just want
他们只是想以不同的方式用它，或者他们只是想
52:38
to build, you know, scalable applications using some of
来构建，你知道的，scalable 应用，使用一些
52:42
the Kamba technology.
Kamba technology。
52:43
And so we have seen a lot of demand.
所以我们看到了很大的需求。
52:45
And yes, it's a new and exciting business.
而且是的，这是一项新的、令人兴奋的业务。
52:48
And we are still learning a lot about how to best serve it.
而且我们还在大量学习如何最好地服务它。
52:52
But I think this week and say, it's not you need to Kamba.
但我觉得这周可以说，并不是你需要 Kamba。
52:55
It's across the industry, right?
这是整个行业的情况，对吧？
52:56
Like there's a lot of companies that all of a sudden like, oh,
就像有很多公司突然之间就，哦，
53:00
we are basically building not only for humans, but also for
我们基本上不只是为人类构建，也是在为
53:03
agents, right?
agents 构建，对吧？
53:04
Even though, you know, humans is our focus, even for AI features,
虽然说，你知道，人类是我们的重点，即使是在 AI 功能上也是，
53:08
but there's going to be in the future, just more agents using
但未来会有更多 agents 使用
53:12
some of those capabilities.
其中一些能力。
53:13
I think that's unavoidable.
我觉得这是不可避免的。
53:15
What do you think those agents need that's different from
你觉得那些 agents 需要的，和
53:19
what humans need?
人类需要的东西有什么不同？
53:20
I think they need more structure.
我觉得他们需要更多 structure。
53:23
They need documentation, you know, in a certain, the final
他们需要 documentation，你知道，得用某种最终的、特定的方式，这样他们才能消化更多东西。
53:28
certain way so they can digest more things.
他们需要能在不订阅的情况下预览内容的能力。
53:30
They need the ability to try to preview things without having
所以很多东西本质上都是为了减少 friction、更方便地找到信息、让我们的工作流程更顺畅，因为最终，很多时候我们就是那个 agent。
53:36
to subscribe.
53:37
And so a lot of things that basically are designed to remove
53:40
friction, finding information, more easily streamlined work
53:44
for us, because ultimately, many times we will be the agent
53:47
that will decide what solution to use.
那会决定用哪个 solution。
53:50
And it's supposed to be the human.
而做这个决定的应该是人。
53:52
And so we have to basically make sure, for example,
所以基本上我们得确保，比如说，
53:55
the agent is able to like spin up with low friction and
agent 能很顺畅地 spin up，而且
53:58
know a lot of burden on like initial credentials or credit
不用在 initial credentials 或 credit 上
54:01
or purchase.
或者 purchase 上承担太多负担。
54:02
And it can spin up something that the agent can test,
而且它能 spin up 一些东西，让 agent 可以测试，
54:05
verify it's working.
验证它是不是在正常工作。
54:07
And then, you know, go back to the human be like, hey, I created
然后，你知道，回去跟那个人说，嘿，我做了
54:10
this like a prototype up using this API.
一个用这个 API 弄的 prototype 出来。
54:13
I tested it works.
我测过了，能用。
54:15
You are not giving a shot.
你都不试一下。
54:16
Okay.
好。
54:17
Now it's approved.
现在批了。
54:18
And now we can go for the real scalable thing.
现在我们可以去做真正 scalable 的东西了。
54:22
So that's the kind of approach.
所以大概就是这种做法。
54:23
Just kind of wrap things up.
就差不多收个尾吧。
54:25
Where do you see Canva evolving with, you know,
你觉得 Canva 接下来会怎么演进，那个，嗯，
54:29
in terms of research in particular?
特别是在研究方面？
54:31
It's a word where like, you know, typically you play
在这个领域里，你知道，通常你会
54:34
in like a year ahead, you know, it's completely false.
提前一年去布局，你知道，那完全不对。
54:38
It's like, it's like just a month ahead.
其实，其实也就提前一个月。
54:41
It's already like a 3D, you know, pretty foggy kind of thing.
那已经像是 3D 的，你知道，挺模糊的那种东西了。
54:46
But at the high level, you know, we have a very clear
但站在高层来看，你知道，我们有一个非常清晰的
54:51
mission and vision.
使命和愿景。
54:52
That one is not changing.
这一点不会变。
54:54
So we want to make AI more accessible.
所以我们想让 AI 更触手可及。
54:57
We want to make it more part of the iterative creative
我们想让它更多地成为 iterative creative process 的一部分。
55:01
process.
我们想消除任何障碍。
55:02
And we want to remove any barrier.
我们想重新构想所有那些，那些为之工作的东西。
55:05
And we want to reimagine all those, those work for.
所以这就是我们一直在追求的方向。
55:08
So that's the direction that we've been pursuing.
55:11
And what we are actually doing, what models we work on,
而我们实际在做的事，以及我们在做哪些 models，
55:14
specifically a Canva.
具体来说，是在 Canva。
55:15
Yeah.
对。
55:16
This is going to change, you know, every, every few months.
你知道，这每隔几个月就会变，每隔几个月。
55:20
And so I, you know, I told my team, basically,
所以我就，你知道，我基本上跟我的团队说，
55:24
don't fight, you know, this ever changing,
别去对抗，你知道，这个一直在变的、
55:28
constantly changing ecosystem, but just like embrace it.
不断变化的生态，而是要去拥抱它。
55:31
And so I think it's really like a mind shift of like,
所以我觉得这真的像是一种思维转变，就像，
55:34
everything is going to change, which mean also like,
一切都会改变，这也就意味着，像是，
55:37
what is important today may not be important tomorrow,
今天重要的东西，明天可能就不重要了，
55:40
which opens up also for opportunities, right?
这也同时打开了机会，对吧？
55:43
Like, it's a double edge sword to some extent,
就像，某种程度上这是一把双刃剑，
55:48
but at the same time, we see that it's a big opportunity.
但与此同时，我们看到这是一个巨大的机会。
55:52
And so we are constantly, you know, analyzing our business,
所以我们一直在，你知道的，分析我们的业务，
55:56
see where there are opportunities, where there's bottlenecks,
看看哪里有 机会，哪里有瓶颈，
55:59
where there are technical part of a workflow that we can automate.
哪里是 workflow 里我们可以自动化的技术部分。
56:02
And then we prioritize those.
然后我们就会优先处理那些。
56:04
And when we, and we work on that, like a year ago,
而大概一年前，我们做这件事的时候，
56:07
none of the projects, the models that we ship now was underway.
我们现在发布的这些项目、这些 models，当时一个都还没启动。
56:12
You know, so it's logic to think that in one year,
你知道，所以按逻辑想，一年之后，
56:15
we will probably release models that we haven't even started working on it today.
我们很可能会发布一些今天甚至还没开始做的 models。
56:20
And as, you know, it could be terrifying or could be very exciting.
而且你知道，这可能很吓人，也可能非常令人兴奋。
56:24
So I think, I think researchers are probably the most exposed to the speed
所以我觉得，我觉得研究人员大概是最能直接感受到
56:28
of which AI is changing.
AI 变化速度的那批人。
56:30
And so it's, yeah, it takes really like some major adaptation.
所以，是啊，这真的需要一些相当大的适应。
56:34
And and mindset shift by the same time is quite exciting.
而且同时，这种思维方式的转变也挺让人兴奋的。
56:38
Well, Seth, thanks so much for taking the time to share a bit about what your new team
嗯，Seth，非常感谢你抽时间分享了一些关于你们新团队
56:42
are up to. It's very cool to hear more about it.
正在做的事情。能多了解一些真的很酷。
56:46
Thank you, Sam.
谢谢你，Sam。
56:46
It was a true pleasure and honor to be on your podcast.
能上你的播客真是我的荣幸。
56:49
Thank you so much.
非常感谢。
56:50
Thank you.
谢谢。