Latent Space
Building Legal Agents When You Can't See the Data - Jacob Lauritzen, CTO of Legora
00:0001:07:21
00:00
People were asking us about Mithos and everything, even though I think really sickly a lot of legal work does not require Mithos level intelligence.
大家一直在问我们 Mithos 什么的，虽然我真的觉得很多法律工作并不需要 Mithos 级别的智能。
00:06
Parallegals gone, junior lawyers gone, I don't need them.
律师助理没了，初级律师没了，我不需要他们。
00:10
That's actually not what we're seeing, and we had one problem where it wouldn't cite properly,
但实际上我们看到的不是这样，我们遇到过一个它没法正确引用的问题，
00:14
and so we'd be like, next maldery solved, it went from like 70% to like 99%.
然后我们就会说，下一个 maldery 解决了，它从大概 70% 变成了大概 99%。
00:18
She's like, we did nothing, and like, great.
她说，我们什么都没做，然后就像，太好了。
00:20
But if you're doing a $400 million M&A transaction, you're going to want to look over.
但如果你在做一笔 4 亿美元的 M&A 交易，你肯定会想仔细过一遍。
00:24
We're just going to be 20 engineers for the rest of the company's history, and like, we're going to beat everyone as much bigger.
我们公司以后整个历史里都只会是 20 个工程师，然后还要打败所有规模大得多的公司。
00:29
We're 100 engineers now, and it feels like we're way too small.
我们现在有 100 个工程师，但感觉我们还是太小了。
00:32
But it's something that I'm talking with is like, how much of our product can just be the agent?
但我一直在跟人聊的一点是，我们的产品里有多少可以直接就是 agent？
00:35
We're sold the other stuff.
我们卖的是其他那些东西。
00:37
Okay, we're here in a studio with special guests, Jacob Lawison, CTO of Legora.
好，我们现在在演播室，和特别嘉宾 Legora 的 CTO Jacob Lawison 在一起。
00:43
Did I get a rate, Lawison?
我没念错吧，Lawison？
00:45
That's fine, yeah, that's okay.
没事，对，没关系。
00:46
I'm not surprised to hear that out. It's Danish, yeah, exactly.
听你这么说我不意外。是丹麦的，对，没错。
00:49
Does it have a special history?
它有什么特别的来历吗？
00:50
It just, no, Sen is like Sun, Sunoff.
就是，不，Sen 就像 Sun，Sunoff。
00:52
Sunoff Lawyers?
Sunoff Lawyers？
00:53
You have Sun, and yeah, Sunoff Lawyers, that's my name.
你叫 Sun，对，Sunoff Lawyers，这就是我的名字。
00:56
Nice.
不错。
00:57
We've crossed paths here and there.
我们时不时会碰上。
00:59
You spoke at AI a few times, and obviously, Legora is one of the fastest-going legal tech
你在 AI 活动上讲过几次，而且很明显，Legora 是史上发展最快的 legal tech 之一；你在节目开始前说的一点我觉得特别有意思。
01:04
in history, and you said something at the start in a pre-show that I thought was really interesting.
你说你的根源非常欧洲，尽管很明显你们正在 New York 和 SF 扩张。
01:09
You said your origin is very European, even though obviously you're expanding in New York and SF.
你们的文化感觉有什么不一样？
01:14
What feels different about your culture?
你们的文化哪里让你感觉不一样？
01:17
I think we are just at the core, probably, very humble about what we're doing,
我觉得我们其实还处在核心位置，可能对自己在做的事非常谦逊，
01:22
because we've always been the underdog from the very beginning.
因为我们从一开始就一直是 underdog。
01:25
No one believed the smallest startup in a basement in Stockholm was going to outcompede
没人相信斯德哥尔摩地下室里最小的 startup 能 outcompete
01:29
the huge beam off. There was way ahead of us in the US.
那个巨大的 beam off。在美国，他们远远领先于我们。
01:32
So I think we've always been underdog, since we've always been very humble.
所以我觉得我们一直是 underdog，因为我们一直都非常谦逊。
01:36
We've always had to fight for our existence.
我们一直都得为自己的生存而战。
01:40
I think it's something we've had to learn as we've also become more and more
我觉得这是我们不得不学会的东西，随着我们也变得越来越
01:44
as we've entered the US and been in the US a lot more,
随着我们进入美国、在美国待得越来越多，
01:46
as to be a little bit more bold, and it's to tell the world how great we are,
就是稍微再大胆一点，然后告诉全世界我们有多厉害，
01:51
because it doesn't come naturally, too, I think some of us.
因为我觉得这对我们中的一些人来说也不是天生的。
01:54
I want to start by going over your guys' product,
我想先聊聊你们的产品，
01:57
because I feel like with a lot of these legal products,
因为我觉得很多这类法律产品，
02:00
they're under wraps, you're not supposed to show it.
都是藏着掖着的，不能随便给人看。
02:03
But yeah, if you could just talk about what does the product look like,
但话说回来，你能不能就讲讲这个产品长什么样，
02:06
and maybe how did you guys get there?
还有你们是怎么走到今天的？
02:08
What have been the product iterations that got you to where you are today?
产品经历了哪些迭代，才让你们走到今天？
02:12
I think that would be really interesting.
我觉得那会非常有意思。
02:13
Absolutely. The way everything started was chat GVT with Swedish loss.
当然。一切的开端就是 chat GVT with Swedish loss。
02:18
It's such a tool in it that was literally the entire thing.
它里面有这么一个工具，那真的就是全部。
02:22
That's where we started with legal research, which is a huge market,
我们就是从 legal research 开始的，那是一个巨大的市场，
02:25
basically, in the world, where lawyer spend a lot of time figuring out
基本上，在全世界都是，律师会花大量时间搞清楚
02:29
what are previous cases that look like this, what's the legal precedent?
有哪些以前的案例看起来像这样，legal precedent 是什么？
02:32
Is this what have previous people done in these situations?
以前的人在这些情况下是不是这么做的？
02:35
That's extremely important.
这极其重要。
02:37
That has always been part of our product.
这一直是我们产品的一部分。
02:39
Basically, we started from an agent, you could ask questions,
基本上，我们是从一个 agent 开始的，你可以提问，
02:42
and it would query loss.
然后它会去查询 loss。
02:44
That was a very simple beginning.
那是一个非常简单的开始。
02:46
Then from that, it's expanded.
然后从那开始，它扩展了。
02:47
It was expanded in what can the agent do?
它是在 agent 能做什么这方面扩展的？
02:50
It will now draft a lot of stuff really well.
它现在能非常好地起草很多内容。
02:53
It will edit word files.
它会编辑 Word 文件。
02:54
It will move your files away, organize everything, run really complex analyses,
它会帮你把文件移走，把所有东西整理好，跑非常复杂的分析，
03:01
and we've added just a bunch of more features to it.
而且我们又给它加了一大堆功能。
03:03
One of our features to call a tabular view, which is now a name that's being used in industry,
我们有个功能叫 tabular view，这现在已经是业内常用的一个名字，
03:08
and we're very happy to see Claude for legal use in our terminology.
我们也很高兴看到 Claude 用于法律用途，用我们的术语来说。
03:14
Which is actually a way to just bulk extract and bulk review many, many agreements,
这其实是一种批量提取和批量审阅大量协议的方式，
03:19
which you do in due diligence and when you raise money or whatever.
也就是你在 due diligence 和融资之类的时候会做的事。
03:21
That's part of it.
这是其中的一部分。
03:23
We help lawyers just do their work much more efficiently, much better.
我们帮助律师更高效、更好地完成工作。
03:27
Outside of just doing the work, we're also now helping orchestrate the work a lot more.
除了做具体的工作之外，我们现在也更多地在帮助统筹整个工作流程。
03:33
Then we were just even a year ago.
比一年前可多多了。
03:37
We have a thing called lists, which is sort of linear for us.
我们有个叫 lists 的东西，对我们来说算是比较线性的。
03:42
We have intake, which is a way to collaborate with clients.
我们有 intake，这是一种和客户协作的方式。
03:46
And we have portal, which is also sort of a way to collaborate.
还有 portal，这也算是一种协作方式。
03:49
You know, lawyer to lawyer, client to lawyer to agent, agent to agent.
你知道的，律师对律师、客户对律师对 agent、agent 对 agent。
03:54
Maybe we're spending a lot of time there because there's just frankly
也许我们之所以在那上面花很多时间，坦白说，是因为你 10 年前跟律师打交道的方式，甚至可能到今天，都真的很痛苦。
03:59
the way that you work with a lawyer 10 years ago or maybe even today is really painful
十年前，甚至可能到今天，你跟律师合作的方式都真的很痛苦
04:04
and there's loads of room to do that.
而且这方面还有很大的空间。
04:06
And a much better way to be honest.
而且说实话，这是个好得多的方式。
04:08
And so those are like, I think the two things.
所以我觉得，大概就是这两件事。
04:09
It's doing the work and it's also like orchestrating the work.
它既是在做具体工作，也像是在 orchestrating 这些工作。
04:12
When did you guys start building the agent?
你们是什么时候开始做 agent 的？
04:15
So I mean, we basically had this switch from the assistant to the agent.
所以我的意思是，我们基本上经历了从 assistant 到 agent 的这个转变。
04:18
Yeah, we call it right.
对，我们就是这么叫的，对吧。
04:20
I guess that the agent, I don't know, I want to say a year ago probably.
我猜，agent 的话，我不知道，我大概想说是一年前吧。
04:25
Yeah, so this is a rough timeline for people who are understanding YC 2023.
对，所以这是给正在了解 YC 2023 的人的一个大致时间线。
04:30
Yes.
是的。
04:30
And then you were YC founder as well.
然后你也是 YC 创始人。
04:32
YC-24.
YC-24。
04:33
YC-24.
YC-24。
04:33
Yeah, I was in YC-23.
对，我当时在 YC-23。
04:35
The Gora was 24.
The Gora 是 24。
04:36
Okay.
好的。
04:36
And then, so there is, this is like a little bit pre-agents really taking it.
然后，所以现在有点像 pre-agents 真的开始接手了，哪怕还只是一点点。
04:41
Exactly.
没错。
04:41
And then like using models obviously change everything.
然后，显然用 models 会改变一切。
04:43
I mean, in one sense today, we had a preview of the products.
我是说，从某种意义上说，今天我们预览了一下产品。
04:49
It does look, well, preview.
看起来确实，嗯，是预览。
04:51
We had a private thing.
我们搞了个私下的活动。
04:52
But we will put up a video on the YouTube.
但我们会在 YouTube 上放个视频。
04:54
It's mostly like ChatGPT for lawyers, right?
它主要就像是给律师用的 ChatGPT，对吧？
04:56
Like a very fair to say that.
这么说挺公道的。
04:58
But then the sidebar has a lot of nice custom tools like the tabular view,
但 sidebar 里有很多不错的自定义工具，比如 tabular view，
05:02
like the intake, which I really liked.
比如 intake，我真的很喜欢。
05:05
And much other things.
还有很多其他东西。
05:07
Exactly.
没错。
05:08
That's exactly what it is.
它就是这么回事。
05:09
It's co-work or ChatGPT.
它是 co-work 或者 ChatGPT。
05:12
Just like a really, really good for legal work.
就像是一个真的、真的非常适合法律工作的工具。
05:13
And yeah, the workflow for either in-house corporate lawyers,
而且对，这个 workflow 不论是给公司内部法务，
05:17
like people looking for upcoming regulations,
比如那些在关注即将出台法规的人，
05:19
not all these things.
不只是这些东西。
05:20
And we can talk about more of the products there.
然后我们也可以多聊聊那边更多的产品。
05:23
Yeah, what models are you guys using underneath?
对，你们底层用的是什么 models？
05:25
Like, can you talk about that?
就是，能聊聊这个吗？
05:27
I mean, we're using a bunch of different ones,
我是说，我们用了好几种不同的，
05:29
depending on tasks, right?
取决于任务，对吧？
05:30
And that's not available to the lawyer to see.
而且那个律师是看不到的。
05:34
You guys for the agents?
你们是支持 agents 的吗？
05:36
Yeah, well, we've been back and forth on this.
是啊，嗯，我们在这个事上一直来回讨论。
05:38
Yeah, it's just like everyone else, I think.
是啊，我觉得就跟其他人一样。
05:40
In the agent, there's a model selector for the primary model.
在 agent 里，有一个给 primary model 用的 model selector。
05:43
But there are certain tools that we use different models in.
但有些 tools，我们会在里面用不同的 models。
05:46
And there's like tabular view.
然后还有像 tabular view 这样的东西。
05:47
And like, we try.
然后吧，我们会试试。
05:49
I think we found it most effective to subdivide the problem
我觉得我们发现最有效的做法是把问题细分
05:51
and route on like, as we subdivide the task.
然后随着我们把任务细分，像那样去做 route。
05:55
But we do right now have a picker in the agent.
但现在我们的 agent 里确实有一个 picker。
05:59
I'm not sure if that's going to stay there always, to be honest.
老实说，我不确定它会不会一直留在那里。
06:01
Ah, so yeah.
啊，所以，嗯。
06:03
Yeah, I mean, so I bring a little bit of context to this
对，我是说，我在这里补充一点背景
06:05
because I'm an advisor for cognition.
因为我是 cognition 的 advisor。
06:07
Yeah.
对。
06:08
And when I joined, you know, part of my thesis for what an agent lab is,
而且我加入的时候，你知道，我关于 agent lab 是什么的论点里，有一部分是，
06:12
an agent lab is, you know, ideally a domain-specific GPT wrapper.
一个 agent lab，你知道，理想情况下就是一个 domain-specific 的 GPT wrapper。
06:17
Yeah, right?
对吧？
06:17
Like, we run the, we are the domain experts.
就像，我们负责运营这个，我们是领域专家。
06:21
We will pay attention to you.
我们会认真关注你。
06:22
We'll serve your needs way, way more than the frontier labs.
我们会满足你的需求，远远超过那些 frontier labs。
06:24
And we'll give you the best of the frontier labs,
而且我们会把 frontier labs 最好的东西给你，
06:26
to put away our job.
来把我们的工作收起来。
06:27
So our job is to decide for you because your job is not to...
所以我们的工作就是替你做决定，因为你的工作不是去……
06:31
Exactly.
没错。
06:32
We'll always keep you on the Pareto frontier.
我们会始终让你处在 Pareto frontier 上。
06:33
Yes.
是的。
06:34
And make sure that it's just great all the time
并且确保它一直都很棒
06:35
and you don't have to dive into all the details.
而你不用深入所有细节。
06:37
Because like, one of the things that I've exactly to this point,
因为，就像，有一件事正好说到这个点上了，
06:39
if you go to a lawyer and you're asked them,
如果你去找律师，然后你问他们，
06:41
oh, you're going to, you know, draft this contract based on this precedent.
哦，你要，你知道，根据这个先例起草这份合同。
06:44
Do you think that's a sonnet higher or an opus mid task?
你觉得那是个 sonnet higher 还是 opus mid 的任务？
06:48
Like, you know, they won't know.
就，你知道，他们不会知道的。
06:49
Just give me this part as well.
就把这部分也给我吧。
06:50
Exactly.
没错。
06:51
Just like, make sure it works.
就是，确保它能正常工作。
06:52
Orally rate is $1,000 an hour.
Orally rate 是每小时 1,000 美元。
06:53
Exactly.
没错。
06:54
Just like, what make it work?
就像，到底是什么让它奏效的？
06:57
So, yes, I originally came in with that.
所以，对，我一开始就是带着这个想法来的。
06:58
I was like, oh, yeah, this is what Devon does differently.
我当时就想，哦对，这就是 Devon 做得不一样的地方。
07:01
And now we have five things in a model picker,
而现在我们在 Astra 的 model picker 里有五个选项，包括 Fable 和 Opus。
07:02
including Fable and Opus, in Astra.
而且大家都想试试那个花哨的新 model。
07:05
And people want to experiment with the fancy new model.
你现在还是这么想吗？
07:10
Do you still have the same?
我觉得是。
07:12
I think so.
我觉得是这样。
07:12
I think it's probably, it's definitely way less
我觉得这大概——不，肯定在 goal 上比在 coding engineering 上少得多。
07:14
in the goal than it is in coding engineering.
嗯。
07:16
Yeah.
但肯定有些人就是想马上要 Fable 1。
07:17
But there are definitely people that I just want
而且你知道，大家一直在问我们 mythos 之类的东西。
07:19
Fable 1 right away.
尽管我觉得，现实地说，很多法律工作并不需要 mythos 级别的智能。
07:20
And, you know, people were asking us about mythos and everything.
而且你知道，大家一直在问我们 mythos 以及各种相关的东西。
07:24
Even though, I think, realistically, a lot of
不过我觉得，现实一点说，很多
07:26
legal work does not require mythos level intelligence.
法律工作并不需要 mythos 级别的智能。
07:29
Yeah, so exactly, right?
对，所以就是这样，对吧？
07:31
Like, I think basically, you're like, okay, we'll give you the option.
就，我觉得基本上，你会说，好吧，我们给你这个选项。
07:34
You ask for this.
这是你自己要求的。
07:35
Exactly.
没错。
07:35
But you're going to run down your credits like 10x faster.
但你的 credits 会消耗得快 10 倍。
07:39
Yeah, I would have loved.
是啊，我本来会很喜欢的。
07:40
We fought introducing it for a very long time.
我们抵制引入它很久了。
07:42
And then in the end, we just kind of caved in and we're like,
然后到最后，我们就有点妥协了，我们说，
07:44
okay, if you want this level of control,
好，如果你想要这种程度的控制，
07:46
you know, then we'll give it to you.
你知道的，那我们就给你。
07:48
Yeah.
嗯。
07:48
So you think that models are already, like,
所以你觉得 models 已经，像是，
07:50
as good as lawyers need.
达到律师需要的水平了。
07:52
Like, there is no need for them to get better.
就是，它们没必要变得更好了。
07:53
It's a, I think it's a part of distribution.
这是，我觉得这是 distribution 的一部分。
07:55
So like, there's probably like six to a percent of the work.
所以，大概有百分之六到百分之一的工作吧。
07:58
Yeah.
嗯。
07:58
We can do really well right now.
我们现在可以做得非常好。
08:00
And the, you know, from now on, it's like a cost and latency thing.
而且，你知道，从现在开始，这就像是个 cost 和 latency 的问题。
08:04
But then there's the rest of the thing where,
但然后还有剩下的部分，就是，
08:07
especially for like very complex analysis,
尤其是对于非常复杂的分析来说，
08:09
we still need more intelligence.
我们仍然需要更强的智能。
08:11
And then I think there's also just the work expands kind of.
然后我觉得，还有一点就是，工作本身也会有点扩展。
08:14
It's like, that's the work that we have right now.
就像，这就是我们现在手头的工作。
08:15
But also the work of what a lawyer does
但律师做的事情
08:17
probably expands with intelligence.
可能也会随着智能提升而扩展。
08:20
Yeah.
是啊。
08:20
I imagine.
我想也是。
08:20
Early on, like, when you guys first started,
早期的时候，呃，你们刚开始那会儿，
08:23
I guess transitioning to the agents or building the agents,
我猜是在转向 agents 或者构建 agents 的时候，
08:26
like, what were, like, what models were you guys using?
就是，你们当时用的是，呃，哪些 models？
08:29
I guess it would have been like Opus 4.6 or something.
我猜大概是 Opus 4.6 之类的吧。
08:31
That would have been, yeah, Sonnet 4.6.
那应该是，对，Sonnet 4.6。
08:34
Opus 4.6 is probably right.
Opus 4.6 应该没错。
08:37
What were the agents, like, not good at that point?
那时候 agents 都，呃，不擅长什么？
08:40
That they're much better right now, even though it's only a little like nine months later?
你是说它们现在好多了，哪怕才过了大概九个月？
08:44
Sub-agent delegation was one of them
Sub-agent delegation 是其中一个
08:47
that we've gotten working really well now
我们现在已经让它跑得很好了
08:48
for like fan-out type retrieval queries.
比如用于 fan-out 类型的 retrieval queries。
08:50
We're just extremely well for us now.
现在对我们来说就是极其好。
08:52
That was difficult before.
这之前挺难的。
08:54
We basically track.
我们基本上会追踪。
08:55
So we have a team that's just doing e-valles.
所以我们有一个团队，专门在做 evals。
08:57
And so we track all of the models performance
然后我们会追踪所有 models 的表现
08:59
in different practice areas, which is like subgroups of legal
在不同 practice areas 里，也就是 legal 的不同子分组
09:02
and different tasks across all of those.
以及这些领域里的各种不同任务。
09:03
So we basically have like a big cheek map.
所以我们基本上有一张很大的 heat map。
09:05
And we're like,
然后我们就，
09:06
and then we can see them move.
然后我们就能看到它们移动。
09:07
And it's like, usually they improve most stuff.
而且一般来说，它们会改进大部分东西。
09:11
But there's actually a lot of model releases
但实际上有很多模型发布
09:13
where they improve some and regress in others.
它们在有些方面改进了，在另一些方面却退步了。
09:15
Okay.
好。
09:16
And that actually happens a lot more than you'd think.
而这实际上比你想的要常见得多。
09:17
Like, what are some of those tasks?
比如，那些任务有哪些？
09:20
Like, are you familiar with how you guys are running a lot of the e-valles?
比如，你熟悉你们是怎么跑很多 e-valles 的吗？
09:22
Like, what are those different sub-signants, I guess, that you look at?
比如说，你关注的那些不同的 sub-signants，我猜，都是些什么？
09:25
So there's different, just like frame-asuf.
所以会有不同的，就像 frame-asuf 那样的东西。
09:28
So we have different basically, like,
所以我们基本上有不同，呃，
09:30
worlds or like, our own environments, essentially,
世界，或者说，基本上就是我们自己的环境，
09:31
which is like, here's a large work in projects
也就是说，这里多多少少总是有一些大型的项目工作。
09:35
always, more or less.
所以就像，这里有一个 10,000 个文件的数据室。
09:35
So it's like, here's 10,000 file data room.
最后你是在做 due diligence。
09:38
You're doing it due diligence in the end.
说到底，你做的还是 due diligence。
09:41
A grid report looks like this.
一份 grid report 大概长这样。
09:42
Like, it contains all these red flags
就比如说，它会把所有这些 red flags 都包含进去
09:44
and like, it doesn't contain whatever weird stuff.
然后呢，它不会包含那些奇奇怪怪的东西。
09:47
And like, we want it to be concise, etc, etc.
还有，我们希望它简洁，等等等等。
09:49
That's kind of how we've built it.
我们差不多就是按这个思路搭的。
09:50
So there's different tasks, which is like,
所以会有不同的 tasks，就有点像，
09:52
part of it is drafting a report.
其中一部分就是起草报告。
09:54
Like, just a draft look good.
就像，先让草稿看起来不错就行。
09:56
Does it contain all the right stuff?
它里面该有的东西都齐了吗？
09:57
Is the wording sound, the analysis sound?
措辞靠不靠谱，分析靠不靠谱？
09:59
There's also the part of finding the red flags,
还有找出 red flags 的那部分，
10:01
like, the analysis before that.
比如在那之前的分析。
10:02
So we can kind of, like, self-divided a little bit.
所以我们可以稍微，呃，自我划分一下。
10:05
And that's how we run the evils.
我们就是这样跑 evals 的。
10:07
Interesting.
有意思。
10:07
What does your evils seem look like?
你们的 evals 大概长什么样？
10:09
Like, do higher lawyers to, like, do that?
比如说，你们会雇律师来做那种事吗？
10:11
So yes, it's a mix we have engineers,
所以是的，是混着来的，我们有工程师，
10:15
and then we have lawyers, and it's one team.
然后我们有律师，而且是一个团队。
10:16
So there's, like, three engineers
所以大概有三个工程师
10:18
and, like, three lawyers right now.
现在大概还有三个律师。
10:19
And then we have a pool of what we call legal engineers,
然后我们有一个人才池，里面是我们所说的 legal engineers，
10:23
which is people that have worked in big law
也就是那些在 big law 工作过的人
10:26
for, like, three or four years.
大概三四年吧。
10:27
Typically, maybe they have a CS degree,
通常，他们可能有 CS degree，
10:29
and, like, they're very tech-forward.
而且，怎么说呢，他们非常 tech-forward。
10:30
And then they come to us,
然后他们会来找我们，
10:31
and because they want to, like, help change the industry.
因为他们想，怎么说呢，帮忙改变这个行业。
10:34
Are they IT in big law?
他们是在 big law 做 IT 的吗？
10:36
Or are they lawyers in big law?
还是说他们是 big law 里的律师？
10:37
They're lawyers in big law.
他们是 big law 里的律师。
10:38
They're lawyers in big law.
他们是 big law 里的律师。
10:39
We've CS degrees.
我们有 CS 学位。
10:40
Yes.
是的。
10:40
Oh, my god.
我的天啊。
10:41
It's a Venn diagram that's, yeah, exactly.
这就像个 Venn diagram，对，没错。
10:43
That's, like, our hottest person to hire for.
这差不多就是我们最想招的那种人。
10:45
But they're also such a force multiplier,
但他们也是非常强的 force multiplier，
10:47
because we can pull on them for the evils stuff.
因为那些 evil 的活儿我们也可以靠他们。
10:48
And for the product development,
还有产品开发方面，
10:49
it's just, like, you have a pool of 100 experts
就是，比如说，你有一个 100 个专家的池子
10:52
that use the product every day and go and talk to clients
他们每天都用这个产品，也每天去和客户聊
10:54
every day, and also, like, they live this and have lived this.
每天都这样，而且，他们也亲身经历这些，而且已经经历很久了。
10:58
And we just pull on them to do evils,
我们就直接找他们来做 evils，
10:59
and, like, help us shape the product.
然后，嗯，帮我们打磨产品。
11:01
And then it's such a, such an advantage.
然后这就是一个特别、特别大的优势。
11:02
And curious, like, when did you guys start that team?
我很好奇，你们是什么时候开始组建那个团队的？
11:05
Evils grew out of the assistant agent team,
Evils 是从 assistant agent 团队里长出来的，
11:08
at some point.
在某个时候。
11:09
I think there was a point up until recently
我觉得直到最近之前，有一段时间
11:12
where it was, like, honestly, just not really worth it.
说实话，那会儿就是真的不太值得。
11:14
Because, like, all models just got better.
因为所有模型就是都变好了。
11:17
So it was, like, do we want to spend a lot of our time
所以就会想，我们真的要把很多时间花在
11:19
doing really complex evils?
做特别复杂的 evals 上吗？
11:21
And it was, like, we want to build,
然后又会想，我们想构建，
11:23
because there was so much to be gained
因为能获得的东西太多了。
11:24
from building a better harness.
从构建更好的 harness 开始。
11:26
And, like, just sort of leaning into everything's going to improve.
而且，就像，干脆就顺着“一切都会变好”这个方向走。
11:28
And everything kind of just improved.
然后一切就真的慢慢变好了。
11:30
But yeah, like, every two, three months,
但是啊，就像，每隔两三个月，
11:32
you get a separate model.
你就会拿到一个单独的 model。
11:33
Yeah, exactly.
对，没错。
11:33
Just pass it through, right?
就直接把它跑一遍，对吧？
11:34
Exactly.
没错。
11:34
I mean, we had one problem where, like,
我是说，我们遇到过一个问题是，就是，
11:36
it wouldn't cite properly.
它 cite 得不对。
11:37
And so it would be difficult to follow.
所以就会很难往下跟。
11:38
And it was, like, next model really solves.
然后就是，下一个模型真的就把它解决了。
11:40
It went from, like, 70% to, like, 99%.
它从，大概 70% 到了，大概 99%。
11:42
It was, like, we did nothing.
就是那种，我们什么都没做。
11:43
And, like, great.
然后，就是，太棒了。
11:44
Yeah.
对。
11:44
Maybe the real work is, you know,
也许真正的工作是，你知道，
11:47
keeping all the data private,
让所有 data 保持私有，
11:49
making it taggable, sort of,
让它变得 taggable，算是吧，
11:51
or adding new features, like, the tabular view.
或者加一些新 features，比如 tabular view。
11:53
No.
不。
11:53
Is that the, you know,
那是，你知道，
11:55
they mean over and above.
他们说的额外的那部分吗？
11:56
I mean, yes.
我是说，是的。
11:58
But obviously, now you're also doing e-vails.
但显然，你现在也在做 evals。
12:00
I assume your use of open models
我猜你对 open models 的使用
12:02
is also increasing.
也在增加。
12:02
We can talk about that.
这个我们可以聊。
12:04
So, yeah, I don't know what I'm saying here apart from, like,
所以，嗯，我也不知道我在这里说这些是什么意思，除了，就像，
12:07
you know, what is the job?
你知道，这份工作是什么？
12:09
The job is, maybe not, in the early days,
这份工作，也许不是，在早期，
12:13
it was not so much to do e-vails,
那时候并不太是做 evals，
12:15
because it was mostly exposed model capability to lawyers.
因为那主要就是向律师展示了 model 的能力。
12:18
Yeah.
嗯。
12:19
The huge part of it is just figuring out
其中很大一部分就是搞清楚
12:22
what does the legal profession look like in five years,
五年后法律行业会是什么样，
12:25
10 years, with all of this stuff.
十年后又会是什么样，在所有这些新东西的背景下。
12:27
Like, coding is maybe a year or two ahead,
比如 coding 可能领先一两年，
12:29
so like, we can kind of look to that a little bit.
所以我们可以稍微参照一下那边。
12:30
But like, what are the primitives?
但问题是，primitives 到底是什么？
12:31
How do people work?
大家到底是怎么工作的？
12:32
How do you collaborate?
你们又是怎么协作的？
12:33
If it's not a 200 thread email
如果不是那种 200 thread 的邮件，
12:35
with, like, v5 word files back and forth,
还带着 v5 word files 来回传，
12:37
like, what does that look like?
就是，那到底会是什么样？
12:38
And that's like,
而且那就像是，
12:39
not even necessarily truly AI.
甚至都不一定真的是 AI。
12:42
That's just, like,
那就只是，就，
12:43
a much better product experience.
产品体验要好得多。
12:45
Yeah.
对。
12:45
And make sure you do it with, like,
而且一定要确保你在做的时候，像，
12:46
auto-locks and data retention,
auto-locks 和 data retention，
12:47
and all of those sort of really important things
以及所有这类真正重要的东西
12:49
in regulated industry.
在受监管行业里。
12:51
And then there's the AI part,
然后还有 AI 那部分，
12:52
which is, like, make the work faster and better
也就是，像，让工作更快、更好。
12:55
and proactive agents that react to stuff when it comes in,
以及 proactive agents，有东西进来时就能做出反应，
12:58
so you don't have to, you know,
这样你就不用，你知道的，
12:59
kick off everything.
什么都自己去启动了。
13:01
So I think a lot of the work is just figuring out,
所以我觉得很多工作其实就是在搞清楚，
13:03
what do we think is going to look like in five years?
我们觉得五年后会是什么样子？
13:04
If, you know, model capabilities keep increasing,
如果，你知道的，model capabilities 一直提升，
13:07
if price keeps coming down,
如果价格一直降，
13:09
what could we do?
我们能做些什么？
13:10
Can you log in and it's, like,
你能登录进去，然后它就像是，
13:11
command center where it's just like,
一个指挥中心，里面就像，
13:12
you need to react to these five things
你需要对这五件事做出反应，
13:14
that the agents have been doing lately
这些是 agents 最近一直在做的，
13:15
and, like, review and check and improve the system?
然后，就像，审阅、检查，再改进系统？
13:18
Maybe that's where we're going.
也许这就是我们要去的地方。
13:19
Or it is where we're going.
或者说，我们确实正在往那儿走。
13:20
Yeah, yeah.
对，对。
13:21
Yeah, I was going to ask, like,
是啊，我正想问，那个，
13:22
could you go into that in a little more down?
你能再稍微展开一点讲讲吗？
13:24
It's like, yeah, what is the, like,
就是，对啊，那个什么，
13:27
while profession going to look like in a few years?
这个职业过几年会是什么样子？
13:32
I should preface this by saying,
我得先说明一下，
13:34
I don't know who knows really,
我真的不知道，谁知道呢，
13:36
but I think we can look to coding a lot.
但我觉得我们可以多参考 coding。
13:39
I think all hours are going to be way more
我觉得所有时间都会更加……
13:42
productive than they were before.
比以前更高效。
13:44
They're going to be able to do a lot more stuff
他们将能够做更多事情
13:46
at a higher quality for lower cost.
以更高的质量、更低的成本。
13:50
I think the profession itself, the work itself,
我觉得这个职业本身、这份工作本身，
13:52
is going to improve a lot.
会有很大提升。
13:54
It's going to be a lot more.
会多得多。
13:55
I think lawyers are going to spend more of their time
我觉得律师会把更多时间
13:57
doing the stuff that they've been lawyers to do,
花在他们作为律师本来要做的事情上，
14:00
you know, figuring out what's the right risk profile,
你知道，弄清楚什么样的 risk profile 才是合适的，
14:03
which manageable risk.
哪些是 manageable risk。
14:04
What's, like,
那是什么，就像，
14:05
all of their judgment, strategic judgment,
他们所有的判断，strategic judgment，
14:07
and, like, working with clients,
还有，就像，和客户合作，
14:08
they're going to do that a lot more
他们会更多做这些，
14:09
and much less of the, like,
而少很多那种，就像，
14:10
trolling through big documents
在大堆文件里翻来翻去
14:12
to, like, find the smoking gun
去，比如说，找到那个实锤
14:14
or, like, do all this menial,
或者，比如说，去做那些琐碎的、
14:15
super annoying work
超级烦人的活儿
14:16
that a lot of lawyers have been doing.
很多律师一直在做的那种
14:18
What do you think is going to happen
你觉得接下来会发生什么
14:19
in the partnership model?
在 partnership model 里？
14:20
Like, do you think that stays?
比如说，你觉得这还会保留吗？
14:21
Or, like, what happens?
还是说，你觉得会怎么样？
14:22
I gave this, I actually gave this a lot of thought
这个我，其实我认真琢磨了很久
14:24
just, like, last week,
就，差不多上周吧，
14:25
because the partnership model,
因为 partnership model 这个事，
14:27
I don't want to get into too much of the details,
我不想讲太多细节，
14:28
but, like, the partnership model
但是，就是，partnership model
14:30
kind of exists
它之所以存在
14:33
because the only type of capital
是因为这些公司唯一拥有的资本类型
14:36
that these companies have is human capital.
就是 human capital。
14:38
So, like, it's not like, you know,
所以，就像，它不是那种，你知道的，
14:39
or, like, a software engineering firm
或者，就像，一家 software engineering 公司
14:41
or, like, a manufacturer,
或者，就像，一个制造商，
14:42
or whatever, you have physical assets
或者随便什么，你有实物资产
14:43
or, like, some sort of intangible assets
或者，就像，某种无形资产
14:45
that you accrue over time
是你随着时间积累起来的
14:46
and they pay dividends to the shareholders.
然后它们给股东带来分红。
14:49
But, if you work in an industry,
但是，如果你在一个行业里工作，
14:50
like, professional services
比如说，专业服务
14:52
and it's the same for, like, consultancies
咨询公司之类的也一样
14:54
and whatever else,
还有别的什么，
14:55
all of your value is in your people.
你所有的价值都在你的人身上。
14:57
And, you can't really own human capital.
而且，你其实没法真正拥有人力资本。
15:00
You know, you can't, you know,
你知道，你没法，你知道，
15:01
slavery is abolish.
奴隶制已经废除了。
15:02
You can't own people,
你不能拥有别人，
15:03
but you can rent them.
但你可以租它们。
15:04
You can pay the money.
你可以付钱。
15:04
And so, partnerships essentially
所以，partnerships 本质上
15:07
is a structure where you make
是一种结构，让你把
15:08
the human capital the owners of the company
human capital 变成公司的所有者
15:11
so that they stay.
好让他们留下来。
15:12
Yeah.
是啊。
15:12
But that change, like,
但那种变化，就像，
15:14
I don't, it doesn't have to,
我不，这不一定非得这样，
15:16
but I think what we really want to do with the lawyers
但我觉得我们真正想和律师一起做的，
15:19
who actually want lawyers to be able,
是让那些确实希望律师能够、
15:21
and law firms to be able to build
也让律所能够去建立
15:23
some sort of intangible capital,
某种无形的资本，
15:25
some durable capital,
某种持久的资本，
15:26
in, like, you can build a system
比如，你可以搭建一个系统，
15:28
that learns of how you work.
它能学习你是如何工作的。
15:29
A lot of firms right now think about, like,
现在很多公司都在想，比如说，
15:31
well, there's the ex law firm way of doing things.
嗯，有那种前律所做事的方式。
15:34
And we can codify that, I think,
我觉得我们可以把那个固化下来，
15:36
and we can build a system
然后我们可以搭一个系统，
15:38
and whether that's, like, a self-improving knowledge graph
不管那是一个 self-improving knowledge graph，
15:41
or whatever it is,
还是别的什么，
15:42
and, like, precedent files and documents,
再加上判例文件和文档，
15:44
where there's, like, a way of working
就像，有一套工作方式。
15:45
that produces a really good output.
能产出非常好的 output。
15:48
And that can be their IP,
而且那可以成为他们的 IP，
15:49
is so it's, like, a way to codify
所以这就像是一种方式，去把
15:51
the human expertise and judgment into a system
人类的专业知识和判断力编码进一个系统里
15:54
that, I think, is not just rambling completely,
我觉得，这不只是在漫无边际地乱说，
15:56
but, like, that's something that we want to help empower.
而是，这确实是我们想要帮助赋能的东西。
16:00
And just that change the partnership model then,
而这也改变了合作模式，
16:02
because then, since it's so different
因为这样一来，既然它是如此不同
16:03
in the partnership model,
在合伙模式里，
16:04
you just got to bring more clients,
你就是得多带客户，
16:06
and you don't want to take away time
而你又不想把时间抽走，
16:08
from bringing in clients
从带客户这件事上，
16:09
to codify your information, basically,
去把你的信息整理成体系，基本上，
16:12
because, like, you get rewarded
因为，你知道，你会得到回报，
16:13
from, you know, either what you kill
要么来自你拿下的东西，
16:14
or make this luck step, whatever.
要么来自 make this luck step，随便啦。
16:16
That needs to change a little.
这一点需要稍微改一改。
16:17
We need to figure out
我们得弄清楚
16:18
in a sense of structure
从结构上来说
16:19
where partnerships, partners,
在合作关系里，合作伙伴
16:22
aren't incentivized to codify things
并没有被激励去把东西规范化
16:23
for, to build an asset for the company,
去为公司打造资产，
16:25
but right now there aren't,
但现在并没有，
16:26
because, as soon as they leave the partnership,
因为一旦他们离开合作关系，
16:28
they don't get any dividends
他们拿不到任何分红
16:30
from the assets that they built.
从他们打造的那些资产里。
16:32
Yeah.
是啊。
16:32
So it's, like, yeah,
所以，就像，是啊，
16:34
I think the whole business model is,
我觉得整个商业模式就是，
16:37
we'll see.
走着瞧吧。
16:38
Yeah, that's all in question right now.
是啊，现在这些都还不好说。
16:40
Yeah, my actual, other take, you know,
是啊，其实我另一个看法，你知道，
16:43
so on the partnership level,
所以在合伙人层面上，
16:44
that's like the top end of the industry.
那基本上就是行业的顶端了。
16:47
Actually, my take is more on the bottom end, right?
其实，我的看法更偏向底端，对吧？
16:49
Yeah, what happened there?
是啊，那边发生了什么？
16:50
Apparently goes gone, right?
据说都没了，对吧？
16:52
Like, junior lawyers gone.
比如，junior lawyers 都没了。
16:53
I don't need them.
我不需要他们。
16:54
Like, it's, that's actually not what we're seeing.
就，其实，我们看到的并不是这样。
16:56
Okay, I think it's, I don't know,
好吧，我觉得这，我也说不准，
16:58
would you say that we're seeing that in engineering?
你觉得我们在 engineering 里看到这种情况了吗？
17:01
Yeah.
嗯。
17:02
So, junior engineers are really the kind of cooks.
所以，初级工程师真的就是那种厨子。
17:05
Like, you kind of have to come out
就像，你差不多得从
17:06
of dropout of university, fully baked,
大学辍学出来的时候，就已经完全成熟了，
17:09
or else you're not going to make it.
不然你就成不了。
17:14
I feel like I'm being a bit dramatic for, for a comedic purpose.
我觉得我为了，为了喜剧效果，有点太夸张了。
17:18
But like, yeah, I know a lot of lawyers,
不过怎么说呢，是啊，我认识很多律师，
17:21
they started as paralegal,
他们是从 paralegal 做起的，
17:22
they go to law school, all these things, right?
他们去读法学院啊，做这些事，对吧？
17:24
And they don't, they're not needed.
而他们并不——他们其实并不被需要。
17:25
They're seriously just not needed.
说真的，他们就是不被需要。
17:26
Yeah.
是啊。
17:27
Yeah.
是啊。
17:28
It's interesting what happens to the partnership
这个合伙制后来会怎样，挺有意思的。
17:30
among the, like, the pyramid.
在……之中，比如说，金字塔。
17:32
I think generally a lot of lawyers,
我觉得一般来说，很多律师，
17:35
you still need to do the next generation.
你还是得做下一代。
17:37
And at least for engineering, the way I think about it,
至少对工程来说，我是这么想的，
17:40
is you have a good engineer today.
就是，你今天有一个好工程师。
17:42
There was a lot more,
而且要多得多，
17:43
even than they were two years ago,
甚至比两年前还要多，
17:45
because they can produce so much more stuff.
因为他们能产出多得多东西。
17:46
Like, if they're good,
就比如，如果他们很厉害，
17:47
and I think it's the same thing in legal,
而且我觉得在法律行业也是一样，
17:48
where they can actually do a lot of really good work,
他们其实能做出很多非常出色的工作，
17:50
if they're good.
前提是他们得厉害。
17:51
So, I think it's more that, like, really good lawyers,
所以，我觉得更像是，真正厉害的律师，
17:53
or junior lawyers are worth a lot more.
或者初级律师，价值要大得多。
17:55
Yeah, it's always been the case, right?
是啊，一直都这样，对吧？
17:56
Yeah, it's even more so.
是啊，现在更是如此。
17:58
But it's, yeah, it's polarizing even more so.
但这个，是啊，它甚至更加两极分化了。
18:00
I'm actually curious about that.
我其实对这点挺好奇的。
18:01
So, like, the whole, like,
所以，就，整个，就，
18:03
law business is basically, like, charging per hour.
法律这行基本上就是，就，按小时收费。
18:07
So, you know, if each project now can be faster,
所以，你知道，如果现在每个项目都能更快，
18:12
is it going to be, like, way cheaper?
那它会不会，就，便宜很多？
18:13
Like, this was actually, so fun story.
就，这其实，说起来挺有意思的。
18:16
I was reading Max's YC application
我当时在看 Max 的 YC 申请
18:19
and, like, helping him get into YC,
而且，就比如，帮他进 YC，
18:21
get layers that was called back then, and it's YC.
拿到当时叫的那个 layers，然后就是 YC。
18:23
And this was one thing.
而这是一件事。
18:24
I was like, and curious,
我当时就想，而且很好奇，
18:25
Leia is a Danish thing for something?
Leia 是丹麦语里表示某个东西的词吗？
18:26
No, it's Swedish Swedish.
不，是瑞典语，瑞典语。
18:28
No, also it means nothing.
不，而且它也没什么意思。
18:31
There's this really funny story,
有个特别好笑的故事，
18:32
and I think it was a Spanish newspaper
而且我觉得那是一家西班牙报纸
18:34
that wrote about us in the early days, Leia.
早期的时候写过我们，Leia。
18:36
And then they explained that the abbreviation was,
然后他们解释说，这个缩写是
18:39
legally efficient Yoda assistant.
legally efficient Yoda assistant。
18:44
Why Yoda?
为什么是 Yoda？
18:44
Yeah, I don't know.
是啊，我也不知道。
18:45
Sometimes I was like, why it was?
有时候我会想，为什么会是这样呢？
18:46
See, hallucinations are not just from elevators.
你看，hallucinations 可不只是来自电梯。
18:48
No, I think journalists hallucinate.
不，我觉得记者才会 hallucinate。
18:51
But no, it doesn't mean anything.
不过，不，这也不代表什么。
18:53
So he wrote the application.
所以他就写了那个应用。
18:54
Yeah, and my feedback to him was,
对，然后我给他的反馈是，
18:57
you know, people have tried this.
你知道，这东西人们已经试过了。
18:58
A million times, legal tech has never been a thing.
都试过一百万次了，legal tech 从来就没成过气候。
19:00
Because, like, there's no incentive for lawyers to,
因为，怎么说呢，律师根本没有动力去，
19:02
why would they be more efficient?
他们为什么要变得更高效呢？
19:03
They earn less money, like, it doesn't make any sense.
他们赚得更少，怎么说呢，这完全说不通。
19:05
But I think the reason that
但我觉得，之所以
19:08
lawyers do want to adopt this is that they understand
律师们确实想采用这个，是因为他们明白
19:11
the gravity of the situation.
情况的严重性。
19:12
They know that, like, the business model and everything
他们知道，怎么说呢，商业模式啊还有这一切
19:14
is under pressure.
都面临压力。
19:15
And it's like, they need to change.
而且怎么说呢，他们需要改变。
19:16
Because the competitive pressure is so high.
因为竞争压力太大了。
19:18
The billing model itself, I think, is also going to change.
计费模式本身，我觉得，也会发生变化。
19:21
We've already seen that, particularly in the US,
我们已经看到这一点了，尤其是在美国，
19:23
that a bunch of stuff is actually being priced
很多东西实际上正在按
19:24
on, like, a project fee, and not on an hour's fee.
项目费来定价，而不是按小时收费。
19:27
Particularly, it's, like, sort of, lower end stuff
尤其是那种偏低端的东西，
19:31
on the spectrum, like, whatever it is.
在这个范围里，差不多就是不管什么吧。
19:32
Like, small reviews and whatever.
比如小型评审之类的，随便什么。
19:34
Some of it honestly is being done for free.
说实话，有些甚至都是免费做的。
19:36
And some of it is just being priced on a flat rate.
而且其中有些只是按固定费率定价。
19:39
So where is the money being made then?
那钱到底是从哪儿赚来的呢？
19:41
Is it because of that?
是因为那个吗？
19:41
Because the strategic work.
因为那些战略性的工作。
19:42
Yeah, I feel like a lot of what you guys talk about
是啊，我感觉你们聊的很多内容
19:45
is M&A specifically.
具体来说就是 M&A。
19:46
Like, that's where all the money is made.
就是说，钱都是在那儿赚出来的。
19:47
It's just a very easy example.
这只是一个非常简单的例子。
19:48
Yeah, yeah, yeah.
对对对。
19:49
Litigation is also the big work still is there.
诉讼也仍然是重要的工作。
19:52
And it's very valuable.
而且它非常有价值。
19:54
And so those spent,
所以那些花掉的，
19:55
lawyers will spend more of their time there.
律师会把更多时间花在那上面。
19:57
Client relationships are also super important.
客户关系也超级重要。
19:59
Like a CRM?
像 CRM 那样吗？
20:00
No, it's just like spending more time with the client.
不，就是多花点时间和客户待在一起。
20:02
Like, a lot of law firms will,
就是，很多律所会，
20:03
or historically, have basically done stuff for free
或者说，过去基本上都是免费干活，
20:06
just to have their relationship with the client.
就为了和客户保持关系。
20:07
Yeah, yeah.
对，对。
20:08
And so now they can do that,
所以现在他们可以这么做，
20:09
but they don't have to lose a lot of money on the way.
但不必在这个过程中亏很多钱。
20:11
I thought I just have to pay the lawyer money
我还以为我得给律师钱
20:14
just to be my friend.
就为了让他做我的朋友。
20:15
Like, this is like,
就像，这就像是，
20:16
isn't that the retainer thing?
不就是那个 retainer 的事吗？
20:18
Yeah, that's how you told me I can do it for free.
对，你就是这么跟我说我可以免费做的。
20:20
Like, yeah, you just have to find the right lawyers.
就，对啊，你只要找到合适的律师就行。
20:22
Yeah, that's right.
对，没错。
20:25
Okay, cool.
好，可以。
20:26
Or Linus Pays has to grow much bigger, I don't know.
或者 Linus Pays 得发展得大得多，我也不知道。
20:27
Yeah, yeah, we're trying.
对对，我们正在努力。
20:30
You know, so, okay, so, so,
你知道，所以，好吧，所以，所以，
20:32
you're saying, you know,
你是在说，你知道，
20:34
obviously the strategic work
显然，那些战略性的工作
20:35
and all these high value add is still there,
还有所有这些高附加值的事，仍然还在，
20:37
but our legal fees are not going down,
但我们的律师费并没有降，
20:39
but you also said paralegals are not affected
但你也说过律师助理没受影响
20:41
or like, they're just shifting.
或者说，就像，他们只是在转移。
20:43
And this is something that we also did worry about with cognition.
而这也是我们在 cognition 上也担心过的事情。
20:47
Like, people worry about the phrasing of like,
就是，大家会担心这种措辞，比如，
20:50
we will replace your junior engineer
“我们会替换掉你的初级工程师”
20:52
because nobody likes that kind of statement.
因为没人喜欢那种说法。
20:54
But in actual fact, yes,
但事实上，是的，
20:56
like, some part of it does happen.
就是，其中一部分确实会发生。
20:58
Other junior engineers embrace it
另一些初级工程师则接受它，
20:59
and they become more productive.
而且他们变得效率更高。
21:00
They're exactly like you just said.
他们完全就像你刚才说的那样。
21:02
So, I guess like, is there a new term
所以，我想问，有没有一个新词
21:04
for this role or this job that is taking,
来形容这个角色或者这份工作，而且它正在
21:07
that is having traction with your customers?
在你的客户那边越来越有 traction？
21:10
You mean the super AI pull junior legal?
你是指 super AI pull junior legal 吗？
21:13
Nothing I found yet.
我目前还没找到。
21:14
Maybe we should call in one.
也许我们该请一个来。
21:15
Isn't there legal legal engineer?
不是有 legal，legal engineer 吗？
21:17
That's our, and also now an industry thing.
那是我们自己的叫法，现在也成了行业里的说法。
21:20
Yeah.
是啊。
21:21
CSJD is exactly.
CSJD 正是如此。
21:22
Yeah, yeah, yeah.
对对对。
21:24
And maybe that's the, there are those other people
也许那就是，还有另外那些人
21:26
that are best positions to do this kind of stuff.
他们最适合做这类事情。
21:28
I think that the derogatory turn will meet proxy
我觉得那种贬损性的转变会遇到 proxy
21:31
where you just like sit in front of the machine
就是你只是坐在机器前面
21:33
and type in something that someone else
然后输入一些别人……
21:34
helps you to type.
帮你打字。
21:35
And then you repeat what the machine tells you to say.
然后机器让你说什么，你就重复什么。
21:38
But like, I do think a very good me proxy
但说真的，我确实觉得一个非常好的 me proxy
21:40
is just like, well, you know agents very well.
就是，嗯，你非常了解 agents。
21:42
And you know the gore very well.
而且你非常了解 gore。
21:44
And you know how they're like,
而且你知道它们是什么样的，
21:46
like, this is my toolkit.
就像，这是我的 toolkit。
21:47
And, you know, every industry has a toolkit.
而且，你知道，每个行业都有 toolkit。
21:50
And I can use my tools to do the thing
而且我可以用我的工具来做那件事
21:51
that you need me to do.
需要我来做的。
21:52
But I guess then you're not really a proxy, right?
但我想这样一来，你就不是真正的 proxy 了，对吧？
21:55
You're actually just doing meaningful work.
你其实就是在做有意义的工作。
21:57
Yeah, it's just like, I no longer, like, you know,
嗯，就像，我不再，那个，你知道，
21:59
well, I'll talk about tabular review
嗯，我会聊聊 tabular review
22:01
because that's the thing that happens a lot.
因为这种情况经常发生。
22:03
I have hundreds, thousands, millions, thousands.
我有几百、几千、几百万、几千。
22:06
We do up to a million cells, I think,
我觉得我们最多会处理一百万个 cells，
22:08
in a thing that's impressive.
这事儿还挺厉害的。
22:10
We should talk about this, the table and the windowing.
我们得聊聊这个，table 和 windowing。
22:13
Yeah, I'm working on a big case.
对，我在处理一个大案子。
22:15
I have a million documents.
我有一百万份文件。
22:17
I type it in.
我把它输进去。
22:18
Usually I'll throw a pair, throw like a team of paralegals
通常我会丢两个人进去，就像把一队律师助理关在房间里。
22:20
locked in the room.
所以就像，对一个……这种情况其实不需要发生，对吧？
22:21
So like, for a, like, that doesn't need to happen, right?
所以就像，对于一个，就像，那不需要发生，对吧？
22:24
Because now you can extract it
因为现在你可以把它提取出来
22:25
and you can go and reprompt everything.
然后你就可以去 reprompt 所有东西了。
22:27
So like, yeah, I think now I don't need a paralegal for that.
所以，对，我觉得现在我不需要律师助理来做这个了。
22:31
A good, like, super AI-pilled paralegal
一个优秀的、特别 AI-pilled 的律师助理
22:34
would do something else, you know?
会去做别的事，你懂吧？
22:36
Yes, exactly.
对，没错。
22:37
Yeah, they can do more stuff,
是啊，他们能做更多事情，
22:38
or like spend more time on the things
或者把更多时间花在这些事情上
22:40
that they, you know, didn't have time for it before.
就是他们，你知道，之前没时间做那个。
22:42
Or actually sleep, who knows, no?
或者其实是去睡觉了，谁知道呢，对吧？
22:44
I have a question.
我有个问题。
22:45
So you guys bought a company, Kira.
所以你们收购了一家公司，Kira。
22:47
Yes, is that how you say it?
对，是这么读的吗？
22:48
No, you guys used your own product to do the M&A?
不是，你们是用自己的产品来做 M&A 的？
22:51
Yes, we do that all the time.
是的，我们一直这么干。
22:52
So how did that work?
那具体是怎么操作的？
22:53
We can talk about that.
这个我们可以聊。
22:55
Yeah, no, that works great.
嗯，对，这样挺好的。
22:56
We're doing that a lot.
我们经常这么做。
22:57
Our legal team is, I mean, intake has been built
我们的法律团队，我是说，intake 已经搭建好了
22:59
with our legal team.
是跟我们法律团队一起弄的。
23:00
It's the right now.
就是现在。
23:01
We have a slack workflow where everyone's
我们有一个 Slack workflow，每个人
23:03
closed their stuff and they're like,
都把自己的东西关掉了，然后他们就说，
23:04
this is annoying, right?
这挺烦人的，对吧？
23:05
And it efficient, so it's great.
而且它很高效，所以很棒。
23:07
Wish you the goal for that.
希望你能实现那个目标。
23:08
Yeah, can you talk a little bit about that?
嗯，你能稍微谈谈那个吗？
23:10
Like, you know, what that process looks like?
就是，你知道的，那个流程是什么样的？
23:11
Yeah, absolutely.
嗯，当然。
23:12
So it's, it's, we do a lot of the do-dillionage ourselves
所以就是，就是，很多尽职调查都是我们自己做的
23:17
because we can do it with regular view and other tools.
因为我们可以用 regular view 和其他工具来做。
23:19
We do still have council coming in
我们确实还是会有 council 进来
23:21
to like assess certain risk
来评估某些风险之类的
23:23
or like, you know, there's payroll
或者说，你知道，还有 payroll 这块
23:25
and so the crew was simple because I was on Sweden,
所以团队这边比较简单，因为我当时在 Sweden，
23:27
but, you know, we've done one in Canada
但你知道，我们在 Canada 做过一个
23:29
and one in Australia.
在 Australia 也做过一个。
23:30
And so there's like payroll in Australia.
所以 Australia 那边也有 payroll 这块。
23:32
Okay, do we want someone to look at this
好，我们要不要找个人来看看这个
23:34
like a bit more strategically?
比如稍微更有策略性一点？
23:35
So there, we will still use Laura's
所以在那块，我们还是会用 Laura's 来做战略部分，让他们真正能当切磋伙伴，在比如怎么想清楚用对的方式把这件事搭起来这方面帮我们很多。
23:37
for the strategic part where they can actually
但很多更琐碎的事情，其实我们自己就能做。
23:39
be sparring partners and help us a lot on like
所以就是，很多那个，嗯，
23:41
how to figure out how to structure this the right way.
怎么想办法用正确的方式来构建这个。
23:43
But a lot of the more menial stuff
但很多比较琐碎的东西
23:44
we could actually just do ourselves.
我们其实可以自己做。
23:46
So like, a lot of the, yeah,
所以就像，很多那个，对，
23:47
a lot of the early things we would find
我们早期会发现的很多东西
23:49
and we have had cases where we found something.
而且我们也有过发现了一些情况的时候。
23:52
That's been like a red flag
那就像一个 red flag
23:53
and then we pulled the brakes.
然后我们就踩了刹车。
23:55
Can you talk a little bit about any of these?
你能稍微谈谈这里面的一些情况吗？
23:56
No, a lot of things.
不，很多事情。
23:58
Yeah, all right.
嗯，好吧。
23:59
Early on you said your early customers were Swedish
你之前说过，你早期的客户是瑞典人
24:03
like Swedish offerings.
比如瑞典那边的服务。
24:04
Have you seen a difference between working
你有没有发现，合作的时候
24:07
with like Swedish law versus like American law firms?
跟瑞典法律那边和跟美国律所相比，有什么不一样？
24:09
That's her thing.
那是她的专长。
24:11
And you guys are mostly working
那你们主要是跟
24:12
with like big, big law firms, right?
那种很大很大的律所合作，对吧？
24:14
Well, we're also smaller law firms also.
嗯，我们也有规模小一点的律所。
24:17
Oh, no, we definitely started really, really big law firms,
哦，不是，我们一开始确实是跟非常非常大的律所合作，
24:19
but 50 Laura's also.
但 50 Laura's 也一样。
24:21
Okay.
好。
24:22
And get a lot of value.
还能收获很多价值。
24:23
Yeah, so have you seen differences between working
嗯，那你有没有发现，跟瑞典的、美国的，甚至，我不知道，亚洲的团队合作时有什么不同？
24:25
with like the Swedish ones versus American ones
嗯嗯。
24:27
maybe like even, I don't know, Asian ones?
我觉得欧洲的团队行动得稍微慢一点。
24:29
Yeah, yeah.
对对对。
24:30
I think the European ones have been a bit slower to move.
我觉得欧洲那些行动得稍微慢了一点。
24:35
The American ones are very forward,
美国那边非常前瞻，
24:38
move really quickly.
行动也特别快。
24:39
I think it's also very high on their agenda
我觉得这在他们的议程上也排得很靠前，
24:41
of the American companies more so than the European ones,
美国公司比欧洲公司更是如此，
24:44
I think it's like a strategic super important thing.
我觉得这就像是一件战略上超级重要的事。
24:46
We need to just like do a lot here.
我们就是得在这方面多做很多。
24:48
The American companies generally both
美国公司总体上，无论是
24:50
corporates and law firms also have a much higher appetite
企业还是律所，胃口也都大得多
24:53
to like custom build stuff.
就是喜欢自己 custom build 东西。
24:55
It's like a thing that they all want to do.
这好像是他们都想做的事。
24:57
Hey, we want to build this like a huge thing with you guys
嘿，我们想跟你们一起搞一个特别大的东西
24:59
and like structured in whatever way,
然后结构怎么安排都行，
25:01
where your opinions are much more like,
而你们的意见更像是，
25:02
we'll just take the off the shelf.
我们就直接用现成的。
25:03
And what's an example of something
那举个例子，有什么东西
25:05
they would want to build themselves?
是他们想自己做的？
25:06
Integration sensor, they're all like,
Integration sensor，他们全都这样，
25:08
some of these customers build their own CRM
有些客户会自己搭建自己的 CRM
25:10
when they're own a document management system
当他们自己有一套 document management system 的时候
25:12
and their own all sorts of stuff.
还有他们自己各种各样的东西。
25:14
And they want to build that and then they want to build
他们想把这个搭起来，然后还想搭
25:16
triggers that like you do a review
triggers，比如你做一个 review
25:18
and sales forcing goes to this one person for whatever reason
然后 sales forcing 不管什么原因就会转到某一个人那儿
25:21
and they do something and then it goes into the goal
他们做点什么，然后它就进到 goal 里了
25:22
and then it goes back in.
然后它又回到里面了。
25:23
And there's that's like the simple end
然后还有一种，算是比较简单的那种结尾
25:25
and then there's some that are extremely ambitious
然后还有一些是特别有野心的
25:27
whereas like a whole new product that they want to build
而有的呢，是他们想做的全新产品
25:29
with us and I, you know, we can't really do that.
跟我们合作，而我，你知道，我们真的做不了那个。
25:31
Yeah.
对。
25:33
But that's definitely a thing where like
但这确实是个事儿，就是那种
25:35
American companies really want to build their own
美国公司真的很想打造自己的东西
25:38
more than European ones do.
比欧洲的更多。
25:39
And the European ones are like,
而欧洲的那些就像，
25:41
hey, like, you know, they give you products suggestions
嘿，就比如，你知道，他们会给你产品建议
25:43
or yeah, for sure.
或者，对啊，当然。
25:44
I mean, we have a great collaboration with all of them.
我是说，我们和所有这些人都合作得很好。
25:46
So they'll let those, hey, this sucks,
所以他们会跟那些人说，嘿，这个很烂，
25:48
this is great, we want more of this
这个很棒，我们想要更多这样的
25:49
and we'll just build a product that does that.
然后我们就做一个能做到这一点的产品。
25:52
Okay, so they don't see like an existential threat here.
好吧，所以他们在自己这儿并不觉得这像是个生存威胁。
25:55
In their business.
在他们的业务里。
25:56
I mean, there's the American ones do
我是说，美国那些公司会
25:58
or I don't know if that's the reason
或者我也不知道那是不是原因
26:00
that they want to cost them both versus not cost them both.
他们是想让双方都付出代价，还是不想让双方都付出代价。
26:02
I think they both know that this is like
我觉得他们双方都知道，这就像是
26:04
extremely important and key to their firm's success
对他们公司的成功极其重要、非常关键
26:07
and their corporate success.
以及他们企业的成功。
26:09
So they both extremely important.
所以这两个都极其重要。
26:10
I think it's just a different way to achieve it.
我觉得这只是实现它的不同方式。
26:12
Okay.
好。
26:13
Another question I think maybe you want to bring this up
另一个问题，我觉得你可能想提一下
26:17
is like product metrics.
就是 product metrics。
26:18
Like how do you know that like, you know,
就像你怎么知道，就是说，你知道，
26:20
we're actually doing something good here.
我们在这里到底是不是真的在做有价值的事。
26:22
Yeah, how do you think about that?
对，你怎么看这个？
26:23
I'm assuming that's also related to evels.
我猜这也和 evels 有关。
26:25
So we have the regular product metrics
所以我们有一些常规的 product metrics，
26:27
that we care a lot about.
这些是我们非常在意的。
26:28
Are people spending a lot of time in our product?
大家会在我们的产品里花很多时间吗？
26:30
Did they seem to get a lot of value out of it?
他们看起来是不是从中获得了很大价值？
26:32
Do they renew it?
他们会续约吗？
26:33
Do they come back a lot?
他们会经常回来吗？
26:34
We have really good numbers for an enterprise software
对于 enterprise software 来说，我们的数字真的很好。
26:38
of like our Dow over mile
比如说我们的 DAU over MAU
26:40
and our like, you know,
还有我们的，那个，你知道的，
26:42
our spent per month and the product are extremely high.
我们每个月的支出，还有产品指标，都特别高。
26:45
And then we do some sort of we do online evels also
然后我们还会做一些，算是 online evals 的东西
26:47
where it's like try to figure out
就是试着搞清楚
26:49
generally like do we solve the problem here?
总的来说，就是我们到底有没有解决这里的问题？
26:51
Just like with a super cheap element of message,
就像用一个超便宜的 message 元素，
26:52
just try to just like have an online metric
就是试着，怎么说呢，有一个 online metric
26:54
and then we can use that also when we do row lots of new things.
然后我们做很多新东西的时候，也能把这个用上。
26:57
We can just like compare feature flag wise, okay?
我们就可以按 feature flag 来对比，对吧？
26:59
Does it improve or like did it catastrophically decline?
它是变好了，还是说灾难性地下滑了？
27:02
Because one thing actually that's might be interesting.
因为其实有一点可能挺有意思的。
27:05
That's difficult for our businesses.
这对我们的业务来说很难办。
27:07
We can't look at any traces basically.
基本上，我们看不了任何 traces。
27:08
We can't look at any data
我们看不了任何 data，
27:09
because it's all super confidential.
因为那些全都超级机密。
27:11
And that's like extremely frustrating to work with
而且跟它打交道真的特别让人抓狂
27:13
because you basically don't know what people are doing.
因为你根本不知道大家都在拿它做什么。
27:16
So it's like here's an agent
就像是，这里有个 agent
27:17
and it can do so many amazing things
它能做那么多很厉害的事
27:18
and I have no idea what people are actually doing.
但我完全不知道大家实际在拿它做什么。
27:20
I would have assumed you guys like
我本来还以为你们会
27:21
to sign some sort of agreement with the law firm
跟那家律所签个什么协议
27:23
and they're like, I wish, yeah, I wish, no.
结果他们说，我也想啊，是啊，我也想，不行。
27:25
But the thing is like a lot of the data that they have
但问题是，他们拥有的很多 data
27:27
is not even there.
根本就不在那儿。
27:28
So I just they're clients and it's super confidential.
所以我就——他们是客户，而且这超级机密。
27:31
So we can look at it.
所以我们能看一下。
27:31
And so basically we have an evels team
所以基本上我们有一个 evels 团队
27:33
that's like trying to approximate
也就是在试着 approximate
27:35
when we think everyone is doing it
我们认为大家都在做的那个东西
27:36
and like build that into evels sets.
然后把它构建进 evels sets 里。
27:39
So that's just like a super annoying kink of the business.
所以这就像是这行生意里一个超级烦人的怪毛病。
27:41
Can you do paraphrase?
你能做一下 paraphrase 吗？
27:43
So for example, ChatgbT has like a slash feedback thing
所以比如说，ChatgbT 有个类似 slash feedback 的东西
27:47
that directly fully says it's full trajectory.
会直接完整地说出它的 full trajectory。
27:50
But obviously that's not possible for you guys.
但显然你们做不到。
27:53
But you can use it all into paraphrase
但你可以把这一切都拿去做 paraphrase
27:55
like from this real thing, make a synthetic thing
比如从这个真实的东西出发，造一个 synthetic 的东西
27:58
that is like close enough, but change all the names,
差不多够接近，但把所有名字都改掉。
28:01
change all the details, but like still.
把所有细节都改掉，但还是那样。
28:03
Yeah, yeah.
嗯，嗯。
28:04
We can't read those.
我们读不了那些。
28:05
I mean, there's always a range right of like if you,
我是说，总归有个范围嘛，就好像如果你，
28:08
if your paraphrase all the way down
如果你的转述一路改到
28:09
to like three values and you pick one of them
差不多三个值，然后你挑其中一个
28:11
then obviously that's fine.
那显然没问题。
28:12
If you just remove the names that has made me
如果你只是把那些名字去掉，那让我
28:15
maybe not fine.
也许不太行。
28:15
Yeah, it's like we are generally pick the most safe way.
对，就像我们通常会选最安全的方式。
28:21
We restrict ourselves as most possible
我们会尽可能约束自己
28:23
because we, you know, there's no risk at all.
因为我们，你知道，完全没有风险。
28:26
But there are things where it's like, okay,
但有些事情是这样的，好吧，
28:28
we know that the failure modes are probably these 10 or 20
我们知道 failure modes 大概就是这 10 个或 20 个
28:30
so we can classify it into one of those
所以我们可以把它归类到其中之一
28:32
and that's a helpful signal.
而这是一个有用的信号。
28:34
We can classify it into the types of tasks
我们可以把它按任务类型来分类
28:36
and the practice areas also
还有实践领域也是
28:37
and then like we have an idea
然后就像，我们有个想法
28:39
it's something that we're always trying
这是我们一直想做的事
28:40
to figure out how to do it in a smart way.
就是想办法用聪明的方式去做
28:42
Also like on the privacy friend I guess
还有就像在 privacy 这块儿，我猜
28:44
I'm assuming even two years ago
我估计就算两年前
28:46
I must have been like insanely difficult
那肯定也是难到离谱
28:48
to get the law firms to be okay with this.
让律所那边能接受这件事。
28:49
But like how do they think about their data being
但他们到底怎么看待自己的数据被
28:52
sent over to Anthropic?
发送到 Anthropic 那边？
28:53
Yeah, yeah, yeah, yeah.
对对对对。
28:54
It must be like a huge thing.
这肯定是个大事吧。
28:56
All of our infrastructure is on Azure
我们所有基础设施都在 Azure 上
28:58
for a marathon for this reason
为了一个马拉松，也是因为这个原因
28:59
because like when we started,
因为就像我们刚开始的时候，
29:01
law firms were like AWS.
律所就像 AWS 一样。
29:03
No way.
不会吧。
29:04
No way.
不会吧。
29:05
We don't trust it.
我们不信它。
29:06
We don't trust AWS.
我们不信任 AWS。
29:07
Yeah, yeah.
对对。
29:08
It's different now.
现在不一样了。
29:08
No, but there was like Amazon, Google.
不是，但那时候有 Amazon、Google 之类的。
29:11
We can't trust them within our data.
我们不能在我们的数据范围内信任他们。
29:12
So we basically we had to host everything
所以我们基本上不得不把所有东西都托管
29:14
on Microsoft because they already have 365.
在 Microsoft 上，因为他们已经在用 365 了。
29:16
So they're okay with Microsoft
所以他们能接受 Microsoft
29:17
exactly because they've already done
正是因为他们已经做过
29:18
like whatever security review,
各种安全审查之类的，
29:20
etc, all of their documents
等等，还有他们所有的文档
29:21
and all of their word and whatever.
以及所有 Word 之类的东西。
29:22
So like Microsoft is an approved vendor in their mind.
所以，在他们眼里，Microsoft 是获批供应商。
29:25
And so we had to host everything on Azure
所以我们就得把所有东西都托管在 Azure 上
29:27
and that's changed over the past few years.
而过去几年，这种情况变了。
29:29
So like now people understand,
所以现在大家明白了，
29:30
okay, we need to get on cloud.
好，我们得上 cloud。
29:31
We need to do all this kind of stuff.
我们得做这一堆事情。
29:33
There's also a lot more people talking about on-prem back then.
那时候也有更多人在聊 on-prem。
29:35
That's also disappeared completely.
现在这也完全消失了。
29:36
People understand that like she's just doing the cloud
大家能理解，她就是在做 cloud 这块
29:39
and it's safe and it's probably safer
而且这很安全，可能还更安全
29:41
than what you're doing.
比你正在做的事更安全
29:42
Yeah.
嗯。
29:43
So that's changed out.
所以这已经变了。
29:43
But by the way,
不过顺便说一句，
29:44
it's only been three years.
这才过去三年。
29:45
So like it's actually like.
所以就是说，它实际上就像……
29:46
Yeah.
是啊。
29:47
Yeah.
是啊。
29:48
Yeah, it's a shoe shift.
是啊，这是一次 shoe shift。
29:49
It's great.
这很好。
29:50
I mean, good for you guys,
我是说，为你们高兴，
29:51
but yeah, I mean, I don't expect the like, you know,
但是啊，我是说，我不指望，你知道，
29:53
the legal firm at legal industry
法律行业里的律所
29:54
has somehow ignored this for the last like 10 years.
会在过去这大概 10 年里就这么莫名其妙地一直忽略这件事。
29:56
Yeah.
嗯。
29:57
Yeah.
嗯。
29:58
I think we needed the AI
我觉得我们需要 AI
30:00
chicken the butt for everyone to understand
chicken the butt，才能让大家都明白
30:01
that we need to change our ways a little bit.
我们需要稍微改变一下我们的方式。
30:03
Okay.
好。
30:04
Okay.
好。
30:04
So also then on like open source models
然后还有就是，关于 open source models 这类东西
30:06
now we're getting really good.
现在我们已经做得非常好了。
30:08
So yeah, how are you thinking about that with the?
所以，嗯，关于那个……你是怎么考虑那件事的？
30:10
Yeah.
对。
30:11
We're just thinking about it to choose your point.
我们只是在考虑这个，来选你的点。
30:14
Just bring our clients along on like the Pareto Frontier.
就是把我们的客户一起带到类似 Pareto Frontier 上。
30:17
Yeah.
对。
30:18
Like the map of all the models
就像所有 models 的地图
30:19
and what they're good at
以及它们各自擅长什么
30:20
and their latency and their cost
以及它们的 latency 和 cost
30:22
and their governance
以及它们的 governance
30:24
and their regional availability
以及它们的 regional availability
30:27
how do we just optimize across all of that.
我们怎么在所有这些方面统一优化。
30:29
So some open ways models are great for certain tasks
所以有些 open ways models 在某些任务上表现很好
30:31
and then we'll use them there.
然后我们就会在这些任务上用它们。
30:33
Some will post train
有些会做 post train
30:34
and some will issues to frontier models.
还有些会把问题交给 frontier models。
30:36
Yeah.
对。
30:37
You call it model arbitrage.
你管这叫 model arbitrage。
30:39
Yeah.
对。
30:40
We have different favoritism research on this
我们在这方面有不同的偏好研究，年初的时候你还坚定地站 Anthropic。
30:42
where you were firmly anthropic at the start of the year.
Grok 进步很大。
30:46
Grok is improving a lot.
对。
30:48
Yeah.
你想举一个 model 的例子吗？
30:49
You want to give an example of a model
你想举个例子说明一个模型
30:50
that wasn't rank highly overall
那个整体排名并不高
30:52
but like you want to shout it out as surprisingly good.
但你会想大声夸它意外地好。
30:55
Yeah.
对。
30:56
I think not overall good.
我觉得整体上不算好。
30:58
Gemini, one of the early Gemini's.
Gemini，早期 Gemini 中的一个。
31:00
Yeah.
对。
31:01
It's extremely good for certain types of transcription
它在某些类型的 transcription 上极其出色
31:04
and speech attribution.
以及 speech attribution。
31:05
So we use that for transcription.
所以我们就用那个来做 transcription。
31:07
Oh, so you have a
哦，所以你们有一个
31:09
very good answer.
非常好的答案。
31:09
I think we don't have,
我觉得我们还没有，
31:11
we actually probably will build like a meeting thing
我们其实可能之后会做一个类似会议的东西，
31:13
but this is just like talk to speech to text
但这只是像 talk to speech to text
31:15
and text to speech just like in any other app
和 text to speech，就像任何其他 app 里
31:19
that has this.
有这个功能一样。
31:19
And we do actually also transcribe files
而且我们其实也会转写文件
31:21
if you upload hearings or whatever
如果你上传听证会之类的
31:23
we'll then like transcribe them
我们就会把它们转写出来
31:25
for the mentor text and so we can do all of our analysis
给 mentor text 用，这样我们就能做我们所有的分析
31:27
on it.
都在它上面做。
31:28
No 11 labs.
不用 11 labs。
31:29
No 11 lab, poor DPA reasons.
不用 11 lab，DPA 方面的原因不太行。
31:31
Same thing is like a data processing agreement
这跟 data processing agreement 是一回事。
31:33
I should say.
我得说明一下。
31:34
So like our data processing agreement
就比如我们的 data processing agreement
31:36
has very few vendors on it.
上面只有很少的供应商。
31:37
Yeah.
对。
31:39
Exactly, we need to keep it super tight
没错，我们得把这一点控制得特别严
31:40
on where any data goes.
就是任何数据会流向哪里。
31:42
And so I think that is one of the benefits of like a Gemini
所以我觉得，这算是像 Gemini
31:46
and OpenAI where they're like,
和 OpenAI 这种的一个好处，他们会说，
31:48
you come to us,
你来我们这儿，
31:49
we have most of the things that you need
你需要的东西我们大部分都有
31:50
and that is actually very valuable.
而且这其实非常有价值。
31:52
Even if it's like it doesn't have to be like
就算它不一定非得是
31:54
still at the art,
state of the art，
31:55
it sounds like it's like mostly good enough for all tasks.
听起来它对所有任务来说基本都够用了。
31:58
Yeah.
嗯。
31:59
Yeah.
嗯。
32:00
Yeah.
是啊。
32:01
Yeah, absolutely.
是啊，完全同意。
32:02
I should actually shout out also Microsoft's
我其实还得特别提一下 Microsoft 的
32:03
transcription model.
transcription model。
32:04
They may want my transcribed.
他们可能想要我转录的内容。
32:06
Yeah.
是啊。
32:07
OK.
好。
32:07
It's great.
很棒。
32:08
It's better than the old Gemini.
它比旧的 Gemini 好。
32:10
Yeah, they did some good results.
是啊，他们做出了一些不错的成绩。
32:12
I don't know, don't even know like,
我不知道，甚至都不太清楚，就像
32:14
I think like usually all of them are like top two,
我觉得通常它们全都大概在前二、
32:16
top three, top four in terms of like the arena scores
前三、前四，就 arena scores 来说
32:19
and that's good enough.
而且这已经足够好了。
32:20
Yeah, exactly.
是啊，没错。
32:21
Yeah, all the governance and it's super scalable
是啊，所有的 governance，而且它超级 scalable
32:22
and it's like that's probably good enough.
而且感觉这大概已经够好了。
32:24
Yeah, I was going to say like at this point
是啊，我正想说，到这个时候
32:26
are pretty much the speech to text models
基本上就是 speech to text models
32:28
like good enough to where you're like,
好到你会觉得，
32:30
oh, one word here there, if it's off.
哦，就算这里那里有一个词不对。
32:32
No, no, no, no, for sure.
不不不不，那肯定。
32:33
Yeah, I've done this a lot.
是啊，这个我做过很多次。
32:35
I'm particularly indegal.
我特别不愿意承认。
32:36
There's like some, we have this thing where I think it's,
就有一些，我们有个情况，我觉得是，
32:39
I forget if it's like in Australia where there's like a certain name
我忘了是不是在 Australia，那边好像有个特定的名字
32:42
for like a person in court.
比如在法庭上称呼某个人。
32:44
And it would consistently transcribe it as condom.
然后它总是会把它转写成 condom。
32:46
Oh, and yeah, so there, yeah, there's certain things
哦，对，所以这里，对，有些东西
32:50
that you don't want there.
是你不希望出现在那里的。
32:52
Yeah, but so usually you run an LLM layer on top of it
对，但通常你会在它上面跑一层 LLM
32:54
and then you extract some keywords
然后再提取一些关键词
32:56
and then you run the transcription model
然后你运行 transcription model
32:58
again with the keywords.
再带着这些 keywords 跑一遍。
32:59
Like this is kind of my work so, so far.
这多少算是我的工作，所以到目前为止吧。
33:02
But yeah, if you've just done this on any large scale,
不过是啊，如果你只要在任何大规模场景下做这个，
33:05
you will pick up tons and tons of transcription errors
你就会发现成吨成吨的 transcription errors
33:07
on any domain that is not like average day-to-day conversation.
在任何不是普通日常对话的 domain 里。
33:12
Yeah, absolutely.
对，完全没错。
33:14
There's any sort of jargon in there.
里面只要有任何行话。
33:17
Yeah, it's gonna be tough.
是啊，这肯定很难。
33:17
Yeah, like I think Whisper and 11 Labs
是啊，我觉得 Whisper 和 11 Labs 还是会把 cloud code 转录成 cloud code。
33:21
still transcribe cloud code as cloud code.
是啊，而且这其实非常非常常见。
33:25
Yeah, and like it's actually very, very common.
是啊，所以你就知道是这样。
33:28
Yeah, so like that's how you know it.
还有很多要改进的地方。
33:30
There's like a lot more to improve.
是啊，绝对。
33:31
Yeah, absolutely.
我确实想稍微讲一点 agent 产品这边。
33:33
I do want to cover a little bit on the agent product side.
我还想稍微聊一下agent产品这一块。
33:36
So we've covered a few of the top-level stuff, right?
所以我们已经讲了一些顶层的东西，对吧？
33:39
We've already named job tabular review
我们已经把 job tabular review 命名为
33:41
with name jobs intake.
jobs intake 了。
33:43
And you've got some other stuff
而且你还有些别的东西
33:45
that we haven't even touched on yet.
我们甚至都还没碰过。
33:46
But I just want to cover the big two.
但我只想讲那两大块。
33:48
Let's call it, is that fair to say
这么说吧，这样说公平吗
33:51
do I do the big two?
那两大块是我来讲吗？
33:52
Agent and tabular review.
Agent 和 tabular review。
33:53
And our project.
还有我们的项目。
33:55
Yeah, and our word add in.
对，还有我们的 Word 插件。
33:57
Okay, it's also something
好，这个也算一个。
33:58
because a lot of people work inside of word
因为很多人都在 Word 里面工作，
34:01
and outlook at it and also.
还有 Outlook，也会看它，而且还有。
34:03
Yeah, so I just want to cover like the big ones
对，所以我就想讲讲那些重点的，
34:05
that everyone must know.
那些大家都必须知道的。
34:06
So tabular review, it's basically like fan out
所以 tabular review，基本上就像 fan out
34:10
of like a lot across each document.
把很多东西铺到每份文档上。
34:12
You can ask the same question across all documents.
你可以在所有文档上问同一个问题。
34:14
You select it in a table.
你在表格里选中它。
34:15
Yeah, and then work back and forth in the agent.
对，然后在 agent 里来回处理。
34:17
A million is impressive.
一百万挺厉害的。
34:19
What's any other like engineering considerations
还有没有其他类似 engineering 上的考虑
34:21
or performance things?
或者 performance 相关的东西？
34:22
Like what's hard about this?
这难在哪？
34:24
Before we do that, can you just like walk through
在做那个之前，你能不能就大概过一遍
34:26
like what is the user experience for a lawyer?
比如一个律师的用户体验是什么样的？
34:28
And then like, okay, you want to as a tabular review?
然后比如说，好，你是想做 tabular review 吗？
34:31
Yeah, yeah, like, so like what are they doing
对对，就是，那他们到底是在做什么
34:33
like with tabular review?
比如在 tabular review 里？
34:34
And then we can go into that.
然后我们可以再深入讲那个。
34:35
Sure, yeah.
当然，好。
34:36
So they, it's typically, I mean,
所以他们通常就是，我是说，
34:38
a good example is just a due diligence, right?
一个很好的例子就是 due diligence，对吧？
34:40
Whereas like we need to review all employment contracts,
就像我们得审查所有雇佣合同，
34:43
all supply contracts, all of whatever.
所有供应合同，所有这类东西。
34:45
And so employment contracts, you put
然后雇佣合同呢，你把
34:46
10,000 employment contracts in there.
10,000 份雇佣合同放进去。
34:48
And then you, like each cell is one employment contract.
然后你，就像每个单元格就是一份雇佣合同。
34:51
Each row is one employment contract.
每一行就是一份雇佣合同。
34:53
And then first cells, you ask questions basically.
然后在最开始的那些单元格里，你基本上就是提问。
34:56
So like the first question,
所以比如第一个问题，
34:57
the first, sorry, per column.
第一个，抱歉，是按列来的。
34:59
So the first column might be what's the termination period?
所以第一列可能是问 termination period 是什么？
35:02
And then it's going to fill out for everyone.
然后它就会给每个人都填好。
35:05
It's 30 days or whatever it is.
是 30 天还是多少来着。
35:06
And the next question is like, is there an IP assignment clause
下一个问题就像是，有没有 IP assignment clause
35:09
or whatever?
之类的？
35:10
Next one is like, how much, you know,
下一个就是，那个，你知道，大概要花多少，
35:12
how much do we have to pay them if we fire them?
如果我们把他们裁了，得付他们多少钱？
35:13
Or whatever it is, like, how much are they getting paid?
或者不管具体是什么，就是，他们现在拿多少钱？
35:15
And then it's going to fill out the entire thing.
然后它就会把整个东西都填好。
35:17
You can also just ask the agent to do this.
你也可以直接让 agent 来做这个。
35:19
Like if you just drop 10,000 files into the agent
比如你直接往 agent 里丢 10,000 个文件
35:21
and say I do a due diligence on this,
然后说，我要对这个做个 due diligence，
35:22
it will spin off a tabular review
它就会生成一个 tabular review。
35:24
and run all of them and wait for it to be done.
然后把它们全都跑一遍，等它跑完。
35:26
And then analyze the results.
然后再分析结果。
35:28
And it can then highlight red flags.
然后它可以标出 red flags。
35:30
OK, the CEO is getting paid an outrageous amount.
好，CEO 的薪酬高得离谱。
35:32
Or there's this weird bonus structure.
或者有个很奇怪的奖金结构。
35:34
OK, then obviously we need to look at that.
好，那显然我们得看看这个。
35:36
And then the agent from that can draft a memo
然后 agent 可以据此起草一份备忘录
35:38
to send to the client.
发给客户。
35:39
Or it can just do the review and then
或者它可以只做审查，然后
35:42
have a lawyer or the user just be like, OK, let's look
让律师或用户直接说，好，我们来看看
35:44
at this here are the red flags.
这个，这些就是警示信号。
35:46
There's also collaborative so that you can comment on it
还有协作功能，你可以对它发表评论
35:49
for other people to look at, et cetera.
方便其他人查看，等等。
35:51
And why do you choose that spreadsheet format?
那你为什么选那种 spreadsheet 格式？
35:54
Yeah.
是啊。
35:54
It's just a lot better to, it's super easy
它就是好太多了，超级简单
35:56
to work in.
可以在里面工作。
35:58
It's really easy to get an overview super quickly
真的很容易就能超快地了解整体情况
36:01
of what's going on and to review it.
知道发生了什么，并进行审阅。
36:03
So all of the stuff that we produce
所以我们产出的所有内容
36:04
also has citations back to source documents.
也都有指向源文档的引用。
36:07
And so you can basically review it super easily.
所以基本上你可以非常轻松地审阅它。
36:08
If you want to just make sure that everything's right,
如果你只是想确认一切都没问题，
36:10
you can also highlight cells.
你也可以高亮单元格。
36:11
These are reviewed or whatever.
这些是审核过的还是什么的。
36:13
These are the ones that are flagged
这些就是被 flagged 的那些，
36:14
and you can comment and collaborate.
然后你可以评论、协作。
36:16
That's much easier than any other form factor that we've tried.
这比我们试过的任何其他 form factor 都容易得多。
36:20
Yeah.
嗯。
36:21
I will say that the spreadsheet form factor
我得说，spreadsheet 这种 form factor
36:23
has a lot of pull in some areas,
在某些领域挺有吸引力的，
36:26
but somehow it hasn't really spread.
但不知怎么的，它一直没怎么真正普及开来。
36:30
I find it very weird.
我觉得这很奇怪。
36:31
Clay is the other one on the GT website
Clay 是 GT 网站上另一个
36:33
that we're very known for the spreadsheets.
我们做 spreadsheet 很出名的那个。
36:35
You guys, I don't really know if anyone else
你们，我真不知道还有没有别人
36:38
does a large spreadsheet AI thing.
在做大型 spreadsheet AI 这种东西。
36:40
No.
没有。
36:41
No.
没有。
36:42
There was V7 that tried to launch, there's like sheets.
之前有个 V7 试图上线，还有类似 sheets 的东西。
36:46
Even in finance domain, I don't really
即使在金融领域，我也没有一个真正叫得上名、专门做 spreadsheets 的顶级产品。
36:48
have a top name that does spreadsheets.
对，我同意。
36:51
Yeah, I agree.
我觉得这很棒，如果你有 fan out，而且你做的是完全一样的事情，并且你在提取东西，那它真是个很好的 form factor，能让你理解到底发生了什么。
36:53
I think it's a great, if you have fan out
你是不是发现你必须 export to Excel？
36:55
and you do the exact same thing,
然后你做完全一样的事情，
36:56
and you're extracting stuff, it's such a great form factor
你在提取东西，这种形态真的特别好
36:59
to understand what's going on.
能理解到底发生了什么。
37:01
Do you find you must export to Excel
你有没有发现你必须导出到Excel
37:03
or people are happy to live in your spreadsheet?
还是说，大家乐意住在你的 spreadsheet 里？
37:05
Do you happen to live there?
你碰巧就住在那儿吗？
37:07
Do you use AG grid?
你用 AG grid 吗？
37:08
Or do you use AG grid?
还是说你用 AG grid？
37:10
That seems to be the leading one.
那似乎是领先的那个。
37:11
I mean, they've done this for like, I don't know, 15 years.
我是说，他们做这个大概，我也不知道，有 15 年了吧。
37:13
Yeah, they know how to.
是啊，他们知道怎么做。
37:15
Yeah, so I'm happy on that spreadsheet angle.
对，所以从 spreadsheet 这个角度来说，我挺满意的。
37:18
I think the only other interesting thing
我觉得唯一另一个有意思的点
37:20
for me, engineering wise, is the fan outs.
对我来说，从工程角度看，就是 fan outs。
37:22
Engineers maybe can are well equipped
工程师也许还挺擅长
37:25
to understand the trade-offs of like, well, okay.
理解那种 trade-offs，比如，好吧。
37:28
Before you use fan instinct out on a million documents,
在你把 fan instinct out 用到一百万份文档上之前，
37:30
maybe test it on like two, and yeah,
也许先拿两份左右试试，然后，嗯，
37:34
does that work before you're going on?
在继续之前，这样行得通吗？
37:35
Do they do that?
他们会这么做吗？
37:36
Yeah, they will.
对，他们会的。
37:37
So our type of review actually will run the first 10 rows.
所以我们这种 review 实际上会跑前 10 行。
37:39
Yeah, just check, right?
对，就检查一下，对吧？
37:40
Yeah, does this make sense?
对，这合理吗？
37:41
Did it show us what we thought it was going to show us?
它有没有给我们展示出我们以为它会展示的东西？
37:44
Yeah, that's really good.
对，那真的很好。
37:45
Oh, but also for people when people built this,
哦，但还有对于人们来说，当人们构建这个的时候，
37:46
it's just really good for them to iterate quickly
能快速迭代对他们来说真的很好。
37:48
on the first 10 rows until like,
先在前 10 行上，直到大概，
37:49
they have a, they're getting the results that they want,
他们有，他们拿到了想要的结果，
37:51
and then they can go ahead.
然后他们就可以继续了。
37:53
Yeah, I am happy to move on to the next thing intake,
好，我很乐意接着聊下一个 intake，
37:55
which is your Slackbot.
也就是你们的 Slackbot。
37:57
Yeah.
嗯。
37:57
Everyone has a Slackbot with this and about yours.
每个人用这个都有 Slackbot，那你们的是什么样的？
38:00
So ours is just legal-specific.
所以我们的是法律专用的。
38:02
And what, what happens with ours is it's not actually,
而我们的情况是，其实并不是，
38:06
it might also, we'll probably also build one
它可能也会，我们可能也会做一个
38:08
that's just an interface to the agent,
只是作为 agent 的 interface，
38:09
but it's actually specifically to basically kick off
但它其实专门就是为了基本上启动
38:12
an agent that then adds a task to our list.
一个 agent，然后这个 agent 会往我们的列表里加一个 task。
38:16
So we have this list thing, which is like linear,
所以我们有这么一个列表的东西，很像 Linear，
38:19
that's the closest equivalent.
那是最接近的对应物。
38:21
And so it will add it to a specific intake list.
所以它会把它加到一个特定的 intake list 里。
38:23
And then lawyers and agents can go into the gory
然后律师和 agents 就可以深入那些又繁琐又糟心的细节，直接说：好吧，这些就是人们发给我们律所的，或者发给大公司里我们法务团队的那些东西。
38:26
and just be like, okay, these are the things
我们可以为这些事项启动 agents，可以围绕它们协作，然后直接把它们标记为完成，接着就会在 Slack 里发回去。
38:27
that people are sending to us as our law firm,
人们发给我们的，作为我们的律所，
38:30
or as our legal team in a big corporate.
或者作为大公司里的法务团队。
38:32
We can kick off agents for them,
我们可以为他们启动agents，
38:34
we can collaborate on them,
我们可以在它们上面协作，
38:35
and then we can just like mark them as done
然后我们就可以把它们标记为完成
38:36
and then it gets sent back in Slack.
然后它就会被发回Slack。
38:38
So it's more of a, it's not just like kicking off an agent,
所以它更像是，不只是像启动一个 agent，
38:41
it's more of like, I don't know, storefront thing.
它更像是，我也不知道，storefront 那种东西。
38:43
Or it's like, here's the ticket, I want this thing reviewed.
或者就像，这儿有个 ticket，我想让这个东西被 review 一下。
38:46
Okay, now it's, now it's here it is, it's back.
好，现在它，现在它就在这儿了，它回来了。
38:48
No one needs to change anything.
没人需要改任何东西。
38:49
And the legal team just sits in Legora
然后法务团队就坐在 Legora 里
38:50
and manages the agents and everything there.
管理 agents 和那儿的一切。
38:52
I think the data structure for this kind of thing
我觉得这类东西的数据结构
38:54
is really interesting,
真的挺有意思的，
38:55
because it's like basically the inbox, right?
因为基本上它就像是收件箱一样，对吧？
38:57
For all these kinds of queries
对于所有这些类型的 query
38:58
that can potentially translate into a thing,
它们可能会转化成一种东西，
39:01
into something else, into something else.
再转化成别的东西，再转化成别的东西。
39:03
And like, I don't know,
而且就像，我也不知道，
39:05
is it hard to integrate all these
把所有这些集成起来会很难吗
39:08
with all the other features,
跟所有其他 features 一起，
39:10
or do you just not worry about it?
还是你根本就不担心这个？
39:12
No, it's not.
不，不是这样的。
39:13
I think we have some primitives that work really well.
我觉得我们有一些用起来特别好的 primitives。
39:15
So like, lists is a great primitive,
所以比如说，lists 就是一个很棒的 primitive，
39:17
because it's essentially, it's like extremely flexible.
因为它本质上就特别灵活。
39:20
It's like a notion database.
它就像一个 Notion 数据库。
39:21
Yeah, a notion database.
对，一个 Notion 数据库。
39:22
It's like a notion database.
它就像一个 Notion 数据库。
39:23
And so you can basically configure it for whatever you want.
所以基本上，你可以把它配置成任何你想要的样子。
39:25
And we, of course, invest a lot into making this
当然，我们投入了大量精力，让它超级 scalable，而且运行得非常好。
39:27
super scalable and work really well.
你可以给它们设置 triggers、automations。
39:29
And you can have triggers on them,
它就像特别好的黏合剂，把所有东西都串在一起。
39:30
automations on them.
因为这样一来，只要列表上有什么事情发生，你就可以触发一次 agent run。
39:31
And it's like a great glue to glue everything together.
它就像一个很棒的粘合剂，把所有东西粘合在一起。
39:33
Because then you can kick off an agent run
因为这样你就可以启动一次agent运行
39:35
based on a list if something happens.
基于一个列表，如果某件事发生了。
39:37
And then that comes back and populates a field
然后它会返回并填充一个 field
39:39
or whatever, ask new documents or whatever it is.
或者随便什么，去请求新文档之类的
39:41
And so it's like, just like you might coordinate
所以这就像，就像你会去协调
39:42
a bunch of coding agents with linear
一堆 coding agents，用 linear
39:44
or some sort of task management tool,
或者某种任务管理工具，
39:45
we can do it similarly with lists.
我们也可以用 lists 做类似的事。
39:47
And so agents can create lists and lists can create agents.
所以 agents 可以创建 lists，lists 也可以创建 agents。
39:49
And engines can read lists.
而 engines 可以读取 lists。
39:51
And so it's like, the agent's warm comes alive.
所以就像是，agent 的 warm 活过来了。
39:54
Yeah.
是啊。
39:54
Do people get excited or overwhelmed by that?
人们会对此很兴奋，还是会觉得不知所措？
39:57
I feel like this is basically end user programming.
我觉得这基本上就是 end user programming。
40:01
You can let people set up crazy automations.
你可以让人们设置各种疯狂的 automations。
40:05
Some people get super obsessed by it,
有些人对它超级着迷，
40:07
and they become like, you know, it'd be cool to have
然后他们会觉得，你知道，要是能有
40:09
a lot of work in place.
大量工作都提前就位，那就太酷了。
40:10
Yeah, it has influencers.
是啊，它有网红。
40:12
Is that worth it?
那值得吗？
40:13
Overcomplicated?
搞得太复杂了？
40:14
We need the flexibility.
我们需要灵活性。
40:16
Even if we don't actually suppose to turn the end user,
即使我们其实并不打算面向终端用户，
40:19
let's say this was just all internal,
假设这完全只是内部使用，
40:21
then it's a really fast way for us to utilize people
那这对我们来说是一种非常快的方式，可以利用像
40:23
like our legal engineers to basically
我们的 legal engineers 这样的人，基本上
40:25
build a lot of products extremely fast.
以超快的速度做出大量产品。
40:28
So like you can build custom like
所以就像，你可以做那种定制的，比如
40:29
fund management dashboards.
基金管理 dashboard。
40:31
You can basically build that on list
你基本上可以在 list 上把这个做出来
40:32
with like your own generative UI.
再配上你自己的 generative UI 之类的。
40:34
And then you basically have a new product
然后你基本上就有了一个新产品
40:36
that some people are gonna love.
会让一部分人特别喜欢。
40:37
And we can have a market place of just like
然后我们就能有一个 marketplace，里面全是那种
40:39
a bunch of specialized products
一堆专门化的产品
40:41
that we can scale super quickly
而且能超快地 scale
40:43
because all of our legal engineers
因为我们所有的法务工程师
40:44
can just build new ones.
都能直接做出新的来。
40:45
If they're out with a client,
如果他们跟客户在外面，
40:46
oh, you want your own easy.
哦，你想要自己的？简单。
40:47
Like we'll just fix one for you.
比如我们就直接给你弄一个。
40:48
So it's like a lovable inside legal or a little bit.
所以这有点像法律领域里的 lovable，或者说有点像。
40:50
And if we don't wanna expose it,
如果我们不想把它暴露出来，
40:51
that's also how we build our own custom features.
这也是我们构建自己自定义功能的方式。
40:54
So we made another acquisition in the UK called Wexler.
所以我们在英国又完成了一次收购，收购了 Wexler。
40:57
And they're doing fact intelligence,
他们做的是 fact intelligence，
40:59
which is for litigation mostly,
主要是用于诉讼，
41:02
which is like you upload a million documents
就像是你上传一百万份文件
41:04
for a lawsuit or whatever.
为了一个官司或者别的什么。
41:06
And then we extract facts from those
然后我们从这些文件里提取事实
41:09
with like named entities and dates and whatever.
包括像 named entities、日期之类的信息。
41:11
And then when you have all the facts,
然后等你掌握了所有事实，
41:13
you can basically hold them up against each other
你基本上就可以把它们拿来互相比对，
41:15
and see which ones are corroborated
看看哪些是得到印证的，
41:16
and which ones are vertical disagreement.
以及哪些是 vertical disagreement。
41:20
There, you know.
你看，就是这样。
41:21
And that's extremely powerful
这非常强大，
41:22
because then you can also transcribe a court hearing.
因为这样你还能转录法庭听证会。
41:24
And you'd be like, oh, he set this thing,
然后你就会想，哦，这东西是他设置的，
41:26
but actually the email over here says that he was not there.
但实际上这边的邮件说，他当时不在那儿。
41:30
And if you see it live, it's like so fucking cool.
而且如果你现场看到它，那真的他妈太酷了。
41:33
And that we can also build on this primitive.
而且我们也可以基于这个 primitive 来构建。
41:34
And so we can build a lot of really cool things
所以我们可以构建很多非常酷的东西
41:37
on this one primitive basically.
基本上就是基于这一个 primitive。
41:39
That's how we think about it.
我们就是这么想的。
41:40
It's like we have to be as we scale the company,
这就像，随着我们 scale 公司，我们必须得……
41:42
we just have to solve so many things and like so hot.
我们就是得解决太多事情了，而且，怎么说呢，特别火。
41:44
So I think a lot about how do we build a core primitive
所以我经常在想，我们怎么打造一个 core primitive
41:47
that we can scale to too many different
能让我们 scale 到很多不同的
41:50
and products in great very customizable experiences.
产品，带来很棒、非常可定制的体验。
41:52
Okay, so what core primitives do you have?
好，那你们有哪些 core primitives？
41:54
Let's just restate for people.
我们给大家重新说一下。
41:56
This is a core one.
这是一个很 core 的。
41:58
Yeah, so we're gonna figure one.
对，所以我们会弄一个出来。
41:59
Yeah, projects, it's basically containers of files,
对，projects，基本上就是文件的容器，
42:02
file systems, the agent.
file systems，还有 agent。
42:04
Of course, you can kick that off
当然，你可以把它启动起来
42:05
and run that in very different configurations.
然后用非常不同的配置去运行。
42:07
Do you have a name for the agent, by the way?
对了，这个 agent 有名字吗？
42:09
No, we don't.
没有，我们没有。
42:10
It's an entity.
它就是一个实体。
42:11
No, okay.
没有，好吧。
42:12
You mean like we call it like a layout
你是说我们就叫它 layout，还是说像是给它起了个名字。
42:14
or like we gave it a name.
哦对，对对对。
42:15
Oh yeah, yeah, yeah.
嗯，layout 应该挺酷的。
42:16
Yeah, layout would be cool.
以前人们会把 layout 当作一个 named entity 来讨论。
42:18
People used to talk about layout
嗯。
42:20
as like a named entity.
不过是啊，我们后来不这么做了。
42:21
Yeah.
是的。
42:22
But yeah, we stopped that.
但是是的，我们停止了那个。
42:24
We were already back.
我们早就回来了。
42:25
It might be confusing, I don't know.
这可能会让人困惑，我也不确定。
42:26
So Liz, Agent, a tabular review.
所以 Liz，Agent，一个 tabular review。
42:28
Tabular review.
Tabular review。
42:29
Which like we're considering
也就是我们正在考虑的那个
42:30
is that actually an instantiation of like
它实际上是不是某种类似……的 instantiation
42:32
you can take this as far down as you want, right?
你想把这个往下做到多细都行，对吧？
42:34
But those are probably the big ones, yeah.
但那些大概就是最主要的部分了，嗯。
42:37
Yeah, so I actually wanted to talk a little bit about,
对，所以我其实想稍微聊一下，
42:40
like FT is because this sounds kind of like
比如 FT，是因为这听起来有点像
42:42
yeah, and I think Max has talked about that a little bit.
对，而且我觉得 Max 已经稍微聊过这个了。
42:45
Now like, okay, when you onboard
现在比如说，好吧，当你 onboard
42:46
a new like big customer,
一个像新的大客户，
42:48
you want to get them value like ASAP.
你会想尽快让他们获得价值，像 ASAP 那样。
42:49
So that, you know, they stay on the Gora, yeah.
这样，你知道，他们就会留在 Gora 上，对。
42:52
Yeah, how has that evolved?
对，那这方面是怎么演变的？
42:53
Like when did you guys start that forward to play team?
你们大概是什么时候开始搞那个 forward to play team 的？
42:56
So we hired our first forward to plan into your,
所以我们招了第一个 forward to plan into your，
42:58
that was our first time in New York,
那是我们第一次在 New York，
42:59
also YC Founder nine months ago maybe?
也是大概九个月前的 YC Founder？
43:03
I'm not actually sure.
我其实不太确定。
43:04
Are they considered people engineers or no?
他们算 people engineers 吗，还是不算？
43:06
No, okay, three engineers.
不算，好，三个工程师。
43:08
But we do, it is something that I think about
但我们确实会，这是我会去想的事。
43:10
is like, how do we exactly enable that at scale?
就是，我们到底怎么在 at scale 的情况下实现这一点？
43:13
And I think a really core thing is like,
而且我觉得一个特别核心的点是，
43:15
how can we empower our legal engineers?
我们怎么赋能我们的 legal engineers？
43:18
Like, how can we power our power users
就是，我们怎么赋能我们的 power users
43:19
and our legal engineers to customize the product
以及我们的 legal engineers，让他们能定制产品
43:22
in a powerful way?
而且是用很强的方式？
43:23
And then of course, we still have forward to play
然后当然，我们还有 forward to play 的工程师
43:25
the engineers to help out, you know,
来帮忙，你知道的
43:26
do all the customer integrations
把所有 customer integrations 都做了
43:28
and the the scoping and figuring out what's possible
还有做 scoping，搞清楚哪些是可行的
43:30
and feed all of that back into core product development?
然后把所有这些都反馈回核心产品开发？
43:32
So that's kind of how I think about it.
所以这大概就是我的思考方式。
43:34
It's like almost a first line, second line defense
这几乎就像是第一道、第二道防线
43:36
where it's like legal engineers should be able to
也就是说，法务工程师应该能够
43:38
customize a lot of stuff and build a great experience
定制很多东西，做出很棒的体验
43:41
and then where they can't or we need to make core product
然后在他們做不到、或者我们需要做核心产品的地方
43:42
changes or customer inspirations
改动，或者客户的灵感
43:44
or whatever we have forward to playing engineers
或者我们转给工程师去玩的那些东西
43:46
who also then cycle in a little bit behind like,
他们随后也会稍微晚一点加入进来，就像，
43:49
sometimes we'll just work on the product
有时候我们就只做产品
43:50
and sometimes the work with clients.
有时候则是跟客户一起工作。
43:52
Can you walk through like an example, like, okay,
你能带我们过一遍一个例子吗，比如，好吧，
43:54
so you just got this new client, like what step number one
比如你刚接到这个新客户，那第一步是什么
43:58
like are you guys like, okay, hey,
比如你们是不是就会，好吧，嘿，
43:59
we have a forward to play the engineer for you guys.
我们会安排一个 forward 来给你们当工程师。
44:01
They're gonna go build some kind of product for you.
他们会去给你们做出某种产品。
44:04
Like how does that work?
那这具体是怎么运作的？
44:05
So it's very different client to client, right?
所以每个客户的情况都很不一样，对吧？
44:06
But if it's a big client
但如果是大客户，
44:08
and they're super engaged and they like want,
而且他们特别投入，也很有意愿，
44:10
they have all these streams of things
他们有一大堆事情，
44:11
that they would want to do or they already have,
有想做的，也有已经在做的，
44:13
typically they have already tried to build something
通常他们都已经试过搭点什么东西了
44:15
and it's failed and they're like,
结果失败了，他们就会说，
44:17
this is what we, this is a problem
这就是我们，这就是个问题
44:18
that we'd like to solve.
是我们想解决的问题。
44:20
Then we'll basically staff them with legal engineers
然后我们基本上会给他们配上 legal engineers
44:22
and forward to play the engineer potentially,
然后可能还要 forward 去扮演 engineer 的角色，
44:24
depending on what, you know, the thing is
取决于，你知道，具体是什么东西
44:25
and then we'll just be like, hey,
然后我们就会说，嘿，
44:27
these guys will come, they'll make you successful.
这些人会来，他们会让你成功。
44:28
They will sit with all your lawyers and teach them
他们会跟你们所有律师坐在一起，教他们
44:30
how to use it and like figure out all the workflows
怎么用它，然后把你们现在所有的 workflows 都梳理清楚
44:32
that you have right now and build them into Lucora
再把这些 workflows 构建进 Lucora
44:35
and some of that will just be a legal engineer
其中有些部分只需要一个 legal engineer
44:36
or some of that will require a forward to play the engineer.
还有些部分则需要一个 forward 来干 engineer 的活。
44:39
What is that separation?
这个区分是什么？
44:41
I guess, if you could like talk about that a little bit.
我猜，如果可以的话，你能不能稍微聊聊这个？
44:43
Legal engineers configure the product
Legal engineers 配置产品
44:45
as a power use of wood.
作为 no-code 的 power user。
44:47
They don't ride code.
他们不写代码。
44:48
Yeah, yeah.
是啊，是啊。
44:49
And a forward to play the engineer
还有一个 forward deployed engineer
44:50
fried code in its production, basically.
基本上就是在 production 里写代码。
44:51
I guess that's the, yeah.
我猜就是这么回事，是啊。
44:53
What kind of background do the forward to play the engineers
这些 forward deployed engineer 都是什么背景？
44:55
normally have a regular engineer?
通常会有普通工程师吗？
44:57
Yeah, founders, regular engineers.
对，创始人、普通工程师。
44:59
You're old.
你年纪大了。
45:00
Yeah, D was your, yeah, exactly that is our first.
对，D 是你的，对，那正是我们的第一个。
45:03
Two WC companies and worked on search at MITTA.
两家 WC 公司，还在 MITTA 做过搜索。
45:05
Yeah, she's awesome.
对，她很棒。
45:06
Yeah, yeah.
对，对。
45:08
OK, that's a high bar.
好，那门槛很高。
45:09
She's great.
她很棒。
45:09
Yeah, yeah.
对，对。
45:11
Yeah, first boost on the ground in the US.
对，美国第一个落地的 boost。
45:13
Yeah, that's right.
对，没错。
45:14
What are some examples of, I don't know, workflows
有哪些例子，比如说，我也不知道，workflows 或者这些人在给客户做的东西？
45:16
or things that these people are building for the clients?
你们有没有，比如说，哦，这些就像，你知道，有几个 use cases 是几乎所有律所都……
45:19
Do you guys have, like, oh, these are like, you know,
你们有没有那种，就是，哦，这些就像是，你知道的，
45:22
there's a couple of use cases that like all the law firms
有几个use case是所有律所都在用的，
45:24
are like, you know, this is the first thing
就像是，你知道的，这是第一件
45:25
that our FDs are going to do when they go in.
我们的 FDs 进去后会做的事。
45:28
It's very different.
这很不一样。
45:30
Some core workflows everyone have and we can just
有些核心 workflows 是大家都有的，我们可以直接
45:33
build in Legora and that could be anything from like,
在 Legora 里搭建，而且可能是任何东西，比如，
45:36
this is how we handle these types of NDAs or whatever.
这是我们怎么处理这类 NDAs 之类的。
45:39
And then you activate it and it's more like skill building
然后你把它激活，它更像是 skill building
45:41
really for at least.
至少确实是这样。
45:43
And most of the FDs, that's more like,
而大多数 FDs 更像是，
45:45
there's something that we can't do yet.
有些东西我们现在还做不到。
45:47
We need some more flexibility and some of the primitives
我们需要更多灵活性，还需要一些 primitives，
45:49
or whatever it might be.
或者不管是什么。
45:50
Or we just have this really complex system
或者我们就是有这么一套非常复杂的系统，
45:52
that we need to integrate into.
需要我们把东西集成进去。
45:54
And then they'll go out there and they'll
然后他们会出去，
45:55
build the integration against our APIs
基于我们的 APIs 来 build integration。
45:56
and maybe expand our APIs or whatever to make that work.
然后也许扩展我们的 APIs 之类的，让那能跑起来。
45:59
And how does the, like, legal structure work there?
那，那个，法律结构那边是怎么运作的？
46:02
Because you said, you guys,
因为你说过，你们，嗯，
46:04
yourselves can't even work on deals.
你们自己甚至都没法做交易。
46:06
Yes, yeah.
是的，对。
46:07
So like, how did they work on client projects?
那，他们是怎么做客户项目的？
46:09
You mean the law firms, how do they work on client projects?
你是说律所，他们怎么做客户项目？
46:12
How do your FDs work on client projects?
你们的 FDs 怎么做客户项目？
46:14
So they don't touch the data still.
所以他们还是不会碰 data。
46:16
So they'll just build the system.
所以他们只会搭 system。
46:18
But they will also be onboarded like an employee sometimes.
但有时候他们也会像员工一样被 onboard。
46:20
So like they might get a laptop and everything.
所以比如他们可能会拿到 laptop 什么的。
46:23
Like Tuesday, too.
比如周二也一样。
46:24
Yeah, yeah, if they need to.
对对，如果他们需要的话。
46:25
Interesting.
有意思。
46:26
That doesn't happen very often to like,
这种情况不常发生，就那种……
46:27
actually they'll just build an integration
其实他们只会做一个 integration
46:28
and like kind of works.
而且差不多算是能用。
46:29
But yeah, they can if they need to.
不过对，如果需要的话他们也可以。
46:31
And you only onboard FDs into a customer,
而且你们只会在客户那边 onboard FDs，
46:33
I'm assuming if there is certain size.
我假设是在有一定规模的情况下。
46:35
Yeah, yeah.
对，对。
46:36
It needs to make sense because our FDs can also work
这得说得通，因为我们的 FDs 也可以参与
46:39
on core product, right?
核心产品，对吧？
46:40
So it's like a constant prioritization thing.
所以这就像是一个不断做优先级排序的事。
46:43
Is there a rule, like, I guess, does every,
有什么规则吗，比如，我猜，是不是每个，
46:46
like, you know, so the parents here have famously had,
就像，你知道，这里的父母们有过一个很有名的安排，
46:48
like, you know, four days you spent on site.
就是你在现场待了四天。
46:50
One day you come back and continue back to products.
过一天你回来，然后又回去继续做产品。
46:52
Anything like that.
有类似这样的吗？
46:53
No, we don't.
不，我们没有。
46:54
Yeah.
嗯。
46:55
I want you to be fully honest with you guys.
我想对你们完全坦诚。
46:57
It's still, it's still, you know,
现在还，还，你知道，
46:58
it's still early days in a lot of this.
很多方面现在还处于早期阶段。
47:00
I mean, where do you see it had a maybe over the next year?
我是说，你觉得在未来一年左右，它可能会发展到什么程度？
47:03
I do think that it's going to be more,
我确实觉得会更多，
47:06
maybe not exactly like a volunteer,
也许不完全像志愿者那样，
47:08
but like we, I think we will have a larger
但就像我们，我觉得我们会有一个更大的
47:11
in constant circulation group of FDs
持续流通的FDs群体
47:13
that way we sit with a client for like two months
这样一来，我们会和客户一起待上大概两个月
47:15
or whatever, to build out this one thing
之类的，就为了把这一个东西做出来
47:16
and then they go to the next one, et cetera, et cetera.
然后他们就转到下一个，等等等等。
47:18
Yeah, but I do imagine us rolling in with legal engineers
对，但我确实能想象我们带着 legal engineers
47:21
and forward-to-plotting engineers
和 forward-to-plotting engineers
47:22
to like work on specific projects.
去参与一些具体项目。
47:24
Yeah.
对。
47:25
The API business is actually interesting, like you said.
API 业务其实挺有意思的，就像你说的。
47:28
Because you know, that's another rule
因为你知道，那又是另一条规则
47:30
that you can have where you will sit
你可以有一条规则，规定你会坐在哪儿
47:32
yo maintainer of the Logura platform, right?
嘿，Logura 平台的维护者，对吧？
47:35
Like we will offer everything that we have
就是说，我们会把我们有的所有东西都提供出来
47:37
in the Logura web app as an API.
在 Logura web app 里，以 API 的形式。
47:39
And then you guys can use whatever.
然后你们想用什么就用什么。
47:41
Is that a promise they make or?
这是他们做出的承诺吗，还是？
47:43
That's not something we're promising now.
这不是我们现在在承诺的事。
47:44
Yeah.
是啊。
47:45
We do have APIs that you can build against,
我们确实有你可以基于它来开发的 APIs，
47:47
but not the entire surface of the product.
但不是产品的全部功能。
47:49
That's the curious thing, like is that the strategy?
这就是有意思的地方，这算是策略吗？
47:52
Or and obviously, you know, some people will love it
或者说，而且很明显，你知道，有些人会很喜欢这一点
47:55
because I don't have to learn a new tool.
因为我不用学一个新工具。
47:57
I can just use the existing tool I have, right?
我可以直接用我现有的工具，对吧？
47:59
And just install the Logura, for example,
比如，直接安装 Logura，
48:02
with the Word document stuff.
还有 Word 文档那些东西。
48:04
Other than that, you know, I do think like
除此之外，你知道，我确实觉得……
48:06
then you have to maintain it.
那你就得维护它。
48:07
Exactly.
没错。
48:08
So that's been my thing.
所以这一直是我的做法。
48:09
That's why we've tried not to do it.
这就是为什么我们一直尽量不做这个。
48:11
We haven't had APIs.
我们一直没有 APIs。
48:12
I mean, we started our APIs like six months ago or something.
我是说，我们大概是六个月前才开始弄 APIs 什么的。
48:14
We forward it for a very long time, mostly because,
我们把它往后推了很久，主要是因为，
48:17
yeah, I just don't want to, we move so quickly
是啊，我就是不想，我们推进得太快了
48:20
in the product that I don't want to be that slowed down
在产品上，我不想被拖慢到那种程度
48:22
by having to maintain all the API versions
因为不得不维护所有 API 版本
48:25
for like a data model that we've changed.
比如针对一个我们已经改过的 data model。
48:27
I mean, now we're bigger and things are different.
我是说，现在我们规模更大了，情况也不一样了。
48:28
So now it's, it's a cost that we can take on.
所以现在，这个成本我们承担得起。
48:31
But like when we're 20 engineers, it's just like,
但你想，当我们只有 20 个工程师的时候，那真的就像，
48:33
we move so quickly that we can't do this.
我们推进得太快了，根本没法这么做。
48:35
I've toyed with this, right?
我琢磨过这个，对吧？
48:36
Like it's just generally good for all companies
就是说，一般来说对所有公司都挺好的，
48:38
to have a platform team and a product team.
有一个 platform team 和一个 product team。
48:41
Because the product team, you have one,
因为 product team 嘛，你有一个，
48:43
but then you can easily spin up a second one
但之后你可以很轻松地再起一个
48:45
as long as the platform is not super tied to a product.
只要 platform 不是特别跟某个产品绑死。
48:47
Yeah, yeah, yeah.
对对对。
48:48
You know, like a very simple one is web app mobile app, right?
你知道，就是一个很简单的例子，web app、mobile app，对吧？
48:51
Or like web app mobile app, and then like, you know,
或者像 web app、mobile app，然后就是，你知道，
48:54
whatever, like other other integrations.
随便什么，像其他那些 integrations。
48:57
Then your FDEs just are one more client
那你的 FDEs 就只是多了一个 client
49:00
to the overall platform anyway.
接到整个 platform 上，反正就是这样。
49:01
Yes, yeah.
对，是的。
49:02
And I think like even the agent you can think about it
而且我觉得，甚至 agent 你也可以这么想
49:04
this way, there's like a harness team
这样去想，就像有一个 harness team
49:05
or like a, that runs the agent and that you can even,
或者像那种，运行 agent 的东西，而且你甚至可以，
49:08
you can build a product around it
围绕它做一个产品
49:09
with like skills and automations and whatever else.
加上 skills、automations 以及各种其他东西。
49:11
Yeah, this is like very famously the Amazon web services
对，这就像那个非常著名的 Amazon web services
49:15
mandate, like a business with a lot of things in the eyes.
mandate，就像一家有很多事情都摆在眼前的企业。
49:18
No more backdoor access.
不能再有 backdoor access。
49:19
You only offer other teams, your APIs,
你只向其他团队提供你的 APIs，
49:21
the same APIs you offer customers.
也就是你提供给客户的同一套 APIs。
49:23
If you don't have it, you build it.
如果没有，那就自己造。
49:24
Yeah.
嗯。
49:25
And then the customers also get it.
然后客户也能用上。
49:26
I think that's also changing a little bit.
我觉得这一点也在慢慢变化。
49:28
A lot of APIs now.
现在有很多 API。
49:31
Like one thing that we're talking with is like,
比如我们正在聊的一件事是，
49:33
can the agent be the API?
agent 能不能就是 API？
49:35
I know it sounds weird, but like for some of this stuff,
我知道这听起来有点怪，但就拿其中一些东西来说，
49:37
what they actually want to do is they
他们实际想做的是，他们
49:38
want to trigger certain things to happen.
想触发某些事情发生。
49:40
For some part of this, you could, you could just be,
对其中一部分来说，你可以，你可以就只是——
49:42
I'm not saying that we're doing this,
我不是说我们现在在这么做，
49:43
but it's like a thing that I'm talking with in my head
但这就像是我脑子里在琢磨的一件事，
49:45
is could that just be that, okay,
就是，那会不会就只是这样，好吧，
49:46
you trigger the agent and ask it to do this thing.
你触发这个 agent，让它去做这件事。
49:48
Like obviously it's way less efficient
就像很明显，这效率要低得多
49:50
than if you just had like an API that you always hit
相比你只是有一个总是会去调用的 API
49:52
and it's like, you know, but it's something
而且就像，你知道的，但这也是某种东西
49:54
that I'm talking with is like, how much of our product
我在聊的是，我们的产品有多少
49:55
can just be the agent or sold the other stuff?
可以直接就是 agent，或者把其他那些东西卖掉？
49:58
Well, you know, unbounded execution and cost and latency.
嗯，你知道的，unbounded execution，还有 cost 和 latency。
50:01
That's the, is that the only whole bet?
那就是——这是唯一的一个整体赌注吗？
50:04
Yeah, and then I think deterministic.
对，然后我觉得是 deterministic。
50:06
Like it may be in, you know, a hundred out of,
就像它可能会在，你知道的，一百个里面，
50:09
or in two out of a hundred times,
或者一百次里会有两次，
50:10
it's going to do something whack that we didn't want it to do.
它就会做出一些我们不想让它做的离谱事。
50:12
We're going to go into like harness
我们待会儿会讲讲像 harness
50:14
and model fine tuning in a bit.
和 model fine tuning 这些。
50:16
But I think there's a, there's a product question
但我觉得这里有个，有个产品层面的问题
50:17
about restraint, but what you don't do
关于克制，也就是你不去做什么
50:20
because you could do all that.
因为那些你其实都能做。
50:22
And obviously there's a prioritization issue like it.
而且很明显，这里有个像这样的优先级问题。
50:24
Are you at more than a point of, let's say no to most things
你现在是不是已经到了这种程度：对大多数事情都说 no，
50:28
and then we'll need to do a short number of stuff?
然后我们只需要做少量的事情？
50:30
Or no, actually we want to do more things
还是说，不，其实我们想做更多的事情，
50:32
and we need just need to hire like 100 people.
而且我们只需要招大概 100 个人。
50:35
I was, I was very strongly the first thing
我曾经，曾经非常坚定地是第一种做法，
50:38
for a very long time.
而且持续了很长时间。
50:39
Let's just say no to everything.
我们就对所有事情都说 no 吧。
50:40
Like keep the end stream.
就像保持 end stream 一样。
50:41
I think I've said this before, but like one of our presentations
我觉得我之前说过这个，但就像我们有一次 presentation，
50:45
that like after I've been at Ligor for half a year,
就是我在 Ligor 待了半年之后，
50:47
I pulled up this slide of, you know, the 300 Spartans
我翻出了这张幻灯片，你知道的，就是 the 300 Spartans
50:51
and for the movie.
那部电影里的。
50:52
And I was like, we're just going to be 20 engineers
然后我就说，我们公司以后就只会有 20 个工程师
50:54
for the rest of the company's history
在公司剩下的全部历史里
50:56
and like we're going to beat everyone as much bigger.
而且我们还要打败所有比我们大得多的人。
50:58
We're 100 engineers now
我们现在有 100 个工程师了
50:59
and it feels like we're way too small.
而且感觉我们规模太小了。
51:00
I want it to do that, but I think
我想让它做到那样，但我觉得
51:02
in the real world we're in like fierce competition.
在现实世界里，我们面临的是那种激烈竞争。
51:05
People want so much stuff.
人们想要的东西太多了。
51:06
It's like I could be a puritan
就好像我可以当个清教徒
51:08
and just be like, no, we have to stay 20 people
然后就说，不行，我们得保持 20 个人
51:10
and we can only build, you know, whatever.
而且我们只能做，你知道，随便什么。
51:12
But then we just lose and like people would go elsewhere.
但那样我们就只会输，然后人们就会去别的地方。
51:15
So we have to build more.
所以我们就得构建更多。
51:16
And so I think my job is to figure out
所以我觉得我的工作就是搞清楚
51:17
how do I build a system where we can build a lot
我该怎么构建一个系统，让我们能构建很多
51:20
but actually only like where we can,
但实际上只在我们能构建的范围内，
51:21
the customer feels like we've built so much stuff
让客户觉得我们已经构建了一大堆东西
51:23
but we've actually only built a few things.
但实际上我们只构建了少数几样东西。
51:25
That's how I think about my job right now.
这就是我现在对自己工作的理解。
51:27
Yeah, I mean, primitives is right, you know?
对，我是说，primitives 说得没错，你懂的？
51:29
Yeah, exactly.
对，没错。
51:30
I think that's going to be, that's really powerful.
我觉得那会非常，那真的会非常强大。
51:32
Do testing your products, just seeing if like,
先测试你的产品，就是看看，比如，
51:34
okay, there's like any pull here
好吧，这里到底有没有什么需求，
51:35
and then you just kind of like, you're like,
然后你就有点像是，你会觉得，
51:36
no, let's just focus on our main thing.
不，咱们还是专注做我们最主要的那件事吧。
51:38
Ooh, we haven't made, I mean,
哦，我们还没做，我是说，
51:41
we've made certain strategic bets,
我们做过一些战略性的押注，
51:43
but we've not tried anything
但我们还没试过任何
51:45
that we don't have a lot of confidence in already,
我们自己本来就没太大把握的东西，
51:47
mostly because we just have like a mountain of demand.
主要是因为我们手上的需求简直堆积如山。
51:49
That's like, we want all of these things
就是那种，这些东西我们全都想要，
51:51
and so it hasn't really made sense for us to try to be like,
所以对我们来说，去试着想“嘿，也许这东西能成”其实不太说得通，
51:53
hey, maybe this thing could work
因为有很多人都在大声喊着要
51:55
because like, there's so many people screaming
另外那些东西，所以我们基本上就做那些。
51:56
for the other things and so we mostly build those.
针对其他那些东西，所以我们主要做那些。
52:00
And then sometimes we'll disagree with clients
然后有时候我们会跟客户意见不一致
52:02
and we'll be like, actually, I think this thing
我们会说，其实我觉得这个东西
52:03
over here will make a lot of sense strategically
放在这边从战略上会很有意义
52:05
and then we'll try to build that anyway.
然后我们还是照样会试着把它做出来。
52:07
Like strategically for you.
就是从你的战略角度来说。
52:08
Yeah, exactly.
对，没错。
52:09
So what are those things?
那这些东西具体是什么？
52:10
So one of those things was the agent actually.
所以其中之一其实就是 agent。
52:15
So like, no one was actually asking for like,
所以就是，其实没人真的在要求，比如，
52:17
I want the assistant
我想要那个 assistant
52:19
that I'm using to have more agency and do things
我正在用的那个，能有更多 agency，能做事
52:21
without me asking it.
不用我开口要求。
52:22
Like no one's no one ever,
就像，没人，从来没人，
52:23
but we're just like, this is definitely
但我们就会说，这绝对就是
52:25
why the industry is going.
行业为什么会往这个方向走。
52:26
This is going to empower you to do so much cool stuff.
这会让你有能力做超多很酷的事。
52:28
Collaboration was another one where we're like,
Collaboration 也是另一个，我们当时是这么想的，
52:30
we think the industry is going to be a place
我们认为这个行业以后会变成这样一个地方，
52:31
where you can collaborate inside of where you're working,
你能在你工作的地方里面直接协作，
52:34
which is LaGoura and not in like emails and text messages.
也就是在 LaGoura 里，而不是在邮件和短信那种地方。
52:37
So yeah, there are a few of those.
所以对，有那么几个。
52:38
And then once you say, okay, yeah,
然后一旦你说，好吧，行，
52:41
we want you guys to use agents.
我们想让你们用 agents。
52:43
I'm assuming like internally at the law firm,
我猜是在律所内部吧，
52:45
it's going to take a lot for them to actually learn
他们真要实际学会怎么做，那得花很大功夫。
52:47
how to do that.
所以这时候你差不多就会说，
52:48
And so that's where you kind of like say,
嘿，FD 会去那边，教他们怎么做这些事。
52:50
hey, FD is going there and teach them what they do.
对，主要是 legal engineers。
52:52
Yeah, legal engineers, mostly.
哦，legal engineers。
52:53
Oh, legal engineers.
对，但他们有点像，有些人叫他们 forward to play legal engineers，如果你愿意这么说的话，
52:54
Yeah, but they're like some people call them forward
对，但有些人管它们叫forward
52:56
to play legal engineers, if you want,
to play legal engineers，如果你愿意这么说的话，
52:57
but that's essentially what they are.
但本质上它们就是这样。
53:00
Then they go in, yeah, and help them structure
然后他们就会介入，对，帮他们梳理
53:02
their workflows around the capabilities of the agent.
围绕 agent 的能力来搭建他们的 workflows。
53:05
You guys said previously, I think Max said this,
你们之前说过，我记得是 Max 说的，
53:08
but that early on specifically,
但特别是在早期，
53:10
you guys weren't fine-tuning your own models.
你们并没有 fine-tune 自己的 models。
53:13
Is that still the case?
现在还是这样吗？
53:14
Yeah, so we haven't fine-tuned any models.
对，所以我们没有 fine-tune 过任何 models。
53:16
I want to tell you now.
我现在想告诉大家。
53:17
I think now in the past few months,
我觉得现在，在过去几个月里，
53:19
open weights models have become extremely capable.
open weights models 已经变得极其强大了。
53:21
And so we will do it for certain tasks,
所以对某些任务，我们会这么做，
53:24
where we think there's a huge delta
在这些任务上，我们认为有一个巨大的 delta
53:26
between what a frontier model can do
介于 frontier model 能做到的
53:28
at the cost and latency,
在 cost 和 latency 上，
53:30
versus what we think a post-trained model can do.
和我们认为 post-trained model 能做到的之间。
53:33
Legal research is a great example.
法律研究就是一个很好的例子。
53:34
It's like, you have to make,
这就像是，你得做出，
53:36
it's a fan-out retrieval problem
这其实是一个 fan-out retrieval 问题，
53:38
and you can make a really effective model
而且你能通过那些 queries，超快、超便宜、以 low latency 做出一个非常有效的模型。
53:40
through those queries super quickly
所以具体来说，你是在对它们做 post-training，让它们更擅长 first specific bond tasks，对，就是这样。
53:42
and super cheaply at low latency.
而且超级便宜、低延迟。
53:43
So specifically, you're post-training them
所以具体来说，你们是在post-training它们
53:45
to be better at first specific bond tasks, yeah, exactly.
让它们更擅长首批特定的bond任务，对，没错。
53:49
Okay.
好。
53:50
We're not trying to train our own fable
我们不是想训练我们自己的 fable
53:54
to be the main agent.
去当主要的 agent。
53:55
I don't think that's like a,
我不觉得那像是，呃，
53:57
we won't win that versus centropic, most likely.
我们大概率赢不了 centropic 那个。
54:00
There's still a lot of,
还有很多，
54:01
let's say search and retrieval research or rag research
比如说 search and retrieval 研究或者 rag 研究
54:04
to do that is like, maybe less glamorous
要做，那可能没那么光鲜。
54:07
is not actually even model training sometimes.
有时候甚至都不是 model training。
54:09
It's just architecture and pipelines.
就只是 architecture 和 pipelines。
54:12
You did a case study with TurboParfour,
你和 TurboParfour 做过一个 case study，
54:14
but there's also,
但还有，
54:17
do you do a late interaction of?
你们会做 late interaction 吗？
54:19
Yes, absolutely.
是的，当然。
54:20
You do that.
你们会做那个。
54:21
Yeah, we actually don't do that,
对，其实我们不做那个，
54:22
but you're completely right that this is,
但你说得完全对，这确实是，
54:25
so there's one guy on the team's own
所以团队里就有一个人，
54:26
who is, he just really cares about search
他呢，就是特别在意搜索
54:29
and he basically comes to me all the time
他基本上老是来找我
54:31
and he's like, we should do these things
然后就说，我们应该做这些事
54:32
and there's new embedding models
而且还有新的 embedding models
54:33
and we can do this late interaction.
我们可以做这个 late interaction。
54:35
He actually talked to TurboParfour teams
他其实已经跟 TurboParfour teams 聊过了
54:36
to get them to build multi-vector
让他们去搭建 multi-vector
54:38
so that we could do these types of things.
这样我们就能做这类事情了。
54:40
But no, there's loads of low-hanging fruit
但其实不是，有很多特别容易见效的地方
54:42
generally even before models
一般来说，甚至在 models 之前
54:43
and even in the harness, honestly,
而且说实话，甚至在 harness 里也是，
54:45
if you just want to create a better output for our clients,
如果你只是想为我们的客户做出更好的 output，
54:47
there's so much stuff that we can do
我们能做的事情太多了
54:49
that has immediate impact before training models.
在 training models 之前就能马上产生影响。
54:51
What makes good harness for the law industry, I guess?
我猜，对法律行业来说，什么样的 harness 才算好？
54:55
You guys are making it,
这是你们在做，
54:56
assuming you're on custom harness.
假设你们用的是 custom harness。
54:57
Yeah, of course.
是啊，当然。
54:58
The way that we edit files is important.
我们编辑文件的方式很重要。
55:01
The way that we do an analysis is important.
我们做分析的方式很重要。
55:04
The way you do legal research is all baked into it.
你们做法律检索的方式全都内置在里面。
55:06
So it's like, you might have a problem
所以这就像是，你可能会遇到一个问题
55:07
that comes up or sees something
它碰到或看到某个东西
55:09
and then it knows how to go
然后它就知道该怎么去
55:10
and do the legal research to figure out
做 legal research，弄清楚
55:11
what the answer is, what the precedent is.
答案是什么，precedent 是什么。
55:13
That's all very important.
这些都非常重要。
55:15
And then, of course,
然后当然了，
55:15
there's all the governance stuff around it as well.
围绕它的还有一堆 governance 相关的东西。
55:17
Yeah, it seems like the search is probably,
嗯，看起来 search 可能大概是，
55:19
like would you say the most important tool there?
那你会说那里最重要的工具是什么吗？
55:21
You guys are really customizing
你们真的是在做定制化啊
55:22
because if you can't pull in the right inflammation
因为如果你没法把对的 inflammation 拉进来，那你就惨了，就惨了。
55:25
and then you're screwed, you're screwed.
我觉得是，是，你们在搜索
55:27
I think it's, it's, you guys are searching
像是在搜大量数据。
55:30
like a lot of data.
还有分析，对。
55:31
And analysis, yeah.
大概是搜索、分析，还有起草。
55:32
It's search, it's analysis and it's drafting, probably.
是搜索，是分析，大概还有起草。
55:35
Cause like drafting word,
因为就像起草文字，
55:36
drafting huge contracts is actually also not super trivial.
起草大型合同其实也没那么轻松。
55:39
Like what about it is not trivial?
比如说，它难在哪儿呢？
55:40
You have to, let's say you're drafting a huge contract
你得，比如说你在起草一份大型合同，
55:44
from scratch.
从零开始。
55:44
Then you have to find precedent of like,
然后你得去找先例，像是，
55:46
what do we do this previously?
我们之前是怎么做这件事的？
55:48
Do we need this type of clause or not?
我们到底需不需要这类条款？
55:49
A specific law firm.
某家特定的律所。
55:50
How did they do it previously?
他们之前是怎么做的？
55:51
Yeah, so I am the law firm.
对，所以我就是那家律所。
55:53
And I'm about to draft a huge, whatever.
然后我正要起草一份巨长的、随便什么吧。
55:55
NDA is very simple, but like there's,
NDA 很简单，但就像，有，
55:56
there are contracts that are way more complex.
有些合同要复杂得多。
55:58
And then you need to find the right precedent first.
然后你得先找到对的先例。
56:00
Cause like, Laura's just don't, don't like sit
因为就像，Laura's 就是，不，不喜欢坐着
56:02
and like stop typing stuff.
然后就像，别打那些东西了。
56:04
They figure out what clauses have we used in similar deals
他们会弄清楚我们在类似交易里用过哪些条款
56:07
and similar can mean a lot of different things.
而“类似”可以有很多种不同的意思。
56:08
As I'm sure you know, in search,
你肯定也知道，在 search 里，
56:09
it could be like, same industry, same type of companies,
可能是，同行业、同类型的公司，
56:12
same whatever.
同样的随便什么。
56:13
And so you need to identify the right precedent.
所以你需要识别出正确的先例。
56:15
You might have like multiple documents
你手上可能会有多份文档
56:17
that you then try to figure out which part should I take
然后你再试着搞清楚，我该取哪一部分
56:20
from what document?
从哪份文档里取？
56:21
And that's actually non-trivial.
这其实并不简单。
56:23
So it's essentially also a search problem.
所以它本质上也是一个 search problem。
56:25
Which model's best at drafting?
哪个 model 最擅长写草稿？
56:27
Yeah.
嗯。
56:28
So I'll say, I told that it's far free this yesterday.
所以我会说，我昨天说它 far free this。
56:31
Definitely fable and opus are extremely chatty.
fable 和 opus 肯定特别话痨。
56:33
I'm sure you guys have known this.
我确定你们已经知道这个了。
56:35
They're very verbose.
它们非常啰嗦。
56:36
That's not good.
这可不好。
56:37
That's generally not good.
这通常不太好。
56:38
No.
不。
56:39
Astro is pretty good at writing.
Astro 写东西挺不错的。
56:40
Astro is pretty good at writing.
Astro 写东西挺不错的。
56:41
Is it much more concise?
它是不是简洁多了？
56:43
And are you running like 100 different problems to like,
而且你是在跑大概100个不同的问题，像是，
56:46
be sure we can, yeah, we can do, we can do certain things.
确保我们能，对，我们能做，我们能做某些事情。
56:50
But I think the chattyness verbosity
但我觉得那种话多、啰嗦
56:52
is actually a little bit difficult for us to do.
其实对我们来说有点难做到。
56:53
So this is, I think, you know, let's see.
所以这，我觉得，你知道，看看吧。
56:56
But probably the next model from Anthropos
但很可能 Anthropos 的下一个 model
56:57
is going to be much better.
会好得多。
56:58
This Astro is actually very concise.
这个 Astro 其实非常简洁。
57:00
Yeah, yeah.
对，对。
57:01
And I assume they all have a Slack channel with you
而且我猜它们都跟你有一个 Slack channel
57:03
and you're telling me it's right.
而你还告诉我它是对的。
57:04
Yeah, exactly.
对，就是这样。
57:04
This is not news today.
这放到今天已经不算新闻了。
57:05
No, no, no, they know this.
不不不，他们知道这个。
57:07
And then the other thing that I really care about is hallucination.
然后我特别在意的另一点是 hallucination。
57:09
Do they make stuff up?
它们会胡编乱造吗？
57:10
So we have a system of a citation system
所以我们有一套 citation system
57:14
where we basically check what they're writing has a link.
基本上就是检查他们写的内容有没有 link。
57:17
So the user has a link.
所以用户那边会有一个 link。
57:18
They can click and it highlights with a bounding box
他们可以点一下，然后会用 bounding box 高亮出来
57:20
like in the PDF or whatever,
比如在 PDF 里或者别的什么里面，
57:22
where this information was taken from.
标出这个信息是从哪儿来的。
57:24
And we can sort of make sure
然后我们差不多能确保
57:26
that what it says always is citable somewhere,
它说的内容总能在某个地方被引用到，
57:29
even multiple layers down.
甚至能往下好几层。
57:30
So like if the agent writes something
所以就像，如果 agent 写了点东西，
57:32
based on a tabular review, you know,
是基于一个 tabular review，你知道，
57:34
it can cite the cell in the tabular view
它就能引用 tabular view 里的 cell，
57:35
or the row in the tabular view.
或者 tabular view 里的那一行。
57:37
You can actually like click multiple layers down
你其实可以往下点好几层，
57:39
and find, you know, sort of validated all the way up.
然后发现，你知道，差不多一路都验证到最上面了。
57:41
Yeah.
对。
57:42
Are you finding that lawyers are like verifying
你发现律师好像会去核实
57:44
that the citations are correct or not?
这些引用到底对不对吗？
57:46
In addition, it's kind of like the other word.
另外，这有点像另一个词。
57:48
But now, yeah, exactly.
但现在，是啊，没错。
57:50
Actually, I have the same thing.
其实，我也遇到过同样的事。
57:51
People should like hallucinated links to super in court.
人们好像会把 hallucinated 的链接交给法庭上的 super。
57:53
They don't care.
他们根本不在乎。
57:54
That happens.
这种事就是会发生。
57:55
Yeah, that happens.
是啊，这种事就是会发生。
57:56
I think it's like coding, you know,
我觉得这就像 coding 一样，你知道，
57:57
you have engineers that just don't give a shit
有些工程师就是压根不在乎
57:59
and just like send agents
就直接把 agents 发出去
58:00
and you have people that like sit and read every comma.
而有些人就会坐下来，把每个逗号都读一遍。
58:02
Sam, just like so much.
Sam，就特别多。
58:04
They're like, how can one here and do that, right?
他们就会想，怎么可能有人在这儿干这种事，对吧？
58:06
Yeah, but I think they probably just like we know
是啊，但我觉得他们大概就是，像我们知道的那样
58:08
in code review what to look for,
在 code review 里要看什么，
58:09
like what is the system or whatever.
比如 system 是什么或者之类的。
58:11
I think they also know, okay, this is the thing,
我觉得他们也知道，好吧，就是这个东西，
58:13
these are the red, like this is the thing
这些就是那些 red，就像，这就是那件
58:14
that's important.
重要的事。
58:15
Let me just like double check these things
让我就再确认一下这些东西，
58:16
and like these sound reasonable so it's probably fun.
而且这些听起来挺合理的，所以应该挺有意思。
58:18
Okay, interesting.
好，有意思。
58:20
Just like if you received something from a junior lawyer,
就像你收到一个初级律师给你的东西，
58:22
it's the same thing you do, right?
你做的也是一样的事，对吧？
58:23
Just like I should double check this
就像我应该再核对一下这个，
58:24
and like this probably is fun.
而且这大概挺有意思的。
58:25
Any other elements of let's call it legal reasoning
还有没有其他一些，姑且称之为 legal reasoning 的要素，
58:28
or what's the legal frontier that models are not good enough
或者说，models 现在还不够好的法律前沿是什么，
58:31
yet that you want from the next generation of models?
是你希望下一代 models 能做到的？
58:35
Nothing I can think of specifically right now.
我现在具体想不出什么。
58:37
I think long document reasoning is still difficult.
我觉得长文档 reasoning 还是很难。
58:41
Long like million token, token contracts is exactly
长到比如 million token 那样，token contracts 就完全是
58:43
like you have a huge contract.
就像你有一份巨大的合同。
58:44
Yeah.
嗯。
58:45
And like you're making edits across it,
而且你要在整份文档里到处改，
58:47
it's a difficult thing to do for a model today.
对现在的 model 来说这是件很难的事。
58:49
And I remember that you changed all these things up here
我还记得你把上面这些东西都改了
58:52
and like the definitions change
然后定义什么的就变了
58:53
and the ordering change and whatever else.
还有顺序变化之类的。
58:56
So the simplest would be one shot,
所以最简单的就是 one shot，
58:57
you'd feed everything in and then one shot I'll put.
你会把所有内容都喂进去，然后 one shot 输出。
59:00
Yes.
对。
59:00
You don't do that, you break it up into chunks
你不那么做，而是把它拆成 chunks，
59:02
and then you orchestrate edits.
然后再 orchestrate 这些编辑。
59:03
Yeah, exactly, exactly.
对，没错，没错。
59:05
And this is like one of those things
而这就像那类事情之一。
59:06
that are funny, like that are fun to be part of
有些挺搞笑的，就是参与进去会很好玩的那种
59:09
in this industry, right?
在这个行业里，对吧？
59:10
So our word at and we've done map reduced
所以我们的 word at，而且我们做过 map reduced
59:13
and we've changed and there was one big one
我们改过，而且曾经有一个大的
59:14
and there was like one big one in multiple sites.
而且就像在多个站点里都有一个大的
59:16
It's like we keep changing the scaffolding
就像我们一直在改 scaffolding
59:18
that we build around as models change.
随着模型变化，我们围绕它搭建的那套
59:20
Yes.
是的。
59:21
I would say I do a lot of news reporting,
我得说，我平时做很多新闻报道，
59:25
obviously because of what I do.
显然是因为我的工作。
59:26
And I've recently shifted from map reduced
而且我最近从 map reduced 转向了
59:28
to the entire thing.
整个东西。
59:30
In the context.
在上下文里。
59:30
Yeah.
嗯。
59:31
And it's better.
而且这样更好。
59:32
Exactly.
没错。
59:33
But it's also it's different.
但它也不太一样。
59:34
I mean, we had this problem with search where it's like
我是说，我们在 search 上遇到过这种问题，就是
59:36
you're searching for certain things in huge contracts
你要在超大的合同里搜某些东西
59:39
and you can map, reduce it.
然后你可以 map、reduce 它。
59:41
But then it might, if you're summarizing any sort of thing,
但这样一来，如果你是在总结任何东西，
59:43
then it might lose some important nuance
就可能会丢掉一些重要的细微差别
59:46
in between the chunks.
在这些 chunk 之间。
59:47
Essentially, if you just summarized individual parts up.
基本上，如果你只是把各个部分分别总结出来。
59:49
And so this has been a problem where it's like
所以这一直是个问题，就像是
59:51
we can do two types of search.
我们可以做两种 search。
59:52
We can also do the other one where it's like
我们也可以做另一种，就是那种
59:54
you need to have a reasoning in the different steps there
你需要在那些不同的步骤里有一个 reasoning
59:57
in between.
在中间。
59:58
So I'm trying to think of a really good example here.
所以我在试着想一个特别好的例子。
60:00
Yeah, there's some documents where they talk about
嗯，有些文档里他们会谈到
60:02
something at length over like
某个东西会讲得很长，类似
60:04
let's say two pages or whatever.
比如说两页之类的吧。
60:05
And then each individual chunk is not
然后每个单独的 chunk 并不是
60:07
to start with the relevant to the search term
一上来就和 search term 相关
60:08
or what they're trying to figure out.
或者跟他们想弄清楚的内容相关。
60:10
But together, it's extremely relevant.
但合在一起，就非常相关了。
60:12
And so, yeah, these are some of the, you know,
所以，对，这些就是一些，你知道的，
60:14
problems where it's, like you said earlier
问题，就像你之前说的，
60:17
to actually make the models better at that
要真正让 models 在这方面变得更好。
60:20
because you can't even look at the contracts, right?
因为你们根本连合同都看不了，对吧？
60:23
It's the real contracts.
那是真实合同。
60:24
So do you guys ever ask them for like,
那你们有没有问过他们，比如，
60:26
hey, can you make example contracts that are similar
喂，能不能做几份跟你们真实合同差不多的示例合同？
60:29
to your real ones?
对对。
60:30
Yeah, yeah.
所以我们会收到反馈，那你们能看到他们给的反馈吗？
60:30
So we have feedback and can you see feedback from them?
而且如果他们愿意配合的话，对，就是这样。
60:32
And if they're willing to engage, yeah, exactly.
如果他们愿意参与的话，对，没错。
60:34
Or they're just like, hey, this is like super important
或者他们就会说，哎，这对我们特别重要，但现在不太好用。
60:36
to us and it's not working well.
然后我们就会说，好啊，那你能不能把文档 anonymize 一下，然后把问题什么的也发过来
60:37
And then we're like, great, can you like anonymize
然后我们要是能 recreate 出来，就能修好。
60:38
the documents and like the problem then whatever
嗯，我是说，所以一种做法就是你买一家律所，有些人正在考虑这个。
60:41
and we, if we can recreate it, then we can fix it.
是的，但这里面有些，嗯，有些东西。
60:43
Yeah, I mean, so one version of this is you buy a law firm
嗯，我是说，一种版本是你买下一家律所
60:47
which some people are considering.
这是有些人正在考虑的。
60:50
Yes, but there's some, there's some stuff there
是的，但那里有一些，有一些东西
60:53
on whether their data is actually their data
关于他们的 data 到底是不是他们自己的 data
60:55
or if it's the client's data.
还是说，那其实是客户的 data。
60:57
So if the clients are all gone, then maybe it's fine.
所以如果客户都没了，那也许就没事。
61:01
I actually don't know.
我其实不知道。
61:02
Interesting.
有意思。
61:02
Yeah, but it's, yes, it's a way to get a lot of data.
是啊，但这也是，对，这是一种获取大量 data 的方式。
61:05
Yeah, people like Google is buying
是啊，像 Google 这样的公司在买
61:07
spirit. Yeah, exactly.
spirit。是啊，没错。
61:08
Yeah, yeah, they bought the, yeah, for like 10 to 10
对对，他们买了那个，对，差不多 10 到 10
61:12
of you know, yeah, like, you know,
你懂的，对，就像，你知道，
61:14
if you have a failed company, you still have your slack.
如果你有个失败的公司，你还有你的 Slack。
61:16
You can sell your slack.
你可以把你的 Slack 卖掉。
61:17
I think there's like a wastey company that like buys
我觉得好像有个 wastey 公司，会收
61:19
your failed company.
你失败的公司。
61:20
Yes, there is.
对，确实有。
61:21
Yeah.
对。
61:22
Yeah, cause like it's just like, oh, I just want,
对，因为就有点像，哦，我就是想要，
61:24
like, we're doing work.
就像，我们在干活。
61:25
Exactly.
没错。
61:26
And then just like exactly stuff up, exactly.
然后就正好把事儿搞砸了，没错。
61:29
The problem with buying a law firm data
买律所数据的问题是
61:31
is that you get all the inputs, the real inputs,
你会拿到所有 inputs，真实的 inputs，
61:33
which is great.
这很好。
61:34
The outputs are a little bit more difficult
outputs 就稍微难一点。
61:36
because you just get what their end result was
因为你只能拿到他们最终的结果
61:38
and it's difficult to figure out if like the way,
而且很难弄清楚，比如说那个过程，
61:41
you have to basically like reverse engineer
你基本上就得去 reverse engineer
61:43
how they got there.
他们是怎么走到那一步的。
61:44
So in our product, when we get real traces
所以在我们的产品里，当我们拿到真实的 traces
61:46
from really uses this extremely high value,
来自真实使用，这个价值极高，
61:48
but if we just got this output
但如果我们只拿到这个 output
61:49
and we don't know if it's good or bad
而且不知道它是好是坏
61:50
or how it was created, it's a little bit difficult.
或者它是怎么被创造出来的，这有点难。
61:53
We're going to wrap up soon.
我们很快就要收尾了。
61:54
The last thing I'll point people to is also,
我最后想让大家注意的一点也是，
61:55
there's a lot of alpha left on embeddings.
embeddings 上还有很多 alpha 空间。
61:57
If you're a rag heavy workload,
如果你的工作负载 RAG 很重，
61:59
you're just looking like MTB
你看起来就像 MTB，
62:01
and you're taking like top model.
而且你用的是 top model 那种。
62:03
You're leaving a lot on the table.
你很多机会都没利用起来。
62:04
You should look at like a voyage
You should look at like a voyage
62:06
or like some of the custom embeddings.
or like some of the custom embeddings.
62:08
You've come to this.
You've come to this.
62:09
Yeah, yeah.
Yeah, yeah.
62:10
So basically don't take default embeddings.
So basically don't take default embeddings.
62:11
Even like opening I embeddings,
Even like opening I embeddings,
62:12
Gemini embeddings doesn't matter.
Gemini embeddings doesn't matter.
62:14
Customize your embeddings, right?
Customize your embeddings, right?
62:16
Yes.
是的。
62:17
You'll support that.
你会支持这个的。
62:18
Yes, I think that's right.
是的，我觉得这是对的。
62:19
I think it's right.
我觉得这是对的。
62:20
The only thing is to make sure that you're multi-language.
唯一要确保的是，你要支持 multi-language。
62:23
So for us, we serve so many languages.
所以对我们来说，我们要服务这么多语言。
62:25
So that's the main problem.
所以这就是主要问题。
62:26
You're a busy king of this.
你在这方面可是个大忙人。
62:27
Yeah, exactly.
是啊，没错。
62:28
Mistral, helpful there.
Mistral，这方面挺有帮助。
62:30
No.
不。
62:33
Until now they just raised a ton of money, right?
直到现在，他们也只是融了一大笔钱，对吧？
62:36
So maybe.
所以也许吧。
62:37
Yeah, let's see.
是啊，看看再说。
62:38
They also raised a ton of money
他们也融了一大笔钱
62:39
and you're going to be your champions.
而你们会成为自己的冠军。
62:41
I think we need way more model companies out of Europe.
我觉得我们需要更多来自欧洲的 model companies，多得多。
62:44
Yes.
对。
62:45
I just wanted to ask you like what predictions
我就是想问问你，比如说
62:46
you have maybe like over the next year
你觉得未来一年左右
62:48
for like the wine industry.
葡萄酒行业会有什么预测。
62:51
We asked a little bit of that earlier.
我们之前稍微问过一点这个。
62:53
I'll modify it to your second question,
我会把它改成你的第二个问题，
62:55
which is in two years,
也就是两年后的那个，
62:56
will lawyers do their work inside LaGoura?
律师会在 LaGoura 里面完成他们的工作吗？
62:58
Will they use other tools like Cloud or Charity
他们会用像 Cloud 或 Charity 这样的其他工具
63:02
and call LaGoura, like the API?
然后像 API 那样调用 LaGoura 吗？
63:04
Yeah, yeah, yeah, yeah.
对，对，对，对。
63:05
I think both, it really depends.
我觉得两者都会，真的看情况。
63:07
I think if you're a big enterprise,
我觉得如果你是一家大企业，
63:08
you might have Cloud Co-Work
你可能会用 Cloud Co-Work
63:09
and then you'll, I think it's similar
然后你就会，我觉得这挺类似的
63:11
to like how I use Linear through Cloud,
就像我用 Linear 是通过 Cloud 这种方式，
63:13
but I still go to Linear.
但我还是会去用 Linear。
63:15
I think enterprise will do that.
我觉得企业会这么做。
63:16
I think law first will just use LaGoura.
我觉得 law first 会直接用 LaGoura。
63:17
Yeah, because that's similar and better.
对，因为那类似而且更好。
63:20
Yeah.
对。
63:20
It kind of reminds me of like open evidence
这有点让我想起 open evidence
63:22
for like medicine, like you're basically that for law.
就像医学领域的那个，你基本上就是法律领域的那个。
63:25
Yeah, yeah, exactly.
对，对，没错。
63:26
In coding, it's a point where people,
在 coding 里，有一个点，就是大家，
63:28
the labs will cut you off for competing with them
那些 labs 会因为你和它们竞争而把你掐掉，
63:31
and this has happened twice.
而且这已经发生过两次了。
63:32
Do you think that would happen to law?
你觉得法律领域也会这样吗？
63:34
No, I don't think so.
不，我不这么觉得。
63:35
I think because like, you know,
我觉得是因为，你知道，
63:36
they are releasing legal services, right?
他们正在推出法律服务，对吧？
63:38
Yeah, yeah, yeah, they are.
对对对，他们是。
63:39
I don't, I mean, we obviously talk to them a lot,
我不——我是说，我们显然经常跟他们聊，
63:41
but it's also, it's like, hey, are you gonna,
但这也，就像是，嘿，你会不会，
63:43
yeah, what?
是啊，什么？
63:44
Yeah, yourselves will say no.
对，你们自己会说不行。
63:46
Exactly, they don't make the call.
没错，不是他们拍板。
63:47
Exactly, that's not the, right?
没错，那不是，对吧？
63:49
And they don't even know
而且他们甚至都不知道
63:50
like often what's actually going on.
比如，实际经常发生的情况。
63:52
I don't think that's the case.
我不觉得情况是这样。
63:54
They have so much stuff to capture in knowledge work
他们在 knowledge work 里有太多东西需要覆盖，
63:56
or in sort of enterprises, knowledge work generally
或者说在企业里，广义上的 knowledge work，
63:59
that like legal has to be part of that,
以至于法务必须成为其中一部分，
64:01
just like data analytics has to
就像 data analytics 也必须参与一样，
64:03
and sales and marketing and finance.
还有销售、市场和财务。
64:05
So like they have to do things that are good there,
所以他们必须在那方面把事情做好，
64:08
but they're never gonna go as deep as we are.
但他们永远不会像我们一样深入。
64:10
And I think if Claude takes a small client,
而且我觉得，如果 Claude 接一个小客户，
64:13
small corporate clients
小型企业客户
64:15
and they do some of their legal via Claude,
然后通过 Claude 处理他们的一些法律事务，
64:17
like that's great, that's fine.
那挺好的，没问题。
64:18
But if you're doing a, you know,
但如果你在做一笔，你知道，
64:19
$400 million M&A transaction,
4 亿美元的 M&A 交易，
64:21
you're gonna want LaGora.
你就会想要 LaGora。
64:22
I had kind of an unrelated question,
我有个可能不太相关的问题，
64:25
but I'm curious, are you guys going to law schools
但我挺好奇的，你们会去法学院吗，
64:28
and kind of like getting them to like teach LaGora
然后有点像是让他们在课堂上教 LaGora，
64:31
in classes like how does that work?
这具体是怎么运作的？
64:33
Yeah, it's definitely a program that we are,
是啊，这肯定是我们正在做的一个项目，
64:36
I think we've started.
我觉得我们已经开始了。
64:37
I actually don't know exactly where we're at,
其实我不太确定我们现在具体到哪一步了，
64:38
but we do want people in law school
但我们确实希望法学院的人参与。
64:40
to learn to use LaGora.
去学习怎么用 LaGora。
64:42
Like are you seeing that like law schools
就是，你有没有看到法学院在课程设置上改了很多，还是其实也没怎么变？
64:44
are changing their curriculums a lot or not really?
我不知道，其实我也不太清楚。
64:46
I don't know, I wouldn't know, actually.
挺有意思的。
64:49
Interesting.
其实我会换个方式来做。
64:50
I would actually do it a different way.
你应该给法学院设奖学金，专门给 CS 专业的学生和工程师。
64:52
You should do scholarships for law school
你应该为法学院做奖学金
64:54
for CS majors, for engineers.
对于计算机专业的学生，对于工程师来说。
64:57
We will cover you've done CS.
我们会讲到你学过CS。
64:58
Yes, we'll put you in law school.
对，我们会把你送进法学院。
65:00
CS plus some software experience.
CS加上一些软件经验。
65:02
We'll put you in law school.
我们会把你送进法学院。
65:03
We'll cover it.
我们会讲到这个。
65:04
Or you do like the MA like masters plus JD programs.
或者你去读那种MA，比如硕士加JD项目。
65:09
Yeah, they don't care about the masters.
对，他们不在乎那个硕士。
65:11
I mean, you know, because there's a lot of people
我是说，你知道，因为有很多人
65:13
who didn't do CS undergrad,
本科没读CS的人，
65:15
so then they do like masters just to kind of like get the job.
然后就为了找工作去读个硕士之类的。
65:18
Yeah, yeah, I don't know.
对对，我也不太清楚。
65:20
I don't see that as often.
这种情况我不太常见到。
65:21
Yeah, yeah.
对对。
65:22
People do JD MBA, that's a thing.
有人读JD MBA，这确实有。
65:25
But JDCS, I haven't seen it.
但JDCS，我还没见过。
65:27
Happens, yeah.
有这种情况，对。
65:29
Yeah, okay, look, overall, you know,
对，好吧，你看，总的来说，你知道，
65:31
super impressive progress.
进展真的非常惊人。
65:32
I mean, congrats on all your success.
我是说，恭喜你取得的所有成功。
65:34
Like are you still going to keep up the pace?
那你还会继续保持这个节奏吗？
65:36
Like absolutely.
那肯定啊。
65:37
Yeah, you know, I think you're hiring in SF New York.
对，你知道，我觉得你们在 SF、New York 招人。
65:41
We're hiring in engineering.
我们在招 engineering 的人。
65:43
We're hiring in New York, Stockholm and London.
我们在 New York、Stockholm 和 London 招人。
65:46
Yeah.
嗯。
65:47
And all other roles we're hiring everywhere.
其他所有岗位我们也在各地招人。
65:50
How many people do you have in New York right now?
你们现在在 New York 有多少人？
65:52
100.
100。
65:53
Oh, okay.
哦，好的。
65:54
And you guys are planning on how many in New York?
那你们计划在 New York 有多少人？
65:56
In New York?
在 New York？
65:57
Yeah, just like generally.
嗯，就大概整体上。
65:58
Let's just say, yeah, generally, I don't know.
这么说吧，嗯，一般来说，我也不知道。
66:00
A lot.
很多。
66:01
As many as you can.
能弄多少就弄多少。
66:02
Yeah.
嗯。
66:03
It's aggressive.
这挺激进的。
66:04
Yeah.
嗯。
66:05
Yeah, I mean, that's the whole McCorpore law, right?
嗯，我是说，这就是整个 McCorpore law，对吧？
66:07
As far as like mid-Americans, New Yorkers.
就像美国中部的人、纽约人这种。
66:10
So like, yeah, I mean, I'm looking forward to you
所以，对，我是说，我很期待你
66:12
buying a law firm and, you know, becoming.
买下一家律所，然后，你知道的，成为……
66:15
Yeah, yeah.
对，对。
66:16
It's super exciting to, I think like, you know,
这真的特别让人兴奋，我觉得，就，你知道的，
66:18
every industry, especially like the top, like,
每个行业，尤其是那些顶尖的，就，
66:21
doctors, lawyers, coders, you know,
医生、律师、程序员，你知道的，
66:23
they all have to have, like, you know,
他们全都得有，就，你知道的，
66:25
through the agent labs that, like, just
通过那些 agent labs，就，只是……
66:27
you dominate the space.
你们主导了这个领域。
66:28
I agree.
我同意。
66:29
So you guys have like a great market.
所以你们算是有一个很棒的市场。
66:30
Yeah.
是啊。
66:31
Yeah, I think so.
对，我觉得是这样。
66:32
And I think it's only going to get much bigger
而且我觉得它只会变得大得多
66:33
as like law changes and becomes more available to more people.
随着法律变化，变得对更多人都更可及。
66:37
Yeah.
是啊。
66:38
Also, I think like, I mean, we're already like super over.
而且我觉得，就，我是说，我们已经，就，彻底结束了。
66:40
But like, the problems that you work on
但就，你研究的那些问题
66:43
are very like the frontier knowledge work, you know?
就非常像是前沿知识工作，你懂吧？
66:46
Yeah.
对。
66:47
I'm like, unless you're literally like doing, like,
我就觉得，除非你真的，就，在做，就，
66:49
science and science, never your stokes and science,
科学和科学，从来不是你的 stokes 和科学，
66:52
other than that, like, most work is, like,
除此之外，就，大多数工作，就，
66:55
basically converges to legal.
基本上都收敛到法律了。
66:57
Yeah, like, yeah, yeah, it's all legal in the end.
是啊，就，是啊是啊，最后全都是合法的。
67:02
Yeah, hiring, finance, everything.
是啊，招聘、财务，所有这些都是。
67:03
Like, I just have to consult my lawyer.
就，我还得咨询我的律师。
67:05
So yeah, I know, I know.
所以是啊，我知道，我知道。
67:06
Which is why I think it's a really exciting thing
这就是为什么我觉得这真的很让人兴奋
67:08
to work on because like modern society
去做，因为就像现代社会
67:09
runs in some way on legal.
在某种程度上是靠法律运转的。
67:11
Yeah, yeah.
是啊，是啊。
67:12
Cool.
酷。
67:13
All right, that's it.
好，就到这里。
67:13
Thank you.
谢谢。
67:14
Thank you.
谢谢。
67:15
Thanks.
谢谢。