TWIML AI
Airbnb Is Building a ‘Company Brain’ for AI Agents with Ahmad Al-Dahle - #781
00:003107
00:00
Security teams don't need another list of alerts.
安全团队不需要又一份告警清单。
00:04
They need to know which vulnerabilities are actually reachable, exposed, and safe to fix.
他们需要知道哪些漏洞是真正可达的、已暴露的、并且可以安全修复的。
00:09
That's what hyperscale remediation from Blitzy does.
这正是 Blitzy 的 hyperscale remediation 所做的事。
00:13
Blitzy understands your entire software state, then finds known and previously undiscovered vulnerabilities in your codebase.
Blitzy 了解你整个软件状态，然后在你的代码库中发现已知的和此前未被发现的漏洞。
00:20
It traces each one through the execution paths, checks the dependencies, and validates the fix.
它会沿着执行路径追踪每一个漏洞，检查依赖关系，并验证修复方案。
00:26
Your engineers approve the plan and Blitzy delivers batches of tested pull requests ready to review.
你的工程师批准计划后，Blitzy 会分批交付经过测试的 pull request，随时可以 review。
00:33
Less alert noise, faster remediation, and confidence that fixes are validated as safe before you review them.
更少的告警噪音，更快的修复，以及在你审查之前就能确信修复方案已被验证为安全的。
00:41
Visit Blitzy.com to see it in action, that's B-L-I-T-Z-Y.com.
访问 Blitzy.com 看看实际效果，网址是 B-L-I-T-Z-Y.com。
00:49
Your company knows a lot more than what's written down.
你的公司知道的远比写下来的多。
00:52
Experience employees know why a system works a certain way, which decisions apply,
有经验的员工知道系统为什么这样运作、哪些决策适用，
00:57
and how to ask when the documentation falls short.
以及当文档不够用时该去问谁。
01:00
Giving AI agents access to that knowledge is a much harder problem than just connecting them to a codebase.
让 AI agents 获取这些知识，比仅仅把它们连接到代码库要困难得多。
01:06
Ahmed Aldale helped build the original iPhone at Apple, and led the team behind Lama at Meta.
Ahmed Aldale曾在Apple参与打造初代iPhone，并在Meta领导了Lama背后的团队。
01:11
Now, as Airbnb's CTO, he's tackling that problem by building what he calls a company brain.
如今，作为Airbnb的CTO，他正通过构建他所谓的“公司大脑”来解决这个问题。
01:17
The system, Everest, captures institutional knowledge from employee's workflows, and makes it available to agents.
这个系统名为Everest，它从员工的工作流中捕获机构知识，并让agents能够使用这些知识。
01:24
But as that shared knowledge grows, how do you keep agents from learning the wrong lessons?
但随着这些共享知识不断增长，你如何防止agents学到错误的东西？
01:30
I think if you aren't careful, Markdown files could be exactly the kind of problem that I was referring to,
我认为，如果你不小心的话，Markdown文件可能正是我所说的那类问题，
01:35
where you have this difficult to find, and there's a lot of context poisoning that's happening.
也就是你很难找到它，而且会发生大量的context poisoning。
01:42
One person uses it, we extract a thing, we put it down a Markdown file, and then the model's very good at finding the thing that's old, or wrong.
一个人用它，我们提取出一个东西，把它写进一个Markdown文件里，然后模型非常擅长找到那个旧的或者错误的东西。
01:50
Because the biggest mistake you can make is you reveal too much context too quickly to the model,
因为你能犯的最大错误就是你向模型透露了太多上下文太快了，
01:54
versus you try to give it enough context to move one step forward to then ask for more context.
而不是你试图给它足够的上下文来向前推进一步，然后再要求更多上下文。
01:59
I'm Sam Charrington, and this is the Twimmel AI podcast.
我是Sam Charrington，这是Twimmel AI播客。
02:03
For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in.
十多年来，我一直通过像这样的对话来探索塑造AI未来的想法和创新，帮助你理解什么是真实的、什么是下一个、什么才是重要的。让我们开始吧。
02:15
When you arrived at Airbnb, talk a little bit about the AI landscape that you encountered, and how you've seen that evolve since you joined.
当你来到Airbnb时，谈谈你遇到的AI格局，以及自你加入以来你看到它如何演变。
02:37
So the good news is there was a lot of momentum already, there was a lot of pretty decent foundation, I would say.
好消息是，当时已经有很大的势头，而且已经有了相当不错的基础，我想这么说。
02:43
And I think it's both the timing that I joined, because really, Agente Coding had just begun to become something that people could really leverage.
我觉得这既跟我加入的时机有关，因为那时候Agentic Coding才刚刚开始真正被人们所利用。
02:54
So the moment and timing of the capability was right, but also I think the Airbnb culture, and you know, Airbnb's had a long lineage of working on AI, and MLOPS and all sorts of things.
所以能力出现的时机和节点是对的，但另外我觉得Airbnb的文化也很重要，你知道，Airbnb在AI、MLOps以及各种相关领域有着很长的积累。
03:06
So it had a strong sort of infrastructure investment. So what I've been able to help the company do is just really accelerate the adoption of AI across like maybe three frontiers or three boundaries.
所以它在基础设施方面有很强的投入。所以我能够帮助公司做的，就是真正加速AI在大概三个前沿或者说三个边界上的采用。
03:20
The first one is like AI for productivity, how does the average engineer use AI to move faster? That turns out to be like you have to sort of engineer your platform to make it easy to facilitate AI agents in your infrastructure.
第一个是AI用于生产力，普通工程师如何用AI来更快地推进工作？这实际上意味着你得对自己的平台进行工程化设计，让你基础设施中的AI agent能够被轻松地促进和运行。
03:36
And then the second one is, you know, how do we deploy AI agents to the marketplace? These are things that maybe the user doesn't see that really add a lot of a creative value to our platform, either build more trust, more reliability, quality, et cetera.
然后第二个是，我们如何把AI agent部署到市场上？这些东西可能是用户看不到的，但它们确实为我们的平台增加了大量的创造性价值，无论是建立更多的信任、更高的可靠性、质量等等。
03:51
These are things that you do in an offline fashion. And then how do we deploy AI agents to improve both the guest experience or the host experience?
这些都是在离线状态下做的事情。然后我们如何部署AI agents来同时提升guest体验和host体验？
04:02
So these are kind of like the three frontiers we pushed on. And each one's made like a tremendous amount of progress in a really short amount of time.
所以这些算是我们推进的三个前沿方向。每一个都在非常短的时间内取得了巨大的进展。
04:09
So when you think about that first pillar and AI for engineering productivity, you mentioned you kind of came on just as agente coding was taking place.
所以当你想到第一个支柱，也就是AI用于工程生产力时，你提到你大概是在agente coding刚刚兴起的时候加入的。
04:22
When you think about like how to help engineers be productive today with AI coding tools, like there are lots of different ways that you could come about that.
当你思考如何用AI coding工具帮助工程师在今天提升生产力时，有很多不同的方式可以实现。
04:31
I see lots of different philosophies about AI coding in the marketplace today. I'm not sure we've like fully nailed. This is the right way to do it or the one way to do it. How do you describe like the Airbnb way to do or even is there an Airbnb way to do it?
我看到当今市场上关于AI coding有很多不同的理念。我不确定我们是否已经完全确定了这是正确的做法，或者说是唯一的做法。你如何描述Airbnb的做法，或者甚至是否存在一种Airbnb的做法？
04:47
Or an omit way to do it or is it team my team?
或者是omit的做法，还是说是我团队的做法？
04:50
Well, I mean, every company has a culture set of principles that are a bit here too.
嗯，我是说，每家公司都有一套文化原则，这里也有点类似。
04:56
So, you know, other, you know, some companies will say like move fast, you know, that's kind of the mantra.
所以，你知道吧，其他一些公司会说，要快速行动，你知道，这算是他们的信条吧。
05:02
So what you'll find is the way that AI is being deployed for activities really about velocity.
所以你会发现，AI被部署到这些活动中的方式，本质上关乎速度。
05:08
I think Airbnb really focuses on craft, which I think is actually really, really good.
我觉得Airbnb真的很注重craft，我觉得这其实非常非常好。
05:14
It's a line to how I think about things. And so because we're very focused on craft, it's not just a velocity, but a speed question velocity actually has a heading which is good.
这是我思考问题的一条准则。因为我们非常注重craft，所以这不仅仅是速度的问题，velocity其实是有方向的，这很好。
05:25
It's really also about quality. So we kind of have to balance the equities of both.
这其实也关乎质量。所以我们得在两者之间权衡取舍。
05:30
And so when we look at productivity, we don't just look at, you know, PR through puts or token rates or things like that.
所以当我们看生产力的时候，我们不只看PR吞吐量或者token速率之类的东西。
05:38
We also look at defect ratios, reliability metrics. So we try to guard rail the velocity with craft and judgment to make sure that we are sort of continuing to stay on track.
我们也会看缺陷率、可靠性指标。所以我们试图用工艺和判断力来为速度设置护栏，确保我们大体上能继续保持在正轨上。
05:51
That way when you get exponential use of AI in your system, you're also like pretty much like reinforcing a culture that you kind of need to maintain the quality of your system as well.
这样一来，当你在系统中实现AI的指数级使用时，你同时也在强化一种文化，一种你需要用来维持系统质量的文化。
06:03
That's like one of the like very high level things that stays true everywhere. But I think there's some common truths in like how to deploy AI agents for your own productivity that like are consistent at every company.
这算是那种在哪儿都成立的高层原则之一。但我觉得在如何部署AI agents来提升自身生产力方面，有一些共同的真理，在每个公司都是一致的。
06:14
So you know, how do you basically like MCP your entire company so that an agent can sort of access what it needs.
所以，你基本上要怎么把整个公司MCP化，让agent能够获取它需要的东西。
06:22
Based on any generic task, how do you construct context in your code base or documentation within your code base so that agents are able to specialize on the work that they need to do.
基于任何通用任务，你如何在代码库或代码库中的文档里构建context，让agents能够专注于它们需要做的工作。
06:33
So if I am a product engineer and I need to like work on a PR that lands in the payment stack, which was like complicated and regulated and like kind of gnarly, is there enough documentation inform the agent so that your PR is actually kind of reasonable for the payments routine to review.
所以如果我是一个产品工程师，我需要处理一个落在支付栈里的PR，那个东西又复杂又有监管、还挺棘手的，那有没有足够的文档来告知agent，让你的PR对支付团队review来说实际上是合理的。
06:49
These are all things that you kind of need to build out in order to get like velocity at the end, like to actually get more productive and also maintain some level of quality.
这些都是你最终需要构建出来的东西，才能获得速度，才能真正提高生产力，同时也保持一定的质量水平。
06:58
And then the last one is like institutional knowledge, which isn't embedded in your organization really figuring out how to extract institutional knowledge encoded so that agents can learn from it.
然后最后一个就是机构知识，它并没有真正嵌入你的组织里，关键是怎么提取编码化的机构知识，让agent能够从中学习。
07:10
So we look under explored but something that we've been working on. So we have something that we call in a project Everest, which is effectively kind of like the company brains as people use AI.
所以我们觉得这块探索不足，但这是我们一直在做的事情。我们有一个叫Project Everest的东西，它实际上就像是人们使用AI时的公司大脑。
07:22
And as they use Everest, it's able to sort of collect a lot of institutional knowledge from everybody using it. So we have a shared context pool of how people do things that maybe isn't in the documentation.
当他们使用Everest时，它能够从每个使用它的人那里收集大量的机构知识。所以我们有一个共享的上下文池，记录人们做事的方式，那些可能不在文档里的东西。
07:32
What's the organizing principle there is a primarily for engineering and engineers. Is it workflow based? Is it very free form?
那里的组织原则是什么，主要是面向工程和工程师的吗？是基于工作流的吗？还是非常自由形式的？
07:42
No, it's for engineers.
不，是给工程师用的。
07:44
I started actually let me be clear. It started for product engineers, but it is now like dispersing across the company. So we even have product managers, which we call actually product marketers at Airbnb that are actually also using it.
我一开始，让我说清楚。它最初是给产品工程师用的，但现在已经在整个公司扩散开了。所以我们甚至有产品经理——在Airbnb我们其实叫产品营销——他们实际上也在用。
07:57
And we'll kind of like bring this as like the company company context graph that is that is like more about everybody's workflows and learning from everyone's workflows.
我们会把它作为公司的上下文图谱来推广，它更多是关于每个人的工作流，并从每个人的工作流中学习。
08:09
So saying that makes you think about.
所以这么说让你想到……
08:13
And many moons ago I was I worked in kind of the knowledge management field and like knowledge management and collaboration are these perpetual challenges within large enterprises like people, you know, create a lot of documents, but there's a lot of implicit knowledge that never gets written down.
很久以前，我曾在知识管理领域工作过，知识管理和协作在大型企业里是永恒的挑战，人们会创建大量文档，但有很多隐性知识从未被写下来。
08:33
And certainly AI and agents change all that because the agents can do some of that documenting for us and they can pull it from places where the work happens, as opposed to being told to do it, per se.
当然，AI和agents改变了这一切，因为agents可以替我们完成一部分文档化工作，它们可以从工作发生的地方提取信息，而不是被明确要求去做。
08:48
But at the same time, you know, at the same time the agents can create a lot of documentation and too much documentation or documentation about the wrong things kind of reduces.
但同时，你知道，同时agents会产生大量文档，而太多文档或者关于错误内容的文档反而会降低效率。
09:00
And you poison you poison so well, yeah, yeah, so talk a little bit about how you've kind of managed that that challenge and and how Everest approaches.
而且你会污染，你污染得非常好，对对，所以聊聊你是怎么管理这个挑战的，以及Everest是怎么处理的。
09:12
You know, kind of curation or prioritization, you know, however you think about that.
你知道，关于策展或者优先级排序，你怎么看这个问题。
09:18
I mean, ultimately the best way to keep your system on track is to have a really strong emails.
我的意思是，最终让系统保持正轨的最好方式是拥有非常强的evals。
09:24
So one of the things that we do with Everest is we have like consistent tasks that we we look at doing and we trying to benchmark those on, you know, are we, if you use Everest, are you able to complete the task and a shorter number and is it more accurate.
所以我们用Everest做的一件事就是，我们有固定的任务，我们去看这些任务并尝试做benchmark，比如，如果你用Everest，你能否用更少的步骤完成任务，而且是否更准确。
09:41
So is Everest, exactly one of those tasks to take a take what a product engineer would do in our pricing domain.
所以Everest，其中一个任务就是拿一个产品工程师在我们的定价领域会做的事情来做。
09:50
If they land that need to land a PR in our pricing domain with a prompt and they're like, you know, apply a new pricing rule change to this to the pricing domain.
如果他们能做到需要在我们pricing领域提交一个PR，用一个prompt，然后他们说，你知道的，给这个pricing领域应用一个新的pricing规则变更。
10:04
Is it able to complete that task given that pricing engineers have been working with Everest giving a context and doing their work with it.
考虑到pricing工程师一直在用Everest提供context并基于它工作，它能不能完成那个任务。
10:12
Is it able to land a PR that is like faster and more accurate as an example.
举个例子，它能不能提交一个更快更准确的PR。
10:19
So it's complicated because there's a lot of policies and a lot of very specific ways of working with pricing.
所以这很复杂，因为有很多策略和很多非常具体的pricing工作方式。
10:25
It's probably one of our most complicated domains. So is Everest able to let people work in the pricing domain encoding a lot of the institutional know how of how to work with something that's as complicated as pricing as an example.
这可能是我们最复杂的领域之一。所以Everest能不能让人们在这个pricing领域工作，把很多关于如何应对像pricing这样复杂事物的机构性know how编码进去，举个例子。
10:38
So the idea in that particular domain, you've got a lot of existing code that illustrates, you know, how the pricing aspects of your system work, but the code isn't everything.
所以在那个特定领域里的想法是，你有很多现有代码，展示了你们系统的pricing部分是如何运作的，但代码并不是全部。
10:49
There's also regulation and policy and other things and ideas to make that available to agents as well.
还有监管、政策以及其他东西和想法，让agents也能用上这些。
10:58
That's right. I mean, that's the like we want to get there. I think it's early, but I think this is to be clear.
没错。我的意思是，那就是我们想要达到的方向。我觉得现在还早，但我想说清楚一点。
11:04
Like it is not that Everest has like completely solved everything for us, but I think it is the beginnings of something that could eventually become, you know, empowering to every engineer who isn't an expert in everything.
不是说Everest已经帮我们把所有问题都彻底解决了，但我觉得这是一个开端，最终可能会变成，你知道，赋能给每一位并非什么都精通的工程师。
11:17
To basically move faster.
让他们基本上能更快地推进工作。
11:20
And do you think of that as a database of some sort or a collection of markdown files of some sort or something else?
你觉得那算是一种数据库，还是某种markdown文件的集合，或者是别的东西？
11:28
It's a database of sorts. Yeah.
算是某种数据库吧。对。
11:30
There's a question of what's the right way to structure it. I think markdown files are really good because frontier models have been trained to navigate file systems.
这里有一个问题，就是什么样的结构方式才是对的。我觉得markdown文件非常好，因为前沿模型已经被训练过如何浏览文件系统。
11:38
So actually, you can get pretty far. There's a question of like, what's the best structure for something like memory? I mean, this is kind of what it is. It's a little bit of a memory structure. And, you know, there's a question.
所以实际上，你可以走得很远。有一个问题是，对于memory这样的东西，什么是最好的结构？我的意思是，这大概就是它本质上的样子。它有点像一个memory结构。然后，你知道，这里有个问题。
11:48
So we've we're experimenting with the right data structure. It's a database is a better approach or actually a markdown file with like, and a harness kind of deciding what the directory structure needs to be is the right approach.
所以我们在试验合适的数据结构。是数据库是更好的方法，还是实际上一个markdown文件加上一个harness来决定目录结构应该是什么，才是正确的方法。
12:01
So I don't think we've like, I'm not convicted that we have the right architecture for it yet.
所以我不认为我们已经，我不确信我们目前已经有了正确的架构。
12:07
Do you think of it as primarily collecting the context and letting the agent figure out what to do with the context?
你是把它主要看作收集context然后让agent自己决定怎么处理这些context？
12:15
Or do you believe that kind of extracting some degree of structure and making that structure available to the agents is something that is a promising direction.
还是你认为提取某种程度的结构并把这种结构提供给agents是一个有前景的方向。
12:26
And I guess I'm wondering, you know, you're kind of creating vector embeddings on it and using it and embeddings to a lot of the agents to find context.
我想知道，你是在对它创建vector embeddings并使用它们，用embeddings让很多agent去找context。
12:34
Are you doing like graph kinds of things to allow the agents to navigate this or is it, you know, here's a bunch of files or way to like list a bunch of files in an area and a database.
你是在做类似graph那种东西让agent去导航，还是说，这里有一堆文件，或者某种在一个区域里列出很多文件的方式和一个数据库。
12:47
And you know, you'll think the agents as they get more and more powerful will be able to figure out what they need to do with it.
你知道，你会觉得随着agent越来越强大，它们能自己搞清楚该怎么处理这些。
12:54
The short answer on this I think is like, I think we're going to end up looking at all of these to figure out which one works best.
这个问题的简短回答，我觉得是，我们最终会把这些都看一遍，找出哪个最有效。
13:04
If you're, if you're not, I find it hard to believe like a vector search is going to scale to where we need it to work.
如果你不是这样的话，我觉得很难相信vector search能scale到我们需要它运作的那个程度。
13:13
Where like the whole company is using it, I think if you aren't careful, mark down files could be exactly the kind of problem that I was referring to where you have like it's difficult to find and is a lot of context poisoning the tapening one person uses it.
比如整个公司都在用它，我觉得如果你不小心的话，markdown文件可能正好就是我之前提到的那个问题，就是你很难找到它，而且有很多context poisoning，一个人用了它之后就会蔓延。
13:28
We extract a thing we put it down a mark down file and then the models like very good at finding the thing that's old or wrong.
我们把东西提取出来，放到一个markdown文件里，然后模型非常擅长找到那些过时的或错误的东西。
13:35
So I think we're going to end up like finding some hybrid where actually the there are some intelligence in between that is figuring out how to search.
所以我觉得我们最终会找到某种混合模式，实际上中间会有一些智能在 figuring out 如何进行搜索。
13:44
How to do like progressive disclosure of context is the biggest mistake you can make is you reveal too much context too quickly to the model versus you try to give it enough context to move one step forward to then ask for more context and like a progressive disclosure approach.
如何进行渐进式上下文披露——你能犯的最大错误就是过快地给模型透露太多上下文， versus 你试图给它足够的上下文来推进一步，然后再请求更多上下文，就像渐进式披露的方法。
14:00
And I think that is that is still kind of open in terms of the kind of tech stack or platforms that she used for agentic coding like have you talked about you know what what that is like are you an extra or why shop or does everybody use their own thing.
而且我觉得在agentic coding所使用的技术栈或平台方面，这仍然是开放的——比如你们有没有讨论过那是什么样的，你们是只用一家还是每家用自己的东西。
14:20
We started we started as one one shop at some point but we've quickly learned like given the pace of the industry it's a little crazy to be an X shop.
我们一开始是只用一家的，但很快我们就意识到，鉴于这个行业的发展速度，只做一家供应商的店有点疯狂。
14:31
So we've at least my my strong inclination was like we need to be a multi model shop.
所以至少我强烈的倾向是，我们需要成为一个多模型店。
14:40
So actually if you look across the company you know for internal productivity we have something called air chat which basically wraps other harnesses with our mcp's but we're able to sort of select which one we want to use and it's kind of nice because codex is also open source it's nice.
所以其实如果你看整个公司，你知道在内部生产力方面我们有一个叫air chat的东西，它基本上用我们的mcp's包装了其他的harnesses，但我们能够选择我们想用哪一个，这挺好的，因为codex也是开源的，这很好。
14:57
This is a look now to open source stuff so generally we're like pretty open and flexible with the engineer preference like people can kind of pick which which agetic tools they want to use and then for product we actually also have our own set of harnesses that we deploy.
现在这是对开源东西的一个视角，所以一般来说我们对工程师的偏好相当开放和灵活，人们可以挑选他们想用哪些agetic工具，然后对于产品我们其实也有我们自己部署的一套harnesses。
15:19
And and we also use open a lot of open source stuff that has our own modifications to it to make it work for our systems what are you doing around evals that's an area that a lot of companies really struggle with to ensure that they're able to kind of get that data flywheel.
而且我们也大量使用很多开源的东西，并加上我们自己的修改，让它适配我们的系统。你们在evals方面在做什么，这是很多公司真正挣扎的一个领域，为了确保他们能够获得那个数据飞轮。
15:37
Going one of the systems that I mentioned earlier zipline was feature engineering like and feature engineering plays a similar kind of data flywheel role in the ML ops platform is.
继续，我之前提到的一个系统zipline是特征工程，特征工程在ML ops平台中扮演着类似的数据飞轮角色，就像。
15:53
Eval tooling does today like how is the way Airbnb thinks about you know evil tooling and kind of closing the loop from interactions to.
Eval tooling今天所做的那样，Airbnb是怎么思考eval tooling以及从交互到。
16:06
To kind of intelligence how is that evolved for agetic tasks work we've worked with a set of engineers we have like a council of like senior engineers and we have a set of like tasks or prompts the people want to complete.
到智能的这种闭环的，这对agetic任务是如何演变的，我们和一组工程师合作，我们有一个类似资深工程师的委员会，我们有一组人们想要完成的任务或prompts。
16:19
The difficulty is like the challenge with the evals you could argue there's like a very binary outcome like did it build the thing or didn't it but there's also like a lot of subtleness to the correctness of the solution and the quality of the solution.
难点在于，就像evals面临的挑战一样，你可以说有一个非常二元的结果——它到底有没有把东西做出来——但解决方案的正确性和质量也有很多微妙之处。
16:34
And so you know we have built.
所以你知道我们已经构建了。
16:37
At like agent to key valves that look at engineering Airbnb engineering tasks I can think of like step off the top of my head but there are like a set of tasks that we've we've worked across various teams to engineer and then we run those evals against all of the models both open source and close source to have a sense of the parade of between the costs of the model and quality and the performance.
比如agent to key valves，它看的是工程类的Airbnb工程任务，我随口能想到一些，但我们有一系列任务，是我们跨各个团队合作设计出来的，然后我们拿这些evals去跑所有的模型，开源的和闭源的都跑，以便了解模型成本与质量和性能之间的pareto前沿。
17:07
And the hardest part is like really trying to push the parade down so that you're trying to not saturate because everything clutter is around clusters around like 92 93% or something like that which a virtual judge is giving you the score so we calibrated a virtual judge to basically say we're given this task given the execution like what's the quality so we get a percentage so we have this like chart that we use that basically shows cost versus accuracy.
最难的部分是真的要努力把pareto前沿往下推，这样你就不会饱和，因为所有东西都聚集在92、93%左右，而这是虚拟judge给你的分数，所以我们校准了一个虚拟judge，基本上就是说给定这个任务、给定执行情况，质量如何，这样我们得到一个百分比，所以我们有一张图表，基本上展示成本对准确率。
17:37
And then we look at all of the open source models and we decide it sounds like without a lot of care what you're seeing is that the model generally say oh that's about 90% good.
然后我们看所有的开源模型，我们决定——听起来好像不用太仔细看，你看到的就是模型一般会说哦差不多90%不错。
17:49
Yeah yeah that's the hard part exactly and getting enough separation between them is like sometimes a chore which actually either says one of two things either evals aren't good or the models are actually probably good enough for a lot of stuff that we're doing.
对对对，这就是难的部分，没错，而且让它们之间有足够的分隔有时候挺费劲的，这其实说明两件事之一：要么evals不够好，要么模型对我们正在做的很多事来说可能确实已经足够好了。
18:05
And then it becomes like a cost and efficiency trade off which and speed trade off really.
然后这就变成了成本和效率的权衡，以及速度的权衡，真的。
18:10
But I think most of the time I'm very suspicious that evals if we're not able to get some separation between frontier and non frontier are these hard enough are really like really calibrating the judge to look for enough signal and the implementation or the completion.
但我觉得大多数时候，我非常怀疑的是，如果evals无法在前沿和非前沿之间做出一些区分，那这些测试够不够难，是不是真的在好好校准judge去找足够的信号，以及实现或者completion。
18:27
A stick into the model costs versus capability a bit deeper you know there have been kind of some famous examples and well you know Uber you know for example you know told all their engineers to go to town on a genticoatting and then like some period of time later was like whoa you guys really went to town on a genticoatting this is expensive.
再深入一点说模型成本与能力的权衡，你知道有一些著名的例子，比如Uber，你知道，他们告诉所有工程师放手去搞agentic coding，然后过了一段时间就说哇你们真的在agentic coding上放开了搞，这很贵。
18:50
Like how do you think about kind of cost versus value versus for obvious reasons the topic of routing has gotten very popular nowadays which is essentially like how do we get the right test of the right model like what have you done to try to optimize there.
比如你怎么看待成本与价值的权衡，出于显而易见的原因，routing这个话题现在变得非常流行，本质上就是如何让正确的测试用上正确的模型，你在优化这方面做了什么。
19:10
I fundamentally think that engineers are problem solvers and you just have to inspire the objective not constrain constrain the execution.
我从根本上认为工程师是问题解决者，你只需要激发目标，而不是约束执行。
19:21
So like things that I adhere to is like we didn't have token leader like we didn't really focus on token leaderboards the only thing that tokens tell me is if someone's not using any tokens that is probably something to work on.
所以我坚持的事情是，我们没有token leader，我们没有真正关注token排行榜，token唯一告诉我的就是，如果某人完全没用任何token，那可能就是需要改进的地方。
19:38
And if someone's using a lot of tokens we should try to understand what they're doing and either by the way we found like two cases one case where people needed help to better understand how to build efficiency into their workflows and others were actually they're doing really great migrations and it's actually really valuable and like go ham and so I like didn't apply a draconian token limit to everybody I didn't really say token max we didn't do any of those fall into any of those bills.
如果有人用了大量token，我们应该试着理解他们在做什么，而且顺便说一下我们发现了两种情况，一种是人们需要帮助来更好地理解如何把效率融入他们的工作流，另一种是他们实际上在做非常好的迁移，而且真的很有价值，那就放手去用。所以我并没有对所有人施加一个严苛的token限制，我没有真的说token上限，我们也没有做任何那些落入那些条款的事情。
20:07
What we did was try to apply nuance to understanding the shape of adoption the thing is like my theory on the right way to approach it is you know people aren't using it need need help and need support.
我们做的是尝试对采用形态的理解施加细微差别，问题是，我关于正确方法的理论是，你知道人们没在用就需要帮助、需要支持。
20:21
People are using a lot of it we need to understand and people who are using and you look for other signals like you know people who are using one model pinned imagine you're using you know name your favorite frontier at maximum thinking.
人们在大量使用，我们需要理解，而对于那些在用的人，你会寻找其他信号，比如你知道有人在用一个固定的模型，想象你在用你最喜欢的frontier模型以最大思考模式运行。
20:35
Yeah yeah and it's like it's like these large you know you call it 10,000 tool calls and every sub agent is like your frontier model and it's like at some point you're like hey like that's slow you know that's slow right and also like are you sure you need that much and so you know we kind of look for those outliers and help bring them into the shape that we're actually going for.
对对对，就像这些大量的，你称之为10,000次工具调用，每个sub agent都像你的frontier模型，然后到了某个时候你会说嘿这很慢，你知道这很慢对吧，而且你确定你需要那么多吗，所以我们就是寻找那些异常值，帮助把它们引导到我们真正想要的方向。
21:00
In terms of like impact I mean this is like what the whole industry is trying to figure out I don't know that we've like completely nailed it which is like okay so you're using all these tokens are they a creative are they actually adding you know growth or velocity what we try to do is not anchor usage we try to anchor on like the downstream metrics so you know we have like a pretty rigorous way of looking at like okay.
就影响而言，我的意思是这就是整个行业都在试图搞清楚的事情，我不确定我们是否已经完全搞定了，就是好吧，你在用所有这些token，它们是有创造性的吗，它们是否真的在增加增长或速度，我们尝试做的不是锚定使用量，我们尝试锚定下游指标，所以你知道我们有一种相当严谨的方式来看，好吧。
21:27
For software engineers do we see PR throughput you know increasing because that shows that the output is increasing which is a good sign that they're producing more but it's not a sign that they're producing more useful stuff is just a producing work.
对于软件工程师，我们是否看到PR吞吐量在增加，因为那表明产出在增加，这是一个好迹象说明他们在产出更多，但这并不是他们在产出更有用的东西的迹象，只是产出了工作。
21:43
And then I think we look at like actual road maps the best indicator that you're using AI well is your roadmap is shifting to the right and compress it.
然后我觉得我们看实际的roadmap，衡量你用AI用得好不好的最好指标就是你的roadmap在往右移，而且在压缩。
21:53
And so like we track that and I think we actually we shared so many confounding factors and everything you're saying now.
所以我们追踪这个，我觉得我们其实——你刚才说的那些，有太多confounding factors了。
22:01
It is 100% your granularity road map distance like yeah oh yeah totally but I think what we feel like what matters is that we feel that we're able to do more and shorter amounts of time and I think that has been true.
这完全是你的granularity roadmap distance——对对对，完全是这样，但我觉得我们感觉重要的是，我们觉得我们能在更短的时间内做更多的事，我觉得这一点是成立的。
22:18
But I to your point there is no perfect metric on whether your AI use is resulting in business impact but there is like trend lines that you can look at that like again the confounding factors almost don't matter if you're able to shift more faster.
但你说的对，关于你的AI使用是否带来了business impact，没有完美的metric，但确实有一些trend lines你可以看，同样地，如果你能更快地推进更多事情，confounding factors几乎就不重要了。
22:34
AI has contributed I also think by the way we underestimate how much there's been a shift in how we work it's not just about oh yeah I'm using lots of tokens but like now you know when you think about product development.
AI确实有贡献，另外我觉得我们低估了工作方式发生了多大的转变，不只是说哦对我在用很多tokens，而是现在你知道，当你想到product development的时候。
22:47
The old days used to be like you have like an MLE and MLE would work with a product engineer who would then work with a designer who would then work with a product manager and then the legal and then a policy.
以前是这样的，你有一个MLE，MLE会和一个product engineer合作，然后product engineer再和一个designer合作，然后designer再和product manager合作，然后是legal，然后是policy。
23:00
And you know each one's producing like a different artifact that needs to be cascaded across each other but now you can just jump straight to the project.
你知道每个人产出的像是不同的artifact，需要互相级联传递，但现在你可以直接跳到项目上。
23:10
Everyone's just like right in code including the designers with the pms, gluing policy, gluing legal like everybody can just jump in to better understand what is the implementation versus trying to pre negotiate what might be at the end.
大家都直接进入代码，包括设计师和PM一起，把policy接进来，把legal接进来，所有人都能直接参与进来，更好地理解实现是什么，而不是试图预先协商最终可能是什么样。
23:24
And then like slowly kind of going through that every time I hear you say the roadmap pull in like I feel like the road map should be expanding because you can be bolder and think bigger about what you want to build.
然后慢慢地推进这个过程，每次我听到你说roadmap在收缩，我就觉得roadmap应该扩张，因为你可以更大胆、更宏观地思考你想构建什么。
23:37
And if it's pulling in.
如果它在收缩的话。
23:40
That is true of companies that don't have more things to do than people.
对于那些事情比人少的公司来说，确实如此。
23:47
We have roadmap is so deep.
我们的roadmap太深了。
23:51
Like the most I can do is pull it in and try to get it done.
我能做的就是尽量收拢一下，把它做完。
23:57
There's a lot to do there's a lot to do.
要做的事情很多，要做的事情很多。
24:01
And so we feel you're right.
所以我们理解你，你说得对。
24:04
I mean there's two ways to look at it. Are we doing more or are we like pulling things in?
我的意思是，可以从两个角度看这件事。是我们做得更多了，还是我们在收拢事情？
24:08
I think we have a lot of ambition.
我觉得我们有很多野心。
24:10
I mean Airbnb wants to reinvent itself.
我是说，Airbnb想重塑自己。
24:12
Our roadmap is like pretty insane.
我们的路线图简直有点疯狂。
24:14
We're trying to move from a core marketplace of stays to like everything that you can do in the physical world is now on our platform to kind of reinventing the entire tech stack
我们正试图从一个核心的住宿市场，扩展到你在现实世界中能做的所有事情都在我们的平台上，再到某种程度上重新发明整个tech stack
24:28
and the like platform to support all of that.
以及类似平台来支持这一切。
24:32
But also to become more AI native.
同时也要变得更加AI native。
24:34
And also there are things that we will introduce hopefully soon that kind of almost shape what the company can become.
而且还有一些我们希望很快推出的东西，几乎能塑造这家公司可以成为的样子。
24:41
So I don't know how much more I wish we could possibly be the Matt.
所以我不知道还能多说多少，我真希望我们能成为那个Matt。
24:46
But so yeah for us it's really like we got so much to do can we do it faster?
但所以是的，对我们来说真的是有太多事情要做，我们能不能做得更快？
24:51
You do it faster.
你们做得更快。
24:53
It's like our main energy but yeah I mean some companies should be asking themselves questions of like are we ambitious enough in the age of AI?
这就像是我们的主要精力所在，但是的，我的意思是有些公司应该问问自己，在AI时代我们够不够有野心？
25:02
So we've talked quite a bit about AI for your engineers and helping them be more productive.
所以我们已经聊了不少关于为你的工程师提供AI并帮助他们提高生产力的话题。
25:08
Certainly there's a lot more detail we can dig into there.
当然还有很多细节我们可以深入探讨。
25:12
But I'd also like to hear about some of the other ways you mentioned that you're applying agents in particular to the marketplace and the guest host experience.
但我也很想听听你提到的其他一些方式，特别是你们如何将agents应用到marketplace和guest host体验中。
25:22
Like what comes to mind in those domains in terms of concrete things that you've been able to achieve with agents?
在这些领域里，你们用agents具体实现了哪些东西，你首先想到的是什么？
25:30
Most people see like the the guest side of Airbnb.
大多数人看到的是Airbnb的guest那一侧。
25:34
It's kind of like my best description of it is like an iceberg where at the top you see like the booking engine and the like UI that most people use is just the guest experience.
我对它最好的描述大概就像冰山一样，最上面你看到的是预订引擎和大多数人使用的UI，那只是guest体验。
25:45
And then underneath that surface is an incredibly complex like two-sided marketplace where like the main thing that we trade in is trust and payments and quality.
而在那表面之下，是一个非常复杂的两边市场，我们交易的核心东西是信任、支付和质量。
25:58
And there's just like this very complicated operational problem customer support insurance.
还有非常复杂的运营问题，customer support、保险。
26:04
There's a lot that goes behind supply and demand.
供需背后有大量的东西。
26:07
And you know there's like a whole host side. There are millions of hosts that we have to support and we have to like they're almost they're a core part of our ecosystem.
你知道，还有整个host这一侧。我们有数百万的host需要支持，他们几乎是我们生态系统的核心部分。
26:18
And so you know we build a whole bunch of stuff for them. So there's a whole very very complicated like stack that lives below what most people see.
所以我们为他们构建了一大堆东西。所以在大多数人看到的东西之下，有一整套非常非常复杂的技术栈。
26:29
And so you know when we think about AI deployment.
所以你知道，当我们思考AI部署的时候。
26:32
A lot of what we've been able to do even in our offline fashion is help support the core tenets of what Airbnb is which is a trusted marketplace.
我们即使在离线模式下能够做的很多事情，都是帮助支撑Airbnb的核心原则，也就是一个可信的市场。
26:41
And so examples payments.
举个例子，支付。
26:45
You know could we can we use AI agents to help identify you know payment disputes or like fraud or can we use AI agents to support better guest host experiences can we detect quality issues bad on experiences.
我们能不能用AI agents来帮助识别支付纠纷或者欺诈，或者我们能不能用AI agents来支持更好的guest和host体验，我们能不能检测质量问题、糟糕的体验。
27:08
And these are like can be done at scale in an offline fashion to help the platform improve its service but it's kind of like behind the scenes can run large scale you know in a large scale large scale fashion without actually getting in front of the user and trying to render anything to them.
而这些是可以在离线状态下大规模完成的，帮助平台改进服务，但有点像是幕后的大规模运行，你知道，以大规模的方式运行，而不需要真正面对用户、试图向他们渲染任何东西。
27:26
And then for customer support I mean it is a massive operational challenge right we have to bring people with train them we have to develop like some level of consistency.
然后对于客户支持来说，我的意思是这是一个巨大的运营挑战，对吧，我们必须招人、培训他们，我们必须建立某种程度的一致性。
27:37
And so you know most people can can easily say well customer support can be an AI problem but actually like we don't see it that way we see it as how can it be a quality improvement and so we we do want to make sure that AI is actually if you have a very simple.
所以你知道，大多数人很容易说客户支持可以是一个AI问题，但实际上我们不这么看，我们把它看作是如何实现质量提升，所以我们确实想确保AI实际上是——如果你有一个非常简单的。
27:52
You know very simple task I don't want to call wait for 45 minutes just to ask can I move my reservation I just want the answer so you know AI has like a really powerful place to to to solve for that and actually I would argue there's a whole bunch of reasons for why that's probably one of the hardest problems it is actually a frontier in a really interesting way.
你知道非常简单的任务，我不想等45分钟只是为了问能不能改我的预订，我只想要答案，所以你知道AI在这方面有一个非常强大的位置来解决这个问题，而且实际上我会说有很多理由说明这可能是最难的问题之一，它实际上以一种非常有趣的方式处于前沿。
28:14
To basically build or two-sided marketplace customer support problem because we don't own the product if you if you are unhappy with something it tends to be a mediation versus like you dealing with a company you're actually dealing with another person through a company.
基本上是构建一个双边市场的客户支持问题，因为我们不拥有产品，如果你对某件事不满意，它往往是调解，而不是你直接和一家公司打交道，你实际上是通过一家公司和另一个人打交道。
28:30
And like two-sided mediation is literally having you know a person in between and two people two very unhappy people and you're trying to get to a very happy outcome which ends up being like a very complicated problem to deal with.
而双边调解实际上就是有一个人夹在中间，两个非常不开心的人之间，你试图达成一个非常开心的结果，这最终是一个非常复杂的问题。
28:44
And so you know that that turns out to be one of the hardest problems that we're working on that is AI related.
所以你知道，这最终成了我们在AI相关工作中最难解决的问题之一。
28:50
What makes it so hard besides the kind of the broader context that you outlined of like a lack of control like do you think of it as fundamentally like an experience or user interface type of problem is it like a game theoretical kind of algorithmic thing is it something all together like what makes it so it's all the it's all the above it's very adversary can be it can be very adversarial right.
除了你刚才描述的那种更宏观的背景——比如缺乏控制——还有什么让这个问题如此困难？你觉得它本质上是一个体验或用户界面类型的问题，还是一个博弈论式的算法问题，还是完全是另一回事？是什么让它变得——以上全都是，它非常对抗性，可以非常对抗性，对吧。
29:14
So you have like two parties with almost like a bounty that they're optimizing for right it's an outcome they're optimizing for.
所以你有两方，几乎像是在为一个赏金而优化，对吧，他们在为一个结果而优化。
29:22
You have like a difficult evidentiary track record so you can ask each entity to produce its its evidence or artifacts but you can't really verify the accuracy of them like I can easily take a picture of a table
你有一个很难的证据链记录，所以你可以要求每一方提供他们的证据或材料，但你没法真正验证它们的准确性，比如我可以很容易拍一张桌子的照片
29:38
work with AI to like modify the table and say well I came to this place and look at the table it's missing a leg.
用AI来修改这张桌子，然后说“我到了这个地方，看这张桌子，它缺了一条腿。”
29:44
But isn't missing a leg is it not missing a leg I can't go inspect it physically so it turns out to be like a really hard adversarial problem.
但它是缺了一条腿吗？它是不是缺了一条腿？我没法去实地检查，所以这最终成了一个非常困难的对抗性问题。
29:52
You saw that leg off.
你看到那条腿被砍掉了。
29:54
Yeah.
是的。
29:56
Exactly.
没错。
29:58
And so you know it's a really if you think about it from like a machine learning perspective you have an AI agent
所以你知道，如果你从机器学习的角度来看，你有一个AI agent
30:04
and you have these two parties that you're trying to resolve but we also have your own policies right.
然后你有这两方你试图去解决，但我们也有你自己的政策，对吧。
30:09
So if you if you're you know if the company if you if you bought a stay or experience and it had a policy of like non-refundable and you're trying to find a reason to refund it
所以如果你，你知道，如果公司，如果你，如果你买了一个住宿或体验，而它有类似不可退款的政策，然后你试图找一个理由去退款
30:19
like you also have to stick to the rules that everybody plays on the marketplace like it has to be a fair equitable relationship that you have with the company so we can't take sides we have to be very deliver it to applying our policy
你也得遵守市场上大家共同遵循的规则，就是你跟公司之间必须是公平对等的关系，所以我们不能偏袒任何一方，我们必须非常严格地执行我们的政策。
30:34
and you have to adversarial like entities that you're trying to move forward.
而且你还得面对那些对抗性的实体，你要推动事情往前走。
30:39
And so it is actually a very complicated how do you solve it it's a frontier problem I don't the models today can't really do a great job.
所以这实际上是一个非常复杂的问题，怎么解决它，这是一个前沿问题，我觉得今天的模型还真的做不好。
30:48
Of like mediating between me and you and then you know if I'm Airbnb and then like Airbnb in the host and let's say you're the guest like it is quite a complex multi party mediation problem.
比如在我和你之间做调解，然后你知道如果我是Airbnb，然后Airbnb和房东之间，假设你是房客，这就是一个相当复杂的多方调解问题。
31:00
You know people talk about multiplayer you know we're all putting agents inside a group chat and it barely works now imagine if you and I had to had a like a financial dispute and we stick on stick an AI agent between our chat like is the AI agent going to really resolve it or is it going to like not.
你知道大家在聊多玩家，我们都把agents放进一个群聊里，现在这都勉强能用，想象一下如果你和我有一个财务纠纷，我们把一个AI agent塞进我们的聊天中间，这个AI agent真的能解决吗，还是说它不会。
31:17
And so and it's a physical interaction like you cannot observe it right right and so you know you're not saying that you're not using agents there you're saying you're using agents or at least thinking about using agents and it's hard like how are you approaching it we're in like research and development to deal with the hardest use cases that I'm referring to but we still apply a lot of human judgment to those use cases where we can train.
而且这是一种物理交互，你没法观察到它，对对，所以你不是说你在那里不用agents，你是说你在用agents，或者至少在考虑用agents，而这很难，你是怎么着手做的，我们处于研发阶段，来处理我提到的那些最难的用例，但在那些我们可以训练的用例上，我们仍然施加了大量的人类判断。
31:45
Like we can train humans to have the human touch to have the interaction to like really like make sure we get again Airbnb is all about like making sure that we give the best quality experience that we possibly can.
就像我们可以训练人类拥有那种人情味，拥有那种互动能力，真正确保我们——Airbnb的核心就是确保我们尽可能提供最高质量的体验。
31:58
So we don't feel like it's ready for the hardest use cases there is a large tale of use cases where it is incredibly useful so we've already deployed you know AI agents into production for customer support.
所以我们不觉得它已经准备好应对最难的用例了，但有一大堆用例它已经非常有用了，所以我们已经把AI agents部署到生产环境里做客服了。
32:09
And I'm you know we've we've resolved we resolve about 50% of all chats with with AI so our agenda solutions already resolving all 50% of support chats in particular yes yes all tickets yeah.
而且我们——我们已经解决了大约50%的聊天，用AI，所以我们的agent解决方案已经解决了50%的客服聊天，特别是，对对，所有工单，是的。
32:25
Is the user guests or hosts interacting directly with an agent or are they interacting with someone that appears to be a person but that person is really an agent or supported by an agent.
是用户、房客或房东直接和一个agent互动，还是他们在和一个看起来像人但实际上是agent、或者由agent辅助的人在互动？
32:36
We're we have a philosophy of being very transparent we're not putting an AI in front of a person and saying it's a person we're putting an AI in front of a person and saying it's an AI so we're pretty we're pretty transparent about about that and but the 50% is purely AI.
我们有一个理念就是非常透明，我们不会把一个AI放在一个人前面然后说它是人，我们是把一个AI放在一个人前面然后说它是AI，所以我们非常透明，但那个50%纯粹是AI。
32:54
So you resolve tickets so like we're not this isn't like a human assisted AI and I'm talking to a human in the AI is assisting the human to close the ticket this is 50% AI to human resolution rate so and it's a hill climbing problem right so like there's a set like informational queries turn out to be really easy that's not the full for 50% like those are pretty easy and actually it's a way more convenient like if I'm calling every being I'm asking like an informational query.
所以你们解决工单，这不是那种人类辅助AI——我在跟一个人说话，AI在辅助那个人关闭工单——这是50%的AI对人类的解决率，而且这是一个爬山问题，对吧，所以有一类信息查询其实很容易，但那不是全部的50%，那些是挺容易的，而且实际上方便多了，比如我打电话——我问一个信息类的查询。
33:22
It's giving me the answer and on my merry way don't put me on hold I don't want to wait for a person like I just want the answer and by the way we also give like cards action cards for people to resolve directly so you can take action it's not just information queries but it's also a genetic which covers like you know a lot of a lot of use cases but there's still some portion of use cases that are extremely hard that as we hill climb to the frontier like that is where you know we really need to push the boundaries and be really
它直接给我答案，我就可以走了，别让我等着，我不想排队等人工，我就想要答案。顺便说一下，我们还会提供类似卡片的东西，行动卡片，让人们可以直接解决问题，所以你可以直接采取行动，不只是信息查询，它还是一个通用的东西，覆盖了很多很多用例，但仍然有一部分用例极其困难，当我们向 frontier 攀登时，那正是我们真正需要推动边界、真正
33:52
to be quite innovative to solve. It's interesting my personal experience as a guest and host with Airbnb like I don't feel like I've run into AI usage that has shifted significantly over the past like two, three years like if I need to go to the support site it all I feel like the flows the same like what stays it about you know is it a guest issues at a host issue and it kind of walks me through that but it's been doing that for a long time.
相当有创新性地去解决。有意思的是，我作为Airbnb房客和房东的个人经历，我感觉过去两三年里并没有遇到AI使用上有显著变化，比如如果我需要去支持网站，我感觉流程还是一样的，比如是关于房客问题还是房东问题，它会引导我走完流程，但这个东西已经做了很久了。
34:21
And I've never had an agent answer or get involved in the chat I have noticed the in the chat like in chat card moderation oh it seems like you're trying to do like a change request here's the card click the button which is cool I'm wondering like
而我从来没遇到过agent来回答或参与到聊天里，我注意到在聊天里，比如在聊天卡片审核里，哦，看起来你是想做一个变更请求，这是卡片，点击按钮，这挺酷的，我在想比如
34:39
you know once that's a really good thing like because most of the time when I think about like a company forcing me to use AI for support it's like not a very good experience I see that as a success metric.
你知道这其实是件好事，因为大多数时候我想到公司强迫我用AI来获得支持，那体验都不太好，我把它看作一个成功指标。
34:49
The goal isn't to shove AI in front of people it is to meet people where they are but there is a chat customer support agent that you can initialize so if you say contact you know customer support via chat I think what you're referring to is like our help center which again people
目标不是把AI硬推到人们面前，而是在人们所在的地方去触达他们，但有一个聊天客服agent你可以初始化，所以如果你说联系客服通过聊天，我想你指的是我们的帮助中心，这个 again 人们
35:06
who are using our help center are actually looking for particular experience are not looking to just call Airbnb or text Airbnb with a problem right.
正在使用我们帮助中心的人实际上是在寻找特定的体验，而不是只想打电话给Airbnb或发短信给Airbnb反映问题，对吧。
35:14
And so those workflows still exist and there's still a lot we should be doing with the help center so that's going to happen but yeah.
所以那些工作流程仍然存在，我们在help center方面还有很多事情应该做，所以那会发生的，但是是的。
35:22
The volume of our tickets actually happened via chat and if you do by the way if you get into a customer support chat and you say like give me a human
我们tickets的数量实际上是通过chat发生的，顺便说一下，如果你进入一个customer support chat然后你说给我一个真人
35:31
AI will like escalate and we'll route it to human so we don't force people I don't know if you've ever used those like terrible call centers where like it just like traps you.
AI会升级然后我们会把它路由到真人，所以我们不会强迫人们，我不知道你有没有用过那些糟糕的call center，就是那种把你困住的。
35:44
Sorry you have to press like zero 18 times before I get you to human or maybe if you yell at me enough I'll get you there like we're pretty careful about getting you to human if that is what you prefer.
抱歉你得按零18次才能让我把你转到真人，或者也许如果你对我吼得够多我就会把你转过去，我们非常小心地确保如果你更愿意的话能转到真人。
35:55
I was just talking to someone about how difficult it is now because like I've been trained to say agent with like increasing urgency when I want to talk to our human yeah but now like what's the agent.
我刚才在跟某人聊现在这有多难，因为我已经被训练成在想要跟真人说话时会用越来越急迫的语气说agent，但现在什么是agent呢。
36:06
Yeah exactly exactly you want an agent well let me introduce you to this new agent human human human human yeah yeah and you know if you if you look at the call volume like if you actually look at the number of people there's people who have zero interest and it's not an insignificant
对对对，你想要一个agent，那让我给你介绍这个新agent，真人真人真人真人，对对，你知道如果你看call volume，如果你真的看人数，有些人是零兴趣的，而且这不是一个小数目
36:24
you're talking about tens of percent of people who will like immediately look that might change you know as we get better systems we get more natural systems as we get more.
你是在说会有百分之几十的人会立刻喜欢上它，这可能会改变，你知道，随着我们的系统变得更好，系统变得更自然，随着我们拥有更多。
36:33
Capable systems people might shift but there is like a non insignificant number of people they just have no interest in talking to to an AI and just one human even if they're ticket even if their item is just
有能力系统的人可能会转变，但有一批不可忽视的人，他们就是没兴趣跟AI说话，只想跟一个人交流，即使他们要排队，即使他们的项目只是
36:44
informational even whatever they're just get me to a curse and those are going to be the hardest ones to resolve for because it is really it's like us it's psychological it's not capability based.
信息类的，甚至不管是什么，他们就是让我去上一个课程，而那些将是最难解决的，因为那真的是，就像我们一样，是心理层面的，不是基于能力的。
36:55
Since we veered into like personal experiences with Airbnb I think one of the domains why I'd be most excited to see interesting AI things is like in terms of a gentick search and yeah I know you've you've worked a lot
既然我们聊到了Airbnb的个人体验，我觉得我最期待看到有趣的AI事物的领域之一就是基因搜索，而且我知道你在这方面做了很多工作
37:13
historically on like trying to capture user preferences but seems like I often have to do like the same mental filter for what I really care about.
以前在捕捉用户偏好这方面，但好像我经常得对真正在意的东西做同样的心理过滤。
37:22
And it would be great to you know be able and they're also there is a lot of information that you have about properties that you don't expose via you know amenities or search filters or something like that.
能这样的话就太好了，而且还有很多关于房源的信息，你们并没有通过设施或搜索筛选之类的方式展示出来。
37:34
It's like deeply embedded in the imagery it's also embedded in the reviews it's embedded in a lot of a lot of context I mean we have a part of why you trust.
这就像是深深嵌入在图像里，也嵌入在评论里，嵌入在很多很多上下文里，我是说，你信任的一部分原因。
37:42
To book book these homes is because we have all these additional signals that make you comfortable to book them right and the signals are just as useful to a person to absorb or an AI to help match.
能预订这些房源，是因为我们有所有这些额外的信号让你放心去预订，对吧，而这些信号对于一个人来吸收，或者对于AI来帮助匹配，同样有用。
37:53
Yeah to a person and you know it like searches like a is such an important part of how people experience our platform.
是的，对人来说，你知道，像搜索这样的东西是人们体验我们平台的重要部分。
38:02
And we're really very deliberate so we've experimented a lot with what to do here and despite your personal preference there is like a there is like a broader you know like broader kind of non consistent user base behavior that we we've been trying to better understand so like people don't want a blinking box where it's like chat.
我们真的非常刻意，所以我们做了很多实验来看这里该怎么做，尽管你有个人偏好，但存在一个更广泛的、你知道，更广泛的非一致性的用户行为，我们一直在试图更好地理解它，所以人们不想要一个闪烁的聊天框。
38:31
They just right it's it's like the text back and forth is like it's friction you know I know I don't want to stay I know how many people are going give me my options.
他们就是，对吧，来回打字就像是一种摩擦，你知道，我知道我不想停留，我知道有多少人，给我我的选项。
38:43
And I don't want to type like a 45 sentence query you know we've brand tests where people know I don't want that either I agree yeah so we've so something we've we like we're recently.
我不想打一个45句话的查询，你知道，我们做过品牌测试，人们知道我也不想要那个，我同意，是的，所以我们，我们最近，我们喜欢，我们最近在。
38:54
I think it just announced that are like we're testing but we're testing a lot of variance but one of the things that we're testing is like just a natural language box where you can just type where you're going we're going with and like why you're going what you're looking for.
我觉得它刚刚宣布的是，我们在测试，我们在测试很多变体，但其中一个我们在测试的东西就是一个自然语言框，你可以直接输入你要去哪里、和谁一起去、为什么去、你在找什么。
39:07
And we should be able to return the results but it's still very multimodal you still have a lot of options you're still in control i'm not getting an AI to go pick the one property and sorry that's like the one or the two or three i'm still giving you the ability to sort of.
我们应该能够返回结果，但它仍然是很多模态的，你仍然有很多选项，你仍然在掌控之中，我不是让AI去挑那一个房源，抱歉，不是那一个或那两个三个，我仍然给你能力去。
39:20
Transpose and like look at different cuts one of the things you see in how people operate these platforms is they don't you don't it's not like a straight line to a book actually yeah it takes like weeks for people to book and they like get inspired they come the search.
调换和查看不同的切法，你在人们如何使用这些平台中看到的一件事是，他们不会，你不是，这不是一条直线到预订，实际上是的，人们预订要花好几周，他们会受到启发，他们来搜索。
39:36
They click around a lot and then they like don't come back for like three more days because I guess they go talk to their partner and they're like hey I don't know right.
他们点来点去很多，然后他们就像三天不再回来，因为我猜他们去跟伴侣聊了，他们说嘿我不知道对吧。
39:45
You know because these trips are like social and they're like multiplayer so you know you end up like going on our platform finding some stuff maybe you heart some stuff and then you go back.
你知道因为这些都是社交的，它们是多人参与的，所以你知道你最终会在我们平台上找一些东西，也许你收藏了一些东西，然后你回去。
39:54
You got to figure out some other things like logistics timing travel like there's so much to figure out and then you come back.
你得搞清楚一些其他事情，比如后勤、时间、旅行，有太多东西要搞清楚，然后你回来。
40:01
And then you click on some more and then you end up like you know maybe removing some hearts adding some new ones and it takes weeks before someone like all right i'm going to spend $10,000 on a trip.
然后你再多点几下，最后可能删掉一些收藏，再加一些新的，然后过了好几周才有人说，好吧，我要在这趟旅行上花一万美元。
40:13
With the company so you don't want AI to sort of remove that agency and exploration process you want to enhance it.
跟公司一起，所以你不希望AI剥夺这种自主性和探索过程，你想增强它。
40:21
So part of part of what we're trying to do with our agenda search is like accommodate for that journey the people go through when they actually decide what they want.
所以我们做agenda search的部分目标就是，去适配人们在真正决定自己想要什么时所经历的那个过程。
40:30
In some ways for some trips at least because I think one of the things that you're identifying is that you know even for me not all trips are the same and so for one type of trip.
至少在某些旅行中是这样，因为我觉得你指出的一个问题是，即使对我来说，不是所有旅行都一样，所以对于某一类旅行。
40:41
I'm just going to put in the city and find the first thing that needs to criteria and bounce and then for others is like this wish list process and like yeah oh hey check these out babe you know and yeah you know weeks later we settle on something.
我就直接输入城市，找到第一个符合条件的东西，然后就走，而对于另一些旅行，就是这种愿望清单的过程，然后说，嘿，看看这些，宝贝，你知道，然后几周后我们才定下来。
40:55
For some kind of trips you know I might not even want to interact with Airbnb experience I want Airbnb to be like an MCP server to a world of places I could stay and like have my agent.
对于某些类型的旅行，你知道我甚至可能不想跟Airbnb体验互动，我希望Airbnb就像一个MCP server，连接到一个我可以住的地方的世界，然后让我的agent来处理。
41:10
Sift through all of the things and you know historically like this has been like whether Airbnb exposes an API of of properties for example like historically this has been not a technical issue but like a political kind of business issue like do we want to lose control of the customer
翻遍所有这些事情，你知道历史上这就像是Airbnb是否暴露一个房源的API，比如历史上这不是一个技术问题，而是一个政治性的商业问题，比如我们是否想失去对客户的控制
41:32
do we want to you know let people scrape our information all these kinds of things but it does strike me that like that argument is starting to disintegrate because of computer use and I could still have my agent do it.
我们是否想，你知道，让人们抓取我们的信息，所有这些类似的事情，但确实让我觉得那个论点开始瓦解了，因为computer use，我仍然可以让我的agent去做这件事。
41:45
It's just like harder and takes a lot more time so why not just give me the MCP access like how do you think about that.
只是更难、需要花更多时间，那为什么不直接给我MCP访问权限呢，你怎么看这个问题。
41:52
I come back to like consumer companies and their DNA and what they need to focus on like fundamentally you need to focus on like what is the best user experience to experience our platform and
我回到消费类公司以及它们的DNA和它们需要专注的东西，从根本上你需要专注于什么是最好的用户体验来体验我们的平台，而且
42:02
I think you kind of need to evaluate on a case per case basis and like what makes sense and what doesn't and so I think that is the lens upon which we will make all of our decisions.
我觉得你需要逐案评估，什么合理什么不合理，所以我觉得这就是我们做所有决策的视角。
42:12
I do want to I do think though that like travel is kind of a unique animal in a lot of ways relative to a lot of other things that you could be doing in the context of like convenience isn't key trust is.
我确实想，我确实认为旅行在很多方面是一种独特的动物，相对于很多其他你可以在便利性语境下做的事情，便利不是关键，信任才是。
42:27
And like a booking isn't just a booking it's like you have first of all you have the inspiration question which obviously you know AI with its world knowledge and its web hooks like fact that it can search the internet and aggregate all this it's like useful but it is not like it's not the only thing they use for inspiration lots of people get inspired by going to like Instagram and watching videos and there's like all kinds of ways people get inspired about where to go.
而且预订不仅仅是预订，它就像，首先你有灵感的问题，显然你知道AI凭借它的世界知识和它的网络钩子，比如它能搜索互联网并聚合所有这些信息，这很有用，但这不是他们用来获取灵感的唯一方式，很多人通过上Instagram看视频来获得灵感，人们获得旅行灵感的方式有各种各样。
42:52
Yeah including talking to friends or events or there's all sorts of things or agents.
对，包括和朋友聊天或者活动或者各种各样的事情或者agent。
42:58
Oh yeah AI is definitely huge and like I think you know 40% or something like that of like humans in the US use AI for inspiration but when you come to book like there's the booking and then there's like going on trip and then there's like everything that happens in between
哦对，AI绝对是巨大的，我觉得你知道美国大概40%左右的人用AI来获取灵感，但当你到预订的时候，有预订，然后有出行，然后中间发生的一切，
43:17
and the trust that you have when you do books so we think that that's going to be something like the best user experience is going to be the one that you trust the most and it because it's such a high judgment high value purchase you really want to trust what you're booking and who you're booking with.
还有你预订时的信任，所以我们认为最好的用户体验将是那个你最信任的，因为这是如此高判断高价值的购买，你真的想信任你预订的东西和你预订的对象。
43:33
And so that's kind of I think one of our main strengths as a company it's a platform that you can trust if you you know we have a insurance we have like payments and a payment stack you can trust we have a customer support to we we will deal with like issues versus like a random you know MCP server somewhere that you're like booking $10,000 trip with.
所以这算是我们认为我们作为公司的主要优势之一，这是一个你可以信任的平台，你知道我们有保险，我们有支付和你可以信任的支付栈，我们有客户支持，我们会处理问题，而不是某个随机的你知道MCP server，你在那里预订一万美元的旅行。
43:59
Yeah so I think I don't think that I'd want to like I don't think I'd want like some agentic commerce thing where the agent just books to stay I think yeah I'd want the agent to help with screening you know just the same way your natural language text box would but presumably the agent can know and store my preferences and kind of get through that more quickly yeah yeah yeah and I think that's what I meant by like case by case is like we have a bar for what we want people to feel experientially.
对，所以我觉得我不会想要那种agentic commerce的东西，agent直接就预订住宿，我觉得对，我会想要agent帮忙筛选，你知道就像你的自然语言文本框会做的那样，但大概agent可以知道并存储我的偏好，然后更快地完成这些，对对对，我觉得这就是我说的case by case的意思，就是我们有一个标准，关于我们希望人们在体验上感受到什么。
44:28
When they see what we're providing and again a home for example is like very personal to you which is why the preferences thing really matters but also finding the right skew that works for you is like a deeply multimodal thing I wouldn't want I wouldn't want to our product to be a bunch of text that a user picks I would want them to be able to experience it in the right way.
当他们看到我们提供的东西时，再比如说家对你来说是非常私人的，这就是为什么偏好这个东西真的很重要，但同时找到适合你的正确方式是一个非常 multimodal 的事情，我不希望我们的产品只是一堆用户挑选的文本，我希望他们能够以正确的方式去体验它。
44:49
So when we feel like there's a way to deliver the right experience to people will will work with them but I think that's kind of very important to us.
所以当我们觉得有办法向人们提供正确的体验时，我们会和他们合作，但我觉得这对我们来说非常重要。
44:59
A little bit about some of the tooling and tech stack on the engineering productivity side of things when you are deploying agents for kind of you know marketplace guest host is that all.
稍微聊一下工程生产力方面的一些工具和技术栈，当你们为 marketplace 的 guest host 部署 agents 的时候，就是这些吗。
45:16
You know what are kind of some of the platforms or text acts that you're you know using and building what are some of the big consumers of open source are you using commercial agent platforms have you built everything from the ground up it's a mix I mean we let's see we are users of open and close source models.
你们在用和构建的一些平台或技术栈是什么，开源的主要消费者是哪些，你们在用商业的 agent 平台吗，还是所有东西都是从零开始构建的，是混合的，我是说让我们看看，我们是 open 和 close source models 的用户。
45:37
We have definitely leveraged open source frameworks and we've also built where we feel we need to where we feel we can innovate and where it's actually valuable for us to build so take customer support for example you know some of the agent frameworks there.
我们肯定利用了开源框架，我们也在觉得需要的地方、觉得可以创新的地方、以及构建对我们真正有价值的地方进行了自建，所以拿 customer support 举例，那里的一些 agent 框架。
45:56
We started building before there were a lot of open source frameworks available so it was more out of like we were ahead of where the industry is and we built what we think we needed to build because there wasn't anything there.
我们开始构建的时候还没有很多开源框架可用，所以更多是因为我们领先于行业所处的位置，我们构建了我们认为自己需要构建的东西，因为当时什么都没有。
46:07
You know arguably like we always evaluate whether we whether the open source community has built something that's like further ahead than what we have if it did we have no qualms like rebasing on the open source and then or migrating to the open source solution and then contributing actually to open source.
你知道，可以说我们一直在评估开源社区是否已经做出了比我们更领先的东西，如果确实如此，我们毫不犹豫地会基于开源来做rebase，然后或者迁移到开源解决方案，然后实际上回馈开源。
46:24
So my stance on open source is like well known I am a big proponent to basically build in the open where we can.
所以我对开源的立场是众所周知的，我非常支持在能公开的地方就公开构建。
46:32
And then on the model side you know there's a strong where we're experiential company there's a strong parade oh that we've been really focused on which is the speed.
然后在模型方面，你知道有一个很强的，我们是一家注重体验的公司，有一个很强的重点我们一直非常专注，那就是速度。
46:43
Of our interactions we don't want people to have like very very tight latency requirements and given our scale like for certain use cases it makes no sense to use large models for very specialized tasks and for that you know we have our own post training infrastructure that we.
我们的交互速度，我们不希望人们有非常非常严格的延迟要求，而且考虑到我们的规模，对于某些用例来说，用大模型来做非常专门化的任务完全没有意义，为此你知道我们有我们自己的post-training基础设施，我们。
47:01
And our own like our own environments etc so that we can like take more efficient smaller faster models that could really kind of beat the frontier at the narrow benchmark that we care about right so like if you have use cases where you know you're you have a narrow thing you're trying to do you're you're dealing with like customer support and adherence to a policy I don't have to worry about regression on mathematics or regression on some other name that I don't care about.
还有我们自己的环境等等，这样我们就能采用更高效、更小、更快的模型，这些模型真的能在我们关心的狭窄benchmark上击败前沿模型，对吧，所以比如如果你有用例，你知道你要做的是一个狭窄的事情，你在处理比如客户支持和政策遵循，我不需要担心数学上的退化或者我不关心的其他某个名字上的退化。
47:30
Then you can actually post train a model to be really good at it I would actually say that's that was why my hot take was like the models are good enough for a lot of consumer experiences because right now we're kind of focused on the jagged edge of like I have now solved a millennium your prize problem versus can I.
那你实际上可以post train一个模型让它在这方面非常擅长，我其实会说这就是为什么我的hot take是模型对于很多消费者体验来说已经足够好了，因为现在我们有点专注于那种参差不齐的边缘，比如我现在解决了一个千禧年大奖难题，对比我能不能。
47:48
Do reliable computer use on on a travel website or whatever like that feels less you know you need a 10 trillion parameter model to achieve right and actually that applies at very like like various scales and so the more you're able to if you have an area use cases which which we have examples of like our search is a good example where we take like the smallest model possible because they keep you so high that we.
在旅行网站上做可靠的computer use之类的任务，感觉不太需要，你知道，你需要一个10万亿参数的模型才能实现，对吧？而且这实际上适用于非常多的规模层面，所以如果你有一个领域用例——我们有例子，比如我们的搜索就是一个很好的例子，我们用尽可能最小的模型，因为它们让你保持如此高的——我们。
48:17
Like really post train and specialize in distill that model to be really good at what it used to be good at doing I mean we're already starting to see a lot of people train different types of models that are just like focus on very narrow things like that you have examples of where yeah people basically like train to model for decision making just awesome but it's a very simple model that very focus on something really really high leverage for a lot of places and so we've learned that that's why we have our own investments in post training open models.
真正地post train并专门化，distill那个模型，让它真正擅长它原本就擅长做的事情。我是说，我们已经开始看到很多人训练不同类型的模型，就是专注于非常窄的领域，你有例子表明，是的，人们基本上训练模型来做决策，太棒了，但它是一个非常简单的模型，非常专注于对很多地方来说真正真正高杠杆的事情，所以我们学到了这一点，这就是为什么我们自己在post training开放模型上有投资。
48:46
And do you have a favorite small model that you that's kind of a go to there's a lot of good ones is like you know Neumatron from Nvidia there's there's a bunch I mean there's a bunch that we look at we're pretty we evaluate everything I'm imagining same is the case for the larger open weights models you just kind of have a stable that you you know e Val everything again so you choose the one that makes most sense at the time yeah that's right that's right any thoughts on kind of what.
那你有没有一个最喜欢的小模型，是你的首选？有很多好的，比如你知道的Nvidia的Neumatron，有一堆——我是说，有一堆我们会看的，我们评估所有东西。我想 larger open weights模型也是一样的情况，你有一个稳定的清单，你你知道，重新评估所有东西，然后你选当时最合理的那个。对，没错，没错。关于这个你有什么想法——
49:16
Where you see this all going at Airbnb like what are you whatever maybe some of the big challenges ahead for you and and how do you think AI will help you solve them of the three things we're doing you know I think we did a good job sort of reshaping the company the AI native I mean like Brian down everybody's using Brian famously I think just shared at a conference that he has a second brain that helps him run the company where it's been trained on he has built a context graph.
你觉得这一切在Airbnb会走向哪里？也许前面有一些大的挑战，你觉得AI会怎么帮你解决它们？我们正在做的三件事中，你知道，我觉得我们在重塑公司方面做得不错，AI native，我是说像Brian down，每个人都在用Brian，很有名，我想他刚在一个会议上分享了他有一个第二大脑，帮他运营公司，它是在——他建了一个context graph。
49:46
Which is local to his machine that is almost like it's been trained on like it has like every document that's ever come to him it is every met you know every vision document manifesto email et cetera communicate piece of communication he's ever written and he uses that to help reason through his day which you know he's using so like if the CEO and the founder is doing that you can imagine like the company is like very very excited about it so I think we've like we're on the right clip I don't know if it's a challenge but but there's a lot of things to do.
那是他机器本地的，几乎就像是——它被训练过，像是——它有所有曾经到过他手上的文档，每一次会议，你知道，每一份愿景文档、宣言、邮件等等，他写过的每一份沟通内容，他用那个来帮助推理他的一天，你知道他在用，所以如果CEO和创始人都在这么做，你可以想象公司非常非常兴奋。所以我觉得我们在正确的轨道上，我不知道这算不算挑战，但有很多事情要做。
50:16
A lot of complexity that comes from a lot of AI use like Everest like is Everest going to be useful or is it going to become problematic are we going to corrupt it as we scale it we don't know are we do even have the right architecture for it we have to rewrite it so but this is all work not not you know when you're trying to do it from zero to one it's much harder than going from like where we are to just keep pushing to stay at the bleeding edge.
很多复杂性来自大量AI使用，像Everest——Everest会有用还是会变得有问题？我们扩展它的时候会不会把它搞坏？我们不知道。我们甚至有正确的架构吗？我们得重写它。但这些都是工作，不是——你知道，当你试图从零到一的时候，比从我们现在的位置继续推进、保持在最前沿要难得多。
50:43
I think for the marketplace I also feel really comfortable I think we still we're working on it but I think there's still a lot of degrees of freedom on how we shape the user experience that people have with every and be using AI while staying true to what we are as a company I think that is actually like I said it's a frontier of interaction and you know the one of the things that's exciting to me is like travel is very complicated you know I like you know I was watching this can go try to vibe code something it looks great try to use it.
我觉得对于这个marketplace，我也感到很自在，我觉得我们还在做，但我认为在如何塑造用户使用every和AI的体验方面，还有很多自由度，同时保持我们作为公司的本色。我觉得这实际上就像我说的，是交互的一个前沿，你知道，让我兴奋的一件事是，旅行是非常复杂的，你知道，我喜欢，你知道，我在看这个，可以去试着vibe code一些东西，看起来很棒，试着用一下。
51:13
It's it for a few days and you realize like oh my god I can't use it I want I want to I do you know it's in your areas I want to be able to share with other people I want to be it turns into this like bloated mess.
用了几天你就意识到，哦天哪，我没法用，我想要，我想，你知道，在你的领域里，我想要能够和其他人分享，我想要，它变成了一团臃肿的乱麻。
51:24
So actually the interaction model and the design of how that comes to life is actually really really hard and then you know for for customer support and for this like multi agent problem multi multi person problem.
所以实际上交互模型以及它如何落地的设计，实际上是非常非常难的，然后你知道，对于customer support以及这种multi agent问题、多人问题。
51:38
How do you deploy agents that can actually resolve things in it in a constructive way.
你如何部署能够以建设性的方式真正解决问题的agents。
51:43
How do we get to 100% that sounds like a meaty challenge to dig into there.
我们如何达到100%，听起来像是一个很有分量的挑战值得深入。
51:48
Yeah awesome well I mean thanks so much for jumping on and sharing a bit about what you're up to there.
是的，太棒了，非常感谢你来聊一聊并分享了一些你在那边做的事情。
51:54
Yeah thanks for having me Sam. This is fun. Thank you.
是啊，谢谢你邀请我Sam。这挺有意思的。谢谢。