Practical AI
How to get discovered in AI search
00:003314
00:01
Welcome to the Practical AI Podcast, where we break down the real-world applications
欢迎来到 Practical AI Podcast，我们会在这里拆解 AI 在现实世界中的应用，
00:07
of artificial intelligence and how it's shaping the way we live, work, and create.
以及它如何塑造我们生活、工作和创作的方式。
00:12
Our goal is to help make AI technology, practical, productive, and accessible to everyone.
我们的目标是帮助让 AI 技术变得实用、高效，并且让每个人都能接触到。
00:18
Whether you're a developer, business leader, or just curious about the tech behind the buzz,
无论你是开发者、企业领导者，还是只是对热点背后的技术感到好奇，
00:23
you're in the right place.
那你就来对地方了。
00:25
Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode
记得在 LinkedIn、X 或 Blue Sky 上关注我们，随时了解节目
00:29
drops, behind the scenes content, and AI insights.
上线、幕后内容和 AI 洞察。
00:32
You can learn more at practicalai.fm.
你可以访问 practicalai.fm 了解更多。
00:35
Now, on to the show.
现在，进入节目。
00:41
Welcome to another episode of the Practical AI Podcast.
欢迎收听又一期的 Practical AI Podcast。
00:45
This is Daniel Lightnack.
我是 Daniel Lightnack。
00:46
I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson,
我是 Prediction Guard 的 CEO，和往常一样，我的联合主持人 Chris Benson 也和我在一起，
00:51
who is a principal AI and autonomy research engineer.
他是首席 AI 和 autonomy 研究工程师。
00:55
How you doing, Chris?
你最近怎么样，Chris？
00:56
I'm doing great today, Daniel.
我今天很好，Daniel。
00:57
How's it going?
你怎么样？
00:58
It's going great.
挺好的。
01:00
It's good to see you.
很高兴见到你。
01:02
You're visible to me.
我这边能看到你。
01:06
That's an interesting part of the awkward segue into how do things become visible to
这其实是那个尴尬过渡里挺有意思的一部分——现在在互联网上，东西到底是怎么变得
01:15
us on the internet these days?
对我们可见的？
01:16
It seems to be increasingly through AI platforms, AI chat interfaces, answer engines, whatever
好像越来越多是通过 AI platforms、AI chat interfaces、answer engines，不管
01:26
you call them, and today we're privileged to have with us Liam Dunne and Ben Moore,
你怎么叫它们，今天我们很荣幸请到了 Liam Dunne 和 Ben Moore，
01:32
who are co-founders at Discovered Labs, to talk through some of these things.
他们是 Discovered Labs 的联合创始人，来聊聊其中一些事情。
01:36
Welcome, Liam and Ben.
欢迎，Liam 和 Ben。
01:37
Great to have you.
很高兴你们能来。
01:38
Good to be here.
很高兴来到这里。
01:39
Thank you.
谢谢。
01:40
Yeah.
嗯。
01:41
Well, for those that maybe are less familiar with this topic in general around AEO,
嗯，对于那些可能对 AEO 这个整体话题不太熟悉的人来说，
01:50
GEO, Answer Engine Optimization, AI visibility, whatever kind of term is around this, and maybe
GEO、Answer Engine Optimization、AI visibility，不管围绕这个的是什么叫法，而且也许
01:56
there are differences between those terms, but for those that aren't as familiar, could
这些术语之间是有区别的，但对于那些不太熟悉的人，能不能
02:00
you all give us a context for what those things mean, and then also how you're involved in
02:08
those topics day to day, what you're doing at Discovered Labs, which is the context
02:14
that you're doing some of the work that we'll talk about.
02:18
Cool.
02:19
I would say everyone's got a different opinion on this, so feel free to take mine with the
02:24
Pinterest, so I would say AI searches like the broad category, and then within that you'd
02:31
have Answer Engine Optimization.
02:34
Some people say, which is AEO, some people say GEO, Generative Engine Optimization.
02:40
I view those as the same thing, and I just call it AEO.
我觉得这俩是一回事，我就直接管它叫 AEO。
02:44
Honestly, for a very simple reason, there are a lot of venture funded companies that have
说实话，原因特别简单，有很多拿了风险投资的公司已经
02:50
spent a lot of money on that term, and so I'm just going to fly behind them and lean
在那个词上砸了很多钱，所以我就打算跟在他们后面借势，然后全力
02:57
into it.
投入进去。
03:00
The background of us, so how we involved with this, so we're co-founders at Discovered
我们的背景，以及我们是怎么参与进来的，我们是 Discovered
03:04
Labs, it's an organic search agency, so we provide end-to-end services.
Labs 的联合创始人，它是一家 organic search 代理机构，所以我们提供端到端服务。
03:10
Now organic search for us splits into two buckets, you've got like traditional SEO,
现在 organic search 对我们来说分成两块，有传统的 SEO，
03:14
and you have AI search, so traditional SEO, we want our clients to be at the top of Google
还有 AI search，所以传统 SEO 方面，我们希望客户能排在 Google 的最上面
03:19
whenever people are searching related keywords, AI search, we want our clients to be appearing
每当人们搜索相关关键词、AI search 时，我们希望我们的客户能出现在
03:23
inside LLNs in a way that they want to be, and so we view that as the AI search side
LLNs 里面，以他们希望的方式，所以我们把这看成 AI search 这方面
03:29
of things.
的事情。
03:30
Gotcha.
明白了。
03:31
I guess maybe for context, people might be familiar with SEO, search engine, optimization,
我想，也许为了给点背景，大家可能熟悉 SEO、search engine optimization，
03:38
and maybe help orient us, is SEO, is that still a thing?
也帮我们理一理，SEO 现在还存在吗？
03:45
Is this topic basically replacing that, is it kind of, how do the two interact?
这个话题基本上是在取代它吗，还是说，这两者到底怎么相互作用？
03:52
I mean, this is always, I guess this is also a fluid thing, but if I am a company, this,
我是说，这一直都是，我想这也是个不断变化的东西，但如果我是一家公司，这个，
03:58
you know, I'm a company in 2026, like what is most important and how are people thinking
你知道，2026 年我是一家公司，就像，什么最重要，人们又是怎么想
04:08
about the effort they're putting into maybe traditional SEO versus these topics that you
关于他们投入的精力，也许是传统 SEO，还是这些你
04:15
have any thoughts there?
有什么想法吗？
04:16
I do.
我有。
04:17
Feel free to, but I can waffle too much, so feel free to ask you questions, I'll go deeper.
你请便，但我可能会啰嗦太多，所以尽管问我问题，我会讲得更深入。
04:25
But ultimately, you won't hear me say that SEO is dead, it's not an opinion I hold.
但归根结底，你不会听到我说 SEO 已死，这不是我持有的观点。
04:30
I just think the space has grown.
我只是觉得这个领域已经变大了。
04:35
We're doing the same three jobs that's on page, so looking after your own website, we're
我们还是在做同样的三件事，也就是 on-page，比如照看你自己的网站，我们
04:41
doing off page, which is building brand authority, and we're doing technical, which is making
做 off page，也就是建立 brand authority，而我们做 technical，也就是确保
04:47
sure agents and Google systems can access your website and understand information.
agents 和 Google 系统能访问你的网站并理解信息。
04:51
It's the same three jobs.
这还是同样的三项工作。
04:52
For me, it comes down to a matter of tactical priorities, and I think this is where people
对我来说，这归结为战术优先级的问题，而我觉得这正是人们
04:58
would like get lost.
很容易迷失的地方。
05:01
I do think there are some things, like, depending on what the goal is, right, like to rank
我确实觉得有些情况，比如，取决于目标是什么，对吧，比如要想排名
05:06
number one on Google for a commercial keyword, you're probably going to use different
在 Google 上某个 commercial keyword 排到第一，你大概会用不同的
05:11
tactics than if you were to optimize for a surface area like chat GPT.
策略，而不是你要针对 chat GPT 这样的 surface area 做优化时会用的策略。
05:16
Again, you're going to be doing the same jobs, you're going to be creating content, you're
再说一次，你还是会做同样的工作，还是会创作内容，还是会打造你公司的品牌，还是会为 technical SEO 做优化，但真正变的是战术上的优先级。
05:19
going to be building the brand of your company, you're going to be optimizing for technical SEO,
比如，从 Google 和 ChatGPT 的角度来看，好的内容到底意味着什么？
05:25
but the tactical priorities is where it changes.
归根结底，SEO 仍然存在，只不过现在已经扩展了。
05:28
For example, what does good content actually mean from a perspective of Google versus chat
我们现在有了 AO，这两者放在一起，我称之为 organic search。
05:33
GPT?
我们还是在做同样的三项工作，只不过战术上的优先级稍微变了一点，我会……
05:34
Ultimately, SEO is still a thing, it's now expanded.
05:37
We now have AO, those together, I call organic search.
05:41
We're doing the same three jobs, but the tactical priorities have just changed a bit, I would
05:47
say.
哎，
05:48
Could you talk a little bit about, like, if you are a business owner out there and you're
你能不能稍微聊一聊，比如说，如果你是个企业主，在外面做生意，而且你
05:55
marketing, you have your website out there, you got your social, how has this changed?
在做 marketing，有自己的网站，有自己的社交媒体，这都发生了哪些变化？
05:59
If you were to take a snapshot of the industry, and a long time ago, I was in the digital
如果你给这个行业拍个快照，很久以前，我在 digital marketing
06:05
marketing industry for a while, and that was obviously before AI came thing, and you
行业待过一段时间，那显然是在 AI 出现之前，而你
06:10
were focused on these things minus the AI engine optimization, and so how is that changed?
当时专注的是这些东西，还没有 AI engine optimization，那这又发生了哪些变化？
06:17
How is the thinking of the marketing department changing to accommodate the fact that you
marketing 部门的思路是怎么变的，来适应这样一个事实：
06:23
now have this whole way that people are going out and searching?
现在人们出去搜索的方式已经完全不同了？
06:27
All of us are AI first, that's very different from maybe 15 years ago, how should business
我们所有人都是 AI first，这跟大概 15 年前很不一样，企业主
06:34
owners listen, listening, be thinking about that in terms of how they approach the whole
该怎么倾听、去听，又该怎么从他们处理整件事的方式
06:39
thing?
去思考这一点？
06:40
Yes, so broad topic.
是的，所以这是个很宽泛的话题。
06:41
It is.
确实是。
06:42
Sorry.
抱歉。
06:43
No, no, no, no, it's all good.
不不不不，没关系。
06:46
I would say the first thing is bio-behavior, so people researching and consuming information
我会说，第一件事是 bio-behavior，所以那些研究并消费信息的人
06:53
inside elements has changed bio-behavior.
内部元素已经改变了 bio-behavior。
06:57
The symptom of this is people will see that, hey, where's all my organic traffic gone?
这件事的症状就是，人们会看到，哎，我的 organic traffic 都去哪了？
07:03
It's because that organic traffic, a big chunk of it, was created by people consuming information
这是因为那些 organic traffic，其中很大一部分，是人们在你的网站上消费信息带来的，
07:10
on your website, but if people are consuming information inside an LLM, the LLM is the new
但如果人们是在 LLM 里消费信息，那 LLM 就是新的
07:15
website visitor that taken that information, they're chewing it up and they're spitting
网站访客，它拿走了那些信息，把它们嚼碎，再吐
07:20
it back out to the user inside an LLM, and so you've now lost those clicks.
回给 LLM 里的用户，所以你现在就失去了那些 clicks。
07:24
As I think that's the first thing, and so the implications there are, or how do we measure
我觉得这是第一点，所以这其中的含义是什么，或者说我们该怎么衡量
07:30
that?
这一点？
07:31
It's really challenging, like I'm a marketer, Ben is the engineer, and measurement has
这真的挺有挑战的，比如我是做营销的，Ben 是工程师，而衡量效果对营销人员来说一直就像一场噩梦，所以 zero-click researchers 这个概念已经存在有一阵子了。
07:37
always been like a nightmare for marketers, and so this concept of zero-click researchers
我觉得 AI 只是把这件事加速了一点，我们一直在社交平台上应对这种情况，我觉得现在 organic searches 开始填补这一块，所以 bio-behavior 肯定发生了变化。
07:44
has been around for a while.
这可能和移动端刚兴起时非常相似，我觉得我们正在这方面看到一些变化。
07:46
I think AI has just sped that up a bit, we've been dealing with that on social platforms,
然后还有，我刚才提到过，现在你的网站多了一个新的访客。
07:53
I think now organic searches start to fill that, so definitely a change in bio-behavior.
07:59
It's probably very similar as when the mobile came around, I think we're seeing some changes
08:04
there.
08:05
And then also, I touched on it there, there is now just a new visitor to your website.
08:13
So websites, for the last however many decades they've been around, have really been optimized
所以 website 这东西，不管已经存在了几十年，一直以来其实都是围绕人类访客来优化的，这就是为什么会有 user experience。
08:19
around the human visitor, that's why we have user experience.
我们怎么设计导航，这就是为什么 website 今天长成这个样子，对吧？
08:24
How do we make the navigate, that's why websites look the way they do today, right?
我们怎么设计 navigation bar，比如 conversion rate、optimization，所有这些都是为了人类访客，而现在它看到的是 agents 在访问你的 website。
08:28
How do we shape the navigation bar, like conversion rate, optimization, all these things
所以我觉得这里面有一些很有意思的考量：它们在找什么，以及你的 website 需要处于什么状态，才能让它们获取信息并理解你。
08:33
are for a human visitor, whereas now it sees agents visiting your website.
08:40
So I think there's some interesting considerations there of what are they looking for, and what
08:45
state does your website need to be in for them to be able to access information and understand
08:51
you.
08:52
And so I think there's some considerations there.
所以我觉得这里面有一些要考虑的点。
08:54
I don't know where this ends.
我不知道这最终会走向哪里。
08:56
I know there are a lot of companies out there that are building towards this.
我知道有很多公司正在朝着这个方向构建。
09:00
You have one version of your website for humans, one website version for agents, but
你有一个给人类看的网站版本，一个给 agents 的网站版本，但
09:07
I'd say those are the, and then obviously there's a bunch of implications, like downstream
我会说那些就是，然后显然会有一大堆影响，比如 downstream
09:11
of all of those things.
所有这些事情的。
09:14
So does that answer your question?
所以这回答你的问题了吗？
09:15
I don't have to go to deeper into any of that.
我不需要再深入讲这些了。
09:17
No, it's a great start.
不，这是个很好的开始。
09:18
I appreciate kind of laid the landscape out there, and I know Daniel has one, but after that
我很感谢你算是把整体格局铺开了，我知道 Daniel 也有一个，但在这之后
09:23
I got plenty more.
我还有不少要补。
09:24
Yeah.
嗯。
09:25
I think my, what triggered in my mind, Liam, as you were talking about that, is that measurement
我觉得，Liam，当你说到那个的时候，我脑子里冒出来的是 measurement
09:31
piece, one of the things that I think I appreciate about your all's approach, and both from a
这一块，我觉得我很欣赏你们大家做法的一点，而且无论是从
09:37
company standpoint, but also a research standpoint is you're interested in knowing actually how
公司的角度，还是从研究的角度来说，你们都有兴趣真正了解
09:43
the mechanics work around these things and understanding it at a deeper level.
这些事情的 mechanics 到底是怎么运作的，并且从更深的层次去理解它。
09:48
Then I'm wondering if kind of turning to that technical side, you've published a variety
然后我在想，如果稍微转到技术那一边的话，你其实已经发表过各种各样的
09:54
of research over time, and you'll have more coming, heard a little bit about that before
研究，随着时间推移，而且你还有更多要出来，之前听说过一点关于这个的，
09:59
the interview, and we were talking, but for example, like citations or mentions in these
在采访之前，我们聊过，但比如说，像 citations 或 mentions 在这些
10:06
answer engines, or in AI systems, from a technical standpoint, like obviously I, as a human
answer engines 里，或者在 AI systems 里，从技术角度来说，显然我作为一个人
10:13
can go into chat GPT or Gemini or caught or whatever, and I can test out some prompts
可以进到 ChatGPT 或者 Gemini 或者 caught 或者随便什么，然后我可以测试一些 prompts
10:18
and see what's cited.
看看什么被引用了。
10:21
How from a technical standpoint, what's required from the technical standpoint, to actually
那么从技术角度来说，到底需要什么，从技术角度来说，要真正
10:26
have a scaffolding and a mechanism that you can track your, what does it mean to track
有一个 scaffolding 和一个 mechanism，让你可以 track 你的，track 到底意味着什么
10:34
your visibility in these systems over time?
随着时间推移，你在这些系统中的可见度如何？
10:37
What are the relevant things, like I mentioned a couple of things, citations mentions, what
相关的因素有哪些，就像我提到过几个，citations、mentions，那
10:41
are the relevant things that you're looking at, and at a high level, like what does it
你正在关注的相关因素是什么，而且从高层来看，要真正做到这些需要
10:46
take to put the right tooling in place to actually measure and track that?
把合适的 tooling 部署到位，去实际衡量和追踪这些，需要什么？
10:51
Yeah, as Lee mentioned, it's a big question, but it's a great one, so I would say this
是啊，就像 Lee 提到的，这是个很大的问题，但也是个很好的问题，所以我会说，这
10:57
is probably the thing that is pretty shaken a lot of the SEO industry at its core a lot
大概就是那件从核心上很大程度上震动了整个 SEO 行业的事，在
11:02
in the sense that everyone kind of showed up in the past with traditional SEO and said,
某种意义上，过去大家带着传统 SEO 出现时都会说，
11:07
look great, let's kind of color by, as it's like paint by numbers kind of thing, like you
看起来很棒，咱们就照着填色，就像按数字涂色那种事，就像你
11:12
fill in the colors and we do this, we do that, everybody's great, everyone's happy, and
把颜色填进去，我们做这个、做那个，大家都很好，每个人都很开心，
11:16
nobody actually thinks about what's going on here.
而其实根本没人去想这里到底在发生什么。
11:19
And you can go into like, essentially unless there was a Google core update, you essentially
而且你可以进入那种，基本上除非有 Google core update，你基本上
11:23
like the rankings were essentially very deterministic and you got to set a key words and it was
rankings 基本上是非常 deterministic 的，你只要设好 keywords，它就很
11:27
like pretty fine, I, pretty manageable, all pretty controllable and again paint by numbers.
就挺不错的，我，挺好管理的，全都挺可控的，而且又是在按数字填色。
11:33
And now we've kind of like gone into a system and go read deep on this as needed, but we've
而现在我们算是进入了一个系统，需要的话就去深入读这个，但我们已经
11:39
gone to a space where there are sources of like a non-determinism or stochastic elements
进入了一个空间，那里有类似 non-determinism 或 stochastic elements 这样的来源
11:47
that people need to start modeling around.
大家需要开始围绕这些做 modeling。
11:48
And the SEO industry, and we've seen it as a whole, it doesn't have like great statisticians
而且 SEO 行业，我们整体上也看到了，它并没有那种很厉害的统计学家
11:54
or like great data science literacy, just being a blunt.
或者说很强的 data science 素养，说白了就是这样。
12:00
And if you're any serious like statistician or control engineer, you would look at the
而如果你是个认真的统计学家或者 control engineer，你会去看
12:04
system you're probing and you would say, well, like what are the requirements on the signals
你正在探测的那个系统，然后你会说，嗯，那对这些 signals 有什么要求
12:09
I'm trying to measure here in order to bound the noise on these measurements essentially?
我在这里试图测量，本质上是为了限制这些 measurements 上的 noise，对吧？
12:16
And the quantities you're trying to measure.
以及你试图测量的那些 quantities。
12:19
And then you would say, okay, well, then you'd set up your model of the system, essentially
然后你会说，好吧，那你就建立这个系统的 model，基本上
12:23
a lot of these LM agents are essentially gray boxes, they're not totally black boxes,
很多这些 LM agents 本质上就是 gray boxes，它们不完全是 black boxes，
12:27
there are research papers out there that are hints in the network traffic and some of
外面有些 research papers，其实就是 network traffic 里的线索，而其中一些
12:31
the difficulty cryptic pockets is also information there.
cryptic pockets 的难点，在那里也是信息。
12:35
And there's also like distilled models.
而且还有像 distilled models 这样的东西。
12:37
So when I looked at the training data, you can also figure out like some information
所以当我看 training data 的时候，你也能从中推断出一些信息
12:40
from the distilled models.
来自 distilled models。
12:41
And so the thing is to set up basically an experimentation kind of engine or pipeline
所以关键是，基本上要搭建一个 experimentation 那种引擎或 pipeline
12:47
where we want to call it and then say, well, how do I bound the, you mentioned their
我们想调用它，然后说，好吧，我要怎么界定那个，你提到了他们的
12:51
citation, like mentioned right, share a voice, how do I bound the uncertainty on those metrics
citation，就像刚才提到的，share a voice，我要怎么界定那些 metrics 上的 uncertainty
12:56
and what do I need to bound to basically bring that uncertainty under, let's say, 5% margin
那我到底需要 bound 什么，才能基本上把那个不确定性压到，比如说，5% 的 margin
13:01
of error.
of error 以下。
13:02
And to be honest, no one in the industry is really doing this or even thinking about if
说实话，这个行业里根本没人真的在做这个，甚至都没在想，如果
13:06
people go on, they buy a profound or some other thing and they're like, here's my problems,
人们继续下去，买了 profound 或者别的什么东西，然后说，这是我的问题，
13:11
let's go away, go do this.
咱们走吧，去做这个。
13:12
I'm profound on interested in either, they just like to say something.
我对 profound 也不是特别感兴趣，他们只是喜欢说点什么。
13:16
And then everyone says, oh, the number goes up, how lovely.
然后大家都说，哦，数字涨了，可真好啊。
13:19
And the SEO industry has like an incredible ability to point at numbers going up and
而 SEO 行业有一种不可思议的能力，就是指着上涨的数字，然后
13:26
claim that it was them.
13:29
And then has an also incredible ability to point at a number that's going down and claim
13:33
that school core update.
13:38
So yeah, I just think in general, I can go into like how we can't get about this and
13:42
how like the various systems are like to space there and how we think about bounding
13:46
this randomness.
13:47
But in general, you need to approach it, like how I'm going to model this system that
13:52
I know hints about and what the data we have, and how do we bound that basically uncertainty
13:58
on those quantities.
就那些数量而言。
13:59
We're trying to measure it for, as you mentioned there, Chris, like for the business that
我们正试着去衡量它，就像你在那儿提到的，Chris，比如对那些企业来说，它们
14:02
has whatever they're selling, you know, gardening tools, whatever it is, right?
不管他们卖的是什么，你知道，园艺工具，随便什么，对吧？
14:07
That's the, that's what people need to be getting to and no one's really talking about
这才是，这才是人们需要真正达到的，而没人真的在谈
14:10
that.
这个。
14:11
And I said that it does take a lot of time.
我说过，这确实要花很多时间。
14:14
But I also think we've got to the point where it's like we've got to like stock market
但我也觉得，我们已经到了这种地步，就好像我们得喜欢股票市场
14:17
dynamics.
动态。
14:18
Like there was a change in Reddit recently and, you know, no one also renews, isn't meaningful.
就像最近 Reddit 有个变化，而且你知道，也没人续订，这没什么意义。
14:24
So lots of times we have to come to our clients and say, so let's say I'll give you
所以很多时候我们得跟客户说，比如说，我会给你
14:28
me for example, right?
拿我举例，对吧？
14:29
Yeah, there'll be like, you'll probably see an end on the background there and someone
是啊，就会有那种，你大概会看到那边背景里有个 end，然后有人
14:32
will say, like, oh, I hate to see news, whatever it is.
会说，哎呀，我讨厌看到新闻，管它是什么呢。
14:34
And then they come down and they say, oh, uh, Dow Jones is down to our percent.
然后他们下来就说，哦，呃，Dow Jones 跌到我们那个百分比了。
14:37
Everyone freaks out.
所有人都慌了。
14:38
And if you actually look to the extent of deviation on the last year, you'd be like, well,
而如果你真的去看过去一年的偏离幅度，你就会觉得，嗯，
14:42
this is not statistically significant as a very basic check.
作为一个非常基础的检查，这并不 statistically significant。
14:46
Like it's very basic, but we need to print news.
就像，这非常基础，但我们需要发布新闻。
14:48
So we do that.
所以我们就这么做。
14:49
And the SEO community has a bit, a bit around that.
而且 SEO 圈子里对此也有一点点，一点点讨论。
14:51
And also people just want to like get eyeballs on stuff.
而且人们也只是想，就是，给东西吸引点眼球。
14:53
But again, you know, a lot of people waste a lot of marking hours and a lot of engineering
但再说一次，你知道，很多人浪费了大量的 marking hours 和大量的 engineering
14:57
time around things and aren't just least significant.
时间花在一些事情上，而且这些东西并不只是 least significant。
14:59
That's a very basic thing that we can check.
这是我们可以检查的非常基础的一件事。
15:01
Like that's not like, that's not rock studs.
就像，那不是，那不是 rock studs。
15:03
But it does a very simple example.
但它确实是个非常简单的例子。
15:04
Anyway, that's why I think we're like, I don't like state of the traditional SEO where
反正，这就是为什么我觉得，我们有点像，我不喜欢传统 SEO 现在的状况，
15:07
we are now and where people are freaking out around it and where we need to add.
我们现在所处的位置，人们围着它抓狂，而且我们还得往里加东西。
15:16
I don't know about you, but I'm sick and tired of showing up at AI or technology events
我不知道你怎么样，但我真的受够了去参加 AI 或科技活动，
15:23
and immediately realizing that they're just going to be full of sales pitches or things
然后马上发现它们无非就是一堆推销话术，或者一些
15:28
that aren't useful for my day to day.
对我的日常没什么用的东西。
15:30
I have more than enough to fill my time and I really need to focus on bringing back value
我有足够多的事情来填满我的时间，我真的需要专注于把价值带回来。
15:36
to my company and my day to day work.
对我的公司和我每天的工作来说。
15:40
That's exactly what the Midwest AI Summit is about October 15th in Indianapolis, Indiana.
这正是 Midwest AI Summit 的主题，10 月 15 日在 Indianapolis, Indiana。
15:47
We as the podcast are partnering as a media partner with the Midwest AI Summit and there's
我们这档播客正在作为媒体合作伙伴和 Midwest AI Summit 合作，而且
15:52
just an amazing amount of practical value that you'll get if you attend the Midwest AI Summit.
如果你参加 Midwest AI Summit，你获得的实用价值会多得惊人。
15:59
There's actually an AI engineering lounge where you can sit down with folks like me and
其实那里有一个 AI engineering lounge，你可以坐下来和像我这样的人，还有
16:04
other AI experts to get feedback on your roadmap, your architecture, tooling, how you're going
其他 AI experts 一起，获得关于你的 roadmap、你的 architecture、tooling，以及你打算怎么
16:10
about your AI transformation in your company and there's amazing speakers, great food.
在你的公司里推进 AI transformation 的反馈，而且还有很棒的讲者和美食。
16:15
This is one not to miss.
这个绝对不能错过。
16:17
This is going to happen in Indianapolis October 15th of this year, 2026.
这场活动将于今年，也就是 2026 年 10 月 15 日，在 Indianapolis 举行。
16:23
We already had one last year.
我们去年已经办过一场了。
16:25
It was an amazing success and you're not going to want to miss the one this year.
那场非常成功，今年的这一场你肯定不会想错过。
16:29
Check it out at MidwestAISummit.com and you can get 20% off with the Code Practical AI20.
去 MidwestAISummit.com 看看吧，用 Code Practical AI20 可以打八折。
16:38
So check it out at MidwestAISummit.com and use the Code Practical AI20 for 20% off registration.
所以去 MidwestAISummit.com 看看吧，注册时用 Code Practical AI20 可以享受八折优惠。
16:50
So Ben, I want to ask a follow-up question.
那么 Ben，我想追问一个问题。
16:53
You did a great job at kind of aligning a foundation for maybe some of the things that we should
你做得很好，某种程度上帮我们奠定了一个基础，让我们在谈到这个话题时，可以从统计角度去思考一些应该考虑的事情。
16:59
be thinking about statistically as we approach this topic.
17:05
I'm wondering if you could help our listeners understand like if I put in a prompt to one
我想知道，你能不能帮我们的听众理解一下：如果我把一个 prompt 输入到这些系统里的某一个，然后某个品牌冒出来了，或者我想让它出现的东西出现了，它可能是通过哪些机制冒出来的？
17:11
of these systems and a brand shows up or a thing that I want to show up shows up, what
而且我知道你做过一些研究，涉及 citations、web search，以及它们和 Reddit、training data 的对比。
17:18
are the mechanisms by which that can show up?
在我们具体聊这些之前，你能不能帮我们理解一下，幕后到底发生了什么，我可能怎样从机制上出现在其中一个系统里？
17:21
And I know you've done some research across like citations and web search versus Reddit
可以。
17:29
versus training data.
17:31
Before we get into any of those very specifically, could you help us understand what happens
17:37
behind the hood, how might I show up mechanically in one of these systems?
17:43
Yeah.
17:44
So let me dig into it just before I do, I would say to anyone that anyone listening, for
所以，让我先深入讲一下，不过就在我开始之前，我想对任何人、任何在听的人说，千万
17:48
sure, do not run down a rabbit hole of being like, this is this one prompt and I would
不要一头扎进兔子洞里，觉得“这就是那一个 prompt，我会
17:52
love to rank on it.
很想在它上面排名靠前。”
17:53
You want to basically model from a topical domain standpoint and get into more of that.
你想基本上从 topical domain 的角度来建模，并更多进入那个方向。
17:59
Basically, what happens is essentially your prompt is fed in, it's tokenized and then
基本上，本质上发生的是，你的 prompt 被输入，它被 tokenized，然后
18:06
ingested by the LM and then there's essentially a reasoning stage, right?
被 LM 摄取，然后基本上就有一个 reasoning stage，对吧？
18:12
So basically, the transformer basically has embedded your prompt and now looks at all those
所以基本上，transformer 基本上已经把你的 prompt embed 了，现在它看着所有那些
18:18
tokens and starts generating its reasoning stage based on its training weights and basically
tokens，并基于它的 training weights 开始生成自己的 reasoning stage，基本上
18:23
it has a, it's running this forward pass and there's two sources of randomness and
它有一个，它正在跑这个 forward pass，然后有两个随机性来源，而且
18:29
reason I mentioned this is kind of important, it's like the kernel function.
我提这个的原因挺重要的，它就像是 kernel function。
18:32
So there's like floating point error in GPUs, that's one source of randomness and the second
所以 GPUs 里会有类似 floating point error，那是一个随机性来源，而第二个
18:36
source of randomness is the actual temperature itself, like the deliberate temperature randomness
随机性来源就是实际的 temperature 本身，就像那种刻意加入的 temperature 随机性
18:42
that generates the next token.
它用来生成下一个 token。
18:43
And then the final part of this is like, it's also speaking with me and that this is now
然后最后一部分是，它也在跟我说话，而现在这就变成了
18:47
there's now a fingerprinting on these LM.
现在这些 LM 上有了 fingerprinting。
18:49
So they'll also nudge and create hashing functions in the background to know where they
所以它们也会在后台轻轻推动，并创建 hashing functions，来知道它们到底在哪儿
18:53
come they came from.
来自它们来的地方。
18:54
Anyway, so it goes through this reasoning phase, that's how you say like a best, I don't
总之，它会经历这个 reasoning phase，这就像你说的，好比一个最佳，我不
18:59
know, gardening serves in, that's a doublet, I don't use that example, and then start reasoning
知道，gardening serves in，那是个 doublet，我不用那个例子，然后开始 reasoning
19:05
and saying, oh, it'll have a certain set of criteria, it'll do some reasoning before
并说，哦，它会有一套特定的 criteria，它会在
19:08
it makes any queries with its retrieval engines.
用它的 retrieval engines 发出任何 query 之前先做一些 reasoning。
19:11
And it's exposed to a series of tools and what is also going to be the retrieval engine.
而且它会接触到一系列 tools，以及也将成为 retrieval engine 的东西。
19:14
I'm saying a series of tools because when we get to agents, they could be exposed
我说一系列 tools，是因为等我们讲到 agents 时，它们可能会接触到
19:17
to whole hosting, but the thing is not aware of.
整个 hosting，但那个东西并不知道。
19:21
Then it accesses this retrieval engine, creates a set of basic query fan heads, you've probably
然后它会访问这个 retrieval engine，创建一组基本的 query fan heads，你可能
19:26
heard the term, and then pulls back in basically a set of initial candidates essentially for
听过这个词，然后基本上拉回来一组 initial candidates，本质上是为了
19:36
to be pulled into the context window.
被拉进 context window。
19:38
But before that happens, essentially, they look basically, it's not always exactly the
但在那发生之前，本质上，他们基本上会看，它并不总是完全
19:41
same algorithm, but deep mind having a greased algorithm, so it's called, and I basically
相同的 algorithm，但 DeepMind 有一个 GREASED algorithm，它是这么叫的，而我基本上
19:46
it's a way to avoid hallucinations.
这是一种避免 hallucinations 的方法。
19:48
So once they pulled in these, let's say, 40 to 60 sources for various places, it's
所以一旦他们拉进来这些，比如说，来自不同地方的 40 到 60 个 sources，它就
19:54
done as, let's say, another set of analysis on which is almost across the board, it's
被当成，比如说，另一组分析，这几乎是全面性的，它是
19:59
very similar to the agree for algorithms from deep mind, it's very similar in anthropic
20:05
models, it's obviously, it's in general as models and also chaffity, we've seen this
20:10
in our own analysis, and also a cloud move to it recently when they moved to Fable and
20:16
Fable 5.1, essentially what it's doing now, I'm trying to see, what is the consensus
20:19
so I can basically use the majority voting system, so I don't make a mistake, I don't
20:24
have that hallucination.
20:26
And then it says, okay, cool, great, this is our kind of like just still set of context
20:30
we want to work with now, let's say, generate the final response, it's actually from that
20:35
cleaned up context, and this actually kind of ties back to like, and read it, people
把 context 清理干净了，这其实有点绕回到，怎么说呢，然后读一下它，大家因为 final citation 都快疯了，再说一遍，我看这件事的方式是，嘿，很明显大家都说，哦，我们当然想知道 citations 是怎么回事，然后它就在 final output 里，但在我们走到那一步之前，还有一大堆其他部分，所以我讲了 weights，你是怎么在 weights 里体现出来的，我们最近还做了些研究，model 是怎么 reasoning 的，它在想什么，当它围绕你的 domain 做出 chord decisions 时，或者说那是 bottom of funnel、top of funnel，不管是什么，它会走一套 criteria，这是它的内部逻辑，本质上就跟人类会做的一样，它不是人类
20:40
freaking out about the final citation, and again, the way I look at like, hey, obviously
20:44
everyone's like, oh, we'd love to know about citations and then she's in the final output,
20:48
but there's all this other part before we got there, so I talked about the weights, how
20:52
you're showing up in the weights, and we did some research on this recently, how is
20:55
the model reasoning, what is it thinking, when it makes chord decisions around your domain
21:00
or be that bottom of funnel, top of funnel, whatever it is, it has a criteria it's going
21:06
through, it's internal logic, just like a human word, essentially, it's not a human
21:09
word, you get my point.
就是说，你懂我的意思。
21:10
And then finally, you have like your retrieval engine in a response, but you need to be
然后最后，你在一个 response 里会有你的 retrieval engine，但你需要
21:14
looking across all of those four pillars in order to dominate your domain.
横跨这四大支柱去看，才能主导你的 domain。
21:19
That's how we look at it for all of our clients.
这就是我们为所有客户看这件事的方式。
21:21
There are lots of things to dive into, but before we do that, I'm curious actually want
有很多东西可以深入聊，但在我们这么做之前，我其实很好奇，想
21:27
to go back to Liam for a second, and like, as I'm listening to Ben explain these things,
先回到 Liam 那儿一下，而且，怎么说呢，当我听 Ben 解释这些的时候，
21:32
the thing in my head was kind of something that you were saying early on about, you know,
我脑子里想的是你一开始说的某个东西，你知道，
21:36
how it changes behavior, and I'm curious as a marketer, I like the ability to go back
它是怎么改变行为的，而且我作为营销人很好奇，我喜欢能够回去
21:41
and forth between the technical and the human behavior side, how is that changing how
以及在技术和人类行为两侧之间来回切换，这会如何改变
21:48
the human behind, you know, that's not now that you have the agent going and doing this,
背后的那个人，你知道，现在已经不是那样了，因为你让 agent 去做这件事、去执行这件事，
21:53
what does that mean for the human that now has this proxy in the agent going out and doing
这对那个人来说意味着什么？现在这个 agent 里有了这个 proxy，代替他们出去做
21:58
that, and how is that changing the behavior and potentially the transactions that are
这些事，而这又会如何改变行为，甚至可能改变那些
22:04
following up on that, just to kind of come full circle for a moment.
随之而来的交易，稍微回到一开始那个点，来个闭环。
22:09
Yeah, so I guess I'll look at this from like demand and supply side.
对，所以我想我会从需求侧和供给侧来看这件事。
22:12
So from a demand side, the buyer, I think this is just a much better deal, right, because
所以从需求侧来看，买家这边，我觉得这就是一笔好得多的交易，对吧，因为
22:17
if we look at search behavior pre-LLM, trying to find relevant information on Google is just
如果我们看 pre-LLM 时代的搜索行为，想在 Google 上找到相关信息简直就
22:23
a nightmare, right, because, you know, us marketers just ruin everything, and you'd look
简直是一场噩梦，对吧，因为你知道，我们这些做营销的就是会把一切都搞砸。然后你去找东西的时候，就像，你只会拿到 10 个 URL 的列表，然后还得一个个去访问、去研究，对吧。
22:29
for things and like, you just get the list of 10 URLs that you then have to go visit
而如果拿今天这个跟 chat-based LLMs 比，它更像对话，所以我可以这样说：嘿，我是 Liam，这是我的公司，它在这个行业，我们的营收是这个数，我们现在真的很挣扎，这些是我们目前的核心约束。
22:35
and research, right, and whereas if we compare that today to like chat-based LLMs, it's
我们在找这样一个解决方案，这些是这个解决方案要有的属性，你知道，我们在找一款软件，必须有 free trial，必须按月收费，必须
22:43
more conversational, so I could be like, hey, I'm Liam, this is my company, it's in this
22:48
industry, we're doing this revenue, we're really struggling, these are our core constraints
22:52
at the moment.
22:53
We're looking for this solution, these are the attributes of the solution, you know,
22:57
we're looking for a piece of software, must have a free trial, must charge monthly, must
23:02
integrate with HubSpot or CRM, all of these requirements, I can frontload that, and then
和 HubSpot 或 CRM 集成，所有这些需求，我可以提前把这些都做好，然后
23:08
the LLM does its sting, and it comes back with personalized recommendations.
LLM 发挥它的作用，然后返回个性化的推荐。
23:12
Now I'm not saying that marked as a, you know, that's our job to influence those recommendations,
现在我不是说那个被标记成 a，你知道，影响那些推荐是我们的工作，
23:17
but it's just a far better deal than like, clicking through all those websites and finding
但这比起点开所有那些网站再自己找，要划算太多了，
23:21
the information myself, like me personally, I just use it for everything, right, date
信息，像我个人的话，我什么都用它，对吧，date
23:25
night, you know, mill prayer, like buying software, everything, so from a demand side, I think
night，你知道，mill prayer，比如买软件，所有事情，所以从需求侧，我觉得
23:31
it's a greater deal, and so therefore I think more people are going to take the deal.
这是更划算的交易，所以因此我觉得更多人会接受这个交易。
23:36
From a supply side, the people who are the vendors, I would say, I probably look at this
从供给侧来看，那些作为供应商的人，我会说，我可能会看这个
23:44
from two ways.
从两个方面来说。第一，如果人们把所有这些上下文都前置了，那现在就会创造出那些模型所使用的查询的一个非常长的长尾分布，它不再只是最好的cold email软件，那是人们在Google上会搜索的东西，而是所有这些实体和东西都挂在，你知道，附加在末尾，对吧，integrations、business model、所有这些独特的需求。所以这就创造出了这种查询的长尾分布。现在长尾的好处在于它们的竞争更小，所以我觉得这
23:45
So number one, if people are frontloading all of that context, that now creates a really
23:51
long tail distribution of queries that those models are using, it's no longer just
23:56
the best cold email software, which is what people would search on Google, it's all
24:01
of these entities and things are kind of hang, you know, appended onto the end, right,
24:07
the integrations, the business model, all of these unique requirements.
24:13
So that creates this long tail distribution of queries.
24:16
Now the great thing about long tails is they're less competitive, and so I think this has
24:21
really been where the opportunity has been over the last year, is where ultimately I would
过去一年里真正的机会所在，最终我会把它
24:26
simplify it down to relevancy, be authority.
简化成 relevancy 和 authority。
24:32
Now there's some nuance in there, but this is really where the edge was, and this is
现在这里面有些细微差别，但这确实是优势所在，而这就是
24:36
where say if I'm competing against HubSpot, who has dominated Google Serbs for all the
比如说，如果我在和 HubSpot 竞争，它已经主导了 Google Serbs 上所有
24:42
money keywords I'd love to rank for, well now I can create relevant content for the
我特别想排名的 money keywords，那现在我可以创作相关内容，来针对
24:47
super niche queries that my buyer is searching inside an LM.
我的买家在 LM 里搜索的超细分查询。
24:52
So I think there's that perspective, is the long tail distribution means we can create
所以我觉得有这样一种视角，就是 long tail distribution 意味着我们可以创作
24:56
relevant content, that targets there, and then I think there's the authority consideration
针对那里的相关内容，然后我觉得还有 authority 方面的考量
25:03
is, okay, well everyone has relevant content, how does the model then reason over who
呃，好吧，既然每个人都有相关内容，那模型接下来怎么推理谁
25:10
to trust, who does it cite, who to trust, historically with traditional SEO, you know, we look
该信谁，它引用谁，该信谁，过去在传统 SEO 里，你知道，我们会看
25:16
at PageRank, how we know Google assesses authority of pages and domains, again it comes
PageRank，我们怎么知道 Google 评估页面和域名的权威性，同样，这又归结
25:24
down to consensus, it's not as weighted, it is my understanding, Google have publicly
于共识，它的权重没那么高了，据我理解，Google 已经公开
25:30
said this, it's not as weighted nowadays, what's like the LLM version of that, and this
说过这一点，现在权重没那么高了，那对应的 LLM 版本是什么，而这
25:37
is what seems to be changing a lot at the moment, right, because marketers use a bunch
正是目前似乎变化很大的东西，对吧，因为营销人员会用一堆
25:43
of tactics, they work well, everyone starts using those tactics, law of sheety click-throughs,
策略，它们很有效，所有人都开始用这些策略，law of sheety click-throughs，
25:49
kicks in, and so then they diminish, and then we need to find the next edge, or like,
就生效了，于是它们就衰减了，然后我们就需要找下一个 edge，或者类似，
25:54
you know, these vendors like patch, patch the vulnerability basically, this is basically
你知道吧，这些 vendor 基本上就是去 patch，patch vulnerability，这基本上就是
25:59
what we're doing, we're exploiting systems, you know, SEO is just exploiting Google systems.
我们在做的事，我们在利用系统，你知道，SEO 无非就是在利用 Google 的系统。
26:04
And so I think there's like a whole thing there of like, well, how do I ensure that my brand,
所以我觉得这里面有一整套东西，就是，我该怎么确保我的品牌、
26:09
my brand's information is chosen as a candidate, and the position I want is communicated back
我品牌的信息被选为 candidate，并且我想要的定位能回传
26:17
to the user inside the LLM, like that's how I'm viewing at the moment from like a marketing
给 LLM 里的用户，这就是我目前从 marketing 的
26:21
perspective.
角度看待这件事的方式。
26:22
Yeah, that's really helpful, and I guess to that point, there can be, you mentioned this
是的，这真的很有帮助，我想说到这一点，可能会有，你提到了这个
26:29
Liam, or maybe it was Ben, like trying to get people to think more about like, oh,
Liam，或者也许是 Ben，就是试着让人们更多地去想，哦，
26:34
here's the prompt I want to optimize around, and I know something that like, people have
这是我想围绕它来优化的 prompt，而且我知道，有件事大家从各种角度都跟我说过：哎，你得有个 Reddit strategy，才能在 AI visibility 上做得特别好吧。我们在这次对话开始之前也稍微聊过这个，但我知道你们做过一些研究，既有早期关于这些 LLM 系统里到底什么会被引用的研究，也有更近期关于权重，以及 Reddit strategy 的那种权重和重要性的研究，还有它是怎么在明面上显现出来，跟影响你之前提到的一些幕后活动相比又是怎样的，Ben，你要不要大概帮我们理解一下你做过的一些研究——
26:39
talked about to me from various aspects is, oh, you need like a Reddit strategy to do great
26:46
in AI visibility, and we are chatting a bit about this before the conversation came about,
26:54
but I know you all have done some research, both earlier research on like, what actually
26:58
gets cited in these LLM systems, but then more recent research around the weight that,
27:06
and the weight and the importance of kind of a Reddit strategy, and how that's surfaced
27:10
visibly versus influencing some of that behind the scenes activity that you were mentioning
27:17
before, Ben, do you want to just kind of help us understand some of the research that you've
27:22
done there, and maybe that Reddit piece, or other things that kind of influence that
在那儿做完了，然后也许那个 Reddit 的部分，或者其他那种会影响那个
27:29
citation that are relevant in relation to Reddit.
与 Reddit 相关的 citation。
27:33
Sure.
当然。
27:34
Yeah.
嗯。
27:35
So on the Reddit front, so obviously, as I mentioned, there is obviously the kind of like,
所以说到 Reddit 这块，很明显，就像我提到的，显然有那种，怎么说呢，
27:39
the fourth pillar of like AI visibility, the actual response, and there's the reason
AI visibility 的第四根支柱，也就是实际的 response，然后原因就是
27:43
you are a Tribal stage, and then we do these research on the chativity retrieval engine
你处在 Tribal stage，然后我们做这些关于 chativity retrieval engine 的研究
27:49
at the time, and in an allocated, like an incentive, I think almost a turn of its easy retrieval
当时，而且是在一个 allocated 的、类似 incentive 的东西里，我觉得几乎算是它的 easy retrieval 的一个转折
27:56
slots for Reddit.
给 Reddit 的 slots。
27:57
So if it could find relevant threads for that topic or that query, it would basically
所以如果它能找到跟那个话题或那个 query 相关的 threads，它基本上就会
28:02
inject that into the context we know at the time, and then obviously as it does, as it
把那个 inject 到我们当时知道的 context 里，然后显然，随着它这么做，随着它
28:08
gets those Reddit threads, that's a 30% of it's retrieval, it's like 60 slots, 20% of
拿到那些 Reddit threads，那占它 retrieval 的 30%，差不多是 60 个 slots，而 20% 的
28:13
the allocated to Reddit, most of those were actually rejected citation times, so they basically
分配给 Reddit 的，大多数其实都是 rejected citation times，所以它们基本上
28:18
are kind of used to ground the model and formatted its thinking on often not actually cited
算是用来 ground 这个 model，并 format 它的 thinking，而往往并不会真的被 cite
28:23
in the final response.
在 final response 里。
28:24
And then there's also been, that's one aspect of it, I mean, there's also also the long
然后还有，那也是它的一个方面，我是说，还有还有那个 long
28:30
form Reddit, like AMAs, questions, and Reddit training data is also used in the training
来自 Reddit，比如 AMAs、提问，而且 Reddit training data 本身也会被用在 training data 里。
28:35
data itself.
我觉得可能差不多，通常是在 RLHF 阶段，那里有点像一种来自人类反馈的 reinforcement，而那对此来说是非常宝贵的 training data。
28:36
I think it's maybe around, it's typically around the oral HF stage where it's like a reinforcement
当我们看这个的时候，实际上会看到，存在种族主义，它正以一些方式显现出来，尤其是来自 Jeff and T，以及 Jam，对吧，当我们看 BTT Albus Wars models 时，它不是特别普遍，但你能看到它确实存在。
28:46
from humanity back, and that's a pretty golden training data for that.
所以考虑到这些，我觉得，你知道，就像我说的，人们会说，哦，好吧，照片是
28:51
And we actually see when we look at this, there is tracism, that is showing up in the
28:55
ways, particularly from Jeff and T, as well as Jam, right, when we looked at the BTT
29:01
Albus Wars models, it's not like super-president, but you can see it's there.
29:06
So with those in mind, I think, you know, as I said, people say, oh, well, photos are
29:12
dropped.
掉了。
29:13
It's kind of like saying, you know, it's one signal on the engine, I would say, but it's
这有点像在说，你知道吧，我会说，这只是 engine 上的一个信号，但它不是全部。
29:17
not everything.
而归根结底，如果你想推动你品牌的叙事，或者把它朝某个方向主导，你需要看所有方面：reasoning engine、retrieval 那一侧，以及 response 和 weights。
29:19
And ultimately, if you want to move the narrative for your brand or own it in a certain
所以是啊，我会说，它已经准备好继续做一个非常重要的玩家了。
29:26
direction, you need to look at all sides, the reasoning engine, the retrieval side of
对。
29:30
it, as well as the response and the weights.
29:35
So yeah, I would say it's ready to still very much a big player.
29:39
Yeah.
29:40
Maybe just to check my understanding here, if I'm understanding what you're saying,
也许只是为了确认一下我的理解，如果我理解你说的意思的话，这就会像是，哦，在那个 retrieval 和 query fan-out 阶段，假设我 retrieve 到 20 个 sources，其中 10 个来自 Reddit，说的是非常相似的东西，可能其中一个来自 TechCrunch，讲的是同一件事，就像它印证了那 10 个 Reddit 的东西。
29:44
this would be like, oh, in that retrieval and query fan-out stage, let's say I retrieve
可能是 answer engine 把那 11 个 sources 当作是在确认同一件事，然后它要讲的就是这个。
29:50
20 sources and 10 of them are from Reddit saying a very similar thing and maybe one of them
但它不 cite Reddit 的那些东西。
29:58
is from TechCrunch talking about the same, like it confirms the 10 Reddit things.
它 cite 的是 TechCrunch 那个，因为就像是，我不知道，不管背后是什么原因。
30:04
It could be that the answer engine uses those 11 sources as confirming the same thing
30:10
and that's what it's going to talk about.
30:13
But it doesn't cite the Reddit things.
30:14
It cites the TechCrunch thing because it's like, I don't know, for whatever reasons behind
30:20
the scenes, it cites.
它引用的那些场景。
30:21
Do I have the right understanding here?
我这样理解对吗？
30:23
Exactly.
没错。
30:24
That's exactly it.
就是这样。
30:25
So essentially it's like, I thought they say, once we retrieve those, that's said, that's
所以本质上就是，我以为他们说的是，一旦我们把这些检索出来，也就是说，那就是
30:31
a 20 sources you mentioned and let's say 10 on Reddit, there's a series of processes
你提到的那 20 个来源，假设其中 10 个在 Reddit 上，接下来会有一系列流程
30:36
that go on like someone that would be like rewranking versus the query and then there's also
比如有人会针对 query 做 reranking，然后还会有
30:40
some later research you're often basically like domain, it's hard to even wear chat if
一些后续研究，你通常基本上就是 domain，甚至很难 wear chat，如果……
30:46
you want to send user itself, but yeah, all of those can be as you can add to say, look,
你想把用户本身发过去，但没错，所有这些都可以，因为你可以加上一句，你看，
30:51
we've used Reddit to ground the answer here.
我们用了 Reddit 来 ground 这里的答案。
30:53
We're confident to move forward with this one, but we haven't, we haven't actually cited
我们对推进这个很有信心，但我们还没，我们其实还没引用
30:58
it.
它。
30:59
That's exactly what's going on there.
那边的情况正是这样。
31:02
If you're like me, you need a good amount of help keeping your website updated, launching
如果你像我一样，你需要不少帮助来保持你的网站更新，发布
31:09
new websites, landing pages, et cetera, and you need an actual platform for your company,
新网站、landing pages 等等，而且你需要一个真正属于你公司的平台，
31:16
not just a builder of websites, but a platform where you can launch and continue improving
不只是一个网站构建器，而是一个你可以发布并持续改进的平台
31:21
your site.
你的网站。
31:22
I'm excited to share with you about Framer, our partner, and what they're doing to
我很高兴能跟你分享 Framer，我们的合作伙伴，以及他们正在做的事情，来
31:27
actually enable this sort of work.
真正让这类工作成为可能。
31:30
They have agents integrated into their website platform that helps streamline collaboration,
他们把 agents 集成到了自己的网站平台里，帮助简化协作，
31:37
they can build custom code components, they can manage CMS content, and much more.
它们可以构建 custom code components，可以管理 CMS 内容，还有更多。
31:43
They work side by side with humans.
它们和人类并肩工作。
31:46
Framer is the pro site builder that's for creatives, teams, and businesses that want a professional
Framer 是专业的建站工具，专为创意人士、团队和企业打造，他们想要一个专业的
31:53
site and care enough to get every detail right.
网站，并且足够用心，把每个细节都做对。
31:56
Learn how you can get more out of your site from a Framer specialist or get started building
了解如何通过 Framer 专家从你的网站获得更多价值，或者开始
32:02
for free today at framer.com slash practical AI for 30% off a Framer pro annual plan.
免费搭建，今天就在 framer.com 斜杠 practical AI，Framer pro 年度计划享 30% 折扣。
32:10
That's framer.com slash practical AI for 30% off framer.com slash practical AI rules and
那就是 framer.com 斜杠 practical AI，享 30% 折扣，framer.com 斜杠 practical AI，规则和
32:18
restrictions may apply.
限制可能适用。
32:20
Then I want to do a quick follow up on what you were just talking about, and that's kind
然后我想就你刚才讲的内容快速追问一下，也就是
32:26
of that relationship between retrieval and citation a little bit.
稍微聊聊 retrieval 和 citation 之间的关系。
32:32
It seems like intuitively I might have thought that there was a positive relationship between
直觉上我可能会以为它们之间是正相关的，
32:41
them and that retrievals would lead to citations.
也就是 retrieval 会带来 citation。
32:44
You just talked about the fact that that isn't necessarily the case.
你刚才说到，情况并不一定就是那样。
32:51
It seems like that's a significant thing.
这似乎是一件挺重要的事。
32:53
I would guess that if you're a marketer out there and going with that intuitive approach,
我猜，如果你是个 marketer，采用那种直觉式的方法，
32:58
that that would be a substantial pivot that you'd have to make to accommodate that.
那你就得做一个相当大的 pivot，来适应这一点。
33:05
Could you talk a little bit about what that means?
你能稍微讲讲这意味着什么吗？
33:09
What does it mean the fact that the retrieval and the citations don't necessarily line up
retrieval 和 citations 并不一定以那种方式对得上这件事，
33:14
that way?
意味着什么？
33:15
Liam, I'd also love to hear what you have to say about that on the marketing side.
Liam，我也很想听听你在 marketing 这边怎么看这一点。
33:20
What I would say is it's a big change because typically it's like, look, again, traditional
我想说的是，这是个很大的变化，因为通常来说，就像，你看，又是老一套，传统的
33:26
SEO would say basically go query an index.
SEO 基本上会说，去 query 一个 index。
33:29
We get something back and it's reasonably stable and minus a global core update.
我们会拿回一些东西，而且它挺稳定的，除了 global core update 的时候。
33:32
Now we have essentially multiple steps, multiple query found out to that, and not only that
现在呢，我们本质上有了多个步骤，多个 query 被找出来指向那里，而且不仅如此
33:38
we have re-ranking involved as well as the LLM basically protecting the information itself
我们还有 re-ranking 参与进来，以及 LLM 基本上在保护信息本身
33:45
in this agree style algorithm I mentioned.
在我提到的这个 agree style algorithm 里。
33:49
Basically, just to link back all the possible sources of that citation time, we talk about
基本上，只是为了把那个 citation time 的所有可能来源都链接回去，我们谈论
33:56
the citation time optimization to say how do we end up in the final citation.
citation time optimization，来说我们要怎么最终进入 final citation。
34:01
One of the things is obviously consistency or an consensus.
其中一个点显然就是 consistency，或者说 consensus。
34:06
If I have this long tail query, basically, one of the query found out we've been retrieved
如果我有一个这样的 long tail query，基本上，就是要找出我们在哪个 query 上被 retrieved 了，还有哪些 supporting materials 可能也会在那个 query 上被 ranked，来支持它。
34:09
on, what other supporting materials are out there that are likely to get ranked on that
这就是 consensus。
34:13
query to support it.
我们在这方面有很多做法，等等。
34:14
That's consensus.
然后另一方面是，那是什么在拉这个——甚至在更早之前，就把我们拉向那组他们看到能增加 consensus、会有帮助的 sources。
34:15
We have a number of all that we do on this, etc.
34:18
And then the other side of it is, so what's that pulling that, even before, pulling
34:24
us that set of sources that they see add consensus that will help.
34:28
And I'll see you want to avoid saying, well, let's say there's some anti-consensus information
而且我明白，你想避免说，嗯，假设有一些 anti-consensus information
34:34
out there.
在外面。
34:35
You don't want to avoid that building up, so let's say there's something about your
你不想避免它越积越多，所以假设你的
34:37
brand where it's actually like negative, let's say reviews and that seems to be like
品牌有些东西其实是负面的，比如说评论，而且那看起来像是
34:41
it's publishing multiple places.
它在多个地方发布。
34:42
That'll be a good example.
那会是个很好的例子。
34:43
That's just one thing we've built out a lot here is AI perception.
这只是我们在这里大量搭建的一件事，就是 AI perception。
34:46
One thing we've found is actually quite interesting is that if there's blank space, you could
我们发现有一件事其实挺有意思，就是如果有空白空间，你可以
34:50
almost publish, you can almost get anything cited.
差不多只要发布出来，你几乎就能让任何东西被引用。
34:54
So for instance, certain industries are very, very sensitive about publishing and pricing.
比如说，某些行业对公开信息和定价非常非常敏感。
34:58
And we've found this works insanely well if you publish numbers that are even like yard
而且我们发现，如果你公布的数字哪怕只是像 yard sticker 那样方向大致正确，这效果也会好得离谱，
35:04
sticker directionally correct because there's blank space in the MLM doesn't want to
因为 MLM 里还有空白空间，而它本质上不想失去任何东西。
35:09
lose any essentially.
所以这只是一个很好的具体例子。
35:10
So that's just a good concrete example of it.
同样，如果竞争对手没有在谈这些，你也可以做类似的事情来加以利用。
35:12
Similarly, if a competitor isn't talking about those, you can also do something similar
35:17
to capitalize on.
35:18
And the second thing is even when it basically perceives that information it's got from your
然后第二点是，即便它基本上感知到了从你的brand获取的信息，你在候补名单里是如何呈现的？
35:24
brand, how are you showing up in the wait?
所以它可能对你的brand有非常负面的sentiment。
35:26
So it might have a very negative sentiment around your brand.
如果它看的是来自你brand的数据，那么打个比方，当我们做model rank analysis的时候，挺有意思的，profound作为一个非常负面的sentiment出现在分数里。
35:32
And if it's looking at data from your brand, then let's say, for instance, it's quite funny
基本上，这会影响到他们，这就是我应该告诉你的。
35:36
when we did the model rank analysis, profound as a really negative sentiment in the score.
所以我甚至会落到很简单的事情上，比如你brand的名字是什么？
35:43
And essentially, that's going to impact them, that's what I should tell you.
所以如果你的brand像是一个词，比如说是已经存在的英文单词，在
35:48
So I'm just going to even come down to simple things like what is the name of your brand?
35:52
So if your brand is like a word that is like, let's say already existing English word in
35:58
the dictionary, for instance, document issue or if it has negative connotations with it
比如说 dictionary、文档问题，或者如果它带有负面含义
36:02
or it doesn't align with the like basic overall, let's say it's a security platform and
或者它跟基本的整体定位不一致，比如说它是个 security platform，然后
36:06
it's like openly or something that could have a negative effect.
它就像是公开的或者什么，可能会产生负面影响。
36:11
So there are multiple points there, both on the retrieval side, like consensus as well
所以那里有多个点，既在 retrieval 侧，比如 consensus，也是
36:15
as like how the model is like basically the sentiment it has towards your brand.
比如 model 基本上对你品牌的 sentiment 是怎么样的。
36:21
And also what, so when these models are trained, coming back to the weight side of things,
还有，那个，所以当这些 model 被训练时，回到 weights 这一侧，
36:25
it's all about co-occurrence, right?
这全都是关于 co-occurrence 的，对吧？
36:26
So there's this attention function basically where it's basically going on, looking at all
所以基本上有这么一个 attention function，它基本上在运行，查看所有
36:31
the tokens in a certain space and it's associating you.
某个空间里的 tokens，然后它把你关联起来。
36:33
So if I have like, let's say, guarding tools.com, what are the other entities I've related
所以如果我有个，比如说，guarding tools.com，那我关联到的其他 entities 是哪些
36:39
to as I've gone through the training data?
在我过 training data 的时候？
36:41
And then when it comes to citation time, it's like, well, that also influences like, okay,
然后到了 citation 的时候，就像，嗯，那也会影响，就像，好吧，
36:45
well, this, you know, we're talking about X. So essentially like, let's say the entity
嗯，这个，你知道，我们在聊 X。所以本质上就像，比如说这个 entity
36:49
shows up, that's related to you, you're more likely to get retrieved again.
出现了，它跟你相关，你就更有可能再次被 retrieved。
36:52
So it's all like essentially like an embedded form of a knowledge graph, so typically most
所以这基本上就像是一种 embedded 形式的 knowledge graph，所以通常大多数
36:57
of these indexes from Google, et cetera, are built on knowledge graphs, but now we've
这些来自 Google 等等的 indexes，都是建立在 knowledge graphs 上的，但现在我们已经
37:01
been embedded form in this transformer layer.
一直以 embedded form 的形式存在于这个 transformer layer 里。
37:03
But you've got to try and assess that.
但你得试着去评估这一点。
37:05
So I'd say basically there are three areas, well, or primarily two areas, the weights and
所以我会说，基本上有三个领域，嗯，或者主要就两个领域：weights 和
37:10
how it's reasoning on those weights for you.
它如何在这些 weights 上为你进行推理。
37:12
And then the consensus around that and the weights embed factual, factual accuracy, sentiment
然后就是围绕这个的 consensus，以及 weights 所 embed 的 factual、factual accuracy、sentiment
37:17
and co-occurrence.
和 co-occurrence。
37:18
That's how I think about it.
这就是我思考这件事的方式。
37:19
But yeah, it could be like, it's not, it's not easy to, it's not easy to operationalize
但是啊，可能就像，这并不，这并不容易，真的不容易 operationalize
37:25
that and then like start weighing that for, for like thinking if you just get profiled
然后就开始权衡那个，为了，为了像是在想你是不是直接就被 profile 了
37:28
or something like that.
或者类似那样。
37:29
I'm curious, so Ben just described kind of, of course, there's this whole chain of
我很好奇，Ben 刚刚描述的，当然，这里有整整一条
37:36
things that influences what eventually shows up in the actual response, everything from
会影响最终实际 response 里出现什么的东西，从
37:42
the model weights to this query fan out to the agreement algorithm, the naming, all of
model weights 到 query fan out，再到 agreement algorithm、命名，所有
37:52
those things, co-occurrence.
这些东西，co-occurrence。
37:53
I'm wondering like over time, obviously this industry of AI visibility is evolving, right?
我在想，随着时间推移，显然这个 AI visibility 行业也在演进，对吧？
38:00
And so at a certain point, maybe it was like, oh, you need FAQs on your website.
所以到了某个阶段，可能就会变成，哦，你需要在网站上放 FAQs。
38:06
This is how like the easiest way to show up now.
这差不多就是现在最容易露脸的方式。
38:09
Now you're kind of thinking across all of these stages of the process.
现在你有点是在通盘考虑这个流程的所有阶段。
38:18
How do you, when you're engaging with new clients, different brands, like, what is,
那当你跟新客户、不同品牌打交道的时候，你会怎么，就是，有没有什么
38:26
are there any generic kind of takeaways around like the strategy of where you start and what
比较通用的要点，比如关于你从哪儿开始的策略，以及什么
38:33
is kind of near-term and long-term important for your brand so that you're hitting all
在近期和长期对你的品牌来说是重要的，这样你才能把所有
38:39
of these things, but also you're able to make progress quickly, any thoughts on that?
这些事情都做到，同时又能快速取得进展，对此你有什么想法吗？
38:45
Yeah, so I would just say three words, and then I'll dig into them.
对，那我就只说三个词，然后我再展开讲讲。
38:51
So I think it's sort of relevancy, consensus, consistency, right?
所以我觉得大概就是相关性、共识、一致性，对吧？
38:56
And all of this, you could also put into buckets of like tradition, like again, same three
而所有这些，你也可以归到几个传统意义上的类别里，就像，还是那三个
39:01
core jobs, I said at the beginning, so, and it really depends on what your current state
核心任务，我一开始就说过，所以，这真的取决于你现在的状态
39:05
is.
是什么。
39:06
Like, do you have a website today that's going to be super important and like all the
比如，你今天有没有一个网站，它会变得超级重要，而且所有那些
39:10
foundational stuff really matters?
基础性的东西都很关键？
39:13
Like, I used to run a paid ad agency, and I was like, man, why does some companies like
比如，我以前经营过一家付费广告代理公司，我当时就想，天哪，为什么有些公司
39:18
really succeed with ads and others don't?
就是能靠广告做得很成功，而有些就不行？
39:21
There's lots of factors that go into it, but one of the key aha moments I had is messaging,
这里面有很多因素，但我有过的关键顿悟时刻之一，就是 messaging，
39:27
is like, the companies that really succeed in marketing, they have, they have a really
就像，那些在营销上真正成功的公司，他们都有，他们都有一个非常
39:32
defined ICP, they really know what their product does, and they're able to map that to clear
明确的 ICP，他们很清楚自己的产品是做什么的，而且能把它对应到清晰的
39:38
the messaging, and they have really good differentiators, and I think that's really important
信息传达上，而且他们有非常好的差异化优势，我觉得这在
39:42
in AI search, is like, what are those terms you want to be associated with?
AI search 里特别重要，就是，你想和哪些词关联在一起？
39:48
Like, what do you actually do, and how do you translate that into a language that people
就像，你实际做的是什么，以及你怎么把它转化成人们
39:51
are going to be searching?
会去搜索的那种语言？
39:52
And I think that's how you create that relevant content, but you can't create good content
而且我觉得这就是你做出相关内容的方式，但如果你其实不知道
39:56
if you don't actually know who it's targeted to or what you're doing.
它是给谁看的，或者你自己在做什么，你就做不出好内容。
39:59
So I think that's like the relevancy piece.
所以我觉得那就是 relevancy 那一块。
40:01
I think that's how you ensure that during that query fan out, these are going to come
我觉得这就是你如何确保在 query fan out 的过程中，这些会
40:06
across your content.
遇到你的内容。
40:08
Now there's, we can go deeper on that, like I think you shouldn't just be posting blog
现在，我们可以再深入聊一下，比如我觉得你不应该只是发博客
40:12
content, you need a website, you know, your service pages, like, you know, all of that
内容，你需要一个网站，你知道，你的服务页面，就像，你知道，所有那些
40:16
good stuff.
好东西。
40:17
I think then consensus is, and Ben covered this in enough detail, is basically, my content
我觉得接下来的共识是，而且 Ben 已经讲得够细了，基本上就是，我的内容
40:24
is like me throwing my hat in the ring, like, hey, we have relevant stuff over here.
就像是我下场参与竞争，就像，嘿，我们这边有相关的东西。
40:30
Consensus are all the votes of confidence, like, why should we choose this company?
Consensus 全都是信任票，比如说，我们为什么要选这家公司？
40:34
Why should we, why should we trust them?
我们为什么要，为什么要相信他们？
40:36
Mark does always ask me, like, well, hey, where should we focus our efforts?
Mark 确实总是问我，比如说，哎，我们该把精力集中在哪儿？
40:39
And my answer is everywhere, really what we're talking about here is, it's a budget and
而我的答案是，哪儿都要。其实我们这里真正在谈的，是预算和
40:45
time constraint.
时间限制。
40:46
Right?
对吧？
40:47
Like if these, if budget and time weren't a consideration, then be everywhere.
就比如，如果这些，如果预算和时间都不是考虑因素，那就哪儿都做。
40:51
If they are a consideration, well, that's really going to depend on, like, industry.
如果它们是考虑因素，那好吧，这真的得看，比如说，行业。
40:55
We're quite bullish on places, like Reddit.
我们对Reddit这样的平台还挺看好的。
40:58
One of the advantages of Reddit is a lot of B2B companies don't really know how to do
Reddit的一个优势是很多B2B公司不太懂怎么做好。
41:03
it well.
所以正因为如此，你就能获得优势。
41:04
And so just, like, because of that, you can gain an edge.
我觉得还有就是做传统的digital PR，在社交渠道上建立存在感，这超级重要。
41:08
I think also doing just traditional digital PR, I think, building a presence on social channels
如果非要我说，拿枪指着我的头让我选，该聚焦在哪里？
41:15
is super important.
我会押注YouTube。
41:17
If I was to, you know, gun to my head, where should I focus?
41:21
I'd bet on YouTube.
41:22
It's actually a native search channel.
其实它本身就是一个 native 的 search channel。
41:25
It's owned by Google.
它是 Google 旗下的。
41:26
I think it's going to be around for a while.
我觉得它还会存在一阵子。
41:29
I don't think you should, I'm, I'm personally not that bullish on LinkedIn as a search channel,
我不觉得你应该，我，我个人对 LinkedIn 作为一个 search channel 没那么 bullish，
41:35
as a social channel.
作为一个 social channel。
41:36
Absolutely.
绝对。
41:37
We're on there.
我们也在上面。
41:38
Am I really believing it?
我真的相信吗？
41:39
But for search activities, I don't know, I think also building customer advocacy.
但就 search 活动来说，我不知道，我觉得还要建立 customer advocacy。
41:46
So we operate in the SaaS industry, that's going to be your G2s of the world, right?
我们是在 SaaS 行业里运营，那就会涉及到 G2 这类平台，对吧？
41:51
Getting people to say positive things about your brand on the internet is always going
让人们在互联网上对你的品牌说好话，永远都
41:55
to be a vote of confidence for consensus.
会是对共识的一种信任票。
41:59
And then consistency, this is kind of where it closes the loop.
然后是一致性，这差不多就是把整个闭环合上的地方。
42:02
What I kind of opened the thought with is you need to get really clear on what you want
我一开始引出这个想法的时候是说，你需要非常清楚地知道自己想
42:07
to be known as as an entity, as a company, right?
以什么被大家知道——作为一个 entity，作为一家公司，对吧？
42:10
So if you go to our website, you're going to see, we use similar words across the place
所以如果你去我们的网站，你会看到，我们在各个地方用的词都很相似。
42:15
because I want those to be embedded in these models and the world.
因为我想让这些东西嵌入到这些 models 和这个世界里。
42:21
And so whether it's your activities on Reddit, your posting videos on YouTube, your posting
所以不管是你在 Reddit 上的活动、你在 YouTube 上发视频，还是你发在
42:26
content on LinkedIn, even like your company LinkedIn profile, you know, content on your website
LinkedIn 上的内容，甚至像你公司的 LinkedIn 主页，你知道，还有你网站上的内容
42:32
and on your blog, always presenting yourselves with a consistent message, right?
以及你博客上的内容，始终用一致的信息来呈现自己，对吧？
42:37
I think this is like, and so like one of the key contrasts here is if we look at like
我觉得这就像是，所以这里的一个关键对比就是，如果我们看像
42:44
link building in SEO, the big check in the box was, hey, we got a link, we've got to do
SEO 里的 link building，那个大大的打勾项就是：嘿，我们拿到了一个 link，我们得做
42:49
follow link.
follow link。
42:50
You know, we're going to get some link Jews back to our website in AI search.
你知道，在 AI search 里，我们会让一些 link Jews 回到我们的网站。
42:54
I actually think the blurbs surrounding, even if that link didn't exist, the blurbs
其实我觉得，围绕你的那些 blurbs，就算那个链接不存在，围绕你这个实体的 blurbs 也重要得多，因为就像，作为一家公司，你到底是谁？
42:59
surrounding you as an entity is far more important because like who are you as a company?
我们希望那个东西非常 dense，对吧？
43:03
We want that to be really dense, right?
就像我希望，如果某个 LN 挑中了这段话，再把它放回到一个回答里，我们是不是被定位得很有竞争力？
43:06
Like I want if an LN picked this paragraph and put it back inside to an answer,
就像我们是从那个角度在想的，所以这就像是一种一致性。
43:11
are we being positioned competitively?
现在我们可以进入策略和所有这些玩意儿，但它真的取决于你当前的状态，因为有些人我们可以聊 Reddit 和所有这些策略，
43:13
Like we're thinking about it from that perspective, so that's like consistency.
43:16
Now we can get into like the tactics and all that stuff, but it really depends on
43:19
like your current state because some people we can talk about Reddit and all these tactics,
43:25
but some people might not even have clear messaging, right? They might just have a website
但有些人可能连清晰的信息都没有，对吧？他们可能就只有一个网站
43:31
with a few paragraphs of text on it, right? And so then your priority is going to be slightly
上面放着几段文字，对吧？所以这时候你的优先级就会稍微
43:35
different, so it does depend on a way you're at.
不同，所以这确实取决于你处在什么阶段。
43:37
I'm curious with kind of that, with the focus on Reddit and the potential there with,
我很好奇，结合这一点，在聚焦 Reddit 以及它那里的潜力时，
43:44
what tends to make one Reddit thread more likely to surface than another are some of the
是什么往往会让一个 Reddit 讨论串比另一个更可能浮现出来，是不是一些
43:52
signals that you would have that are like upvotes and comment cowl and user reputation.
你会有的信号，比如 upvotes、comment cowl 和用户声誉。
43:59
Do those correlate with retrieval? Are those, are those, and so when you're looking across
这些会和 retrieval 相关吗？那些，那些，所以当你在浏览
44:06
different threads on Reddit, does that make a substantial difference?
Reddit 上不同的讨论串时，这会有很大的差别吗？
44:11
How do you think about that?
你怎么看这件事？
44:13
Yeah, so I'm not going to speak in absolute, because I think these things are always changing.
嗯，所以我不会说得太绝对，因为我觉得这些东西一直在变。
44:16
What we have observed is engagement in terms of upvotes. We haven't seen that correlate
我们观察到的是，从 upvotes 来看的 engagement。我们没看到它跟
44:26
with increased citations. What we have seen highest correlation with is the actual content itself,
citations 增加相关。我们看到相关性最高的是实际内容本身，
44:33
right? So I always like to break things into like easy ways to understand, because that's how
对吧？所以我总是喜欢把事情拆成容易理解的方式，因为这就是
44:40
my brain works. But if we just look at this from the principle of user prompts LLM, LLM does
我大脑的运作方式。但如果我们只从 user prompts 给 LLM 的原理来看，LLM 会
44:45
query fan out and it's looking for information. Contained with those queries, there's lots of
query fan out，然后它在找信息。那些 queries 里包含了很多
44:51
things and entities, right? So if I go back to that prompt example, SaaS founder, like doing this
东西和 entities，对吧？所以如果我回到那个 prompt 例子，SaaS 创始人，比如在做这个
44:55
in revenue, in those query fan outs, it's containing all these entities, right? And so I want those
在收入方面，在那些 query fan outs 里，它包含了所有这些 entities，对吧？所以我想要那些
45:00
to be contained within my content on Reddit. And so we found very often that the passage
被包含在我 Reddit 上的内容里。然后我们发现，很多时候那个 passage
45:06
being extracted from Reddit is like 50% down the page, and the comment has like one upvote or
从 Reddit 里被提取出来的时候，大概在页面往下 50% 的位置，而且那条评论只有大概一个 upvote 或
45:12
two upvotes. And it's because the passage was relevant. Now, I think that might change over time,
两个 upvote。这是因为那个 passage 是 relevant 的。现在，我觉得这可能会随着时间改变，
45:19
right? Because similar to how relevancy is super important, but authority is becoming even more
对吧？因为就像 relevancy 超级重要，但 authority 正变得越来越
45:24
important. I think these models are going to keep similar like, you know, how Google always releases
重要。我觉得这些模型会继续类似，就像，你知道，Google 总是发布
45:30
a core update is a catamount game between these providers and marketers constantly trying to catch
一个 core update，就是这些服务商和营销人员之间不断试图追赶的猫鼠游戏
45:37
up. And I know there are commercial agreements between OpenAI, Reddit, Reddit, Google. And so maybe
而且我知道 OpenAI、Reddit、Reddit、Google 之间有商业协议。所以也许
45:44
things like upvotes, maybe even the history of those accounts, things like that might factor into
比如 upvotes，甚至那些账号的历史，诸如此类的东西可能会影响
45:51
it. I don't know the complexities of that might change over time. But yeah, just what we've observed
这件事。我不知道这里面的复杂之处会不会随时间变化。但没错，就我们目前观察到的
45:58
so far is the overlap between what the user is searching and the content on Reddit. Liam,
到目前为止，就是用户搜索的内容和 Reddit 上的内容之间的重叠。Liam，
46:05
you started getting us towards kind of like how things may or may not change how they're evolving
你开始把我们带向，像是这些东西可能会或可能不会改变它们演变的方式
46:11
over time as we get kind of close to an end here for this conversation. I'd love to close out by
随着时间推移，在我们这场对话差不多要结束的时候。我想收尾的方式是
46:19
just asking each of you, maybe circling back to Ben to start each of you, like as you look forward,
就是问问你们每个人，也许从 Ben 开始，让你们每个人来说，像是你们展望未来时，
46:26
what is kind of top of your mind, either in terms of like, oh, there's like a huge opportunity here
你们最先想到的是什么，要么是像，哦，这里有个巨大的机会
46:33
for people that are willing to jump into it, or something that's like, oh, there's a really open
给那些愿意投身其中的人，或者类似，哦，这里有一个非常开放的
46:40
challenge here. We haven't figured it out yet. There's more work to be done in this area in either
46:45
of those areas. Like what are you thinking about as you end your days or as you start out in the
46:51
morning that's kind of top of your mind going into this next phase of work?
46:56
Let Liam go. He's ready to go.
47:00
I'd probably say two things. So offsite, like how are those third party
47:12
earned sources going to influence how I am or clients are perceived inside LLMs?
47:21
I think there's just so much change happening there. And ultimately, brand is the final
47:26
mode, right? You see this plastered everywhere. And so I really think that's where things are
47:34
going to change a lot. And so I'm always thinking about that. I'm always thinking about our clients
会改变很多。所以我一直在想这件事。我一直在想我们的客户
47:38
only have certain budget and time they can allocate and they're on strict timelines.
只有一定的预算和能分配的时间，而且他们的时间线很紧。
47:45
And so how do we increase the probability that the bet they make is going to have the highest
所以我们怎么提高概率，让他们下的注能有最高的
47:51
expected value. So I'm always thinking about that. The second area is agent accessibility.
expected value。所以我一直在想这件事。第二个领域是 agent accessibility。
48:02
So I think everyone at the moment, like everyone, I'm speaking of such
所以我觉得现在每个人，就像每个人，我这么说很
48:08
generalities, but I think everyone's been focusing on discoverability, right? Like how do we
笼统，但我觉得每个人一直关注的都是 discoverability，对吧？就像我们怎么
48:14
get my content discovered? And I think there's so many edges to gain there. This is going back to
让我的内容被发现？我觉得那里有太多优势可以争取。这又回到
48:18
my original point. It's like walk before you can run. But we're quickly moving to a place where
我最初的观点。就像先学会走再学会跑。但我们正在快速走向一个
48:26
it's not just going to be agents crawling your website and retrieving information.
这不会只是 agents 爬取你的网站并检索信息。
48:32
They're actually going to be taking actions on your website. And we've already seen evidence
它们实际上会在你的网站上采取行动。而且我们已经看到了证据
48:36
of this. We have clients report into us that, hey, an agent, an AI agent booked a demo last
这一点。我们有客户向我们反馈说，嘿，一个 agent，一个 AI agent 上周预约了一个 demo，
48:41
week, we're seeing things like that happen in. And so then that just, again, websites have
我们正看到这类事情发生。然后这就，再一次，网站一直以来
48:46
historically been optimized for humans. Humans, if your website takes a little bit to load or
都是为人类优化的。人类呢，如果你的网站加载要花点时间，或者
48:53
your form, your demo form is confusing, it's okay, we can figure it out. We can use our intelligence,
你的表单，你的 demo 表单让人困惑，没关系，我们能搞明白。我们可以用我们的智慧，
48:58
we can figure it out. It's not fun and it's going to impact conversions, but we'll figure it out.
我们能搞明白。这不好玩，而且会影响 conversions，但我们会搞明白的。
49:02
Agents, are they going to wait around to figure it out? I think there's actually some standards
Agents，它们会干等着去搞明白吗？我觉得其实已经有一些标准
49:07
that I'll be seeing some movement here from Google. And so I think that's going to really impact
我觉得接下来会看到 Google 这边有一些动作。所以我觉得这真的会影响到
49:14
how websites look and behave. And I think that's really exciting because I don't think anyone's
网站的外观和行为方式。而且我觉得这真的很令人兴奋，因为我不认为有人
49:19
looking there yet. I know Ben wants to research more in this area. I'm kind of holding him back
已经在关注那里了。我知道 Ben 想在这个领域做更多研究。我有点在拖他后腿
49:24
a bit because I don't think it's sexy enough just yet. But I see that definitely as the next frontier.
因为我觉得它现在还不够性感。但我确实认为那绝对是下一个前沿。
49:32
And we have clients that are generating a lot of conversions, tens of thousands of conversions per
而且我们有客户正在产生大量 conversions，每周数万次 conversions。
49:38
week. And a lot of their users are coming via command line interface. So they're saying, hey,
而且他们很多用户都是通过 command line interface 来的。所以他们会说，嘿，
49:43
I want to build X. And then the agent is the one going out there, coming back like a private
我想构建 X。然后 agent 就是那个出去、回来像一支私人
49:49
procurement team. And like not just saying, hey, I recommend these tools. They're saying, hey,
采购团队一样的东西。而且不只是说，嘿，我推荐这些工具。他们在说，嘿，
49:53
I installed these tools for you. And they're good to go. I think that expands the surface area
49:59
that we need to look at. So yeah, I'd say those two is what I'm thinking about. Awesome. Yeah, Ben,
50:04
anything to add there? Yeah, definitely with the agents. I just think it's
50:09
moving from like a, I always described as like a read only environment to a read right environment
50:15
is where we're headed. And his dreams had the very start of this conversation. It's just a better
50:20
deal. He gets a better deal to get all the, all the context in one place, just talk to an agent.
50:25
It will be even better deal if you don't have to fill out the form. It will be even better deal if
50:29
you know the pick a slot of the calendar. Yeah, yeah, yeah, get the idea. And so that's for sure where
50:34
we're heading. I think, you know, you've seen evidence of this. And if you look at the Google
我们正朝这个方向走。我觉得，你知道，你已经看到这方面的证据了。而且如果你看 Google
50:39
web or cp program, this is going to probably, it'll probably, it'll be very, very slow and overnight
web 或 cp program，这可能会，它可能会，它会非常非常慢，然后一夜之间
50:46
to go boom. And that's that's what I expect it out. Just because it's human, it's just human
突然爆发。这就是，这就是我预期会发生的。就因为这是人性，这就是人类
50:50
behavior. It's just easier. It's just easier like low restriction path. That's where we tend to.
行为。就是更容易。就是更容易，像限制更少的路径。我们往往就会往那儿走。
50:55
And then I would say, I do think like to date, we've been very focused around basically like,
然后我会说，我确实觉得，到目前为止，我们基本上一直非常专注于，像是，
51:06
like, here's some creative fan ads and here's the citations. How can I map them off? And we're kind
比如，这里有一些 creative fan ads，这里是 citations。我怎么把它们对应起来？而我们有点
51:11
of like playing this, this, a little bit of a, most marketing teams are playing this game. I'm like,
像是在玩这个，这个，有点像，大多数营销团队都在玩这个游戏。我就会想，
51:16
here's my comp, and I now rank four kind of thing. But I think people have started focusing more on
这是我的 comp，然后我现在排第四之类的。但我觉得人们已经开始更多关注
51:23
like the weights and what's driving the weights longer term. Because the two reasons that the
51:29
marketing general is like sophisticated people are probably more aware of how these systems work.
51:33
The second thing is that the training cycles are actually coming down. So when we first started,
51:37
it was a cool way of like a new version of chat to be key once every like nine, 12 months, right.
51:42
And now obviously these engines are getting trained like on a weekly and bi-weekly basis at the top
51:48
layers. So even like the top layers are being retrained and being tuned. And then the backbones
51:54
are being retrained every, let's say three months or so now. And so they are more influenceable.
51:59
And if you can get influence or imprinting those tokens, it's worth like borders of magnitude
52:06
more than being in a single context window. I mean, you said that you ultimately won't pose and
52:12
it's not easy to get into that training data. But that's where I see things having in terms of
52:17
otherwise outside of that. Yeah. Also, I was going to be a big thing. I think being able to
52:24
because essentially what do they kind of overall trend is that Google is having to,
52:29
where all these all these ages are having to avoid basically like dead internet theory, right.
52:35
They're trying to avoid training on like basically model clouds, training on their own systems.
52:40
And so they're looking at all this content out there. That's part of what the finger printing
52:43
releases. It's part regulatory. It's part. It's funny. A part by T play. It's part like multiple
52:48
plays they have out there. But it's ultimately it's one of the reasons I also think they have is
他们能拿出来的那些玩法。但说到底，这也是我觉得他们有的一个原因，就是
52:52
they don't want to train on their own. So if you do post a Reddit, I get a generator told
他们不想自己训练。所以如果你真的在 Reddit 上发帖，我就能让一个 generator 去告诉
52:56
Colin, they will know that they look up their hash function very simple. Boom, the probability of
Colin，他们会知道，他们查一下他们的 hash function，非常简单。Boom，这个是我们 model 的 generator 的概率
53:00
this being a generator from our model is very high. And they just want they essentially that
非常高。而他们只是想要，他们基本上就是
53:08
I think that's going to ratchet up. And I think there's going to be a lot of cleaning and basically
我觉得这会愈演愈烈。而且我觉得会有很多清理，基本上
53:11
all these like the basically the protection of human level context. And actually when
所有这些，基本上就是在保护 human level context。而其实当
53:16
open AI started out, they have this manifesto online to they wanted to all flow the verification
OpenAI 刚起步的时候，他们在网上有一份宣言，他们想要允许文本 verification 的
53:22
of text that third parties like trust pilots, a good example in partnership with them.
这种 verification 由像 Trustpilot 这样的第三方来做，就是一个跟他们合作的很好的例子。
53:28
They'll keep trying to do that. And I think they'll keep trying to up the anti here. So get
他们会继续试着这么做。而且我觉得他们会继续在这方面加码。所以，会变得
53:32
harder and harder to basically evade them. So you need to keep climbing the edges around that.
越来越难，基本上很难避开它们。所以你得不断在那个边界周围往上爬。
53:37
So that's where one other kind of like let's say AI content, bot content, I guess, or bot
所以这就是另一种，比如说，AI content、bot content，我猜，或者 bot
53:43
behavior in general is going to become the human signal is going to become very valuable there.
行为，总的来说，会变成——human signal 在那里会变得非常有价值。
53:48
But also emulating the world too. Makes sense. Yeah. Well, I was writing down furiously in the
但也要模拟这个世界。有道理。是啊。嗯，我刚才一直在
53:55
background of a few things that you all mentioned kind of throughout. I need to level up on my
背景里飞快记下你们大家大概从头到尾提到的几件事。我得提升我的
54:01
understanding things are just moving so fast. I really appreciate both of you joining us to
理解，事情实在发展得太快了。我真的很感谢你们两位加入我们，
54:05
like with your expertise really helping us and our listeners understand this topic. I would
用你们的专业真正帮我们和我们的听众理解这个话题。我会
54:10
encourage everyone listening to go check out discovered labs in our show notes. We'll link some
54:15
of the research that we talked about here. We'll link their research page, which is which is
54:20
really great. Thank you so much Liam and Ben for joining us looking forward to having you
54:25
back on the show sometime when everything is different. Thank you so much. Thanks for having us.
54:30
Cheers. All right, that's our show for this week. If you haven't checked out our website,
54:42
head to practical AI.fm and be sure to connect with us on LinkedIn, X, or Blue Sky.
54:48
You'll see us posting insights related to the latest AI developments and we would love
54:52
for you to join the conversation. Thanks to our partner prediction guard for providing
54:56
operational support for the show. Check them out at predictionguard.com. Also, thanks to
为这档节目提供运营支持。
55:02
break master cylinder for the beats and to you for listening. That's all for now.
去 predictionguard.com 看看吧。
55:06
But you'll hear from us again next week.
另外，感谢 break master cylinder 制作的 beats，也感谢你的收听。