本集简介
双语字幕
仅展示文本字幕，不包含中文音频；想边听边看，请使用 Bayt 播客 App。
大家好，欢迎收听MINDSCAPE播客。
Hello, everyone, and welcome to the MINDSCAPE podcast.
我是主持人肖恩·卡罗尔。
I'm your host, Sean Carroll.
我想大家都听说过图灵测试，或者艾伦·图灵自己称之为的模仿游戏。
I presume that everyone has heard of the Turing test or as Alan Turing himself called it, the imitation game.
这应该是用来判断计算机能否思考的一种方法。
Supposed to be a way to figure out whether computers can think.
当然，难点不仅在于计算机能做什么，还在于你说的“思考”是什么意思。
And, of course, the difficulty in that is not only what computers can do, but what do you mean by thinking?
所以图灵采用了非常科学、物理和数学的方法来解决这个问题。
So Turing took a very science y physical science y, mathematically, approach to the problem.
他说，我不知道思考是什么意思，但我们可以看看事物做了什么。
He says, I don't know what it means to think, but we can look at what things do.
如果一台计算机做的事情和思考无法区分，也就是说，计算机的输入输出反应和人的无法区分，那么我们就应该称之为思考。
And if you have a computer that does things that are impossible to tell the difference between that and thinking, that is to say the input output responses of the computer are indistinguishable from those of a person, we should call that thinking.
现在呢，我们有了这些大语言模型，也就是LLM，作为AI的一种实现方式。基本上，很明显它们已经能通过图灵测试了。
Now these days, we have these LLMs, these large language models as an approach to AI, and more or less, it's clear that they do pass the Turing test.
我知道有人会对此有争议，但我觉得这有点吹毛求疵了。
I know there's some people who argue about that, but I think it's kind of nitpicky myself.
在我看来，按照图灵本人想象的那种方式来看，它们确实已经通过测试了，这点我毫不怀疑。
I think that there's no question in my mind that they're passing the test as Turing himself would have imagined it.
那么，这是不是意味着大语言模型真的能像人类一样思考呢？
So does that mean that the LLMs are really thinking in the same way that human beings are thinking?
而且我觉得，你知道，这个问题一直来回争论。
And I think, you know, there's been an argument back and forth.
有些人说，是的。
Some people say, yes.
就是这么回事。
That is what it means.
也有人说，不对。
Others say, well, no.
其实，我们得更深入地思考一下思考意味着什么。
Actually, turns out we have to think harder about what it means to be thinking.
然后另一方会说，哦，不。
And then the other side says, oh, no.
现在你是在偷换概念。
Now you're moving the goalpost.
我以为我们同意图灵测试了。
I thought we agreed on the Turing test.
我实际上是站在偷换概念这一边的。
I'm actually on the side of the goalpost movers.
我认为完全可以这样说：那并不是对思考的足够严谨的定义，至少不是人类思考的那种方式。
I think it's perfectly okay to say, well, that wasn't a careful enough definition of what it means to think, at least not in the same way as human beings do.
即使承认，在给定一组输入的情况下，LLM 产生的输出与某种人类输出几乎无法区分。
And even granting that given a certain set of inputs, the LLMs will produce outputs that are more or less indistinguishable from some kind of human output.
但问题依然存在。
There's still a question.
它们这样做是因为，LLM架构在建立、训练、微调等过程中，本质上重新发现了人类思考的机制，还是因为一个完全合理的情景，它们发现了另一种方式，来实现与人类相同的输入输出机制，就像外星智能一样。
Are they doing it because, the LLM architecture in the process of being set up and then trained and and fine tuned and so forth has essentially rediscovered the mechanisms by which human beings think, or is it because it is a absolutely plausible scenario, they've discovered a different way to have the same input output mechanisms as human beings, the kind of alien intelligence.
而且有一些证据，我一直被这些证据所打动，这些证据表明，你可以问LLM一些看起来不像人类答案的问题。
And there is some evidence, and I have absolutely been on the side of being impressed by the evidence that says, look, there's questions you can ask in LLM that don't look like human answers.
你知道，最著名的例子就是问它草莓这个词里有几个r，或者类似的问题。
You know, the famous ones are how many r's in the word strawberry or something like that.
对我来说，这是很好的证据，表明它们不像人类那样思考。
And that to me was very good evidence that they're not thinking in the same way that human beings are.
很多人反对我的观点，他们说，你知道，所有伟大的计算机科学家和AI行业的领导者都持相反意见。
And a lot of people push back against my view on that saying, well, you know, all the great computer scientists and leaders of the AI industry are are saying otherwise.
这对我来说从来都不是特别有说服力，原因很简单，那些人是计算机编程、计算机科学和AI的专家，但不是智力和认知科学的专家。
And that was never especially convincing to me for the simple reason that those people are experts in computer programming and computer science and AI, but not experts in intelligence and cognitive science.
所以最近，约翰霍普金斯大学主办了哲学与心理学学会的会议，我们有些人想到，在会议期间做一个现场播客采访会很有趣。
So recently, Johns Hopkins hosted a meeting of the Society for Philosophy and Psychology, And some of us had the idea it'd be fun to do a live podcast interview as part of that meeting.
但结果并没有那样实现。
It didn't pan out that way.
我当时正好在旅行，和会议时间冲突了。
I was traveling at the same time the meeting was happening, etcetera.
但我们浏览了来参会的人，寻找合适的播客嘉宾，然后找到了今天的嘉宾，钱德拉·斯里帕达。
But we looked through the people who were visiting, for prospective good podcast guests, and we found today's guest, Chandra Sripada.
他是密歇根大学精神病学系的哲学家和认知科学家，同时也是大语言模型专家。
He is a philosopher and a cognitive scientist in the, psychiatry department at University of Michigan and also an expert on LLMs.
对吧？
Okay?
所以他是思考和大语言模型方面的专家，非常适合提出并回答这个问题：大语言模型的思考方式与人类相似吗？
So he's an expert on thinking and on LLMs and is very well positioned to ask and answer the question, do LLMs think in similar ways to humans do?
他有力地论证了，它们确实经常在很多方面使用与人类相同的思考方法。
And he makes a strong case that they do often, in many ways, think using the same kind of thinking methodologies that human beings do.
我觉得他很有说服力。
And so I find him very persuasive.
我认为他很好地证明了，至少在许多重要方面，大语言模型重新发现了，或者说被引导重新发现了人类的思考方式。
I think that he's made a really good case that in at least in an important set of ways, LLMs have rediscovered or, you know, been coaxed into re finding out the ways that human beings think.
利用认知科学的理念，你知道，不同的认知过程测试在LLM和人类之间是如何匹配的。
Using ideas from cognitive science, you know, how do different, tests of what happens during the cognitive process match up between LLMs and human beings.
这样你就可以自己判断这是否让你信服。
So you can tell for yourself whether or not it's convincing to you.
这并不是说LLM有意识或者是负责任的道德主体之类的，但这是我们应该朝着那个方向确立的东西。
It's not the same as saying that LLMs are conscious or responsible moral agents or anything like that, but they are but this is a something we should establish in that direction.
你知道，认知比意识更容易理解。
You know, cognition is easier to understand than consciousness.
所以我觉得这是那种以重要方式改变了我信念的播客之一，而且，你知道，人总该是个好的贝叶斯主义者。
So I think this is one of those podcasts that has shifted my credences, in important ways, and, you know, one should always be a good Bayesian.
如果有更多相反的证据出现，信念就会回调；如果证据继续朝这个方向推进，信念就会继续朝那个方向移动。
If more evidence comes in the other way, then they'll shift back, or if the evidence keeps pushing in this direction, they'll keep moving in that direction.
但我觉得很吸引人的是，那个一直摆在桌面上的选项——我一直承认LLM以某种方式重新发现了人类的思维模式——结果竟然以有趣的方式得到了证据支持。
But I think it's fascinating that, you know, the option, which was always on the table, and I always admitted certainly, that LLMs sort of have for by some way or another, rediscovered human modes of thinking, has turned out to be something that has evidence for it in an interesting way.
所以我觉得这能构成一场很棒的对话。
So I think that makes for a great conversation.
我们开始吧。
Let's go.
Chandra Sripada，欢迎来到MINDSCAPE播客。
Chandra Sripada, welcome to the MINDSCAPE podcast.
哦，谢谢，Sean。
Oh, thanks, Sean.
我很高兴来到这里。
I'm glad to be here.
现在我们要比较我最喜欢的两个认知系统：人脑和LLM。
Now we're gonna be comparing two of my favorite cognitive systems, the human brain and LLMs.
所以，我们大多数人对人脑都有点了解。
So, most of us are a little bit familiar with the human brain.
当然，我们现在都听说过LLM。
Of course, we've all heard of LLMs by now.
但你能给我们简单介绍一下LLM是如何工作的吗？
But could you give us a quick overview of how LLMs work?
你知道，我们虽然在使用它们，但可能有点忘了它们背后是怎么运作的。
You know, we we use them, but we maybe we've forgotten what is going on underneath the hood a little bit.
对。
Yeah.
当然。
Of course.
对。
Yeah.
让我来简单说一下。
Let me let me say a bit about that.
而且，我意识到我之前没来得及说这个。
And and, I realized that I I didn't get to say this already.
我是你的长期粉丝了。
I'm a long time a long time fan.
哦，谢谢。
Oh, thank you.
能来到这里真是太棒了。
It's great to be here.
而且，我好像从来没听过你语速低于1.5倍的。
And, I don't think I've ever heard you at anything less than 1.5 speed.
所以能听到你现场说话真好。
So it's good to hear you live.
按这个语速听下来，你显得更口齿清晰、表达流畅了。
You're even more eloquent at this rate.
那大语言模型是怎么运作的呢？
So what LLMs, how do they work?
首先，大语言模型本质上是神经网络。
For starters, LLMs are neural networks.
所以里面会有一组节点和连接它们的权重。
So there's going to be a set of nodes and weights that connect them.
基本的处理方式是：当输入进来时，激活值会与权重相乘，然后在网络中传播。
And the basic processing strategy is when an input comes in, the activations will be multiplied by the weights and get propagated through the network.
如果这些权重调整得当，输出的答案就会是输入内容的合理延续。
And if those weights have been tuned correctly, the answer will be a sensible continuation of the input that came in.
所以大语言模型在几个方面上很有特点。
So LLMs are distinctive in that, in a few ways.
其中一个特点就是它们采用的方法是自回归的。
And one of them is the approach that they're going to take is autoregressive.
打个比方，我喜欢喝咖啡加糖；而大语言模型经过训练，拥有海量参数——前沿模型的参数达到几千亿甚至万亿级别——这些参数被调整到最佳状态，使其在处理互联网规模的数据时，能够预测下一个词是什么。
So I like to have my coffee with sugar and and they have been trained so that massive set of parameters, hundreds of billions, trillions in frontier models, are tuned up in such a way that over the course of Internet scale data, they are making predictions of the next word.
而且每一次，这些参数都会微调一点点，让预测结果越来越好。
And every single time, those parameters have been changed just a bit so their predictions get better and better.
打个比方，我喜欢咖啡加糖，一开始可能是加点糖浆或者黄油，最后慢慢就变成加奶油了。
And so initially, I like my coffee with sugar and, you know, it might be dorts or butter, and eventually you're going to get to cream.
对。
Yeah.
所以自回归生成是一个词一个词来的，每生成一个新词，就会把它加回提示里，作为上下文的一部分。
And so the the autoregressive generation, one word at a time, after that one word is generated, it's added back into the prompt as part of the context.
这是它们的一个独特之处。
That's one thing that's distinctive about them.
在海量互联网数据上进行训练，这也是它们的一大特点。
The training on Internet scale data, that's another thing that's distinctive about them.
也许我们可以先打个伏笔，稍后再详细说，但它们并不是那种毫无差别的普通神经网络。
And maybe it's best to just foreshadow this and say this a bit later, but it's not a big undifferentiated neural net.
回想一下PDP（并行分布式处理）的黄金时代，那时候我还年轻些，那些网络的结构是：有一组输入节点，一组权重，最后输出结果。
In the old heyday of PDP, you know, when I was a bit younger, those nets, there is a set of input nodes and a set of weights and eventually you get to the output.
这些就是 Transformer 模型。
Well these are transformers.
它们的结构要复杂得多。
And they have a lot more structure.
在 Transformer 中，每个词或 token（我们不用太深入这个细节）都是作为一列输入进来的。
And so with the transformer, every word slash token, we don't have to get too much into that, enters as part of a column.
处理过程是按列进行的，但关键在于那些注意力机制头，它们能把信息从左边的位置传递到右边的 token 位置上。
And the processing happens in a column specific way, but for the presence of these attention heads that move information from left over to token positions on the right.
所以这是另一个特点，虽然不是变压器的唯一特点，但它们是一种高度结构化的神经网络。
And so that's another, it's not the only feature of transformers, but they're a distinctive highly structured kind of neural net.
数十亿的参数，通过预测下一个词来训练，自回归生成，这就是变压器的结构。
And so billions of parameters trained on next word prediction, prediction, autoregressive generation, the structure of a transformer.
好的。
Okay.
这就是大语言模型的主要运作方式。
That's a lot of what's going on in the Large Language Model.
你知道，如果我要写一本书，我不会从第一句话开始一直打下去。
You know, if I am going to write a book, I do not start at the first sentence and just keep typing.
对吧？
Right?
我会先构思一个提纲，然后逐章展开。
Like, I imagine an outline, and then I develop it chapter by chapter.
我可以让大语言模型来写一本书。
I could ask an LLM to write a book.
它会那样做吗？
Would it do that?
它会先给出一个大概的概述，然后再填充细节，还是直接从第一句话开始？
Would it sort of give a break overview and then fill in, or would it just start at the first sentence?
对。
Right.
是的，我会尝试整理一些论据，说明这些大语言模型的处理方式与人类心智和大脑的核心处理原则有相似之处或类比关系。
Already and I'm going to try to, you know, assemble some arguments that these LLMs process in ways that are reminiscent of or analogous to core processing principles of the human mind brain.
但你已经抓住我的把柄了。
But you caught me already.
看起来，自回归的下一词预测训练和生成过程确实有些非常不同的地方，在生成过程中，一次生成一个词，并添加到上下文中。
There there seems to be something very different about the autoregressive next word prediction training, and then the generation process, where one word is generated at a time and added to the context.
对此该怎么说呢？
What to say about that?
我的意思是，有一点已经可以说了，生成下一个词并不妨碍模型在前向传播过程中拥有大量的记忆资源和处理能力来提前预测后续会发生什么。
I mean, one thing to say already is next, generating the next word is not incompatible with the model itself during the forward password has a substantial amount of mnemonic resources and processing capacity to start anticipating what happens later on.
所以 Anthropic 有个例子，你给它一个押韵的开头，大概意思是‘我看见一根胡萝卜，我得把它抓住’。
So Anthropic has this example where you give it a stem of a rhyme, something to the effect of, I saw a carrot, I had to grab it.
好的。
Okay.
然后模型其实会先吐出下一句的最后一个词，也就是‘兔子’。
And the model actually generates the last word of the next sentence, rabbit, first.
我们通过机制可解释性知道了这一点，也就是我们深入模型内部，观察它在生成‘兔子’之前的各个处理阶段。
And we know this via mechanistic interpretability where we're peering into the model and looking at its processing stages before it generates the prior words that will culminate in Rabbit.
这说明，以生成下一个词作为训练目标和方法，其实跟模型在早期就能预测后面发生的事情并不冲突。
And that's highlighting that next word generation as a generation method as well as closely aligned training objective is quite compatible with the model generating things that come much later, early on.
所以这种规划能力还是可以存在的。
And so that kind of planning can still happen.
我再补充一个维度。
I'll add another dimension to it as well.
我们待会儿再详细聊这个。
We'll come to this a bit later, I think.
但这些模型可以把上下文窗口当作一种草稿纸，生成内部标记，作为它们所谓的“思考”的一部分。
But these models can use the context window as a kind of scratch pad and generate internal tokens as part of what they call thinking.
所以如果你给它一个复杂的任务，模型现在会先把它分解成子任务，作为它最初要做的事情的一部分。
So if you give it a complex kind of task, the models now will break it into subtasks as part of the initial thing that it'll do.
子任务还可以进一步分解成更小的子任务，并委托给一系列子代理等等。
And subtasks can be further broken down into subtasks and delegated to a series of sub agents and things like that.
所以，令人惊讶的是，这与下一个词生成和下一个词预测作为训练目标非常兼容，可以分阶段地攻击问题，将其分解为子组件，在下一个词生成开始之前就预见到子组件的最终部分。
So it is compatible remarkably enough with next word generation and next word prediction as the training objective for substantial staged attacking of a problem, breaking it down into subcomponents, looking ahead at terminal parts of the subcomponents before next word generation begins.
所以，下一个词预测和自回归的下一个词生成乍一看似乎违反直觉，但实际上可以涵盖人们使用的许多普通策略。
So next word prediction is and and the training objective, next word prediction and autoregressive next word generation seems counterintuitive at first, but can actually encapsulate a lot of the ordinary strategies that people use.
那么，这种草稿纸和多次尝试答案的想法，
And is this idea of, like, a scratch pad and, multiple sort of swipes at the answer.
是人类引导LLM这样做的，还是LLM在某种程度上自己摸索出来的？
Is that something that human beings have coaxed the LLM into doing, or is that something that the LLMs have sort of figured out for themselves in some sense?
对。
Right.
答案主要是大语言模型自己摸索出来的，当然也有一些引导的成分。
And the answer is mostly the LLMs have figured out for themselves, and there is some coaxing involved as well.
那我们为什么会有这样的看法呢？
And so why why do we why do we believe something like that?
你刚才提到了使用草稿纸的具体做法。
The use of you talked about the use of the scratch pad specifically.
不过，现在适合深入讨论双过程理论的区别吗？
But, is this a good time to go into the dual process distinction?
不。
No.
我们留到后面再说。
We're going to keep that later.
我们留到后面再讲。
We're going to keep that for later.
那我就先笼统地说一下。
Then I'll I'll say this a bit of in a very general way.
在互联网规模的数据上进行下一个词预测，能让模型构建出惊人的表征景观和潜在策略，来处理复杂问题，比如你说你想去塔斯马尼亚。
Next word prediction on Internet scale data gets the model to assemble an astonishing representational landscape and latent strategies for, you know, how complex problems like, let's say you tell me that you wanna go to Tasmania.
模型内部已经潜藏着完成这件事所需的各类子任务。
The model has latent within it the kind of subtasks that would be needed in order to do that.
所以这一切似乎都是下一个词预测所产生的表征能力的一部分，尤其是在大规模训练时。
So all of this just seems to be something as part of the representational repertoire that next word prediction generates, especially when it's done at scale.
但即使你有了这些乐高积木，也不意味着你已经有了那个漂亮的游乐园，你知道，你可以用乐高搭出来的那个。
Now when you've got these LEGO pieces, it doesn't mean that you've got that pretty amusement park already that, you know, you can build with the LEGOs.
这需要一些引导、一些监督训练、一些指令微调，尤其是强化学习。
There is some coaxing, some supervised training, some instruction tuning, and especially reinforcement learning.
对。
Right.
你用来自各种来源的评估反馈来训练模型。
Where you train the model with evaluative feedback that can emerge from various sources.
这往往会将那种惊人的、潜在的表征能力，整合成有用的部分，并组织成有目标导向的控制。
That tends to take this astonishing representational repertoire that's there, that's latent, and assemble it into useful pieces and organized goal directed, control.
夏天是旅行的好时节，也是学习新语言的好时机。
Summer is a great time for travel, and that's a great time to think about learning a new language.
哪怕只是学几句日常用语，我亲身经历过，懂一点当地语言会让出国旅行有趣得多。
Even if it's just a few phrases to get around, I can testify from experience that knowing a bit of the local language makes foreign travel so much more fun.
一个很好的途径就是使用Babbel。
One great way to get there is with Babbel.
每天只需十分钟，短短三周内，Babbel就能帮你开始进行真实的对话。
Even just ten minutes a day with Babbel can help you start having real conversations in as little as three weeks.
它不是让你死记硬背单词，而是教你实际会用到的短语，比如点餐、问路或者和当地人聊天。
Instead of memorizing random words, you're learning phrases you would actually use to order dinner, ask for directions, or talk with locals.
Babbel有互动对话、个性化复习，甚至还有播客，都是为了让你快速、自信地开口说。
Babbel has interactive dialogue, personalized reviews, even podcasts, all designed to get you speaking quickly and confidently.
而且不像旅行前临时抱佛脚，Babbel能融入你的实际日程：喝咖啡的间隙、通勤路上，或者睡前几分钟。
And unlike cramming before a trip, Babbel fits into your actual schedule: coffee break, commute, or a few minutes before bed.
Babbel这款屡获殊荣的应用已售出超过2500万份订阅，并提供十四天退款保证。
Babbel's award winning app has sold over 25,000,000 subscriptions and is backed by a fourteen day money back guarantee.
如果你夏天有旅行计划，现在就该开始学了，这样你才能在旅途中真正用上你学到的东西。
If you've got summer travel coming up, now's the time to start so you can actually use what you learn on the trip.
现在，Babbel 给听众提供高达六折的优惠。
Right now, Babbel's offering listeners up to 60% off.
访问 babbel.com/mindscape。
Go to babbel.com/mindscape.
就是 Babbel，babbel.com/mindscape，最高可享六折优惠。
That's Babbel, babbel,.com/mindscape for up to 60% off.
可能适用相关条款和限制。
Rules and restrictions may apply.
互联网是个有趣又有教育意义的地方，但也可能极其危险。
The Internet is a fun and educational place, but it can also be incredibly dangerous.
Incogni 来帮你避开其中一些危险。
Incogni is here to help you avoid some of those dangers.
使用你的优惠码 MINDSCAPE，订阅 Incogni 年度计划可享六折优惠。
And with your code MINDSCAPE, you can get 60% discount on an annual Incogni plan.
互联网的运行离不开数据，而现实情况是，你的很多数据都散落在网络上的各个角落。
The Internet runs on data, and the fact is a lot of your data is scattered around different places online.
其中有些信息你可能不希望公开，比如你的住址、电话号码和社会安全号码。
Some of it you probably don't want to be out there, including your address, phone number, and Social Security number.
Incogni 是一项服务，它会帮你追踪并从网络上多个网站中移除你的个人信息。
Incogni is a service that tracks down and removes your personal data from multiple sites across the web.
不仅限于暗网上的恶意泄露，还包括在线目录、搜索引擎和各类商业数据库。
Not just malicious leaks on the dark web, but online directories, search sites, and commercial databases.
这些都可能被不法分子利用。
All of those can be used by malicious actors.
Incogni 有自动化的功能，可以帮你搜索并删除你的数据，同时也提供定制化的删除服务。
Incogni has an automated component where they search for your data and get it removed, but also a custom removal feature.
如果你发现个人信息出现在了不该出现的地方，告诉 Incogni，会有专门的隐私专家为你处理。
If you find your info somewhere you don't want it to be, let Incogni know and a dedicated privacy expert will handle it for you.
我已经使用这项服务好几个月了，你绝对想不到你的名字在网上随处可见的记录有多少，也不知道把这些信息删掉后能带来多大的安心感。
I have been using the service for several months now and you wouldn't believe how many times your name appears in easily available records online or how much peace of mind it confers once they are removed.
所以，用 Incogni 把你的个人数据收回来吧。
So take your personal data back with Incogni.
访问 incogni.com/mindscape，使用优惠码 MINDSCAPE 即可享受年度计划六折优惠。
Go to incogni.com/mindscape to get 60% off an annual plan using code MINDSCAPE.
就在 incogni.com/mindscape，享受六折优惠。
That's 60% off at incogni.com/mindscape.
只要他们找不到你，就无法对你造成伤害。
They can't harm you if they can't find you.
所以这太棒了，因为确实引出了我真正想问的问题，就是在一个非常宏观的层面上。
So that's great because, yeah, it leads right into what I really wanna ask, which is, at a at a very high level.
所以我们稍后再深入细节。
So we'll get into details later.
对。
But Yeah.
对我来说，核心问题在于：我们打造了一个极其出色的系统，它能高效接收输入，并生成极具人性化的输出。
To me, the basic question is, look, we've built something that is really, really good at taking in inputs and giving outputs that are remarkably human.
对吧？
Right?
这足以骗过任何人，如果是老式的图灵测试的话。
That that that could fool anybody, if if you had the old fashioned Turing test.
我能想象两种不同的可能性。
And I can imagine two different possibilities.
一种是，LLM基本上重新发现了人脑中存在的相同机制；另一种是，LLM找到了一种全新的方式来模仿人类说话，我们可以称之为一种外星智能。
One is that's because, the LLMs have basically rediscovered the same mechanisms that are going on in the human brain, Or alternatively, the LLMs have figured out a wholly new way of sounding human, what we might think of as an alien kind of intelligence.
你知道我们是无情的拟人化者，所以如果它们表现得像人，我们很容易就会认为它们一定是人。
And you know that we're, you know, relentless anthropomorphizers, so it's gonna be easy to guess that they must be human if they're acting human in this way.
那么，在我们开始支持任何一种观点之前，有哪些选项呢？
So what are the options here before we start advocating for either one?
比如，有哪些可能性可以帮助我们理解盒子内部发生了什么？
Like, what are the possibilities to to help conceptualize what's going on inside the box?
是的。
Yeah.
我是说，我觉得你分析得非常到位。
I I mean, I I think you've laid it out really, really well.
我们能看到的是外在表现：输入提示词，然后看到输出的下一串文字流畅得让人心里发毛。
We see the outward we see the inputs, the prompts, and we see the next words come out and they're eerily fluent.
其中一种可能性是，它们只是高级的自动补全工具。
And one option is they are fancy autocompletes.
它们是统计近似器。
They're statistical approximators.
它们算是某种意义上的吹牛者。
They are bullshitters of sort.
它们是那些背熟了各种技巧和启发式规则，用来生成听起来像人话的下一个词的实体。
They are entities that have memorized a bunch of tricks and heuristics about how to generate next words that sound like people.
但在底层，发生的过程跟人类大脑里的运作方式几乎没有相似之处。
But under the hood, very little is going on that closely resembles what happens in the human mind brain.
没错。
Right.
这确实是一个连续谱。
And this is definitely a continuum.
中间会存在各种各样的过渡状态。
There's going to be various middle positions.
而在另一端，有一幅图景——我把话摊开来说，我接下来要论证的观点，也是我更倾向、更被吸引的观点是：事实证明，预测能力极其强大。
And at the other end, a picture that that that you know, I'll put my cards on the table that I'm gonna argue for a bit more and I think I'm more inclined to and attracted to is, it just turns out that prediction is incredibly powerful.
它是所有训练信号中最核心的那个。
It is the training signal that is the mother of all training signals.
回想我读研那会儿，大家可没这么看重这一点。
And whereas when I was going in grad school, this was much less appreciated.
那时候这也不是主流思潮。
Wasn't part of the zeitgeist.
不过在认知科学领域，这些年情况变了。
It has been over the years in cognitive science.
现在它已经成了舞台中心，因为大脑里发生的很多事情本质上都是预测。
It's been taking over center stage as a lot of what happens in the mind brain is prediction.
很多我们原本以为是先天设定、或者由某种偶然进化路径决定的认知原则，其实都是涌现出来的。
And a lot of cognitive principles that we thought were innately specified or due to some sort of contingent evolutionary trajectory, they actually are emergent.
在认知科学里，双系统理论也好，其他各种用来描述人脑心智的认知原则也罢，
In cognitive science, dual process distinction or various other cognitive principles that are, you know, that we describe the human mind brain.
它们其实都源于预测机制。
They are downstream of prediction.
结果发现，在最基础的认知原则上，大语言模型和人类识别出了相似的表征，采用了类似的处理技巧，只是在推理组织的模式等方面有所不同。
And so it turns out that at the level of basic core cognitive principles, the LLMs and humans, they identify similar representations, similar procedural techniques, different modes of inferential organization and so forth.
所以它们有很多相似之处，而这些相似性源于预测之后的过程。
And so there's a lot of similarity there and it arises downstream of prediction.
就像你之前说的，用“心智大脑”这个词，既避免了只说“大脑”可能让人不舒服，也避免了只说“心智”可能引起的争议。
Like how you called them the mind brain that sort of avoids worrying about calling it the brain and people getting upset or calling it the mind of people getting upset.
没错。
Exactly.
完全正确。
Exactly.
我受过公关公司的培训，不
I've been coached by a PR firm not
要谈那个。
to get into that.
那好吧。
So okay.
好的。
Good.
所以，各种可能性都存在。
So the the the options are out there.
也许LLM已经想出了一种不同的方式来表现人类行为，或者也许存在某种趋同进化，使得表现人类行为的最佳方式也正是LLM所发现的。
Maybe LLMs have thought of a or have come up with a different way of acting human, or maybe there's some sort of some form of convergent evolution where the best way of acting human is also what the LLMs found.
我必须承认，有一些类似轶事的证据给我留下了深刻印象，那就是LLM犯的错误似乎是人类永远不会犯的那种。
There is this, like, quasi anecdotal evidence, that has made a big impression on me, I will confess, that the kinds of mistakes that LLMs make seem to be the kinds that no human would ever make.
当然，每个人都会犯错，但错误的类型是不同的。
Like, of course, everyone makes mistakes, but there's different kinds of mistakes.
比如，大家都知道，LLM 数不清 strawberry 这个词里有几个 r，除非你专门教它们。
Like, famously, the LLMs can't count the number of r's in the word strawberry until you really teach them.
那么，这种证据对你有多大影响？
So, how much of an impact does that kind of evidence make on you?
是的。
Yeah.
我觉得，你知道，我是个完全靠证据说话的人。
I think, you know, I'm a total evidence guy.
所以，对，那是一个数据点。
So yeah, that's a data point.
你看着它，确实让人困惑。
You look at it, that's puzzling.
然后，你知道，下一刻它又在阐述广义相对论中非常微妙的部分。
And you know, and then the next moment, it's expounding on parts of general relativity you know, are very subtle.
所以，你知道，这到底是怎么回事？
So, you know, what is going on there?
有一件事我们不应该做，就是只停留在这些行为观察上。
And one thing that we shouldn't do is just stick with these kind of behavioral observations.
你会注意到很多地方，LLM的行为不像人类。
You're going to notice various places where the LLMs do not act like in very human like ways.
我们不应该满足于只看外在行为，然后统计一下，这个看起来有点像，那个看起来不太像。
And we shouldn't settle for just looking at outward behavior and counting and tabulating, well, looks a little similar, this looks a little dissimilar.
如果一直停留在那个层面，很快就会发现，任何一种可能性都仍然存在。
Very quickly, if we stay at that level, it's always gonna remain any one of these possibilities could be live.
你知道，外星智能，或者我更喜欢称之为“认知表亲”，指的是另一种情况，即它们实际上和我们很像。
You know, alien intelligence versus the name I like to give is Cognitive Cousin for the other version where they're actually much like us.
关于草莓那个问题，这些模型显然有不同的学习历史，它们与所谓“世界”的接触完全是通过文本，通过那些标记，这些标记对它们来说就像是感官原始数据，决定了它们接收什么。
It turns out with the strawberry, these things, they do have obviously a different kind of learning history, and their contact with the quote unquote world is exclusively textual via these tokens, which essentially serve as kind of sensory primitives for them about what comes in.
所以它们甚至无法接触到字母级别。
So they don't even have access to the letter level.
不是说它们无法接触，而是当单词输入时，它们通常不会处理单个字母。
It's not they don't have access, but they don't typically operate with individual letters when words come in.
所以这就很自然地解释了为什么他们数不清草莓里有多少个r。
So this is a very natural explanation for why they can't count the number of r's in strawberry.
每当我们看到这类异常现象，而且这类现象还很多，我们马上可以问的下一个问题是，在行为层面上，甚至在我们开始深入研究底层机制之前，LLM在行为上是否重现了很多我们熟悉的现象？
Whenever we see these kinds of anomalies, and there are many, an immediate next question we can ask is at the behavioral level, before we even start looking too closely at underlying mechanisms, are there a bunch of ways in which LLMs reproduce behaviorally?
这些现象是我们在认知科学领域几十年来记录下来的。
Phenomenon that are familiar, that we've documented over the decades in cognitive science.
他们是否重现了其中一些现象？
Do they reproduce some of those phenomenon?
事实证明，这样的案例有很多，而且很多都很引人注目。
Turns out there are many such cases and many of them are notable.
它们很吸引眼球。
They catch your eye.
我能举几个例子吗？
Can I go through some examples?
当然可以。
Sure.
好的。
Okay.
在语言领域，心理语言学家会讨论很多效应。
In in the area of language, there are a bunch of effects that psycholinguists talk about.
比如，中心嵌入句就是一个例子。
So an example is the center embedded sentence.
所以，“一个男人跑了”很容易理解。
So a man ran is easy to understand.
“一个爱着女人所爱的男人的女人所爱的男人跑了”很容易理解。
A man that a woman a man that a woman loves ran is easy to understand.
嗯。
Uh-huh.
“一个孩子认识的女人所爱的男人跑了”就开始变得难以处理了。
A man that a woman that a child knows loves ran starts to become hard to process.
对。
Right.
这就是中心嵌入。
That's center embedding.
心理语言学家已经发现了，你知道，现在我先不讨论机制。
And psycholinguists have identified, you know, for now, I actually won't talk about mechanisms.
这个我留到稍后再说。
Let me let me save that for a little bit later.
还有花园路径句。
There's garden path sentences.
马跑过谷仓。
The horse raced past the barn.
对。
Right.
摔倒了。
Fell.
这个“摔倒”好像凭空冒出来一样。
The fell seems to come out of nowhere.
这是因为我不是在说那匹马跑过了谷仓。
And it's because I'm not saying the horse raced past the barn.
我说的是那匹被赶着跑过谷仓的马摔倒了。
I'm saying the horse that was raced past the barn fell.
所以这里有一个顺序效应。
And so there's an order effect.
当你进行增量解析时，这些花园路径句就会出现。
When you do incremental parsing, these garden paths arise.
还有其他类型的效应，比如基于相似性的干扰。
There are other kinds of effects, similarity based interference.
还有深度变化。
There are depth changes.
所以心理语言学家已经发现了许多这类效应。
So there's a bunch of these kind of effects that psycholinguists have identified.
大型语言模型也表现出这些效应。
LLMs exhibit those.
还有一个很典型、很引人注目的例子，出现在序列列表记忆这种情景记忆的分支里。
Another nice and striking one is in serial list memory, a kind of branch of episodic memory.
如果你给人们看一个单词列表，让他们复述出来，你会发现存在近因效应。
If you give people a list of words and ask them to say the words back, what you'll find is that there's a recency effect.
排在最后的单词更有优势。
The very last word has an advantage.
排在最前的单词也更有优势。
The very first word has an advantage.
中间丢失效应。
There's a lost in the middle of effect.
中间的那些内容，就没有这种优势。
Those things in the middle, they don't have that advantage.
还有邻近效应，也就是说，如果你提到一个词，它周围的词会更容易被想起来。
There's a contiguity effect where if you say a word, the words around it come to mind more easily.
还有时间不对称效应。
There's a temporal asymmetry effect.
所以如果你说一个词，后面的词更容易被想起来。
So that if you say a word, the one after it comes to mind more easily.
大型语言模型也表现出所有这些效应。
LLMs exhibit all these effects.
当你在提示中放入一串词时，第一个词和最近的词更容易进入处理过程。
When you put a bunch of words in their prompt, the first word, the most recent word are more prominent to enter into processing.
中间的词容易被忽略。
There's a lost in the middle.
还有时间邻近性，也就是邻近效应。
There's temporal contigu there's a contiguity effect.
然后还有时间不对称效应。
And then there's the temporal asymmetry effect.
我再举一个例子。
I'll give one more example.
好的。
Yeah.
这些效应，视觉搜索已经被广泛研究了。
And and these so there's visual search has been extensively studied.
有一个经典效应，分离搜索和联合搜索。
And there's a classic effect, disjunctive versus conjunctive search.
比如说，如果我说在一堆红色的L中找绿色的L。
So if I say find the green l among a bunch of red l's.
想象一下那个画面。
So picture that.
有一个绿色的L，还有一堆红色的L。
There's a one green l and there's a bunch of red l's.
绿色的L会一下子跳出来。
The green l's gonna pop out.
因为绿色是一个独特的特征，能把那个东西区分出来。
Because the greenness is a singleton concept that distinguishes that one thing.
如果我说，在红色的L和绿色的T中找绿色的L。
If I say, find a green l among red l's and green t's.
这时候你就遇到难题了。
Now you got a problem.
没有任何单一特征能把你找的东西区分出来。
There's no one feature that distinguishes the thing that you're looking for.
所以大家的速度会变慢。
And so people slow down.
他们得按顺序一个个看。
They do this in serial order.
完成任务所需的时间与数组中元素的数量成正比。
And their time in completing the task is proportional to the number of elements in the array.
这是非常经典的现象。
So this is classic.
我喜欢这些研究的一点是，认知科学家早在几十年前就已经把这些效应记录下来了。
What I love about these is cognitive scientists have already written down these effects decades ago.
我们可以检验一下大语言模型，比如这里的视觉语言模型，是否也表现出这种效应？
And you can check the LLMs, in this case vision language models, do they exhibit the effect?
确实如此。
And they do.
嗯。
Mhmm.
所有这些例子，我们都清楚背后的机制。
All of these examples, we know the mechanisms behind it.
我们知道花园小径现象是因为增量解析，也就是逐词解析，导致错误的解析路径，让你走偏，然后你得回头重新组织。
We know that garden paths arise due to incremental pars, parses that happen one word at a time that lead to bad parses leading you astray, and then you've gotta go back and regroup.
我们知道情景记忆效应。
We know the episodic memory effects.
我们知道一些起作用的机制。
We know some of the mechanisms that play a role.
而在视觉搜索中，这涉及到什么是概念原语。
And in the visual search, it has to do with what are the conceptual primitives?
什么通过组合编码来编码，什么需要特征绑定。
What is encoded via a compositional code, and what requires feature binding.
你在大型语言模型和人类身上都观察到了这些不那么明显的相似模式，尤其是在认知科学中我们已经知道某些效应背后的机制时。
The fact that you're seeing these non obvious patterns of similarities in LLMs in people, especially where we know some of the mechanisms that happened in these effects in cognitive science.
这表明大型语言模型和人类可能使用了相似的运作机制。
They point to similar mechanisms being operative in the LLMs in people.
那我试着总结一下：草莓这个例子也许能证明，大型语言模型找到了一种截然不同的方式来模拟人类说话。而我们通过草莓这个案例揭示出‘皇帝的新衣’其实不存在，也就是说，大部分时候大型语言模型确实是在进行类似人类的认知活动，我们只是发现了一个小小的阿喀琉斯之踵。
So if I can try to summarize that, the strawberry thing could conceivably be evidence that the LLMs have found a very different way of sounding human, and here we're showing that the emperor has no clothes with the strawberries, or mostly the LLMs are doing human type cognition and we've just found one little Achilles heel.
我会用更温和的方式来表述这一点。
I I I would actually put it in an even more benign way.
我们一开始就知道，他们的感官证据，对应到我们的情况，就是英语单词的离散化表征。
We knew at the get go that their sensory evidence, what is analogous to that is tokenized representations of English language words.
他们很少会深入到单个字母的层面，所以我们从一开始就清楚这一点。
They very rarely descend to the level of individual So we knew that at the get go.
这本来就不该成为他们和我们之间类比的主要维度。
It wasn't supposed to even be a major dimension of the analogy between them and us.
也就是说，单个字母在他们的处理策略中并不特别突出。
That individual letters would be particularly salient in their processing strategies.
所以我要说的两个教训是，在行为输出层面进行计数和制表可能不会让我们走得太远。
So the two lessons I would say are that counting and tabulating at the level of behavioral outputs is probably not going get us very far.
我们需要从机制上入手。
We need to look mechanistically.
我们需要思考哪些机制才是我们真正关心的，是人类心智大脑的核心处理原则。
And we need to think about which are the mechanisms that we actually care about, that are core processing principles for the human mind brain.
处理单个字母，这从来不是我提升到核心处理原则层面的东西。
Dealing with individual letters in, that's never something that I would elevate to the level of a core processing principle.
对。
Right.
而其他一些涉及情景记忆和视觉搜索、组合编码和增量解析的机制，这些是认知科学家长期以来思考的基本内容。
Whereas some of the others that deal with episodic memory and visual search, compositional codes and incremental parses, that's bread and butter of what cognitive scientists thought about for a long time.
所以这些机制得以保留可能更重要。
And so the fact that those are preserved are probably more important.
我不想纠结于LLM的各种特殊之处，但我觉得你对草莓问题的回答似乎没有回答洗车问题。
I don't wanna like dwell on all of the different, idiosyncrasies of the LLMs, but I guess your answer to the strawberry question doesn't seem to provide an answer to the car wash question.
你知道这个的。
You know this one.
对吧？
Right?
你了解这个。
You know this
我不清楚这个。
I do not know this one.
哦，这是个很好的例子。你问大模型：我现在离洗车店五百英尺远。
Oh, this is a good example where you you ask the LLM, I am 500 feet away from the car wash.
我想去洗车店洗个车。
I'd like to get my car washed at the car wash.
我是开车过去好，还是直接走过去？
Should I drive my car there or just walk?
而大模型肯定会回答：你就直接走过去吧。
And the LLM will inevitably say, you should just walk.
展开剩余字幕（还有 480 条）
才五百英尺远。
It's only 500 feet away.
虽然没明说，但走路去可不行，因为我要把车开过去洗。
Not with the implication being, but it does be no good to walk because I wanna get my car washed there.
哪怕很近，我也得开车过去。
I need to drive my car there even though it's very close.
所以呢
And that
你知道这件事有多好笑吗？
You know what's hilarious about that?
我本来也想说你应该走过去。
I was about to say you should walk.
才500英尺远而已。
It's only 500 feet away.
你跟大模型混得太久了，Chandra。
You've been hanging out with LLMs for too long, Chandra.
是的。
Yeah.
没错。
Exactly.
要么是它们把我的脑子搞坏了，要么就是无意中搞的。
Either they've rotted my brain or unwittingly.
我们现在看到的是，它们的那些小毛病和怪癖，至少在某些方面，有点像我们。
What we're seeing is the foibles and idiosyncrasies of them are a little bit like us, at least in some cases.
对。
Yeah.
我觉得就是这样。
I think that's it.
好。
Okay.
总之，我就是想把那件事说清楚。
Anyway, I just wanted to get that on the table there.
但是，好吧。
But but okay.
你之前很客气，没有深入讲双过程理论，但现在是个好时机，因为如果你要论证LLM的认知策略非常像人类，这里就有一个人类如何运作的例子。
Now you've been you were very gracious in not going into the dual process theory when when you wanted to, but here is a good time to do it because if you're gonna make the argument that LLMs are sort of remarkably human like in their cognitive strategies, here's one example of how humans work.
所以解释一下，我们别假设大家都知道双过程理论是什么。
So explain let let's not assume that people know what the dual process theory is.
当然。
Sure.
对。
Yeah.
那么什么是双过程理论呢？
So what is dual process theory?
人们通常熟悉卡尼曼的《思考，快与慢》。
People often are familiar with Kahneman's Thinking Fast and Slow.
他获得了诺贝尔奖，在他的诺贝尔演讲中。
And his Nobel he won the Nobel Prize in his Nobel lecture.
他在诺贝尔演讲中，用双过程理论重新梳理了他整个精彩的研究项目。
He revisits his entire amazing research program in terms of dual process theory.
这个理论的核心就是，你有一种快速思维。
And so the idea is that you have fast thinking.
它很快。
It's quick.
它是自动的。
It's automatic.
它是直觉性的。
It's intuitive.
它的结果就像一道闪光一样出现在你面前。
It results appear to you as a kind of a flash.
然后你还有慢速思维，它很慢，是序列式的，会消耗更多的认知资源。
And then you have slow thinking, which is slow, it's serial, it draws on more cognitive resources.
而快速思维被认为是通过与环境长期耐心的、依赖经验的互动中产生的。
Whereas fast thinking is thought to arise via slow patient experience dependent practice with the environment.
慢思考的一个特点是，它能非常迅速地处理复杂新颖的问题。
A characteristic of slow thinking is it deals very rapidly with sophisticated novel problems.
以上就是快思考和慢思考的一些特征。
So those are some of the features of fast and slow thinking.
举个例子，如果我给你看一张约翰·特拉沃尔塔的照片，你脑子里会立刻蹦出他的形象。
You know, an example, if I show you a picture of John John Travolta, it flashes out to you.
那就是特拉沃尔塔。
That's Travolta.
并不是说你要一步步去推理。
It's not as if you reason through it.
对。
Yeah.
比如我说，因为参加的人太多，所以把球搬到了楼上。
If I say, because so many people attended, they moved the ball upstairs.
你知道我是在说人群聚会，而不是指那个圆形的物体。
You know that I'm talking about a gathering of people, not the spherical thing.
这就是快思考。
That's fast thinking.
而慢思考呢，比如我给你做多位数的心算。
And know, and slow thinking if I give you, for example, multi digit mental arithmetic.
脑子里不会立刻闪现出答案。
Nothing flashes before your head.
而是有个过程。
There's some sort of procedure.
也许你作为医生，不，
Maybe you as a physician No,
不，不，不。
no, no, no.
算术不是这样的。
Arithmetic is not it.
对。
Yeah.
这就是快思考和慢思考。
So that's thinking fast and slow.
这是一个影响极其深远的理论。
And it's just a toweringly influential theory.
这个理论并非卡尼曼首创。
It doesn't originate with Kahneman.
虽然术语是基思·斯坦诺维奇和理查德·韦斯特提出的。
Though Keith Stanovich and Richard West coined the terminology.
在社会心理学和刻板印象研究中，人们会用到这个理论。
And social psychology, stereotyping, they use this theory.
在临床方面，我本身是精神科医生。
Clinical, I'm a psychiatrist as part of my background.
药物成瘾、多动症，这些领域都会借鉴这个理论。
Substance addiction, ADHD, this is a theory that people draw on.
发展心理学中，青少年的行为方式就是因为快思考超过了慢思考。
Developmental psychology, adolescents behave the way they do because fast thinking outstrips flow thinking.
这只是一个在心理学领域广泛使用的解释框架。
It's just a explanatory form that is widespread throughout psychology.
所以这是一个例子。
And so this is an example.
我们得看看，如果说大语言模型和我们相似的话……
We better see if we're going to say that LLMs are like us.
大语言模型里有没有类似的东西呢？
Is there something like that in LLMs?
这对我们认知科学家理解这种区分有帮助吗？
And can help us as cognitive scientists understand this distinction?
答案是肯定的。
And the answer is yes.
好的。
Okay.
所以在大型语言模型中，人们有时在权重处理方面提到的一个概念之间存在着非常有趣的区别。
So in LLMs, there is a really interesting distinction between what people sometimes call in weight processing.
然后还有一对，所以是权重处理，就是w e i g h t。
And then there's a pair of, so in weight processing, so that's w e I g h t.
我一会儿再讲这个。
I'll come to that in a second.
这很像快速处理。
It's very akin to fast processing.
然后LLM中有两种机制支撑所谓的系统二，即慢速处理。
And then there's two mechanisms in LLMs that help to undergird that so called system two, the slow processing.
但我会从权重处理开始讲。
But I'll start with inweight.
LLM通过预测目标，获取大量信息，这些信息被编译到它们的权重中。
LLMs via the prediction objective, they acquire massive amounts of information that's compiled into their weights.
这就是为什么叫权重处理。
That's why it's called in weights.
从刺激信息到LLM中填充的内容，这是一条相对容易的路径。
It's a relatively easy pathway from the stimulus information to something that's populated in the LLM.
这不需要在它们的激活空间里进行复杂的构建。
It doesn't require elaborate constructions within their activation space.
如果这听起来有点晦涩，我会再多说一点。
And I'll say a little bit more about that if that sounds obscure.
所以阿尔伯特·爱因斯坦是德国人，它们就是知道。
So Albert Einstein is German, they just know it.
迈克尔·乔丹打篮球。
Michael Jordan plays basketball.
达拉斯所在的州是得克萨斯州。
The state containing Dallas is Texas.
如果你打碎一个花瓶，它会碎掉。
If you drop a vase, it will shatter.
这些都是它们权重中存储的常识性事实。
These are all the kind of common sense factual things that they know in their weights.
有两个机制支撑着它们的系统二。
There are two mechanisms that undergird their system too.
第一个机制，就是在处理上下文的时候。
And the first one, in context processing.
我想稍微详细讲一下这个。
I'm going to dwell a little bit on it.
因为我之前没提过，现在应该说说。
Because I haven't mentioned this yet and I should say this.
有一篇论文，和我今天要讲的一些论点相关。
There's a paper that goes along with some of the arguments that I'm covering today.
那篇论文，我有一个合著者。
And that paper, I have a co author.
就是我的同事里克·刘易斯，他是密歇根大学的心理学和语言学教授，也是我的朋友和合作伙伴。
It's my Rick Lewis, professor of psychology and linguistics at Michigan, and my colleague and my friend and collaborator.
里克说，我们最终会把论文放到节目笔记里，我想是的。
So Rick calls, we'll put the paper up in the show notes eventually, I think.
里克称这是认知科学史上最伟大的发现。
Rick calls this the greatest discovery in the history of cognitive science.
所以在语境处理方面。
So in context processing.
也许这么说还不够夸张。
So maybe that's not hyperbolic enough.
那什么是语境处理呢？
What is in context processing?
这是一种令人惊讶的现象，预测训练会在模型中植入这种机制。
It's a surprising phenomenon that prediction training installs in the model.
比如一种非常通用的流程，能从提示中提取模式、从基础案例中归纳推广，仅凭指令就能学会新的映射关系。
Something like a very general routine for extracting patterns from the prompt, generalizing from base cases, learning a novel mapping from instructions alone.
我之前描述系统二的时候说过，它的一个主要特征就是：复杂的新颖模式会突然被快速理解。
When I characterized system two, I said one of its primary properties is it's the place where rapid novel grokking of complex novel patterns suddenly happens.
认知科学一直搞不清楚这个现象，也没法对它做细致的刻画。
And cognitive science has been at a loss to understand this phenomenon and characterize it in any detail.
在布拉德福德等人2019年以及布朗2020年的两篇论文里……
And in a pair of papers, Bradford and colleagues in 2019 and Brown in 2020.
第二个是来自OpenAI的。
These the the second one is from OpenAI.
其中一篇的领衔作者是Ilya Sutzkever，另一篇的资深作者是Dario Amode。
And Ilya Sutzkever is lead author of one of these, and Dario Amode is lead author I mean, he's a senior author in the other one.
他们都是大牛，尤其是GPT-3那篇2020年的论文。
These are like They, giants of the especially in GPT-three, so the 2020 paper.
这些模型仅仅通过预测训练，就展现出了惊人的上下文推理能力。
These models, when they're just trained on prediction, they have an astonishing ability to display this context reasoning.
有一篇论文指出了上下文处理的一个机制，里面用了一个例子。
And an example that was used in a paper that identified one of the mechanisms of in context processing.
我来举那个例子。
I'll give that example.
比如，居里是……
You say, Curie is.
我是说玛丽·居里，她是镭的发现者。
Well, I mean Marie Curie, she's is the discoverer of radium.
她是一位了不起的科学家。
She's an amazing scientist.
这些是你脑海中浮现的东西。
Those are things that come to your mind.
也许这些就存在于模型的权重中。
Maybe that's in the weights of the model.
但如果我说阿尔伯特·爱因斯坦是德国人，圣雄甘地是印度人，玛丽·居里是——你会说是法国人。
But if I say Albert Einstein is German, Mahatma Gandhi is Indian, Marie Curie is, you go French.
有些人可能知道她出生在波兰。
Some may know that she was born in Poland.
波兰人。
Polish.
我正想说波兰人。
I was gonna say Polish.
对。
Yeah.
我知道得太多了。
I know too much
关于那些用这个例子的计算机科学家，他们可能甚至不知道她是波兰人。
about The this computer scientists that that use this example, they may not have even known that she was Polish.
所以你就直接假设答案是法国。
So you just assume the answer is French.
模型会意识到这里在要求一种模式。
The model gets the idea that there is a pattern being requested.
他们确实对此进行了很多详细的研究。
And they they actually study this in a lot of detail.
他们使用隐马尔可夫模型，以及它们的混合模型，并表明模型在做类似隐式贝叶斯推断的事情。
They use hidden Markov models, mixtures mixtures of of them, them, and and they they show the model is doing something like implicit Bayesian inference.
它对被问的任务有一个假设。
It's got a hypothesis about what task is being asked.
它利用第一个例子作为似然。
It's using the first example, the likelihood.
它大概是在用某种先验乘以这个可能性，从而得出关于任务类型以及在这种情况下哪种补全最合适的后验。
It's multiplying that presumably by some sort of prior to arriving at a posterior about what kind of task is there and what kind of completion is the best for this circumstance.
也有人把这比作模型内部发生的梯度下降。
Others have likened this to more like gradient descent happening within the model.
我想强调的是，这发生在权重被冻结之后。
I want to emphasize this is happening after the weights are frozen.
那么我说的梯度下降是什么意思呢？
So what do I mean gradient descent?
梯度下降就是这些模型训练的方式。
Gradient descent is how these models are trained.
然后它们被发布，之后权重就被冻结了。
Then they're shipped, and then the weights are frozen.
你说的梯度下降是什么意思？
What do you mean gradient descent?
嗯，模型有激活值。
Well, the models have activations.
这些激活值在变化。
Those activations are changing.
这些激活值就是当你把节点乘以权重时，节点上发生的事情。
Those activations are what are happening on the nodes when you multiply the nodes by the weights.
模型内部编译了机器学习例程，比如线性回归、最近邻和贝叶斯方法。
Well, the model has compiled within it machine learning routines like linear regression, nearest neighbor, and Bayesianism.
它们能理解激活空间中非常复杂的模式。
And they can understand very complicated patterns in their activation space.
这就是所谓的上下文学习。
That's what this in context learning is.
它通过非常通用的预测训练产生。
And it arises via very general prediction training.
作为认知科学家，我们一直在努力阐述什么是系统二。
As a cognitive scientist, where we have struggled to articulate what is this system two.
认知科学的一个不为人知的秘密是，我们实际上并没有关于系统二或相关术语的理论。
The dirty secret of cognitive science is we don't actually have theories of things like system two or related terms.
我们有的只是关于它什么时候在青春期发展的理论。
What we have is theories of when does it develop during adolescence?
当人们进入扫描仪时，什么会被激活？
What lights up when you put people in the scanner?
实际上，对于很多这类东西的运作方式，我们并没有机制层面的过程理论。
We don't actually have mechanistic process level theories of a lot of how this stuff works.
而这里我们有一个展示这种现象的产物，我们可以从机制上对它进行探究。
And here we have an artifact that's displaying this phenomenon, and we can mechanistically interrogate it.
我们可以理解支撑它的那一堆常规方法。
And we can understand the grab bag of routines that underpin it.
所以这很惊人，这种上下文学习。
So this is startling, this in context learning.
而且它非常像系统二。
And it's very system two ish.
他们还有另一个系统二的机制。
There's another system two mechanism that they have.
这就是你在利用内部，把上下文当作草稿纸来用。
And that's where you're using the internal, you're using the context as a scratch pad.
然后你在里面进行连续的推理。
And you're doing extended serial reasoning within it.
所以我简单说一下这个，借用梅兰妮·米切尔的一个例子。
So I'll say a little bit about this quickly, borrowing an example from Melanie Mitchell.
比如你有这样一个问题。
So you have a problem like this.
朱莉娅有两个姐妹。
Julia has two sisters.
她的兄弟马丁有几个姐妹？
How many sisters does her brother Martin have?
哦，她刚告诉我她有两个，那里有两个姐妹。
Oh, Well, she just told me she has two there's two sisters there.
嗯，朱莉娅也是一个姐妹。
Well, Julia is a sister.
对。
Yeah.
所以，如果你把这类问题给一个没有使用思维链的模型，它很可能直接回答两个。
And so what if you give this type of problem to a model that is not using chain of thought, very likely to say two.
但如果你允许它生成不向用户展示的、会进入上下文并影响模型后续步骤的内部思考标记，它就能进行推理，比如：好的，朱莉娅是女孩，她有两个姐妹，这意味着家里有三个女孩，等等。
But if you allow it to generate internal thinking tokens that are not displayed to the user, that end up in the context, that influence the next steps that the model takes, it can reason through and says, okay, Julia is a girl, she has two sisters, that means there's three girls in the family and so forth.
因此，你可以证明，拥有这种思维链草稿纸功能的模型。
So you can show that a model that has access to this chain of thought scratch pad.
像那些在2024年最近获得IMO金牌的模型，我认为这放大了模型的推理能力。
Models like the ones went IMO gold recently, in 2024, that this amplifies the inferential power of the model.
好的。
Okay.
所以我们的起点是双过程理论的区别。
And so where we begin is dual process distinction.
这是人类心理学中一个非常核心的区分。
It's a towering distinction in human psychology.
我们发现这些模型具备上下文处理能力，也支持思维链推理。
And what we see is these models have in context processing and they have chain of thought.
这两者结合起来，再加上模型参数里内化的知识（对应人类的直觉反应），它们共同构成了一个近似人类认知的系统。
And between the two of them, they approximate the, and they also have the in weight which corresponds to the automatic.
是的。
Yeah.
所以，模型内部这种区分，其实跟人类大脑里的认知分工是如出一辙的。
And so you're ending up with a distinction in the model that mirrors that in people.
你可能会问，有没有确凿的证据表明，这种上下文中的思维链机制，其实跟人类大脑里的运作方式非常相似？
And you ask, is there clear evidence that this in context and chain of thought, that actually is much like what happens in the human mind brain?
答案是肯定的，确实有证据支持这一点。
And yes, there is evidence.
而且你知道的，我们还有反应时间的数据作为佐证。
And you know, there is reaction time evidence.
我自己的实验室做过一项研究，采用了类似斯特鲁普（Stroop）任务的范式，结果清楚地展示了模型内部的运作过程与人类大脑的活动之间存在着高度的一致性。
My own lab has a study using paradigms like the Stroop task that illustrate that there's a very tight correspondence between what is happening in the model and what happens in the human mind brain.
我们经常用系统一和系统二的区别来解释这个。
For which we often invoke the system one, system two distinction.
我不想太快带过这个话题。
So I don't wanna gloss over it too quickly.
你是说LLM发现了贝叶斯定理吗？
Are you're that LLMs have discovered Bayes' theorem.
对。
Yeah.
这个怎么说呢？
You you know, what what to say about that?
众所周知，LLM能够近似后验概率。
So it has been widely known that LLMs approximate the posterior.
所以，你有一个上下文，然后下一个词。
So the you know, you've got a context and then there's a next word.
然后有一个条件分布。
And there's a conditional distribution.
而且它们对那个条件分布的逼近效果非常好。
And they approximate that conditional distribution very well.
贝叶斯学派会说，这是规范的做法。
And a Bayesian would say, here's the normative way to do it.
你有这些假设的分解表示。
You have these factorized representations of hypotheses.
你有一个先验，然后乘以这些似然。
You've got a prior and you multiply it by, you know, these likelihoods.
事实证明，你可以近似贝叶斯后验或正确的条件分布，不是通过显式地使用分解表示，而是通过建模或学习它。
It turns out that you can approximate the Bayesian posterior or the correct conditional distribution, not by explicitly going through a factorized representation, but in a sense, by modeling it or learning it.
通过捷径，直接看上下文中有哪些特征。
By shortcutting your way through what are the features present in the context?
我能把这些特征输入一个强大的函数逼近器，得到类似贝叶斯后验的结果吗？
Can I stick those into a powerful function approximator and get something Bayesian posterior like?
这叫做摊销推断。
It's called the amortized inference.
这是一个统计学原理，在机器学习领域已经知道了几十年，你不必通过贝叶斯计算就能得到贝叶斯后验。
And it's a statistical principle that's been known in machine learning community for decades, that you don't have to go through a Bayesian calculation to get to the Bayesian posterior.
你可以通过非常漫长的学习过程来摊销它。
You can amortize it via the very long learning process.
这些东西可能就是在做这件事。
That's what these things are probably doing.
而且它们不仅仅是近似贝叶斯后验。
And it's not simply that they approximate the Bayesian posterior.
那实际上并不是上下文学习的本质。
That's not actually what in context learning is.
它们能够将一个新情况映射到它们激活中的贝叶斯近似器，在权重被冻结之后。
It's they can take a novel situation and map it into a Bayesian approximator in their activations after their weights have been frozen.
所以贝叶斯主义不仅仅发生在那个扩展的预测下一个词的互联网规模训练阶段。
So the Bayesianism doesn't just happen during that extended predict the next word internet scale training phase.
贝叶斯主义渗透到了权重被冻结之后它们如何处理信息的层面。
The Bayesianism penetrates at the level of after their weights are frozen frozen, how they process.
对。
Yeah.
而且这很可能也正是人脑里发生的事。
And that's very plausibly what is happening in the human brain too.
我们不是天生就带着贝叶斯定理，或者说它并没有印在我们的神经元上。
We didn't we're not born with basis theorem imprinted on our on our neurons.
对。
Yes.
Sean，在聊完这个话题之前，我还想再说几句。
Sean, I I wanna say just a little bit more before we leave this topic.
因为对这种双过程的区别，我真是特别来劲。
Because I get so excited about this dual process distinction.
对我来说，这就像是在喊：嘿，看着我！
To me, this is, you know, screaming, Hey, look at me.
我是一个大语言模型。
I'm an LLM.
看看我。
Look at me.
看看我。
Look at me.
我并没有那么陌生。
I'm not so alien to me.
对。
Yeah.
我研究斯特鲁普任务，也就是认知科学中的冲突任务，已经二十年了，从我读认知科学哲学研究生时就开始。
I have studied Stroop tasks, they're called conflict tasks, in cognitive science for the last twenty years, since I was a graduate student in philosophy of cognitive science.
在实验室里，认知科学家花很多时间试图理解这两种相互竞争的过程等等。
They are, there's just a laboratory where cognitive scientists spend a lot of time in trying to understand what are these two processes that are competing and things like that.
在大型语言模型中，你可以找到一些地方，比如我们用的提示是：蜡笔是红色的。
In LLMs, you can find places where, so the prompt we use is, the crayon is red.
所以你说蜡笔是。
So you say the crayon is.
模型会想说红色。
The model wants to say red.
这是它自动编译出来的反应倾向。
That's its automatic compiled disposition.
当然，我告诉过你蜡笔是红色的。
Of course, I told you the crayon's red.
你肯定会说是红色。
You're gonna say it's red.
然后我们加一个基于规则的前缀，把红色和蓝色或者任意两种颜色的映射关系反转过来。
Then we append a kind of rule based prefix that reverses the mapping between red and blue or any pair of colors.
我们发现，而这正是涉及到上下文和推理推断。
What we find, and and that taps in context, inferential reasoning.
对。
Mhmm.
我们发现的是模型里承载自动性的因果路径。
What we find is causal pathways through the model where the automaticity resides.
模型中的因果路径，也就是那些反对自动性的上下文处理出现的地方。
Causal pathways through the model where the in context processing that opposes the automaticity arises.
你会发现冲突任务中典型的相容效应。
You find classic congruency effects that you find in conflict tasks.
你会发现所谓的相容促进效应，这是另一个效应。
You find what's called the congruency facilitation effect, another effect.
你会发现相容序列效应，这是认知科学家描述过的另一个效应。
You find the congruency sequence effect, another effect that cognitive scientists have described.
你可以微调模型来增强自动性，然后得到预期的效应。
You can fine tune the model to ramp up automaticity, and you get the predicted effects.
你可以通过一种类似人类的操作来削弱上下文处理，比如用一个很长的规则来给模型增加负担和压力。
You can impair in context processing using a human like manipulation where you burden and tax the model by a very long rule.
这正是那种会让人出错的情况。
Exactly the kind of thing that would mess up a human.
然后你会发现上下文处理变弱了，而且它只影响不一致的条件。
And you find that in context processing is weaker and it selectively affects the incongruent condition.
整个过程和人类的表现很像。
The whole thing resembles people.
所以我特别喜欢这个例子，因为它就是一个实体。
And so I love this example because this is just an entity.
而且我们是在Gemma 2b上做的，这个模型甚至没有经过指令微调。
And we're doing this on Gemma 2b that hasn't even been instruction tuned.
所以它是个小模型。
So it's a small model.
它只是预测下一个词，却得到了所有这些基础设施。
It's just predict the next word that gets all of this infrastructure there.
对。
Right.
如果你在2010年问我，这些双过程效应是从哪里来的？
If you asked me in 2010, where do these dual process effects come from?
我会说，是进化给了我们爬行动物的大脑，然后它又安装了高级皮层的其他基础设施，而且这些都是天生的。
I'd say, well evolution gave us a reptilian brain and it installed this other infrastructure of higher cortex and it's all innate.
而且你知道吗，我在罗格斯大学读书的时候，杰瑞·福多就在那里。
And I was a student, you know, at Rutgers where Jerry Fodor was there.
对。
Yep.
所以我也会说，我们可能永远都搞不清楚系统二到底是怎么运作的。
And so I would also throw in, we'll probably never figure out how the system two part ever ever works.
那些话我都会说。
I would say all those things.
但那些全都是错的。
All of that is wrong.
没错。
Right.
预测目标看起来就搭建了这整个基础设施。
Prediction objectives looks like it sets up this entire infrastructure.
哇。
Wow.
好的。
Okay.
所以换一种说法，如果我理解错了请纠正我，就是，你看。
So one way of rephrasing that, again, correct me if I'm wrong, is, look.
当然，LLM的回答和人类的回答之间确实存在表面上的相似性，因此一方面，你会忍不住想把它们拟人化。
There of course, there's this superficial similarity between responses from LLMs, responses from humans, and therefore, on the one hand, you're tempted to anthropomorphize them.
另一方面，你又想抵制这种诱惑。
On the other hand, you wanna resist that temptation.
但你说的是，情况远不止如此。
But you're saying that it goes beyond that.
人类推理中有很多更微妙的方面，普通人甚至察觉不到，但训练有素的认知科学家却非常清楚。
There are much more subtle aspects of human reasoning the typical person on the street doesn't even recognize, but the highly trained cognitive scientist is is well aware of.
而且我们发现LLM中也存在这些方面。
And we're finding those in the LLMs also.
完全正确。
Exactly.
没错。
Exactly.
而且关于双重过程这个区分，这其实是那种普通人确实能察觉到的例子。
And with the dual process distinction, that's one of those, though, where the man on the street actually does recognize this.
《思考，快与慢》之所以有影响力，部分原因就在于它和我们从内省中得知的东西产生了共鸣。
Thinking fast and slow is is influential in part because it resonates with something that we know from introspection.
对。
Right.
所以这是那种我们可以通过内省看到这种区分的例子。
So this is one of those where we can look inside and see this distinction.
是的。
Yeah.
不过，冒着跑题的风险，我们有多大把握说这真的是一个双重过程理论，而不是一个多重过程理论呢？
And at the danger of derailing, how confident are we that it is a truly dual process theory versus just a multiprocess theory?
它真的只有两个尺度，还是说在不同尺度上会发生不同的过程？
Is it truly just two scales, or are there different processes that happen on all sorts of different scales?
对。
Yeah.
我的意思是，权重内和上下文中的区别，到底是什么？
I I mean, the in weight, in context distinction, what is it?
我要说的是，认知科学家们对这个问题有争议。
One thing I will say that cognitive scientists debate this question.
双过程理论的一个弱点，任何认知科学家都会感到沮丧的是，它很多时候只是一堆形容词。
And one of the weaknesses of dual process theory, any cognitive scientist gets frustrated is a lot of it operates as a list of adjectives.
其中一个过程是快速的、自动的、不费力的。
Well, one of these processes is fast, automatic, effortless.
另一个是缓慢的、依赖工作记忆的，然后你有了这些形容词，但机制细节在哪里呢？
One of these is slow, working memory dependent, and you've got these adjectives and hey, where's the mechanistic details?
没有机制细节，就很难知道如何计数，理论到底承诺了什么，等等。
Without the mechanistic details, it's hard to know how you count, what exactly is the theory committed to, and so forth.
现在你有了一个人工制品，也就是大语言模型，我们对上下文中的处理与权重内的处理有了更多了解。
Now you have an artifact, that is the LLM, and we understand a bit more how in context processing works versus in weight.
我更愿意把它描述为一个连续体。
I would describe it much more as a continuum.
LLM所做的每一件事都需要它的权重和激活。
Everything that an LLM ever does requires its weights and it requires activation.
但在权重内的情况下，刺激和反应之间的联系更直接，因为权重中已经包含了大部分反应。
But there's a much more direct link between the stimulus and the response via weights that already have most of the response within it in the case of in weight.
所以迈克尔·乔丹打篮球。
So Michael Jordan plays basketball.
不需要构建复杂的结构，对。
There isn't an elaborate structure that needs to get built Yeah.
在激活空间中才能得到那个反应。
In the activation space in order to get to that response.
但如果你有更复杂的提示，比如阿尔伯特·爱因斯坦是德国人，圣雄甘地是印度人，所以玛丽·居里是。
But if you have more complicated prompts like that Albert Einstein is German and Mahatma Gandhi is Indian, so Marie Curie is.
你必须在激活空间中构建一个结构。
You have to build up a structure in the activation space.
它会推断出任务是什么，然后将该任务应用到玛丽·居里身上，提取出相应的回答。
Something that infers what is the task, applies that task to Marie Curie, extracts what that response is.
激活空间现在充满了各种结构。
The activation space is now crowded with a lot of structure.
它很容易受到干扰等等。
It's very prone to interference and so forth.
所以虽然权重内和上下文内是一个连续体，但肯定有两个极端。
So while it is a continuum between in weight and in context, there's definitely poles
对。
Yeah.
分别锚定两端。
Anchoring each end.
所以现在你更有条件去理解这个区别的真正含义了，因为你已经有了非常清晰的机制性假设。
And so you're now in a better position to start to understand what this distinction really means because you've got, you know, very clear mechanistic hypotheses.
所以当我们深入探讨，试图通过先思考人类如何思考来弄清楚LLM是否以非常人类的方式思考时，双过程理论就是其中的一个例子。
So while we're digging into we're trying to figure out if the LLMs are thinking in very human ways by first thinking about how humans think, the dual process theory is one of the examples there.
你还有另一个例子，我完全不了解。
You had another example, which I'm not at all familiar with.
我知道双重过程理论，但你提到了产生式系统的概念，这是埃米尔·波斯特提出的。
I knew about the dual process theory, but you bring up the idea of production systems, an idea due to Emil Post.
我完全不知道那是什么，所以给我们讲讲吧。
And, I have no idea what's going on there, so tell us what is going on with that.
对。
Right.
现在我们不再讨论推理组织，而是进入更基础的层面，也就是计算组织。
So now we've moved away from inferential organization to an even more basic level, a kind of computational organization.
一开始我就说过，看，Transformer不是老式神经网络。
So at the very beginning, said, look, transformers, they're not grandpa's neural nets.
它们有很多结构。
They have a lot of structure.
它们的运作方式，我接下来会讲到，涉及逐层的变换，这与传统神经网络不同。
And the way that they operate, and I'm gonna be getting to it, involves a layer wise transformations that differ from traditional neural nets.
很多计算机科学领域的人对Transformer的运作方式非常熟悉，而且他们完全清楚导致这种架构出现的历史。
And many people in computer science would be very familiar with how the transformer operates and they know exactly the history that led to that.
但他们可能不知道，在认知科学领域，我们某种程度上早就经历过类似的情况。
What they may not know that in cognitive science, we've been there before, in a sense.
这就是整个生产系统。
And so that's the whole production system.
所以我会简单介绍一下。
So I'll say a little bit about that.
好的。
Okay.
生产系统是计算认知科学中一个非常重要的建模框架。
Production systems are a very important modeling framework in computational cognitive science.
它们被广泛使用。
And they're used extensively.
有大量的研究成果。
There's a huge body of results.
那么具体有哪些内容呢？
And so what exactly are there?
好的，一个很好的切入点就是大家熟悉普通数字计算机的工作原理，比如你桌面上用的电脑是怎么运行的。
Well, a good starting place is people are familiar with ordinary the way ordinary digital computers on your desktop, how they work.
它们通过顺序执行的指令和取指-执行周期来工作。
They operate with sequential instructions, a fetch execute cycle.
对。
Mhmm.
所以你知道，你收到一个指令，然后解码它，执行里面的内容，接着再处理序列里的下一条指令。
And so you know, you get an instruction, you decode it, you execute what it says, and then you move on to the next instruction in the sequence.
这就是生产系统，它是一个非常有用的通用计算框架。
Production systems, and that's a very potent general framework for computation.
生产系统的运作基于一种叫做‘识别-行动’循环的机制。
Production systems operate with something called a recognize act cycle.
具体来说，它的设置是这样的：有一个状态表示，也就是一种持久的记忆。
And so the way they would, things are set up is there is a state representation, a kind of persistent memory.
然后还有一组条件规则。
And then there is a population of conditionals.
这些条件规则的形式是：如果状态具备某些特征，就执行一个或多个动作。
And the form of those conditionals is if the state has so and so features, then do one or more actions.
在计算认知科学里，它们都是离散的。
And these, in computational cognitive science, they're all discrete.
我待会儿会稍微放宽一点这个限制。
And I'll loosen that a little bit later.
打个比方，你可以给机器人编程让它泡茶，直接告诉它每一步该怎么做：走进厨房、拿起水壶、把水装满。
And an analogy that might be helpful is you could program a robot to make tea by telling it exactly, walk into the kitchen, grab the kettle, fill it up, tell exactly what to do.
还有另一种做法。
And here's another way to do it.
如果水壶是空的，就把它装满。
If the kettle is empty, fill it.
明白吗？
Okay?
如果水是凉的，就烧开。
If the water is cold, boil it.
如果壶里的水是热的，就倒到茶上。
If the kettle water is hot, pour it on the tea.
所以它读取环境的状态，并执行这些条件中的一个或多个。
So it's reading in the state of the environment, and it's executing one or more of these conditionals.
有趣的是这种另一种做法。
And what's interesting is this other way of doing it.
这样你有一个状态，还有条件。
This you've got a state and you've got conditionals.
这看起来更自然。
It just seems more natural.
对。
Yeah.
它很稳健。
It's robust.
比如说我走进去，拿起水壶，往里面加冰块，水又变冷了。
So let's say I walk in there, I grab the kettle and I fill it with ice cubes and the water's cold again.
其实机器人知道该怎么做。
Actually the robot knows what to do.
它会判断，如果水壶里的水是冷的，就烧开它。
It's going, if the kettle water is cold, boil it.
对。
Right.
我编程的时候根本没想到会有人用冰块这一招。
I didn't even think when I programmed it that somebody was gonna do the ice cube trick.
但认知科学家注意到，取指令再执行这个框架根本不适合模拟人类的心智和大脑。
But cognitive scientists noticed that fetch execute is just not going to be a good framework for modeling the human mind brain.
于是他们转向了产生式系统。
And they gravitated towards the production system.
也就是状态表征加上一堆条件规则。
That is the state representation and a population of conditionals.
所以区别在于，一种是逐步执行的算法，做这个，做那个，另一种是与环境持续互动，决定下一步该做什么。
So the way that The distinction is between, like, just a step by step algorithm, do this, do this, do this, versus a continual give and take with the environment and what should I do next.
没错。
That's right.
但这里的环境是以状态表征的形式编码的，这是我们的主角——机器人——用来决定应用哪些条件规则的典型位置。
And and but the environment here is encoded in the form of a state representation, which is the canonical place where our hero, the robot, will be looking in order to know which conditionals to apply.
对。
Right.
而且，你知道，在很久以前的逻辑学或其他课程中，我们学到了不同的通用计算方法。
And, know, way back in logic or whatever, we learned these different approaches to universal computation.
早在1940年，埃米尔·波斯特——人们听说过图灵，但这种状态表征加条件规则群是另一种完整的框架，任何可计算函数都可以用这种方式表达。
And Emil Post way back in 1940, people have heard about Turing, but this state representation population of conditionals is another complete framework for any computable function can be expressed this way.
好。
Good.
但认知科学家非常喜欢这种生产框架，它成为了认知科学中的主导建模框架。
But cognitive scientists very much like this production framework, and it became the dominant modeling framework in cognitive science.
这个类比的关键在于，工作记忆就是存储状态的地方。
And the analogy is really with kind of working memory is where your state is stored.
而大脑长期记忆里存放的很多内容，就是大量这样的“如果-那么”规则。
And a lot of what your brain houses in long term memory is a lot of this population of if thens.
这些规则是你随着时间慢慢学到的。
And you have learned over time.
所以你可以用这个生产系统框架来模拟句子解析。
And so you can model sentence parsing in this production system framework.
事实上，我的合著者里克·刘易斯和他的前学生，他们有一个非常有影响力的生产系统模型，专门解释解析是如何进行的。
In fact, Rick Lewis, my co author on some of this work, and his former student, They have a very influential production system model of how parsing works.
这个模型能预测反应时间。
It predicts reaction time.
它能预测中心嵌入问题。
It predicts center embedding problems.
它还能预测相似性干扰效应。
It predicts similarity interference effects.
有关于工作记忆提取的理论。
There's theories of working memory retrieval.
有多任务处理的理论，所有这些都用这个生产系统框架来表达。
There's theories of multitasking, all expressed in this production system framework.
所以这里的要点是，这是一个强大且可能是认知科学中领先的有影响力的理论，认为心智以中央状态表征和大量条件规则运作。
So the the take home lesson there is it is a good, powerful, probably a leading influential theory in cognitive science that the mind operates with kind of a central state representation and massive populations of conditional rules.
这是对人脑计算组织的一个很好的近似。
The good approximation of the computational organization of the human mind brain.
好的。
Okay.
明白了。
Got that.
对。
Yeah.
我会补充一些名字，比如艾伦·纽厄尔和约翰·安德森。
And I'll add some names like Alan Newell and John Anderson.
如果观众想知道该去哪里了解这些学者的研究，可以看看他们的著作。
If your viewers wanna know where do we go read these folks.
《认知统一理论》，那是一本非常精彩的书。
Unified Theories of Cognition, beautiful book.
约翰·安德森的那本《心智如何在物理世界中产生？》
John Anderson's book, How Can the Mind Occur in the Physical World?
还有《心智的规则》，大家应该都听说过。
Rules of the Mind, you know.
这些内容大家都能查到。
These are read all about it.
这部分内容非常精彩。
This is great stuff.
好的。
All right.
那我们就先把这部分搞定。
So we got that in place.
与此同时，在城市的另一个角落，也就是计算机科学领域。
Meanwhile, far away in another part of town, this is in computer science.
这里稍微引用了一下鲍勃·迪伦，有兴趣的朋友应该能get到。
A little Bob Dylan reference there for anybody that's interested in that.
计算机科学家们正努力尝试构建更深层的神经网络。
Computer scientists are struggling trying to create the neural nets that will go deeper.
他们希望增加更多的中间层。
They want more intermediate layers.
他和他的同事在一篇著名的论文《ResNet '20 16》中发现，仅仅增加层数并不能带来显著的提升。
And he and his colleagues, in a famous paper called ResNet '20 16, they find that you're not getting very far, just putting more layers.
令人惊讶的是，不仅在测试数据上表现变差，在训练数据上的表现也变差了，这不仅仅是因为过拟合。
And shockingly, you're actually doing worse on the train data, not just on the test data because of overfitting.
他们在训练集上的表现确实变差了。
You're doing worse on train.
他们展示了一种改进的方法。
And they show there's a way to improve this.
通常，每一层的权重都会学习对前一层的变换。
Normally, each layer, the weights learn a transformation of the previous layer.
所以第l加一层会是某个函数f作用于l，也就是前一层。
So layer l plus one will be some function f of l, the previous layer.
你需要做的是把前一层的信息传递到下一层。
What you need to do is you need to propagate the previous layer up to the next one.
所以f(l加一)就等于x，也就是前一层。
So f of l plus one is going to be x, the previous layer.
而x就是前一层本身，加上某个关于前一层的函数。
And x is just what the, or l, the the previous layer, plus some function of the previous layer.
这迫使模型不要重新学习整个表示。
What that forces the model to do is not learn to rewrite the entire representation.
它会让那个状态在层与层之间持续存在。
It's going to let that state persist across layers.
而每一层要做的只是做一点小的修正，根据前一层出现的特征和表示来做一些小的调整。
And what each layer will do is make a small amendment, some small change that's conditional on what appear, what features are present, what representations are present at that previous layer.
好的。
Okay.
所以，持续传递的那个东西叫做残差流。
So that the thing that's persisting is called the residual stream.
好的。
Okay.
这是Transformer的关键设计组件之一。
And that's one of the key design components of the transformer.
顺便说一下，这篇ResNet论文的引用次数实际上比《Attention Is All You Need》还多，这两篇都是史上被引用最多的十篇论文之一。
So this ResNet paper, by the way, is actually cited more than attention is all you need, the 20 Both 17 of those are top 10 papers of all time.
是的。
Yeah.
Transformer论文还明确了一点，就是模型必须学习条件关系。
And the other thing that the transformer paper does is it makes explicit that the model is going to have to learn conditionals.
那它是怎么做到的呢？
And how does it do that?
大家可能熟悉KQV。
People may be familiar with KQV.
那是注意力机制的线性代数方程。
That's the linear algebra equation for attention.
它说的是某个叫查询的东西之间的匹配程度。
And what that's saying is that the degree of match between something called a query.
所以在每个词元上，你都有这个表示。
So at every token, you have this representation.
你把它映射到所谓的查询。
You map that into something called a query.
左边每个其他词元都会被映射到所谓的键。
Every other token to the left is gonna get mapped into something called a key.
你确定键和查询之间的匹配程度。
And you ascertain the degree of match between key and query.
那个数字是标量。
And that number is scalar.
你会把这个结果乘以另一个变量，叫做值（value），这个值是把左边每个token的位置映射过去。
You're gonna multiply that by another variable called a value, which is a mapping each token position to the left.
所以，根据键和查询之间的匹配程度，你会对值的贡献量进行放大或缩小。
So you're gonna scale up and down the amount the value gets added by the degree of match between key and query.
好的。
Okay.
我刚才说了什么呢？
What did I just say there?
我的意思是，如果键和查询的匹配程度达到那个水平，就把对应的值加进来。
I said, if key matches query to that extent, add value.
我用线性代数的术语把这个条件给形式化了。
I formulated in linear algebra terms a conditional.
这是一个渐进式的条件判断。
It's a graded conditional.
它不是那种‘有这个特征就加上另一个特征’的二元逻辑，认知科学家通常就是这么处理的。
It's not if this feature is present, then add this feature, which is how the cognitive scientists would do.
为什么要用分级条件呢？
Why do it as graded conditional?
嗯，这样表达力更强，而且可以通过梯度下降来学习。
Well, you get a lot more expressivity, but you can learn the thing via gradient descent.
这就是关键。
And that's the key.
如果你能用梯度下降大规模训练，就像预测那样，那你就成功了。
And if you can train something at scale with gradient descent, which you can with prediction, now you're off to the races.
你就有了一种类似生产系统的架构，在学习生产规则。
You've got a production system like architecture that's learning productions.
不仅仅是注意力机制这样运作。
It's not just attention that operates this way.
MLP单元，实际上模型中的大部分参数都在MLP单元里，它们有两个矩阵。
The MLP units, which are actually where most of the parameters in the model are in the MLP units, they have two matrices.
第一个矩阵的行包含特征。
And so the rows of the first matrices contain features.
如果它们和残差流匹配到这个程度，就把第二矩阵这一列里的特征加进去。
If they match the residual stream to that extent, you add the features in the column of the second matrix.
公式完全一样。
The exact same formulation.
说到底，Transformer 做的其实就是大量向量之间的点积运算，得到一个标量，也就是这两个向量的匹配程度，然后把这个值加上去。
Ultimately, what transformers are doing is tons and tons of dot products between two vectors, that's a scalar, that you add which is the degree of match between those two vectors.
对。
Yeah.
然后你再加上第三个向量。
And then you add the third vector.
这就是 Transformer 中 99% 的操作内容。
That's 99% of what happens in a transformer.
大部分情况都是这样。
A lot of that.
所以这就是那种生产系统（production system）的思路。
So that's this production system idea.
这些东西恰恰就是认知科学家们已经发现的东西。
These things are involved in exactly what the cognitive scientists had figured out.
这些人在城市的另一边独立地发现，你需要一个这样的结构。
These people independently in another part of town had figured out, you're gonna need a structure like this.
一个残差流和一组条件。
A residual stream and a population of conditions.
所以我的合著者里克，他实际上用那篇旧的ACT-R生产系统论文来研究transformer。
And so my co author, Rick, he actually looked at transformers using that old ACTR production system paper.
他研究的是，嘿，transformer是否预测了我们旧论文中得到的许多相同效应？
Looked at whether, hey, do transformers, do they predict a lot of these same effects that we got from our old paper?
果然，它们确实如此。
Sure enough, they do.
而且还有很多。
And on and on.
有各种各样的证据表明这些类似于生产系统。
There's there's various lines of evidence that these are production system like.
它们并非不可理解，至少在计算组织层面是这样。
They're not inscrutable, at least at the level of computational organization.
我们明白，你知道，这些东西所使用的策略。
We understand, you know, the the strategy being used in these things.
所以从多个方面来看，告诉我这是否夸张：LLM，真是好心，只是试图预测接下来会发生什么，却重新发明了人类常用的认知策略。
So in multiple ways, tell me whether this is an exaggeration or not, the LLMs, bless their hearts, just trying to predict what's gonna happen next, have reinvented strategies, cognitive strategies that are well used by human beings.
好的。
Okay.
这里有一个有效的双过程，这里，有一个小的修正。
Here in a dual process that works, here, there's a small amendment.
LLM是硬编码的，其架构中包含了残差流。
The LLM is hard coded, is architected with the residual stream.
这不是它学来的。
It didn't learn that.
好。
Good.
好。
Good.
事实上，我认为他和同事在ResNet论文中展示的是，至少在他们所研究的数据集规模下，模型是学不会的。
In fact, I think what he and colleagues showed in their ResNet paper is, at least with the dataset sizes they were looking at, it can't learn.
它卡住了。
It's stuck.
从原理上讲，它是可以学习的
In principle, it could learn
好的。
Sure.
必要的变换。
The needed transformations.
它本可以学会，你只需要做一点小调整，而不是重写整个内容。
It could learn that you need to make a little amendment rather than rewrite the whole thing.
但它只是卡在参数优化的空间里，根本学不会这一点。
It just gets stuck in parameter optimization space, and it just can't learn it.
好的。
Okay.
所以你们设计了一堆这样的东西。
So you architect a bunch of these things.
不过，这些计算机科学家在做这些设计选择时， convergence（收敛）是自然发生的，他们也是好心帮忙，并不是因为他们在盯着艾伦·纽厄尔和约翰·安德森的肩膀看。
But the convergence is happening with these computer scientists are making these design choices, God bless them, not because they're looking over the shoulder of Alan Newell and John Anderson.
他们根本不知道这些人是谁。
They don't know who those folks are.
他们其实是在重新发明艾伦·纽厄尔和约翰·安德森早就搞明白的东西。
They're just reinventing what Alan Newell and John Anderson figured out.
就是搞一套这样的状态表示，也就是条件句的集合。
Let's have this kind of state representation, population of conditionals.
他们 basically 是在重新发明这种策略。
They're kind of reinventing that strategy.
我是说，你之前多次提到预测的重要性，认为这是理解大语言模型运作方式的关键。
I mean, you've mentioned a lot the importance of prediction as thinking about what the LLM does.
也许这就是理解大脑运作的正确方式。
Maybe that's the right way to think about what brains do.
你知道，我们节目请过卡尔·弗里斯顿。
You know, we've had Carl Friston on the program.
是的。
Yes.
确实有一种学派认为，大脑的正确理解方式是作为预测机器，比如贝叶斯大脑、自由能原理等等。
There's this there's certainly a school of thought out there that says that the right way to think of the brain is as a prediction machine, the Bayesian brain, the free energy principle, things like that.
所以，也许你能给我们补充一些。
So, I mean, maybe fill in for us a little bit.
那是对实际情况的准确描述吗？
Is that an accurate, representation of what's going on?
有没有可能是别的样子？
Is could it have been different than that?
或者说，说我们是预测机器，我们能学到什么？
Or, you know, what are we learning by saying that we are prediction machines?
对。
Yeah.
这观点很棒。
That's great.
这有点像是借力打力，因为人们常说大语言模型不像我们，理由是你如果给某个东西投喂了数据，让它沉浸在预测之中，那它就是在做预测。
And it it it's a little bit of jujitsu because one of the things that people say of when they say LLMs are not like us is what you've done if you've tainted something and you've, you're bathing it in prediction.
比如基于互联网规模的数据来预测下一个词。
It's predict the next word on internet scale data.
这跟人们平时做的事情完全不一样。
And that's so unlike what people are, you know, what people are up to.
对。
Yeah.
而且，乍一听挺有道理的，但你要意识到，在认知科学领域，预测早就成为核心议题了，远在大语言模型出现之前就已经开始了。
And that's a, that, that seems to be plausible until you realize that in cognitive science, prediction has been moving to center stage way before LLMs you know, came on the scene.
所以你看，特里斯坦和哲学家安迪·克拉克就把他们的研究项目追溯到了19世纪末赫尔姆霍茨那里。
So know, Tristan and Andy Clark, the philosopher, they attribute their research program to Helmholtz, you know, in the late eighteen hundreds.
所以，认为心智在持续预测的想法，其实由来已久。
And so the idea that the mind is continuously predicting, it has an old history.
在感知领域，这些预测性层级编码模型非常有影响力。
And in perception, there's these predictive hierarchical coding models that are very influential.
在克拉克、赫尔姆霍茨和弗里斯顿的手中，他们将这些模型推广为基本的认知原则。
And in the hands of Clarke and Helmholtz, Friston, they generalize this to just core cognitive principles.
我们在感知和认知中普遍做的事情就是预测。
What we are doing ubiquitously in perception and cognition is predicting.
具体到语言方面，你知道，我一直在教认知科学导论，我们总是展示库迪斯和希利亚德1980年的研究。
In language specifically, you know I've been teaching Intro to Cognitive Science and we always show Kudis and Hilliard in 1980.
从2010年开始，我和里克·刘易斯就一直这样做。
Since 2010 we've been doing this, me and Rick Lewis.
你给人们一个句子，比如“他用热面包涂抹”，你可以预期是黄油，但那里的词是“袜子”。
And you give people a sentence like he spread the warm bread with, you can expect butter but the word there is socks.
那篇发表在《科学》杂志上的老论文显示，几百人的脑电图亮起红灯，你知道，就像警报一样。
And that old paper in Science shows, you know, a couple 100 people, their EEG is red alert, red alert, you know.
所以人们会追踪不一致之处，而他们这样做的方式就是看它是否偏离了预测。
And so people are tracking incongruity, and the way they do it is it's a deviation from what's predicted.
在2010年，我们并没有真正把预测视为语言中的普遍原则。
In 2010, we didn't really think of prediction as a ubiquitous principle in language.
然而，罗伯特·黑尔和罗杰·利维这些伟大的思想家却将其应用于语言领域。
Yet Robert Hale and Roger Levy and these great thinkers were installing it there in language.
然后其他人说这是认知科学中一个非常普遍的理论。
And then other people were saying it's a very general theory in cognitive science.
接着，大型语言模型出现了。
And then along come LLMs.
你知道，它们所做的预测就是，它们拥有所有这些参数。
And you know, what prediction what they're doing is they've got all these parameters.
它们做出预测。
They make a prediction.
而观察值与预测值之间的偏差，使它们能够通过链式法则、梯度下降等方法智能地调整参数。
And that allows them, the deviation between observed and predicted, allows them to make intelligent revisions to their parameters via chain rule gradient descent and things like that.
关于 Bayt 播客
Bayt 提供中文+原文双语音频和字幕，帮助你打破语言障碍，轻松听懂全球优质播客。