亚当·马尔布洛斯特 – 人工智能忽略了大脑的一些基本特性

我的意思是，首先，概率性AI领域的人会说，当然需要推理时的计算资源，因为这个推理问题非常困难。

I mean, first of all, I think the probabilistic AI people would be like, of course, you need test time compute because this inference problem is really hard.

而我们目前知道的唯一方法都需要大量的推理时计算资源。

And the only ways we know how to do it involve lots of test time compute.

否则，那只是一个糟糕的近似，永远无法实现——你得需要无限的数据才能做到这一点。

Otherwise, it's just a crappy approximation that's never gonna, like you have to do infinite data or something to, like, make this.

所以我认为，一些概率论者会说：不。

So I think that some of the probabilistic people will be like, no.

这本质上是概率性的，以这种方式进行摊销根本说不通。

It's, like, inherently probabilistic, and, like, amortizing it in this way, like, just doesn't make sense.

因此，他们可能会进一步指出大脑：好吧。

And so and they might then also point to the brain and say, okay.

大脑中的神经元具有一定的随机性，它们在采样，执行各种操作。

Well, the brain, the neurons are kind of stochastic, and they're sampling, and they're doing doing things.

所以也许大脑实际上进行的是更多非摊销的推理，即真正的推理。

And so maybe the brain actually is doing more like the non amortized inference, the real inference.

但感知能在几毫秒内完成，这也很奇怪。

But it's also kind of strange how perception can work in just, like, milliseconds or whatever.

看起来它并没有使用太多采样，所以显然它也在某种程度上把一些东西预计算到近似的前向传递中，以实现这一点。

It doesn't seem like it uses that much sampling, so it's also clearly also doing some kind of baking things into into, like, approximate forward passes or something like that to do this.

是的。

And yeah.

所以未来，你知道，我不确定。

So in the future, you know, I don't know.

我的意思是，某种程度上，已经出现一种趋势：人们之前需要在测试时计算的东西，现在正被用来反向训练基础模型。

I mean, I think is it already a trend to some degree that things that are people were having to use test time compute for are getting, like, used to train back the the base model.

对吧？

Right?

是的。

Yeah.

是的。

Yeah.

现在它可以在一次前向传播中完成。

That now it can do it in one pass.

对。

Right.

是的。

Yeah.

所以我的意思是，也许进化确实这样做了，也可能没有。

So I mean, I think, yeah, you know, maybe evolution did or didn't do that.

我认为进化仍然必须通过基因组来传递所有信息，以构建神经网络，对吧？

I think evolution still has to pass everything through the genome, right, to build the network.

而人类所处的环境是非常动态的。

So and the environment in which humans are living is very dynamic.

对吧？

Right?

所以，如果我们相信这个观点是正确的，即存在一个像史蒂夫·伯恩斯所说的学习子系统和一个引导子系统，那么这个学习子系统可能并没有太多的预初始化或预训练。

And so, maybe that's if we believe this is true, that that there's a learning subsystem per Steve Burns and a steering subsystem, that the the learning subsystem doesn't have a lot of, like, preinitialization or pretraining.

它具有某种架构，但在个体生命周期内会进行学习。

It has a certain architecture, but then within lifetime, it learns.

因此，进化并没有真正将太多东西固化到这个网络中。

Then evolution didn't, you know, actually, like, immortize that much into that network.

对。

Right.

相反，它将这些固化为一组本能行为和一组引导性成本函数，是的。

It immortized it instead into so set of innate behaviors in a set of these bootstrapping cost functions Yeah.

或者构建特定奖励信号的方式。

Or ways of building up very particular reward signals.

是的。

Yeah.

是的。

Yeah.

这个框架有助于解释人们指出并我曾问过几位嘉宾的谜题：如果你把进化类比为预训练，那你怎么解释基因组传递的信息如此之少这一事实？

This framework helps explain this mystery that people have pointed out and I've asked a few guests about, which is, if you want to analogize evolution to pre training, well, do you explain the fact that so little information is conveyed through the genome?

所以，三吉字节是整个人类基因组的大小。

So three gigabytes is the size of the total human genome.

显然，其中只有一小部分与大脑的编码真正相关。

Obviously a small fraction of that is actually relevant to coding at the brain.

是的。

Yeah.

如果以前人们将这个类比为：进化实际上找到了模型的超参数，也就是那些告诉你应该有多少层、基本架构的数字，对吧？

And if previously people made this analogy that actually pre, evolution has found the hyper parameters of the model, the numbers which tell you how many layers should there be, the architecture basically, right?

比如，各个部分应该如何连接？

Like how should things be wired together?

但如果说，提高样本效率、促进学习、普遍提升系统性能的一个重要部分是奖励函数或损失函数的话，

But if a big part of the story that increases sample efficiency, aids learning, generally makes systems more performant is the reward function is the loss function.

是的。

Yeah.

如果进化找到了这些促进学习的损失函数，那么这就很好地解释了，为什么可以用如此少的信息构建出智能，因为奖励函数本身就很关键。

And if evolution found those loss functions, which aid learning, then it actually kind of makes sense how, so you can like build an intelligence with so little information because, like, the reward function hey.

你在Python里就是这样。

You're, like, right in Python.

对吧？

Right?

奖励函数实际上就是一行代码。

The reward function is, like, literally a line.

是的。

Yes.

所以你只是有上千行这样的代码，但它们并不占多少空间。

And so you just, have, a thousand lines like this, and that's doesn't take up that much space.

是的。

Yes.

而且它还能实现我之前提到的泛化功能，比如我们讨论过的蜘蛛，它能学会仅仅看到“蜘蛛”这个词就会触发蜘蛛的反射之类的行为。

And it also gets to do this generalization thing with the the thing I the thing was describing where we were talking with about the spider, right, of where it learns that just the word spider, you know, triggers the spider, you know, reflex or whatever.

它也能利用这一点。

It gets to exploit that too.

对吧？

Right?

所以它能够构建一个奖励函数，这个函数通过指定这些天生的蜘蛛相关机制和史蒂夫所说的评估器来实现大量泛化，这些评估器负责学习。

So it gets to build a reward function that actually has a bunch of generalization in it just by specifying these innate spider stuff and the thought assessors, as Steve calls them, that do the learning.

因此，这可能是构建你所需的更复杂奖励函数的一种非常紧凑的解决方案。

So that's, like, potentially a really compact solution to building up these more complex reward functions too that you need.

所以它不需要预判奖励函数未来的所有细节，只需预判哪些变量是相关的，以及哪些启发式方法能帮助找到这些变量。

So it doesn't have to anticipate everything about the future of the reward function, just anticipate what variables are relevant and what are heuristics for finding what those variables are.

然后，是的。

And then yeah.

因此，它必须为学习算法和学习子系统的基础架构提供非常紧凑的规范。

So then it has to have a very compact specification for the learning algorithm and basic architecture of the learning subsystem.

然后它还必须指定所有这些Python代码，比如关于蜘蛛的一切、关于朋友的一切、关于你母亲的一切、关于相遇的一切、关于社交群体的一切、关于共同眼神交流的一切。

And then it has to specify all this Python code of, like, all the stuff about the spiders, and all the stuff about friends, and all the stuff about your mother, and all the stuff about meeting, and and social groups, and joint eye contact.

它必须指定所有这些东西。

It has to specify all that stuff.

那么，这真的正确吗？

And so, is this really true?

所以，我认为有一些证据支持这一点。

And so, I think that there is some evidence for it.

因此，费·陈、埃文·马科斯科以及许多其他研究人员一直在进行这些单细胞图谱研究。

So so, Fay Chen and and Evan Macosko and various other researchers who have been doing, like, these single cell atlases.

神经科学技术的扩展——这又是我众多痴迷之一——通过脑计划这一大型神经科学资助项目，已经对大脑的不同区域，尤其是小鼠大脑，进行了测绘，以确定不同细胞类型的位置。

So, one of the things that neuroscience technology or scaling up neuroscience technology, again, this is kind of like my one of my obsessions, has done through through the brain initiative, a big, you know, neuroscience funding program is they've basically gone through different areas, especially the mouse brain, and mapped where are the different cell types.

皮层不同区域中存在多少种不同类型的细胞？

How many different types of cells are there in different areas of cortex?

这些细胞类型在不同区域之间是否相同？

Are they the same across different areas?

然后你再观察这些皮层下区域，它们更像是转向子系统或奖励功能生成区域。

And then you you look at these subcortical regions, are more like the steering subsystem or reward function generating regions.

它们有多少种不同类型的细胞？又有哪些神经元类型？

How many different types of cells do they have, and which neurons types do they have?

我们不知道它们是如何全部连接的，也不清楚它们具体做什么、电路是什么、意味着什么，但你可以通过测序RNA来量化到底有多少种不同类型的细胞。

We don't know how they're all connected and exactly what they do or what the circuits are or what they mean, but you can just, like, quantify, like, how many different kinds of cells are there with sequencing the RNA.

在控制子系统中，存在大量更奇怪、更多样化、更独特的细胞类型，远多于学习子系统中的。

And there are a lot more weird and diverse and bespoke cell types in the steering subsystem, basically, than there are in the learning subsystem.

比如皮层细胞类型，似乎已经足够多，足以在那里构建一个学习算法并指定一些超参数。

Like, the cortical cell types, there's enough to build it seems like there's enough to build a learning algorithm up there and specify some hyperparameters.

而在控制子系统中，有成千上万种极其奇特的细胞，可能有一种是负责蜘蛛惊跳反射的，有一种是负责‘我即将尝到咸味’的，还有

And in the in the steering subsystem, there's, like, a gazillion, you know, thousands of really weird cells, which might be, like, the one for the spider flinch reflex and the one for I'm about to taste salt and the

那为什么每个奖励函数都需要一种不同的细胞类型？

one So why would each reward function need a different cell type?

这就要说到天生固有的神经回路了。

Well, so this is where you get innately wired circuits.

对吧？

Right?

在学习算法部分，也就是学习子系统中，你设定初始架构，学习算法的所有‘机制’都通过突触的可塑性实现，即这个庞大网络中突触的变化，但它的初始化结构是相对重复的。

So in the in the learning algorithm part, in this in the learning learning subsystem, you set up that's why the initial architecture, you specify a learning algorithm is all all the all the all the juices is happening through plasticity of the synapses, changes of the synapses within that big network, but it's kind of like a relatively repeating architecture, how it's initialized.

编写一个八层Transformer所需的Python代码量，和编写一个三层Transformer的差别并不大。

It's just like the amount of Python code needed to make, you know, a eight layer transformer is not that different from one to make it a three layer transformer.

对吧？

Right?

你只是在重复复制。

You're just replicating.

没错。

Yeah.

但所有这些用于奖励函数的Python代码，比如当一个优越的点击列表看到某个东西在爬行并落地时，你皮肤上会起鸡皮疙瘩之类的，就会触发蜘蛛反射，这其实就是一堆特定物种、特定情境的杂乱代码，根本不是通用的。

Whereas, all this Python code for the reward function, you know, if superior click list sees something that's skittering and it lands, you know, you're feeling goosebumps on your skin or whatever, then trigger spider reflex, that's just a bunch of, like, bespoke species specific situation specific crap that no.

大脑皮层并不知道蜘蛛，它只知道层级结构，没错。

The cortex doesn't know about spiders, it just knows about layers and Right.

还有学习。

And learning.

要写出这样的奖励函数，唯一的办法就是。

The the only way to have this, like, write this reward function Yeah.

需要一种特殊的细胞类型。

Is to have a special cell type.

是的。

Yeah.

好的。

Okay.

是的。

Yeah.

我认为是的。

Well, I think so.

我认为你必须拥有特殊的细胞类型，或者必须有其他特殊的连接规则，让进化能够规定：这个神经元需要与那个神经元连接，而无需任何学习。

I think you either have to have a special cell types or you have to otherwise get special wiring rules that evolution can say, this neuron needs to wire to this neuron without any learning.

而最有可能实现这一点的方式，我认为是这些细胞表达出不同的受体和蛋白质，比如说：好吧。

And the way that that is most likely to happen, I think, is that those cells express, like, different receptors and proteins that say, okay.

当这个细胞与那个细胞接触时，就形成一个突触。

When this one comes in contact with this one, let's form a synapse.

所以这是基因层面的布线。

This So it's genetic wiring.

是的。

Yeah.

实现这一点需要不同的细胞类型。

And those need cell types to do it.

是的。

Yeah.

我确信，如果我懂一点神经科学的话，这会更容易理解，但看起来转向子系统中仍然存在很多复杂性或普遍性。

I'm sure this would make a lot more sense if I knew one zero one neuroscience, but, like, it seems like there's still a lot of complexity or generality rather in the steering subsystem.

转向子系统有自己的视觉系统，与视觉皮层是分开的。

So in the steering subsystem has its own visual, system that's separate from the visual cortex.

是的。

Yeah.

不同的

Different

这些特征仍然需要接入那个视觉系统，比如蜘蛛机制需要接入，爱的机制也需要接入，等等。

features still need to plug into that vision system in the so like the spider thing needs to plug into it and also the, the, love thing needs to plug into it, etcetera, etcetera.

是的。

Yes.

所以这看起来很复杂。

So it seems complicated.

就像，我

Like, I

我知道这仍然很复杂，这恰恰说明基因组中大量基因组区域，以及不同细胞类型等，都用于构建转向系统的布线。

know it's still complicated, that's that's all the more reason why a lot of the genomic, you know, real estates in the genome and in terms of these different cell types and so on would go into wiring up the steering subsystem.

我们能知道预布线吗？我们能

And can we tell Prewiring Can we

能判断有多少基因组是明显在起作用的吗？

tell how much of the genome is like clearly working?

所以我想你可以判断有多少基因与产生RNA或在大脑不同细胞类型中表现出来的表观遗传有关，对吧？

So I guess you could tell how many are relevant to the producing the RNA that manifest or the epigenetics that manifest in different cell types in the brain, right?

是的，这正是细胞类型帮助你理解的地方。

Yeah, this is what the cell types helps you get at it.

我不认为这就像说，基因组中有百分之几在做这件事。

I don't think it's exactly like, oh, this percent of the genome is doing this.

但你可以说，好吧。

But you could say, okay.

在这些所有转向子系统的亚型中，有多少不同的基因参与了指定它们各自的身份以及它们如何连接，这些基因占用了多少基因组区域，而那些指定视觉皮层与听觉皮层的基因又占了多少，你实际上只是重复使用相同的基因来做同样的事情两次。

In these all these steering subsystem subtypes, you know, how many different genes are involved in sort of specifying which is which and how they wire, and how much genomic real estate do those genes take up versus the ones that specify, you know, visual cortex versus audio auditory cortex, you kinda are just reusing the same genes to do the same thing twice.

而蜘蛛反射的连接，是的。

Whereas the spider reflex hooking up yes.

你说得对。

You're right.

它们必须构建自己的视觉系统、听觉系统、触觉系统和导航系统。

They have to they have to build their vision system, they have to build some auditory systems and touch systems and navigation type systems.

所以，即使是输入海马体之类的地方，也有头朝向细胞。

So, you know, even feeding into the hippocampus and stuff like that, there's head direction cells.

即使是果蝇的大脑，也具有先天的神经回路，是的。

Even the fly brain, it has innate circuits Yeah.

它能确定自己的方向，帮助它在环境中导航，利用视觉感知飞行时的光流，以及它的飞行与风向之间的关系。

That, you know, figure out its orientation and help it navigate in the world, and it uses vision, figure out its optical flow of of how it's flying, and, you know, how is it how is its flight related to the wind direction.

它拥有所有这些先天机制，我认为在哺乳动物大脑中，我们都会把这些归入转向子系统。

It has all these innate stuff that I think we in the mammal brain, we would all put that and lump that into the steering subsystem.

所以有很多工作要做。

So there's a lot of work.

因此，所有决定果蝇所需各种功能的基因，我们在转向子系统中也会有类似的东西。

So all the genes basically that go into specifying all the things a fly has to do, we're gonna have stuff like that too just all in the steering subsystem.

但我们有没有大概的估计，比如，指定这些功能需要多少个核苷酸，多少兆碱基？

And but do we do we have some estimate of, like, here's how many nucleotides, here are many megabases it takes to

我不知道。

I I don't know.

我的意思是，也许你可以去问问生物学家，某种程度上可以，因为可以说，我们和它们有很多共同之处。

I mean, but but but, I mean, I think peep you might be able to talk to biologists about this, you know, to to some degree because you can say, well, we just have a ton in common.

我的意思是，从基因的角度来看，我们和酵母有很多共同之处。

I mean, we have a lot in common with yeast from a genes perspective.

酵母至今仍被用作模式生物，没错。

Yeast is still used as a model Yeah.

用于生物学中的某些药物开发等工作。

For, you know, some amount of drug development and stuff like that in biology.

因此，基因组的大部分都只是用于维持细胞的基本存在。

And so so much of the genome is just going towards you have a cell at all.

它能回收废物。

It can recycle waste.

它能获取能量。

It can get energy.

它能复制自身。

It can replicate.

然后你就会看到我们和小鼠的共同点。

And then it then you see what we have in common with a mouse.

所以我们确实知道，在某种程度上，我们和黑猩猩之间的差异，包括社会本能以及大脑皮层等更高级的差异。

And so we we do know at some level that, you know, the difference is us and a chimpanzee or something, and that includes the social instincts and the more advanced, you know, differences in cortex and so on.

实现这些额外功能——比如从六层变换器升级到八层变换器，或调整奖励函数——所需的基因数量其实非常少。

It's it's a it's a tiny number of genes that go into these additional amount of making the eight layer transformer instead of the six layer transformer or tweaking that reward function.

是的。

Yeah.

这有助于解释为什么人科动物的大脑尺寸会如此迅速地膨胀，据我理解，这大概是因为社会学习或其他某种机制提升了我们从环境中学习的能力，也就是提高了我们的样本效率，对吧？

This would help explain why the hominid brain exploded in size so fast, which is presumably like, tell me this is correct, but under the story we, social learning or some other thing increased the ability to learn from the environment, like increased our sample efficiency, right?

不用非得自己去猎杀野猪、摸索怎么干，只需要听长辈说：‘这是做矛的方法。’

Instead of having to go and kill the boar yourself and figure out like how to do that, can just be like, the elder told me this is how you make a spear.

这样一来，就有更强的动力去发展更大的大脑皮层，以便能够学习这些技能。

And then now it increases the incentive to have a bigger cortex, which can like learn these things.

这可以通过相对较少的基因实现，因为本质上只是复制了老鼠已有的功能，并加以扩展。

And that can be done with a relatively few genes because it's really it's really replicating what the mouse already has is making more of it.

可能并不完全相同，或许有些微调，但从整体上看，你不需要从头发明一切。

And it's maybe not exactly the same, and there may be tweaks, but it's like from a perspective, you don't have to reinvent Right.

所有这些东西。

All this stuff.

对吧？

Right?

那么，大脑皮层在进化史上可以追溯到多久以前？

And so then how far back in the history of the evolution of the brain does the cortex go back?

是不是说，大脑皮层一直都在解决这种全方位推理的问题，这已经是一个长期存在的难题了？

And is the idea that like the cortex has always figured out this omnidirectional inference thing that that's been a solve problem for a long time.

灵长类动物的重大突破在于我们获得了奖励机制，从而提高了全方位推理的回报，还是说，大脑皮层的全方位推理功能本身也需要很长时间才能解锁？

And then with the big unlock with primates is this, we got the reward function which increased the returns to having omnidirectional inference or is the is cortex the omnidirectional inference also something that took a while to unlock?

我不确定在这方面是否达成共识。

I'm not sure that there's agreement about that.

我认为关于语言可能存在一些具体问题，比如是否通过听觉和记忆，或者听觉与记忆区域的某种组合进行了调整。

I think there might be specific questions about language, you know, are there tweaks to be whether that's through auditory and memory, some combination of auditory memory regions.

可能还需要对听觉区域与记忆区域，以及这些社会本能进行宏观上的连接。

There may also be macro wiring of you need to wire auditory regions into memory regions or something like that and into some of these social instincts to get

我明白了。

I see.

比如，语言要发生。

Language, for example, to happen.

所以可能只需要很少的基因变化

So there might be but that might be also a small number of gene changes

嗯。

Yep.

就能实现从我的颞叶到这里到听觉皮层的连接。

To be able to say, oh, I just need from my temporal lobe over here going over to the auditory cortex something.

对吧？

Right?

有一些证据表明，布洛卡区、韦尼克区。

And there is some evidence for the, you know, De Broca's area, Wernicke's area.

它们与海马体等区域相连。

They're connected with this hippocampus and so on.

还有前额叶皮层。

And so prefrontal cortex.

所以可能只有少数几个基因，使得人类能够真正很好地进行语言活动。

So there's, like, some small number of genes maybe for, like, enabling humans to really properly do language.

那可能是一个关键的基因。

That could be a big one.

但没错。

But yeah.

我的意思是，我认为是大脑皮层发生了某种变化，才使得这些能力成为可能？

I mean, I think that is it that something changed about the cortex, and it became possible to do these things?

还是说这种潜力原本就存在，只是缺乏推动其扩展并将其与社交本能连接起来使用的动力，从而更充分地利用它？

Or is it that potential was already there, but there wasn't the incentive to expand that capability and then use it, wire it to these social instincts, and and use it more.

嗯。

Mhmm.

我的观点更倾向于后者。

I mean, I would lean somewhat toward the latter.

我的意思是，我认为小鼠的大脑皮层与人类有很多相似之处。

I mean, I think a mouse has a lot of similar similarity in terms of cortex as a human.

对。

Right.

不过，苏泽特和赫库拉·胡萨尔的研究表明，灵长类动物的大脑神经元数量与体重的关联性比啮齿类动物更好。

Although there's that, the Suzette and Hercula Hussal work of the the, the the number of neurons scales better with weight with primate brains than it does with rodent brains.

对吧？

Right?

所以，是的。

So Yeah.

这是否表明大脑皮层的可扩展性实际上有所提升？

Does that suggest that there actually was some improvement in the scalability of the cortex?

也许吧。

Maybe.

也许吧。

Maybe.

我对这个了解得不是特别深入。

I'm not I'm not super deep on this.

可能确实存在一些架构上的变化、折叠方式的改变、神经元特性的调整等等，这些都略微影响了这一点，但仍然存在某种规模效应。

There may there may have been, yeah, changes in architecture, changes in the folding, changes in neuron properties and stuff that that somehow slightly tweak this, but there's still a scaling.

没错。

That's right.

没错。

That's right.

无论如何。

Either way.

对吧？

Right?

我并不是说人类在学习子系统的架构上没有任何特别之处。

And so I I was not saying there aren't something special about humans in the architecture of the learning subsystem at all.

但确实，我认为普遍认为这一点得到了扩展，但接下来的问题是，好吧。

But, yeah, I mean, it's I think it's pretty widely thought that this has expanded, but then the question is, okay.

那么，这如何与引导子系统的改变以及利用这种机制并使其有效启动的本能相契合呢？

Well, how does that fit in also with the steering subsystem changes and the instincts that make use of this and allow you to bootstrap using this effectively?

但话说回来，我再补充几点。

But, yeah, I mean, just to say a few other things.

我的意思是，即使是苍蝇的大脑，也具备某种程度的——比如，即使是非常早期的生物，我想你读过那本很棒的书《智力简史》。

Mean, so even the fly brain has some amount of, for example, even even very far back I mean, I think you've read this this great book, The Brief History of Intelligence.

对吧？

Right?

我觉得这本书真的很好。

I think this is a really good book.

很多人工智能研究人员认为这本书非常出色，看起来是这样。

Lots of AI researchers think this is a really good book, it seems like.

是的。

Yeah.

基本上，任何拥有大脑的生物，都具备一定程度的学习能力。

You have some amount of learning going back all the way to anything that has a brain, basically.

你至少可以追溯到脊椎动物，它们拥有一种类似原始强化学习的东西。

You have something kind of like primitive reinforcement learning, at least, going back at least to, like, vertebrates.

比如，想象一条斑马鱼。

Like, imagine, like, a zebrafish.

这些其他的分支。

Kind of these other branches.

鸟类可能重新发明了某种类似皮层的东西，但没有六层结构。

Birds maybe kind of reinvented something kind of cortex like, but it doesn't have the six layers.

嗯。

Mhmm.

但它们拥有一些类似皮层的结构，因此，在爬行动物之后，鸟类和哺乳动物都各自发展出了某种类似皮层的结构，只是组织方式不同。

But they have something a little bit cortex like, So that that some of those things, after reptiles, in some sense, birds and mammals, both kind of made us up somewhat cortex like, but differently organized thing.

但即使是苍蝇的大脑，也有联想学习中心，能够实现类似贝伦斯提出的‘思维评估者’概念的功能——通过特定的多巴胺信号，训练果蝇蘑菇体中的特定神经元群，将不同的感官信息与‘我现在会得到食物吗？还是会受伤吗？’联系起来。

But even a fly brain has associative learning centers that actually do things that maybe look a little bit like this thought assessor concept from Behrens, where there's a specific dopamine signal to train specific subgroups of neurons in the fly mushroom body to associate different sensory information with, am I gonna get food now or am I gonna get hurt now?

是的。

Yeah.

稍微跑题一下。

Brief tangent.

我记得读过达伦·米德盖写的一篇博客文章，说大脑皮层中与听觉和视觉相关的部分，在其他灵长类动物和人类之间出现了不成比例的扩张，而与嗅觉相关的部分则没有。

I remember reading in, one blog post that Darren Milledge wrote that the parts of the cortex, are associated with audio and vision have scaled disproportionately between other primates and humans, whereas the parts associated say with odor have not.

我记得他说过，这种现象是因为这类数据的规模定律特性较差。

And I remember him saying something like, this is explained by that kind of data having worse scaling law properties.

但我认为，也许他本意如此，但另一种解释是：这些嵌入在前额叶系统中的社会奖励功能，需要更多地利用观察长辈、观察视觉线索和聆听他们说话的能力。

But I think the, and maybe he meant this, but another interpretation of actually what's happening there is that these social reward functions that are built into the Syrieg subsystem needed to make use more of being able to see your elders and see what the visual cues are and hear what they're saying.

为了理解这些引导学习的线索，你需要更多地激活视觉和听觉系统，是的。

In order to make a sense of these cues, which guide learning, you needed to activate these Yeah.

更多地激活视觉和听觉系统，而不是

Activate the vision and audio more than

我的意思是，还有很多这样的事情。

I mean, there's all this stuff.

我觉得在你的节目里之前就提到过，比如人类眼睛的设计——有瞳孔、眼白等等，我们天生就被设计成可以通过共同的眼神接触来建立关系。

I feel like it's come up in in your your shows before actually, but, like, even like, the design of the human eye where you have the pupil and the white and everything, like, we are designed to be able to establish relationships based on joint eye contact.

也许这在《Sudden》那一集中提到过，我不太记得了。

And and maybe this came up in the Sudden episode, I can't remember.

但没错。

But yeah.

我们必须从零开始，直到能够检测到眼神接触并通过语言进行交流。

We're we we have to bootstrap to the point where we can detect eye contact and where we can communicate by language.

对吧？

Right?

而生命的头几年正是在努力实现这一点。

And that's what the first couple years of life are are trying to do.

没错。

Yeah.

好的。

Okay.

我想问你关于强化学习的问题。

I wanna ask you about RL.

目前，这些LN的训练方式是，如果它们解决了单元测试或数学题，那么整个轨迹中的每一个标记都会被加权提升。

So currently the way these LNs are trained, you know, they are, if if they solve the unit test or solve a math problem, that whole trajectory, every token in that trajectory is up weighted.

那人类的情况是怎样的呢？

And what's going on with humans?

大脑的不同区域是否存在不同类型的基于模型和无模型的机制？

Is there are there different types of model based versus model free that are happening in different parts of the brain?

是的。

Yeah.

我的意思是，这又是另一件事。

I mean, this is this is another one of these things.

我的所有回答，对于这些问题，我所说的任何具体事情都只是大致方向上的，我们可以围绕这个方向进行探索。

I mean, again, all my answers to these questions any specific thing I say is all just kind of, like, directionally, this is we can kind of explore around this.

我觉得这很有趣。

I find this interesting.

我觉得文献在某种程度上指向了这些方向。

Maybe the lit I feel like the literature points in these directions in some very broad way.

我真正想做的是，去绘制整个小鼠大脑的图谱，全面地弄清楚这个问题，让神经科学成为基础科学。

What I actually wanna do is, like, go and map the entire mouse brain and, like, figure this out comprehensively and, like, make neuroscience the ground truth science.

所以，说实话，我不知道。

So I don't know, basically.

但确实，我的意思是，我和伊利亚在播客里讨论时，他说，你不使用价值函数，这很奇怪。

But but, yeah, I mean, there so first of all, I mean, I think with Ilya on the podcast, I mean, he was like, it's weird that you don't use value functions.

对。

Right.

对吧？

Right?

你们用的是一种最原始的强化学习形式，当然，这些人都极其聪明，他们优化的是如何在GPU上实现，他们取得的成就确实令人惊叹。

You use, like, the most dumbest form of RL based on of course, there are these people are incredibly smart, and they're optimizing for how to do it on GPUs, and it's really incredible what they're achieving.

但从概念上讲，这种强化学习方式甚至比十年前的做法还要原始。

But, like, conceptually, it's a really dumb form of RL even compared to, like, what was being done in, like, ten years ago.

对吧？

Right?

就像，即使是玩雅达利游戏的那些东西，对吧，用的是Q学习，本质上就是一种时序差分学习，没错。

Like, even, you know, the Atari game playing stuff, right, was using, like, Q learning, which is basically like it's a kind of temporal difference Yep.

学习。

Learning.

对吧？

Right?

而时序差分学习基本上意味着，你有一个某种价值函数，它告诉你现在选择的动作不仅仅会立即产生结果。

The And temporal difference learning basically means you have some kind of a value function of, like, what action I choose now doesn't just tell me literally what happens immediately after this.

它告诉你，从预期的总奖励之类的角度来看，这个动作的长期后果是什么。

It tells me, like, what is the long run consequence of that from my expected, you know, total reward or something like that.

所以，像我们现在的大型语言模型中完全不使用价值函数，这简直太疯狂了。

And so, you would have value functions like the fact that we don't have, like, value functions at all is, like, in the LLMs is, like it's crazy.

我的意思是，因为Ilya说过，所以我现在可以说出来。

I mean, I I think I think because Ilya said it, I I can say it.

我知道自己对AI的理解只有他的百分之一，但这种东西居然能奏效，确实有点疯狂。

I know, you know, one one hundredth of what he does about AI, but, like, it's kinda crazy that this is working.

是的。

Yeah.

但话说回来，就大脑而言，我认为大脑中有一些部分被认为在做类似于无模型强化学习的事情。

But, yeah, I mean, in terms of the brain, well so I think there are some parts of the brain that are thought to do something that's very much like model free RL.

这主要是基底神经节、纹状体和基底神经节这些区域。

That's sort of parts of the basal ganglia, sort of striatum and basal ganglia.

它们具有某种有限的、相对较小的动作空间，这是人们普遍认为的。

They have, like, a a certain finite like, it is thought that they have a certain, like, finite relatively small action space.

它们可能采取的动作类型，首先可能是告诉脊髓或脑干和脊髓执行某个运动动作。

And the types of actions they could take, first of all, might be, like, tell the spinal cord or tell the brain stem and spinal cord to do this motor action.

是的，不是。

Yes, no.

或者可能是更复杂的认知类动作，比如告诉丘脑允许大脑皮层的这一部分与另一部分交流，或者释放海马体中的记忆并启动一个新的记忆之类的。

Or it might be more complicated cognitive type actions, like tell the thalamus to allow this part of the cortex to talk to this other part or release the memory that's in the hippocampus and start a new one or something.

对吧？

Right?

确实存在一组有限的动作，这些动作源自基底神经节，这是一种非常简单的强化学习。

There is but there's some finite set of actions that kinda come out of the basal ganglia, and that it's just a very simple RL.

因此，我们大脑中可能还有其他部分正在执行非常简单、原始的强化学习算法。

So there are probably parts of other brains in our brain that are just, like, doing very simple naive type RL algorithms.

此外，神经科学领域的一些重要工作，比如彼得·戴安的研究，以及大量促使我認為深層動態學習最初會發展時間差分學習的原因，都是因為他們對神經科學非常感興趣。

Layer one thing on top of that is that some of the major work in neuroscience, like Peter Diane's work and a bunch bunch of work that is part of why I think DeepMind did the temporal difference learning stuff in the first place, is they were very interested in neuroscience.

而且有大量的神經科學證據表明，多巴胺傳遞的是獎勵預測誤差信號，而不是直接的獎勵，也不是幾百萬個時間步之後的獎勵。

And there's a lot of neuroscience evidence that the dopamine is giving this reward prediction error signal, rather than just reward, yes, no, you know, a gazillion time steps in the future.

這是一種預測誤差，這與學習這些價值函數是一致的。

It's a prediction error, And that's consistent with learning these value functions.

所以這一點是存在的。

So there's that.

然後可能還有一些更高階的東西。

And then there's maybe higher order stuff.

我們的大腦皮層正在構建一個世界模型。

So we have these cortex making this world model.

皮层世界模型可以包含的一个内容是，你何时会获得奖励、何时不会获得奖励的模型。

Well, one of the things the cortex world model can contain is a model of when you do and don't get rewards.

对吧？

Right?

它再次预测的是转向子系统会做什么。

Again, it's predicting what the steering subsystem will do.

它也可能预测基底神经节会做什么。

It could be predicting what the basal ganglia will do.

因此，你的皮层中有一个模型，它具有更强的泛化能力和更多概念，能够说明：在这些类型的环境中，这些类型的计划和行动会带来奖励。

And so you have a model in your cortex that has more generalization and more concepts and all this stuff that says, okay, these types of plans, these types of actions will lead in these types of circumstances to reward.

所以我有一个关于我自身奖励的模型。

So I have a model of my reward.

有些人还认为，你可以反过来思考。

Some people also think that you can go the other way.

因此，这是推理图景的一部分。

And so this is part of the inference picture.

有一种观点认为强化学习就是推理。

There's this idea of RL as inference.

你可以说，假设我的奖励很高，那么采样一个能达成这种结果的计划。

You could say, well, conditional on my having a high reward, sample a plan that I would have had to get there.

这是从奖励部分推断计划部分。

That's inference of the plan part from the reward part.

我把奖励固定为高值，然后推断出确实如此。

I'm clamping the reward as high and inferring Yeah.

从可能导向该结果的计划中进行采样。

The plan sampling from plans that could lead to that.

所以，如果你拥有这种非常通用的皮层系统，以及一个包含计划和奖励等要素的通用模型，那么你基本上就能自动获得这种能力。

And so if you have this very general cortical thing, it can just do if you have this, like, general very general model based system, and the model, among other things, includes plans and rewards, then you just get it for free, basically.

所以，

So,

换句话说，在神经网络术语中，有一个价值头与发生在

like, in neural network parlance, there's a value head associated to the the the omnidirectional inference that's happening in

是的。

Yes.

嗯，对。

The Yeah.

或者有一个价值输入。

Or there's a value input.

嗯。

Yeah.

哦，明白了。

Oh, okay.

嗯。

Yeah.

它能够预测其中一个几乎属于感觉的变量，也就是将会获得的奖励。

And it and it it can predict one of the one of the almost sensory variables it can predict is is what rewards is gonna get.

嗯。

Yeah.

不过顺便说一下，说到这种摊销的事情，没错，价值就像是摊销的滚动预测

But by the way, speaking of this thing about amortizing things, yeah, obviously, value is like amortized rollouts

查找奖励。

of looking up reward.

没错。

Yeah.

类似这样的东西。

Something like that.

没错。

Yeah.

没错。

Yeah.

它就像是对它的统计平均值或预测。

It's like a statistical average or prediction of it.

没错。

Yeah.

对。

Right.

离题了。

Tangential thought.

你知道吗，乔、亨里克和其他人有一个观点，认为人类社会学会做事的方式，就像是如何发现某种几乎总是有毒的豆子，只要经过一个极其复杂的十步过程，就变得可以食用。

You know, Joe, Henrik and others have this idea that the way human societies have learned to do things is just like, how do you figure out that, you know, this kind of bean, which actually just almost always poisons you is edible if you do this 10 step incredibly complicated process.

这十个步骤中的任何一个步骤如果做错了，豆子就会有毒。

Any one of which, if you fail at, the bean will be poisonous.

嗯。

Uh-huh.

你是怎么知道在一年中的这个时候，用这种特定的武器，以这种方式捕猎海豹的呢？

How do you figure out how to hunt the seal in this particular way with this, like, particular weapon at this particular time of the year, etcetera?

除了代代相传地不断尝试，别无他法。

There's no way but, just like trying shit over generations.

这让我想到，这实际上非常类似于在文明层面上发生的无模型强化学习。

And it strikes me that this is actually very much like model free RL happening at, a civilizational level.

不。

No.

不完全是。

Not exactly.

我的意思是，进化是

I mean, Evolution is

在某种意义上是最简单的算法。

the simplest algorithm in some sense.

对吧？

Right?

如果我们相信这一切都来自进化，那么外层循环可以是非常缺乏远见的，没错。

And if we believe that all of this can come from evolution, like, the outer loop can be, like, extremely not foresighted and yeah.

对。

Right.

是的。

Yeah.

这很有趣。

That that that's interesting.

就像是，进化模型的层次结构，用于文化进化模型，用于

Just like, hierarchies of evolution model for a culture evolution model for a

那这说明了什么？

So what does that tell you?

也许这意味着，只要你反复去做，简单的算法就能让你获得任何东西。

Maybe that simple algorithms can just get you anything if you do it enough or something.

对。

Right.

对。

Right.

是的。

Yeah.

是的。

Yeah.

不知道。

Don't know.

所以

So

但没错。

But yeah.

所以你呢，可能有这个，嗯。

So you you have, like, maybe this yeah.

进化模型是无模型的，基底神经节是无模型的，皮层是基于模型的，嗯。

Evolution model free, basal ganglia model free, cortex model based Mhmm.

文化可能是无模型的。

Culture model free potentially.

我的意思是，比如要听长辈的话之类的。

I mean, there's like pay attention to your elders or whatever.

所以可能这种群体选择之类的机制，更像是无模型的。

So there's like Maybe this, like, group selection or whatever of of these things is, like, more model free.

是的。

Yeah.

但现在我认为，文化确实存储了部分模型。

But now I think culture well, it stores some of the model.

是的。

Yeah.

对。

Right.

假设你想训练一个代理来帮助你处理贷款申请之类的事情。

So let's say you want to train an agent to help you with something like processing loan applications.

训练一个代理来做这件事，仅仅给模型提供正确的工具（比如浏览器、PDF阅读器和风险模型）是不够的。

Training an agent to do this requires more than just giving the model access to the right tools, things like browsers and PDF readers and risk models.

还有一种隐性知识，只有实际在行业中工作才能获得。

There's this level of tacit knowledge that you can only get by actually working in an industry.

例如，某些贷款申请尽管风险极高，却能通过所有自动化检查。

For example, certain loan applications will pass every single automated check despite being super risky.

申请中的每一个单独部分可能看起来都很安全，但有经验的核保人员知道要跨文件比较，以发现暗示风险的细微模式。

Every single individual part of the application might look safe, but experienced underwriters know to compare across documents to find subtle patterns that signal risk.

Labelbox 拥有您所在领域的此类专家，他们会搭建高度真实的训练环境，涵盖您需要关注的所有细微差别和注意事项。

Labelbox has experts like this in whatever domain you're focused on, and they will set up highly realistic training environments that include whatever subtle nuances and watchouts you need to look out for.

除了构建环境本身，Labelbox 还提供了捕捉代理训练数据所需的所有支持框架。

Beyond just building the environment itself, Labelbox provides all the scaffolding you need to capture training data for your agent.

他们提供工具来评估代理的表现、录制每次会话的视频，并在每个episode之间将整个环境重置为初始状态。

They give you the tools to grade agent performance and capture the video of each session and to reset the entire environment to a clean state between every episode.

因此，无论您从事哪个领域，Labelbox 都能帮助您训练出可靠、贴近现实世界的代理。

So whatever domain you're working in, Labelbox can help you train reliable, real world agents.

了解更多，请访问 labelbox.com/thorcash。

Learn more at labelbox.com/thorcash.

退一步说，与当前的计算机相比，人类使用生物硬件是劣势还是优势？

Stepping back, how, is it a disadvantage or an advantage for humans that we get to use biological hardware in comparison to computers as they exist now?

我问这个问题的意思是，如果有一个算法，当它被实现于当今的硬件上时，其表现会明显更差还是更好？

So, by what I mean by this question is like, if there's the algorithm, would the algorithm just qualitatively perform much worse or much better if, inscribed in the hardware today?

所以我认为可能的原因是，让我解释一下。

And the reason to think it might like, here's what I mean.

你知道，大脑显然不得不做出许多权衡，而这些权衡对竞争性的硬件来说并不相关。

Like, you know, obviously the brain has had to make a bunch of trade offs, which are not relevant to competing hardware.

它必须更加节能。

It has to be much more energetically efficient.

也许正因为如此，它必须以较慢的速度运行，以便使用更小的电压差。

Maybe as a result, has to learn, run on slower speeds so that it can get smaller voltage gap.

因此，大脑以200赫兹运行，功耗仅为20瓦。

And so the brain runs at 200 Hertz, and has to like run on 20 Watts.

另一方面，你知道，在机器人领域，我们已经明显发现，手指的灵活性远超我们目前能制造的电机。

On the other hand, you know, with like robotics, we've clearly experienced that fingers are way more nimble than we can make motors so far.

所以，大脑中或许存在某种类似认知灵活性的东西，这可能是因为我们能够实现非结构化的稀疏性，能够将内存和计算能力紧密整合。

And so maybe there's something in the brain that is equivalent of like cognitive dexterity, which is like, maybe due to the fact that we can do unstructured sparsity, we can co locate the memory and the compute.

是的。

Yes.

这一切最终会走向何方？

Where does this all land out?

你是觉得，天啊，如果我们不用处理这些大脑，事情会聪明得多吗？还是说你认为？

Are you like, fuck, would be so much smarter if we didn't have to deal with these brains or are you like,

哦。

oh.

我的意思是，最终我们将会得到两者的最佳优势，对吧。

I mean, I think in the end we will get the best of both worlds Right.

以某种方式。

Somehow.

对吧？

Right?

我认为大脑一个明显的缺点是它无法被复制。

I think I think an obvious downside of the brain is it cannot be copied.

是的。

Yeah.

你无法像操作外部读写那样访问每一个神经元和突触。

You don't have, you know, external read write access to every neuron and synapse.

而你可以，我只需在权重矩阵中编辑一些东西，对吧。

Whereas you do, I can just edit something in the weight matrix Right.

你可以在Python里，或者随便什么语言，加载它，原则上就能复制。

You know, in Python or whatever, load that up and copy that in principle.

所以，它无法被复制和随机访问这一点非常烦人。

So the fact that it can't be copied and random accessed is very annoying.

但除此之外，它可能还有很多优势。

But otherwise, maybe it has a lot of advantages.

所以，或者这也告诉你，你希望以某种方式实现算法与硬件的协同设计，也许这并没有改变我们之前讨论的太多内容，但你确实希望进行这种协同设计。

So or it also tells you that you wanna somehow do the codesign of the algorithm and the it maybe that that even doesn't change it that much from all of what we discussed, but you wanna somehow do this codesign.

那么，你如何用那些非常缓慢、低电压的开关来实现呢？

So, yeah, how do you do it with really slow, low voltage switches?

这对能耗来说将至关重要。

That's gonna be really important for the energy consumption.

将内存与计算单元放在一起。

The co locating memory and compute.

所以，我认为硬件公司很可能会尝试将内存和计算单元集成在一起。

So, like, I I think that probably just like hardware companies will try to co locate memory and compute.

他们会尝试使用更低的电压，并允许一些随机性。

They will try to use lower voltages, allow some stochastic stuff.

有些人认为，我们之前讨论的这些概率性方法，比如能量模型等，实际上是在进行大量采样。

There are some people that think that this like, all this probabilistic stuff that we were talking about, oh oh, it's actually energy based models and so on, is doing lot it is doing lots of sampling.

这不仅仅是对所有内容进行摊销。

It's not just amortizing everything.

神经元天然适合这一点，因为它们本身就具有随机性。

That the neurons are also very natural for that because they're naturally stochastic.

因此，你不需要用随机数生成器和一大堆Python代码来生成样本。

And so you don't have to do a random number generator and a bunch of Python code basically to generate a sample.

神经元本身就能生成样本，并且可以调节不同概率。

The neuron just generates samples, it can tune what the different probabilities are.

是的。

Yeah.

所以，要学习这些调整。

And so and, like, learn learn those tunings.

因此，它可能与某种推理方法紧密协同设计。

And so it could be that it's very codesigned with, like, some kind of inference method or something.

是的。

Yeah.

这简直太搞笑了。

It'd be hilarious.

我的理解是，这场访谈中我注意到，那些在推特上被人嘲笑的人，比如杨立昆、贝丝·贾索斯之类的。

I mean, the the message I'm taking over this interview is, like, you know, all these people that folks make fun of on Twitter, you know, Yan Lakul Yan Lakul and Beth Jasos and whatever.

他们就像是

They're like

谁知道呢？

Who knows?

不，不是。

Nope.

呃，什么意思啊。

Like, what yeah.

也许我还不知道这个。

Maybe I don't know of it.

这实际上是一种解读方式。

That is actually that is actually one read of of the read.

你知道，自从大语言模型兴起后，我就没再从事过AI相关的工作。

You know, I I haven't really worked on AI at all since LLMs, you know, took off.

所以我现在已经跟不上潮流了。

So I'm I'm just, like, out of the loop.

但我很惊讶，我觉得缩放机制的效果真的很棒，一切都很了不起。

But I'm I'm surprised, and I'm I I I think it's amazing how the scaling is is working and everything.

不过，我觉得杨·莱库恩和贝丝·杰佐斯在概率模型方面确实有些见解，或者至少有可能。

But, yeah, I think Jan Lecun and Beth Jezzos are kinda onto something about the about the probabilistic models or at least possibly.

事实上，直到2021年左右，所有神经科学家和AI领域的人都是这么认为的。

And in fact, that's what, you know, all the neuroscientists and all the AI people thought, like, until 2021 or something.

对吧？

Right?

大脑中发生着大量细胞层面的活动，而这些并不仅仅是神经元之间的突触连接。

So there's a bunch of cellular stuff happening in the brain that is not just about neuron to neuron synaptic connections.

这些细胞活动中有多少是在功能上比突触本身做了更多的工作，而不是仅仅为了支持突触运作而必须进行的辅助性工作？

How much of that is functionally doing more work than the synapses themselves are doing versus it's just a bunch of collage that you have to do in order to make the synaptic thing work.

在数字思维中，你可以非常轻松地调整突触——抱歉，是参数，但要通过细胞根据梯度信号来调节突触，却需要动用这一整套复杂的机制。

So the way you need to, you know, with a digital mind, you can nudge the synapse, sorry, the parameter extremely easily, but with a cell to modulate a synapse according to the gradient signal, it just takes out all of this crazy machinery.

所以，这些细胞活动真的比用极少量代码就能完成的工作还要多吗？

So, like, is it actually doing more than it takes extremely little code to do?

我不知道，但我并不相信那种激进的观点，比如记忆主要不是靠突触，或者学习主要是基因变化之类的。

So I don't know, but I'm I'm not a believer in the, like, radical, like, oh, actually, memory is not synapses mostly or, like, learning is mostly genetic changes or something like that.

我觉得这种说法很有道理。

I think it would just make a lot of sense.

我觉得你表达得非常好，更像你所说的第二种情况。

I think you put it really well for it to be more like the second thing you said.

比如，假设你想对从神经元发出或输入神经元的所有权重进行归一化，对吧？

Like, let's say you wanna do weight normalization across all of the weights coming out of your neuron, right, or into your neuron.

那么你可能需要以某种方式通知细胞核，然后让细胞核将这些信息重新传递给所有的突触之类的。

Well, you probably have to go, like, somehow tell the nucleus about this of the cell and then have that kind of send everything back out to the synapses or something.

对吧？

Right?

因此，会有很多细胞层面的变化。

And so there's gonna be a lot of cellular changes.

对吧？

Right?

或者说，你经历了大量的可塑性，参与了某个记忆的形成，而现在这个记忆已经被巩固到皮层中了。

Or let's say that, you know, you just had a lot of plasticity and, like, you're part of this memory, and now that's got consolidated into the cortex or whatever.

现在我们想让你重新发挥作用，再次学习。

And now we wanna reuse you as, like, a new one that can learn again.

这将带来大量的细胞变化。

It's gonna be a ton of cellular changes.

所以细胞里会发生很多事，但从算法上讲，它并没有超越这些算法增加什么新东西。

So there's gonna be tons of stuff happening in the cell, but algorithmically, it's not really adding something beyond these algorithms.

对吧？

Right?

它只是实现了一种在数字计算机上对我们来说非常容易就能找到权重并进行修改的东西。

It's just implementing something that in a digital computer is very easy for us to go and just find the weights and change them.

这毕竟是一个细胞。

And it is a cell.

它必须完全依靠分子机器自己完成所有这些工作，是的。

It just literally has to do all this with molecular machines itself Yeah.

没有任何中央控制器。

Without any central controller.

对吧？

Right?

这简直令人难以置信。

It's kind of incredible.

有些细胞做的事情，我觉得更令人信服。

There are some things that cells do, I think, that that seem like more convincing.

在小脑中，小脑需要做的一件事就是随时间进行预测。

So in the cerebellum so one of the things the cerebellum has to do is, like, predict over time.

比如，预测时间延迟。

Like, predict what is the time delay.

比如说，我看到一个闪光，然后几毫秒后，我的眼睑会受到一股气流的冲击之类的。

You know, let let's say that, you know, I see a flash and then, you know, some number of milliseconds later, I'm gonna get, like, a puff of air in my eyelid or something.

对吧？

Right?

小脑能够非常准确地预测闪光和气流之间的时序，这样你的眼睛就会自动闭上。

The cerebellum can be very good at predicting what's the timing between the flash and the air puff so that now your eye will just, like, close automatically.

小脑参与了这种类型的反射，也就是习得性反射。

Like, the cerebellum is, like, involved in that type of reflex, like, learned reflex.

在小脑中有一些细胞，似乎细胞体在存储这种时间常数，改变延迟的时间常数，而不是通过构建更长的突触环路来延长延迟。

And there are some cells in the cerebellum where it seems like the cell body is playing a role in storing that time constant, changing that time constant of delay versus that all being somehow done with, like, I'm gonna make a longer ring of synapses to make that delay longer.

不是这样的。

It's like, no.

细胞体本身就会为你存储这种时间延迟。

The cell body will just, like, store that time delay for you.

所以确实有一些例子，但我并不相信，至少在直觉上，那种认为神经元之间连接变化就是发生机制的理论。

So there are some examples, but I'm not a believer, like, out of the box in, like, essentially this theory that, like, what's happening is changes in connections between neurons.

是的。

Yeah.

而那正是主要的算法机制。

And that's, like, the main algorithmic thing that's going on.

我认为，坚持认为是这种机制，而不是某些疯狂的细胞层面的变化，是有充分理由的。

Like, I I think that's a very good reason to to still believe that it's that rather than some, like, crazy cellular stuff.

是的。

Yeah.

回到这个观点，我们的智能并不仅仅是一个全方位的推理系统，构建一个世界模型，而实际上是一个教会我们该关注什么、哪些是重要的显著因素来学习的系统。

Going back to this whole perspective of, like, our our intelligence is not just this omnidirectional inference thing that builds a world model, but really this system that teaches us what to pay attention to, are the important salient factors to learn from, etcetera.

我想看看我们能否从这个角度获得一些直觉，来理解不同种类的智能可能是什么样子。

I want to see if there's some intuition we can drive from this, but what different kinds of intelligences might be like.

因此，似乎AGI或超人智能仍然应该具备学习一个相当通用的世界模型的能力。

So it seems like AGI or superhuman intelligence should still have this, like ability to learn a world model that's quite general.

是的。

Yeah.

但它可能会被激励去关注那些在后奇点环境中相关的事物。

But then it might, be incentivized to pay attention to different things that are relevant for what, you know, the the modern post singularity environment.

我们该预期不同的智能之间有多大差异呢？

How different should we expect different intelligences to be basically?

是的。

Yeah.

换句话说，这个问题的一种思考方式是：我们真的有可能制造出纸夹最大化器之类的东西吗？

Mean, think one way of this question is like, is it actually possible to, like, make the paper clip maximizer or whatever?

对吧？

Right?

如果你试图制造一个纸夹最大化器，它会不会最终根本不够聪明之类的？

If you make if you try to make the paper clip maximizer, does that end up, like, just not being smart or something like that?

因为它的唯一奖励函数就是制造纸夹。

Because it's it was just the only reward function it had was, like, make paper clips.

有趣。

Interesting.

是的。

Yeah.

是的。

Yeah.

如果我更多地代入史蒂夫·伯恩斯的观点，我认为他非常担心，要让一个系统变得聪明，所需的最小化引导子系统组件，远少于让它具备人类社会本能、伦理道德等所需的最小组件。

If I channel Steve Burns more, I mean, I think he's very concerned that the the sort of minimum viable things in the steering subsystem that you need to get something smart is way less than the minimum viable set of things you need for it to have human, like, social instincts and ethics and stuff like that.

因此，你真正想了解的引导子系统，实际上本质上是关于对齐的具体方式，或者说是人类行为与社会本能， versus 仅仅实现能力所需的东西。

So a lot of what you wanna know about the steering subsystem is actually the specifics of how you do alignment essentially, or what human behavior and social instincts is versus just what you need for capabilities.

我们以前以一种稍微不同的方式讨论过这个问题，因为我们当时在说，人类要进行社会性学习，需要进行眼神交流并从他人那里学习。

And we talked about it in a slightly different way because we were sort of saying, well, in order for humans to learn socially, they need to make eye contact and learn from others.

但我们已经从大语言模型中知道，对吧？根据你的起点，你可以在没有这些东西的情况下学会语言。

But we already know from LLMs, right, that depending on your starting point, you can learn language without that stuff.

对吧？

Right?

所以，是的。

And so yeah.

因此，我认为很可能可以创造出非常强大的基于模型的强化学习优化系统之类的东西，而它们并不具备人类大脑中大部分的奖励机制。

And and so I think that it probably is possible to make super powerful model based RL, optimizing systems and stuff like that, that don't have most of what we have in the human brain reward functions.

因此，它们可能会想要最大化回形针的数量，这是一个令人担忧的问题。

As And a consequence, might wanna maximize paper clips, and that's a concern.

是的。

Yeah.

对。

Right.

但你指出，要制造一个称职的回形针最大化器，确实如此。

But but you're pointing out that in order to make a competent paper clip maximizer Yeah.

那种能够建造飞船、学习物理等的系统，需要具备一些促使学习的驱动力，比如好奇心和探索欲。

The kind of thing that can build the spaceships and learn the physics and whatever, it needs to have some drives which elicit learning, including, say, curiosity and exploration.

是的。

Yeah.

好奇心以及对他人的兴趣，没错。

Curiosity and and interest in others Yeah.

也就是说，对社交互动的兴趣、好奇心。

Of so interest in social interactions, curiosity.

是的。

Yeah.

但这些其实已经非常少了，我认为，人类也是如此。

But but that that's pretty that's pretty minimal, I think, and it and that's true for humans.

对。

Right.

但对于像已经预训练好的LLM之类的东西，这可能就不那么成立了。

But it might be less true for, like, something that's already pretrained as an LLM or something.

对吧？

Right?

所以，如果我站在史蒂夫的角度来看，我们之所以想了解这个引导子系统，大部分原因是为了对齐问题。

And so so most of why we wanna know the steering subsystem, I think, if I'm channeling Steve, is alignment reasons.

对。

Yeah.

没错。

Right.

我们有多大的把握，自己拥有的算法和概念词汇，能够准确描述大脑在做什么？

How how confident are we that we even have the right algorithmic conceptual vocabulary to think about what the brain is doing?

我的意思是，神经科学对人工智能曾有一个重大贡献，那就是神经元的概念，这源于20世纪50年代威廉和菲特等人的开创性工作。

And what I mean by this is, you know, there was one big contribution to AI from neuroscience, which was this idea of the neuron, which like William and Fitt, you know, 1950s, like this original contribution.

但此后，我们从大脑中了解到的许多关于其高层算法的内容——比如大脑中是否存在类似CNN在V1层运作、TD学习、贝尔曼方程、演员-评论家等机制的类比过程——似乎就少多了。

But then it seems like a lot of what we've learned afterwards about what the high level algorithm the brain is implementing from the backdrop to, or if there's something analogous backdrop happening in the brain to always V1 doing something like CNNs to TD learning and Bellman equations, actor critic, whatever

对。

Yeah.

这似乎是受到某种启发，我们提出某个想法，比如我们可以让人工智能神经网络以这种方式工作。

Seems inspired by what is like, we come up with some idea, like, maybe we can make AI neural networks work this way.

对。

Yeah.

然后我们发现大脑中的某些东西也是以这种方式运作的。

And then we notice that something in the brain also works that way.

是的。

Yes.

所以为什么不去想，可能还有更多类似的情况，其中

So why not think there's more things like this where

哦，也许吧。

Oh, may be.

对。

Yeah.

我认为我们可能有所发现的原因是，基于这些想法构建的AI表现得异常出色。

Think the reason that I'm not I think that we might be onto something is that, like, the AIs we're bake making based on these ideas are working surprisingly well.

还有一大堆经验性的东西，比如卷积神经网络及其各种变体。

There's also a bunch of, like, just empirical stuff, like like, convolutional neural nets and variants of convolutional neural nets.

我不确定最新的情况到底如何，但与其他计算神经科学中关于视觉系统运作的模型相比，它们的预测能力更强。

I'm not for sure what the absolute latest latest, but compared to other models in computational neuroscience of what the visual system is doing are just more predictive.

对吧？

Right?

所以，即使只是在猫的图片上预训练过，你也可以衡量CNN在某些任意图像上的表征相似性，与通过不同方式测量的大脑激活数据进行对比。

So you can just score even pre trained on cat pictures and stuff, CNNs, what is the representational similarity that they have on some arbitrary other image compared to the brain activations measured in different ways.

吉姆·德卡洛的实验室有脑评分这个指标。

Jim DeCarlo's lab has the brain score.

AI模型确实显示出一定的相关性，就神经科学而言，目前似乎还没有更好的模型能超越它。

And the AI model is actually there seems to be some relevance there in terms of neurosciences don't necessarily have something better than that.

所以，是的，我的意思只是重申一下你的观点：我们目前最好的计算神经科学理论似乎都是偶然发现的。

So, yes, I mean, that's just kind of recapitulating what you're saying is that the best computational neuroscience theories we have seem to have been invented Right.

很大程度上是因为AI模型找到了有效的方法。

Largely as a result of AI models and find things that work.

所以发现反向传播有效后，我们是否可以用皮层回路之类的东西来近似反向传播？

And so find backprop works and then say, can we approximate backprop with cortical circles or something?

确实出现过类似这样的尝试。

And there's there's kind of been things like that.

现在有些人完全不同意这种观点。

Now some people totally disagree with this.

对吧？

Right?

比如，尤里·布扎基是一位神经科学家，他写了一本叫《从内而外的脑》的书，书中基本认为，我们所有的心理学概念、AI概念，这些东西都是凭空捏造的。

So, like, Yuri Buzaki is a neuroscientist who has a book called The Brain from Inside Out, where he basically says, like, all our psychology concepts, like AI concepts, all this stuff is just, like, made up stuff.

我们真正需要做的是，找出大脑实际使用的那些基本单元，而我们的现有词汇根本不足以描述它们。

What we actually have to do is, like, figure out what is the actual set of primitives that, like, the brain actually uses, and our vocabulary is not gonna be adequate to that.

我们必须从大脑出发，创造新的术语，而不是先提出反向传播，再试图把它套用到大脑上之类的做法。

We got to start with the brain and make new vocabulary rather than saying back prop and then try to apply that to the brain or something like that.