吉姆·凯勒：摩尔定律、微处理器、抽象与第一性原理

所以即使在今天，大多数人认为每次运行程序都会得到相同的答案。

And so most people think even today, every time you run a program, you get the same answer.

不。

No.

我知道。

I know.

但问题是，这正是编程语言的形式定义。

But the the question is, that's the formal definition of a programming language.

确实存在一些语言，它们运行时不会得到相同的结果，但使用这些语言的人总是希望得到确定的结果，因为当你得到一个错误的答案时，你会怀疑这到底是算法的问题，还是因为这个原因。

There is a definition of languages that don't get the same answer, but people who use those you always want something because you get a bad answer, and then you're wondering, is it because Right.

是因为你的算法出了问题，还是因为这个？

Of something in your algorithm or because of this?

所以每个人都希望有一个开关，无论怎样，都以确定性的方式运行。

And so everybody wants a little switch that says, no matter what, do it deterministically.

这真的很奇怪，因为现代计算中几乎所有的输入都是有噪声的。

And it's really weird because almost everything going into modern calculations is noisy.

那么，为什么答案必须这么明确呢？

So why do the answers have to be so clear?

这是

It's

对的。

Right.

那你持什么观点？

So where do you stand?

我为运行程序的人设计计算机。

I design computers for people who run programs.

所以如果有人说我想要一个确定性的答案，就像大多数人希望的那样。

So if somebody says, I want a deterministic answer, like most people want that.

你能提供一个确定性的答案吗？这是问题所在。

Can you deliver a deterministic answer, guess, is the question.

比如，当你

Like, when you

是的。

Yeah.

希望如此，当然。

Hopefully, sure.

那就是

That that's What

人们没有意识到的是，即使执行流程非常不确定，你仍然能得到一个确定的答案。

people don't realize is you get a deterministic answer even though the execution flow is very undeterministic.

所以你运行这个程序一百次。

So you run this program a 100 times.

它从来没有两次以相同的方式运行。

It never runs the same way twice ever.

而答案，它得出了相同的答案。

And the answer, it arrives at the same answer.

但它每次都得到相同的结果。

But it gets the same answer every time.

就这样，没什么大不了的。

It's just it's just Okay.

在许多人眼中，你已经达到了芯片设计大师的传奇地位，嗯。

You've achieved in the eyes of many people, legend status as a chip art Mhmm.

架构师。

Architect.

你最自豪的设计作品是哪一个？

What design creation are you most proud of?

也许是因为它极具挑战性，或者是因为它的影响力，又或者是因为其中蕴含的一系列卓越创意才让它得以实现。

Perhaps because it was challenging, because of its impact, or because of the set of brilliant ideas that that were involved in bringing it to life.

我觉得这个说法很奇怪，我有两个年幼的孩子。

I find that description odd, and I have two small children.

我向你保证，他们觉得这特别好笑。

And I promise you, they think it's hilarious.

这个问题。

This question.

是的。

Yeah.

所以我会为他们去做。

So I'd do it for them.

所以我真的对建造计算机很感兴趣，而且我曾与非常、非常聪明的人共事。

So I I am I'm really interested in building computers, and I've worked with really, really smart people.

我并不是特别聪明。

I'm not unbelievably smart.

我对人们如何将计算机整合在一起——无论是作为一种活动还是作为一种人类追求——感到着迷。

I'm fascinated by how they go together both as a a thing to do and as a endeavor that people do.

人和计算机如何相互结合？

How people and computers go together?

是的。

Yeah.

就像人们如何思考并建造一台计算机。

Like how people think and build a computer.

我发现，有时最好的计算机架构师对人并不那么感兴趣，而最好的团队管理者也不一定擅长设计计算机。

And I find sometimes that the best computer architects aren't that interested in people or the best people managers aren't that good at designing computers.

整个由人类组成的体系都很迷人。

The So whole stack of human beings is fascinating.

管理者、个体工程师……

So the managers, the individual engineers

是的。

Yeah.

对。

Yeah.

没错。

So yeah.

经过多年构建计算机的经历，我意识到，当你从晶体管、逻辑门、功能单元、计算单元这些部件来构建计算机时，你也可以用同样的方式看待人。

I said I realized after a lot of years of building computers, where you sort of build them out of transistors, logic gates, functional units, computational elements, that you could think of people the same way.

所以，人就是功能单元。

So people are functional units.

然后你可以把组织设计看作是一个计算机架构问题。

And then you could think of organizational design as a computer architecture problem.

然后我就觉得，这太棒了，因为每个人都不一样，就像计算单元各不相同一样。

And then it was like, oh, that's super cool, because the people are all different, just like the computational elements are all different.

他们喜欢做不同的事情，因此我在重新思考如何理解组织方面找到了很多乐趣。

And they like to do different things, and and so I had a a lot of fun, like, reframing how I think about organizations.

就像我们之前说的计算机一样，有各种执行路径。

Just like with with computers, we were saying execution paths.

你可以有很多不同的路径，最终都到达同一个好的目标。

You can have a lot of different paths that end up at a at the at a at the same good destination.

从单个的人类功能单元到更广泛的组织，你从人类抽象中学到了什么？

So what have you learned about the human abstractions from individual functional human units to the the broader organization?

要创造出特别的东西，需要什么？

What what does it take to create something special?

大多数人想得还不够简单。

Well, most people don't think simple enough.

好的。

Alright.

那么，你知道食谱和理解之间的区别吗？

So do you know the difference between a recipe and the understanding?

理解？

Understanding?

这可能有一个哲学上的描述。

There's probably a philosophical description of this.

想象你要烤一条面包。

So imagine you're gonna make a loaf of bread.

对。

Yep.

食谱上说：拿一些面粉，加一些水，加一些酵母，混合均匀，让它发酵，放进烤盘，然后放入烤箱。

The recipe says, get some flour, add some water, add some yeast, mix it up, let it rise, put it in a pan, put it in the oven.

这只是一个食谱。

It's a recipe.

对吧？

Right?

真正理解了做面包，你就能理解生物学、供应链，比如谷物研磨机、酵母、物理学，还有热力学。

Understanding bread, you can understand biology, supply chains, you know, grain grinders, yeast, physics, you know, thermodynamics.

这里面有太多层次的理解了。

Like, there's so many levels of understanding there.

当人们构建和设计事物时，他们常常只是在执行一系列的流程步骤。

And then when people build and design things, they frequently are executing some stack of recipes.

对吧？

Right?

但问题是，这些流程步骤都有其局限性。

And the problem with that recipes all have limited scope.

比如，一本非常好的面包制作食谱书，不会告诉你如何做煎蛋卷。

Like, if you have a really good recipe book for making bread, it won't tell you anything about how to make an omelet.

对。

Right.

对吧？

Right?

但如果你对烹饪有深入的理解，比如面包、煎蛋卷，还有三明治。

But if you have a deep understanding of cooking, right, then bread, omelets, you know, sandwich.

你知道，看待一切的方式会完全不同。

You know, there's there's a different, you know, way of viewing everything.

当人们在某件事上成为专家时，你希望获得的是更深层的理解，而不仅仅是掌握一大堆可以执行的食谱。

And most people, when you get to be an expert at something, you know, you're you're hoping to achieve deeper understanding, not just a large set of recipes to go execute.

有趣的是，当你带领一群人时，执行食谱如果正是你想要的，效率会高得惊人。

And it's interesting to walk groups of people because executing recipes is unbelievably efficient if it's what you want to do.

但如果你不希望这样，就会陷入困境，而这种差异至关重要。

If it's not what you wanna do, you're really stuck, and and that difference is crucial.

每个人都在深层理解与食谱之间保持着某种平衡。

And and everybody has a balance of, let's say, deeper understanding of recipes.

有些人非常擅长判断何时需要深入理解某件事。这说得通吗？

And some people are really good at recognizing when the problem is to understand something deep Does that make sense?

这完全说得通。

It totally makes sense.

团队在每个开发阶段都需要深入理解吗？

Does every stage of development deep understanding on the team needed?

这又回到了艺术与科学的问题。

Well, this goes back to the art versus science question.

当然。

Sure.

如果你总是为了深入理解而拆解一切，那就永远无法完成任何事。

If you constantly unpacked everything for deeper understanding, you never get anything done.

对。

Right.

但如果你在需要时不去拆解理解，就会做错事。

And if you don't unpack understanding when you need to, you'll do the wrong thing.

在每一个关键节点上，人类都是很奇怪的生物，因为你告诉他们的每件事都有无数种可能的反应。

And then at every juncture, like human beings are these really weird things because everything you tell them has a million possible outputs.

对吧？

Right?

然后它们以一种非常有趣的方式相互作用。

And then they all interact in a hilarious way.

是的。

Yeah.

这非常有趣。

It's very interesting.

而要对何时该告诉他们什么、该做什么、何时介入、何时不介入有某种直觉，这很复杂。

And then having some intuition about what do you tell them, what do you do, when do you intervene, when do you not, it's complicated.

对。

Right.

所以

So

这本质上是计算上无法解决的。

It's it's it's, you know, essentially computationally unsolvable.

是的。

Yeah.

这是一个难以解决的问题。

It's an intractable problem.

当然。

Sure.

人类太混乱了。

Humans are a mess.

你说的深刻理解，也包括一些根本性的问题，比如什么是计算机？

With deep understanding, do you mean also sort of fundamental questions of things like what is a computer?

比如为什么、为什么这样的问题，我们为什么要构建这些东西，关于目的的问题？

Like or why like, like, the why questions, why are we even building this, like, of purpose?

还是说你指的是更接近物理学的根本极限，真正深入科学的核心？

Or do you mean more, like, going towards the fundamental limits of physics, sort of really getting into the core of the science?

嗯，从构建计算机的角度来看，想得简单一点，再简单一点。

Well, in terms of building a computer, think simple think a little simpler.

所以通常的做法是，你先构建一台计算机。

So common practice is you build a computer.

然后当有人提出想让它快10%时，你会去想：好吧，我得把这个缓冲区调大一点，也许还可以加个广告位。

And then when somebody says, want to make it 10% faster, you'll go in and say, all right, I need to make this buffer bigger, and maybe I'll add an ad unit.

或者，你知道，我有个东西是三条指令宽的。

Or, you know, I have this thing that's three instructions wide.

我要把它改成四条指令宽。

I'm gonna make it four instructions wide.

你会发现，每个部分都变得越来越复杂。

And what you see is each piece gets incrementally more complicated.

对吧？

Right?

然后在某个时刻，你遇到了瓶颈。

And then at some point, you hit this limit.

比如，再添加一个功能或缓冲区似乎并不能让它更快了。

Like, adding another feature or buffer doesn't seem to make it faster.

然后人们会说，这是因为达到了根本性的极限。

And then people will say, well, that's because it's a fundamental limit.

然后另一个人会仔细观察并说，实际上，是你划分问题的方式以及不同功能之间的交互方式限制了你，必须重新思考和重写。

And then somebody else will look at it and say, well, actually, way you divided the problem up and the way that different features are interacting is limiting you, and it has to be rethought, rewritten.

对吧？

Right?

于是你重构并重写了它，人们通常发现，重写后的版本不仅更快，而且复杂度还降低了一半。

So then you refactor it and rewrite it, and what people commonly find is the rewrite is not only faster, but half as complicated.

从头开始？

From scratch?

是的。

Yes.

那么在你的职业生涯中，你见过多少次需要彻底抛弃整个系统的情况呢？

So how often in your career, but just have you seen as needed maybe more generally to just throw the whole out the whole thing out

而我所处的这一端，正是从这里开始的。

and This start a is where I'm on one end of it.

每三到五年。

Every three to five years.

你站在哪一边？

Which end are you on?

等等。

Wait.

更频繁地重写。

Rewrite more often.

重写。

Rewrite.

三到五年是如果你想要的话

And three to five years is If you want

要想在计算机架构上取得重大进展，每五年就应该从头开始做一次。

to really make a lot of progress on computer architecture, every five years you should do one from scratch.

所以

So

x86-64标准是从哪里来的？

where does the x eighty six sixty four standard come in?

或者你多久写一次？我曾经是那个的合著者，那时候

Or what How often do you I wrote the I was the coauthor of that back

在1998年。

in '98.

那是二十年前了。

That's twenty years ago.

是的。

Yeah.

所以它现在还在用。

So that's still around.

指令集本身已经扩展过很多次了。

The instruction set itself has been extended quite a few times.

对。

Yes.

指令集本身不如其底层实现有趣。

And instruction sets are less interesting than the implementation underneath.

在x86架构上，英特尔设计了几种，AMD也设计了几种，不同的架构。

Been on x86 architecture, Intel's designed a few, AMD's designed a few, different architectures.

我不想过多讨论具体频率，但通常每十年就会重写一次，而实际上每五年就应该重写一次。

And I don't don't wanna go into too much of the detail about how often, but there's a tendency to rewrite it every, you know, ten years, and it really should be every five.

所以你的意思是，从这个角度来看，你是少数派？

So you're saying you're an outlier in that sense in the

我们写得更频繁。

We write more often.

我们写得更频繁。

We write more often.

那这难道不可怕吗？

Well and here's Isn't it scary?

是的。

Yeah.

当然。

Of course.

对谁来说可怕呢？

Well, scary to who?

对所有相关方来说都很可怕，因为就像你说的，重复这个流程是高效的。

To everybody involved because like you said, repeating the recipe is efficient.

公司想赚钱，嗯，不是。

Companies wanna make money well, no.

而个体工程师想成功，所以你想逐步改进，把缓冲区从三增加到四，对吧。

And the individual engineers wanna succeed, so you wanna incrementally improve, increase the buffer from three to four while we Yeah.

所以这就涉及到收益递减曲线了。

So this is where you get into diminishing return curves.

我觉得史蒂夫·乔布斯说过这句话。

I think Steve Jobs said this.

对吧？

Right?

所以每当你有一个项目，你从这里开始，它会不断上升，但回报会递减。

So every you have a project and you start here and it goes up and they have diminishing return.

是的。

Yep.

要达到下一个水平，你必须从头开始一个新的项目。

And to get to the next level, you have to do a new one.

新的起点会低于之前的优化点，但最终会变得更高。

And the initial starting point will be lower than the old optimization point, but it'll get higher.

所以你现在面临两种恐惧：短期灾难和长期灾难。

So now you have two kinds of fear, short term disaster and long term disaster.

你们真是大人了。

You're So grown ups.

对。

Right.

是的。

Yes.

比如，那些以季度为单位设定业务目标的人，对彻底改变一切感到恐惧。

Like, you know, people with a quarter by quarter business objective are terrified about changing everything.

是的。

Yeah.

而那些致力于长期目标、经营企业或开发计算机的人，知道短期的限制会阻碍他们实现长期成功。

And people who are trying to run a business or build a computer for a long term objective know that the short term limitations block them from the long term success.

所以，如果你观察那些取得卓越长期成功的企业领导者，每次他们意识到必须重做某件事时，他们都会去做。

So if you look at leaders of companies that had really good long term success, every time they saw that they had to redo something, they did.

所以必须有人站出来发声。

And so somebody has to speak up.

或者你可以同时进行多个项目。

Or you do multiple projects in parallel.

比如，在优化旧项目的同时，开发一个新的项目。

Like, you optimize the old one while you build a new one.

但市场部门的人总是说：向我保证，新电脑在每一方面都更快。

And but the marketing guys are always like, make promise me that the new computer is faster on every single thing.

而计算机架构师会说，新计算机在平均性能上会更快。

And the computer architect says, well, the new computer will be faster on the average.

但结果和性能存在一个分布，会有一些较慢的异常情况。

But there's a distribution of results and performance, and you'll have some outliers that are slower.

这非常困难，因为他们有一个客户特别在意那个异常情况。

And that's very hard because they have one customer who cares about that one.

说到长期发展，五十多年来，摩尔定律对我来说以及数百万人而言，一直是一盏鼓舞人心的明灯，指引着卓越工程师能够创造的奇妙未来。

So speaking of the long term, for over fifty years now, Moore's Law has served a for me and millions of others as an inspiring beacon of what kind of amazing future brilliant engineers can build.

是的。

Yep.

我今天一直在逗你的孩子们笑。

I'm just making your kids laugh all of today.

这太棒了。

That was great.

那么首先，在你看来，如果要向不了解的人解释，摩尔定律是什么？

So first, in your eyes, what is Moore's Law if you could define for people who don't know?

嗯，戈登·摩尔的简单表述是，每两年晶体管数量翻一番，大概是这样。

Well, the the simple statement was from Gordon Moore was double the number of transistors every two years, something like that.

而我的操作模型是，我们每两到三年将计算机性能提升两倍，但这个数值在历史上波动很大。

And then my operational model is we increase the performance of computers by two x every two or three years, and it's wiggled around substantially over time.

而且，我们交付性能的方式也发生了变化。

And also in how we deliver performance has changed.

对，没错。

So right.

但这个

But the

核心理念是每两年晶体管数量翻一番。

the foundational idea was two x the transistors every two years.

现在的节奏更像是他们所说的‘缩小因子’，大约是每两年缩小到0.6倍，而不是0.5。

The current cadence is something like they call it a shrink factor, like point six every two years, which is not point five.

但这个说法，严格来说，还是指最初的定义吧？

But that that's referring strictly, again, to the original definition?

是的

Yeah.

晶体管数量

Transistor count.

缩小因子就是让晶体管变得越来越小。

And shrink factor is just getting them smaller and smaller and smaller.

嗯，这是在芯片面积不变的前提下。

Well, as you it's for a constant chip area.

对。

Right.

如果你把晶体管尺寸缩小到0.6倍，就能多容纳1除以0.6个晶体管。

If you make the transistor smaller by point six, then you get one over point six more transistors.

你能再详细讲讲吗？

So can you linger on it a little longer?

从更广的角度来看，当你提到你对性能的理解时，你认为摩尔定律应该怎样更全面地定义？

What's what's a broader what do you think should be the broader definition of Moore's Law when you mentioned how you think of performance?

总的来说，怎样才能更好地理解摩尔定律？

Just broadly, what's a good way to think about Moore's Law?

首先，我了解摩尔定律已经有三十年了。

Well, first of all so I've I've I've been aware of Moore's Law for thirty years.

指的是哪个方面？

In which sense?

嗯，我已经设计计算机四十年了。

Well, I've I've been designing computers for forty.

你就在自己眼前亲眼看着它发生。

You're just watching it before your eyes kind of thing.

而且在我第一次了解到摩尔定律的时候，我也听说它将在十到十五年内终结。

Well, and somewhere where I became aware of it, I was also informed that Moore's Law was gonna die in ten to fifteen years.

起初我觉得这没错，但十年后，又有人说它将在十到十五年内终结。

And then I thought that was true at first, but then after ten years it was gonna die in ten to fifteen years.

后来一度说它将在五年内终结，接着又改回了十年。

And then at one point it was gonna die in five years, and then it went back up to ten years.

后来我决定，这辈子再也不去操心这个具体的预言了，这还挺有意思的。

And at some point I decided not to worry about that particular prognostication for the rest of my life, which is which is fun.

然后我加入了英特尔，人人都说摩尔定律已经死了。

And then I joined Intel, and everybody said Moore's Law is dead.

是的。

Right.

我觉得这很遗憾，因为这是摩尔定律的公司，而它并没有死，它一直都被说要死了。

And I thought that's sad because it's the Moore's Law company, and it's not dead, and it's always been gonna die.

你知道，人类喜欢这种似是而非的说法，比如我们会耗尽食物、空气、空间，或者某种其他东西。

And, you know, humans like these apocryphal kind of statements, like, we'll run out of food, or run out of air or run out of room or run out of, you know, something.

对。

Right.

但令人惊叹的是，它竟然已经持续了这么久。

But it's still incredible that it's lived for as long as it has.

是的，现在有很多人认为摩尔定律已经死了。

And, yes, there's many people who believe now that Moore's Law is dead.

我知道。

I know.

他们可以加入过去五十年来持有同样观点的行列。

And they can join the last fifty years of people who had the same

是的。

Yeah.

这有着悠久的传统。

There's a long tradition.

但如果你能试着理解的话，你认为它为什么没有消亡呢？

But why do you think if you can try to understand, why do you think it's not dead?

好吧，让我们想想，人们认为摩尔定律是一回事。

Well, let's just think people think Moore's Law is one thing.

晶体管变得越来越小。

Transistors get smaller.

但实际上，在表象之下，有成千上万项创新。

But actually, under the sheets, there's literally thousands of innovations.

而这些创新中的绝大多数都有各自递减的回报曲线。

And almost all those innovations have their own diminishing return curves.

所以如果你把它画成图，看起来就像一连串递减的回报曲线。

So if you graph it, it looks like a cascade of diminishing return curves.

我不知道该怎么称呼这种现象。

I don't know what to call that.

但结果却是一条指数曲线。

But the result is an exponential curve.

至少到目前为止是这样的。

At least it has been.

我们不断发明新的东西。

And we keep inventing new things.

所以，如果你是某个处于回报递减曲线上的领域的专家，能看到它的平台期，你很可能会告诉别人：哇，这个已经到头了。

So if you're an expert in one of the things on a diminishing return curve, right, and you can see its plateau, you will probably tell people, wow, this is this is done.

与此同时，还有一些其他的人在做不同的事情。

Meanwhile, some other pilot people are are doing something different.

所以这很正常。

So that's that's just normal.

然后就是关于开关器件可以多小的观察。

So then there's the observation of how small could a switching device be.

现代晶体管大约由一千乘一千乘一千个原子组成。

So a modern transistor is something like a thousand by a thousand by a thousand atoms.

对吧？

Right?

当你缩小到两到十十个原子左右时，就会出现量子效应。

And you get quantum effects down around two to two to 10 atoms.

所以你可以想象一个只有十乘十乘十大小的晶体管。

So you can imagine the transistor as small as 10 by 10 by 10.

那就要小一百万倍。

So that's a million times smaller.

而量子计算领域的人正在研究如何利用量子效应。

And then the quantum computational people are working away at how to use quantum effects.

所以

So

一千乘一千乘一千。

A thousand by thousand by thousand.

原子。

Atoms.

这真是一个非常清晰的表达方式。

That's a really clean way of of putting it.

嗯，现代晶体管的栅极，如果你看它的宽度，大约有120个原子宽，但我们可以让它更薄。

Well, a fan like a modern transistor, if you look at the fan, it's like a 120 atoms wide, but we can make that thinner.

然后围绕它有一个栅极，还有间距。

And then there's there's a gate wrapped around it, and then there's spacing.

涉及很多几何结构。

There's a whole bunch of geometry.

一个有经验的晶体管设计师能够数出每个方向上的每一个原子。

And, you know, a competent transistor designer could count both atoms in every single direction.

现在已经有技术可以直接在单原子层上排列原子。

Like, there's techniques now to already put down atoms in a single atomic layer.

对吧？

Right?

如果你愿意，是可以精准放置原子的。

And you can place atoms if you want to.

但从制造工艺的角度来看，如果放置一个原子要花十分钟，而你需要把大约10的23次方个原子组合起来制造一台电脑，那时间就太长了。

It's just, you know, from a manufacturing process, if placing an atom takes ten minutes and you need to put, you know, 10 to the twenty third atoms together to make a computer, it would take a long time.

所以，这些方法既包括不断缩小尺寸，也包括找到有效控制过程的方法。

So the the methods are, you know, both shrinking things and then coming up with effective ways to control what's happening.

稳定且低成本地制造。

Manufacture stably and cheaply.

是的。

Yeah.

所以，创新的层面非常广泛。

So so the innovation stack is pretty broad.

你知道，这里有设备、光学、化学、物理、材料科学、冶金学。

You know, there's there's equipment, there's optics, there's chemistry, there's physics, there's material science, there's metallurgy.

当不同材料组合在一起时，它们如何相互作用，有很多想法。

There's lots of ideas about when you put different materials together, how do they interact.

它们稳定吗？

Are they stable?

它们在温度变化下稳定吗？

Are they stable over temperature?

比如，它们可重复吗？

You know, like, are they repeatable?

你知道，实际上涉及成千上万种技术。

You know, there's there's, like, literally thousands of technologies involved.

但就缩小尺寸而言，你并不认为我们已经接近基本极限了吧？

But just for the shrinking, you don't think we're quite yet close to the fundamental limits I of

我做过一个关于摩尔定律的演讲，当时我要求一份通往100的路线图。

did a talk on Moore's Law, I asked for a road map to a path of 100.

两周后，他们说我们只达到了50。

And after two weeks, they said we only got to 50.

100什么？

A 100 what?

抱歉。

Sorry.

100倍的缩小。

A 100 x shrink.

100倍的缩小？

A 100 x shrink?

我们只达到了

We only got to

50。

50.

我说，为什么不再给两周时间呢？

And I said, why don't you give it another two weeks?

关于摩尔定律，这里有个关键点。

Well, here's the thing about Moore's Law.

对吧？

Right?

所以我认为，未来十年或二十年的晶体管缩小趋势将继续发生。

So I believe that the next ten or twenty years of shrinking is gonna happen.

对吧？

Right?

作为一名计算机设计师，你有两种立场。

Now, as a computer designer, there's you have two stances.

一种是认为晶体管会继续缩小，因此你会以能够利用更多晶体管的方式设计和思考架构。

You think it's going to shrink, in which case you're designing and thinking about architecture in a way that you'll use more transistors.

另一种则是避免被所获得的大量晶体管带来的复杂性压垮。

Or conversely, not be swamped by the complexity of all the transistors you get.

对吧？

Right?

是的。

Right.

你必须有一个策略。

You have to have a strategy.

你知道的。

You know?

所以你愿意接受并等待一大批新晶体管准备好发挥作用的可能性。

So you're you're open to the possibility and waiting for the possibility of a whole new army of transistors ready to work.

我期待着，期待着。

I'm expecting Expecting.

每两三年晶体管数量就会增加，是的。

More transistors every two or three years Yeah.

增加的数量大到足以改变你对设计和架构的思考方式。

By a number large enough enough that how you think about design, how you think about architecture has to change.

想象一下，你用砖块建造砖房，而每年或每两年砖块的尺寸就减半。

Like, imagine you're you you build build brick buildings out of bricks, and every year the bricks are half the size or every two years.

如果你继续以相同的方式建造砖块，比如每人每天砌这么多砖，那么建造一栋建筑所需的时间将会呈指数级增长。

Well, if you kept building bricks the same way, you know, so many bricks per person per day, the amount of time to build a building would would go up exponentially.

对。

Right.

对吧？

Right?

但如果你预见到这一点，现在就设计出能更快搬运砖块、更高效利用砖块的设备，因为更小的砖块可能带来更强的强度、更薄的墙体，你知道的，节省材料，提升效率。

But if you said, I know that's coming, so now I'm going to design equipment that moves bricks faster, uses them better, because maybe you're getting something out of the smaller bricks, more strength, thinner walls, you know, less material, efficiency out of that.

所以，一旦你有了关于晶体管发展趋势的路线图——我们会拥有更多的晶体管。

So once you have a roadmap with what's gonna happen, transistors, they're gonna get we're gonna get more of them.

然后你就要围绕它设计所有配套的系统，以充分利用它，同时应对它带来的挑战。

Then you design all this collateral around it to take advantage of it and also to cope with it.

这就是人们不理解的地方。

Like, that's the thing people don't understand.

就好像，如果我不相信摩尔定律，而摩尔定律所预测的晶体管突然出现了，我的设计团队就会被彻底压垮。

It's like, if I didn't believe in Moore's Law and then Moore's Law transistors showed up, my design teams were all drowned.

所以什么是正确的？

So what's the Right.

这种新晶体管激增中最困难的部分是什么？

What's the hardest part of this influx of new transistors?

我的意思是，即使只回顾你职业生涯中的历史，当你在设计中增加更多晶体管时，真正改变的是什么？

I mean, even if you just look historically throughout your career, what's what's the thing that what fundamentally changes when you add more transistors in the task of designing

嗯，有两个不变的因素。

Well, there's there's two constants.

对吧？

Right?

一个是人们并没有变得更聪明。

One is people don't get smarter.

我认为，顺便说一下，有一些科学研究表明，由于营养等因素，我们确实变得更聪明了。

I think By the way, there's some science showing that we do get smarter because of nutrition, whatever.

是的。

Yeah.

抱歉提起这个。

Sorry to bring that up.

影响。

Effect.

是的。

Yes.

对。

Yeah.

我对此很熟悉。

I'm familiar with it.

没人理解它。

Nobody understands it.

没人知道它是否还在继续。

Nobody knows if it's still going on.

所以这就是

So that's a

或者它是否真实。

Or whether it's real or not.

但确实。

But yeah.

那是一个我有点儿总之。

That's a I I sort of Anyway.

但不是指数级的。

But not exponentially.

在大多数情况下，人们并没有变得更聪明。

Believe for the most part people aren't getting much smarter.

证据并不支持这一点。

The evidence doesn't support it.

那是而且

That's And

那么团队也无法增长那么多。

then teams can't grow that much.

对。

Right.

对。

Right.

所以人类在团队规模为10人左右，最多到100人的时候，表现得特别好。

So human beings under you know, we're really good in teams of 10, you know, up to teams of 100.

他们能彼此了解。

They can know each other.

超过这个规模，你就需要设立组织边界。

Beyond that, you have to have organizational boundaries.

所以你实际上面临着这些相当严格的限制。

So you're kind of you have those pretty hard constraints.

对吧？

Right?

所以你必须分而治之。

So then you have to divide and conquer.

随着设计规模变大，你必须把它分成若干部分。

Like, as the designs get bigger, you have to divide it into pieces.

你知道，抽象层的力量非常强大。

You know, the power of abstraction layers is really high.

我们过去是用晶体管来制造计算机的。

We used to build computers transistors.

现在，有一支团队把晶体管转化为逻辑单元，另一支团队把它们转化为功能单元，还有一支团队把它们组装成计算机。

Now we have a team that turns transistors into logic cells and another team that turns them into functional units, another one that turns into computers.

对吧？

Right?

所以我们这里也有抽象层。

So we have abstraction layers in there.

你必须思考什么时候该切换这种层级。

And you have to think about when do you shift gears on that.

我们还用更快的计算机来设计更快的计算机。

We also use faster computers to build faster computers.

所以有些算法在新电脑上运行速度会快一倍，但很多算法的时间复杂度是 n 的平方。

So some algorithms run twice as fast on new computers, but a lot of algorithms are n squared.

因此，晶体管数量多一倍的电脑，运行时间可能会变成四倍。

So a computer with twice as many transistors in it might take four times as long to run.

所以你必须重构软件。

So you have to refactor the software.

仅仅依靠更快的电脑来构建更大的电脑是行不通的。

Like simply using faster computers to build bigger computers doesn't work.

所以你必须考虑所有这些因素。

So So you have to think about all these things.

那么在计算性能方面，以及更强大电脑带来的令人兴奋的可能性中，我们刚才讨论的这种缩小是否是你眼中性能提升最重要的激动人心的方向？还是说你对其他方向也感兴趣？

So in terms of computing performance and the exciting possibility that more powerful computers bring, is shrinking the thing we've just been talking about one of the for you, one of the biggest exciting possibilities of advancement in performance, or is there other directions that you're interested in?

比如，像强制并行性，或者大规模并行——比如堆叠大量 CPU，这类并行方式，或者其他任何形式的并行性？

Like like in the direction of sort of enforcing given parallelism or, like, doing massive parallelism terms of many, many CPUs, you know, stacking CPUs on top of each other, that kind of that kind of parallelism or any kind of parallelism?

换个角度想想。

Think about it in a different way.

旧电脑，你知道的，慢的电脑，你说 a 等于 b 加上 c 乘以 d。

So old computers, you know, slow computers, you said a equal b plus c times d.

很简单。

Pretty simple.

对吧？

Right?

然后我们造出了带有向量单元的更快电脑，可以处理真正的方程和矩阵。

And then we made faster computers with vector units and you can do proper equations and matrices.

对吧？

Right?

而现代计算，比如卷积神经网络，你需要将一个大型数据集与另一个卷积运算。

And then modern, computations or like convolutional neural networks, where you convolve one large data set against another.

因此，数学呈现出一种层次结构，从简单方程到线性方程，再到矩阵方程，再到更深层的计算。

And so there's sort of this hierarchy of mathematics, from simple equation to linear equations to matrix equations to the deeper kind of computation.

数据集变得如此庞大，以至于人们开始把数据看作是一个拓扑问题。

And the data sets are getting so big that people are thinking of data as a topology problem.

数据以某种巨大的结构组织起来。

Data is organized in some immense shape.

而计算则试图从这种庞大的结构中获取数据并对其进行处理。

And then the computation, which sort of wants to be get data from immense shape and do some computation on it.

因此，计算机让人们能够将算法推得更远、更深入。

So the what computers have allowed people to do is have algorithms go much, much further.

所以你提到的那篇萨顿的论文。

So that that paper you you referenced, the Sutton paper

是的。

Mhmm.

他们谈到，当人工智能刚起步时，就是将规则集应用到某些东西上。

They talked about, you know, like, when AI started, it was apply rule sets to something.

这是一种非常简单的计算场景。

That's a very simple computational situation.

当他们第一次下国际象棋时，他们解决了深度搜索的问题。

And then when they did first chess thing, they they solved deep searches.

所以有一个移动和结果的数据库，进行深度搜索，但本质上仍然只是一种搜索。

So have database of moves and results, deep search, but it's still just a search.

对吧？

Right?

现在我们使用大量图像来训练这些卷积权重集合。

Now we we take large numbers of images, and we use it to train these weight sets that we convolve across.

这是一种完全不同的现象。

It's a completely different kind of phenomena.

我们称之为人工智能。

We call that AI.

现在他们正在推进下一代。

Now they're doing the next generation.

如果你仔细看，他们正在沿着这条数学曲线向上发展。

And if you look at it, they're going up this mathematical graph.

对吧？

Right?

然后，计算能力和数据集共同推动了这条数学曲线的上升。

And then computations both computation and data sets support going up that graph.

是的。

Yeah.

我所说的这种计算，我认为所有这些本质上仍然是一种搜索。

The kind of computation that might I mean, I would argue that all of it is still a search.

对吧？

Right?

就像你所说的，数据集的拓扑问题，你是在数据集中搜索有价值的信息，同时神经网络的实际优化也是一种搜索，我不知道该怎么形容。

Just like you said, a topology problem of datasets, you're searching the datasets for valuable data, and also the actual optimization of neural networks is a kind of search for the I don't know.

如果你

If you

曾经观察过识别猫的内部层，那并不是搜索。

had looked at the inner layers of finding a cat, it's not a search.

而是一系列无穷无尽的投影。

It's it's a set of endless projections.

所以，你知道，投影就像是这个手机的影子。

So, you know, projection is here's a shadow of this phone.

是的。

Yeah.

对吧？

Right?

然后你可以把那个影子投射到别的东西上，再把那个东西的影子投到其他东西上。

And then you can have a shadow of that onto something and a shadow on that of something.

如果你观察各层，会发现这一层实际上描述了尖耳朵、圆眼睛、毛茸茸的特征，但提取这些属性的计算过程并不是搜索。

If you look in the layers, you'll see this layer actually describes pointy ears and round eye ness and fuzziness and but the the computation to tease attributes is not search.

对。

Right.

我的意思是，推理部分可能是搜索，但训练过程不是搜索。

I mean Like the inference part might be search, but the training's not search.

好的。

Okay.

从技术上讲，在深度网络中，它们会查看各层，但甚至不知道其中的表示方式。

Well, technically And then in deep networks, they look at layers and they don't even know it's represented.

然而，如果你把各层去掉，它就无法工作了。

And yet, if you take the layers out, it doesn't work.

好的。

Okay.

所以

So

我认为这并不是搜索。

refer I don't think it's search.

好吧。

Alright.

不过你得去问问我的数学家，这到底是什么意思。

Well But you have to talk to my mathematician about what that actually is.

嗯，我们可能会有不同意见。

Well, we could disagree.

但这只是语义问题，想想看。

But the the it's just semantics, think.

这不对。

It's not Right.

但绝对不是

But it's certainly not

我认为这绝对不是语义问题，但是

Which I would say it's absolutely not semantics, but

好的。

Okay.

行吧。

Alright.

好吧，如果你非要这么讲的话。

Well, if you wanna go there.

所以对我来说，优化就是搜索，我们试图优化神经网络识别猫耳朵的能力。

So optimization to me is search, and we're trying to optimize the ability of a neural network to detect cat ears.

国际象棋和那个极其多维的十万维空间之间的区别完全不同，而神经网络正是在这样的空间中进行优化，这和国际象棋的数据库完全不是一回事。

And the difference between chess and the space, the incredibly multidimensional 100,000 dimensional space that, you know, networks are trying to optimize over is nothing like the chessboard database.

所以这是完全不同的东西。

So it's a totally different kind of thing.

好的。

Okay.

从这个意义上说，你可以说

In that sense, you could say

它失去了意义。

it loses meaning.

我能理解你为什么会这么说。

I can see how you you might say.

有趣的是，这介于给定的搜索空间和已找到的搜索空间之间。

If if you the funny thing is it's the difference between given search space and found search space.

对。

Right.

没错。

Exactly.

对。

Yeah.

也许这是另一种描述方式，那就是

Maybe that's a different way to describe That's

这么说真好。

a beautiful way to put it.

好的。

Okay.

但你提到的是，从支持这些运算的基本数学运算和架构、计算机硬件角度来看，你的看法是什么？

But you're saying what's your sense in terms of the basic mathematical operations and the architectures computer hardware that enables those operations?

你认为今天的CPU仍然是执行这些数学运算的核心部分吗？

Do you see the CPUs of today still being a really core part of executing those mathematical operations?

是的。

Yes.

这些操作，比如加法、减法、加载、存储、比较和分支。

Well, the operations, you know, to be add, subtract, load, store, compare, and branch.

这真是了不起。

It's it's remarkable.

所以有趣的是，计算机或晶体管这些基本构件，其底层是原子。

So it's it's interesting that the building blocks of, you know, computers or transistors, you know, under that atoms.

你有原子、晶体管、逻辑门、计算机，对吧，也就是计算机的功能单元。

So you got atoms, transistors, logic gates, computers, right, you know, functional units of computers.

从某种层面来说，数学的基本构件是加法、减法、乘法等，但数学所能描述的空间我认为几乎是无限的。

The building blocks of mathematics at some level are things like adds and subtracts and multiplies, but the space mathematics can describe is, I think, essentially infinite.

但运行算法的计算机仍然在做着相同的事情。

But the computers that run the algorithms are still doing the same things.

现在，一个特定的算法可能会说，我需要稀疏数据，或者需要32位数据，或者需要像卷积操作这样天然处理8位数据、将其相乘并以某种方式求和的操作。

Now a given algorithm might say, I need sparse data, or I need 32 bit data, or I need, you know, like a convolution operation that naturally takes eight bit data, multiplies it, sums it up a certain way.

因此，TensorFlow中的数据类型暗示了一组优化策略。

So the like the data types in TensorFlow imply an optimization set.

但当你深入查看计算机时，会发现其实是与门和或门在执行加法和乘法。

But when you go right down and look at the computers, it's AND and OR gates doing adds and multiplies.

就像这样，变化并不大。

Like, that hasn't changed much.

现在，量子研究人员相信他们将彻底改变这一点。

Now, the quantum researchers think they're going to change that radically.

还有一些人思考模拟计算，因为你看大脑，它似乎更偏向模拟方式。

And then there's people who think about analog computing because you look in the brain and it seems to be more analog ish.

你知道，也许存在一种更高效的方式来实现这一点。

You know, that maybe there's a way to do that more efficiently.

但我们现在的计算能力已经提升了百万倍。

But we have a million x on computation.

我不知道计算强度与实现数学抽象能力之间的关系。

And I don't know the reference the relationship between computational, let's say, intensity and ability to hit mathematical abstractions.

我不记得有谁描述过这一点，但就像你在人工智能中看到的那样，你从规则系统发展到简单搜索，再到复杂搜索，最后到深度搜索。

I don't know anybody's described that, but but just like you saw in AI, you went from rule sets to simple search to complex search to, say, found search.

就像需要多几个数量级的计算一样。

Like, are orders of magnitude more computation to do.

当我们再提升两个数量级时，我的朋友罗杰·吉多里说，每个数量级都会改变计算的本质。

And as we get the next two orders of magnitude, a friend Roger Gidori said, like every order of magnitude changes the computation.

它从根本上改变了计算所做的事情。

It fundamentally changes what the computation is doing.

对。

Yeah.

你知道那句表达吗？量的不同就是质的不同。

Oh, you know the expression, the difference in quantity is a difference in kind.

你知道，蚂蚁和蚁群之间的区别，对吧？

You know, the difference between ant and anthill, right?

或者神经元和大脑之间的区别。

Or neuron and brain.

你知道，有一个无法界定的临界点，当数量达到一定程度时，质量就发生了变化。

You know, there's there's indefinable place where quantity changed the quality.

对吧？

Right?

我们在数学中已经多次看到这种情况发生，而且我认为这种趋势还会继续。

And we've seen that happen in mathematics multiple times, and, you know, my my guess is it's gonna keep happening.

所以，在你的理解中，这确实是，如果你专注于低头缩小晶体管的话。

So in your sense, this is, yeah, if you focus head down and shrinking the transistor.

但这不仅仅是低头苦干。

Well, it's not just head down.

我们清楚地了解正在运行的软件栈和计算负载，也在思考：当你拥有一个PB级内存、需要以稀疏方式访问，并支持AI程序员所需的计算类型时，你该怎么做？

We're aware of the software stacks that are running and the computational loads, and we're kind of pondering what do you do with a petabyte of memory that wants to be accessed in a sparse way and have, you know, the kind of calculations AI programmers want.

所以这中间存在一种对话和互动，但当你深入到计算机芯片层面时，你会发现加法器、减法器、乘法器等等。

So there's a there's a dialogue and interaction, but when you go in the computer chip, you know, you find adders and subtractors and multipliers and

所以如果你拉远视角，就像你提到的雷·萨顿所说，过去几十年AI研究的大部分进展，其实只是利用了算力的提升，以及那些等待算力进步的简单算法。

So if you zoom out then with, as you mentioned, Ray Sutton, the idea that most of the development in the last many decades in AI research came from just leveraging computation and just simple algorithms waiting for the computation to improve.

软件工程师们有一个说法，叫作‘过早优化’的问题。

Well, software guys have a thing that they call it the problem of early optimization.

对。

Right.

所以如果你编写了一个大型软件栈，并且一开始就进行优化，那么你最先写的部分很可能并不是性能瓶颈。

So if you write a big software stack, and if you start optimizing, the first thing you write, the odds of that being the performance limiter is low.

但当你把整个系统跑起来后，能否通过优化正确的地方将速度提升两倍？

But when you get the whole thing working, can you make it 2x faster by optimizing the right things?

当然可以。

Sure.

在你优化的过程中，你有没有可能

While you're optimizing that, could you have

直接开发一个新的软件栈，从而做出更好的选择？

written a new software stack, which would have been a better choice?

也许吧。

Maybe.

现在你面临了创造性张力。

Now you have creative tension.

所以，在你编写的过程中，我们所讨论的正是这个软件。

So But the whole time as you're doing the writing, the that's the software we're talking about.

底层的硬件变得越来越快

The hardware underneath gets faster

越来越快。

and faster.

这又回到了摩尔定律。

Goes back to the Moore's Law.

如果摩尔定律继续下去，你的AI研究就应该预期到这一点，并据此做出与我们已触碰瓶颈、什么都不会发生时不同的选择。

If Moore's Law is gonna continue, then your AI research should expect that to show up, and then you make a slightly different set of choices than we've hit the wall, nothing's gonna happen.

从这里开始，我们只是在重写算法。

And from here, it's just us rewriting algorithms.

这看起来像是过去三十年摩尔定律走向终结时的一种失败策略。

Like, that seems like a failed strategy for the last thirty years of Moore's Law's death.

所以，你能再详细说说吗？

So So can you just linger on it?

我觉得你已经回答了，但我还是想一遍又一遍地问这个简单的问题。

I I think you've answered it, but I just I'll just ask the same dumb question over and over.

那么，你为什么认为摩尔定律不会消亡呢？

So why do you think Moore's Law is not going to die?

在未来五到十年内，最有可能、最令人兴奋的摩尔定律持续存在的理由是什么？

Which is the most promising exciting possibility of why it won't die in the next five, ten years?

那么，是晶体管的持续缩小，还是会出现另一条完全不同的S型曲线呢？

So is it the continued shrinking the transistor, or is it another s curve that steps in in a totally sort of Well,

缩小晶体管实际上包含了成千上万项创新。

shrinking the transistor is literally thousands of innovations.

对。

Right.

所以，这背后有一整套

So there's So this there's a whole

许多S型曲线正在各自推进、被重新发明，并不断涌现新事物。

bunch of s curves just kind of running their course and and being reinvented and new things.

你知道，半导体制造商和技术人员都宣布了所谓的纳米线技术，他们将原本带有栅极的鳍状结构改成了细小的导线，从而实现了更好的控制，并且尺寸更小。

You know, the the semiconductor fabricators and technologists have all announced what's called nanowires, so they took a fin which had a gate around it and turned that into little wires so you have better control of that and they're smaller.

从那里开始，关于如何进一步缩小这些结构，有一些显而易见的路径。

And then from there, there are some obvious steps about how to shrink that.

因此，围绕导线堆叠的冶金技术有着非常明显的缩小潜力。

So the metallurgy around wire stacks and stuff has very obvious abilities to shrink.

而且，那里还有许多可以结合使用的整体方案。

And, you know, there's a whole combination of things there to do.

你的感觉是我们会从中获得大量进展，是的。

Your sense is that we're gonna get a lot Yeah.

仅靠这种缩小所带来的创新就已经很多了。

If this innovation from just that shrinking.

是的。

Yeah.

像提升一百倍这样的幅度，已经非常多。

Like a factor of a 100 is a lot.

是的。

Yeah.

我会说。

I would say.

这太不可思议了。

That's incredible.

而且这完全是未知的。

And it's totally unknown.

只有十年或十五年。

It's only ten or fifteen years.

你现在更聪明了，也许你知道，但对我来说，这100倍的提升会带来怎样的计算方式变革，完全是不可预测的。

Now you're smarter, you might know, but to me it's totally unpredictable of what that 100x would bring in terms of the nature of the computation that people would be

是的。

Yeah.

你熟悉贝尔定律吗？

Are you familiar with Bell's Law?