杨立昆：深度学习、卷积神经网络与自监督学习

什么图灵奖之类的荣誉都不算什么。

Turing Award, whatever.

你用Lisp写的这些程序才是真本事。

This what you programmed in Lisp.

这简直是

That's

它至今还是我最喜欢的编程语言。

It's still my favorite language.

不过我们并不是用Lisp语言编程，而是得自己编写Lisp解释器。

But it's not that we programmed in Lisp, it's that we had to write our Lisp interpreter.

因为那时候根本不存在什么现成的

Because it's not like

我们能用的

we used

成熟解释器。

one that existed.

所以我们编写了一个Lisp解释器，并将其连接到我们自己开发的后端库，用于神经网络计算。

So we wrote a Lisp interpreter that we hooked up to a backend library that we wrote also for sort of neural net computation.

几年后，大约在1991年，我们提出了一个想法：让各个模块具备前向传播和反向传播梯度的能力，然后将这些模块以图的形式连接起来。

And then after a few years around 1991, we invented this idea of basically having modules that know how to forward propagate and back propagate gradients, and then interconnecting those modules in a graph.

莱昂·布托在八十年代末就提出过类似的想法，而我们能够用我们的Lisp系统实现它。

Leon Botou had made proposals about this in the late eighties and we were able to implement this using our Lisp system.

最终，我们希望用这个系统在贝尔实验室开发用于字符识别的生产级代码。

Eventually we wanted to use that system to make, build production code for character recognition at Bell Labs.

我们实际上为这个Lisp解释器编写了一个编译器，帕特里夏·西马德——现在在微软——与莱昂和我一起主要完成了这项工作。

We actually wrote a compiler for that Lisp interpreter so that Patricia Simard, who's now Microsoft kind of did the bulk of it with Leon and me.

这样我们就可以用Lisp编写系统，然后编译成CE代码，从而得到一个独立完整的系统，能够完成整个任务。

So we could write our system in Lisp and then compile to CE, and then we'll have a self contained complete system that could kind of do the entire thing.

直到今天，PyTorch和TensorFlow都还做不到这一点。

Neither PyTorch nor TensorFlow can do this today yet.

好的。

Okay.

快了。

It's coming.

我的意思是，PyTorch 里有个叫 TorchScript 的东西，差不多是那样的。

I mean, there's something like that in PyTorch called TorchScript.

我们得写所有的解释器，得写整个编译器，投入了巨大的精力才做到这一点。

And we had to write all this meter player, had to write all this compiler, we had to invest a huge amount of effort to do this.

如果不是完全相信这个理念，没人会愿意花时间去做这种事。

Not everybody, if you don't completely believe in the concept, you're not going to invest the time to do this.

当时也好，现在也好，这都会演变成 Torch、PyTorch、TensorFlow 之类的工具。

Now at the time also, or today this would turn into Torch or PyTorch or TensorFlow or whatever.

我们把它开源了。

We put it in open source.

大家都用，然后就会发现它确实很好。

Everybody would use it and realize it's good.

在1995年之前，我在AT&T工作时，律师根本不可能允许我们发布这种性质的开源代码。

Back before 1995, working at AT and T, there's no way the lawyers would let you release anything in open source of this nature.

所以我们实际上无法分发我们的代码。

And so we could not distribute our code really.

关于这一点，抱歉扯得太远了，但我还读到过，贝尔实验室曾对卷积神经网络申请过类似专利的东西。

And on that point, and sorry to go on a million tangents, but on that point, I also read that there was some almost pat like a patent on Convolutional Neural Networks at Bell Labs.

是的，首先，我的意思是，

Yes, first of all, I mean,

实际上有两个。

just There's two, actually.

那个专利过期了，幸好如此，

That ran out, that's Thankfully,

在2007年。

in 2007.

在2007年。

In 2007.

我们能稍微聊一下这个吗？

What, can we just talk about that for a second?

我知道你在Facebook，但你也在纽约大学，那么对这类软件想法申请专利到底意味着什么？

I know you're at Facebook, but you're also at NYU, and what does it mean to patent ideas like these software ideas, essentially?

或者什么是数学想法？

Or what are mathematical ideas?

或者它们到底是什么？

Or what are they?

好的。

Okay.

所以这些并不是数学想法。

So they're not mathematical ideas.

所以确实存在一些算法。

So there are, you know, algorithms.

曾经有一段时间，美国专利局允许对软件进行专利保护，只要它被具体实现。

And there was a period where the US Patent Office would allow the patent of software as long as it was embodied.

欧洲人则非常不同。

The Europeans are very different.

他们并不完全接受这一点。

They don't quite accept that.

他们有不同的理念，但你知道，我其实从未真正强烈相信过这种观点，我现在也不相信这种类型的专利。

They have a different concept, but you know, I don't, I no longer, I mean, I never actually strongly believed in this, but I don't believe in this kind of patent.

Facebook 基本上不相信这种类型的专利。

Facebook basically doesn't believe in this kind of patent.

Google 申请专利是因为他们曾被苹果公司伤害过。

Google files patents because they've been burned with Apple.

所以现在他们这么做主要是出于防御目的，但通常他们会说，只要你不侵犯我们的专利，我们就不会起诉你。

And so now they do this for defensive purpose, but usually they say, we're not going to sue you if you infringe.

Facebook 也有类似的政策。

Facebook has a similar policy.

他们说，我们会为某些技术申请专利，主要是为了防御目的。

They say, you know, we file patents on certain things for defensive purpose.

只要你不起诉我们，我们也不会因为你侵犯专利而起诉你。

We're not going to sue you if you infringe unless you sue us.

所以这个行业并不相信专利。

So the industry does not believe patents.

它们存在是因为法律环境和其他各种因素。

They're there because of, you know, the legal landscape and various things.

但我真的不认为这类东西应该有专利。

But I don't really believe in patents for this kind of stuff.

好的。

Okay.

这真是件好事。

So that's a great thing.

所以我来告诉你一个更糟糕的故事。

So I- I'll tell you a worst story actually.

当时第一个关于卷积网络的专利，是关于卷积网络的早期版本，它没有独立的池化层。

So what happens was the first pattern about convolutional net was about kind of the early version of convolutional net that didn't have separate pooling layers.

它只有卷积层，这些层试图做更多的事情，如果你愿意这么说的话。

It had convolutional layers which tried more than one, if you want.

对吧？

Right?

然后第二个是带有独立池化层的卷积网络，使用反向传播进行训练。

And then there was a second one on convolutional nets with separate pooling layers, train with back prop.

在1989年和1999年左右，有人提交了相关专利申请。

And there were files filed in '89 and ninety nine years, something like this.

当时，专利的有效期是十七年。

At the time, the life of a patent was seventeen years.

所以在接下来的几年里，我们开始基于卷积网络开发字符识别技术。

So here's what happened over the next few years is that we started developing character recognition technology around convolutional nets.

1994年，支票识别系统被部署在自动取款机中。

And in 1994, a check reading system was deployed in ATM machines.

1995年，它被用于银行后台的大规模支票识别设备等。

In 1995, it was for large check reading machines in back offices, etcetera.

这些系统是由我们与AT&T合作的工程团队开发的。

And those systems were developed by an engineering group that we were collaborating with at AT and T.

这些系统由NCR公司商业化，当时NCR是AT&T的子公司。

And they were commercialized by NCR, which at the time was a subsidiary of AT and T.

AT&T在1996年初分拆了。

Now AT and T split up in 1996, early nineteen ninety six.

律师们审查了所有专利，并将专利分配给各个公司。

And the lawyers just looked at all the patents and they distributed the patents among the various companies.

他们把卷积网络的专利给了NCR，因为NCR确实在销售使用这项技术的产品，但NCR内部没人知道卷积网络是什么。

They gave the convolutional net patent to NCR because they were actually selling products that used it, but nobody at NCR had any idea what a convolutional net was.

是的。

Yeah.

好的。

Okay.

所以从1996年到2007年，有一整个时期直到2002年，我实际上没有从事机器学习或卷积网络的工作。

So between 1996 and 2007, so there's a whole period until 2002 where I didn't actually work on machine learning or convolutional net.

我大约在2002年重新开始研究这个领域。

I resumed working on this around 2002.

在2002年到2007年之间，我一直在研究它们，心里祈祷NCR的人别发现，结果他们真的没发现。

And between 2002 and 2007, I was working on them, crossing my finger that nobody at NCR would notice, and nobody noticed.

是的。

Yeah.

我希望，正如你所说，撇开律师不谈，如今这个社区相对开放的氛围能够持续下去？

And I and I hope that this kind of somewhat, as you said, lawyers aside, relative openness of the community now will continue?

这加速了整个行业的发展。

It accelerates the entire progress of the industry.

你知道，Facebook、Google和其他公司今天面临的问题，并不是Facebook、Google、微软或IBM谁领先谁。

And, you know, the problems that Facebook and Google and others are facing today is not whether Facebook or Google or Microsoft or IBM or whoever is ahead of the other.

而是我们还没有技术来实现我们想构建的东西。

It's that we don't have the technology to build these things we want to build.

我们想打造具有常识的智能虚拟助手。

We want to build intelligent virtual assistants that have common sense.

我们对这方面的优秀想法并没有垄断。

We don't have monopoly on good ideas for this.

我们不认为我们有。

We don't believe we do.

也许别人认为他们有，但我们没有。

Maybe others believe they do, but we don't.

好的。

Okay.

如果一家初创公司告诉你，他们掌握了实现人类水平智能和常识的秘诀，别信他们。

If a startup tells you they have the secret to, you know, human level intelligence and common sense, don't believe them.

他们没有。

They don't.

要达到那个阶段，让各个公司都能基于此开始构建东西，还需要全球研究界共同努力一段时间。

And it's going to take the entire work of the world research community for a while to get to the point where you can go off and in each of those companies kind of start to build things on this.

我们还没到那一步。

We're not there yet.

所以确实如此。

So absolutely.

这正体现了你经常提到的理念与实际验证之间的差距。

And this this calls to the the gap between the space of ideas and the rigorous testing of those ideas of practical application that you often speak to.

你曾建议过：不要被那些声称已解决通用人工智能、声称拥有像人脑一样工作的AI系统，或声称已破解大脑工作原理的人所迷惑。

You've written advice saying, don't get fooled by people who claim to have a solution to artificial general intelligence, who claim to have an AI system that work just like the human brain, or who claim to have figured out how the brain works.

问问他们在MNIST或ImageNet上的错误率是多少。

Ask them what the error rate they get on MNIST or ImageNet.

是的。

Yeah.

不过，这有点过时了。

This is a little dated, by the way.

我的意思是，五年前。

Two I mean, five years.

对。

Yes.

谁还在乎呢？

Who's counting?

好的。

Okay.

但我认为你的观点，即MNIST和ImageNet可能已经过时了，仍然是对的。

But I think your opinion is still MNIST and ImageNet, yes, may be dated.

可能会有新的基准测试。

There may be new benchmarks.

对吧？

Right?

但我认为这种理念是，你仍然坚持认为，基准测试和实际应用才是真正检验这些想法的地方。

But I think that philosophy is when you still and and and somewhat hold that benchmarks and the practical testing, the practical application is where you really get to test the ideas.

但也许这并不完全实用。

Well, it may not be completely practical.

比如，它可能是一个玩具数据集，但必须是整个社区普遍接受的某种标准基准任务，如果你想要的话。

Like, for example, know, it could be a toy dataset, but it has to be some sort of task that the community as a whole has accepted as some sort of standard benchmark, if you want.

它不需要是真实的。

It doesn't need to be real.

比如很多年前，在公平会上，杰森·韦斯顿和其他一些人提出了婴儿任务，这是一类玩具问题，用于测试机器推理和访问工作记忆等能力。

So for example, many years ago here at fair, people, know, Jason Weston, and few others proposed the, the baby tasks, which were kind of a toy problem to test the ability of machines to reason actually to access working memory and things like this.

尽管这不是一个真实任务，但它非常有用。

And it was very useful, even though it wasn't a real task.

MNIST 算是介于真实任务和非真实任务之间的一种。

MNIST is kind of halfway real task.

玩具问题可能非常有用。

Toy problems can be very useful.

我只是特别注意到，很多人，尤其是那些有资金投资的人，会被别人忽悠，说‘我们掌握了皮层的算法，你应该给我们五千万’。

It's just that I was really struck by the fact that a lot of people, particularly a lot of people with money to invest would be fooled by people telling them, oh, we have, you know, the algorithm of the cortex, and you should give us 50,000,000.

是的。

Yes.

当然。

Absolutely.

确实有很多人试图利用这种炒作来谋取商业利益。

So there's a lot of people who who try to take advantage of the hype for business reasons and so on.

但让我谈谈这个观点：推动领域前进的新想法，可能还没有相应的基准测试。

But let me sort of talk to this idea that the new ideas, ideas that push the field forward, may not yet have a benchmark.

或者建立基准测试可能非常困难。

Or it may be very difficult to establish a benchmark.

我同意。

I agree.

这是过程的一部分。

That's part of the process.

建立基准测试是这个过程的一部分。

Establishing benchmarks is part of the process.

那么，你对这些基准测试有什么看法？我们现在有围绕图像的基准测试，从分类到描述，到从图像表面提取各种信息。

So, what are your thoughts about so we have these benchmarks on, around stuff we can do with images, from classification to captioning, to just every kind of information you can pull off from images in the surface level.

还有音频数据集。

There's audio datasets.

有一些视频数据。

There's some video.

我们能从自然语言开始，关注哪些类型的东西、哪些基准，逐渐向更像智能、推理这样的方向发展，也许你不喜欢这个词，但类似于AGI的那些影子

What can we start, natural language, what kind of stuff, what kind of benchmarks do you see that start creeping on to more something like intelligence, like reasoning, like, maybe you don't like the term, but AGI echoes of that kind of

是的

Yeah.

许多人在研究交互式环境，在这些环境中可以训练和测试智能系统。

So a lot of people are working on interactive environments in which you can train and test intelligent systems.

例如，经典的监督学习范式是：你有一个数据集，将其划分为训练集、验证集和测试集，并且有明确的协议。

So there, for example, you know, it's the classical paradigm of supervised learning is that you have a dataset, you partition it into a training set, validation set, test set, and there's a clear protocol.

但传统数据假设样本是统计独立的，可以互换，你看到它们的顺序无关紧要；但如果你的回答决定了你接下来看到的样本——比如在机器人领域就是这种情况呢？

But what if data assumes that the samples are statistically independent, you can exchange them, the order in which you see them doesn't, shouldn't matter, things like But what if the answer you give determines the next sample you see, which is the case, for example, in robotics, right?

你的机器人做了一件事，然后进入一个新房间，而它去的地方不同，房间也会不同。

You robot does something and then it gets exposed to a new room and depending on where it goes, the room would be different.

这就带来了探索问题。

So that creates the exploration problem.

这也导致了样本之间的依赖性。

So that creates also a dependency between samples.

如果你只能在空间中移动，那么你接下来看到的样本很可能还是在同一个建筑里。

If you can only move in space, the next sample you're going to see is going to be probably in the same building, most likely.

因此，只要机器的行为会对世界产生影响，所有关于训练集和测试集假设的有效性就会被打破。

So all the assumptions about the validity of this training set test set hypothesis break whenever a machine can take an action that has an influence in world.

而这正是它将看到的内容。

And it's what it's going to see.

所以人们正在构建这样的虚拟环境，对吧？

So people are setting up artificial environments where that takes place, right?

机器人会在房屋的三维模型中四处移动，并与物体进行交互，诸如此类。

The robot runs around a three d model of a house and can interact with objects and things like this.

所以你进行的是基于模拟的机器人研究。

So you do robotics based simulation.

你有那些类似健身房的环境，或者Mujoco这样的模拟机器人，还有各种游戏之类的。

You have those, you know, opening a gym type thing, or Mujoco kind of simulated robots and you have games, you know, things like that.

所以这个领域真正的发展方向就是这种类型的环境。

So that's where the field is going really, this kind of environment.

现在回到AGI的问题上。

Now back to the question of AGI.

我不喜欢AGI这个术语，因为它暗示人类智力是通用的，而人类智力根本不是通用的。

Like I don't like the term AGI because it implies that human intelligence is general and human intelligence is nothing like general.

它非常、非常专门化。

It's very, very specialized.

我们以为它是通用的。

We think it's general.

我们喜欢认为自己拥有通用智能。

We like to think of ourselves as having general intelligence.

我们并没有。

We don't.

我们非常专门化。

We're very specialized.

我们只是比……稍微更通用一点

We're only slightly more general than

为什么它会感觉是通用的？

Why does it feel general?

所以你对‘通用’这个术语有点误解。

So you kind of the term general.

我认为人类令人印象深刻的是学习能力，就像我们之前讨论的那样，能够在如此多不同的领域中学习。

I think what's impressive about humans is the ability to learn, as we were talking about learning, to learn in just so many different domains.

它或许不是无限通用的，但你确实能在多个领域学习并以某种方式整合这些知识。

It's perhaps not arbitrarily general, but just you can learn in many domains and integrate that knowledge somehow.

好的。

Okay.

知识是持续存在的。

The knowledge persists.

让我举一个具体的例子。

So let me take a very specific example.

对。

Yes.

这算不上是一个实例。

It's not an example.

它更像是一种准数学层面的论证。

It's more like a quasi mathematical demonstration.

人的一只眼睛里会延伸出大约一百万根神经纤维。

So you have about 1,000,000 fibers coming out of one of your eyes.

两只眼睛加起来总共有两百万根，不过我们先只聊其中一只眼睛的情况。

2,000,000 total, but let's talk about just one of them.

你的视神经包含整整一百万条神经纤维。

It's 1,000,000 nerve fibers, your optical nerve.

我们假设这些信号都是二进制的，也就是说它们要么处于激活状态，要么处于未激活状态。

Let's imagine that they are binary, so they can be active or inactive.

那么进入你视觉皮层的输入数据量就是一百万比特。

So the input to your visual cortex is 1,000,000 bits.

现在这些输入会以特定的方式接入你的大脑，而大脑里的连接有点像卷积神经网络——它们在空间上是局部化的，还有诸如此类的特性。

Now they connect it to your brain in a particular way, and your brain has connections that are kind of a little bit like a convolutional net, they kind of local in space and things like this.

现在我来跟你玩个把戏。

Now imagine I play a trick on you.

我承认，这是个相当恶劣的把戏。

It's a pretty nasty trick I admit.

我切断你的视神经，然后安装一个设备，对所有神经纤维进行随机排列重组。

I cut your optical nerve and I put a device that makes a random perturbation of a permutation of all the nerve fibers.

现在传入你大脑的信息，是所有像素的一个固定但随机的排列。

So now what comes to your brain is a fixed but random permutation of all the pixels.

无论我是在你婴儿时期就对你这么做，你的视觉皮层都不可能再达到你原本的视觉质量水平。

There's no way in hell that your visual cortex, even if I do this to you in infancy, will actually run vision to the same level of quality that you can.

明白了。

Got it.

你是说，你根本不可能重新学会这个吗？

And you're saying there's no way you've relearned that?

不可能。

No.

因为现在现实中相邻的两个像素会在你的视觉皮层中出现在完全不同的位置。

Because now two pixels that are nearby in the world will end up in very different places in your visual cortex.

而那里的神经元彼此之间没有连接，因为它们只与邻近的神经元相连。

And your neurons there have no connections with each other because they're only connected locally.

所以我们的整个硬件在很多方面都是为了支持

So this whole our entire the hardware is built in many ways to support

现实世界的局部性。

The locality of the real world.

是的。

Yeah.

对。

Yes.

这就是特化。

That's specialization.

是的。

Yeah.

但这仍然相当令人印象深刻。

But it's still pretty damn impressive.

所以这并不是完美的泛化。

So it's not perfect generalization.

甚至差得远。

It's it's not even close.

不。

No.

不。

No.

并不是说它差得远。

It's it's it's It's not that it's not even close.

根本不是。

It's not at all.

是的，不是。

Yes, not.

这是专门化的。

It's specialized.

是的。

Yeah.

那么有多少个布尔函数？

So how many Boolean functions?

所以，让我们想象一下，你希望训练你的视觉系统来识别这100万个比特的特定模式。

So let's imagine you want to train your visual system to recognize particular patterns of those 1,000,000 bits.

好的。

Okay.

所以这是一个布尔函数，对吧？

So that's a Boolean function, right?

要么这个模式存在，要么不存在。

Either the pattern is here or not here.

这是一个具有100万个二进制输入的二分类问题。

There's a two way classification with 1,000,000 binary inputs.

有多少这样的布尔函数？

How many such Boolean functions are there?

好的。

Okay.

你有2的100万次方种输入组合。

You have two to the 1,000,000 combinations of inputs.

对于每一种组合，你都有一个输出位。

For each of those you have an output bit.

因此，这类布尔函数有2的100万次方个，这是一个难以想象的巨大数字。

And so you have two to the 1,000,000 Boolean functions of this type, which is an unimaginably large number.

你的大脑皮层实际能计算出其中多少个函数？

How many of those functions can actually be computed by your usual cortex?

答案是极小极小极小极小极小的一小部分，简直微乎其微。

The answer is a tiny, tiny, tiny, tiny, tiny sliver, like an enormously tiny sliver.

是的。

Yeah.

所以我们极其专业化。

So we are ridiculously specialized.

你知道的。

You know?

好吧。

Okay.

但好吧。

But okay.

这是对‘通用’这个词的一个反驳。

That's an argument against the word general.

我觉得这里有点重复，我同意你的直觉，但我并不确定，大脑似乎具有令人印象深刻的适应能力。

I think there's there's a I I there's I agree with your intuition, but I'm not sure it's it seems the the brain is impressively capable of adjusting to things.

所以

So

这是因为我们无法想象超出我们能力范围的任务，对吧？

It's because we can't imagine tasks that are outside of our Right?

是的。

So yeah.

所以我们认为

So we think

我们认为自己是通用的，因为我们能理解所有我们能感知到的事物。

we think we are general because we're general of all the things that we can apprehend.

哦，是的。

Oh, so yeah.

但外面还有一个我们完全不了解的广阔世界。

But there is a huge world out there of things that we have no idea.

顺便说一下，我们称那个为热量。

We call that heat, by the way.

热量。

Heat.

热量。

Heat.

所以至少物理学家称其为热，或者称之为熵，你知道的，就是你有一个充满气体的容器。

So at least physicists call that heat or they call it entropy, which is you know, you have a thing full of gas.

对吧？

Right?

密闭的气体系统。

Closed system for gas.

对吧？

Right?

无论是密闭还是非密闭的。

Closed or not closed.

它具有，你知道的，压强。

It has, you know, pressure.

它具有温度。

It has temperature.

它还具有，你知道的，你可以写出像 PV 等于 nRT 这样的方程，诸如此类的东西。

It has, you know, and you can write equations PV equal NRT, you know, things like that.

对吧？

Right?

当你减小体积时，温度会上升，压力也会上升，诸如此类。

When you reduce the volume, the temperature goes up, the pressure goes up, know, things like that.

对吧？

Right?

至少对于理想气体来说是这样。

For perfect gas, at least.

这些是你能够了解的关于这个系统的特性。

Those are the things you can know about that system.

与整个系统状态的完整信息相比，这仅仅是极少极少的比特数，因为整个系统状态会给出气体中每个分子的位置和动量。

And it's a tiny, tiny number of bits compared to the complete information of the state of the entire system, because the state of the entire system will give you the position of momentum of every molecule of the gas.

而你对它不了解的部分就是熵，你将其解释为热。

And what you don't know about it is the entropy and you interpret it as heat.

那个系统中包含的能量就是我们所说的热。

The energy contained in that thing is what we call heat.

事实上，这些分子的运动方式可能存在非常强的结构。

Now it's very possible that, in fact, there is some very strong structure in how those molecules are moving.

只是这种结构的方式是我们根本无法感知的。

It's just that they are in a way that we are just not wired to perceive.

是的，我们对它一无所知。

Yeah, we're ignorant to it.

在我们无法感知的无限多事物中，还有更多。

And there's, in your infinite amount of things we're not wired to perceive.

你说得对，这样表达真不错。

And you're right, that's a nice way to put it.

我们能想象的一切只是所有可能性中极其微小的一部分。

We're general to all the things we can imagine, which is a very tiny subset of all things that are possible.

所以这就像复杂性的共同核心，或者说是复杂性的共同核心。

So it's like common core of complexity or the common core of complexity.

对。

Yeah.

你知道，每一个比特串或每一个整数都是随机的，除了那些你实际上能写出来的。

You know, every bit string or every integer is random, except for all the ones that you can actually write down.

是的。

Yeah.

好的。

Okay.

说得真好。

So beautifully put.

但你知道，我们完全可以称之为人工智能。

But, you know, so we could just call it artificial intelligence.

我们不需要拥有一个通用的

We don't need to have a general

或者人类水平的。

Or human level.

人类水平的智能是

Human level intelligence is a

你知道，只要你一涉及到人类，事情就会变得有趣，因为归根结底，是我们自己把情感投射到了人类身上，而很难准确定义什么是人类智能。

good You know, you'll start anytime you touch human, it interesting because, you know, it's it's just because we attach ourselves to human, and it's difficult to define what human intelligence is.

是的。

Yep.

不过，我的定义可能是极其惊人的智能。

Nevertheless, my definition is maybe damn impressive intelligence.

明白吗？

Okay?

无论怎样，这都是一次令人印象深刻的智能展示。

Demo impressive demonstration of intelligence, whatever.

关于这个话题，深度学习的大多数成功都来自于监督学习。

And so, on that topic, most successes in deep learning have been in supervised learning.

什么？

What

你对无监督学习有什么看法？

is your view on unsupervised learning?

有没有可能减少人为干预，同时仍能开发出具有实际应用价值的成功系统？

Is there a hope to reduce involvement of human input and still have successful systems that are have practical use?

是的。

Yeah.

我的意思是，这确实是有希望的。

I mean, there's definitely a hope.

这甚至不止是希望。

It's more than a hope actually.

目前已经有越来越多的证据支持这一点。

It's mounting evidence for it.

而这基本上就是我所有的工作。

And that's basically all I do.

目前我唯一感兴趣的是我称之为自监督学习的东西，而不是无监督学习。

Like, the only thing I'm interested in at the moment is I call it self supervised learning, not unsupervised.

因为‘无监督学习’这个术语带有太多歧义。

Because unsupervised learning is a loaded term.

懂机器学习的人会告诉你，你在做聚类或主成分分析，但其实并不是这样。

People who know something about machine learning, you know, tell you, so you're doing clustering or PCA, which is not the case.

而普通公众一听‘无监督学习’，天啊，机器能自己学习，完全不需要人为干预。

And the white public, you know, you say unsupervised learning, oh my God, you know, machines are going learn by themselves and without supervision.

你知道，他们看到这个

You know, they see this

那——父母在哪？

as- Where's the parents?

是的。

Yeah.

所以我称之为自监督学习，因为实际上所使用的底层算法与监督学习的算法是一样的，只不过我们训练它们的目标不是预测特定的变量，比如图像的类别，也不是预测由人类标注者提供的变量。

So, so I call it self supervised learning because in fact, the underlying algorithms that are used are the same algorithms as the supervised learning algorithms, except that what we train them to do is not predict a particular set of variables like the category of an image and not to predict a set of variables that have been provided by human labelers.

而是训练机器去重建其输入中被遮蔽的部分。

But what you train the machine to do is basically reconstruct a piece of its input that is being masked out essentially.

你可以这样理解，对吧？

You can think of it this way, right?

所以给机器展示一段视频，然后让它预测接下来会发生什么。

So show a piece of video to machine and ask it to predict what's going to happen next.

当然，过一段时间后你可以展示实际发生的情况，机器就会逐渐学会更好地完成这个任务。

And of course, after a while you can show what happens and the machine will kind of train itself to do better at that task.

如今所有在自然语言处理领域最先进、最成功的模型都使用自监督学习，比如BERT这类系统，对吧？

You can do like all the latest, successful models in natural language processing use self supervised learning, you know, sort of BERT style systems, for example, right?

你向它展示一个包含一千个词的文本窗口。

You show it a window of a thousand words on a test corpus.

你去掉其中15%的词语，然后训练机器去预测那些缺失的词。

You take out 15% of the words, and then you train a machine to predict the words that are missing.

这就是自监督学习。

That's self supervised learning.

它并不是在预测未来。

It's not predicting the future.

它只是，你知道的，预测中间的内容，但你也可以让它去预测未来。

It's just, you know, predicting things in the middle, but you could have it predict the future.

这就是语言模型所做的。

That's what language models do.

你以一种无监督的方式构建了一个语言模型。

You construct, so in an unsupervised way, you construct a model of language.

你觉得

Do you think

或者视频、物理世界，或者

Or video or the physical world or

不管怎样，你觉得这能走多远？

How whatever, far do you think that can take us?

你觉得AI真的理解什么吗？

Do you think Art understands anything?

在某种程度上，它对文本有浅层的理解，但要真正达到人类水平的智能，我认为必须将语言与现实联系起来。

To some level, it has, you know, a shallow understanding of text, but it needs to mean, to have kind of true human level intelligence, I think you need to ground language in reality.

所以有些人正在尝试这样做，对吧？

So some people are attempting to do this, right?

拥有能够可视化所讨论内容的系统，这正是为什么你需要那些交互式环境的原因之一。

Having systems that kind of have some visual representation of what is being talked about, which is one reason you need those interactive environments actually.

但这是一项巨大的技术难题，尚未解决，这也解释了为什么自监督学习在自然语言领域有效，但在图像识别和视频领域却效果不佳（尽管进展迅速）。

But it's like a huge technical problem that is not solved and that explains why self supervised learning works in the context of natural language, but does not work in the context, or at least not well, in the context of image recognition and video, although it's making progress quickly.

原因在于，在自然语言的语境中表达预测的不确定性，比在视频和图像等场景中容易得多。

And the reason, that reason is the fact that it's much easier to represent uncertainty in the prediction, in the context of natural language than it is in the context of things like video and images.

例如，如果我让你预测被删掉的15%的单词缺失了什么内容。

So for example, if I ask you to predict what words are missing, you know, 15% of the words that are taken out.

可能性是有限的。

The possibilities are small.

这意味着

That means

范围很小。

It's small.

对吧？

Right?

词典中有十万词汇，而机器输出的是一个巨大的概率向量。

There is a 100,000 words in the the lexicon, and what the machine spits out is a a big probability vector.

对吧？

Right?

这是一堆介于零和一之间的数字，一一对应。

It's a bunch of numbers between zero and one that's one to one.

我们知道如何用计算机来实现这一点。

And we know how to do this with computers.

所以

So

它们在预测中表示不确定性相对容易。

they are representing uncertainty in the prediction is relatively easy.

在我看来，这就是这些技术在自然语言处理中有效的原因。

And that's in my opinion, why those techniques work for NLP.

对于图像，如果你遮住图像的一部分并要求系统重建该部分，会有许多可能的答案。

For images, if you ask, if you block a piece of an image and you ask the system reconstruct that piece of the image, there are many possible answers.

它们都是完全合理的，对吧？

They are all perfectly legit, right?

那么，你如何表示这一组可能的答案呢？

And how do you represent that, this set of possible answers?

你无法训练一个系统只做出一个预测。

You can't train a system to make one prediction.

你无法训练一个神经网络说：这就是图像。

You can't train a neural net to say, here it is, that's the image.

因为有一整套与之兼容的可能性。

Because there's a whole set of things that are compatible with it.

那么，你如何让机器表示的不是一个单一输出，而是一整组输出呢？

So how do you get the machine to represent not a single output, but a whole set of outputs?

同样地，在视频预测中，未来可能发生的事情有很多。

Similarly with video prediction, there's a lot of things that can happen in the future of video.

你现在看着我，我并没有大幅度移动头部，但我可能会向左或向右转头。

You're looking at me right now, I'm not moving my head very much, but I might turn my head to the left or to the right.

对。

Right.

如果你没有一个能够预测这一点的系统，而是用最小二乘法来训练它，试图最小化预测结果与我实际行为之间的误差，那么你得到的将是我所有可能未来位置的模糊图像，这并不是一个好的预测。

If you don't have a system that can predict this, and you train it with least square to kind of minimize the error with a prediction and what I'm doing, what you get is a blurry image of myself in all possible future positions that I might be in, which is not a good prediction.

但也许对于视觉场景，还有其他方法可以实现自监督，对吧？

But, so there might be other ways to do the self supervision, right, for visual scenes.

比如哪些？

Like what?

如果我知道的话，我不会先告诉你该怎么发表。

If I knew, I wouldn't tell you how to publish it first.

我不知道。

I don't know.

不，也许有其他方法。

No, there might be.

所以，我的意思是，这些方法可能有一些人为的途径，比如在游戏中进行自我对弈，通过模拟部分环境来实现。

So, I mean, these are kind of, there might be artificial ways of like self play in games, the way you can simulate part of the environment.

可以

Can

哦，这并不能解决问题。

Oh, that doesn't solve the problem.

这仅仅是一种生成数据的方法。

It's just a way of generating data.

但因为你有更多的控制权，也许你可以控制一下。

But because you have more of a control like, maybe you can control yeah.

这是一种生成数据的方式。

It's a way to generate data.

没错。

That's right.

而且因为你能生成海量数据，你说得对。

And because you can do huge amounts of data generation, that doesn't you're right.

这个嘛，它是从数据的角度逐步逼近问题的，但你不觉得这是正确的逼近方式

This well, it's it's a creeps up on the problem from the side of data, and you don't think that's the right way to creep

这并不能解决处理世界不确定性的问题。

up It doesn't solve this problem of handling uncertainty in the world.

对吧？

Right?

所以，如果你让机器在一种确定性或准确定性的游戏中学习世界的预测模型，那就很简单。

So if you if you have a machine learn a predictive model of the world in a game that is deterministic or quasi deterministic, it's easy.

给卷积网络几帧游戏画面，加上多层网络，然后它就能生成接下来的几帧。

Give a few frames of the game to a Convolent, put a bunch of layers and then half the game generates the next few frames.

如果游戏是确定性的，那就没问题。

If the game is deterministic, it works fine.

这包括向系统输入你角色即将采取的动作。

And that includes, you know, feeding the system with the action that your little character is going to take.

问题在于，现实世界和大多数游戏都不是完全可预测的。

The problem comes from the fact that the real world and most games are not entirely predictable.

因此，你会得到模糊的预测，而无法基于模糊的预测进行规划。

And so there you get those blurry predictions and you can't do planning with blurry predictions.

所以，如果你对世界有一个完美的模型，你就可以在脑海中用一组动作假设来运行这个模型，从而预测这一系列动作的结果。

So if you have a perfect model of the world, you can, in your head, run this model with a hypothesis for a sequence of actions, and you're going to predict the outcome of that sequence of actions.

但如果你的模型不完美，你该如何规划呢？

But if your model is imperfect, how can you plan?

是的。

Yeah.

这会迅速变得复杂不堪。

It quickly explodes.

关于这个方向的延伸，我非常感兴趣，它与你之前提到的机器人技术有关，那就是主动学习。

What are your thoughts on the extension of this, which topic I'm super excited about, it's connected to something you were talking about in terms of robotics, is active learning.

与完全无监督或自监督学习不同，你会向系统寻求人类的帮助，以选择接下来需要标注的部分。

So, as opposed to sort of completely unsupervised or self supervised learning, you ask the system for human help, for selecting parts you want annotated next.

如果你想象一个机器人在探索空间，或者一个婴儿在探索空间，又或者一个系统在探索数据集，每隔一段时间就寻求一次人类的输入。

If you think about a robot exploring a space, or a baby exploring a space, or a system exploring a dataset, every once in a while asking for human input.

你认为这种做法有价值吗

Do see value in that kind

的工作？

of work?

我看不出有变革性的价值。

I don't see transformative value.

它只会让那些我们已经能做的事情更高效，或者让我们学得稍微快一点，但不会让机器变得显著更智能。

It's going to make things that we can already do more efficient or that we'll learn slightly more efficiently, but it's not going to make machines sort of significantly more intelligent.

而且顺便说一下，自监督学习、强化学习、监督学习、模仿学习和主动学习之间并没有对立，也没有冲突。

Think, and by the way, there is no opposition, there's no conflict between self supervised learning, reinforcement learning and supervised learning or imitation learning or active learning.

我认为自监督学习是上述所有方法的前期阶段。

I see self supervised learning as a preliminary to all of the above.

所以我经常举的例子是：如果你使用经典的强化学习，或者说是当今最好的所谓无模型强化学习方法来学习玩雅达利游戏，需要大约八十小时的训练才能达到人类大约十五分钟就能达到的水平。

So the example I use very often is how is it that So if you use classical reinforcement learning, deep reinforcement learning, if you want, the best methods today so called model free reinforcement learning to learn to play Atari games, take about eighty hours of training to reach the level that any human can reach in about fifteen minutes.

它们

They

能超过人类，但需要很长时间。

get better than humans, but it takes them a long time.

AlphaStar，你知道的，还有他的团队开发的用于玩《星际争霸》的系统，只针对单一地图、单一类型的玩家，通过大约相当于两百年的自我对弈训练，就能达到超越人类的水平。

AlphaStar, you know, and his teams, the system to play Starcraft plays, you know, a single map, a single type of player, and can reach better than human level with about the equivalent of two hundred years of training playing against itself.

是两百年，对吧？

It's two hundred years, right?

这不是人类永远无法做到的事情。

It's not something that no human can, could ever do.

我的意思是，我不确定Lex能从中学到什么。

I mean, I'm not sure what Lex can take away from that.

现在，把这些目前最先进的强化学习算法应用到自动驾驶汽车的训练上。

Now take those algorithms, the best RL algorithms we have today to train a car to drive itself.

它可能需要驾驶数百万小时。

It would probably have to drive millions of hours.

它必须撞死数千名行人。

It will have to kill thousands of pedestrians.

它必须撞上数千棵树。

It will have to run into thousands of trees.

它还必须从悬崖上冲下去。

It will have to run off cliffs.

是的。

Yeah.

而且在它意识到这是个糟糕的主意之前，它已经多次从悬崖上冲下去了。

And it had to run off cliff multiple times before it figures out that it's a bad idea, first of all.

其次，在它学会如何避免这样做之前，也经历了多次这样的事故。

And second of all, before it figures out how not to do it.

因此，这种学习方式显然与动物和人类的学习方式不同。

And so, I mean, this type of learning obviously does not reflect the kind of learning that animals and humans do.

这其中缺失了一些非常重要、非常关键的东西。

There is something missing that's really, really important there.

我的假设——我已经倡导了五年了——是我们拥有对世界的预测模型，这些模型包含在不确定性下进行预测的能力。

And my hypothesis, which I've been advocating for like five years now, is that we have predictive models of the world that include the ability to predict under uncertainty.

而这正是让我们在学习驾驶时不会冲下悬崖的原因。

And what allows us to not run off a cliff when we learn to drive.

我们大多数人只需大约二十到三十小时的训练就能学会开车，而且从未发生过碰撞或事故。

Most of us can learn to drive in about twenty or thirty hours of training without ever crashing, causing any accident.

如果我们开车靠近悬崖，我们知道如果向右打方向盘，车就会冲下悬崖，结果肯定不好。

And if we drive next to a cliff, we know that if we turn the wheel to the right, the car is going to run off the cliff and nothing good is going to come out of this.

因为我们对直觉物理有相当好的理解，知道车会掉下去。

Because we have a pretty good model of intuitive physics that tells us, you know, the car is going to fall.

明白重力的作用。

Know about gravity.

婴儿在八九个月大时就学会了物体不会漂浮，而是会下落。

Babies learned this around the age of eight or nine months that objects don't float, they fall.

而且，我们对转动方向盘对车辆的影响有相当清楚的认识，也知道必须留在道路上。

And, you know, we have a pretty good idea of the effect of turning the wheel on the car and, you know, we know we need to stay on the road.

因此，我们带入了许多东西，本质上就是我们对世界的预测模型。

So there's a lot of things that we bring to the table, which is basically our predictive model of the world.

这个模型让我们不会做愚蠢的事，并且基本上能保持在我们需要做的事情的范围内。

And that model allows us to not do stupid things and to basically stay within the context of things we need to do.

我们仍然会遇到不可预测的情况，这正是我们学习的方式，但这也让我们能够学得非常非常快。

We still face unpredictable situations and that's how we learn, but that allows us to learn really, really quickly.

这被称为基于模型的强化学习。

That's called model based reinforcement learning.

其中包含一些模仿和监督学习，因为我们偶尔会有一个驾驶教练告诉我们该怎么做。

There's some imitation and supervised learning because we have a driving instructor that tells us occasionally what to do.

但大部分学习是学习模型，学习我们从婴儿时期就开始掌握的物理知识。

But most of the learning is learning the model, learning physics that we've done since we were babies.

几乎所有学习都发生在这一部分。

That's where all almost all the learning

而物理知识在不同场景之间是可迁移的。

And the physics is somewhat transferable from is transferable from scene to scene.

愚蠢的行为在任何地方都是一样的。

Stupid things are the same everywhere.

是的。

Yeah.

我的意思是，如果你对世界有经验，你并不需要是特别聪明的物种，才能明白如果把水从容器里洒出来，剩下的地方就会湿。

I mean, if you, you know, you have experience of the world, you don't need to be particularly from a particularly intelligent species to know that if you spill water from a container, you know, the rest is going to get wet.

你可能会弄湿自己。

You might get wet.

你知道，猫也懂这个，对吧？

You know, cats know this, right?

是的。

Yeah.

所以我们需要解决的主要问题是，如何学习世界的模型？

So the main problem we need to solve is how do we learn models of the world?

这正是我感兴趣的。

And that's what I'm interested in.

这正是自监督学习的核心所在。

That's what self supervised learning is all about.

如果你要尝试构建一个基准，比如看看MNIST。

If you were to try to construct a benchmark for let's let's look at MNIST.

我非常喜欢这个数据集。

I love that dataset.

但你觉得只用每个数字的一个样本在MNIST上表现良好，是否可行、有趣或有意义？

But if do you think it's useful, interesting slash possible to perform well on MNIST with just one example of each digit?

我们该如何解决这个问题？

And how would we solve that problem?

答案可能是肯定的。

The answer is probably yes.

问题是，你被允许使用哪种其他类型的学习方法？

The question is what other type of learning are you allowed to do?

如果你被允许在一个包含大量标注数字的大数据集上进行训练，这就叫做迁移学习。

So if what you're allowed to do is train on some gigantic dataset of labeled digit, that's called transfer learning.

我们知道这种方法是有效的。

And we know that works.

好的。

Okay.

我们

We

我们在Facebook的生产环境中就是这样做的。

do this at Facebook, like in production.

对吧？

Right?

我们训练大型卷积网络来预测人们在Instagram上输入的标签，训练数据是数十亿张图像，真的是数十亿张。

We we train large convolutional nests to predict hashtags that people type on Instagram and we train on billions of images, literally billions.

然后我们去掉最后一层，针对我们想要的任何任务进行微调。

And then we chop off the last layer and fine tune on whatever task we want.

这种方法效果非常好。

That works really well.

用这种方法甚至可以超越ImageNet的记录。

You can beat the ImageNet record with this.

我们实际上几周前就把整个方案开源了。

We actually open sourced the whole thing, like, a few weeks ago.

是的。

Yeah.

那确实还是很酷。

That's that's still pretty cool.

但没错。

But yeah.

那么，什么会让人印象深刻？

So what and what would be impressive?

什么既实用又令人印象深刻？

And and what's useful and impressive?

什么样的迁移学习会既实用又令人印象深刻？

What kind of transfer learning would be useful and impressive?

是维基百科吗？

Is it Wikipedia?

类似这样的东西？

That kind of thing?

不。

No.

所以我认为我们不应该把重点放在迁移学习上。

So I don't think transfer learning is really where we should focus.

我们应该尝试设定一种基准场景，其中有大量未标记的数据。

We should try to do, you know, have a kind of scenario for benchmark where you have unlabeled data and can, and it's very large number of unlabeled data.

它可以是视频片段。

It could be video clips.

它可以是帧预测任务。

It could be where you do frame prediction.

它可以是图像，你可以选择遮蔽其中一部分。

It could be images where you could choose to mask a piece of it.

可以是任何东西，但它们都是未标记的，而且你不能给它们打标签。

Could be whatever, but they're unlabeled and you're not allowed to label them.

所以你先在这上面进行训练，然后在特定的监督任务上训练，比如ImageNet或NIST，并测量随着标记训练样本数量的增加，你的测试误差或验证误差如何降低。

So you do some training on this and then you train on a particular supervised task, ImageNet or NIST, and you measure how your test error decrease or validation error decreases as you increase the number of labeled training samples.

好的。

Okay.

你希望看到的是，你的错误率下降速度远快于从随机权重开始完全从头训练的情况。

And what you'd like to see is that your error decreases much faster than if you train from scratch, from random weights.

因此，要达到与完全监督系统相同的性能水平，你所需的样本数量会少得多。

So that to reach the same level of performance than a completely supervised, purely supervised system would reach, you would need way fewer samples.

这是关键问题，因为它将回答那些对医学图像分析感兴趣的人的疑问。

So that's the crucial question because it will answer the question to like, you know, people interested in medical image analysis.

好的。

Okay.

你知道，如果我想在这个任务上达到特定的错误率，我知道我需要一百万个样本。

You know, if I want to get a particular level of error rate for this task, I know I need a million samples.

我能否通过自监督预训练，将这个数量减少到大约一百个左右？

Can I do, you know, self supervised pre training to reduce this to about a 100 or something?

你认为这里的答案是自监督预训练吗？

And you think the answer there is self supervised pre training?

是的

Yeah.

某种形式。

Some form.

某种形式的。

Some form of it.

你提到主动学习，但你不同意。

Telling you active learning, but you disagree.

不，这并不是无用的。

No, it's not useless.

它不会带来质的飞跃。

It's just not gonna lead to a quantum leap.

它只是让我们已经做的事情变得更好。

It's just gonna make things that we already do.

所以你比我聪明多了。

So you're way smarter than me.

我只是不同意你的观点。

I just disagree with you.

但我没有证据支持这一点。

But I don't have anything to back that.

这仅仅是直觉。

It's just intuition.

所以我接触过很多大规模数据集，我觉得主动学习中可能有些神奇之处。

So I worked with a lot of large scale data sets, and there's something that might be magic in active learning.

但好吧。

But okay.

至少我说了

At least I said

公开地表达了。

it publicly.

至少我公开地当了个傻瓜。

At least I'm being an idiot publicly.

好的。

Okay.

这并不是傻。

It's not being an idiot.

这其实是，你知道的，利用你现有的数据进行工作。

It's, you know, working with the data you have.

我的意思是，当然，人们正在做这样的事情：比如，我有三千小时的自动驾驶模仿数据，但其中大部分都极其枯燥。

I mean I mean, certainly, people are doing things like, okay, I have three thousand hours of imitation running for a self driving car, but most of those are incredibly boring.

我喜欢的是从中挑选出10%最有信息量的样本。

What I like is select 10% of them that are kind of the most informative.

仅凭这些，我可能就能达到同样的效果。

And with just that, I would probably reach the same.

如果你愿意这么说的话，这是一种弱形式的主动学习。

It's a weak form of of active learning, if you want.

是的。

Yes.

但可能还存在一个更强大的版本。

But there might be a a much stronger version.

是的。

Yeah.

那就是

That's

对。

right.

那就是问题所在，这还是一个悬而未决的问题：它是否存在。

That's what and that's an open question if it exists.

问题是，你能做到多强？

The question is how much stronger can you get?

埃隆·马斯克很有信心。

Elon Musk is confident.

我最近刚和他谈过。

Talked to him recently.

他相信，大规模数据和深度学习能够解决自动驾驶问题。

He's confident that large scale data and deep learning can solve the autonomous driving problem.

你对深度学习在这一领域的潜力极限有什么看法？

What are your thoughts on the limits possibilities of deep learning in this space?

嗯，这显然是解决方案的一部分。

Well, it's It's obviously part of the solution.

我的意思是，我认为我们永远不可能拥有一个自动驾驶系统——至少在可预见的未来——不使用深度学习的。

I mean, don't think we'll ever have a self driving system, or at least not in the foreseeable future, that does not use deep learning.

让我这么说吧。

Let me put it this way.

那么，它占多大比例呢？

Now, how much of it?

在工程历史中，尤其是类似AI的系统，通常会经历第一阶段，即所有东西都是手工构建的。

So in the history of sort of engineering, particularly sort of AI like systems, there's generally a first phase where everything is built by hand.

然后进入第二阶段。

Then there is a second phase.

二十、三十年前，自动驾驶也是如此。

And that was the case for autonomous driving, you know, twenty, thirty years ago.

有一个阶段，会用到一些学习技术，但需要大量工程工作来处理边缘情况和设定限制等，因为学习系统并不完美。

There's a phase where there's a little bit of learning is used, but there's a lot of engineering that's involved in kind of, you know, taking care of corner cases and putting limits, etcetera, because the learning system is not perfect.

随着技术进步，我们越来越依赖学习方法。

And then as technology progresses, we end up relying more and more on learning.

这正是字符识别、语音识别、计算机视觉、自然语言处理的发展历程。

That's the history of character recognition, the history of speech recognition, computer vision, natural language processing.

我认为自动驾驶也会经历同样的过程。目前，最接近实现某种程度自主性——即你不需要司机干预——的方法，是通过限制运行环境来实现的。

And I think the same is going to happen with autonomous driving that currently methods that are closest to providing some level of autonomy, some decent level of autonomy, where you don't expect a driver to kind of do anything is where you constrain the world.

所以你只在亚利桑那州凤凰城方圆一百平方公里或平方英里的区域内运行，而且天气良好、道路宽阔，这正是Waymo的做法。

So you only run within, you know, a 100 square kilometers or square miles in Phoenix, but the weather is nice and the roads are wide, which is what Waymo is doing.

你为汽车配备了大量激光雷达和昂贵的高级传感器，这些对普通消费者车辆来说太贵了，但如果你只运营一个车队，那就没问题。

You completely overengineer the car with tons of LIDARs and sophisticated sensors that are too expensive for consumer cars, but they're fine if you just run a fleet.

你把其他所有方面都彻底工程化到极致。

And you engineer the thing, the hell out of everything else.