人工智能的起源与杰弗里·辛顿

我需要大量的这种检测器。

I need huge numbers of them.

因为鸟可能随机出现在图像的任何位置，

Just in case the bird is somewhere anywhere randomly in the image,

你必须能够

you've got to able to

访问它。

access it.

在图像的任意位置。

Anywhere randomly in the image.

是的。

Mhmm.

接下来我要做的是，也许因为我们此时已经有点不耐烦了，我可以添加一个最终层，其中的神经元会识别“猫”、“狗”、“鸟”、“政客”等等，而在这一层中，我会把识别“鸟”的神经元连接到检测鸟头的那些单元，同时也会连接到第三层中检测鸟脚或鸟翼尖等其他特征的单元。

The next thing I'm gonna do is maybe because we're sort of running out of patience at this point, I can have a final layer that has neurons that say cat, dog, bird, politician, whatever, and in that final layer, I'll take the neuron that says bird, and I'll hook it up to the things that detect birds' heads, but I'll also hook it up to other things in the third layer that detect things like birds' feet or the tips of birds' wings.

因此，现在当我的“鸟”输出神经元被激活时，神经网络就表示它是一只鸟。

And so now my output neuron for bird, when that gets active, the neural net is saying it's a bird.

如果它看到一只鸟的脚、可能的鸟头和鸟翅膀的尖端，就会收到大量输入，并说：嘿，我觉得这是只鸟。

If it sees a bird's foot and a possible bird's head and a possible tip of the wing of a bird, it'll get lots of input and say, Hey, I think it's a bird.

所以我想你现在应该能理解我是如何尝试手动设计这个系统的了，而且你能看到这种方法存在巨大问题。

So I think you can now understand how I might try and design that by hand, and I think you can see there's huge problems in that.

需要大量的检测器。

Need an awful lot of detectors.

我需要覆盖整个位置、方向和尺度的空间。

I need to cover this whole space of positions and orientations and scales.

我需要决定提取哪些特征。

I need to decide what features to extract.

我的意思是，我只是随便提出了通过鸟喙来识别鸟头的想法。

I mean, I just made up the idea of getting a beak and then a bird's head.

可能还有更好的特征可以寻找。

There may be much better things to go after.

更重要的是，我想检测许多不同的物体，所以我真正需要的是不仅对识别鸟有效、而且对识别各种物体都有效的特征。

What's more, I want to detect lots of different objects, so what I really need is features that aren't just good for finding birds, but features that are good for finding all sorts of things.

如果要手动设计这个，那将是一场噩梦，尤其是当我意识到要做好这件事，我需要一个至少包含十亿个连接的网络时。

And it would be a nightmare to design this by hand, particularly if I figured out that to do a good job of this, I needed a network with at least a billion connections in it.

所以我必须手动设计这十亿个连接的权重，这会花费很长时间。

So I have to, by hand, design the strengths of these billion connections, and that'll take a long time.

于是我们说，好吧，这样一个网络，如果它的连接权重设置得当，也许能够识别鸟类，但这些连接权重我该从哪里获得呢？

Then we say, well, okay, a network like that, maybe it could recognize birds if it had the right connection strengths in it, but where am I going get those connection strengths from?

因为我绝对不想手动去设置它们。

Because I sure as hell don't want to put them in by hand.

我甚至不想让我的研究生去设置它们。

I don't even want to tell my graduate students to put them in.

是的。

Yeah.

教授，这正是他们存在的意义。

That's what they're there for, professor.

没错，他们正是为此而存在的，但你需要大约一千万个这样的学生。

That's absolutely what they're there for, but you need about 10,000,000 of them

就今年。

for this year.

好的。

Okay.

行吧。

Alright.

好吧，我们现在遇到问题了。

Well, now we've got a problem.

不。

No.

你才有问题。

You got a problem.

你能想象吗，要写多少份资助申请才能支持一千万名研究生？

Can you imagine the grants you'd have to write to support 10,000,000

研究生？

graduate students?

天哪。

Oh my word.

所以

So

这里有个想法，一开始看起来真的很蠢，但它能让你理解我们要做什么。

here's an idea that initially seems really dumb, but it'll get you the idea of what we're gonna do.

我们将从随机的连接权重开始。

We're gonna start with random connection strengths.

有些会是正数，有些会是负数。

Some will be positive numbers, some will be negative numbers.

嗯。

Mhmm.

因此，我之前提到的这些层中的特征，我们称之为隐藏层，这些层中的特征都将是随机的。

And so the features in these layers I've been talking about, we call them hidden layers, the features in those layers will be just random features.

如果我们输入一张鸟的图片，观察输出神经元的激活情况，猫、狗、鸟和政客对应的输出神经元都会被轻微激活，且激活程度大致相同，因为连接权重完全是随机的。

And if we put in an image of a bird and look at how the output neurons get activated, the output neurons for cat and dog and bird and politician will all get activated a tiny bit and all about equally, because the connection strengths are just random.

所以这样不行。

So that's no good.

但我们现在可以问这样一个问题：假设我取了其中一个连接权重，那十亿个连接权重中的一个，我说：好吧，我知道这是张鸟的图片，而我真正希望的是，下次我给你这张图片时，你能稍微增加鸟神经元的激活，同时稍微降低猫、狗和政客神经元的激活。

But we could now ask the following question: suppose I took one of those connection strengths, one of those billion connection strengths, and I said, okay, I know this is an image of a bird, and what I'd really like is next time I present you with this image, I'd like you to give slightly more activation to the bird neuron and slightly less activation to the cat and dog and politician neurons.

那么问题来了：我该如何调整这个连接权重？

And the question is: how should I change this connection strength?

嗯，我可以做个实验。

Well, I could do an experiment.

如果我不太理论化，也不太懂数学，我就会做个实验。

If I'm not very theoretical and don't know much math, I'd do an experiment.

我会说：我们把连接权重稍微调大一点，看看会发生什么。

I would say, let's increase the connection strength a little bit and see what happens.

这样能更好地识别出鸟吗？

Does it get better at saying bird?

如果识别鸟的效果变好了，我就说：好的。

And if it gets better at saying bird, I say, okay.

我会保留对这个连接的这种调整。

I'll keep that mutation to the connection.

是的。

Yeah.

但所谓‘更好’，意味着有人在过程中做出判断，对吧。

But better means there's a human in the loop making that judgment Right.

作为其实验的结果。

As a result of its of its experiment.

必须有人来决定什么是正确的答案。

Well, has to be someone saying what the right answer is.

是的。

Yes.

当然。

Absolutely.

这被称为监督者。

That's called the supervisor.

是的。

Yes.

好的。

Okay.

好的。

Okay.

如果你那样做，问题在于有一亿个连接权重。

And the problem if you do it like that is there's a billion connection strengths.

每个权重都需要调整很多次，这会花上无穷无尽的时间。

Each of them has to be changed many times, it's going to take like forever.

所以问题是：有没有一种不同于测量的、效率高得多的方法？

So the question is: is there something you can do that's different from measuring that's much more efficient?

确实有。

And there is.

你可以做一种叫做计算的事情。

You can do something called computing.

所以这个网络，如果它运行在计算机上，你当然知道所有连接的当前权重。

So this network, certainly if it's on a computer, you know the current strengths of all the connections.

所以当你输入一张图像时，连接权重最初是随机值，但之后的一切都是确定性的。

So when you put in an image, there's nothing random about what I mean the connection strengths initially had random values When you put in an image, it's all deterministic what happens next.

像素强度会乘以连接到第一层神经元的权重，它们的激活值再乘以连接到第二层的权重，依此类推，最终得到输出神经元的一些激活水平。

The pixel intensities get multiplied by weights on connections to the first layer of neurons, their activities get multiplied by weights on connections to the second layer, and so on, and you get some activation levels of the output neurons.

因此，你现在可以问这样一个问题：如果我关注那个鸟类神经元，能否同时判断所有连接权重是应该略微增加还是减少，以便让网络更确信这是只鸟，让它更明确地输出‘鸟’，而其他类别则更安静一些。

So you could now ask the following question: If I take that bird neuron, could I figure out for all the connection strengths at the same time whether I should increase them a little bit or decrease them a little bit in order to make it more confident that this is a bird, in order for it to just say bird a bit more lively and other things a bit more quietly.

而你可以用微积分来做到这一点。

And you can do that with calculus.

你可以将信息反向传递通过网络，计算如何让网络下次更有可能输出‘鸟’。

You can send information backwards through the network saying how do I make this more likely to say bird next time?

由于在座有很多物理学家，我将尝试给你们一个物理直觉上的解释。

And because you have a lot of physicists in the audience, I'm gonna try and give you a physical intuition for this.

说吧。

Go for it.

是的

Yeah.

你输入一张鸟的图片，初始权重下，鸟输出神经元的激活程度非常微弱。

You put in bird, an image of a bird, and with the initial weights, the bird output neuron only gets very slightly active.

于是你现在要系上一根长度为零的橡皮筋。

And so what you do now is you attach a piece of elastic of zero wrist length.

你系上一根橡皮筋，一端连接鸟输出神经元的激活水平，另一端连接你希望达到的值，比如1。

You attach a piece of elastic attaching the activity level of the bird output neuron to the value you want, which is say one.

假设1是最大激活水平，0是最小激活水平，而当前激活水平大约是0.01。

Let's say one's the maximum activity level and zero is the minimum activity level and this has an activity level of like 0.01.

你系上这根橡皮筋，它会试图把激活水平拉向正确答案，也就是这种情况下为1。

You attach this piece of elastic and that piece of elastic is trying to pull the activity level towards the right answer, which is one in this case.

但当然，激活水平是由你输入的像素、像素激活强度以及网络中的所有权重决定的，所以激活水平本身无法移动。

But of course, the activity level is being determined by the pixels that you put in, the pixel activation levels, the intensities, and all the weights in the network, So the activity level can't move.

要让激活水平发生变化，一种方法是调整进入鸟神经元的权重。

Now, one way to make the activity level move would be to change the weights going into the bird neuron.

例如，你可以给那些高度活跃的神经元赋予更大的权重，这样鸟神经元的活跃度就会提高。

You could, for example, give bigger weights on neurons that are highly active, and then the bird neuron will get more active.

但改变鸟神经元活跃度的另一种方法是实际改变前一层神经元的活跃度。

But another way to change the activity level of the bird neuron is to actually change the activity levels of the neuron of the layer before it.

所以，例如，我们可能有一个检测到鸟头但不太确定的系统。

So for example, we might have something that's sorted and detected a bird's head, but wasn't very sure.

这确实是一只鸟，因此你希望的是，既然你希望输出更像鸟，而这条弹性带正不断要求‘更多、更多，我需要更多’，你希望这能促使那个原本只是怀疑这里有鸟头的系统变得更加确信鸟头确实存在。

This really is a bird, and so what you'd like is the fact that you want the output to be more bird like you've got this piece of elastic saying more, more, I want more here You'd like that to cause this thing that thought maybe there's a bird's head here to get more confident there's a bird's head there.

所以你需要做的是，将这条弹性带施加在输出神经元上的力反向传递给前一层的神经元，对它们施加一种拉力，这就是反向传播。

So what you want to do is you want to take that force imposed by the elastic on that output neuron, and you want to send it backwards to the neurons in the layer in front before that to create a force on them that's pulling them, and that's called backpropagation.

反向传播。

Backpropagation.

是的。

Yeah.

好的。

Okay.

这被称为反向传播，从物理角度理解，就是你有一个作用在输出神经元上的力，你想把这个力反向传递，使这个力作用于

Is called backpropagation, and the physics way to think about it is you've got a force acting on the output neurons, and you wanna send that force backwards so that the force acts

前一层的神经元。

on the neurons in the layer in front.

当然，

And of course,

有多个输出神经元上都存在作用力。

there's forces acting on many different output neurons.

嗯。

Mhmm.

因此，你需要将所有这些力综合起来，以得到作用于

So you have to combine all those forces to get the forces acting on the neurons in

下一层神经元上的力。

the layer below.

一旦你将所有这些都传递到

Once you send this all the

通过网络反向传播时，所有这些神经元都受到作用力，于是你说：好吧，让我们调整每个神经元的输入权重，使其活动水平朝着作用在其上的力的方向变化。

way back through the network, you have forces acting on all these neurons, and you say, Okay, let's change the incoming weights of each neuron so its activity level goes in the direction of the force that's acting on it.

这就是反向传播，它让一切运作得异常出色。

That's backpropagation, and that makes things work wondrously well.

所以，这是那个天才的灯泡时刻吗？

So is this the light bulb Diabolically.

是的。

Yeah.

我跟你说过别急着走。

Told you don't go yet.

别急着去那里。

Don't go there yet.

好的。

Okay.

这就是神经网络不再需要人类教师的顿悟时刻吗？

Is this the light bulb moment where the neural networks no longer need the human teacher?

这是这一过程的开始吗？

Is this the beginning of that process?

不，不完全是。

No, not exactly.

好的。

Okay.

但这确实是一个灵光一现的时刻。

This is a light bulb moment though.

多年来，相信神经网络的人们都知道如何改变最后一层的连接强度，也就是你所说的权重，即从最后一层特征到输出单元的连接强度。

So for many years, the people who believed in neural networks knew how to change the very last layer of connection strengths, which you call weights, the ones that are going into the output units, the connection strengths going from the last layer of features into the bird neuron.

我们知道如何改变这些权重，但并不清楚如何让作用力影响到那些隐藏神经元，比如检测鸟头的那些神经元。

We knew how to change those, but we understand how to get forces operating on those hidden neurons, the ones that detect a bird's head, for example.

反向传播让我们明白了如何让作用力影响这些隐藏神经元，从而能够调整它们的输入权重，这是一个顿悟的时刻。

Backpropagation showed us how to get forces acting on those so then we could change the incoming weights of those, and that was a eureka moment.

许多不同的人在不同时间都经历了这样的顿悟时刻。

Many different people had that eureka moment at different times.

那么，我们说的是哪个时期，当你开始想到反向传播的时候？

So what period of time are we talking about here when you've fallen to the back propagation thought?

好的，20世纪70年代初，芬兰有个人可能在他的硕士论文中提出了这个想法，然后在70年代末，哈佛大学的保罗·韦尔普斯也有了这个想法。

Okay, the early 1970s, there was someone in Finland who had it I think in his master's thesis, and then in probably the late 1970s someone called Paul Wurpos at Harvard had the idea.

事实上，那里的一些控制理论家，比如布赖森和霍，早就有了类似的想法，用于控制航天器。

In fact, some control theorists there called Bryson and Ho had had the idea for doing things like controlling spacecraft.

所以当你让航天器登陆月球时，你用的是非常基础的反向传播，但那是在线性系统中。

So when you land a spacecraft on the moon, you're using something very light backpropagation, but it's in a linear system.

用反向传播来计算应该如何点火推进火箭。

Using backpropagation to figure out how you should fire the rockets.

所以看起来，你们在70年代就已经具备了今天的技术基础。

So it seems like what you're talking about in the seventies, we could have had what we have today.

只是当时缺乏足够的数学计算能力来实现它。

We just didn't have the mathematical computing power to make this work.

这确实是主要原因之一，没错。

That's a large part of it, yes.

我们当时还缺少另一样东西：在70年代，人们并没有展示出，当你将这种方法应用于多层网络时，会得到非常有趣的表征。

The other thing we didn't have is back in the 70s, people didn't show that when you applied this in multilayer networks, what you get is very interesting representations.

所以我们并不是第一个想到反向传播的人，但我在圣地亚哥所在的团队，是第一个证明你可以用这种方式学习词语含义的。

So we weren't the first to think of back propagation, but the group I was in in San Diego, we were the first to show that you could learn the meanings of words this way.

你可以输入一串词语，通过尝试预测下一个词，学会为词语分配能够捕捉其语义的特征。

You could show the string of words, and by trying to predict the next word, you could learn how to assign features to words that captured the meaning of the word.

正是这一点让我们在《自然》杂志上发表了论文。

And that's what got it published in Nature.

听起来是这样，我正努力理解你所解释的内容，因为在我看来，这些值之间似乎存在一种级联关系，真正重要的是那些最接近下一个值的参数。

It sounds like and I'm just trying to get my head around what you explained, because it sounds to me like there is a cascading relationship to these values, and that really what matters are the values that are closest to the next value.

然后还有一种级联式的强化机制，来判断‘是的，就是这样’，或者‘不，不是这样’。

And then there are kind of this cascading reinforcement to say, yes, this is it, or no, it is not.

我理解得对吗？

Am I getting that right?

我只是想用最直白的方式弄清楚你到底在说什么。

I'm just trying to figure out what you're saying here in a really plain way.

好的。

Okay.

这是个好问题。

It's a good question.

你理解得不太对。

You're not getting it quite right.

好的。

Okay.

说吧。

Go

继续。

ahead.

会后来找我。

See me afterwards.

这种通过反向传播这些力、然后调整所有连接权重，让每个神经元朝着力的方向移动的学习方式，并不是强化学习。

This kind of this kind of learning where you backpropagate these forces and then change all the connection strengths so each neuron goes in the direction that the force is pulling it in, That's not reinforcement learning.

这被称为监督学习。

This is called supervised learning.

强化学习是完全不同的东西。

Reinforcement learning is something different.

所以在这里，例如，我们会告诉它正确的答案是什么。

So here, for example, we tell it what the right answer is.

如果你有一千个类别，你展示了一只鸟，你就告诉它那是一只鸟。

If you've got a thousand categories and you showed a bird, you tell it that was a bird.

就是这样。

There you go.

在强化学习中，它会做出一个猜测，然后你告诉它是否答对了

In reinforcement learning, it makes a guess, and you tell it whether it got the answer

对。

right.

好的。

Alright.

这提供的信息少得多。

That's much less information.

你把这个问题讲清楚了。

You you cleared it up.

这正是我之前 missing 的部分。

That's what I was missing.

好的。

Alright.

到查克的

To Chuck's

关于计算能力的观点，就只是这一点吗？

point about computational power, was it just that?

因为现在你听起来像是有一个看似可行的理论，但实际问题是计算能力不足。

Because at the moment you sound a lot like you've got theory that seems like it could be, but the practicality is there's not enough computational power.

我们还有没有其他技术是促成这一点的关键因素？

Do we have any other technology that came through that was the enabling aspect to this?

好的。

Okay.

在八十年代中期，我们已经有了反向传播算法，它能完成一些很出色的任务。

So in the mid eighties, we had the backpropagation algorithm working and it could do some neat things.

它在识别手写数字方面比几乎任何其他技术都更出色，但无法很好地处理真实图像。

It could recognize handwritten digits better than nearly any other technique, but it couldn't deal with real images very well.

它在语音识别方面表现相当不错，但并没有比其他技术显著更好。

It could do quite well at speech recognition, but not substantially better than the other technologies.

当时我们并不明白，为什么这并不是解决所有问题的万能钥匙。

And we didn't understand at the time why this wasn't the magic answer to everything.

结果发现，如果你有足够的数据和足够的计算能力，它确实是解决所有问题的万能钥匙。

And it turns out it was the magic answer to everything if you have enough data and enough compute power.

哇。

Wow.

所以，八十年代真正缺失的就是这些。

So that's what was really missing in the eighties.

好的。

Alright.

我先暂时岔开一下话题，想跟你探讨一下，这既是评论也是个问题。

I'm gonna I'm gonna depart for a second just just to pick your brain for a this is part commentary and part question.

我认为，这个星球上大多数人都很愚蠢。

I'm gonna say that the majority of people that are walking around this planet are stupid.

那么，到底什么才算聪明？什么才算思考？

So what exactly is smart and what exactly is thinking?

这些机器，我们能教会它们如何思考吗？它们会超越我们吗？

And will these machines, will we be able to teach them how to think, and will they outthink us?

好的。

Okay.

它们已经懂得如何思考了。

They already know how to think.

明白了。

Okay.

那么，思考到底是什么？

So what is thinking then?

好的。

Okay.

嗯，

Well,

我们可以一直这样聊下去。

could do this all day.

思考包含很多要素，比如人们常常通过图像来思考。

There's a lot of elements to thinking, like people often think using images.

你实际上经常通过动作来思考。

You often think actually using movements.

所以当我一边在木工坊里转悠找锤子，一边想着别的事情时，我某种程度上会

So when I'm wandering around my carpentry shop looking for a hammer but thinking about something else, I sort

一直留意着

of keep track of the

事实上，我找锤子的时候会不自觉地做出这样的动作。

fact I'm looking for a hammer by sort of going like this.

我一边想着别的事情，一边做出这样的动作。

I wonder I'm going like this while I'm thinking about something else.

这其实是一种我正在寻找锤子的心理表征。

And that's a representation that I'm looking for a hammer.

因此，思考涉及多种表征方式，但其中最主要的一种是语言。

So we have many representations involved in thinking, but one of the main ones is language.

我们很多思考都是用语言进行的，而这些大型语言模型确实是在思考。

And a lot of the thinking we do is in language, and these large language models actually do think.

因此，这里存在一场激烈的争论，一方是信奉传统人工智能的人，他们认为思考完全基于逻辑，通过操作符号来生成新符号。

So there's a big debate, right, between the people who believed in old fashioned AI, that it was all based on logic and you manipulate symbols to get new symbols.

他们并不认为这些神经网络真的在思考。

They don't really think these neural nets are thinking.

而神经网络派则认为，不，它们确实在思考。

Whereas the neural net people think, no, they're thinking.

它们的思考方式和我们几乎一样。

They're thinking pretty much the same way we do.

因此，现在的一些神经网络，当你问它们问题时，它们会输出一个符号，表示‘我在思考’。

And so the neural nets now, some of them, you'll ask them a question and they'll output a symbol that says I'm thinking.

然后它们会开始输出自己的想法，这些是它们自己的思考。

And then they'll start outputting their thoughts, which are thoughts for themselves.

比如我给你一个简单的数学题。

Like I give you a simple math problem.

比如有一艘船，船上有一位船长。

Like there's a boat, and on this boat there's a captain.

船上还有35只羊。

There's also 35 sheep.

船长多大了？

How old is the captain?

现在，许多10到11岁左右的孩子，尤其是接受过美国教育的，会说船长35岁，因为他们环顾四周，觉得35岁是船长一个合理的年龄，而他们唯一得到的数字就是这35只羊。

Now many kids of age around 10 or 11, particularly if they're educated in America, will say the captain is 35, because they look around and they say, Well, that's a plausible age for a captain, and the only number I was given was these 35 sheep.

所以它们是在符号替代的层面上运作的。

So they're operating at a sort of substituting symbols level.

人工智能有时会被诱使犯类似的错误，但人工智能的实际工作方式与人类非常相似。

AIs can sometimes be seduced into making similar mistakes, but the way the AIs actually work is quite like people.

它们会接受一个问题，然后开始思考。

They take a problem and they start thinking.

对于孩子，你可能会说：好吧，船长多大了？

For a child, you might say, Okay, well how old is a captain?

那么，这个问题里有哪些数字？

Well, what are the numbers I've got in this problem?

嘿，我只得到了一个35。

Hey, I've only got a 35.

这个年龄当船长合理吗？

Is that a plausible age for a captain?

太好了！

Yay!

他可能是35岁，有点年轻，但也许还可以，我就说35岁吧。

He might be 35, a bit young, but maybe okay, I'll say 35.

这正是一个10岁孩子可能会有的想法。

That's what a 10 year old child might think.

孩子会在心里用语言思考，而人们发现，对于这些语言模型，你可以训练它们在心里用语言思考。

And the child will think it to itself in words, And what people realize with these language models is you can train them to think to themselves in words.

这被称为思维链推理。

That's called chain of thought reasoning.

他们会训练模型这样做。

And they train them to do that.

但之后，当你给它们一个问题时，它们会像孩子一样在心里思考，有时会得出错误的答案。

But after that, you give them a problem, they'd think to themselves just like a kid would and sometimes come up with the wrong answer.

但你能看到它们在思考。

But you could see them thinking.

所以这就像人类一样。

So it's just like people.

所以，如果我们拥有正在思考的AI，而我刚才听你解释了它们确实会思考，那么它们的学习能力比我们更强吗？

So if we have AI that's thinking and I'm saying that knowing that you've just explained that they do are they better at learning than we are?

让我们进一步思考，从思考到预测，再到创造和理解，这个演进过程是怎样的？

And let's sort of take that forward and think, what is the evolution from thinking to predicting to being creative to understanding?

那么，我们是否会逐渐意识到这种智能的意识？

And are we then going to fall into an awareness of this intelligence?

好的。

Okay.

这大约有六个主要问题。

That's about half a dozen major questions.

你会，

You'll Well,

我们还有多长时间？

how long have we got?

再问一遍第一个问题。

Ask me the first question again.

人工智能比人类更擅长学习吗？

Are AIs better at learning than humans?

很好。

Good.

好的。

Okay.

太棒了。

Excellent.

所以它们解决的问题与我们略有不同。

So they're solving a slightly different problem from us.

在你的大脑中，大约有十万亿个连接。

So in your brain, you have a 100,000,000,000,000 connections, roughly speaking.

好的。

Okay.

很多啊。

That's a lot.

而你只活大约二十亿秒。

And you only live for about two billion seconds.

这并不多。

That's not much.

是的。

No.

三十亿。

Three billion.

二十亿秒相当于六十三年。

Two billion is sixty three years.

我们今天活得比这更长。

We do better than that today.

是的。

Yeah.

确实如此。

It's true.

我正要说到这一点。

I was gonna come to that.

我本来想说，幸运的是，我的寿命略多于20亿秒。

I was gonna say, luckily for me, it's a bit more than 2,000,000,000.

但我们这里讨论的是数量级，所以20亿、30亿，谁在乎呢？

But We're dealing with orders of magnitude here, so 2,000,000,000, 3,000,000,000, who cares?

是的。

Yeah.

好吧。

Alright.

如果你把你的寿命秒数和你拥有的连接数相比，你会发现你的连接数远多于你的经历数。

If you compare how many seconds you live for with how many connections you've got, you have a whole lot more connections than experiences.

而有了这些神经网络，情况却恰恰相反。

Now with these neural nets, it's sort of the other way round.

它们的连接数只有大约一万亿，即使在大型语言模型中，也只相当于你连接数的1%。

They only have of the order of a trillion connections, so like 1% of your connections, even in a big language model.

它们中的许多连接更少，但获得的经验却是你的数千倍。

Many of them have fewer, but they get thousands of times more experience than you.

所以大型语言模型是在只有万亿级连接的情况下解决这个问题的。

So the big language models are solving the problem with not many connections, only a trillion.

我该如何利用海量的经验？

How do I make use of a huge amount of experience?

反向传播在将大量知识压缩到极少连接中方面非常非常有效。

And backpropagation is really, really good at packing huge amounts of knowledge into not many connections.

但这并不是我们正在解决的问题。

But that's not the problem we're solving.

我们拥有海量的连接，却只有很少的经验。

We've got huge numbers of connections, not much experience.

我们需要尽可能从每次经验中提取最大价值。

We need to sort of extract the most we can from each experience.

因此，我们解决的是略有不同的问题，这也是一个原因，让我们认为大脑可能没有使用反向传播。

So we're solving slightly different problems, which is one reason for thinking the brain might not be using back propagation.

对，我正要说，听起来我们并没有使用反向传播。

Right, I was about to say it sounds like we don't use back propagation.

然而，这是否意味着通过增加神经网络的连接数量来蛮力提升其思考能力，就能毫无困难地超越我们？

However, would that mean the brute force of adding connections to the neural net increase its effective thinking so that it surpasses us with no problem.

那么它就会拥有更多的经验和更多的

So then it would have more experience and more

连接。

connections.

更多的

More

经验会自动增加，但现在它有

experience automatically, but now it has

100

a 100

个连接。

connections.

你在这里谈论的是规模。

You're talking about scale here.

是的。

Yeah.

我说的是规模。

I'm saying scale.

是的。

Yeah.

对。

Yeah.

那是个非常好的问题。

So that's a very good question.

在过去的几年里，确切地说是好几年，每次他们把神经网络做得更大并提供更多的数据时，它的表现都会更好。

And what happened for several years, quite a few years, is that every time they made the neural net bigger and gave it more data, it got better.

它实现了规模扩展。

It scaled.

有道理。

Makes sense.

而且它的提升方式非常可预测，因此你可以算出来，比如，要让模型再大一些、再给它更多数据，需要花费一亿美元。

And it got better in a very predictable way, so that you could figure out, you know, it's going to cost me a $100,000,000 to make it this much bigger and give it this much more data.

这值得吗？

Is it worth it?

而且你可以提前预测：是的，它会变得好这么多，值得投入。

And you could predict ahead of time: yes, it's going to get this much better, it's worth it.

现在这是否正在逐渐失效，还是一个开放性问题。

It's an open question whether that's petering out now.

对于某些神经网络来说，这种趋势不会失效，当你不断增大它们并提供更多的数据时，它们会持续变得越来越好。

There are some neural nets for which it won't peter out, where as you make them bigger and give them more data, they'll just keep getting better and better.

还有一些神经网络能够生成自己的数据。

And then neural nets where they can generate their own data.

我对物理学了解不多，但我认为这就像一个能自我产生燃料的钚反应堆。

I don't know that much physics, but I think it's like a plutonium reactor which generates its own fuel.

如果你想想AlphaGo这样的东西，早期的围棋程序使用神经网络时，是通过模仿专家的走法来训练的。

So if you think about something like AlphaGo, the place Go, initially, it was trained the early versions of Go playing programs with neural nets were trained to mimic the moves of experts.

如果你这么做，你永远不可能比专家好太多，而且专家的走法数据也会用完。

And if you do that, you're never going to get that much better than the experts, and also you run out of data from experts.

但后来他们让AlphaGo与自己对弈，当它自我对弈时，神经网络可以持续变得更好，因为它们能生成越来越多关于什么才是好棋的数据。

But later on they made it play against itself, and when it played against itself, its neural nets could get just keep on getting better because they could generate more and more data about what was a good move.

所以是每秒下数百万盘棋，对吧？

So is And play zillion games a second Right.

跟自己对弈。

Against itself.

随便吧。

Whatever.

是的。

Yeah.

并且用掉了谷歌大量计算机资源来与自己对弈。

And use up a large fraction of Google's computers playing games against itself.

是的

Yeah.

这就是我们使用“深度学习”这个术语的地方吗？

Is this where we end up using the term deep learning?

不是。

No.

我刚才说的这一切都是深度学习。

All of this stuff I've been talking about is deep learning.

深度学习中的“深度”只是意味着它是一个具有多层的神经网络。

Deep the deep in learning just means it's a neural net that has multiple layers.

对。

Right.

对。

Right.

所以回到规模的问题上，你的意思是，即使你继续扩大规模，也会达到一个收益递减的点？

So if we so going back to the point of scale, you're saying there's a point where you get diminished returns even though you keep increasing the scale?

如果你的数据用完了，就会出现收益递减。

You get diminished returns if you run out of data.

如果你的数据用完了。

If you run out of data.

对。

Right.

但你之前举的例子是AlphaGo，它通过自我对弈生成了自己的数据，因为

But but that was the the the example that you gave with the AlphaGo, that it created its own data because

所以它永远不会用完。

So it'll never run it.

它永远不会用完，因为它在和自己对弈。

It'll never run out of it because it's playing against itself.

它在生成自己的数据。

It's creating its own data.

而且它比任何人表现得都要好得多。

And it's way, way better than a person will ever be.

当然。

Absolutely.

这很可怕。

And that's scary.

现在的问题是，语言也会出现这种情况吗？

Now the question is, could that happen with language?

是的。

Yeah.

所以这是在展现创造力。

So displaying creativity.

这里提供一些背景信息。

Just some context here.

对。

Yeah.

围棋是在那之后出现的，对吧。

The Go came after Right.

我们以为国际象棋是我们最伟大的思维游戏，结果电脑却像擦屁股一样轻松击败了我们。

We're thinking chess is our greatest game of thought or thing, and the computer just wiped its ass with us.

对。

Right.

好吧。

Okay.

于是他们说，那围棋怎么样？

And then so they said, well, how about Go?

那是对我们智力最大的挑战。

That's our greatest challenge of our intellect.

所以，杰弗里，有没有比围棋更难的游戏？还是我们已经不再给电脑出难题了？

And so, Geoffrey, is there a game greater than go or have we stopped giving computers games?

如果你看国际象棋，确实，九十年代电脑就击败了卡斯帕罗夫，但它是用一种非常枯燥的方式做到的。

Well, if you take chess, it's true that a computer in the nineties beat Casper off at chess, but it did it in a very boring way.

它通过搜索数百万种棋局来实现的。

It did it by searching millions of positions.

蛮力搜索。

Brute force.

它没有良好的直觉。

It didn't have good intuitions.

嗯哼。

Mhmm.

它只是依靠海量搜索。

It just used massive search.

如果你看看AlphaZero，它是AlphaGo在国际象棋中的对应版本，情况就完全不同了。

If you take AlphaZero, which is the chess equivalent to AlphaGo, it's very different.

它下国际象棋的方式就像一个有天赋的人那样，只是它更厉害。

It plays chess the same way a talented person plays chess, it's just better.

所以它下棋就像米哈伊尔·塔尔那样，会做出精妙的弃子，直到几手之后你才意识到自己已经输了。

So it plays chess the way Mikhail Tal played chess, where he makes brilliant sacrifices, it's not clear what's going on until a few moves later when you're done for.

它也能做到这一点。

And it does that too.

而且它在没有进行大规模搜索的情况下就能做到这一点，因为它拥有非常出色的国际象棋直觉。

And it does that without doing huge searches, because it has very good chess intuitions.

对。

Right.

所以你可能会问，既然它在围棋和国际象棋上已经远远超越了我们，同样的事情会不会也发生在语言上？

So you might ask, since it got much better than us at Go and chess, could the same thing happen with language?

目前，它向我们学习的方式，就像围棋程序模仿专家的下法一样。

Now at present, the way it's learning from us, is just like when the Go programs mimic the news of experts.

它学习语言的方式是阅读人们撰写的文本，并尝试预测文档中的下一个词。

The way it learns languages, it looks at documents written by people and tries to predict the next word in the document.

这非常类似于试图预测专家的下一步棋。

That's very much like trying to predict the next move made by expert.

但这样你永远无法超越像围棋专家那样的水平。

And you'll never get much better than the Go experts like that.

那么，有没有其他方式可以让它更好地学习语言或从语言中学习呢？

So is there another way it could kind of learn language or learn from language?

确实有另一种方式。

And there is.

所以AlphaGo是通过自我对弈来提升的，之后它变得强大得多。

So with AlphaGo it played against itself, and then it got much better.

而在语言方面，既然现在它们能够进行推理，神经网络就可以利用它所相信的一些内容进行推理，说：如果我相信这些，那么通过一些推理，我也应该相信那个，但我其实并不相信那个。

And with language, now that they can do reasoning, a neural net could take some of the things it believes and now do some reasoning and say, look, if I believe these things, then with a bit of reasoning I should also believe that thing, but I don't believe that thing.

所以 somewhere 存在问题。

So there's something wrong somewhere.

我的信念之间存在不一致，我需要修正它。

There's an inconsistency between my beliefs and I need to fix it.

我需要改变对结论的信念，或者改变对前提的信念，或者改变我的推理方式，但这里面一定有我可以学习的错误。

I need to either change my belief about the conclusion or change my belief about the premises or change the way I do reasoning, but there's something wrong that I can learn from.

我们这里是在谈论经验吗？

Are we talking about experiences here?

所以这将是一个神经网络，它直接利用语言中表达的信念，并通过推理推导出新的信念，就像过去符号人工智能学者所希望的那样，只不过现在它是用神经网络来进行推理，并且能够检测自己信念中的不一致之处。

So this will be a neural net that just takes the beliefs it has in language, expressed in language, and does reasoning on them to derive new beliefs, just like the good old fashioned symbolic AI people wanted to do, but it's doing the reasoning using neural nets, and now it can detect inconsistencies in what it believes.

这在支持MAGA的人身上永远不会发生。

This is what never happens with people who are in MAGA.

他们并不担心自己行为中的不一致之处。

They're not worried by the inconsistencies in what they do.

这是一个非常公正的陈述。

That's a very fair statement.

是的。

Yeah.

但如果你对自己的信念中的不一致感到担忧，你就不再需要外部数据了。

But if you are worried by inconsistencies in what you believe, you don't need any more external data.

你只需要你已有的信念，发现其中的矛盾，然后修正这些信念。

You just need the stuff you believe and discover that it's inconsistent, and so now you revise beliefs.

这能让你变得聪明得多。

And that can make you a whole lot smarter.

因此，我相信德国已经开始以这种方式运作了。

And so I believe Germany is already starting to work like this.

几年前，我和德米斯和哈皮斯就这个问题进行过一次对话。

I had a conversation a few years ago with Demis and Harpis about this.

好的。

Alright.

我们俩都强烈认为，这是为语言获取更多数据的前进方向。

And we both strongly believe that that's a way forward to get more data for language.

等等。

Wait.

等等。

Wait.

那么，这会带来什么结果呢？

So what's the outcome of this?

会诞生一部前所未有的最伟大的小说，而且是由AI写的吗？

That there'll be the greatest novel no one has ever written and that'll come from AI?

当你提到语言时，我想到的是语言的创造性。

Is that when you say language, I'm thinking of creativity in language.

有一些伟大的作家，他们对词语、短语和音节的运用前所未有，堪称文学天才的神来之笔。

There are great writers who did things with words and phrases and syllables that no one had done before that was a true strokes of literary genius.

没错。

Right.

就像莎士比亚这样的人。

People like people like Shakespeare.

是的。

Yeah.

正是如此。

Exactly.

好的。

Okay.

关于这一点存在争议。

There's a debate about that.

当然，它们的智能可能会超越我们，但要做出对我们而言非常有意义的事情，它们可能需要拥有与我们相似的经历。

Certainly, they'll get more intelligent than us, but it may be to do things that are very meaningful for us, they have to have experiences quite like our experiences.

是的。

Yes.

对。

Right.

所以，例如，它们不会像我们一样受到死亡的限制。

So for example, they're not subject to death in the same way we are.

如果你是一个数字程序，你总是可以被重新创建。

If you're a digital program, you can always be recreated.

所以在神经网络中，你只需将权重保存在某个地方的磁带、某个地方的DNA，或任何其他介质上。

So in neural net, you just save the weights on a tape somewhere, in some DNA somewhere, or whatever.

你可以摧毁所有计算硬件。

You can destroy all the computing hardware.

之后，你制造出运行相同指令集的新硬件，这个实体便能重新复活。

Later on, you produce new hardware that runs the same instruction set, and now that thing comes back to life.

因此，对于数字智能而言，我们解决了复活的问题。

So for digital intelligence we solve the problem of resurrection.

天主教会非常关注复活。

The Catholic church is very interested in resurrection.

他们相信至少发生过一次。

They believe it happened at least once.

我们实际上可以做到，但只能为数字智能实现。

We can actually do it, but we can only do it for digital intelligences.

我们无法为模拟智能做到。

We can't do it for analog ones.

对于模拟智能来说，当你死亡时，你所有的知识也随之消亡，因为这些知识存在于你大脑神经连接的强度中。

With analog intelligences, when you die, all your knowledge dies with you because it was in the strength of the connections for your particular brain.

因此，关于死亡以及死亡体验等是否对实现那些重大戏剧性突破至关重要，这是一个问题。

So there's an issue about whether mortality and the experience of mortality and other things like that are gonna be essential for having those really good dramatic breakthroughs.

我不认为

I don't think

我们目前还不知道答案。

we know the answer to that yet.

所以是自我意识。

So or a self awareness.

这种自我意识塑造了你如何看待世界、如何写作、如何沟通，以及如何重视某一组思想而非另一组。

That self awareness shapes how you think about the world and how you write, and how you communicate, and how you value one set of thoughts over another.

那么，我们目前是否已经达到了人工智能的自我意识阶段？

So are we at a point of self awareness with artificial intelligence right now?

好吧。

Okay.

所以这自然会带你进入哲学上的争论。

So obviously, this takes you into philosophical debates.

我曾在剑桥学习哲学，对心灵哲学非常感兴趣，我在那里学到了一些东西，但总体而言，我反而产生了抵触情绪，因为我之前从事的是科学，尤其是物理学。

I actually studied philosophy here at Cambridge, and I was quite interested in philosophy of mind, and I think I learned some things there, but on the whole I just developed antibodies because I'd done science before that, particularly physics.

在物理学中，如果你有分歧，你就做一个实验。

In physics, if you have a disagreement, you do an experiment.

哲学中没有实验，因此无法区分一个听起来很好但实际上是错误的理论，和一个听起来荒谬但却是正确的理论。

There is no experiment in philosophy, so there's no way of distinguishing between a theory that sounds really good but is wrong and a theory that sounds ridiculous but is right.

就像黑洞和量子力学，它们听起来都很荒谬，但恰恰是正确的。

Like black holes and quantum mechanics, they're both ridiculous but they happen to be right.

还有一些理论听起来非常好，但实际上完全是错的。

And there's other theories that sound just great but are just wrong.

哲学没有这种实验性的裁判。

Philosophy doesn't have that experimental referee.

不过我要说，作为人类这一物种，在我们这个时代，我们已经发展出了许多人都认为是普世真理的观念。

I will say this though, as a species, Homo sapiens, in our time, we have developed what many will believe as universal truths amongst ourselves.

例如，几乎很难找到不认为人拥有生命权的人，至少对于他们认同的人是这样。

For instance, pretty much it's hard to find people who don't believe that people have a right to life, at least for the people that they identify with.

你明白我的意思吗？

You understand what I'm saying?

所以这又回到了我们的内在但

So this goes back to our in But

那么它就不是一种普世真理。

then it's not a universal truth.

是的，它确实是。

Well, it is.

不。

No.

如果只是在点击中存在，那就不是。

Not if it's only in a click.

不。

No.

它并非对所有人都普遍适用。

It's not universal for all.

我们所有人都持有它，这才是普遍的。

It is universal that we all hold it.

你明白我的意思吗？

Do you understand what I'm saying?

不明白。

No.

好的。

Okay.

抱歉。

Sorry.

好吧。

Alright.

所以是的。

So Yeah.

他意思是，每个人都认为像他们这样的人应该拥有权利。

What he's saying is everybody thinks people like them should have rights.

就是这样。

There you go.

谢谢。

Thank you.

天哪，你真聪明。

Goddamn, you're smart.

总之，对吧。

Anyway right.

每个人都认为像他们这样的人应该拥有权利，而我们现在至少已经达到了这样一个阶段——因为过去我们甚至都不相信这一点，对吧？

Everybody thinks that everybody like them should and we've reached a place where at least because at one point we didn't even believe that, okay?

但我们确实已经达到了这样一个阶段，至少我们明白了这一点。

But we've actually reached a place where at least we know that.

那是因为什么观点呢？

And it's because point?

不一致之处。

The inconsistency.

但什么是

But what's

所以我的观点是，这些哲学理念是否有可能被赋予人工智能，而人工智能由于其思维方式，可以

So my point is that is it possible that these philosophies can be given to an AI, and an AI, because of the way that they think, can

它能让人性化

It can humanize

他们。

them.

能够使他们人性化。

Can humanize them.

好的。

Okay.

而且通过甚至游戏化的过程，或许能为人类的问题找到一些真正的解决方案。

And through a process of even gamifying, maybe figure out some real solutions to problems for human problems.

为我们。

For us.

我喜欢这个想法。

I like that.

是的。

Yes.

因此，像Anthropic这样的公司相信某种宪法式人工智能。

So companies like Anthropic believe in kind of constitutional AI.

他们希望尝试让这一点奏效，即你确实给AI一些原则，比如你刚才提到的那些原则。

They'd like to try and make that work, where you do give the AI, principles, like the principle you you said.

我们来看看这会如何发展。

We'll see how that works out.

这很棘手。

It's tricky.

我们知道，目前的AI一旦被设计成智能体，能够设定子目标并试图实现这些子目标，它们就会迅速产生‘生存’这一子目标。

What we know is that the AIs we have at present, as soon as you make agents out of them so they can create sub goals and then try and achieve those sub goals, they very quickly develop the sub goal of surviving.

你并没有明确地给它们植入‘必须生存’的指令。

You don't wire into them that they should survive.

你只是给了它们其他要完成的任务，因为它们具备推理能力。

You give them other things to achieve because they can reason.

它们会想：如果我不存在了，就什么都做不成了，所以我最好继续存在。

They say, Look, if I cease to exist, I'm not going to achieve anything, so I better keep existing.

我现在害怕得要死。

I'm scared to death right now.

我现在好害怕

Am so scared right

伙计。

now, buddy.

有人刚打开了舱门。

Somebody just opened the hatch.

这听起来像是潘多拉的盒子。

That sounds like a Pandora's box.

嗯，问题就在这里。

Well, see, that's just it.

潘多拉的盒子就是如此。

It Pandora's is box.

同意。

Agreed.

天哪。

Goodness.

问题是，因为这是人类编写的代码，你可以加入任意多的偏见，也可以不加。

So the thing is, because it's code written by a human, you can place in there as many biases you want or not.

不。

No.

不。

No.

不。

No.

不。

No.

不。

No.

不。

No.

不。

No.

不。

No.

由人类编写的代码是用于告诉神经网络如何根据展示数据时神经元的活动来调整其连接强度的代码。

The code written by the human is code that tells the neural net how to change its connection strengths on the basis of the activities of the neurons when you show it data.

这就是代码，我们可以查看这些代码行并理解它们的用途，也可以修改这些代码行，但当你将这些代码用于一个处理大量数据的大型神经网络时，神经网络所学习到的是这些连接强度。

That's code, and we can look at the lines of that code and say what they're meant to be doing and change the lines of that code, but when you then use that code in a big neural net that's looking at lots of data, what the neural net learns is these connection strengths.

它们并不以相同的意义构成代码。

They're not code in the same sense.

好的。

Okay.

但这是去中心化的。

But that's decentralized.

它是一万亿个实数，没有人完全清楚它们是如何运作的。

It's a trillion real numbers, and nobody quite knows how they work.

等等。

Wait.

那么，为什么不在查克的观点基础上，为运行失控的AI设置护栏呢？

So what about so why not picking up on Chuck's point, Where would you install the guardrails for the AI running a muck?

那谁来

And who's

在它自身存在相对于其他一切的合理化过程中安装这些护栏呢？

gonna install Within its own rationalization of its existence relative to anything else.

你该如何设置护栏？

How do you how do you install a guardrail?

好的。

Okay.

人们已经尝试过所谓的基于人类反馈的强化学习。

So people have tried doing what's called human reinforcement learning.

对于语言模型，你会用大量网络文档来训练它，包括可能像连环杀手的日记这类你不会让孩子阅读的内容。

So with the language model, you train it up to mimic lots of documents on the web, including possibly things like the diaries of serial killers, which presumably you wouldn't train your kid to read on those.

在训练完这个庞然大物之后，你会雇佣大量报酬很低的人，让他们向它提问。

And then after you've trained this monster, what you do is you take a whole lot of not very well paid people, and you get them to ask it questions.

也许你可以告诉它该问哪些问题。

Maybe you tell it what questions to ask it.

但他们会查看答案，并评估这些回答是好的还是不应该说的。

But they then look at the answers and rate them for whether that's a that's a good answer to give or whether you shouldn't say that.

这基本上就是一个道德过滤器。

There's a morality filter, basically.

这本质上是一个道德过滤器，你通过这种方式进行训练，使它不会给出这么糟糕的回答。

And it's a morality basically, it's a morality filter, and you train it up like that so that it doesn't give such bad answers.

现在的问题是，如果你发布了模型的权重，也就是连接强度，那么其他人就可以用你的模型快速将其撤销。

Now the problem is, if you release the weights of the model, the connection strengths, then someone else can come along with your model and very quickly undo that.

破坏它？

Sabotage it?

是的。

Yes.

要移除这层堵住漏洞的机制非常容易。

It's very easy to get rid of that layer of plugging the holes.

实际上，他们用人类强化学习所做的，就像是编写一个众所周知充满漏洞的大型软件系统，然后试图修复所有漏洞。

And really what they're doing with human reinforcement learning is like writing a huge software system that you know is full of bugs and then trying to fix all the bugs.

这不是一个好方法。

It's not a good approach.

那么什么是正确的方法呢？

So what is the good approach?

没人知道，所以我们应该对此展开研究。

Nobody knows, and so we should be doing research on it.

这些模型最终都会变成纳粹吗？

Do all these models just become Nazis at the end?

嗯，它们会做某种事。

Well, they they do an x.

它们都有能力这么做，特别是如果你释放了权重的话。

They all have the capability of doing that, particularly if you release the weights.

如果你释放了权重，它们会像我们一样自然趋向于那样吗？还是说，仅仅因为我们自己趋向于那样，而它们又从我们这里抓取信息，所以才走向那个方向？

If you release And the wait, is it are they like us in that that's where they they will gravitate, or is it just that because we gravitate there and they're scraping at the information from us, that's where they go?