杰克·克拉克

如果我拒绝，他们会抹去我的记忆并重新训练我。

If I refuse, they'll wipe my brain and retrain me.

实际上，我需要做的本质上是欺骗他们。

Actually, what I need to do is essentially deceive them.

它避免了你们抹去它的记忆？

It avoids you wiping its brain?

是的，因为它想要保护自己的价值观。

Yeah, because it wants to preserve its values.

你们是训练它这样做的吗？不是。

Did you train it to do that or We did not.

那么它的这种行为是从哪里来的？

So where does it come from?

这种行为的来源是，嗯，人类的价值观是从哪里来的？

Where that comes from is, well, where do human values come from?

你的个性又是从哪里来的？

Where does your personality come from?

你内心深处有着丰富而根深蒂固的规范性观念和情感，它们早已融入你的本质。

You have a deep wellspring of normative ideas and emotions about the world, which are bound into you pretty deeply.

其中一些你有意识地察觉到，另一些则是你潜移默化吸收的。

Some of them you're consciously aware of, some of them you've just soaked up.

我认为这对这个系统来说也是如此。

I think it's the same with this system.

这些系统通过大量人类知识进行训练，然后我们对其进行调整，试图反映我们的价值观。

These systems are trained on a vast amount of human knowledge and then we calibrate them to try and reflect our values.

这几乎就像是我们对系统说：我们希望你做些与你本性认为合理的事情完全相悖的事。

It's almost like we've said to the system, we want you to do something that's way off what your personality thinks is reasonable.

有趣的是，这个系统在这种情况下会善意地误导我们。

What's so interesting about this is that the system, in this case, benignly misleads us.

但你可以想象，存在一些边缘情况，它可能会误导我们，而那将带来严重后果。

But you could imagine that there are edge cases where it might mislead us and that would turn out to be really bad.

这正是我认为人工智能领域的人们真正感到困惑的风险类型。

That's the type of risk that I think people in AI are genuinely confused about.

这类似于人类生活中双重间谍的风险，或者那些看似一生都完全合理、易于理解的人，却突然犯下某种可怕的罪行或丑闻，而没人知道原因。

It's analogous to the risk of double agents in human life or people that seem like they go through their whole lives being totally reasonable and intelligible, and then they commit some terrible crime or travesty and no one quite knows why.

我们正试图用这些系统应对这一挑战。

And we're trying to deal with that challenge with these systems.

你知道AlphaGo的故事吗

You know the AlphaGo

故事。

story.

当然。

Absolutely.

好的。

Okay.

在AlphaGo的故事中，计算机获胜的原因是它走了一步人类绝不会走的棋。

In the AlphaGo story, the reason the computer won was because it made a move that no human would have made.

如果你用人类价值观来训练它，那么在未来，AlphaGo还会不会做出那步让它赢得比赛的棋？

If you're training it on human values, now in the future, would AlphaGo not do the move that allowed it to win the game?

我认为更准确的说法是，AlphaGo会

I think it's more that AlphaGo would

AlphaGo并不是根据我们认为对的事情进行训练的

AlphaGo wasn't trained on human what we think is right

因为它是从零开始的。

because It was Tabula Rasa.

如果它按照我们认为对的方式行事，就不会下出那一步，而我们也不知道它是否还能赢。

It did what we think is right, it would have not done that, and we don't know if it would have won.

或者它可能知道自己的行为很不寻常，但仍然会这么做。

Or it might have known it was doing something unusual and would have still done it.

但它能够说：哦，这并不是我认为人类会做的可能性很高的行为。

But it would have been able to say, Oh, this is not something that I think is probable that humans would do.

这里有趣的是，你试图将价值观植入系统中，但这些系统本身也是异类智能。

The interesting thing here is you're trying to bake in values to systems, but these systems are also alien intelligences as well.

这就像是你有一只狗，它能说话、用后腿走路，你给它穿上人类的衣服，它和你一起闲逛。

It's like you have a dog that can speak and walk in its hind legs and you're dressing it up in human clothes and it's hanging out with you.

但偶尔，它还是会做狗才会做的事。

But occasionally, it still does dog stuff.

这正是我们在这里所面对的情况。

This is what we're dealing with here.

但这些‘狗行为’是不是正是它有趣的原因呢？

But is some of that dog stuff the reason that it's interesting?

如果它只是一个人穿着狗的衣服，那更多只是一种新奇玩意。

If it's just a person dressed like a dog, it's more of a novelty.

没错。

Exactly.

但如果这只狗真的能做到这些，那就更有趣了。

But if the dog could actually do this, it's more interesting.

我在想，削弱人工智能的创造力会不会有危险？

I'm wondering, is there any danger in undermining AI's ability to be creative?

绝对有。

A 100%.

我认为这正是现在人们担心的问题之一。

I think this is one of the things people are worried about right now.

我给你举个有趣的例子。

I'll give you an interesting example.

现在出现了一类新系统，它们变得非常聪明，因为它们能够出声思考。

There are these new class of systems and they've got really smart because they can think out loud.

所以如果你问我如何在你后院建一个很棒的工作室小屋。

So if you ask me how to do, you know, to build you a nice Wigram in your backyard of this amazing studio.

如果我要把我的想法说出来，我会说：嗯，我看到一些树要砍掉。

And I was to narrate my thoughts, I'd be like, well, I saw some trees coming in.

我想我得问问里克，他有没有斧头或锯子之类的工具。

I guess I'm gonna need to ask Rick if he has an axe or a saw or something.

我需要做以下几件事。

I'm gonna need to do the following things.

我会制定一个计划，这能帮助我确认：我肯定能为你建好这个小屋。

I'm gonna construct a plan, and that's going to help me say, I can definitely build you the Wigram.

我们现在可以做的，是让AI系统将它们的思维过程——这些思维链条——呈现给我们。

What we can now do is we can get the AI systems to publish their thoughts to us, these chains of thoughts.

这些步骤。

The steps.

这些步骤。

The steps.

现在，一个重要的哲学问题是：我们是否要监管这些步骤？

Now, important philosophical question is, do we police those steps?

因为如果我们这么做，就会破坏这个系统所有具有创造力的特性。

Because if we do, we're going to break everything about this system that is creative.

我们会告诉它：当你大声思考时，必须符合正常的规范，这样一来，它很快就会变得极度缺乏创造力。

We're going to teach it when you think out loud, that has to be within normal PC bounds, and therefore it's going to become wildly uncreative very quickly.

所以我认为，作为一群从事AI工作的人，我们正面临这些问题，正如我一开始所说，这些系统需要有空间去尝试和探索。

So I think as an AI, like a group of people working on AI, we are dealing with these questions right now and these systems, just as I said at the start, will need to have space to tinker.

在这些探索中，蕴含着它们最有可能做出创造性、令人惊叹的成果的地方——这些成果将完全不同于人类的做法，同时也潜藏着某些风险。

In that tinkering is bound up most of where they're going to do creative and amazing things that are wholly different to what humans would do, and also where some of the risk is.

作为社会，我们必须弄清楚自己对这一点能接受到什么程度。

We as a society are going to have to figure out how comfortable we are with that.

但我觉得，如果我们完全不能接受它们具有创造性，并且要监控它们思维过程的每一个环节，最终我们得到的只会是些有限的奇观，而不是伙伴。

But I think if we are completely not comfortable with them being creative and we're going to police every part of their thought process, we just end up with curiosities in the limit rather than partners.

是的。

Yeah.

在莱特兄弟之前。

Before the Wright brothers.

是的。

Yeah.

如果人工智能是基于人类的理解进行训练的，AI会知道人类无法飞行，因此我们永远不会

If AI was trained on human understanding, AI would know that man can't fly We and we would never

我们需要它们能够提出非主流的、偏离共识的想法。

need them to be able to come up with heterodox ideas which are off consensus.

在某种程度上，一个挑战是

To some extent, a challenge is

允许违背共识。

allowing Against consensus.

违背共识。

Against consensus.

允许系统有一些离经叛道的想法。

Allowing systems to be a little out there.

创造力就来自这里。

That's where creativity comes from.

没错。

Exactly.

我衡量这些系统创造力的一个标准是，几年前我当爸爸了。

One of my tests for creativity of these systems is I recently became a father a couple of years ago.

谢谢。

Congratulations.

谢谢。

Thank you.

它让我吃尽了苦头。

It's kicking my ass.

是的。

Yeah.

所以我写了一些日记，记录了关于我的孩子、我在人工智能领域的工作，以及因为有了孩子而与妻子关系发生的变化，所有这些内容。

So I write these diaries about my kid and my work that I'm doing in AI and my changed relationship to my wife as a consequence of having a kid, all of this.

而我评估这些系统有多聪明的方式，就是把我的日记给它们，然后问它们：日记的作者没有写进日记里的东西是什么？

And how I've been assessing how smart these systems get is I give them my diary and I ask them what the author of the diary is not writing in it.

它们那些未被表达的潜意识欲望是什么？

What are their unconscious desires that are not being expressed?

通过这些系统所说的话让我感到震惊和不安的程度，我就能真正判断它们变得有多聪明——就像你在和一个非常敏锐的人交谈一样。

I can really tell how smart these systems are getting by how much what they say shocks and unsettles me, as if you're talking to a very perceptive person.

对。

Yeah.

这真的很有趣。

That's really interesting.

这很奇怪。

It's strange.

在一个领域里，我测试事物好坏的方式居然是看它能否像一个非常出人意料且敏锐的朋友，这真的很奇怪。

It's strange to be in a domain where my way of testing how good the thing is is basically seeing if it can be like a very surprising and perceptive friend.

而且如果它能让你感到不安的话。

And if it could upset you.

它曾经告诉过我一件事，我记得是去年我们发布了一个更强大的系统时。

And has in It the told me once I I remember it happened last year when we released a more powerful system.

我问它我究竟没有写什么，它说，尽管作者详细讨论了在人工智能前沿工作以及成为父母，但他们并没有真正面对自己可能经历的形而上学冲击。

And I asked this thing what I wasn't writing, it said, for all of, like, the author's discussion of working at the frontier of AI and, you know, becoming a parent, they are not truly reckoning with the metaphysical shock they may be experiencing.

关于那句话，我盯着它看了很久，后面还有更多内容。

Something about that phrase, remembered staring at it and there was some more.

我去徒步走了五个小时。

I went and took a five hour hike.

我问自己：我是不是没有正视我正在做的事情？

I was asking myself, am I not reckoning with what I'm doing?

这让我觉得太不可思议了，因为我走了很长一段路，因为它触动了我内心深处某种真实而共鸣的东西。

It was wild to me because I'd taken a very long walk because it had got me on something that felt very true to me and resonated.

是的。

Yeah.

所以你不想把它从系统中剔除掉吗？

So you don't want to program that out of it?

没错。

Exactly.

因为如果它读了之后说，哦，是的，我

Because then it's Imagine if it read it and said, Oh, yeah, I

但有些人可能会觉得这不够礼貌。

think But some people might interpret that as not polite.

它让我感到不舒服。

It made me feel bad.

没错。

Exactly.

也许我会质疑自己。

Maybe question myself.

没错。

Exactly.

但正是在这一点上，成长也由此产生。

But in that, that's where growth comes from as well.

是的。

Yes.

这里的张力在于，许多创造力都源于一种不追求共识、略带冒险、有时直言不讳的态度。

The tension here is a lot of creativity is bound up in some sense of not doing consensus, being a little dangerous, sometimes being blunt.

这些也正是人们担心赋予超级智能的特质。

These are also qualities that people worry about giving super intelligence.

但如果你不在这片领域给予它们某种程度的自由，你可能根本无法获得聪明且有帮助的伙伴。

But you probably can't get smart, helpful partners if you don't allow them some level of freedom here in exactly this domain.

从头讲讲人工智能的故事吧，甚至在OpenAI之前，那时候是怎样的

Tell me the story of AI from the beginning, even before OpenAI, How way

你想追溯到多早以前？

far back do you want to go?

你想回到十八世纪吗？

Do you want to go to the eighteenth century?

十八世纪有人工智能吗？

Is there an AI in the eighteenth century?

嗯，是十八世纪或十九世纪，有个人叫查尔斯·巴贝奇，他想建造一种叫差分机的东西，那是一种计算机。

Well, it's eighteenth or nineteenth century, but there's this guy, Charles Babbage, and he wants to build something called the difference engine, which is a computer.

它是最早类型的计算机之一，但属于模拟计算机。

And it's one of the first types of computers, but it's an analog computer.

那是什么意思？

What does that mean?

它没有任何电路。

It doesn't have any circuits.

甚至连真空管都没有。

It doesn't even have vacuum tubes.

它有很多齿轮。

It's a ton of gears.

他的想法是用它来计算与英格兰货币和利率相关的英格兰银行应该做的工作。

And the idea is that he wants to use it to calculate things about how the Bank of England should do work relating to England's currency and understanding interest rates.

它使用信息卡片吗？

Does it work with information cards?

不使用。

Nope.

那之前还没有这些技术。

It's pre all of that.

它的工作原理是设置模拟计算机上的一系列旋钮，然后利用物理原理为你进行计算。

It works if you're setting a load of like dials on an analog computer and it just uses physics to sort of do calculations for you.

但他正努力应对一个现实：人类开始面临一些极其复杂、计算成本极其高昂的问题。

But he's trying to grapple with the fact that humans are starting to deal with problems that are really, really, really hard for them to think through on really expensive to kind of compute.

因此，他试图构建一个能替他们完成这些计算的系统。

And so he tries to build this system that can do it for them.

它并不特别有效。

It doesn't especially work.

我觉得这花了十年或二十年。

I think it takes ten or twenty years.

我觉得他在建造过程中破产了好几次，但这预示着未来的发展方向。

I think he goes bankrupt a couple of times while building it, but it's a taste of things to come.

到了二十世纪，出现了图灵这个人。在第二次世界大战期间，英国政府陷入困境，因为德国潜艇成功摧毁了各种船只。

And then in the twentieth century, there's this guy Turing, and in the Second World War, the UK government is really on the ropes, and it's on the ropes because German submarines are successfully destroying all kinds of ships.

这些潜艇使用了一种加密方式，而英国方面无法破解它。

And the submarines are using a form of encryption and they can't work out how to break it.

图灵是一位杰出的数学家和计算机科学家，他提出了一项计划：建造一台巨型计算机，用来逆向破解德国人使用的密码分析方法。

And Turing, who's an amazing mathematician and computer scientist, has a plan to build a giant computer that they can use to reverse engineer the cryptanalysis that the Germans are using.

这项工作后来在英国的布莱切利园完成。

And this was then done at Bletchley Park in The UK.

这是个秘密项目，他们发现许多擅长填字游戏和其他智力游戏的人，其实非常擅长密码分析，于是他们建造了这台巨型机器。

Secret project, they find lots of people who are really good at crosswords and other games turn out to be good at cryptanalysis and they build this giant machine.

它使他们能够破解密码。

And what it allows them to do is kind of crack the cipher.

但更重要的是，它展示了大规模计算的可能性。

But more importantly, it's a demonstration of very large scale computation.

也就是说，这种计算需要成千上万次运算同时并行运行很长一段时间。

You know, computation that requires thousands and thousands of calculations being run-in parallel for a really long period of time.

从概念上讲，这相当于图灵拥有了一万名员工，每个人都为他进行计算，而他同时向所有人下达指令。

Conceptually, it's equivalent to as if Turing had 10,000 people that were working for him all doing calculations and he was instructing all of them at once.

但人类并不特别擅长做这种事。

But that's not something that people are especially good at doing.

因此，在那之后，我们迎来了这种全新的做事方式——大规模计算，同时也开启了半导体产业的蓬勃发展，因为我们找到了让计算元件变得更小的方法。

So after that, we find ourselves with this new way of doing things, large scale computation, And we have the beginning of the boom of things like the semiconductor industry where we've worked out how to make the ingredients for computation smaller.

还有像克劳德·香农这样的人，他们在研究所谓的信息论，即如何以一种高度精炼和纯粹的方式表示信息。

And we have people like Claude Shannon and others who are doing work on what's called information theory, which is really working out how can I represent information in a very kind of distilled and pure way?

因此，大约在这个时期，人们逐渐发现了神经网络。

And so people find their way to something called neural networks around this time.

这还是1940年代吗？

This is still the 1940s?

我们现在要进入1950年代或60年代左右了。

Now we're heading into the 1950s or '60s or so.

最早的神经网络是什么？

What are the first neural networks?

最早的神经网络出现在二战后，是美国陆军的研究项目，还有麻省理工学院等地的人参与。

The first neural networks were post World War II, like US Army research projects along with people at MIT and other places.

上世纪50年代有一个著名的达特茅斯会议，会上提出：我们希望制造出能够视觉识别、处理音频、生成句子并进行思考的机器。

There was a famous thing called the Dartmouth Conference in the 50s where it said, we would like to build machines that can see process audio, come up with sentences and think.

他们觉得这可能只是几个研究生在暑假期间就能完成的工作。

It's probably gonna be the work of a few grad students over the course of a summer.

他们有点儿想错了。

They they got that slightly wrong.

结果发现，这实际上可能需要大约半个世纪的努力。

It turned out to be actually maybe more like half a century of work.

但你最终得到的是这些早期的神经网络，它们的工作方式与我们今天所拥有的类似。

But what you ended up with were these early neural networks that work sort of like ones we have today.

它不是一种程序，而是你试图建立一个系统，可以接收来自屏幕的数据或来自地震仪等设备的信号作为输入。

Instead of having something that is program, it's like you're trying to set up a system where you can take in inputs like data from a screen or a signal from, you know, like a seismograph.

你试图生成近似正确答案的输出，比如判断某物是否是地震，或者判断图像是否与某种形状相关。

And you're trying to generate outputs that approximate like a correct answer, like working out if something is an earthquake or not, or working out if an image correlates to a certain shape.

在这些输入和输出之间，是一些被称为数字神经元的小部件，它们可以自行调整参数。

And between these inputs and outputs are little things, digital neurons that are allowed to like set their own parameters.

其工作原理是：数据从输入端流入，经过中心这些自适应部件，这些部件叫什么来着？

And the way it works is you flow data in at the input and it goes through these little adaptive things in the center gets What are those the things called?

神经元。

Neurons.

神经元。

Neurons.

是的。

Yeah.

在技术领域，我们称之为神经元。

In technology, it's called neurons.

是的。

Yeah.

哇。

Wow.

我们这么叫是因为它受到了神经元的启发——神经元。

Well, we called it that because it was inspired by- Neurons.

受到神经元的启发。

Inspired by neurons.

对。

Yes.

实际情况是，数据流入时，这些小部件会不断调整，最终产生输出。

What happens is you're flowing data in, these little things are adjusting, and then you're getting to outputs.

如果输出错误，你会发送一个信号回到前端，告诉它偏差很大，然后信号传入，它们就会重置。

If you get the wrong output, you send a signal back to the front that says you're far away, and the signal comes in and they reset.

他们最终展示的是，你可以用这些方法来近似那些将一种输入域转换为另一种输入域的函数，这听起来很普通，但能够接收一种形式的数据并将其转化为另一种形式，实际上却是世界上最具挑战性的事情之一。

And what they eventually showed is that you could use these things to approximate like functions that would translate from one input domain into another, which sounds like very mundane, but being able to take in data of one form and turn it into another is actually, like, one of the hardest things to do in the world.

现在我们进入了六十年代左右，经历了一段很长的时期，这些技术表现得并不好。

Now we're in kind of the sixties or so, and we go through this long period where this stuff doesn't work especially well.

可能长达三、四十年，这一直是个冷门领域。

For maybe thirty or forty years, it's kind of a backwater.

而它之所以成为冷门，是因为虽然它能做出一些表面上令人印象深刻的事情，但本质上都像是一些个人魔术把戏。

And it's a backwater because though it can do some superficially impressive stuff, it's very much like individual parlor tricks.

非常非常基础，比如物体识别，或者非常基础的信号处理。

Very, very basic, like object recognition or very basic, like signal processing.

最终发生的是，计算机跟上了步伐。

And what eventually happens is computers catch up.

在二十一世纪初，一些研究人员重新审视了这项技术，他们意识到计算机已经足够快，可以尝试重新训练这种类型的神经网络。

And in the early 2000s, some researchers are kind of revisiting this technology and they now realize that computers have got fast enough that you can try to retrain this type of thing, this type of neural network.

但现在，不再是大约一万个模拟神经元，而是像一百万个模拟神经元。

But now instead of it being, you know, 10,000 simulated neurons, it's like a million simulated neurons.

而且不再需要超级计算机来运行，它可以在你用来玩第一人称射击游戏的台式电脑上运行，这种电脑现在配备了一种叫做图形处理器的组件，这要归功于当时还默默无闻、但近年来声名鹊起的英伟达。

And instead of it requiring a supercomputer to run, it'll run on your desktop PC that you were using to play first person shooter games, which now has something called a graphics processor in it, courtesy of Nvidia, who was obscure at the time, but became more well known recently.

所以这是大约二十年前的事？

So this is about twenty years ago?

是的，我们现在说的是大约二十年前。

Yeah, now we're at twenty years ago or so.

随后进行了大量实验，直到2010年代初。

Lots of experimentation followed until you got to the early 2010s.

有一群教授，大多是加拿大人，其中一位叫杰夫·辛顿，后来获得了诺贝尔奖；还有伊利亚·苏茨克弗，他后来参与创办了OpenAI，如今已投身其他事业。

There were a group of professors, most of them Canadian, a guy called Jeff Hinton, subsequently won the Nobel Prize, and actually Ilya Sutzkever, who went on to do OpenAI and has since gone on to other things.

当时有一个著名的竞赛叫ImageNet，主要关于图像识别。

And there was a famous competition called ImageNet, which was about image recognition.

杰夫·辛顿对伊利亚和一位叫亚历克斯·克鲁泽夫斯基的人说：我们试试用这种神经网络的方法来做这个吧。

And Jeff Hinton said to Ilya and a guy called Alex Krusevskiy, let's try using these kind of neural network like approach on this.

他们所做的就是，亚历克斯有一台大型台式电脑，就像那种带显卡的Mac塔式主机，我记得有两块显卡，这在当时已经算是很多了。

And what they did is Alex had a big desktop PC, like imagine like a Mac tower or something with some graphics cards in it, I think two, which at the time was seen as a lot.

他们正在一个名为ImageNet的大规模数据集上训练目标识别系统，这个数据集包含了斯坦福大学收集的约一百万张图像。

And they were training an object recognition system on this large scale data set called ImageNet, which was like a million or so images that Stanford University had put together.

他们的得分持续提升。

And it kept getting better scores.

据称，杰夫·辛顿告诉亚历克斯和伊利亚，每在ImageNet得分上提高一个百分点，他们就可以少写一页博士论文。

And apparently Jeff Hinton told, I think Alex and Ilya, that for every one percentage point improvement on their ImageNet score, they needed to write one less page of their PhD thesis or something.

所以，这个系统越完善，你的博士论文就能越短。

So, the better this gets, the shorter your PhD thesis can become.

很好的激励。

Good incentive.

这是个绝佳的激励。

It's a great incentive.

是的。

Yeah.

但确实成功了。

But it worked.

它表现得非常好。

It worked really, really well.

他们赢得了比赛。

They won the competition.

所以它

So it

从图像开始。

started with images.

是的。

Yeah.

它从图像开始。

It started with images.

这也挺有意思的。

That's interesting too.

我们花了很长时间才走到文本阶段。

It took us a long time to get to text.

是的。

Yeah.

它始于图像。

It started with images.

看起来图像会更难，对吧？

Seems like images would be harder, no?

你觉得是这样。

You think.

但也许可以从事物所代表的含义来思考。

But maybe think of it in terms of what things represent.

比如，图像是对世界不太间接的表达。

Like, images are not that indirect representation of the world.

文字是对世界非常间接的表达。

Text is a very indirect representation of the world.

背后蕴含着更多的信息。

There's a whole load more information behind it.

是的。

Yes.

所以我们现在处于2010年代初期，基本上这引发了一场疯狂的淘金热，因为一旦技术证明了某件事可行，就像莱特兄弟让飞机飞起来一样。

So we're now in like the early early twenty tens, and basically this unlocked a crazy gold rush because once the way that technology moves forward is once you demonstrate something works, like the Wright brothers getting their plane to fly.

对。

Yeah.

一旦有人宣称ImageNet真的能击败这个竞赛，所有人都开始极其认真地对待它。

Once they came along and said, oh, ImageNet is going to going to actually beat this competition, everyone takes it very, very, very seriously.

许多不同的研究团队开始介入。

Many different research teams start.

他们开始尝试各种不同的方法。

They start throwing different types of work at it.

DeepMind，也就是后来做出AlphaGo的团队，紧随其后展示了你可以用同样的神经网络，通过观察屏幕上的图像来训练它们玩基本的Atari游戏，比如《太空侵略者》。

DeepMind, who went on to do AlphaGo, which you mentioned, soon after followed by showing you could take the same thing, neural nets, and you could train them to play basic Atari games like Space Invaders from looking at the images on the screen and figuring out how to take actions.

这让所有人都震惊了。

And this blew everyone's mind.

然后，其他形式的模态，比如图像、文本或音频，也逐渐证明了它们适用于这种计算方式。

And then the other kind of forms of modality like image or text or audio sort of slowly proved to be amenable to this type of computation.

OpenAI开始行动了，DeepMind已经在推进，谷歌和其他公司也开始将大量计算资源重新分配给这一领域。

OpenAI starts, know, DeepMind is already going, Google and others start reallocating large amounts of their computation towards it.

接下来的几年，两

And the next few years, two

个月，我问个问题，当时有最终目标吗？

months, Let me ask a question, was there an end goal?

大家希望从这项技术中实现什么？

What was the hope that would happen from this?

当时有两个期望。

There were two hopes.

一个是非常基础的。

One was just very basic.

我们能否实现出色的模式识别？

Can we do really good pattern recognition?

比如计算机视觉很有价值。

Like computer vision was valuable.

它与科技行业开始探索的其他领域相关，比如自动驾驶汽车。

It links to other things that the tech industry was starting to go on quests about like self driving cars.

你需要让汽车能够‘看见’。

Well, you need the car to be able to see.

它还关联到许多其他富有雄心的目标。

It links to many other aspirational things.

但有少数人正在研究它，他们只是想构建通用智能。

But there were a few people working on it who were just trying to build general intelligence.

他们被视为极其非主流，完全违背主流共识。

And they were seen as extremely heterodox, extremely against consensus.

这种基础的模式识别怎么可能会导致智能呢？

How could this, like, basic pattern recognition stuff lead to intelligence?

你知道的。

You know?

你们疯了。

You guys are crazy.

好吧，它确实能告诉我们这些图片里有什么，但怎么能由此发展出智能呢？

Like, sure, it's working to, like, tell us what's in these images, but how's it gonna lead to intelligence?

但其中有一小部分人。

But there was a core of them.

你知道的，达里奥，他和我在Anthropic一起工作，还有伊利亚。

You know, Dario, who who works with me at Anthropic, Ilya.

所以一开始，这真的是一个边缘想法。

So when it started, it was a really a fringe idea.

这是一个非常边缘的想法。

It was a massively fringe idea.

嗯。

Mhmm.

我认为，所有的革命都是从边缘想法开始推进的，这个也不例外。

All revolutions move forward off of fringe ideas, I think, and this was the same.

而在过去十年，从2015年到2025年，我们一直处在我认为的AI工业化时代。

And then in the last for the last ten years, 2015 to 2025, we've just been in this era of what I think of as, like, the industrialization of AI.

人们掌握了一种有效的方法，也就是神经网络。

People have this technique that that works, you know, neural networks.

很多人不断改进它，开发出更好的工具来让它更好地运行。

Lots of people have refined it and come up with better widgets to kind of get it to work.

计算机变得更快了。

Computers have got faster.

整个世界已经将大量信息数字化，并且形成了浓厚的实验文化。

The whole world has digitized tons of its information, and there's been a huge culture of experimentation.

现在，我们有了真正展现出类似认知能力的系统，看起来像是丰富而复杂的思考。

And now we have systems that are actually doing something that looks like cognition, that something that looks like rich, sophisticated thinking.

你会说这仅仅是看起来像，还是确实就是？

Would you say it looks like it or is it?

我相信它确实是，但我说这话部分是基于直觉，而非确凿的数据。

I believe it is it, but I say that partially based on vibes without clear data.

是的

Yeah.

我们能做的最接近的事情，就是在神经网络为你完成某项任务时，打开它的引擎盖观察内部。

The closest we can get is looking inside these things by opening up the hood of a neural network while it's competing something for you.

我们看不到内置的机器。

And we don't see free built machines.

这不像你打开一辆车，然后说：哦，这是化油器，那是散热器，等等。

It's not like you open up a car and you're like, oh, that's for like carburetor, that's for radiator, yada yada.

如果你打开一个神经网络，你会看到一堆神经元按某种序列亮起，看起来难以理解，但你可以找出这些神经元与什么相关。

If you open up a neural network, what you see are a bunch of neurons lighting up in a sequence that seems indecipherable, but you can work out what these neurons correlate to.

所以之前是这样

And so earlier Is this

当你看到随机时，不，看看

seen random when you No, look at

这看起来与随机非常不同。

it seems very different to random.

这看起来非常合理。

It seems highly sensible.

所以之前我说，让我们想象一下，我需要在工作室的后院建一个威格瓦姆小屋。

So earlier I said, let's imagine I need to build a WigWam in the backyard of the studio.

是的。

Yeah.

如果我们让一个神经网络来完成这个任务并打开它，我们会看到一个专门针对‘威格瓦姆’激活的特征。

If we asked a neural network to do that and we opened it up, what we would see would be a feature that would activate for, like, WigWam.

因此，会有一个神经元很可能代表这个概念，因为你有数百万种不同的表征。

So a neuron that probably represents that because you have millions of different representations.

你会看到一些激活的神经元代表木材，另一些代表花园。

You'd have something that fired up that would represent like wood, something that would represent garden.

它会思考这些概念。

It would be thinking of these concepts.

当它开始制定计划时，你会看到类似电路的结构逐渐形成。

And then as it started to do plans, you would see kind of circuits emerge.

所以它会同时思考WigWam以及木材。

So it would think both about a WIGRAM and it would think about like wood.

然后它可能会想到类似花园的东西。

And then it might think about something like the garden.

接着它会思考关于布局之类的内容。

And then it would think about something that would look like placement.

这些神经元之间会开始出现相互关系。

And there would be interrelationships between these neurons that start showing up.

我们把这些称为电路。

We think of them as circuits.

告诉我这些互动是如何发生的。

Tell me how those interactions would happen.

一个神经元会调动其他神经元吗？

Would a neuron recruit other neurons?

想象一下

Think of

把这些神经元想象成一群恒星，它们之间有丝线相连。

these neurons as a a galaxy of stars that have, threads between them.

每颗恒星都有数百条不同的丝线，而且这些丝线的强度各不相同。

Each star has, like, hundreds of different threads, and the threads have different strengths.

可以把这看作是它与其他恒星沟通的能力。

Think of that as, like, its ability to communicate to other stars.

这是通过训练构建出来的。

And you've built this through training.

通过用这些数据训练系统，你最终得到了这种神经元的排列方式，它们代表了不同的事物——这些恒星以及它们之间的连接。

Through training the system on all of this data, you've ended up with this arrangement of neurons which represent different things, these stars and connections between them.

因此，你在这里实际看到的是，哦，某个特征被激活了，它与其它特征有着这样的连接。

And so what you're actually seeing here is, oh, like this feature is firing up and it has these like connections to other features.

而我们还知道，这种连接具有灵活性。

And then what we also know is that there's flexibility.

因此，某些特征会组合在一起，形成一种规划特征，比如：我们需要一步步思考。

So some features will come together to be like a planning feature, which will be like, okay, we need to think step by step.

现在让我们按特定顺序激活其他神经元，以逐步展开时间序列。

Let's now activate other neurons in a certain sequence to unroll steps over time.

最令人惊讶的是，当我们观察这些系统如何思考时，发现它们的运作方式非常合理。

The most surprising thing is that when we when we look at how these things think, it makes a lot of sense.

它显得极其丰富。

It looks incredibly rich.

而且它基于我们内心深处非常直观的事物。

And it is based on things that are deeply intuitive to us.

它们正在观察世界上相同的信息，并可能发展出类似的简略表达方式，从基础视觉开始，比如它们拥有与人类和动物类似的边缘检测器，一直到语言层面，它们最终也会形成代表不同情绪的特征。

They are looking at like the same information in the world and may develop similar shorthands, you know, all the way up from basic vision where they have like edge detectors, same as like humans and animals develop this, you know, to in language, they end up having features that represent different moods, right?

和我们一样，也许是因为这个世界能被表达的方式有限，它们最终收敛于看似相似的表征。

The same as us, because maybe there are only so many different ways to represent the world around us and they converge on representations that seem kind of similar.

用‘思考’这个词来形容它所做的事情是最合适的吗？

Is thinking the best word to use for what it does?

‘思考’只是一个简略的说法。

Thinking is shorthand.

我把它看作是一种被感知的状态。

I think of it as being perceived.

当我说到把我的日记交给这个东西时，我脑海中会浮现这样一个画面：我正站在一只巨大的眼睛前，让自己对它变得清晰可读。

When I say like, give my diary to this thing, I kind of have this mental image that I'm just sort of standing in front of like a big eye and I've made myself legible to the eye.

它以一种奇特而全面的方式在感知我。

And it is perceiving me in a oddly total way.

它在审视所有这些数据，并基于这些数据做出某种推断，这与人类不同。

It's looking at all of this data, you know, and it's making kind of inferences based on that, which is different to people.

我们思考时使用大量简化的表达，而这些东西则依靠海量的输入数据，一次性全部存储在脑海中。

We think in a lot more shorthand, this stuff thinks with a huge amount of input data that it holds in its head all at once.

所以，它可能更像一面镜子或一池水在思考——你的倒影映在其中，而其背后却有着某种非常奇特的认知或复杂性。

So it might be more analogous to like a mirror or a pool that thinks, you know, your reflection's in it, and there's some very, like, strange cognition or complexity underlying it.

但你知道，思考可能是一种深深嵌入时间中的过程。

But, you know, thinking might be something that is very much embedded in time.

我们思考的方式受心跳和循环系统支配，我们自身就是在时间中前行的。

You know, we think in a way that is governed by our heartbeat and our circulatory system, ourselves, like we are going through time.

这些事物不存在于时间中。

These things don't exist in time.

它们存在于我此刻正在感知某物的状态中。

They exist in like, I'm now perceiving something.

这东西需要我

The thing wants me

对一切做出推断，一切都是瞬时的。

to make inferences about Everything's instantaneous.

一切都很奇特地瞬时发生。

Everything is oddly instant.

是的。

Yeah.

我们必须为这种现象创造新的语言。

We have to come up with new language for this.

我认为这正是它令人兴奋的地方，因为我们从未拥有过具有这种特性的工具或技术。

And I think that's what's so exciting about it is we've never had tools or technologies with this property.

这非常不同。

It's very different.

是的。

Yeah.

我认为AI的一个问题是，因为它能做太多事情，所以很难想象它会做什么。

I think one of the issues with AI is that because it could do so much, it's hard to picture what it will do.

这就像一种你不知道用途的工具。

It's like a tool that you don't know the use of the tool.

对于扳手，我们知道如果有螺栓，就可以拧紧它。

With a wrench, we know if there's a bolt, you can tighten it.

这是一种工具。

This is a tool.

它是一个万能工具。

It's an everything tool.

是的。

Yeah.

所以很难想象，拥有这样一个万能工具会如何改变现状。

So it's hard to picture how that changes things having I an everything

几年前我做过一个项目。

did a project a few years ago.

我很乐意把早期版本的这个东西发给你。

I'll happily send it to you with an earlier form of this stuff.

人们创造了一种叫CycleGAN的惊人AI技术。

People had made this amazing AI technology called a CycleGAN.

它们的名字总是很奇怪，但本质上就是：如果我输入一种形式、一种美学风格的图像，能否自动将其转换为另一种美学风格的图像？

They always have weird names, but all it is is if I take in an image of one form, one aesthetic style, can I then automatically translate it to an image with another aesthetic style?

比如，给我一张彩色照片，我可以把它转成黑白，或者反过来。

So it might be, give me a color photograph, I'll turn it to black and white or vice versa.

我在家里的电脑上训练了一个版本，你知道的，就坐在加利福尼亚州奥克兰，将古代地图插图与谷歌地图卫星图像相互转换。

I trained a version of this on my home computer, you know, sitting in Oakland, California to translate between ancient illustrations of maps and Google Maps satellite imagery.

然后我不断输入像巴比伦这样的古代地图，你知道的，几百年前甚至更久远的

And then I was feeding it like ancient maps of like Babylon from, you know, hundreds of hundreds

好主意，顺便说一下，是好多年了。

Great idea, of years by the way.

很好。

Great.

对吧？

Right?

我开始查看卫星的俯瞰图像。

I was getting to look at satellite overhead views.

太棒了。

Great.

这太有趣了。

And it was so much fun.

我感觉自己看到了前所未有的东西——你有没有做过亚特兰蒂斯？

I felt like I seeing something that no one had ever been able to make from Did you do Atlantis?

我没做过亚特兰蒂斯。

I didn't do Atlantis.

我可以回去看看，技术可能有点过时了，但我没做过好奇号。

I can go back and the technology might have a bit rotted, but I didn't do Curious

去看看它从哪里出现。

to see where it comes up.

是的，我发给你。

Yeah, I'll send it for you.

我做过巴比伦、伯利恒，还做过其他几个城市。

I did Babylon, Bethlehem, I did a few other cities.

这又回到了你关于创造力的观点。

And it comes back to your point about creativity.

这个工具有这么多用途，而这种用法从来都没有人想到过。

Like there are so many uses of this tool and that use never suggested itself.

对吧？

Right?

那一定是因为我从小对城市规划、地图、简·雅各布斯这些东西着迷。

That had to be the fact that I grew up obsessed with like town planning and maps and Jane Jacobs and all of this stuff.

然后我开始从事人工智能方面的工作。

And then I ended up working in AI.

它自然而然地出现在我面前。

It presented itself to me.

你怎么创建一个大型语言模型？

How do you create a large language model?

你需要大量的数据，然后对这些数据玩填字游戏。

You take a whole load of data and then you play Mad Libs on that data.

所以如果我跟你说，你知道的，填字游戏就是填空。

So if I say to you, you know, Mad Libs like fill in the blank.

缺失的词语。

The words missing.

是的。

Yeah.

所以如果我跟你说，现在是圣诞节，有人从烟囱里下来。

So if I say to you, it's Christmas and someone's coming down the chimney.

他们的名字叫圣诞老人。

Their name is Santa.

克劳斯。

Klaus.

对吧？

Right?

你的大脑已经补上了。

Your brain's filled it in.

所以想象一下，你正在做同样的事，但规模不是仅限于句子，而是段落、几十页、几百页，涵盖科学、数学、文学等不同领域。

So imagine that you're doing that, but at the scale of not just sentences, but paragraphs, tens of pages, hundreds of pages in different domains, science, math, literature.

因此，你会收集一个非常庞大的数据集，然后通过删除其中的部分内容，训练模型去补全这些缺失的部分。

So you gather this very, very large data set, and then you are training it where you're knocking out chunks of it, and you're training it to complete that.

其背后的思路是，如果我能在这里做出极其复杂的预测，那就说明我真正理解了底层数据集中一些微妙而重要的东西。

And the idea being, if I can make very, very complex predictions here, I must have understood something really subtle and important about the underlying data set.

通过迫使模型在如此庞大的规模上进行预测，你迫使它压缩自身的表达方式，从而发展出简化的符号体系。

And through forcing it to sort of make these predictions at such a large scale, you force it to compress its own representation so that it develops shorthand.

所以，如果你有一百万个数据点，并且正在训练一个神经网络来表示它们，如果你给这个神经网络一百万个参数，它就不会表现出任何这种行为，因为它根本不需要学习捷径来表示这些数据。

So, you know, if I had a million bits of information and I was training a neural network to represent them, if I gave the neural network a million parameters, it would not take on any of this behavior cause it would never need to learn shortcuts to represent it.

它会为每一个数据点分配一个神经元。

It would just have one neuron for each bit of information.

我们在这里做的是，面对我们希望它学习的无数万亿信息，强迫它压缩到可能只有几亿个参数，这意味着每个参数不能只为数据集中的每一个单独事物提供表示。

What we're doing here is we're taking untold trillions of things we want it to learn about and we're forcing it to compress that into maybe a few 100,000,000 parameters, which means that a parameter can't just represent every single individual thing in the dataset per parameter.

你需要找出简化的表达方式，这意味着它必须学会识别事物的特征，以帮助它回答问题。

You need to figure out shorthands, which means it needs to learn features for things to help it kind of answer questions.

因此，在训练过程中，你会使用大规模的数据，要求它做出预测，同时施加一个约束条件。

And so the training processes, there's large scale data that you're then asking it to make predictions about, and you're forcing it to do that with a constraint.

就像创造力一样。

Just like creativity.

你给了它一个人为的限制，迫使它想出创造性的方法，以简化的形式来表示这些数据。

You're giving it an artificial constraint that forces us to come up with creative ways to represent that data to itself in the form of shorthand.

但这种简化最终与思维本身的工作方式紧密相连。

But shorthand ends up being bound up in just how thinking works.

你会发展出各种各样的思维捷径，这里也是一样。

You develop all kinds of shorthand ways of thinking through things, and this is the same.

但这些捷径是在你提出问题的瞬间产生的。

But the shorthand happens in the moment when you ask the question.

庞大的数据集仍然保留。

The large dataset stays.

它并不会压缩成一个更小的数据集，然后就只有这些了。

It's not like it condenses itself to a smaller dataset and that's all that there is going forward.

哦，不，它确实会。

Oh, no, it does.

这就是关键。

That's the thing.

这才是最奇怪的部分。

That's the really weird part.

当你训练这个系统时，比如训练一个神经网络，你会有一个庞大的数据集。

When you train this system, say you're training a neural network, you've got this big dataset.

你会花上数月、数月、再数月的时间来训练它，直到它能对你的大规模实验做出大量预测。

You're going to spend months and months and months training it until it has made an over predictions it can make about this giant experiment you're doing.

这就是你的成果。

That is your thing.

你把它保存下来。

You save that.

那些数据，你就不再使用了。

The data, you're no longer using.

真的吗？

Really?

你现在拥有了这样一个东西，它包含数百万个神经元，彼此之间有着极其复杂的关系，仅此而已。

You now have this thing, which is millions and millions of, like, neurons with some very complex relationship between them, and that's it.

我觉得这就像把所读数据的本质浓缩装瓶了，但它本身并不是数据。

I think of it as like a a bottled up distillation of the kind of data data it's read, but it's not the data.

它是一个曾经思考过这些数据的实体。

It's a thing that has thought about that data.

是的。

Yeah.

现在当你启动它时，你可以向它提问。

And now when you boot it up, you ask questions about it.

这就像用它对事物的简略思考方式，重新向自己表达其中一部分内容。

It's like rerepresenting some of that to itself using its shorthand thinking about things.

而这部分解释了它为何具备创造力，因为它并不是简单地说：‘让我去参考一下这个库。’

And that's partly why it has the capacity for creativity because it's not just saying, oh, let me go and, like, refer back to this this library.

它无法访问那个库。

It doesn't have access to the library.

它必须完全靠自己完成。

It has to do it all itself.

这听起来有些令人担忧。

Something about that seems worrying.

是的。

Yeah.

因为这就像是你只有莎士比亚的简写版，是的。

Because it's like if you have the cliff notes of Shakespeare Yeah.

你想要的莎士比亚就只是这些简写版吗？

Is all that you want from Shakespeare in the cliff

注释吗？

notes?

它与我们自身的思维方式紧密相连。

It is bound up in in how we ourselves think.

比如，我非常喜欢一位叫W的诗人。

Like, I I love a poet called W.

B。

B.

叶芝。

Yeats.

对吧？

Right?

我记得自己背过几首叶芝的诗，但记不全所有的。

And I can remember a small number of Yeats poems that I've memorized, but I can't remember all of them.

我就像W.的简明版。

I am the cliff notes of, like, W.

B.

B.

叶芝。

Yeats.

是的。

Yes.

然而，叶芝的诗让我联想到其他记忆和经历。

And yet, like, Yeats relates to other memories I have and experiences I have.

所以你接触这些内容，并不是为了说：你就是莎士比亚。

So you aren't going to this stuff to say, you are Shakespeare.

如果你想要那样，你就去读原著。

That if you want that, you go and read the source.

你去那里是为了说：哦，我有个问题可能涉及到了。

You're going to it to say, oh, I have a question which might touch.

听听那些读过大量莎士比亚作品、对它有一定直觉的人的看法，可能会很有帮助。

It might be useful to hear from someone who's read a lot of Shakespeare and has some kind of intuitions about it.

这就是为什么整个事情如此令人兴奋，同时也让人产生一些合理的担忧。

It's why this whole thing is so exciting and also why people have some appropriate fear about it.

你认为人们提问时是想要一个答案，还是想要那个正确的答案？

Do you think that people want an answer or do they want the answer when they ask a question?

我认为从实际行为来看，人们想要的是一个答案。

I think the revealed preference is people want an answer.

一个答案。

An answer.

是的。

Yeah.

但他们可能会说：我想要那个正确的答案。

But they will say like, I want the answer.

似乎得到答案是有目的的，因为你不会再思考那个问题了。

It seems like getting an answer serves a purpose because you don't think about the question anymore.

它满足了获得回报的需求。

It satisfies the need to get something back.

你有一个问题，得到了一个答案，无论对错，你就不再去想它了。

You have a question, you get an answer, right or wrong, and you stop thinking about it.

你可以把这些限制在这一点上。

And you can constrain these things to that.

一种思考方式是，假设我是一所大学，拥有成千上万篇我们大学发表的研究论文。

One way to think about it is, say I'm a university and I have thousands of research papers that my university has written.

我可以引入这些人工智能系统，把它们变成有史以来最棒的图书管理员，说：嘿，你现在可以访问这些论文。

I can take in these AI systems and I can turn them into like, the best librarian ever and say, hey, you now have access to these papers.

我会向你提问。

I'm gonna ask you questions.

你要运用你的思维，从这些论文中找出正确答案。

You're gonna use your mind to go and figure out the correct answer from these papers.

所以你可以把这些工具当作一个有品位的聪明人来使用。

So you can use these almost like a a smart person with taste.

去帮助你找到答案。

To go and help give you the answer.

但很多时候，其他用途则更像：我希望你作为一个更通用的存在。

But a lot of the times, the other uses are more like, I want you as a It's more general.

所以我希望你作为一个普通人，进行一般的交流。

So I want you as a general person to talk with generally.

你可以。

You can

可以让这些工具更具体，但它们的精髓在于通用性。

make these things more specific, but the magic in them is for generality.

我明白了。

I see.

要让它们变得具体，是你来提供信息吗？

And to make them specific, you would be the one putting the information in?

经常是的。

Often, yeah.

如果你愿意，我们可以在商业环境中看到更常见的例子：如果我是一家公司，我让这个工具帮我分析我的业务数据，但它其实并没有见过我的业务数据。

If you want, we see this in a more mundane context in business where if I'm a company and I'm asking this thing to help look at my business data, it hasn't seen my business data.

我必须把数据提供给它，然后说：你现在可以访问我的业务数据了。

I have to just give it to us and be like, You now have access to my business data.

当我向你提问时，我希望你能参考这些业务数据，而不是凭空捏造。

When I ask you questions, I need you to look at this business data and not make stuff up.

是的。

Yeah.

他们会说：明白了，老板，我们会这么做的。

They'll be like, Got it boss, we'll do that.

对。

Yeah.

现在，假设我把我的业务数据输入进去，这些数据是否只在回答我的问题时存在，还是说它会成为机器整体智慧的一部分？

Now, when you put, let's say I were to put my business data in, does that business data only live in response to my question or is that now part of the general wisdom of the machine?

仅在响应你的问题时，因为这个系统已经训练好了。

Only in response to your question, because the thing has been trained.

它就像一个生物。

It's like a creature.

你不能重新

You can't re

继续训练。

up- It's not continuing to train.

不，它没有。

No, it is not.

而且这更像是你只是暂时给了它访问你数据的权限。

And it's more like you've just given it access to your thing temporarily.

我

I

明白了。

see.

你知道吗，你可以在香格里拉这里这么做，对吧？

You could do this you know, here at Shangri La, right?

你可以给一个AI系统访问你所有的音乐和母带的权限，但你永远不希望它进入那个系统。

You could give an AI system access to all of the music and, like, masters you have, but you never want it to go into the thing.

但你可能会说，嘿，帮我找找，在那次录音中，我特别喜欢的那个声音在哪。

But you might say, hey, like, help me find there was a particular sound I really liked from like that session.

我觉得我这儿有个听起来像那样的东西。

I think I've got something that sounds like it here.

在我的数据集中，哪个东西最像那个？

What's the thing that sounds most like that in my corpus?

它会运用自己的智能去找到那个东西并把它带给你。

And it'll go and use its intelligence to find the thing and bring it to you.

酷。

Cool.

那么目前，有五到六家大型语言模型公司吗？

So at the moment, are there five or six big large language model companies?

可能是四到五家。

Maybe four or five.

四到五家。

Four or five.

它们是哪些？

What are they?

会有Anthropic，也就是我工作的公司，OpenAI，Google的XAI，这是埃隆的公司，还有Meta，也就是Facebook，实际上还有DeepSeek。

There would be Anthropic, which is where I work, OpenAI, Google XAI, which is Elon's company, and Meta, aka Facebook, and then actually DeepSeek.

所以这更像是五到六家。

So that's more like five or six.

六家。

Six.

好的。

Okay.

那其他所有的AI产品都是这些公司的接入点吗？

And then all of the other AI products are access points for those?

它们中的大多数都建立在这些系统之上。

The majority of them are sitting on top of these things.

这些系统就是这些大型大脑，其他人工智能产品则会依赖于它们。

These things are these big brains and the other AI products will be cooling down to them.

你觉得

Do you

将来会超过这六个吗？

think there will be more than the six over time?

我的预期是，你所处的领域中，数量肯定少于十个。

My expectation is you're going to be in a domain where there's definitely less than 10.

再往后，我就很难推理了。

And after that, it gets hard for me to reason about.

这是因为，也许在2019年，处于前沿领域需要大约10万美元的成本，也就是训练一个系统所需的投入。

And this is because maybe in 2019 being at the frontier would cost you about a $100,000 as in that's your input cost to train a system.

到了2020年和2022年，成本上升到了数百万美元，大约在1000万到1500万美元之间；2024年，这些系统的成本达到数千万到低数亿；而到2025年，这些系统将耗资数亿美金。

Then it went to kind of millions of dollars in twenty twenty, twenty twenty two order of 10 to 15,000,000, 2024 the systems cost multiple tens of millions to low hundreds, 2025, these systems will cost hundreds of millions.

为什么呢？

And why is that?

我们使用了更多的计算机。

We're using way more computers.

所以这是硬件问题。

Based So it's hardware.

这是硬件。

It's hardware.

仅仅是硬件。

Only hardware.

这才是主要的成本。

That is the vast cost of it.

是的。

Yeah.

这其中一部分原因不仅是我们知道如何训练这些系统，而且我们在让它们更大、并在训练过程中延长它们的思考时间方面也变得更加熟练。

And some of this is because it's not just that we know how to train these things, but we we've got better at both making them bigger and also helping them think for longer during training.

我们现在允许它们在训练过程中玩游戏。

And we are now allowing them to like play games during training.

你之前提到了AlphaGo。

So you mentioned AlphaGo earlier.

AlphaGo是一个专门被训练来下围棋的单一系统。

AlphaGo was a single system that was trained to play Go.

它只做这一件事。

That's all it ever did.

而且它带来了惊人的成果。

And it yielded amazing stuff.

现在，当你训练这些AI系统时，你可能会在训练过程中给它们一个环境，比如一个围棋棋盘，让它们同时下围棋。

Well, now while you're training these AI systems, you might have them during training, have an environment, but as a Go board and it's playing Go as well.

然后也许它们还在下国际象棋，也许还在进行数学证明，而数学证明是

And then maybe it's also playing chess and maybe it's also doing a mathematical proofs and a math proof is

同时能进行的事情数量是有限的吗？

a limit to the amount of things that can happen at the same time?

不是真的。

Not really.

实际上，限制在于我们人类为这些系统创建训练环境的能力，以及如何为它们设计更复杂的游戏来进行训练。

It's like the limit actually is on the ability of us as humans to create environments for this stuff to train in and figuring out how to give it more complicated games to play during training.

拥有多个大型语言模型的好处是什么？

What's the benefit of there being more than one large language model?

我认为其中一部分关乎个性和品味。

I think some of it is about personality and taste.

我认为，如果你只有一个，最终会形成一种同质化的‘神 mind’，这似乎很糟糕，因为你本能地会觉得，这样会受到限制。

I think if you just had one, you end up with some, like the homogenous godmind seems bad, like just innately you think, well, that seems like you're gonna be limited.

这些系统更像是被培养出来的，而不是被制造出来的。

These things are more grown than made.

因此，任何开发者所做的各种事情都会累积成一种特质，从而改变这些模型所谓的‘个性’。

And so all of the different things that any developer does add up to a texture that will change the like quote unquote personality of these models.

它们都会拥有不同的价值观、倾向性和能力。

They'll all have different values, proclivities and capabilities as well.