马克斯·泰格马克：《生命3.0》

的

to the

n次方对于所有整数等等，当n大于二时？

n for all integers, etcetera, etcetera, when n is bigger than two?

这是个关于智能的问题。

That's a question about intelligence.

你能造出具备那种智能的机器吗？

Can you build machines that are that intelligent?

我认为等到我们能造出独立完成那种级别证明的机器时，可能已非常接近通用人工智能了。

And I think by the time we get a machine that can independently come up with that level of proofs, probably quite close to AGI.

第二个问题是关于意识的问题。

The second question is a question about consciousness.

这样的机器有多大可能性会真正拥有体验，而不是像僵尸一样？

How likely is it that such a machine would actually have any experience at all as opposed to just being like a zombie?

我们是否期待它对此产生某种情感反应，或任何类似人类情感的东西——当它达成机器目标时，会将其视为某种极其积极、崇高且意义深远的事物。

And would we expect it to have some sort of emotional response to this or anything at all akin to human emotion where when it accomplishes its machine goal, it views it as somehow something very positive and sublime and deeply meaningful.

我当然希望，如果未来我们真的创造出机器同伴甚至机器后代时，它们能拥有这种对生命的崇高感悟。

I would certainly hope that if in the future we do create machines that our peers or even our descendants, I would certainly hope that they do have this sort of sublime appreciation of life.

而我最深层的噩梦是：在遥远的未来某天，我们的宇宙中遍布后生物生命体，做着各种看似酷炫的事情——

A way my absolutely worst nightmare would be that at some point in the future, the distant future, maybe our cosmos is teeming with all this post biological life doing all this seemingly cool stuff.

当人类种族最终消亡时，最后的人类可能会说‘没关系，我们为这些后代感到骄傲’。但最可怕的是我们始终没解决意识问题，

And maybe the last humans, by the time our species eventually fizzes out, will be like, well, that's okay because we are so proud of our descendants here and look at all the My worst nightmare is that we haven't solved the consciousness problem.

甚至没意识到这些全都是行尸走肉。

And we haven't realized that these are all the zombies.

它们比录音机强不了多少，没有任何真实的体验，

They're not aware of anything any more than the tape recorders, it hasn't any kind of experience.

整个文明最终变成一场空看台的表演。

So the whole thing has just become a play for empty benches.

这对我而言就是终极的僵尸末日。

That would be like the ultimate zombie apocalypse to me.

所以我宁愿这些存在能真正理解生命奇迹的珍贵。

So I would much rather in that case that we have these beings, we just really appreciate how amazing it is.

在那个图景中，创造力的角色会是什么？

In that picture, what would be the role of creativity?

有几个人问到了关于创造力的问题。

I had a few people ask about creativity.

你认为，当你思考智能时——我是说，你书开头讲的故事涉及制作电影等等赚钱的方式，在现代社会通过音乐和电影确实能赚很多钱，所以如果一个智能系统存在，它可能会想擅长这方面。

Do you think, when you think about intelligence, I mean, the, the story you told at the beginning of your book involved, you know, creating movies and so on, sort of making money, you know, you can make a lot of money in our modern world with music and movies, so if you are an intelligent system, you may want to get good at that.

是的。

Yeah.

但这并不完全是我所指的创造力。

But that's not necessarily what what I mean by creativity.

在那些复杂目标中——比如海平面上升的背景下，创造力是否重要？还是说我过于人类中心主义，认为创造力相对于智能有某种特殊性？

Is it important on that complex goals where the sea is rising for there to be something create creative, or am I being very human centric and thinking creativity is somehow special relative to intelligence?

我的直觉是，我们应该把创造力简单地视为智能的一个方面。

My hunch is that we should think of creativity simply as an aspect of intelligence.

我们必须对人类的虚荣心保持高度警惕。

And we have to be very careful with human vanity.

我们常常有一种倾向，一旦机器能做某件事，就试图贬低它，说‘哦，但那不是真正的智能，因为它们没有创造力或这样那样的特质’。

We have this tendency to very often want to say, as soon as machines can do something, try to diminish it and say, oh, but that's not like real intelligence, know, because they're not creative or this or that, the other thing.

如果我们要求自己写下对‘创造力’的实际定义，比如安德鲁·威尔所展现的那种创造力，我们是否常指某人做出了出人意料的飞跃？

If we ask ourselves to write down a definition of what we actually mean by being creative, what we mean by Andrew Weil's, what he did there, for example, don't we often mean that someone takes a very unexpected leap?

这不像用573乘以224那样遵循直白的固定规则步骤，而是可能在人们从未想过有联系的两件事物之间建立关联。

It's not like taking five seventy three and multiplying it by two twenty four by just a step of straightforward cookbook like rules, You can maybe make it you make a connection between two things that people had never thought was connected.

非常出人意料。

Very surprising.

或类似这样的特质。

Or something like that.

我认为这是智能的一个方面，实际上是最重要的方面之一。

I think this is an aspect of intelligence and this is actually one of the most important aspects of it.

或许我们人类在这方面比传统计算机更擅长的原因在于，作为神经网络，这种能力比基于传统逻辑门的计算机更自然地产生。

Maybe the reason we humans tend to be better at it than traditional computers is because it's something that comes more naturally if you're a neural network than if you're a traditional logic gate based computer machine.

我们生理上拥有所有这些连接。

We physically have all these connections.

如果你激活这里，激活这里，再激活这里，砰的一声。

And if you activate here, activate here, activate here, ping.

我的直觉是，如果我们真能造出这样的机器，你只需对它说：'嘿，我刚决定这个月要去环游世界，你能替我教我的AGI课程吗？'

My hunch is that if we ever build a machine where you could just give it the task, hey, you say, hey, I just realized I want to travel around the world instead this month, can you teach my AGI course for me?

它就会回答：'好的，我来教。'

And it's like, okay, I'll do it.

然后它会完成你本应做的一切，还能即兴发挥等等。

And it does everything that you would have done and improvises and stuff.

在我看来这需要极大的创造力。

That would in my mind involve a lot of creativity.

是啊。

Yeah.

这么说其实很精妙。

So it's actually a beautiful way to put it.

我认为我们确实试图抓住——你知道的——智能的定义就是我们还不懂如何构建的一切。

I think we do try to grab grasp at the, you know, the the definition of intelligence is everything we don't understand how to build.

所以，就像，我们人类试图找到我们拥有而机器没有的东西，也许创造力只是其中之一，是我们用来描述这一点的词汇之一，这种说法真的很有趣。

So, like, so we as humans try to find things that we have and machines don't have, and maybe creativity is just one of the things, one of the words we use to describe that, that's a really interesting way to put it.

我认为我们不需要如此防御性，我不认为声称'我们很特别'能带来什么好处。

I don't think we need to be that defensive, I don't think anything good comes out of saying, oh, we're somehow special.

相反，历史上有许多例子表明，试图假装自己比其他智能生命更优越往往会导致非常糟糕的结果，对吧？

Contrary wise, there are many examples in history of where trying to pretend they were somehow superior to all other intelligent beings has led to pretty bad results, right?

纳粹德国就声称他们比其他民族更优越。

Nazi Germany, they said that they were somehow superior to other people.

如今，我们仍然通过对动物施加大量残忍行为来证明我们某种程度上的优越性，声称它们感受不到痛苦。

Today, we still do a lot of cruelty to animals by saying that we're so superior somehow and they can't feel pain.

奴隶制也是通过类似非常站不住脚的论点来正当化的。

Slavery was justified by the same kind of just really weak arguments.

我认为如果我们真的继续发展人工通用智能，它能比我们做得更好。

I don't think if we actually go ahead and build artificial general intelligence, it can do things better than us.

我不认为我们应该把自我价值建立在某种关于智力优越性的虚假主张上。

I don't think we should try to found our self worth on some sort of bogus claims of superiority in terms of our intelligence.

我认为我们应该从自身经历中寻找人生使命和生命的意义。

I think we should instead find our calling and the meaning of life from the experiences that we have.

即使有比我更聪明的人存在，我依然能拥有非常有意义的体验。

I can have very meaningful experiences even if there are other people who are smarter than me.

当我参加系里会议讨论时，突然意识到：天啊，这位得过奖，那位也得过奖，还有那位也得过奖。

When I go to a faculty meeting here and we're talking about something and then I suddenly realize, oh, boy, he has an old prize, he has an old prize, he has an old prize.

这难道会让我减少对生活的享受或与这些人交谈的乐趣吗？

Does that make me enjoy life any less or enjoy talking to those people less?

当然不会。

Of course not.

相反，我觉得能与许多方面都比我优秀的智慧生命交流，是莫大的荣幸和特权。

And contrariwise, I feel very honored and privileged to get to interact with other very intelligent beings that are better than me at a lot of stuff.

所以我认为我们没有理由不能以同样的态度对待智能机器。

So I don't think there's any reason why we can't have the same approach with intelligent machines.

这个观点很有意思，人们通常不会这么想——当谈到更智能的机器时，人们本能地认为那不会是一种有益的智能。

That's a really interesting, so people don't often think about that, they think about when there's going, if there's machines that are more intelligent, you naturally think that that's not going to be, a beneficial type of intelligence.

你没意识到这可能是，就像和诺贝尔奖得主做朋友一样，和他们交谈会很有趣，他们在某些话题上很聪明，你还可以和他们小酌几杯。

You don't realize it could be, you know, like peers with Nobel Prizes that that would be just fun to talk with, and they might be clever about certain topics and, you can have fun having a few drinks with them.

所以

So

嗯，还有，我们都能理解为什么身边有比我们更聪明的人不一定是件坏事，比如你我两岁时。

Well, also, you know, another example we can all relate to it of why it doesn't have to be a terrible thing to be in present in presence of people who are even smarter than us all around is when when you and I were both two years old.

我是说，我们的父母那时比我们聪明得多。

I mean, our parents were much more intelligent than us.

对吧？

Right?

是啊。

Yeah.

结果也挺好的。

Worked out okay.

没错。

Yep.

因为他们的目标与我们的目标一致。

Because their goals were aligned with our goals.

是啊。

Yeah.

我认为这才是我们必须解决的首要关键问题。

And that I think is really the number one key issue we have to solve.

是价值观对齐问题。

It's value alignment.

价值观对齐问题，正是如此。

The value alignment problem, exactly.

因为看了太多好莱坞烂科幻片的人，他们总在担心错误的事情，对吧？

Because people who see too many Hollywood movies with lousy science fiction plot lines, they worry about the wrong thing, right?

他们担心某些机器会突然变坏。

They worry about some machines suddenly turning evil.

我们该担心的不是恶意，而是能力问题。

It's not malice that we should, that's the concern, it's competence.

根据定义，智能使你非常能干。

By definition, intelligence makes you very competent.

如果你有一个更智能的围棋程序，那么电脑就是相对不够智能的一方。

If you have a more intelligent Go playing, computer playing is the less intelligent one.

当我们把智能定义为赢得围棋比赛的能力时，更智能的一方自然会胜出。如果你有一个人类和一个在各方面都更强大的通用人工智能，而他们目标不同，你猜谁会实现自己的目标？

When we define intelligence as the ability to accomplish Go winning, It's gonna be the more intelligent one that And if you have a human and then you have an AGI that's more intelligent in all ways, and they have different goals, guess who's gonna get their way, right?

我刚读到关于这种几年前灭绝的犀牛物种，看着母犀牛和它幼崽的可爱照片，真是令人难过。

So I was just reading about this particular rhinoceros species that was driven extinct just a few years ago, kind of a bummer, was looking at this cute picture of mommy rhinoceros with its child.

为什么我们人类会导致它们灭绝？

Why did we humans drive it to extinction?

并不是因为我们整体上是邪恶的犀牛仇恨者。

It wasn't because we were evil rhino haters as a whole.

只是因为我们与犀牛的目标不一致，而这对犀牛来说结果很糟糕，因为我们更聪明。

It was just because our goals weren't aligned with those of the rhinoceros and it didn't work out so well for the rhinoceros because we were more intelligent.

所以我认为最关键的是，如果我们真的创造了通用人工智能，在释放任何东西之前，我们必须确保它能理解并采纳我们的目标，且始终坚守这些目标。

So I think it's just so important that if we ever do build AGI, before we unleash anything, we have to make sure that learns to understand our goals, adopts our goals and it retains those goals.

所以这里一个既酷又有趣的问题在于，我们人类要如何尝试去明确我们的价值观。

So the cool interesting problem there is being able, us as human beings trying to formulate our values.

你可以把美国宪法看作一个例子，当时一群人坐下来——虽然是一群白人男性，但这是个不错的例子——他们为这个国家制定了目标，许多人都认为这些目标至今仍相当有效。

So, you know, you could think of the United States constitution as a, as a way that people sat down, at the time a bunch of white men, but which is a good example, should we should say, they formulated the goals for this country and a lot of people agree that those goals actually held up pretty well.

这是一种有趣的价值观表述方式，但在其他方面却惨败。

That's an interesting formulation of values and failed miserably in other ways.

因此，对于价值对齐问题及其解决方案，我们必须能够将人类价值观落实到纸上或程序中。

So for the value alignment problem and the solution to it, we have to be able to put on paper, or in program human values.

你认为这有多困难？

How difficult do you think that is?

非常困难。

Very.

但这至关重要。

But it's so important.

我们必须全力以赴。

We really have to give it our best.

这困难的原因有两个方面。

And it's difficult for two separate reasons.

首先是技术层面的价值对齐问题，即如何让机器理解、采纳并坚守其目标。

There's the technical value alignment problem of figuring out just how to make machines understand their goals, adopt them and retain them.

其次是与之分离的哲学部分——究竟该采用谁的价值观？

And then there's the separate part of it, the philosophical part, whose values anyway?

鉴于地球上我们对价值观尚未达成广泛共识，我们该建立何种机制来汇总并决定一个合理的折中方案？

And since it's not like we have any great consensus on this planet on values, what mechanism should we create then to aggregate and decide, okay, what's a good compromise?

这第二个议题不能只留给我这样的技术宅来决定。

That second discussion can't just be left to tech nerds like myself.

确实如此。

That's right.

如果我们拒绝讨论这个问题，等到通用人工智能被创造出来时，实际上将由谁来决定采用谁的价值观？

And if we refuse to talk about it and then AGI gets built, who's going to be actually making the decision about whose values?

最终只会是某家科技公司里的一群男人来做决定。

It's going to be a bunch of dudes in some tech company.

他们真的能代表全人类吗？我们是否应该就这样将决定权托付给他们？

And are they necessarily so representative of all of humankind that we want to just entrust it to them?

仅仅因为擅长编程人工智能，他们就具备定义未来人类幸福的特权吗？

Are they even uniquely qualified to speak to future human happiness just because they're good at programming AI?

我更希望这是一场真正包容各方的对话。

I'd much rather have this be a really inclusive conversation.

但你认为有可能构建一个美好的愿景吗？这个愿景要能容纳文化多样性，以及关于权利、自由、人类尊严的各种观点，但要达成这样的共识有多困难？

But do you think it's possible sort of, so you create a beautiful vision that includes, sort of the diversity, cultural diversity, and various perspectives on discussing rights, freedoms, human dignity, but how hard is it to come to that consensus?

这无疑是我们都应该努力去做的重要事情，但你认为它可行吗？

Do you think, it's certainly a really important thing that we should all try to do, but do you think it's feasible?

我认为拒绝讨论或拒绝尝试是确保失败的最佳方式。

I think there's no better way to guarantee failure to refuse to talk about it or refuse to try.

我也认为这种策略非常糟糕：我们先进行长期讨论，直到达成完全共识后，再尝试将其植入机器。

And I also think it's a really bad strategy to say, let's first have a discussion for a long time and then once we reach complete consensus, then we'll try to load it into some machine.

不，我们不应该让完美成为优秀的敌人。

No, we shouldn't let perfect be the enemy of good.

相反，我们应该从几乎所有人都认同的基本道德准则开始，现在就将其植入机器。

Instead, we should start with the kindergarten ethics that pretty much everybody agrees on and put that into machines now.

我们甚至还没做到这一点。

We're not doing that even.

任何制造客机的人都希望它在任何情况下都不会撞向建筑物或山体。

Anyone who builds a passenger aircraft wants it to never under any circumstances fly into a building or a mountain.

然而9·11事件的劫机者却做到了。

Yet the September eleven hijackers were able to do that.

更令人难堪的是，德国之翼航空那位抑郁的飞行员安德烈亚斯·卢比茨，当他驾驶客机撞向阿尔卑斯山导致百余人遇难时，他只是向自动驾驶系统下达了指令。

And even more embarrassingly, Andreas Lubitz, this depressed German wings pilot, when he flew his passenger jet into the Alps, killing over a 100 people, he just told the autopilot to do it.

他居然直接让电脑将高度改为100米。

He told the freaking computer to change altitude to a 100 meters.

尽管系统拥有GPS地图等所有数据，电脑还是照做了。

And even though it had the GPS maps, everything, the computer was like, okay.

所以我们应该从这些不存在分歧的基本价值观入手。

So we should take those very basic values where the problem is not that we don't agree.

问题只是我们一直太懒，没有尝试将这些基本准则植入我们的机器，并确保从现在起，所有装有电脑的飞机都不会执行此类指令。

The problem is just we've been too lazy to try to put it into our machines and make sure that from now on airplanes will just, which all have computers in them, but we'll just refuse to do something like that.

进入安全模式，或许锁定驾驶舱门，飞往最近的机场。

Go into safe mode, maybe lock the cockpit door, go to the nearest airport.

而且，我们现在的世界还有如此多的其他技术领域，确实已经到了该植入这类基本价值观的恰当时机。

And and there's so much other technology in in our world as well now where it's really quite becoming quite timely to put in some sort of very basic values like this.

甚至在汽车中也是如此。

Even in cars.

迄今为止我们已经遭遇了足够多的车辆恐怖袭击事件，有人驾驶卡车和货车冲撞行人，因此将这种防护机制内置到汽车中绝非疯狂的想法。

We were we've had enough vehicle terrorism attacks by now where people have driven trucks and vans into pedestrians, that it's not at all a crazy idea to just have that hardwired into the car.

因为确实，总会有出于某种原因想要伤害他人的人，但其中大多数人并不具备技术能力来绕过这种防护机制。

Because yeah, there are a lot of there's always gonna be people who for some reason wanna harm others, but most of those people don't have the technical expertise to figure out how to work around something like that.

所以如果汽车直接拒绝执行危险操作，就能起到防范作用。

So if the car just won't do it, it helps.

那么，我们就从这里开始着手吧。

So let's let's start there.

这是个非常好的观点。

So there's a lot of that's a that's a great point.

所以不必追求完美。

So not not chasing perfect.

世界上有很多事情是大多数人达成共识的。

There's a lot of things that most of the world agrees on.

是的，让我们从这里开始。

Yeah, and let's start there.

就从这里开始吧。

Let's start there.

一旦我们迈出第一步，就会逐渐习惯这类讨论：我们还需要加入哪些内容，并持续进行这些对话。

And then once we start there, we'll also get into the habit of having these kind of conversations about, okay, what else should we put in here and have these discussions.

这应该是一个渐进的过程。

This should be a gradual process then.

很好，但这同时也意味着需要把这些原则描述清楚，并转化为机器能理解的指令。

Great, so, but that also means describing these things and describing it to a machine.

有件事，我们和史蒂芬·沃尔夫勒姆有过几次交流，不知道你是否熟悉他。

So one thing, we had a few conversations with Stephen Wolfram, I'm not sure if you're familiar with Stephen Wolfram.

哦，是的，我对他相当了解。

Oh yeah, I know him quite well.

他涉猎广泛，研究细胞自动机这类简单的可计算系统和计算体系。

So he is, you know, he plays, you know, works with a bunch of things, but you know, cellular automata, these simple computable things, these computation systems.

他提到过，我们可能在这些系统中已经拥有了某种通用人工智能，只是因为我们无法与之交流而不知道它的存在。

And he kind of mentioned that, you know, we probably have already within these systems already something that's AGI, meaning, like, we just don't know it because we can't talk to it.

所以，如果给我这个机会尝试从中提炼出一个问题的话——我认为这个观点很有趣：我们可能拥有智能系统，却不知如何向它们描述事物，它们也无法与我们沟通。

So, if if you give me this chance to try to try to at least form a question out of this is I think it's an interesting idea to think that we can have intelligent systems, but we don't know how to describe something to them and they can't communicate with us.

我知道你在可解释AI领域做了一些工作，试图让AI能够自我解释。

I know you're doing a little bit of work in explainable AI, trying to get AI to explain itself.

那么你对自然语言处理或其他沟通方式有什么看法？

So what are your thoughts of natural language processing or some kind of other communication?

AI要如何向我们解释事物呢？

How does the AI explain something to us?

我们该如何向机器解释事物？

How do we explain something to it, to machines?

或者你有不同的看法？

Or you think of it differently?

你的问题其实包含两个独立的部分。

So there are two separate parts to your question there.

其中一部分涉及沟通，这非常有趣，我稍后会谈到。

Of them has to do with communication, which is super interesting and I'll get to that in a sec.

另一个问题是，我们是否已经拥有AGI，只是还没注意到它的存在。

The other is whether we already have AGI, we just haven't noticed it there.

在这一点上我持不同意见。

There I beg to differ.

我不认为任何细胞自动机、互联网或其他系统中存在真正的人工通用智能。

Don't think there's anything in any cellular automaton or anything or the internet itself or whatever that has artificial general intelligence in it.

它真的能比我们人类更擅长做所有事情吗？

It's going to really do everything we humans can do better.

我认为当那一天到来时，我们会很快注意到，甚至可能在此之前就以非常显著的方式察觉到。

I think the day that happens, when that happens, we will very soon notice, we'll probably notice even before in a very, very big way.

不过关于第二部分——

But for the second part though-

等等，我能问一下，我可以回答吗？

Wait, can ask, can I answer?

抱歉。

Sorry.

因为你有这种将意识表述为信息处理的精妙方式，你可以将智能视为信息处理，也可以将整个宇宙看作这些具有信息处理能力的粒子和系统在四处游动。

So, because you have this beautiful way to formulating consciousness as information processing, you can think of intelligence as information processing, and you can think of the entire universe as these particles and these systems roaming around that have this information processing power.

你不认为存在某种能以我们人类方式处理信息的能力存在于外界，需要与之建立某种联系吗？

You don't, you don't think there is something with the power to process information in the way that we human beings do that's out there, that, that needs to be sort of connected to?

这可能听起来有点哲学意味，但这个观点有其引人之处：能力已经存在，重点应更多放在能否与之沟通上。

It seems a little bit philosophical perhaps, but there's something compelling to the idea that the power is already there, the focus should be more on being able to communicate with it.

嗯，我同意在某种意义上硬件处理能力已经存在，因为我们的宇宙本身就可以被视为一台计算机，对吧？

Well, I agree that in a certain sense the hardware processing power is already out there, because our universe itself can think of it as being a computer already, right?

它不断计算水波的动态，查尔斯河如何演变，以及空气分子如何运动。

It's constantly computing what water waves, how it evolved the water waves and the River Charles and how to move the air molecules around.

我的同事塞思·劳埃德曾指出，你甚至可以用非常严谨的方式将整个宇宙视为一台量子计算机。

Seth Lloyd has pointed out, my colleague here, that you can even in a very rigorous way think of our entire universe as being a quantum computer.

很明显我们的宇宙支持这种惊人的处理能力，因为即使在我们生活的这个物理计算机中，我们也能建造实际的笔记本电脑等设备。

It's pretty clear that our universe supports this amazing processing power because you can even within this physics computer that we live in, right, we can even build actual laptops and stuff.

所以显然这种能力是存在的。

So clearly the power is there.

在我看来，大自然拥有的大部分计算能力都浪费在无聊的事情上，比如模拟某个无人关注的海洋波浪。

It's just that most of the compute power that nature has, it's in my opinion, of wasting on boring stuff like simulating yet another ocean wave somewhere where no one is even looking.

对吧？

Right?

所以在某种意义上，生命所做的、我们建造计算机时所做的，就是将大自然原本就在进行的计算重新引导，去做比又一道海浪更有趣的事情，让我们在这里做些酷炫的事。

So in a sense what life does, what we are doing when we build computers is we're re channeling all this compute that nature is doing anyway into doing things that are more interesting than just yet another ocean wave, you know, and let's do something cool here.

所以原始硬件能力确实存在，即使只是计算这个水瓶接下来五秒会发生什么，如果用人类计算机来做，也需要惊人的计算量。

So the raw hardware power is there for sure, then even just like computing what's gonna happen for the next five seconds in this water bottle, you know, it takes a ridiculous amount of compute if you do it on a human computer.

这个水瓶刚刚做到了。

This water bottle just did it.

但这并不意味着这个水瓶具有人工通用智能(AGI)，因为AGI意味着它应该还能完成像写我的书、做这个采访这样的事情。

But that does not mean that this water bottle has AGI, because AGI means it should also be able to, like have written my book, done this interview.

我不认为这只是沟通问题，我认为它做不到这些。

And I don't think it's just communication problems, don't think it can do it.

虽然佛教徒说当他们观察水时能感受到某种美，认为自然界中存在他们可以与之交流的深度与美。

Although Buddhists say when they watch the water and that there is some beauty, that there's some depth and beauty in nature that they can communicate with.

不过沟通确实非常重要，因为你看，我工作的一部分就是当老师。

Communication is also very important though, because I mean, look, part of my job is being a teacher.

我认识一些非常聪明的教授，他们甚至也有点难以沟通。

And I know some very intelligent professors even who just have a bit of hard time communicating.

他们提出

They come

了所有这些绝妙的想法，但要与他人沟通，你还必须能够模拟对方的思维。

up with all these brilliant ideas, but to communicate with somebody else, you have to also be able to simulate their own mind.

是的，同理心。

Yes, empathy.

并且要构建得足够好，理解他们心智的模型，这样你才能说出他们能理解的话。

And build well enough and understand model of their mind that you can say things that they will understand.

这相当困难。

And that's quite difficult.

这就是为什么今天如果你有一台电脑做出某种癌症诊断，而你问它‘为什么你认为我应该做这个手术？’会如此令人沮丧。

And that's why today it's so frustrating if you have a computer that makes some cancer diagnosis and you ask it, well, why are you saying I should have this surgery?

如果你不想回答‘我是在五万亿字节的数据上训练的，这是我的诊断，哔哔，嘟嘟’。

And if you don't want to reply, I was trained on five terabytes of data and this is my diagnosis, boop, boop, beep, beep.

这并不能真正让人产生多少信心，对吧？

Doesn't really instill a lot of confidence, right?

所以我认为我们在沟通方面还有很多工作要做。

So I think we have a lot of work to do on communication there.

那么，我想你正在做一些可解释人工智能方面的工作。

So what of, I think you're doing a little bit of work in Explainable AI.

你认为最有前景的途径是什么？

What do you think are the most promising avenues?

主要是像Alexa那样的自然语言处理问题吗？即能够实际使用人类可理解的沟通方式？

Is it mostly about sort of the Alexa problem of natural language processing of being able to actually use human interpretable methods of communication?

所以是能够与系统对话并得到回应，还是存在一些更基础的问题需要解决？

So being able to talk to a system and talk back to you, or is there some more fundamental problems to be solved?

我认为以上所有方面都很重要。

I think it's all of the above.

自然语言处理显然很重要，但也存在更技术性的基础问题，比如你会下国际象棋吗？

The natural language processing is obviously important, but there are also more nerdy fundamental problems, like if take you play chess?

嗯，当然。

Mhmm, course.

我是俄罗斯人。

I'm Russian.

我必须会下。

I have to.

你什么时候学的俄语？

When did you learn Russian?

你会多少种语言？

How many languages do you know?

哇，这真的很厉害。

Wow, that's really impressive.

我不确定，伙计，我妻子有她的算法，但我想说的是，如果你下国际象棋，你看过AlphaZero的对局吗？

I don't know, man, my wife has some calculation, but my point was, if you played chess, have you looked at the AlphaZero games?

那些

The

实际对局，没看过。

actual games, no.

去看看，有些对局简直令人震撼，非常精彩。

Check it out, some of them are just mind blowing, really beautiful.

如果你问它是怎么做到的？

And if you ask how did it do that?

你去和DeepMind的Sabbaths等人交流，他们最终能给你的只是一大堆数字表格，那些定义神经网络的矩阵。

You go talk to them as a Sabbaths and others from DeepMind, all they'll ultimately be able to give you is big tables of numbers, matrices that define the neural network.

你可以盯着这些数字表格看到脸色发青。

And you can stare at these tables numbers till your face turned blue.

但你仍然不会明白它为何做出那个决策。

And you're not gonna understand much about why it made that move.

即便有自然语言处理技术能用人类语言告诉你'七点二八'这样的数据，也依然无济于事。

And even if you have natural language processing that can tell you in human language about, seven, points two, eight, still not gonna really help.

所以我认为这其中涉及一系列有趣的挑战——如何将执行智能行为的计算过程，转化为同样优秀且智能但更易理解的形式。

So I think think there's a whole spectrum of of of fun challenges there involved in in taking a computation that does intelligent things and transforming it into something equally good, equally intelligent, but that's more understandable.

我觉得这非常有价值，因为随着我们让机器掌管世界上越来越多的基础设施——电网、股市交易、武器系统等等。

And I think that's really valuable because I think as we put machines in charge of ever more infrastructure in our world, the power grid, the trading on the stock market, weapon systems and so on.

我们能够信任这些AI完成我们期望的所有任务，这一点至关重要。

It's absolutely crucial that we can trust these AIs to do all we want.

信任从根本上来源于深刻的理解。

Trust really comes from understanding in a very fundamental way.

这就是我正在研究这个的原因。

And that's why I'm working on this.

因为我认为，如果我们希望确保机器采纳了我们的目标并保持这些目标，这种信任必须建立在你能真正理解的基础上，最好是能构建甚至能证明定理的东西。

Because I think the more, if we're gonna have some hope of ensuring that machines have adopted our goals and that they're gonna retain them, that kind of trust, I think needs to be based on things you can actually understand, preferably even make, preferably even prove theorems on.

即使是自动驾驶汽车，对吧？

Even with a self driving car, right?

如果有人只是告诉你它经过海量数据训练且从未出过事故，这远不如有人能提供实际证明来得让人安心。

If someone just tells you it's been trained on tons of data and it never crashed, it's less reassuring than if someone actually has a proof.

或许这是个计算机验证的证明，但它至少能说明在任何情况下这辆车都不会突然驶入对向车道。

Maybe it's a computer verified proof, but still it says that under no circumstances, is this car just going to swerve into oncoming traffic?

这类信息有助于建立信任，促进目标对齐，至少能让人意识到你们的价值观是一致的。

And that kind of information helps build trust and helps build the alignment of goals, at least awareness that your goals, your values are aligned.

我认为即便在短期内，你看看我们现有的网络安全状况有多糟糕——30亿雅虎账户泄露，几乎每个美国人的信用卡信息都被盗等等。

And I think even in the very short term, you look at how you know, right, this absolutely pathetic state of cybersecurity that we have, where is it, 3,000,000,000 Yahoo accounts, can't pack almost every American's credit card and so on.

这种情况正在发生吗？

Is this happening?

这最终会发生，是因为我们使用的软件没人完全理解其运作原理。

It's ultimately happening because we have software that nobody fully understood how it worked.

这就是为什么那些漏洞一直没被发现。

That's why the bugs hadn't been found.

对吧？

Right?

我认为AI既能非常有效地用于攻击和黑客行为，也能用于防御。

And I think AI can be used very effectively for offense, for hacking, but it can also be used for defense.

希望它能自动化可验证性，并构建以不同方式设计的系统，这样你就能真正证明它们的某些特性。

Hopefully automating verifiability and creating systems that are built in different ways so you can actually prove things about them.

这很重要。

And it's important.

说到没人理解其运作原理的软件，当然很多人会问到你那篇论文，关于你对'为什么深度廉价学习效果这么好'的看法——就是那篇论文，但你对深度学习本身有什么看法？

So speaking of software that nobody understands how it works, of course a bunch of people ask about your paper, about your thoughts of why does deep and cheap learning work so well, that's the paper, but what are your thoughts on deep learning?

这类对我们大脑的简化模型已经能够成功完成一些感知工作、模式识别工作，现在通过AlphaZero等还能做些聪明的事情。

These kind of simplified models of our own brains have been able to do some successful perception work, pattern recognition work, and now with alpha zero and so on, do some some clever things.

你如何看待这一领域的潜力与局限？

What are your thoughts about the promise limitations of this piece?

很好。

Great.

我认为，从这类成功案例中我们总能汲取许多非常重要的见解和教训。

I I think there are a number of of of very important insights, very important lessons we can always draw from these kind of successes.

其中之一是，当你观察人脑时，会发现它非常复杂——有100亿到110亿个神经元，而且存在各种不同类型的神经元等等。

One of them is when you look at the human brain, you see it's very complicated, 10 to 11 neurons, and there are all these different kinds of neurons and yada yada.

长期以来一直存在一个争论：我们拥有数十种不同类型的神经元这一事实，是否对智能而言是必需的。

And there's been this long debate about whether the fact that we have dozens of different kinds is actually necessary for intelligence.

现在我认为我们可以相当有说服力地回答：并非如此。

We can now, I think, quite convincingly answer that question of no.

只需要一种类型就足够了。

It's enough to have just one kind.

如果你深入研究AlphaZero的内部结构，会发现它只有一种神经元，而且是一种简单得可笑的数学构造。

If you look under the hood of alpha zero, there's only one kind of neuron, and it's ridiculously simple, simple mathematical thing.

所以这就像物理学中的情况一样，不在于气体中分子本身的详细性质，而在于某种集体行为表现。

So it's not the it's just like in physics, it's not the if you have a gas with waves in it, it's not the detailed nature of the molecules of matter, it's the collective behavior somehow.

同理，重要的是网络的高层结构，而非拥有20种神经元。

Similarly, it's this higher level structure of the network that matters, that you have 20 kinds of neurons.

我认为我们的大脑之所以如此复杂混乱，是因为它不仅仅是为了智能而进化，还要实现自我组装、自我修复以及在进化过程中可达等目标。

I think our brain is such a complicated mess because it wasn't evolved just to be intelligent, it wasn't evolved to also be self assembling and self repairing and evolutionarily attainable and so on.

还有各种补丁之类的。

And patches and so on.

是的，我的直觉是，我们会在完全理解大脑运作原理之前，先掌握如何构建通用人工智能。

Yeah, so I think it's pretty my hunch is that we're going to understand how to build AGI before we fully understand how our brains work.

就像我们在能造出机械鸟之前，早就理解了如何制造飞行器。

Just like we understood how to build flying machines long before we were able to build a mechanical bird.

没错，你举的这个机械鸟与飞机的例子很贴切，飞机飞行得很好却并不真正模仿鸟类的飞行方式。

Yeah, that's right, you've given the example exactly of mechanical birds and airplanes, and airplanes do a pretty good job of flying without really mimicking bird flight.

甚至在一百年后的今天，你看过那个德国机械的TED演讲吗，就是那个会飞的

And even now after a hundred a hundred years later, did you see the TED talk with this German machine, heard the mechanical

你提到它了。

you mention it.

快去看看，简直太神奇了。

Check it out, it's amazing.

但即便如此，我们依然没有造出机械鸟，因为事实证明我们想出的方法更简单，也更符合我们的需求，我认为人工智能领域可能也是如此。

But even after that, we still don't fly a mechanical bird because it turned out that the way we came up with was simpler and it's better for our purposes, and I think it might be the same there.

这是第一个启示。

That's one lesson.

另一个启示——也是我们论文的主题——作为物理学家，我最初发现人工神经网络与我们物理研究中许多概念（比如重正化群方程、哈密顿量等晦涩术语）之间存在极其密切的数学关系时，觉得非常着迷。

Another lesson, which is what our paper was about, first I as a physicist thought it was fascinating how there's a very close mathematical relationship actually between artificial neural networks and a lot of things that we've studied for in physics go by nerdy names like the renormalization group equation and Hamiltonians and yada yada yada.

当我更深入研究时，最初的反应是：哇，这简直疯狂到不合常理。

And when you look a little more closely at this, you have at first I was like, woah, there's something crazy here that doesn't make sense.

我们现在都知道，即便是构建一个能区分猫狗图片的超级简单神经网络，也能做得非常出色。

We know that if you even want to build a super simple neural network with parallel park cat pictures and dog pictures, we can do that very, very well now.

但仔细想想又会觉得这根本不可能——因为假设有一百万像素的图像，即便每个像素只有黑白两色，可能的图像组合也有2的100万次方种，这数量远超宇宙中的原子总数。

But if you think about it a little bit you convince yourself it must be impossible because if I have one megapixel, even if each pixel is just black or white, there's two to the power of 1,000,000 possible images, which is way more than there are atoms in our universe.

对吧？

Right?

因此，对于每一个这样的图像，我都需要分配一个数字，表示它是狗的概率。

So in order to and then for each one of those, I have to assign a number, which is the probability that it's a dog.

所以，一个任意的图像函数所包含的数字量，比我们宇宙中的原子总数还要多。

So an arbitrary function of images is a list of more numbers than there are atoms in our universe.

显然，我无法将这些数据存储在我的GPU或电脑里，但它却莫名其妙地能运行。

So clearly I can't store that under the hood of my GPU or my computer, yet it somehow works.

这意味着什么呢？

So what does that mean?

这意味着，在所有你可能尝试用神经网络解决的问题中，几乎没有一个能用合理规模的网络解决。

Well, it means that out of all of the problems that you could try to solve with a neural network, almost all of them are impossible to solve with a reasonably sized one.

但我们在论文中表明，根据物理定律，我们真正关心的所有可能问题所占的比例也是微乎其微的。

But then what we showed in our paper was that the fraction of all the problems that you could possibly pose that we actually care about given the laws of physics is also an infinitesimally tiny little part.

而令人惊讶的是，它们基本上属于同一部分。

And amazingly, they're basically the same part.

是啊。

Yeah.

这几乎就像我们的世界是为...我是说，它们某种程度上是相契合的。

It's almost like our world was created for I mean, they kind of come together.

对。

Yeah.

但你可以说也许世界是为我们而创造的，但我有个更谦逊的解释：进化赋予我们神经网络正是出于这个原因。

But you could say maybe where the world created that the world was created for us, but I have a more modest interpretation, which is that instead evolution endowed us with neural networks precisely for that reason.

因为这种特定架构——与你笔记本电脑上的不同——极其适合解决自然界不断向我们祖先提出的那类问题，对吧？

Because this particular architecture, as opposed to the one on your laptop is very, very well adapted to solving the kind of problems that nature kept presenting our ancestors with, right?

所以我们最初为何拥有大脑？这就有道理了。

So it makes sense that why do we have a brain in the first place?

是为了能够预测未来等等。

It's to be able to make predictions about the future and so on.

如果我们有个永远解决不了问题的糟糕系统，它就不会进化出来。

So if we had a sucky system which could never solve it, wouldn't have evolved.

所以我认为这是一个非常美妙的事实。

So this is, I think a very beautiful fact.

我们也意识到之前已有关于为何深度网络表现优异的研究，但我们能够展示一个额外有趣的事实：即便是极其简单的问题，比如我给你一千个数字要求相乘，你只需几行代码就能轻松搞定。

We also realize that there's been earlier work on why deeper networks are good, but we were able to show an additional cool fact there, which is that even incredibly simple problems like suppose I give you a thousand numbers and ask you to multiply them together, you can write a few lines of code, boom, done, trivial.

如果你尝试用仅含一个隐藏层的神经网络来完成这个任务，虽然可行，但你需要2的1000次方个神经元来相乘一千个数字——这数量远超宇宙中原子的总数。

If you just try to do that with a neural network that has only one single hidden layer in it, you can do it, but you're going to need two to the power of a thousand neurons to multiply a thousand numbers, which is again more neurons than there are atoms in our universe.

因此，如果采用多层深度网络结构，你仅需4000个神经元就能实现，这完全可行。

So, if you allow yourself make it deep network with many layers, you only need 4,000 neurons, which is perfectly feasible.

这确实很有意思。

That's really interesting.

关于另一种架构类型，你提到了薛定谔方程，那么你对量子计算及其在构建智能系统中的作用有何看法？

So on another architecture type, I mean you mentioned Schrodinger's equation and what what are your thoughts about quantum computing and the role of this kind of computational unit in creating an intelligence system?

在某些不便提及名字的好莱坞电影里，他们实现通用人工智能的方式就是建造量子计算机。

In some Hollywood movies that are not mentioned by name because I don't want to spoil them, the way they get AGI is Building a quantum computer.

因为'量子'这个词听起来很酷，诸如此类。

Because the word quantum sounds cool and so on.

没错。

That's right.

首先，我认为我们不需要量子计算机来构建人工通用智能。

First of all, I think we don't need quantum computers to build AGI.

我怀疑你的大脑在任何可验证的意义上都不是量子计算机。

I suspect your brain is not quantum computer in any found sense.

多年前你甚至没写过相关论文，计算过退相干——所谓的退相干时间，即神经元活动中的量子计算特性被环境随机噪声抹除所需的时间。

You don't even wrote a paper about that many years ago, calculated the decoherence, so called decoherence time, how long it takes until the quantum computerness of what your neurons are doing gets erased by just random noise from the environment.

这个时间大约是10的负21次方秒。

And it's about 10 to the minus twenty one seconds.

所以尽管我脑袋里有台量子计算机听起来很酷，但我的思维速度可没那么快。

So as cool as it would be to have a quantum computer in my head, I don't think that fast.

另一方面，量子计算机确实能实现一些非常酷的应用——我认为当我们拥有更强大的量子计算机时，它们或许能帮助机器学习超越人脑。

On the other hand, there are very cool things you could do with quantum computers, or I think we'll be able to do soon when we get bigger ones, that might actually help machine learning do even better than the brain.

举个例子（虽然这还只是天方夜谭），学习本质上与搜索非常相似。

So for example, one this is just a moonshot, learning is very much the same thing as search.

如果你试图训练一个神经网络使其真正学会某项技能，你需要定义一个损失函数，并调整一系列可调节的参数（用数字表示），通过不断微调这些参数，使网络在该任务上达到最佳表现。

If you try to train a neural network to get really learned, to do something really well, you have some loss function, have a bunch of knobs you can turn, represented by a bunch of numbers, and you're trying to tweak them so that it becomes as good as possible at this thing.

想象一个多维度的地形景观，每个维度对应一个可调整的参数，你正试图找到其中的最低点（最小值）。

So if you think of a landscape with some valley, where each dimension of the landscape corresponds to some number you can change, you're trying to find the minimum.

众所周知，在高维复杂地形中寻找最小值是极其困难的。

And it's well known that if you have a very high dimensional landscape, complicated things, it's super hard to find the minimum.

而量子力学在这方面表现出惊人的优势。

Quantum mechanics is amazingly good at this.

没错。

Right.

如果我想知道水分子可能达到的最低能量状态——这在计算上极其困难——但自然界会轻松解决这个问题：你只需将其冷却到极低温度。

If I want to know what's the lowest energy state this water can possibly have, incredibly hard to compute, nature will happily figure this out for you if you just cool it down, make it very, very cold.

就像放置一个小球时它会滚落到最低点一样，这种机制在能量景观中也以隐喻的方式发生。

If you put a ball somewhere it will roll down to its minimum and this happens metaphorically at the energy landscape too.

量子力学甚至运用了一些当今机器学习系统尚未采用的巧妙技巧。

And quantum mechanics even uses some clever tricks which today's machine learning systems don't.

就像你试图寻找最小值，结果被困在这个局部最小值里。

Like you're trying to find the minimum and you get stuck in the little local minimum here.

在量子力学中，你实际上可以隧穿势垒，从而摆脱困境。

In quantum mechanics, you can actually tunnel through the barrier and get unstuck again.

这真的很有趣。

That's really interesting.

是啊。

Yeah.

所以也许有一天我们会用量子计算机来帮助更好地训练神经网络。

So maybe for example, we'll one day use quantum computers that help train neural networks better.

这真的很有趣。

That's really interesting.

好的。

Okay.

比如说，作为学习过程的一个组成部分。

So as a component of kind of the learning process, for example.

是的。

Yeah.

让我问一下，差不多要结束了，让我回到之前提到的人性和爱的问题上。

Let me ask, sort of wrapping up here a little bit, let me let me return to, the questions of our human nature and and love as I mentioned.

你认为，你提到过一种辅助机器人，也可以考虑个人机器人。

So do you think, you mentioned sort of a helper robot, you could think of also personal robots.

你认为人类坠入爱河并建立联系的方式，在人工智能系统中、在人类级别的人工智能系统中可能实现吗？

Do you think the way we human beings fall in love and get connected to each other is possible to achieve in an AI system, in human level AI intelligence system?

你认为我们能看到那种联系吗？

Do you think we would ever see that kind of connection?

或者，你知道，在所有这些关于解决复杂目标的讨论中

Or, you know, in all this discussion about solving complex goals

是的。

Yeah.

像这种人类社交联系，你认为这是随着海平面上升我们能够达到的山峰和低谷中的目标之一，还是你认为这是最终、至少在短期内相对于其他目标无法实现的？

As this kind of human social connection, do you think that's one of the goals on the peaks and valleys that with the raising sea levels that we'll be able to achieve, or do you think that's something that's ultimately, or at least in the short term relative to the other goals is not achievable?

我认为这一切都是可能的。

I think it's all possible.

我的意思是最近，正如你所知，在AI研究者中对何时能实现通用人工智能有着非常广泛的猜测。

And I mean in recent, there's very wide range of guesses as you know among AI researchers when we're going to get AGI.

有些人比如我们的朋友罗德尼·布鲁克斯说至少还需要几百年时间，但也有许多人认为会快得多，最近的调查显示约半数AI研究者认为我们将在几十年内实现通用人工智能。

Some people like our friend Rodney Brooks said it's going to be hundreds of years at least and then there are many others who think it's going to happen much sooner and recent polls maybe half or so of AI researchers think we are going get AGI within decades.

如果那成为现实，当然，我认为这些事情都是可能的。

So if that happens, of course, then I think these things are all possible.

但关于它是否会发生，我认为我们不应该花太多时间询问我们认为未来会发生什么，仿佛我们只是某种可悲的被动旁观者，等待未来降临到我们身上。

But in terms of whether it will happen, I think we shouldn't spend so much time asking what do we think will happen in the future as if we are just some sort of pathetic passive bystanders waiting for the future to happen to us.

嘿，我们才是创造这个未来的人，所以我们应该积极主动，问问自己希望实现什么样的未来。

Hey, we're the ones creating this future, so we should be proactive about it and ask ourselves what sort of future we would like to have happen.

说得对。

That's right.

努力让它变成那样。

Trying to make it like that.

那么，我会不会更喜欢某种极其无聊的僵尸般的未来，那里只有机械性的事情发生，没有激情，没有情感，甚至可能没有体验。

Well, would I prefer it to some sort of incredibly boring zombie like future where there's all these mechanical things happening and there's no passion, no emotion, no experience maybe even.

不，我当然更希望，如果我们所珍视的人性中最宝贵的东西——主观体验、激情、灵感、爱——都能存在，我们可以创造一个这些事物都存在的未来。

No, I would of course much rather prefer it if all the things that we find that we value the most about humanity are subjective experience, passion, inspiration, love, we can create a future where those things do exist.

我认为最终不是宇宙赋予我们意义，而是我们为宇宙赋予意义。如果我们构建更高级的智能，就要确保它以包含意义的方式被构建。

I think ultimately it's not our universe giving meaning to us, it's us giving meaning to our universe and if we build more advanced intelligence let's make sure we build it in such a way that meaning is part of it.

许多深入研究这个问题并从不同角度思考的人发现，在大多数情况下，如果他们仔细推敲会发生什么，结果往往是对人类不利的。

A lot of people that seriously study this problem and think of it from different angles have trouble in the majority of cases, if they think through that happen, are the ones that are not beneficial to humanity.

那么，你有什么想法？

And so, yeah, so what are your thoughts?

人们应该...我真的不希望人们感到恐惧。

What should people, you know, I really don't like people to be terrified.

有什么方式能让人们以可以解决问题、改善现状的方式来思考？

What's a way for people to think about it in a way in a way we can solve it, in a way can make it better?

是的，我认为恐慌没有任何帮助，它不会增加事情顺利发展的几率，即使你处于真实威胁的情况下，大家都惊慌失措有用吗？

Yeah, no, I don't think panicking is gonna help in any way, it's not gonna increase chances of things going well either, even if you are in a situation where there is a real threat, does it help if everybody just freaks out?

不，当然，当然不会。

No, of course, of course not.

我认为，是的，确实存在事情可能变得非常糟糕的途径。

I think, yeah, there are of course ways in which things can go horribly wrong.

首先，重要的是当我们思考这个问题和风险时，也要记住如果我们做对了，收益会有多么巨大，对吧？

First of all, it's important when we think about this thing, about the problems and risks, also remember how huge the upsides can be if we get it Right?

你所热爱的社会和文明都是智能的产物。

You love about society and civilization is a product of intelligence.

所以如果我们能用机器智能增强我们的智力，不再因为所谓的绝症而失去所爱之人等等，我们当然应该追求这些。

So if we can amplify our intelligence with machine intelligence and not anymore lose our loved ones to what we're told is an incurable disease and things like this, of course, we should aspire to that.

这可以成为一种动力，提醒自己我们解决问题的原因不仅是为了避免阴霾，更是为了成就伟大的事业。

So that can be a motivator, think reminding ourselves that the reason we try to solve problems is not just because we're trying to avoid gloom, but because we're trying to do something great.

但就风险而言，我认为真正重要的问题是：我们今天能做些什么来真正帮助实现好的结果？

But then in terms of the risks, think the really important question is to ask, what can we do today that will actually help make the outcome good?

而忽视风险并不在其中。

And dismissing the risk is not one of them.

我经常觉得很有趣，当我参加关于这些议题的讨论小组时，那些为公司工作的人总是说'不用担心，不用担心，不用担心'。

I find it quite funny often when I'm on discussion panels about these things, how the people who work for companies always like, oh, nothing to worry about, nothing to worry about, nothing to worry about.

而有时只有学术界人士会表达担忧。

And it's only academics sometimes express concerns.

如果你仔细想想，这一点都不奇怪。

That's not surprising at all if you think about it.

正如厄普顿·辛克莱那句名言：当一个人的收入取决于他不相信某件事时，很难让他相信这件事。

Upton Sinclair quipped, right, that it's hard to make a man believe in something when this income depends on not believing in it.

坦率地说，我们知道公司里很多人和其他人一样担忧，但如果你是公司CEO，你不会想公开说这些——尤其当有愚蠢的记者引用你时还会配终结者机器人的图片。

And frankly, we know a lot of these people in companies that they're just as concerned as anyone else, but if you're the CEO of a company, that's not something you wanna go on record saying when you have silly journalists who are gonna put a picture of a Terminator robot when they quote you.

所以这些问题确实存在。

So so the the the issues are real.

我认为问题的本质在于：我们真正面临的选择首先是——我们是否要直接忽视这些风险，然后继续建造比人类做得更好更便宜的机器。

And the way I the way I think about what the issue is is basically, you know, the the the real choice we have is, first of all, are we gonna just dismiss this, the risks and say, well, let's just go ahead and build machines that can do everything we can do better and cheaper.

干脆以最快速度让自己被淘汰算了。

Let's just make ourselves obsolete as fast as possible.

还能出什么差错呢？

What could possibly go wrong?

这是一种态度。

That's one attitude.

我认为相反的态度是，认识到这巨大的潜力，让我们认真思考我们真正向往的未来是什么样的。

The opposite attitude I think is to say, here is this incredible potential, know, let's think about what kind of future we're really, really excited about.

有哪些我们真正渴望实现的共同目标？

What are the shared goals that we can really aspire towards?

然后让我们认真思考如何才能真正实现这些目标。

And then let's think really hard on how about how we can actually get there.

所以从否定开始。

So start with no.

不要先考虑风险。

Don't start thinking about the risks.

先考虑目标。

Start thinking about the goals.

目标。

The goals.

是的。

Yeah.

当你这样做之后，你才能开始思考你想要避免的障碍。

And then when you do that, then you can think about the obstacles you want to avoid.

对吧？

Right?

经常有学生来我办公室寻求职业建议。

I often get students coming in right here into my office for career advice.

我总是问他们这个问题：你未来想成为什么样的人？

I always ask them this very question, where do you want to be in the future?

如果她只能回答：也许我会得癌症。

If all she can say is, maybe I'll have cancer.

也许我会被车撞

Maybe I'll get run over by

关注障碍而非目标。

on obstacles instead of the goal.

她最终只会变成一个疑病症妄想症患者。

She's just gonna end up a hypochondriac paranoid.

而如果她走进来时眼中闪烁着光芒，仿佛在说‘我要到达那里’。

Whereas if she comes in and fire in her eyes and it's like, I want to be there.

然后我们才能讨论障碍，看看如何规避它们。

And then we can talk about the obstacles and see how we can circumvent them.

我认为这是一种更健康得多的态度。

That's I think a much, much healthier attitude.

这确实很好，而且我觉得构想一个让我们毫无保留为之兴奋的未来愿景非常具有挑战性。

That's really well And I feel it's very challenging to come up with a vision for a future, which we are unequivocally excited about.

我现在不是在说‘让我们治愈癌症’这样模糊的概念，而是在讨论我们想要创造什么样的社会？

I'm not just talking now in the vague terms like, yeah, let's cure cancer, I'm talking about what kind of society do we want to create?

在AGI时代，我们想让人性意味着什么？

What do we want it to mean to be human in the age of AGI?

因此，如果我们能展开这场广泛包容的对话，逐步朝着某个未来方向收敛——至少是我们希望引导的方向——那么我们就会有更强动力去建设性地应对各种障碍。

So if we can have this conversation, broad inclusive conversation, and gradually start converging towards some future that with some direction at least that we want to steer towards, then we'll be much more motivated to constructively take on the obstacles.

如果要更简洁地总结，我想我们现在都能认同：我们应当致力于打造不会压制人类、而是赋能人类的通用人工智能。

And I think if I had to, if I try to wrap this up in a more succinct way, think we can all agree already now that we should aspire to build AGI that doesn't overpower us but that empowers us.

想想它们能实现这一目标的多种方式，比如从我所处的自动驾驶汽车领域来看。

And think of the many various ways they can do that, whether that's from my side of the world of autonomous vehicles.

我个人其实属于这一派：认为要实现真正让人乐于使用并融入生活的车辆，就需要达到人类水平的智能。这只是其中一个例子，医疗等领域当然还有许多其他类型的机器人。

I'm personally actually from the camp that believes this human level intelligence is required to achieve something like vehicles that would actually be something we would enjoy using and being part of, So that's one example, and certainly there's a lot of other types of robots in medicine and so on.

所以专注于这些目标，然后找出障碍，找出可能出错的地方，并逐一解决它们。

So focusing on those and then coming up with the obstacles, coming up with the ways that that can go wrong and solving those one at a time.

仅仅因为你能造出一辆自动驾驶汽车，即使它能完美地自行驾驶，也许生活中有些事情我们实际上还是想亲力亲为。

And just because you can build an autonomous vehicle, even if you could build one that would drive this fine without you, maybe there are some things in life that we would actually want to do ourselves.

没错。

That's right.

举个例子，如果把我们的社会看作一个整体，有些事情对我们来说非常有意义。

Like for example, if you think of our society as a whole, there's some things that we find very meaningful to do.

这并不意味着仅仅因为机器能做得更好，我们就必须停止做这些事。

And that doesn't mean we have to stop doing them just because machines can do them better.

我可不会因为哪天有人造出了能打败我的网球机器人就放弃打网球。

I'm not gonna stop playing tennis just the day someone builds a tennis robot and beat me.

人们现在依然在下国际象棋甚至围棋。

People are still playing chess and even Go.

是的，甚至在不久的将来，有些人已经在提倡用基本收入来取代工作。

Yeah, and in the very near term even some people are advocating basic income to replace jobs.

但如果政府愿意无条件向民众发放现金，那么也应该认真考虑是否应该同时大幅增加教师、护士这类常让人获得巨大成就感的岗位数量。

But if the government is going to be willing to just hand out cash to people for doing nothing, then one should also seriously consider whether the government should also just hire a lot more teachers and nurses and the kind of jobs which people often find great fulfillment in doing.

我们已厌倦听到政客们说'我们雇不起更多教师，但或许可以推行基本收入'这类论调。

We get very tired of hearing politicians saying, oh we can't afford hiring more teachers, but we're going to maybe have basic income.

如果能更深入研究什么赋予我们生命意义，就会明白工作带来的远不止收入，对吧？

If we can have more serious research and thought into what gives meaning to our lives, the jobs give so much more than income, right?

然后再思考未来，我们希望人类在哪些角色上能持续获得来自机器的赋能？

And then think about in the future, what are the roles that we want to have people continually feeling empowered by machines?

我来自俄罗斯，来自苏联，我认为对二十世纪的许多人来说，登月、探索太空是件鼓舞人心的事。

And I think sort of I come from the Russia from the Soviet Union and I think for a lot of people in the twentieth century, going to the moon, going to space was an inspiring thing.

我觉得宇宙

I feel like the universe

的

of

心智，所以人工智能，理解并创造智能就是二十一世纪的太空探索。

the mind, so AI, understanding creating intelligence is that for the twenty first century.

这确实令人惊讶，我也听你提到过，让我惊讶的不仅是研究资金方面投入不足，更重要的是在政治层面，除了杀人机器人、终结者这类观点外，它尚未进入公共讨论范畴——人们似乎还未对我们能共同构建的积极未来感到兴奋。

So it's really surprising, and I've heard you mention this, it's really surprising to me both on the research funding side that it's not funded as greatly as it could be, but most importantly on the politician side that it's not part of the public discourse except in the killer bots, Terminator kind of view, that people are not yet, I think, perhaps excited by the possible positive future that we can build together.

而我们

And we

应该如此，因为政客通常只关注下一次选举周期，对吧？

should be, because politicians usually just focus on the next election cycle, right?

我认为在整个人类科学史上我们学到的最重要一课就是：我们最擅长低估自己。

The single most important thing I feel we humans have learned in the entire history of science is that we're the masters of underestimation.

我们一次又一次低估了宇宙的规模，逐渐认识到我们以为存在的一切都只是更宏大存在的一小部分——行星、太阳系、银河系、星系团、宇宙。

We underestimated the size of our cosmos again and again, realizing that everything we thought existed was just a small part of something grander, planet, solar system, the galaxy, clusters of galaxies, universe.

而我们现在知道，未来蕴含的潜力远超我们祖先所能想象的。

And we now know that the future has so much more potential than our ancestors could ever have dreamt of.

想象一下这个宇宙：如果整个地球除了马萨诸塞州剑桥市外完全没有任何生命。

This cosmos, imagine if all of earth was completely devoid of life except for Cambridge, Massachusetts.

如果我们毕生的追求只是永远待在剑桥市，然后在一周后灭绝——尽管地球还会继续存在很久——这难道不是很可悲吗？

Wouldn't it be kind of lame if all we ever aspired to was to stay in Cambridge, Massachusetts forever and then go extinct in one week even though earth was gonna continue on for longer.

我认为我们现在在宇宙尺度上就抱着这种态度。

That sort of attitude I think we have now on the cosmic scale.

生命可以在地球上繁荣发展不是短短四年，而是数十亿年。

Life can flourish on earth not for four years, but for billions of years.

我甚至能告诉你当太阳变得过热时，如何让地球避开危险。

I can even tell you about how to move it out of harm's way when the sun gets too hot.

而且我们在太空中拥有更多资源——虽然今天可能有许多其他星球存在细菌或类似牛的生命——但就我们目前所知，这些机遇大多像撒哈拉沙漠一样死寂。而我们却有机会帮助生命在此繁荣数十亿年。

And then we have so much more resources out here, which today, maybe there are a lot of other planets with bacteria or cow like life on them, but most of this, all this opportunity seems as far as we can tell to be largely dead, like the Sahara Desert, and yet we have the opportunity to help life flourish around this for billions of years.

所以，让我们别再为某条边界该往左还是右挪一英里而争吵不休了，抬头看看天空，意识到，嘿，我们其实能成就如此非凡的事业。

So like let's quit squabbling about whether some little border should be drawn one mile to the left or right and look up into the skies and realize, hey, know, we can do such incredible things.

是啊，所以我觉得你和其他人参与埃隆·马斯克的部分工作特别令人振奋，因为他实实在在地迈向太空，真正探索我们的宇宙，这就是

Yeah, and that's I think why it's really exciting that you and others are connected with some of the work Elon Musk is doing because he's literally going out into that space, really exploring our universe and it's That

这正是埃隆·马斯克被严重误解的原因。

is exactly why Elon Musk is so misunderstood.

对吧？

Right?

别把他曲解成某种悲观的末日预言者。

Misconstrue him as some kind of pessimistic doomsayer.

他如此重视AI安全的原因，正是因为他比几乎所有人都更清楚这些绝佳机遇——如果我们在地球上自我毁灭，就会白白浪费这些机遇。

The reason he cares so much about AI safety is because he more than almost anyone else appreciates these amazing opportunities that we'll squander if we wipe out here on earth.

我们毁灭的不只是下一代，而是所有未来的世代。

We're not just gonna wipe out the next generation, but all generations.

而那个存在于星空中的绝佳机遇，那样被浪费就太可惜了。

And this incredible opportunity that's out there, and that would really be a waste.

对于那些认为没有技术会更好的人，我想谈谈人工智能。

And AI for people who think that there'd be better to do without technology.

让我说明一点：如果我们不改进技术，问题不在于人类是否会灭绝。

Let me just mention that if we don't improve our technology, the question isn't whether humanity is going to go extinct.

问题只在于我们是被下一颗大陨石、下一座超级火山，还是其他我们可以用更多技术轻易预防的愚蠢灾难毁灭，对吧？

The question is just whether we're going to get taken out by the next big asteroid or the next super volcano or something else dumb that we could easily prevent with more tech, right?

如果我们想让生命在宇宙中繁荣发展，人工智能就是关键。

And if we want life to flourish throughout the cosmos, AI is the key to it.

正如我在书中详细提到的，我觉得即使是最富想象力的科幻作家，也完全低估了太空旅行的可能性——尤其是前往其他星系——因为他们没有考虑通用人工智能的可能性。嗯。

As I mentioned in a lot of detail in my book right there, even many of the most inspired sci fi writers I feel have totally underestimated the opportunities for space travel, especially at the other galaxies because they weren't thinking about the possibility of AGI Mhmm.

这让一切变得简单多了。

Which just makes it so much easier.

没错。

Right.

是的。

Yeah.

所以这正符合你对AGI的看法，它推动我们的进步，创造更美好的生活，这种表述方式很美好，也是我们应当追求的目标。

So that that goes to your view of AGI that enables our progress, that enables a better life, so that's a beautiful way to put it and something to strive for.

马克斯，非常感谢你，谢谢你今天抽空参与，这次对话太棒了。

So Max, thank you so much, thank you for your time today, it's been awesome.

非常感谢。

Thank you so much.

谢谢。

Thanks.