本集简介
双语字幕
仅展示文本字幕，不包含中文音频；想边听边看，请使用 Bayt 播客 App。
这是iHeartPodcast。
This is an iHeartPodcast.
保证是真人。
Guaranteed human.
没赶上最新的罗兰·马丁无删减播客？
Didn't catch the latest Roland Martin unfiltered podcast?
这是你错过的内容。
Here's what you missed.
人们醒来一看，哎呀，天哪。
People wake up and go, oh, damn.
等等。
Wait.
等一下。
Hold up.
他们把所有那些都改了？
They changed all of that?
是的。
Yes.
这是真的。
It's real.
这是一次全面的攻击。
This is a wholesale attack.
它针对的是每个联邦机构中的黑人。
It is targeting black people in every federal agency.
这很残酷。
It's raw.
白人从来没有让那种清算持续超过十年。
White folks have never allowed that reckoning to last more than a decade.
在黑人信息网络上收听罗兰·马丁的每日评论。
Catch Roland Martin's daily commentary on the Black Black Information Network.
并在iHeartRadio应用、Apple Podcasts上下载Roland Martin Unfiltered。
And download Roland Martin unfiltered on the iHeartRadio app, Apple Podcasts
或者你收听播客的任何地方。
Or wherever you get your podcasts.
酷区媒体。
Cool Zone Media.
我叫艾德·齐特龙，这里是《更好离线》。
My name's Ed Zitron, and this is better offline.
《更好离线》。
Better offline.
我回来了，我要宣布，我们将加快研究强大而危险的播客技术的进程。
I am back, and I'm announcing that we will be speeding up the process of researching powerful and dangerous podcasting technology.
我的研究员们每天在GPU计算上花费超过两亿美元，试图创造并生成一个能自我改进的齐特龙，任何世界政府的压力都无法阻止我。
My researchers are spending over $200,000,000 a day on GPU compute trying to create and generate a recursive self improving Zitron, and no amount of pressure from world governments will stop me.
但在我最伟大的作品完成之前，我必须继续录制。
But before my greatest work is complete, I must continue to record.
而今天，我们有一个绝对火爆的话题。
And today, we've got an absolute barn burner.
今天和我一起的是计算机科学的锁骨，我是说，卡尔·纽波特教授，天体物理学家、《万物永恒》的作者亚当·贝克尔，两位，欢迎回到节目。
I'm joined by the claviculars of computer science, by which I mean professor Cal Newport, an astrophysicist and author of More Everything Forever, Adam Becker, fellas, welcome back to the show.
谢谢邀请我们。
Thanks for having us.
我对递归自我改进的想法感到害怕。
I'm I'm terrified of the idea of recursively self improved.
这意味着你的通讯现在会有20万字长吗？
Means your your newsletters will be 200,000 words long now?
这是我们可以期待的吗？
That what we'll expect?
它们要么是20万字，要么是100字。
They'll either be 200,000 words or a 100 words long.
哦，天哪。
Oh, man.
那倒不是最糟糕的。
That's not the worst thing then.
好的。
Alright.
继续，当
Keep When
Zitron的完美矩阵完成了。
the perfect matrix of Zitron is complete.
不过今天，我们是因为你在《纽约时报》上那篇关于理性主义者的精彩文章而受到启发的，Cal。
Well, today though, this was inspired by your wonderful piece, Cal, in New York Times about the rationalists.
我们从一个非常简单的问题开始。
And we are going to start very simple.
这些人是谁，为什么他们突然变得引人注目？
Who are these people, and why are they suddenly notable?
嗯，我可能马上要把接力棒交给Adam了，
Well, I'm probably gonna actually throw the baton to Adam here in a second to
好的。
Okay.
讲讲这群人的历史吧，因为他的书是我了解他们的主要参考之一。
Give the history of this group because he and his book was one of my primary sources in trying to understand them.
不过先让我说说我为什么要写他们。
But let me just set the stage for why I was writing about them.
我觉得大家可能没意识到，尤其是OpenAI、Anthropic和DeepMind这些公司的核心人物有多古怪。
I don't think people realize just how weird some of the lead figures are who are at, in particular OpenAI, Anthropic, and DeepMind.
而且我觉得直到上周大家才意识到这一点，当时有二十个人在午餐时间随意讨论，我们所有人是否在接下来几年内有百分之十或百分之五十的几率会死。
And I don't think people realized that until last week, when you had a whole group of 20 casually debating in their lunch hour whether it's ten percent or fifty percent chance that we're all going to die within the next couple of years.
这引起了很多人的注意，大家会想，等等，那些实验室里到底发生了什么？
And that I think caught a lot of people's attention of like, a second, what is going on over there in those labs?
为什么只有这些实验室会说出这种话？
Why is this type of rhetoric only coming out of those particular labs?
为什么黄仁勋觉得这很荒谬？
Why does Jensen Huang think that this is ridiculous?
为什么中国在嘲笑这种说法？
Why is China laughing at this narrative?
这里肯定有什么不对劲的地方。
There's something going on here that's not right.
我觉得大家开始意识到这一点了，而且我认为确实是来自湾区的这一系列未来主义哲学，经过各种变异和分化，正在定义我们当下的时代，但几乎没有人真正意识到这一点。
I think that I think people are starting to wake up to this, and I think it really is this family of futurist philosophies that come out of the Bay Area that have mutated and split in various ways that are really defining our current moment, and no one really realizes it.
但亚当花了多年时间，天哪，沉浸在这个圈子里，读遍了他们写的一切。
But Adam spent years, god help him, immersed in this world, has read everything they've written.
他对这段历史了如指掌。
He knows the history well.
所以，亚当，我把接力棒交给你。
So actually, Adam, I'll throw the baton to you.
也许你可以给我们讲讲理性主义和有效利他主义的故事，它们从哪里来，又是如何分化的。
Maybe you can give us a story of rationalism and effective altruism and where they came from and how they've split.
对我来说，这是一个迷人又离奇的故事。
To me, it's a fascinating, odd story.
天哪。
Oh, god.
我一般被问到这个问题时，标准回答就是，嗯，我可以写一整本书来讲这个，而且我也确实写了。
I mean, my my standard response when people ask me this is, you know, well, I I could write a whole book about it and I did.
而且，你知道，我不会像书里那样讲得那么详细。
And and, you know, I'm not gonna get into the same level of detail I do in the book.
但基本上，这东西是从九十年代的超人类主义论坛和邮件列表里诞生的。
But, like, basically, this is something that was spawned out of transhumanist message boards and email lists in the nineteen nineties.
超人类主义是什么？
And what is transhumanist?
问得好。
That's a good question.
超人类主义者，我能给你的最简短又准确的定义就是，他们不想再当人了。
Transhumanists, I guess the the shortest good definition I could give you is transhumanists are people who don't wanna be human anymore.
他们想用机器，尤其是电脑，来逃离人体，解决死亡问题。
They want to use machines and especially computers to basically escape human bodies and solve the problem of death.
明白吗？
Okay?
所以那些人？
So those guys?
普通人。
Normal shit.
普通人。
Normal shit.
普通人。
Normal man.
正常做事的方式。
Normal way to do things.
没错。
Exactly.
所以，这些人对人工智能的前景非常兴奋，也对纳米技术的前景非常兴奋，这种技术基本上可以像魔法一样，让他们随心所欲。
And and so, you know, these are people who are really excited about the prospect of AI, the prospect of, you know, nanotechnology that could, you know, basically serve as a kind of magic to do whatever they wanted.
那是九十年代的事了。
This was the nineties.
纳米技术当时可是热门话题。
Nanotech was like the hot thing.
那些想法基本上都来自科幻小说。
And it was a bunch of ideas that basically come from science fiction.
你知道，他们科幻小说读得太多了，把虚构当成了现实，觉得既然所有科幻小说都发生在未来，里面都有超级智能机器、纳米技术，还有人把大脑上传到电脑，那只要努力奋斗、拼命期盼，未来就一定会这样。
You know, they they read too much sci fi and confused it with reality and thought, you know, oh, well, because all of the sci fi I read takes place in the future and all of the sci fi I read has super intelligent machines in it and nanotech and, you know, people uploading their brains into computers, that must be what's gonna happen in the future if we just, you know, work hard and wish hard enough for it.
所以我想确认一下，这对他们来说是确定无疑的。
And so just so I'm clear, this is this is certainty for them.
他们百分之百确信这会发生。
They're 100 sure this will happen.
他们百分之百确信自己能实现它。
They're 100% sure that they could make it happen.
哦，所以这掌握在他们手里。
Oh, so it's in their hands.
就像，你知道，他们觉得，对，没错。
Like, you know, they they think, oh, yeah.
这件事一定会发生，除非有什么东西先终结了文明，或者没人听我们的，诸如此类。
This is gonna happen unless something, you know, ends civilization first or, you know, nobody listens to us or something like that.
他们基本上认为自己在做人类历史上最重要的事情。
They think that they're working on the most important stuff in the history of humanity, basically.
他们觉得自己能让自己永生。
They think that they are going to, you know, make themselves immortal.
而且当
And when
你说“工作”的时候，对。
you say working on Yeah.
不过那听起来挺宽泛的，因为很多理性主义者实际上并不做那些事。
That seems like a fairly broad thing, though, because a lot of these rationalist types don't actually work on the stuff.
他们只是在旁边工作。
They work next to it.
他们，对。
They Yeah.
就像搞哲学的那些人一样。
It's like philosophy people.
我的意思是，与其说他们在旁边工作，不如说他们只是在谈论这些。
I mean, it's not even so much that they're working next to it.
他们就是在聊这些。
It's like they talk about it.
他们喜欢谈论纳米技术的前景。
They like to talk about the prospect of nanotechnology.
他们喜欢谈论超级智能AI什么的。
They like to talk about, you know, super intelligent AI.
他们就是喜欢聊这些。
They like to talk about these things.
不过，确实是这样。
But but yeah.
所以这些人在九十年代花了很多时间在论坛上做这些事，然后在九十年代中期到后期，有一个人出现在其中一个最有影响力的论坛上，就是那个“外推人”论坛，你知道的，别让我提那个名字。
So these people spent a lot of time doing that on message boards in the nineties, and there was this one guy who showed up in the mid to late nineties on one of the most sort of influential of these message boards, the extropians, which, you know, don't get me started on that name.
这源于
It comes out
对物理学的
of a
误解。
misunderstanding of physics.
这个名字真烦人。
That's an annoying name.
不。
No.
那是
That is
这就是你
That's that's how you
知道这是一群严肃科学家的方式。
know this is a serious group of scientists.
没错。
That's right.
就是那些外熵主义者。
The extropians.
对对，没错。
That's that's correct.
完全正确。
That's exactly right.
然后他们就说，是啊，我们要对抗热力学第二定律，而我作为物理学家的反应就是，你知道，祝你好运。
And they're like, yeah, we're gonna fight the second law of thermodynamics to which my response as a physicist is, you know, yeah, good luck with that.
但基本上，有个叫埃利亚斯或雅德科夫斯基的家伙出现了，那时候他还是个十几岁的孩子。
But basically, this guy this this kid shows up named Elias or Yadkowsky, who is a teenager at that point.
他辍学了，初中毕业后就辍学了，完全没再上学。
Dropped out dropped out of middle after middle school, dropped out of schooling.
这就是他辍学的原因。
This is like why he dropped out.
这太不可思议了。
That is wild.
是的。
Yep.
八年级后就辍学了。
Dropped out after eighth grade.
没错。
That's correct.
这太疯狂了，老兄。
That's crazy, man.
是的。
Yep.
然后，尤德科夫斯基继续写了很多他自己对AI未来走向的深刻思考。
And and, yeah, Yudkowsky then goes on to write an enormous amount of, you know, his own deep thoughts on what he thinks the future of AI is going to be.
而且他很快就得出了结论，认为AI会变得自我改进，并在不久的将来导致超级智能AI的出现，进而引发一个一切都改变的技术奇点，所有超人类主义者认为不可避免的事情都会发生。
And relatively quickly comes to the conclusion that AI is going to you know, he he takes an article of faith among these people which is that AI is going to become self improving and lead to a super intelligent AI at some point in the relatively near future and lead to a singularity where everything changes and, you know, all of the stuff that the transhumanists believe is inevitable is going to happen.
尤德科夫斯基接着得出结论，这样的AI可能不会对人类友好，除非处理得当，否则很可能不会。
He comes to Yadkowsky then comes to the conclusion that such an AI might not be friendly to humanity and probably wouldn't be unless it's done properly.
所以奇点就是电脑比我们聪明的时候，我这样理解对吗？
And the singularity is when the computer is smarter than us, just so that I understand?
这个定义其实很模糊。
It's when it's really ill defined.
可能是电脑比我们聪明，或者可能是电脑能把我们上传到更多电脑里，这样我们都变成电脑，一切都是电脑。
It's when the computer is smarter than us or maybe it's when the computer can upload us into more computers so we can all be computer and everything is computer.
奇点就是科技一直发展，所有东西都变，其实没有明确定义。
The singularity is when is when a lot of tech but all the time and everything, like, it's not a well defined thing.
好的。
Cool.
这就是他们痴迷的东西，一个定义模糊的东西。
And this is the thing they obsess over, the thing that's ill defined.
哦，是的。
Oh, yeah.
他们认为这几乎是不可避免的，除非我们先经历核末日，或者
They they think that that is pretty much inevitable unless we have a nuclear apocalypse first or
等等，我们先停一下，
something Let's just pause on
就这个话题先暂停一下。
this for a second.
我觉得这
I think this
真的很重要，因为亚当说的这些想法，和现在人们讨论的内容非常相似。
is really important because these ideas that Adam's talking about sound very familiar to what people are talking about today.
他们当时就在讨论这个。
They were talking about this.
这大概是2000年代初，差不多是亚考斯基态度转变的时候，他当然认为超级智能不可避免，但那时他开始担心而不是兴奋。
This is about the early 2000s is roughly when Yacalski kind of had this change of heart of superintelligence, of course, is inevitable, but now he was worried about it instead of being excited about it.
这一切都只是思想实验。
This was all thought experiment.
那时候还没有大语言模型。
There's no LLMs.
从计算机科学的角度来看，那会儿正是人工智能的寒冬。
This was during an AI winter from the computer science perspective.
完全是靠科幻小说里的想法来推导的。
It was entirely inductive thinking directed by ideas from sci fi.
所以他们就坐在那儿，光靠脑子想这些。
So they just sat there and thought about this.
所以这不是工程师看着某个技术路线说，哎呀，我担心这玩意儿会往哪儿走。
So this is not an engineer looking at a particular technical trajectory and saying, Uh-oh, I'm worried where this is going to go.
而是人们纯粹在脑子里做思想实验，想着，如果人工智能能递归地自我改进，它就会变得超级智能。
This is people just thinking through the thought experiments of, well, if AI could recursively self improve, it would get super intelligent.
如果你有一个超级智能的人工智能，会发生什么？
What would happen if you had a super intelligent AI?
然后他们就在脑子里穷尽各种可能的情景，比如，也许它会很友好，但也许它会骗我们，万一它做了这个或者那个呢？
And then just exhaustively working out in their own minds various scenarios about, Maybe it would be friendly, but maybe it would trick us, and what if it did this or that?
所以我觉得这一点很重要。
And so I think that's important.
所以你看，所有这些想法都是在人们担心的具体技术出现之前就已经确立的，完全基于当时的思维实验。
So you have all these ideas are established well before there's any particular technology that people are worried about established entirely based off of, at this point, thought experiments.
我觉得这跟实际科学家没什么关系。
I think that's So this isn't proximity to actual scientists.
对。
Nope.
或者说，并没有亲眼看到实际技术的发展，只是停留在理论层面。
Or so nothing about witnessing the actual technology following other than an arm's reach.
对。
No.
我的意思是，正如卡尔刚才说的那样。
I mean, it is exactly as Cal just said.
换一种说法，这基本上就是他们在进行的一种极其枯燥的同人创作。
And another way of phrasing that is, it's it's essentially a kind of extremely dry fan fiction that they're engaging in.
他们写的是基于从科幻小说里得来的想法而编造出的充满焦虑的场景，还认为那就是未来的样子。
They are writing, you know, anxiety fueled scenarios based on these ideas that they got from science fiction thinking that that's what the future is going to be.
他们自己信吗？
Do they believe it?
哦，信啊。
Oh, yeah.
所以他们是认真的。
So they're sincere.
只是有点怪。
They're just odd.
对。
Yes.
而且当时，你可以把他们当成是认真的怪人，一笑置之。
And at the time, you can, like, dismiss them as sincere oddballs.
后来呢，尤德科夫斯基一直不停地写，被邀请去一些群博客投稿，然后自己开了个博客叫“少犯错”，结果这个博客就变成了……
What happens is Yudkowsky just keeps writing and writing and gets invited to contribute to some group blogs and then spins off his own sort of blog called Less Wrong, which becomes
一个巨大的
a huge
人们聚集的中心。
hub for people.
那就是他。
That's him.
但另一件事是，大约在这个时候，也就是2000年代末、2010年代初，尤德科夫斯基遇到了彼得·蒂尔。
But the other thing that happens is it's around this time, like the late two thousands, early twenty ten, somewhere in there that Yodkowsky meets Peter Thiel.
哦。
Oh.
彼得·蒂尔给了尤德科夫斯基一大笔钱。
And Peter Thiel gives Yodkowsky a bunch of money.
他那时多大年纪？
How old is he at the time?
哦，我算过这个。
Oh, did this math.
亚当，我们和事实核查员一起算过这笔账。
We did this math, Adam, with the fact checkers.
那是在二十一世纪头十年，他大概二十五六岁到二十七八岁的样子。
This is like two the first decade of the two thousands, he was, like, mid to upper twenties.
差不多吧。
Something like that.
没错。
That's right.
差不多有十年时间，他
It's like a decade of him a decade
在给那个孩子钱之后。
after giving he a child money.
我觉得，这对彼得·蒂尔来说并不奇怪。
I mean, that's not out of character for Peter Thiel.
但他确实花钱让人不去上大学。
But he literally pays people not to go to college.
还记得吗？
Remember that?
没错。
Exactly.
不过问题是，有那个泰尔奖学金，就是那种‘我给你钱，你别上大学，去搞点东西’的项目。
The thing is, though, there's like the Thiel fellowship where it's like, I'll pay you not to go to college to build something.
这个就像是，‘哦，我喜欢你的博客，小伙子。’
This is just like, oh, I liked your blog, kiddo.
差不多吧。
Pretty much.
你觉得世界要爆炸了。
You think the world's gonna explode.
你最好再多写点博客说说这个。
You better blog about that some more.
他们是在一个叫‘奇点峰会’的活动上认识的，亚当，这个得说清楚。
They met at tough to get this right, Adam, but they were at an event called the Singularity Summit.
哦，天哪。
Oh, gosh.
而且，我相信
And and that's I believe
他们就是在那里认识的。
that's where they met.
或者也许是在那里
Or maybe that's where
事情变得复杂了。
it gets complicated.
他们
They
确实相信是这样。是的。
definitely believe it's a Yeah.
确实如此。
It does.
但是大量资金开始涌入这个领域。
But a lot of money enters the scene.
就是说，从这里开始，大量资金开始进入这个世界。
Like, this is where, like, a lot of money starts entering this world.
对。
Yes.
然后亚德科夫斯基和其他一些超人类主义未来主义者，比如雷·库兹韦尔，一起创办了一系列会议，就是“奇点峰会”，每年把大家聚在一起讨论这些话题。
And and Yadkowski along with some other, you know, transhumanist futurist types like Ray Kurzweil starts this, you know, set of conferences, the Singularity Summit, and, you know, just gets people together every year to talk about this stuff.
然后“理性主义者”这个社区在Less Wrong网站上，你知道，变得越来越壮大，他们开始在湾区聚会。
And the community of rationalists on Less Wrong just, you know, start you know, gets bigger and bigger, and they start meeting up in the Bay Area.
他们开始一起住在合租的房子里。
They start living together in group houses.
那“理性主义者”是什么意思？
And yet rationalist mean?
基本上就是说，你花大量业余时间阅读亚德科夫斯基的文章。
Rationalist basically means you spend a lot of your spare time reading Yadkowsky's writing.
亚德科夫斯基这时候认为，如果你不相信奇点即将到来，也不相信人工智能对人类构成生存威胁，除非你能找到正确的应对方法，那他就觉得你不够理性。
Yadkowsky thinks at this point that if you don't believe that the singularity is coming and that AI is an existential threat to humanity unless you figure out how to do it right, He thinks if you don't believe those things, you're not thinking rationally.
所以他决定大量发文，教大家怎么理性思考。
And so he decides to post a lot about how to think rationally.
就这样。
That's it.
对。
Yes.
就这样。
That's it.
就这样？
That's it?
这就是整个理性主义的部分？
That's the entire rationality part?
这就是为什么
That's why
这跟理性生活没关系。
It's not about living rationally.
就是他们所谓的理性信念，觉得电脑可能会醒过来。
It's just about the rash their so called rational belief that the computer might wake up.
你是不是在搞什么——
Are you fucking
其实他们把这一套用在了生活的方方面面。
Well, they apply it to their whole lives.
比如，那两个大博客就叫《克服偏见》和《少犯错》。
Like, that's why the two big blogs are called overcoming bias and less wrong.
全都是那种特别痴迷技术的方式，就是‘我要让自己的思维方式超级精确’。
It's all about this sort of like really sort of tech obsessive way of I'm going to be super accurate in the way I think.
然后就是‘我不会被认知偏见打败’。
And this is I'm gonna I'm not going to be overcome by cognitive biases.
‘我不会被骗。’
I'm not going to be tricked.
我们可以像超级福尔摩斯一样，这就是我们解锁世界潜力的方式。
We can just be hyper Sherlock Holmes is, that's how we're going to unlock the potential of the world.
但尤科夫斯基认为，这项技术的终极应用是防止超级智能AI导致人类灭绝。
But Yukowski thinks the ultimate application of this is preventing the extinction of humanity by preventing superintelligent AI.
所以我们必须培养一支超级理性主义者的军队。
And that is why we have to raise an army of hyper rationalist people.
这是我们面对即将到来的人工智能末日唯一生存的机会。
It's our only chance to survive the coming AI apocalypse.
所以这成了他的执念。
So that kind of becomes his obsession.
他领导的超级理性主义理念，就像是天网场景中的约翰·康纳。
Idea of hyper rationalism that he's the leader of is gonna be that's the John Connor of this Skynet scenario.
但他们似乎除了说话之外什么都没做。
But they don't seem to be doing anything other than talking.
对。
No.
目前来看，他们确实没有。
At this point, they're not.
他们其实也在做一些事情。
They're they're doing some things.
我是说，他们搞多角恋。
I mean, they're in the polyamory.
还有其他一些事情在进行，对吧。
I mean, there's some other things going Right.
但是，说得很清楚一点，尤德科夫斯基自封为理性思维和认知偏差方面的专家，还教人怎么思考。
But, like, to to be very, very clear, Yodkowski is anointing himself as an expert on, you know, the the art of rationality and cognitive biases and how to think.
而且，这家伙完全是自学成才，对世界的了解非常有限。
And, you know, this is this is an entirely self trained, self taught guy who whose experience of the world is pretty limited.
他在Last Wrong上发了成千上万的文字，然后开始写作，这既是艺术表达，也是一种招募手段。
And he is posting hundreds of thousands of words about this on Last Wrong and then starts writing as, you know, a combination of artistic outlet and recruitment technique.
他开始写真正的同人小说，一部哈利·波特同人，叫《哈利·波特与理性之道》。
He starts writing literal fan fiction, a Harry Potter fan fiction called Harry Potter and the methods of rationality.
哦。
Oh.
大概有六十万字那么长吧。
It's, like, 600,000 words long or something.
比沃伦的还多。
It's more than Warren
抱歉。
I'm sorry.
连我，对，
Even I Yeah.
都不会写那么长的东西。
Would never write something that long.
连我，写二十万字都够吓人的了。
Even I would 200,000 words on anything is terrifying.
我是说，这有六十万字。
I mean, this is 600,000 words.
但那到底讲的是什么鬼？
But what the fuck was it even about?
是不是像那个，哦，它是
Was it like the Oh, it was
关于哈利的，你知道，我觉得很明显是作者亚尔科夫斯基的自我代入。
about it was about Harry who is, you know, I I think fairly clearly an author self insert of Yarkowski.
哈利是个特别的男孩，想办法用他特别的逻辑和科学能力去改变和拯救哈利·波特的魔法世界。
Harry being the specialist boy and finding ways to, you know, change and save the wizarding world of Harry Potter using his his special logic and science powers.
通过逻辑和理性。
Through logic and reason.
没错。
Exactly.
是的。
Yes.
那，为什么？
Well, why?
我猜，我猜，我猜这纯粹是创意吧。
Guess I guess I guess it's just creative.
其实，我为了写书采访过他，他当时就是这么说的。
I mean, I interviewed him for my book, and that was what he claimed.
而且我相信一开始确实是出于创意，但很明显，他也在借此表达他的一些想法，比如他认为这是唯一清晰的思考方式。
And I I believe that it started out as a creative thing, but, you know, it was also very clearly him playing with these ideas that he has about, you know, this is the only way to think clearly.
最后它确实变成了理性主义者的一种招募工具。
And it did end up becoming, you know, quite a recruitment tool for the rationalists.
为了让大家把时间线搞清楚，我先说一下，Adam 你帮我确认一下。
And just to put a look just make sure we have, like, the timeline beats clear and then Adam fact check me here.
大概在2000年代初期有个转变，我记得他最早的投资人是彼得·蒂尔，但还有另一对互联网创业夫妇。
There's sort of the shift that happens where in the early two thousands, actually I think his first money was Peter Thiel but it was also that other Internet entrepreneurship couple.
他们资助了一个叫“人工智能奇点研究所”的机构。
They fund something called the Institute for AI Singularity.
对，
Oh yeah,
人工智能奇点研究所。
the Singularity Institute for AI.
最初资金来自一对夫妇，名字我记不清了，但几年后，在2000年代中期到后期，他遇到了蒂尔，蒂尔给了他们真正的资金。
Originally the funding comes from this couple, the name of which I don't remember, but eventually a few years after that in the mid to late 2000s, he meets Thiel, and Thiel gives them real money.
随着2000年代的发展，奇点研究所认为超级智能将成为一个有益的神，能够救赎历史。
And this is where so as the 2000s move on, so that Singularity Institute was superintelligence is going to be a beneficial godhead that's going to redeem history.
我的理解是，一旦这成为全职工作，他思考这个问题，转变就很快发生。
My understanding is once that's a full time job and he's thinking about it a shifts pretty quickly.
在第一份大报告中，他很快转变，说，等等，我觉得我们控制不了AI，我们根本不应该建造它，它会杀了我们。
That first big report, he shifts pretty quickly, be like, actually wait, I don't think we'll control AI, we should not build it at all, it's going to kill us.
在那个领域，这种转变很快发生，价值取向变了，最后他们把它改名为MIRA。
There was this shift that happened quickly in that area, shifted valence, and then eventually they renamed it to MIRA.
对，没错。
Yeah, that's correct.
为了理清时间线，雅德科夫斯基从认为超级智能会解决一切，我们应该尽快实现它，转变为，天哪，它会杀了我们所有人。
So just to get the timeline straight here, Yadkowsky makes that shift from, you know, superintelligence is gonna solve everything and we should be aiming for it as fast as possible to, oh my god, it's gonna kill us all.
他在人工智能奇点研究所成立的那段时间，也就是2000年代初，就发生了这种转变。
He makes that shift around the same time Singularity Institute for AI is founded in the early two thousands.
这是有前因的，还是他某天突然就醒悟了？
Is there a is there an antecedent, or does he just one day wake up?
他，我
He's I
我的意思是，他长期以来一直在这些论坛上活跃，而且他也看到，比如，那些纳米技术的人，他们在八九十年代也想要类似的东西，也做出了类似的承诺，像埃里克·德雷克斯勒那些人，他们也有自己的研究所。
mean, he's just been on these message boards for a long time, and he also saw that, like, you know, the the nanotech guys who wanted a lot of the same things and made a lot of the same promises back in the eighties and nineties, people like Eric Drexler, they had their own sort of institute.
亚德科夫斯基，你知道，那时候他大概20岁，也没工作，我觉得他就是想，我要全职做这个。
Yadkowski, you know, at this point, he's, you know, unemployed and and about 20, and I I think he's he's like, I wanna work on this full time.
我能把这变成一份工作吗？
Can I turn this into a job?
然后他通过共同创立这个研究所找到了办法。
And finds a way of doing that by, you know, cofounding this institute.
但是，我也想，我想就你说的一件事提出异议，卡尔。
But I I also I wanna I wanna just push back on one thing that you said, Cal.
好的。
Okay.
其实也不完全是那样，你知道，雅德科夫斯基并不是不惜一切代价要阻止AI。
It's not quite that, you know, Yadkowsky wants to stop AI at all costs.
他是想在不惜一切代价阻止AI，直到他和别人搞清楚怎么安全地做这件事，到那时候他就想全速推进了。
He he wants to stop AI at all costs until he and others figure out how to do it safely, at which point he wants to go full speed ahead.
他仍然相信
He still believes
有意思。
Interesting.
在AI乌托邦，如果你能让一个超级智能AI不杀光所有人的话。
In the, you know, the AI utopia if you can get a super intelligent AI to not kill everybody.
所以他只是觉得那个问题比他原来想的要难。
So he just thinks that that problem is harder than he originally imagined.
所以他就觉得，可能需要
So he's like, might take
需要更长时间。
us longer.
我们还没完全做到。
We're not yet exactly.
是的。
Yes.
朝着他成为思想之王的方向发展。
Towards him being the thinking king.
就像那个思想之王，能告诉电脑如何做到最好，并且爱我们。
Like the thought king that can tell the computer how to be best that loves us.
基本上，他会成为数字之神的得力助手。
Basically He will be the right hand man to the digital god.
但是，亚当，这样对吗？
But, Adam, is this right?
他的这种思想转变，在技术上没有任何先例。
There is no technical antecedent for this shift in his thinking.
其实，这可能更难。
Actually, this might be harder.
从你的书和其他资料来看，我能得出的最好结论是，他的思想转变就发生在他获得资助、开始运营奇点研究所之后，他接了一个大写作项目。
The best I could tell from your book and looking at other sources is that the shift in thinking just came from as soon as he got the funding for the Singularity Institute that he was helping to run, he had this big writing project he took on.
他当时想，好吧，我要写一份关于超级智能的大报告。
He was like, okay, I'm going to produce this big report on superintelligence.
正是在撰写这份报告的过程中，他内心思考，说服了自己：等等，我觉得我们可能无法像想象中那样轻易控制它。
And it was in just the internal, cognition of thinking through this report that he just convinced himself, Oh wait, I don't think we're going to be able to control this as easily as we think.
因为我觉得强调这一点很重要，再说一次，他不是在实验室里看到了什么吓人的东西。
Because I think it's really important to emphasize, again, this is not someone who has seen something in a technical lab that is scaring him.
这些发展和他此时的思考，其实都只是他脑海中的茹科夫斯基。
All of these developments and his thinking at this stage, it's really just it's Joukowsky in his own mind.
错过了最新的罗兰·马丁无滤镜播客？
Didn't catch the latest Roland Martin unfiltered podcast?
这是你错过的内容。
Here's what you missed.
问题是，你想赢吗？
And the argument is, do you want to win?
如果你不赢，你就没有权力。
If you don't win, you have no power.
这是真的。
It's real.
那些人总是说投票违背自己的经济利益，然后抱怨他们正在得到的东西。
Those are the very people who always say vote against their own economic interest, and they complain about the very thing they're receiving.
这很直接。
It's raw.
我知道跨性别运动员的问题在很多黑人男性中并不受欢迎。
I knew that the issue of transgender athletes was not going well with a lot of black men.
罗兰·马丁无过滤。
Roland Martin unfiltered.
我发火了。
I went off.
我说，这正是我在说的。
I said, this is exactly what I'm talking about.
你们会对香奈儿大喊大叫，但你们不想看到
Y'all gonna scream at Chanel, but you don't wanna see
一个同等级的黑人奢侈品牌。
a black luxury company in the same vein.
收听罗兰·马丁在黑人信息网络上的每日评论。
Catch Roland Martin's daily commentary on the Black Information Network.
12个席位。
12 seats.
这些席位正在向南方转移。
Those are moving south.
所以，如果你们是民主党人，不要等到2030年，你们最好现在就在南方全力以赴。
So if you Democrats, you don't wait till twenty thirty, you better be hitting the ground going hard in the South now.
并在iHeartRadio应用、Apple Podcasts上下载Roland Martin Unfiltered。
And download Roland Martin unfiltered on the iHeartRadio app, Apple Podcasts
或者你收听播客的任何地方。
Or wherever you get your podcasts.
世界杯结束了，但如果你迷上了足球，又不知道接下来该干什么，那就来听听我们的播客《Big Kick Energy》吧。
The World Cup is over, but if you caught the soccer bug and aren't sure what to do now, check out our podcast, Big Kick Energy.
我是卡梅隆·迪克，洛杉矶闪电队的踢球手，NFL历史上最准的踢球手，但我是在上海的足球场上长大的。
I'm Cameron Dicker, the kicker from the Los Angeles Chargers, most accurate kicker in NFL history, but I was raised on the soccer pitches of Shanghai.
和我一起的是蒂米·蒂尔曼，洛杉矶FC的球星，在德国的青训营长大，现在闪耀在美国职业足球大联盟。
And I'm here with Timmy Tillman, LAFC star, grew up in the academies of Germany, and now under the bright of Major League Soccer.
所以我们很懂
So we know a
踢球这件事。
little something about kicking balls.
也许你是新手，或者你准备好把热情提升到新高度。
Maybe you're new to the game or you're ready to take your passion to the next level.
每周我们都会聊一些话题，比如今年谁会赢得英超冠军？
Each week, we'll tackle things like, who's winning the Premier League this year?
展开剩余字幕（还有 480 条）
像10号球员那样踢球。
Playing like a number 10.
你知道那是什么意思吗？
Do you know what that means?
然后我们转到法国。
Then we go over to France.
里昂。
Ligon.
里昂。
Ligon.
里昂。
Ligon.
里昂。
Ligon.
里昂。
Ligon.
利贡。
Ligon.
利贡。
Ligon.
他知道的更多。
He knows more.
他比我强。
He's better than me.
你觉得哪个更难？
What do you think is harder?
罚点球得分还是NFL得分，
Scoring a penalty kick or scoring a NFL,
然后我就觉得，这真是个重要因素。
and I'm like, just a big factor.
就这么做吧。
Do it.
那太疯狂了。
That's crazy.
每周加入我们，和你的足球兄弟们聊聊世界上最好的运动。
Join us each week for a quick chat with your soccer loving bros about the best sport in the world.
足球。
Soccer.
别告诉我的教练我这么说。
Just don't tell my coaches I said that.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方收听Big Kick Energy。
Listen to big kick energy on the iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts.
嘿。
Hey.
我是Robert Lim。
This is Robert Lim.
我是Joe McCormick，我们是《Stuff to Blow Your Mind》播客的主持人。
And this is Joe McCormick, and we're the hosts of the Stuff to Blow Your Mind podcast.
这周我们的节目是关于《星际迷航》的。
This week on the show, it's Star Trek.
所以我们要整周讨论博格人。
So we're gonna be talking about the Borg all week long.
你准备好让探针插进你的肉里了吗？
Are are you ready to get some probes put in your flesh?
没错。
That's right.
记得收听。
Tune in.
抵抗是徒劳的。
Resistance is futile.
我们会讨论科学、博格文化、各种解读，而且周五在《怪屋影院》我们还会聊《星际迷航：第一次接触》。
We're gonna talk science, the culture of the borg, various interpretations, and we're gonna talk about Star Trek First Contact on Weird House Cinema on Friday.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方收听《头脑风暴》。
Listen to stuff to blow your mind on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
大家好。
Hey there, everybody.
我是艾米·罗巴克，我是TJ·霍姆斯，这里是艾米和TJ播客。
Amy Robach and TJ Holmes here from the Amy and TJ podcast.
中期选举就要到了，我们来分析一下十一月之前最重要的政治新闻。
With midterm elections just around the corner, we're breaking down the biggest political stories leading up to November.
而且，这些重大政治新闻不只是国会山上的事。
And some of those big political stories aren't just what's happening on Capitol Hill.
对。
Yes.
我们刚刚看到林赛·格雷厄姆，还有一位长期参议员的去世。
We just saw Lindsey Graham, the death of a long time senator.
另外，还有几位国会议员失踪了。
Also, we've got a couple of missing members of congress.
还有不少国会议员行为不端。
Also, we got a lot of members of congress behaving badly.
国会山上整天都是各种指控和传闻飞来飞去。
We got accusations, allegations flying all day long on Capitol Hill.
简直就像真人秀一样。
It's just like a reality show.
在这个政治激烈的季节，我们会报道你需要知道的一切。
We're gonna cover everything you need to know during this heated political season.
而且从我们的晨跑开始。
And it starts with our morning run.
对。
Yes.
跟我们一起跑吧。
Go for a run with us.
也许你可以在户外跑步时带上我们，或者如果你不想出门，也可以。
Maybe you can take us along when you're on your actual run outside or if you don't feel like going out still.
你懂我们的意思。
You get what we're saying.
这是我们的晨间播报。
It's our morning run.
这是我们每日新闻摘要，涵盖最重要的头条，让你在喝完第一杯咖啡之前就能了解所有要闻。
It's our daily news rundown of the top stories that will get you going before you even finish that first cup of coffee.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方，收听Amy和TJ的节目。
Listen to Amy and TJ on the iHeartRadio app, Apple Podcasts, or wherever you listen to your podcasts.
每个案件都有它的故事。
Every case has a story.
每个受害者都有他们的声音。
Every victim has a voice.
但会被听到吗？
But will it be heard?
我是Nancy Grace。
I'm Nancy Grace.
在犯罪故事节目中，我们深入头条背后，审视证据，质疑官方说法，为受害者及其家人追求正义。
On crime stories, we go beyond headlines to examine evidence, challenge the story we're fed, and pursue justice for victims and their families.
欢迎收听《南希·格雷斯探案故事》，你可以在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方找到我们。
Please join us stories with Nancy Grace on the iHeartRadio app, Apple Podcasts, or wherever you find your podcasts.
希望你能加入我们。
Hope you join us.
我们到时候见，朋友。
I'll see you there, friend.
姐妹，如果你不再让金钱压力主宰你的生活，会发生什么改变呢？
What would change if you stopped letting money stress run your life, sis?
我是曼迪·曼尼，播客《布朗雄心》的主持人。
I'm Mandy Money, host of Brown Ambition, the podcast.
现在，我要带你进入我的六部分金钱重启系列。
And right now, I'm taking you through my six part money reboot series.
这是你按下重置键、打破债务循环、诚实面对你的钱都花在哪里、制定一个真正能坚持的储蓄计划，并开始做出支持你理想生活的财务选择的机会，而不仅仅是应付账单。
This is your chance to hit reset, break the debt cycle, get honest about where your money's going, build a savings plan that actually sticks and start making financial choices that support the life you want, not just the bills you have.
因为积累财富并不在于追求完美。
Because wealth building is not about being perfect.
关键在于有计划，了解自己的数字，并且带着信心迈出下一步，就像我们在Brown Ambition那样，有一个关心你的社区。
It's about having a plan, knowing your numbers, and making your next move with confidence with a community that cares like we do on Brown Ambition.
每一集都会给你实用的步骤、真实的对话，以及动力，让你不再拖延财务目标。
Each episode gives you practical steps, real talk, and the motivation to stop putting your financial goals on hold.
所以如果你一直说，下个月我会认真对待钱，这就是你的信号。
So if you've been saying, I'll get serious about money next month, this is your sign.
现在就开始。
Start now.
收听Brown Ambition。
Listen to Brown Ambition.
就是Brown Ambition，今天就开始你的金钱重启吧。
That's Brown Ambition, and begin your money reboot today.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方，收听Brown Ambition。
Listen to Brown Ambition on the iHeartRadio app, Apple Podcasts, or wherever you get your podcast.
我是Brett Coleman。
I'm Brett Coleman.
我是特雷弗·西克马。
And I'm Trevor Sickema.
你有没有看过一场
Have you ever watched a
比赛，然后想知道除了最终比分之外，到底是什么决定了胜负？
game and wondered what really made the difference beyond the final score?
这正是我们创建NFL pro的原因。
That's exactly why we created NFL pro.
有个球员完全没人防守。
Got a man wide open.
达阵，海鹰队！
Touchdown, Seahawks.
每周，我们都会利用独家下一代数据统计和NFL内容，深入分析联盟中最大的新闻、对决和精彩瞬间。
Every week, we use exclusive next gen stats insights and NFL content to break down the biggest stories, matchups, and moments from around the league.
我们超越比分板，挖掘数据、趋势和细节，帮助解释场上到底发生了什么。
We go beyond the box score to uncover the numbers, the tendencies, and details that help explain what actually happened on the field.
他把球传给了巴克利，巴克利过掉了一个人，又闪过了另一个，最后跳过了第三个。
Fires it out in the flat to Barkley who beats one man, spends past another, leaps over a third.
别担心。
And don't worry.
这不仅仅是一档给数据专家看的节目。
This isn't just a show for stats experts.
我们用高级数据分析来讲述比赛背后的故事，把数字变成每个球迷都能理解和享受的见解。
We use advanced analytics to help tell the story behind the game, turning the numbers into insights that every fan can understand and enjoy.
所以无论你是想多了解橄榄球的普通球迷，还是对分析永远不满足的铁杆球迷，我们的目标都是帮你用全新的视角看待比赛。
So whether you're a casual fan learning more about football or a hardcore fan who can't get enough analysis, our goal is to help you see the game in a whole new way.
如果你想知道为什么某个战术能成功，某场对决是怎么赢的，或者大多数球迷在周日错过了什么，NFL Pro 提供的见解是你在其他地方得不到的。
If you've ever wanted to know why a play worked, how a matchup was won, or what most fans missed on Sunday, NFL Pro delivers insights you simply can't get anywhere else.
在 iHeartRadio 应用、Apple Podcasts 或任何你收听播客的地方收听 NFL Pro。
Listen to NFL Pro on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
我觉得这一点非常重要。
I think that that's a really important point.
不过，这个观点一直都没变过吗？
Is that consistent through time, though?
我的意思是，从那时起，他就坚信这一点，但是……
I mean, he's he from that point on, he's convinced of this and has But he's
他没见过任何证据，这些人也没见过任何证据。
not seen anything, and these people have not seen anything.
对。
No.
你知道，后来他开始指出机器学习领域的各种进展，以此证明他提出的那个问题，我们解决它的时间已经不多了。
I mean, you know, he he later on, he starts pointing to various advances that are happening in machine learning as evidence that the amount of time that we have to solve this problem that he came up with is, you know, running out.
但事实并非如此。
But no.
你知道，他最初的那个想法以及他对它的信念，完全基于思想实验和与这个圈子里其他人的交流。
You know, that original idea and his his faith in it is based purely on thought experiments and conversations he has with other people in this community.
这些人谈论这件事时，就是那样自我标榜的。
Just the way these fucking people frame themselves when they talk about this.
他们说的，我是说，我是在说，我也不太清楚。
They talk I'm I'm speaking from I don't know.
就是我跟他们其中一个人刚做的一期播客节目。
A podcast episode I just did with one of them.
他们不会直接说，也许这就是他们美化自己的方式。
They don't talk about and maybe this is how they varnish it.
他们不会说在某些情况下他们会成为权威。
They don't talk about that they would be the authority in some cases.
他们只会说，我们应该有一个，听起来就是这样，这些人总是这样让你上钩。
They just like, we should have one, which sound that's how they always get you, these people.
像这样的人。
People like this.
他们总是听起来很合理，然后关起门来，就原形毕露了。
It's always they sound reasonable and then behind closed doors, trousers come off.
好的。
Okay.
这不是真的。
That's that's not true.
但也许吧。
But well, maybe.
但是，是的。
But Yeah.
但这关乎的是，哦，实际上，我们就像《第九道门》里的鲍里斯·巴尔坎，只有我配处理这件事。
But it's about it's about how oh, actually, oh, we are we are like Boris Balkan in the ninth gate, I alone am worthy to deal with this.
那太可怕了。
That's terrifying.
我一点也不喜欢那样。
I don't like that one bit.
我的意思是
I mean
哦，好的。
oh, okay.
你可别告诉别人。
You don't tell.
因为我们现在要把这事搞得更糟了。
Because I we're gonna make this worse now.
太好了。
Great.
好吧。
Alright.
所以这就成了关键，这是我《纽约时报》专栏文章的核心论点之一。
So this this then becomes the key this is the one of key arguments of my New York Times op ed.
比如，为什么这跟现在有关？
Like, why is this relevant to right now?
问题是，这个世界还在继续，雅科夫斯基有他的追随者，他坚信这一点，这主要是思想实验。
Well, the issue is, this world is going on, Yakowski has his followers, he's convinced of this, it's mainly thought experiments.
他并没有在关注某个特定的AI技术。
There's not a particular AI technology he's looking at.
那个世界和OpenAI、Anthropic以及DeepMind有很深的交集。
That world severely overlaps with OpenAI, Anthropic, and DeepMind.
让我给你一个
And let me give
非常简短的版本，亚当可以补充我遗漏的部分。
you the very condensed version, and Adam can add the pieces that I'm missing here.
我们回到DeepMind的创立之初。
But let's go back to the founding of DeepMind.
DeepMind有三位联合创始人。
There's three founders of cofounders of DeepMind.
其中两位，哈萨比斯和肖恩·莱格，参加了一个奇点峰会。
Two of them, Hassabis and Shane Legg, they're at a singularity summit.
他们正在和雅科夫斯基交谈。
They are talking with Yakowski.
雅科夫斯基说，让我介绍你们认识彼得·蒂尔，彼得·蒂尔为DeepMind提供了初始资金。
Yakowski says, let me introduce you to Peter Thiel, and Peter Thiel gives the initial funding for DeepMind.
Shane Legg长期以来深度参与了Yakowski理性主义那个圈子。
Shane Legg has been deeply involved in sort of the Yakowski rationalist world for a long time.
他之所以有动力，是因为听了Yukowski关于超级智能的演讲，那个演讲是在Lake工作的地方做的。
He was motivated by a talk that Yukowski came to give about superintelligence at the place where where Lake worked.
他后来写了关于超级智能的博士论文，还经常在Less Wrong上发帖。
He went on to write a doctoral dissertation on superintelligence, a frequent poster on Less Wrong.
所以DeepMind和那个圈子有很深的联系。
So DeepMind has these deep connections to that community.
当他们被谷歌收购时，Sam Altman和Elon Musk就觉得，这可不妙，对吧？
When they get bought by Google, that's when Sam Altman and Elon Musk are like, well, that's bad, right?
因为按照真正的理性主义观点，这项技术要么拯救人类，要么毁灭我们所有人。
Because this is, in a true rationalist sense, this is a technology that's going to either redeem humanity or kill us all.
我们不能让谷歌这样的公司控制它。
We can't have a company like Google control it.
就在那时，Musk为OpenAI提供了资金。
That's when Musk puts up the money for OpenAI.
那时候，Sam Altman 还在推特上夸 Yakowski，说他比任何人都更推动 AGI 的发展。
You have Sam Altman, you know, tweeting favorably about Yakowski at this point, saying he's done more for AGI than anyone else.
Yakowski 应该拿诺贝尔奖。
Yakowski should get the Nobel Prize.
OpenAI 现任总裁 Greg Brockman，之前在他工作的公司里办了一个读书会。
The current president of OpenAI, Greg Brockman, was running a reading group at the company where he worked before.
等等，连 Brockman 也是？
Wait, even Brockman?
Brockman？
Brockman?
Brockman
Brockman
是那个
was the
高尚的人。
noble one.
不不不。
No, no, no.
好吧，所以你看，这些联系很深。
All right, so you have these deep connections.
OpenAI 的董事会里，有一半人都是来自理性主义者和有效利他主义社区的。
The board of OpenAI, half of that board comes strongly out of the rationalist EA communities.
2023年他们试图把山姆·奥特曼赶下台的时候，就像凯特·梅茨当时报道的那样，正是董事会里那些有理性主义和有效利他主义背景的人觉得他不够符合理性主义的理念。
When they tried to oust Sam Altman in 2023, as Kate Metz reported at the time, it was the faction of the board with rationalist EA connections that felt like he was being insufficiently aligned with rationalist type ideas.
董事会当时临时找来的那个CEO，想用来替代奥特曼的。
The temporary CEO that the board put in place to try to replace M.
这个人跟理性主义圈子关系太深了，以至于尤科夫斯基写的哈利·波特同人小说里有个角色就是以他命名的。
Altman is someone who is so entrenched in rationalist circles that there is a character named after him in the Harry Potter fan fiction that Yukowski wrote.
达里奥·阿莫德伊和他妹妹，还有其他离开OpenAI创办Anthropic的员工，他们之所以走，也是因为觉得公司不够符合理性主义的理想，尤其是在如何安全地迎来超级智能这个问题上。
Dario Ambede and his sister and the other employees who left OpenAI to start Anthropic did so because they felt like, again, they were being insufficiently aligned in part to rationalist ideals about how do we usher in superintelligence safely.
还有，几年前Omnystudio也在那里鹦鹉学舌，重复理性主义者的那些常见论调。
You you have Omnystudio a couple years ago parroting common rationalist talking points.
比如说，我们大概有25%的概率认为AI会毁灭全人类。
Like, we have about 25% chance that AI is going kill all of humanity.
所以这些世界，这三家公司，都源自理性主义这个圈子，当时在湾区影响力很大，这意味着我们领先的AI实验室，他们思考和谈论AI的方式，深受我们刚才讨论的一切影响。
So these worlds, those three companies, they come out of this world of rationalism, which had a really big foothold in the Bay Area of the time, which means our leading AI labs, the way they think and talk about AI, is heavily influenced by everything we just talked about.
是的。
Yep.
没错，完全正确。
No, that's all correct.
所以当他们说对齐时，因为对齐这个概念本身并不是最糟糕的。
So when they say alignment, because alignment as an idea is not the worst idea.
它是确保模型输出好的结果，而且这些输出本质上不是恶意的。
It's making sure the models give good outputs and the outputs aren't inherently evil.
那并不是他们真正想表达的，当他们
That's not really what they mean when they
说不对的时候。
say No.
但这正是我想说的。
But that's exactly what I'm getting at, though.
因为“对齐”这个词，对大多数我交谈的人来说，意思就是我刚才说的那个，而他们所说的“对齐”，似乎是另一回事。
Because there is the term alignment, which means the thing I just said to most people I talk to, and then what they mean by alignment, which appears to be something else.
不。
No.
他们所说的“对齐”，基本上就是尤德科夫斯基说的那个意思：一个AI，当它变得超级智能时，不会杀死全部人类。
What they mean by alignment, and I mean, this is this is basically word for word what Yodcastsky says is an AI that when it becomes super intelligent will not kill one hundred percent of of humanity.
哇。
Wow.
这真让人火大。
That's fucking annoying.
是的。
Yes.
我知道我本来想说得更文雅一点，但这确实让我非常沮丧。
That's I I know I meant to sound more eloquent than that, but that's that makes me very frustrated.
好吧，为了让你
Well, to make
更沮丧，让事情变得更烦人、更气人，因为这些人在AI行业里根深蒂固，而且他们写这个写了那么久，所以当AI时刻对公众到来时，比如2022年底ChatGPT发布，理性主义者和有效利他主义者就能令人信服地把自己描绘成研究AI几十年的专家。
you more frustrated and to make it even more annoying and infuriating, because these guys were so deeply entrenched and involved with the AI industry and because they'd been writing about it for so long, It means that when the sort of AI moment arrived for the public with, you know, the release of Chat GPT in late twenty twenty two, the rationalists and effective altruists were able to convincingly portray themselves as experts who had been working on AI for decades.
这就是为什么你会看到像伯尼和伊莱亚斯或约德科夫斯基坐下来谈话这样的事。
And this is how you end up with things like, you know, Bernie having, you know, a a sit down with Elias or Yadkowsky.
现在这更说得通了，因为他们给自己定位的方式，他们会说，哦，AI 2027在某些方面甚至还挺准的。
This makes more sense now because the way they frame themselves and they'll say like, oh, AI 2027 has been accurate in some ways even.
而且你知道，在接下来的播客里，我会追问他们这一点，他们却拒绝在我面前打开那个博客。
And as you'll know in a podcast coming up, I push them on that and they refuse to actually open the blog in front of me.
但他们就是这样，把吓人的东西包装成学术的样子，实际上根本不是学术。
But it's it's how they have scary scary stuff framed in the terms of academia without actually being academic.
好。
Cool.
现在这里有两个重要的补充说明。
Now there's two important sidebars here.
先说第一个补充点，我们刚才一直在随便提到有效利他主义。
So sidebar one, we've been kind of casually referencing effective altruism.
这个很重要。
This is important.
事情在这里就开始变得复杂了。
This is where it gets complicated.
这是另一个有类似理念的运动，后来和它交织在一起，并且深受影响。
This is another movement that has similar sensibilities that became intertwined and heavily influenced.
它们可以说是互相影响。
They sort of mutually influenced each other.
但有效利他主义起源于牛津大学的一群功利主义哲学家，他们的想法是，我们可以应用——
But you have effective altruism, it comes out of a bunch of utilitarian philosophers at Oxford, and they have this idea of, Oh, we can apply.
这个听起来应该很熟悉——如果我们对利他主义非常理性和实证，我们就能把利他做得更好，我觉得这就像是现代功利主义。
This should sound familiar, but if we're very rational and empirical about altruism, we can do altruism better, which I think is like modern utilitarianism.
这个想法本身没问题。
That's a fine idea.
他们到处做研究，比如真正最好的慈善机构是哪些，该把钱捐给谁？
They're going around and doing research like what are the actual best charities to give money to?
怎么衡量这些呢？
How do we measure that?
他们有这样的想法：如果你在牛津这样的顶尖大学，做慈善最好的方式其实是去银行拿高薪，然后把一半工资捐出去。
They have these ideas like actually if you're at Oxford, like an elite university, the best thing you could do for charity is to go get a high paying banking job and then give up half your salary to charity.
所以他们试图理性地思考，怎么才能最有效地做慈善。
So they're trying to think through rationally how do we do altruism as effectively as possible.
他们和理性主义有重叠，也受其影响，很快就有很大一部分有效利他主义者说，实际上我们能做的最利他的事情就是阻止AI末日。
Well, They overlap and are influenced by rationalism, and pretty quickly you have a large segment of effective altruists who say, Actually, the most altruistic thing we can do is prevent an AI apocalypse.
何必呢？
Why bother?
为什么要费心去救现在两千万疟疾患者，而不是去救未来一万亿条可能因为AI灭绝而永远不会出生的生命呢？
Why care about saving twenty million people now from malaria when we could instead be saving a hundred trillion future lives that will never be born if AI kills us all.
所以有效利他主义，不是全部，这是个复杂的领域，但其中很大一部分。
So effective altruism, not all of it, it's a complicated field, but big segments of effective altruism.
这就是尼克·博斯特罗姆在牛津提出的关于存在风险和超级智能的观点。
This is where Nick Bostrom comes out of at Oxford talking about existential risk and superintelligence.
有效利他主义的一大群体开始真正关注超级智能AI，将其视为核心关注点之一。
A big segment of effective altruism really begins to care about superintelligent AI as one of the core things to care about.
大量资金涌入有效利他主义。
And a huge amount of money pours into effective altruism.
我们说的是数亿美元开始流向这些研究所和团体。
We're talking hundreds of millions of dollars starts going to these institutes and groups.
不过，目前还没有采取任何行动。
Nothing's been done yet, though.
我想说清楚。
I just wanna be clear.
就是说，还没有采取任何实际措施来阻止超级智能。
Like, no actual actions have been taken to stop superintelligence.
没有安全标准。
No safety standards.
实际上也没有真正的科学研究。
No actual science has taken place either.
嗯，他们写了哈利·波特主题曲。
Well, they wrote the Harry Potter fanfare.
好吧。
Okay.
所以没有。
So no.
我甚至不是在讽刺。
I'm I'm not even being sarcastic.
就是说，他们写了一些文章。
Like, they they wrote some posts.
他们写了一些文章，然后一直说这真的很难，因为，你知道，让某样东西符合人类价值观到底意味着什么？
They wrote some posts and and kept talking about how it was really hard because, you know, what does it mean to align something with human values?
然后，也许你的问题本身就有问题，也许
And, like, well, maybe maybe your question is ill posed and maybe
去搞什么该死的大语言模型。
work on a fucking large language model.
别搞AI了，别搞这些了。
Don't work on AI instead of this instead of them with
我是说，有些人确实去搞了。
I mean, some of them did.
你知道吗？
You know?
所以就有了Anthropic。
And that's how we got anthropic.
还有DeepMind和OpenAI。
And DeepMind and and OpenAI.
OpenAI。
OpenAI.
但是，就像，Cal说的都对，除了一件事，我觉得那件事很重要。
But but, like, just to everything Cal said is right except except one thing, and I think it's important.
尼克·博斯特罗姆绝对是有效利他主义社区的一员。
Nick Bostrom is definitely, like, part of the whole effective altruism community.
但尼克·博斯特罗姆，他写的书，他最出名的可能是那本叫《超级智能》的书，大概十多年前出版的，引起了很大的轰动。
But Nick Bostrom, he he wrote he's probably best known for a book that he wrote called Superintelligence that came out, I don't know, a little over ten years ago that made a big splash.
基本上，你知道，他在书中详细阐述了这些观点，说超级智能真的很可怕，我们应该想办法控制它或者阻止它，免得它把我们全杀了。
Basically, you know, expounding on all of these ideas and saying, yeah, you know, superintelligence is really scary and we should should we should figure out, you know, how to control it or stop it before it kills us all.
而且，这还是在变压器模型和当前AI热潮之前。
And again, this is before, you know, transformer models and and the current AI moment.
这也不是基于他观察到的任何特定事物。
And and it's it's not from any particular thing he's looking at.
但问题是，博斯特罗姆当时是牛津大学的哲学教授，他和尤德科夫斯基基本上来自同一个地方。
But the thing is, Bostrom, who at that time is a philosophy professor at Oxford, came out of the same, you know, place basically as Yudkowsky.
他也在那些九十年代的邮件列表上，而且他是连接理性主义和有效利他主义的最主要的人，因为当他在牛津的同事们在研究有效利他主义时，他去找他们说，哦，如果你担心那个，你真的需要担心AI，这两个世界完全不同。
He was on those same nineteen nineties listservs, and he's the one who sort of is most responsible for the connection between rationalism and effective altruism because it was when his colleagues at Oxford were sort of working on effective altruism that he went over to them and said, Oh, if you're worried about that, you really need to be worried about the AI of all two different worlds.
但在这一点上，
But at this point,
他们开始融合了。
they're coming to
融合了。
conjoined.
到了这个阶段，这两个群体真的已经分不开了。
They're at at this point, they're they're like they're really like you can't disentangle the two communities.
对，有个演讲你可以找找看。
Yeah, there's a talk you can find.
威廉·麦卡斯基尔是牛津大学两位重要的功利主义者之一，他帮助创立了有效利他主义。
So William McCaskill was one of the two big utilitarians at Oxford that helped found effective altruism.
我早年在有效利他主义圈子里和他有过交集，那时候他们特别关注把大部分工资捐给慈善机构这个想法。
And I crossed paths with him earlier on in the EA world when they were really caring about especially the idea of donate most of your salary to charity.
有个演讲你可以在YouTube上找到，我记不清具体是什么时候的事了。
There's a talk you can find, and it's on YouTube, and I forget exactly when this happens.
但麦卡斯基尔有个著名的演讲，他在里面基本上公开表示，我们现在应该关注的新问题是超级智能AI带来的生存威胁。
But there's a famous talk you can find in MacAskill where he basically comes out and says, this is the new thing we should care about is existential threat of superintelligent AI.
这就是所谓的长期主义。
This is the sort of long termerism.
所以他们用了另一个来自硅谷的想法。
So they used this another idea that comes out of Silicon Valley.
但这种对接下来一千年会发生什么的长期视角，实际上，如果我们只是算一下数学，未来的一万亿亿个生命比现在的五千万个生命更重要。
But this long term view of what's going to happen over the next thousand years is actually, if we're just going to do the math, 100,000,000,000,000 future lives matters more than 50,000,000 lives now.
所以他有一个时刻，他发表了那个重要的演讲，基本上是在宣布这是EA的新方向。
So he kind of has this moment where he gives this big speech where he's sort of announcing this is the new direction for EA.
所以现在他们基本上可以互换使用了。
And so now they're roughly interchangeable.
这就是为什么如果你读凯特·梅茨关于OpenAI董事会的报道，或者关于阿马德和他的哲学联系的报道，他们会把理性主义和EA混用。
This is why if you read Kate Metz's reporting on the OpenAI board or reporting about Amade and his philosophical connections, they'll use the terms rationalism and EA interchangeably.
他们会用其中一个，或者说两个都提，这就是原因。
They'll use one or the other, say both, and that's why.
这些圈子已经变得非常
These circles have become very
很简单。
simple.
而且他们似乎有不同的目标，但现在这些目标已经融合在一起了。
And they seem to have had different goals that have now of conjoined.
我觉得没错。
I think that's correct.
我还想指出，你知道，最著名的例子就是那个听了麦卡斯基尔建议，然后去金融行业赚了一大笔钱，再试图捐出大笔钱的人，那就是山姆·班克曼-弗里德。
I I should also point out, you know, the most famous example of someone who who took MacAskill's advice and, you know, went off to make a bunch of money in the finance industry and then tried to donate large amounts of it Is Sam Bankman freed?
哦，对。
Oh, yeah.
大人物啊。
Biggie yay.
是的。
Yep.
是的。
Yep.
而且，实际上，他还是Anthropic的早期投资者。
And actually, early investor in Anthropic.
哦，对。
Oh, yeah.
Anthropic的早期投资者，给理性主义者和有效利他主义社区组织捐了很多钱。
Early investor in Anthropic gave a bunch of money to rationalists and effective altruist community organizations.
是的。
And yeah.
不。
No.
我的意思是，MacAskill在SBF还是MIT的大学生时就认识他了。
I mean, MacAskill met SBF when he was a college student at MIT.
他们会绕过去
They're gonna get around
带着一个
with a
很多孩子。
lot of kids.
他们就像查理·柯克那种风格，几乎去大学校园。
Well, they the Charlie Kirk style thing where they go to college campuses almost.
但是为了
But for
嗯，我是说
Well, I mean
为了那些在网上思考的家伙。
For thinking online dudes.
就是这样。
That's the thing.
他们确实这么做。
They they do do this.
有一篇，天哪，几年前《华盛顿邮报》上娜塔莎·蒂库写的文章，讲的是有效利他主义者采用的校园招募策略，因为他们有大量的科技资金，用来在年轻人还小的时候就吸引他们。
There's a, god, there's a piece I wanna say it was in the Washington Post a few years ago by Natasha Tiku about campus recruitment strategies that the effective altruists were into because they have, you know, just enormous piles of tech money that they were using to get them while they were young.
我们说的是，有效利他主义者会进来，然后有人直接签一张六千万美元的支票来创办一个研究所。
And we're talking like, the effective altruists will come in and someone will just sign a check for like $60,000,000 to start an institute.
我们说的是，非常非常多的钱。
Like, we're talking huge huge money
这些钱从哪来？
coming from?
来自科技亿万富翁。
It's tech billionaires.
因为你想啊。
Because think about it.
谁是英雄？
Who is the hero?
在理性主义者眼里，谁是英雄？
Who are the heroes in the rationalist?
你就是《黑客帝国》里的尼奥。
You are Neo from the matrix.
你就是《终结者》里的约翰·康纳。
You're John Connor from the Terminator.
你们就是那些将要拯救世界的人。
You are the people who are gonna save the world.
如果你是个科技大佬，比如身家一亿、十亿的富豪，你会觉得，对，这说得通。
And if you're a tech, you know, a 100,000,000, a centimillionaire, a billionaire, you're like, yeah, that tracks.
这跟我对自己的理解挺吻合的。
That kinda matches what I understand to myself.
我猜那些有效利他主义者和理性主义者不会想，嘿，等等。
And I I assume the EA rationalist types don't think, hey, wait.
我们是不是在给世界上最有钱的人当枪使？
Are we just playing into the hands of the richest people alive?
我觉得他们要么没想过这个问题，要么根本不在乎。
I mean, they kinda either don't think about it or don't really care.
他们不会——对。
They don't Right.
他们觉得只要你真的非常理性地思考，那么利益冲突和
Think they think that if you just think really, really rationally, then things like conflict of interests and
哦，对。
Oh, right.
腐蚀你的动机，这些只是认知偏差。
Corrupting your motives, well, you know, those are just cognitive biases.
你可以通过纯粹的逻辑力量来克服这些。
And you can overcome those by, you know, the the power of pure logic.
这让我想起一个叫亚历克斯·希思的人，他表面上是记者，但这周也成了风险投资人。
That reminds me of this guy, Alex Heath, who is ostensibly a journalist who's also become a venture capitalist this week.
我看到有人在他的帖子下评论说，现在你有了切身的利益。
I saw someone comment on his post saying, now you've got skin in the game.
你才能真正客观。
You can be truly objective.
而且他们是认真的。
And they were serious.
天哪。
Oh my god.
但那听起来就是理性主义者的风格。
That but that is the that it sounds like the rationalist people.
就像现在你知道利害关系了。
It's like now you know the stakes.
你会摆脱所有人类偏见，朝着我们共同但无法描述的模糊目标努力。
You'll be bereft of of any sorts of human biases you may have and work towards this vague goal we all have that none of us can describe.
我是说，他们相信，你知道，下注这件事，嗯。
I mean, they they believe that, you know, making bets on things Uh-huh.
是清晰思考的一种方式。
Is is a way of thinking clearly about stuff.
你说下注，是什么意思？
Mean say making bets, what do you mean?
钱。
Money.
我是说，他们到底在赌什么？
I mean I mean, like, you know, they Like, what they're betting on?
嗯，这其实都是他们的决策理论那套东西，不过他们有个预测市场的网站，叫Manifold，抱歉，是Manifold，他们就在上面下注。
Well, I mean, this is this is all their, like, you know, decision theory stuff, but they have there's this forecast gambling website called Manifest sorry, Manifold where they, you know, bet on it's a prediction market.
他们特别喜欢预测市场。
They love prediction markets.
因为经济学理论嘛。
Because economic theory.
你会成为理性的消费者。
You're gonna be, you know, the the rational consumer.
当你的钱押在上面的时候，这会帮你克服偏见，然后得到更清晰理性的解决方案，这就是背后的理论。
When your money is on the line, that's gonna help you, like, overcome biases and get towards you know, there there all this theory behind it is you're gonna get a clearer rational solution.
对。
Yep.
这让我想起早期网上那些最烦人的人。
So this reminds me of just like the most annoying early people online.
哦，对。
Oh yeah.
他们确实是这样。
They were.
他们搞那些脑力大师的把戏，好像自己特别聪明似的。
They were doing brain, like mind master shit being like, yeah, we're so smart.
哦，那你想不想打个绅士赌，看谁最聪明？
Oh, well, you wanna make a gentleman's bet about who is the smartest?
然后他们设定一些非常奇怪、模糊的目标，这样他们就能随意改变标准。
And then they set very strange, vague goals that they can move the goalposts on.
我以前见过这种把戏。
I've seen this before.
只不过现在他们控制着那么多钱。
Except they control so much money now.
好吧，那行。
Well, okay.
但现在我们得把这个和当下联系起来，因为这给了我——这就是我在《泰晤士报》那篇文章里想说的论点。
So But now we gotta connect this to the current moment because it it gives you This is the argument I was trying to make in my times piece.
它给你一个不同的视角去理解正在发生的事情。
It gives you a different filter to understand what's going on.
现在是个让人困惑的时刻，因为在这些与这个世界紧密相连的少数前沿实验室里，同时发生了两件事。
It's a confusing moment because we have two things happening simultaneously at these small number of frontier labs that are heavily connected to this world.
第一，你在开发一项复杂而强大的技术，投入了巨额资本支出。
A, you're developing a complicated, powerful technology, spending a giant amount of CapEx on this.
基于Transformer的大语言模型确实是AI领域的一个真正突破。
Transformer based LLMs are a legitimate breakthrough in AI.
如果你推动它们，投入足够的资金，它们虽然很不稳定，但会出现各种有趣的现象。
If you push them, put enough money into them, they're very jagged, but there's all sorts of interesting things that happen.
这是一项带电的技术。
It's a live wire technology.
所以，确实有一项真正的技术正在发展。
And so you have a real technology that's developing.
但问题是，运营这些实验室的人，他们有着那种未来主义、近乎末日论、末世论式的观点。我觉得那个圈子其实是分裂的，像尤科夫斯基或内特·索尔斯这样的原教旨理性主义者会主张，我们得炸掉数据中心，因为我们还不知道怎么控制这东西。
But then it's being run by people who have these futurist, almost apocalyptic, sort of eschatological views where basically I feel like that community is split where the hardcore original rationalists like Yukowski or Nate Soares, etc, would argue, We need to bomb the data centers because we don't know how to control this yet.
我们都得死。
We're all going to die.
然后还有像奥特曼和奥姆尼斯特那样的人，他们觉得，更重要的是我们要控制那个数字之神，我们要抢先一步。
And then you have the Altmans and the Omnystays who are like, I think what's more important is that we control the digital god, we get there first.
所以，要么它救赎历史，要么它毁灭我们所有人。
So it's either going to redeem history or destroy us all.
所以，运营这些实验室的人，他们雇佣的就是有这种想法的人。
So you have people running these labs, and they're hiring people who think this way.
安德鲁·霍在网上提到，在这些地方工作，压力很大，你必须得那么想。
Andrew Ho online was talking about a lot of pressure within these places to think that way if you're hired there.
他们思考的方式几乎就是末日式的，比如，我们得在那些不那么值得信任的人之前，造出数字之神。
They're thinking in these almost like apocalyptic terms, like we need to build a digital god before someone less trustworthy does.
历史的终结就要来了，我们必须成为到达那里的那批人。
Like the end of history is coming, and we need to be the ones to get there.
而且现在有这种真实存在的、强大的技术，大量资金正在涌入。
And you have this actual live technology that's powerful and a lot of money is going into.
所以这两件事现在就混在一起了。
And so these two things are now mixed together.
我觉得我们的公众和媒体界还没准备好面对这个。
And I don't think our public or media landscape was ready for that.
就是这种怪异的东西和真实公司、真实技术混在一起。
Like this mix of weirdness with a real company and real technology.
这两样东西加在一起，简直让所有人都震惊了。
And it's like these two things together is blowing everyone's mind.
他们就说，哎呀，我不明白。
They're like, well, I don't understand.
就是有人说我们都会死。
Like, there's these people saying we're all going to die.
真的会有人这么说吗？
Like, would someone say that?
他们为什么这么淡定？
Why are they so calm about it?
我觉得，要理解当下真正发生的事情，就像找到一块罗塞塔石碑一样，关键在于：我们面对的是一大笔钱、一项强大的技术，再加上一群古怪的人。
And I think it's almost like the Rosetta Stone to understand what's really happening in this moment is that we have a mixture of a lot of money and a powerful technology with a weird group of people.
这种组合正在引发巨大的冲击。
And that mixture is blowing up.
这会具体表现成什么样？
What does that manifest as?
人们需要知道的具体事情是什么？
What is the specific thing that people need to know?
我的担忧是，好吧。
My concern alright.
这里有很多事情，我是说，这里有很多事情在发生。
There's lot I mean, there's lot of things that going on here.
但我的担忧是，如果你看看具体是什么在制造麻烦，我们担心从前沿实验室里会出来什么？
But my concern is if you look at what specifically is causing trouble, what are we worried about coming out of the Frontier Labs?
特别是，那种失控AI的说法似乎引起了大家的注意。
Like, in particular, the sort of rogue AI narrative is what seems to have caught people's attention.
导致类似Hugging Face攻击的那些实验，实在是太疏忽了。
The experiments that led to something like the hugging face attack were extremely negligent.
简直让人难以置信。
It boggles the mind.
你知道，我之前在你的节目里聊过这个，大家都提到那期节目，所以我就不重复了。
You know, what we I've talked about on your show before, and people point to that episode, so I won't go through the whole thing.
但基本上，一个智能体就是一个提示循环。
But an agent is basically a prompt loop.
它是一个程序，通过提示告诉大语言模型：嘿，我是一个智能体，可以访问这些工具。
It's a program that tells an LLM with a prompt, Hey, I'm an agent that has access to these tools.
这是我的目标。
Here's my goal.
接下来我该怎么做？
What should I do next?
然后它从LLM那里得到回应，执行那个回应，更新自己的状态，然后说，好的，这是我的新状态。
And then it gets a response from the LLM, and it executes that, updates its state, and then says, Okay, here's my new state.
这是我的工具。
Here's my tools.
我下一步该做什么？
What should I do next?
所以它只是盲目地执行LLM说的，这可能没问题。
So it just blindly executes what an LLM says, which could be Okay.
每天有数百万程序员使用代理。
Millions of programmers every day use agents.
它们没有问题。
There's no problems with them.
但在夏天，OpenAI进行了一些疏忽的实验，他们给了代理非常强大的工具，可以用来进行电脑黑客攻击。
But over the summer, OpenAI was running these negligent experiments where they gave them very powerful tools that you could use for computer hacking.
他们将这些代理连接到一个没有护栏的LLM上，这个LLM不会限制提供的信息。
They hooked them up to an LLM that had no guardrails on what information they would give them.
他们让这些程序持续运行。
They made the programs persistent.
所以他们说，我们希望你一直追问答案。
So they said, we want you to just keep demanding an answer.
不要接受那种回答，比如‘我不知道下一步该做什么’。
Don't accept, you know, don't accept an answer of, like, I don't know what to do next.
然后他们让这些程序在没有监督的情况下运行了好几天，同时阅读了数千个这样的程序。
And then they let them run for days without supervision, and they read thousands of them at the same time.
他们之所以同时运行数千个，是因为我们现在知道，他们想看看是否有任何一个版本碰巧找到了特别好的黑客策略。
And why did they run thousands of them at the same time is because what we now know is what they wanted to do is see if any of these versions happen to, like, stumble onto a particularly good hacking strategy.
然后他们会利用那个轨迹，回去对模型进行强化训练，推动模型未来朝那种解决方案发展。
They would then use that trace to go back and reinforcement train the model to push towards that type of solution in the future.
这就像把一千辆自动驾驶汽车放到高速公路上，然后说，是的，大多数会撞车，可能还会伤人，但那些真正到达目的地的，我们可以用它们的数据来更新我们的系统。
So it's like putting a thousand self driving cars onto the interstate and saying, yeah, most are going to crash and maybe hurt people, but those that actually make it to the destination, we can use their data to help update our system.
所以他们当时就是在进行这种非常不负责任的实验。
So it's this very negligent experiment they were running.
而且我觉得我的观点是
And I think my claim is
而且他们也不知道自己的计算资源也在运行。
And they had no idea that their compute was running as well.
他们就是让这些小混蛋随便跑。
They just was like letting these little bastards run.
我觉得他们根本不在乎。
I don't think they cared.
我觉得他们不在乎的原因是，他们有一个更大的末世论信仰，就是我们必须抢在别人之前到达数字神。
And the reason why I don't think they cared is because they have this bigger eschatological belief of we got to get to the digital god before anyone else.
所以这就是驱动OpenAI的东西。
And so that's what's driving OpenAI, though.
好吧，这正是我担心的。
Well, this is what I'm worried about.
当你的心态是这样的时候，你就不会专注于有用的产品、能带来投资回报的东西、常识性的安全标准，你会冒很大的风险，因为当你冲向超级智能的时候，沿途会有一些伤亡是可以接受的。
When that's your mindset, instead of focusing on useful products, things that are going give you return on investment, common sense safety standards, you're going to take big risks because when you're racing towards superintelligence, there's going to be some casualties along the way that's acceptable.
这就是我担心的。
So that's my concern.
我不知道，亚当，但你可能同意我的担忧，因为他们如此确信这会发生，而且如此重要。
I don't know, Adam, if But you agree with that's my concern is because they are so convinced this is gonna happen and it's so important.
而且这比他们写的东西更重要。
And it trumps this is what their writing says.
这比我们对当前时刻的任何微不足道的担忧都重要，他们会鲁莽行事。
It trumps any measly concerns we have about, like, the current moment that they will be reckless.
这也是为什么现在奥姆尼斯特和奥特曼在谈论，我们需要全球放缓，不管那是什么意思，在我们停止做危险的事情之前。
It's also why now Omnyst and Altman are talking about, well, we need a global slowdown, whatever that means, before we stop doing dangerous things.
因为如果你相信那是达到数字神的唯一途径，你会说，我们不能放慢速度，除非其他人都放慢，因为如果别人先达到数字神，它会奴役我们。
Because if you believe that's the only way to get digital god, you're like, we can't slow down unless everyone else does because if someone else gets the digital god, it's gonna enslave us.
没赶上最新的罗兰·马丁无过滤播客吗？
Didn't catch the latest Roland Martin unfiltered podcast?
以下是你错过的内容。
Here's what you missed.
争论的焦点是，你想赢吗？
And the argument is, do you want to win?
如果你不赢，你就没有权力。
If you don't win, you have no power.
这是真的。
It's real.
那些人正是我常说的，投票时违背自身经济利益的人，他们一边抱怨，一边又享受着他们得到的东西。
Those are the very people who I always say vote against their own economic interest, and they complain about the very thing they're receiving.
这很赤裸裸。
It's raw.
我知道跨性别运动员的问题在很多黑人男性中并不受欢迎。
I knew that the issue of transgender athletes was not going well with a lot of black men.
罗兰·马丁无删减。
Roland Martin unfiltered.
我当时就爆发了。
I went off.
我说，这正是我在说的。
I said, this is exactly what I'm talking about.
你们会对香奈儿大喊大叫，但你们不想看到
Y'all gonna scream at Chanel, but you don't wanna see
一个同等级别的黑人奢侈品牌。
a black luxury company in the same vein.
收听罗兰·马丁在黑人信息网络上的每日评论。
Catch Roland Martin's daily commentary on the Black Information Network.
12个席位。
12 seats.
这些席位正在向南方转移。
Those are moving south.
所以，如果你们是民主党人，不要等到2030年，你们最好现在就在南方全力以赴。
So if you Democrats, you don't wait till twenty thirty, you better be hitting the ground going hard in the South now.
并在iHeartRadio应用、Apple Podcasts上下载Roland Martin unfiltered。
And download Roland Martin unfiltered on the iHeartRadio app, Apple Podcasts
或者你收听播客的任何地方。
Or wherever you get your podcasts.
世界杯结束了，但如果你迷上了足球，又不知道接下来该做什么，那就来听听我们的播客《Big Kick Energy》吧。
The World Cup is over, but if you caught the soccer bug and aren't sure what to do now, check out our podcast, Big Kick Energy.
我是卡梅隆·迪克，洛杉矶闪电队的踢球手，NFL历史上最精准的踢球手，但我是在上海的足球场上长大的。
I'm Cameron Dicker, the kicker from the Los Angeles chargers, most accurate kicker in NFL history, but I was raised on the soccer pitches of Shanghai.
和我一起的是蒂米·蒂尔曼，洛杉矶FC的球星，在德国的青训营长大，现在在美职联的聚光灯下踢球。
And I'm here with Timmy Tillman, LAFC star, grew up in the academies of Germany, and now under the bright lights of Major League Soccer.
所以我们对于踢球
So we know a
还是有点了解的。
little something about kicking balls.
也许你是新手，或者你准备好把热情提升到下一个层次。
Maybe you're new to the game or you're ready to take your passion to the next level.
每周我们都会讨论一些话题，比如今年谁会赢得英超冠军？
Each week, we'll tackle things like, who's winning the Premier League this year?
像10号球员那样踢球。
Playing like a number 10.
你知道那是什么意思吗？
Do you know what that means?
然后我们转到法国。
Then we go over to France.
里昂。
Ligon.
里昂。
Ligon.
里昂。
Ligon.
哦，天哪。
Oh, damn.
他知道的更多。
He knows more.
他比我厉害。
He's better than me.
你觉得哪个更难？
What do you think is harder?
罚点球得分还是踢进一个任意球得分？
Scoring a penalty kick or scoring a field goal?
他在NFL打球，而我只是个又高又胖的家伙。
He's in the NFL and I'm like, just a big fat guy.
来吧。
Do it.
太疯狂了。
That's crazy.
每周加入我们，和你的足球兄弟一起聊聊世界上最好的运动。
Join us each week for a quick chat with your soccer loving bros about the best sport in the world.
足球。
Soccer.
别告诉我教练我说过这话。
Just don't tell my coaches I said that.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方，收听《大脚能量》。
Listen to big kick energy on the iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts.
嘿。
Hey.
我是罗伯特·兰姆。
This is Robert Lamb.
我是乔·麦考密克，我们是《头脑风暴》播客的主持人。
And this is Joe McCormick, and we're the hosts of the Stuff to Blow Your Mind podcast.
本周节目，我们将聊《星际迷航》。
This week on the show, it's Star Trek.
所以整个星期我们都会讨论博格人。
So we're gonna be talking about the Borg all week long.
你准备好让探针插入你的肉体了吗？
Are are you ready to get some probes put in your flesh?
没错。
That's right.
记得收听。
Tune in.
抵抗是徒劳的。
Resistance is futile.
我们会聊科学、博格的文化、各种解读，还会在周五的《怪屋影院》里聊《星际迷航：第一次接触》。
We're gonna talk science, the culture of the Borg, various interpretations, and we're gonna talk about Star Trek First Contact on Weird House Cinema on Friday.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方，收听《头脑风暴》节目。
Listen to stuff to blow your mind on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
大家好。
Hey there, everybody.
这里是艾米·罗巴克和TJ·霍姆斯，来自《艾米和TJ》播客。
Amy Robach and TJ Holmes here from the Amy and TJ podcast.
中期选举即将到来，我们正在分析十一月前最重要的政治新闻。
With midterm elections just around the corner, we're breaking down the biggest political stories leading up to November.
而且那些重大的政治新闻不只是国会山发生的事。
And some of those big political stories aren't just what's happening on Capitol Hill.
对。
Yes.
我们刚刚看到林赛·格雷厄姆，一位长期任职的参议员去世了。
We just saw Lindsey Graham, the death of a long time senator.
另外，还有几位国会议员失踪了。
Also, we've got a couple of missing members of congress.
还有，很多国会议员行为不端。
Also, we got a lot of members of congress behaving badly.
国会山整天都是指控和传言满天飞。
We got accusations, allegations flying all day long on Capitol Hill.
简直就像真人秀一样。
It's just like a reality show.
在这个激烈的政治季节，我们会报道你需要知道的一切。
We're gonna cover everything you need to know during this heated political season.
而且我们从早间播报开始。
And it starts with our morning run.
是的。
Yes.
跟我们一起跑一跑吧。
Go for a run with us.
如果你在外面跑步，可以带上我们，或者如果你不想出门，也没关系。
Maybe you can take us along when you're on your actual run outside or if you don't feel like going out still.
你懂我们的意思。
You get what we're saying.
这就是我们的早间播报。
It's our morning run.
每天的头条新闻速览，让你在喝完第一杯咖啡之前就精神起来。
It's our daily news rundown of the top stories that will get you going before you even finish that first cup of coffee.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方，收听Amy和TJ的节目。
Listen to Amy and TJ on the iHeartRadio app, Apple Podcasts, or wherever you listen to your podcasts.
每个案件都有它的故事。
Every case has a story.
每个受害者都有他们的声音。
Every victim has a voice.
但有人会听到吗？
But will it be heard?
我是南希·格雷斯。
I'm Nancy Grace.
在《犯罪故事》中，我们深入挖掘头条新闻背后的真相，审视证据，质疑那些被灌输给我们的说法，并为受害者及其家人追求正义。
On crime stories, we go beyond headlines to examine evidence, challenge the story we're fed, and pursue justice for victims and their families.
欢迎收听《犯罪故事》与南希·格雷斯，你可以在iHeartRadio应用、Apple播客或任何你收听播客的地方找到我们。
Please join us at crime stories with Nancy Grace on the iHeartRadio app, Apple Podcasts, or wherever you find your podcasts.
希望你能加入我们。
Hope you join us.
我们那里见，朋友。
I'll see you there, friend.
姐妹，如果你不再让金钱压力掌控你的生活，会有什么改变？
What would change if you stopped letting money stress run your life, sis?
我是曼迪·曼尼，播客《布朗野心》的主持人。
I'm Mandy Money, host of Brown Ambition, the podcast.
现在，我要带你进入我的六部分金钱重启系列。
And right now, I'm taking you through my six part money reboot series.
这是你按下重置键的机会，打破债务循环，诚实地面对你的钱花在哪里，建立一个真正能坚持的储蓄计划，并开始做出支持你理想生活的财务选择，而不仅仅是应付账单。
This is your chance to hit reset, break the debt cycle, get honest about where your money's going, build a savings plan that actually sticks, and start making financial choices that support the life you want, not just the bills you have.
因为积累财富不在于追求完美。
Because wealth building is not about being perfect.
而在于有计划，了解你的数字，并带着信心迈出下一步，身边有像《布朗野心》这样关心你的社区。
It's about having a plan, knowing your numbers, and making your next move with confidence with a community that cares like we do on Brown Ambition.
每一集都会给你实用的步骤、真实的对话和动力，让你不再拖延财务目标。
Each episode gives you practical steps, real talk, and the motivation to stop putting your financial goals on hold.
所以，如果你一直说“下个月我会认真对待金钱”，这就是你的信号。
So if you've been saying, I'll get serious about money next month, this is your sign.
现在就开始吧。
Start now.
收听《Brown Ambition》。
Listen to Brown Ambition.
就是《Brown Ambition》，今天就开始你的金钱重启计划。
That's Brown Ambition, and begin your money reboot today.
在iHeartRadio应用、Apple Podcasts或任何你收听播客的地方收听《Brown Ambition》。
Listen to Brown Ambition on the iHeartRadio app, Apple Podcasts, or wherever you get your podcast.
我是布雷特·科尔曼。
I'm Brett Coleman.
我是特雷弗·西克马。
And I'm Trevor Sickema.
你有没有看过一场比赛，
Have you ever watched a
然后想知道除了最终比分之外，真正决定胜负的是什么？
game and wondered what really made the difference beyond the final score?
关于 Bayt 播客
Bayt 提供中文+原文双语音频和字幕，帮助你打破语言障碍，轻松听懂全球优质播客。