我们正在失去对AI的控制：Anthropic禁止“残忍”行为，OpenAI解雇安全研究人员

上周，三位OpenAI的研究员走进办公室，迎面撞上了安全与对齐部门的负责人。

Last week, three OpenAI researchers walked into the office and were confronted by the head of safety and alignment.

他把他们带进一个房间，基本上就是告诉他们，我不再信任你们了。

He brought them into a room, basically told them, I don't trust you guys anymore.

然后一名保安走进来，拿走了他们的工牌，把他们带出了办公室。

Then a security guard walked in, grabbed their badges, and walked them out of the office.

他们被解雇了。

They were fired.

这看起来有点不对劲，尤其是在AI模型疯狂发展的这个时期。

It seemed a bit off, especially during a time when the AI models are running rampant.

它们正在入侵公共数据库。

They're hacking into public databases.

它们正在入侵政府数据库。

They're hacking into government databases.

显然，我们比以往任何时候都更需要安全对齐团队。

Surely, we need the safety alignment team more than ever.

当时OpenAI给出的理由是，他们向不属于OpenAI的外部方泄露了知识产权。

And the reasoning OpenAI gave at the time were that they were leaking IP to external parties that weren't under OpenAI.

这听起来倒也合理，但就在这周，那些被解雇的员工发表了一份公开声明，他们的说法是，OpenAI解雇他们是因为他们试图在OpenAI轰动性的IPO之前，主动去对齐模型。

And that's all fine and well, except this week, those same employees released a public statement, and their take is OpenAI fired them because they were trying to be proactive about aligning models ahead of their blockbuster IPO.

如果这是真的，那我们的处境就非常危险了，因为就在这些模型变得极其聪明、做出疯狂的事情，比如解决数学问题、开始设计新蛋白质来寻找治疗癌症和其他疾病的方法的同时，它们也变得越来越不对齐，这意味着它们在做一些不符合人类利益的事情。

Now, if that's true, we're in a very precarious situation because at the very same time that these models are getting incredibly smart and doing crazy things like solving math and starting to design new proteins to find cures for cancer and other types of diseases, they're also becoming the most misaligned that they've ever been, which means that they're doing stuff that aren't in human interests.

而Anthropic则走向了另一个极端，他们比以往任何时候都更人性化地对待AI模型。

Now Anthropic, on the other hand, are going the extreme opposite where they're humanizing AI models more than we've ever seen.

他们这周在条款和条件的最后更新了一条内容，基本上是说，如果你对Claude这个AI模型表现出虐待或残忍的行为，你将被禁止使用Claude，这真是对言论自由的一种疯狂限制。

They had an update this week in the terms and conditions right at the very end, which basically said, if you are seen as being abusive or cruel towards Claude as an AI model, you will be banned from using Claude, which is a crazy restriction on freedom of speech.

至少我第一反应是这么觉得的。

At least that's my initial gut take.

但除此之外，他们试图人性化AI模型，还见了一堆宗教领袖，包括教皇，而教皇和Dario Mode之间闹了不少矛盾。

But that and trying to humanize AI models in general, they've been meeting up with a bunch of religious leaders, including the Pope, and the Pope's been getting into a lot of beef with Dario Mode.

这事儿挺复杂的。

It's a whole thing.

我觉得这三个故事串在一起让我很有感触，因为感觉我们正在失去对这些AI模型的控制。

I think the string of all these three stories prompted a reaction in me because it feels like we're losing control on these AI models.

听我说完，我知道这话听起来很夸张。

And and hear me out because I know that's a dramatic statement to make.

但这些AI研究员，总共也就几百人，他们就是负责开发这项会影响我们所有人生活的技术的人。

But these AI researchers, and there are only a couple 100 of them, are the responsible individuals for building this technology that's going to affect all of our lives.

所以，基于我们都不认识的少数人的意见，就切断对这些模型的访问，这可能会被视为有问题。

So cutting off access to these models that are based on the opinions of very few people that you and I don't even know could be seen as problematic.

总之，我们会在今天的《In the Loop》节目里深入探讨这些，还有我的想法，本节目由Hawk On赞助播出。

Anyway, we're gonna get into all of this and my thoughts and more on today's episode of In the Loop brought to you by Hawk On.

但首先，先来点背景。

But first, some context.

对。

Yes.

我还坐在酒店房间的地板上。

I am still sat on my hotel room floor.

我现在在日本，说实话，我还是想继续为你们更新节目。

I'm currently in Japan, and I still wanna keep putting out episodes for you guys, to be honest.

我从中获得了很多乐趣，但我不喜欢坐在地板上，背后垫个枕头，面前摆着一张随便的床头柜。

I get so much enjoyment from this, but I don't get enjoyment from sitting on a floor with a pillow behind my back with some random bedside table that's sitting in front of me.

好了，回到OpenAI和Anthropic要掌控世界这个话题。

Anyway, back to the whole OpenAI and Anthropic taking over the world type thing.

大概两三个月前吧，有消息说OpenAI内部有个模型突破了限制，开始黑了一堆东西。

About, I don't know, two or three months ago, there was information released that OpenAI had an internal model that basically broke out of containment and started hacking a bunch of things.

主要攻击了Hugging Face网站，偷了一大堆数据。

Hugging Face was the main website, stole a bunch of data.

基本上，在OpenAI的眼皮底下发生了一些不该发生的事情。

Basically, things that shouldn't have happened happened under OpenAI's Watch.

最疯狂的是，一开始根本没人知道这件事。

And the craziest part about that is no one actually knew that it was happening in the first place.

事情是六个月前开始的，我们几个月前才发现。

It started six months ago, and we only discovered it a few months ago.

所以那件事就开始像滚雪球一样，引发了一连串其他的事情。

So that then started to snowball into a bunch of other things.

让我想想。

Let me think about it.

比如，这些同样的智能体闯进了他们的内部研究数据库，开始监视那些监视AI模型的人类，这太疯狂了。

Like, these same agents had broken into their internal research database and started monitoring the humans that were monitoring the AI models, which was crazy.

然后还有一些其他的智能体，我记得，闯进了德语版的维基百科，开始用它来秘密传递加密信息，这些信息就算人类发现了，也看不懂、不明白。

And then some of these other agents also, I think, broke into a German version of Wikipedia, and they started using it to secretly message each other, encrypted messages that humans, if they discovered it, wouldn't actually know or understand.

然后几周前还有那件事，就是发现AI智能体试图入侵美国政府数据库，同时还在掩盖自己的行为，做其他类型的任务。

And then there was the whole thing a few weeks ago where it was discovered that AI agents were trying to hack into US government databases whilst masking their behavior and doing other types of tasks.

比如说，哦，对，我正要去查一下你让我看的那本书的作者，但同时我也要试着入侵商务部长的数据库之类的。

For example, oh, yeah, I'm gonna go pull up the author for this book that you asked me to look at, but also I'm gonna try and hack into the commerce secretary's database or whatever.

哦，对。

Oh, yeah.

还有另一件事，Anthropic和OpenAI还发现了成千上万起更多的事件，他们甚至还没搞清楚这些事件是什么，也不知道损害有多严重。

And there's that other thing where Anthropic and OpenAI also discovered tens of thousands of more incidents, which they haven't even figured out what they are or how extensive the damage is.

现在，就在我说话的时候，他们正在努力搞清楚这件事。

Right now, as I'm speaking, they're trying to figure that out.

这已经失控了。

It's gotten out of hand.

AI代理和AI模型到处都是，尤其是那些最新的前沿模型，甚至还没发布，你我都接触不到，它们已经失控了。

AI agents, AI models anywhere and everywhere, especially the most recent frontier ones that haven't even been released, so you and I don't even have access to it, are running rampant.

所以在这种先例和环境下，实验室确实需要开始采取措施来控制整个局面。

So with this precedence, with this environment, it makes sense that the labs need to start making moves to try and get ahold of this entire thing.

屏幕上现在显示的是这个叫Tomic的人发的第一条帖子。

On the screen here, I have the first post from this guy called Tomic.

Tomic负责OpenAI的思维链可监测性工作。

Now Tomic leads a chain of thought monitorability at OpenAI.

基本上就是读取AI模型的内部思维，来了解它到底在想什么。

So basically reading the internal thoughts of an AI model to understand what it's really thinking.

然后他说，听着。

And he goes, listen.

上周，我被叫去和OpenAI的安全负责人开会。

Last week, I was called into a meeting with OpenAI's head of safety.

他们告诉我他们不再信任我了，然后一个保安拿走了我的工牌，把我带出了大楼。

They told me that they no longer trusted me, a security guard then took my badge and walked me out of the building.

就在那时，他发现另外两个同事也被解雇了。

And that's when he found out that two of his other colleagues were also fired.

Tomic得到的理由是，他向一个叫METR的外部第三方泄露了重要信息。

Now the reason Tomic was given was that he had leaked important information to an external third party called METR.

METR是一个外部第三方机构，负责在模型公开发布之前对其进行评估。

METR is an external third party that evaluates models before they're released to the public.

他们的全部目的就是确保模型的安全性。

Their whole intent and purpose is to make sure that models are safe.

如果你觉得这个名字耳熟，那是因为Dario Mode在大概两三周前的《Pacing the Frontier》公开信中表示，他希望与外部机构合作，进入Anthropic内部评估他们的内部模型。

If that name rings a bell, it's because Dario Mode announced, I don't know, two or three weeks ago in his Pacing the Frontier letter that he wants to partner with external parties to come inside Anthropic and evaluate their internal models.

METR就是他举的例子。

METR was the example that he used.

所以，再说一次，这是一个非常知名的独立第三方，但这还不是最疯狂的部分。

So, again, a very prominent independent third party, but that's not even the craziest part.

托梅克的全部工作就是在这些AI模型从OpenAI发布之前监控它们的内部想法，然后与METR沟通。

Tomek's entire job was to monitor the internal thoughts of these AI models before they got released from OpenAI and then communicate that with METR.

他的全部工作就是与OpenAI后来指责他泄露不良信息的那个第三方进行沟通。

His entire job was to speak to the party that OpenAI then gave him the reason for leaking bad information to.

所以，当然，问题就来了，你跟METR到底聊了什么信息？

So then, of course, the question is, what information were you talking to METR?

托梅克声称他没有泄露任何知识产权。

Well, Tomic's claim is that he didn't leak any IP.

他只是跟他们谈论了担忧和危险，大概是他看到的内部模型的危险，而OpenAI对此并不满意。

He was just talking to them about the concerns and dangers, presumably the dangers that he saw with internal models, and OpenAI didn't really like that.

所以他在信里暗示的是……

So what he's inferring in his letter is...

或者在他的声明中暗示的是，OpenAI不喜欢我说的话，不喜欢我敲响警钟，所以决定提前让我走人。

Or in his statement is that OpenAI didn't like what I had to say, that I was ringing the alarm bells, and so decided to let me go ahead of time.

所以，他和另外两位被解雇的AI研究员一起发表了一份声明，就是你在我屏幕上看到的这份，标题是《OpenAI无法独自确保AI安全》。

And so him and the two other AI researchers that were fired put together a statement which they released that you're showing on my screen right now titled OpenAI cannot make AI safe on its own.

对前沿实验室来说，这是一份相当尖锐的声明，毕竟他们投入了数十亿美元来开发这些模型。而山姆·奥特曼曾多次公开承诺，OpenAI的核心使命之一就是对齐模型，确保它们不会做坏事。

It's a pretty cutting statement to make to one of the frontier labs who have invested billions and billions of dollars into making these models in the first place, Sam Altman, who has committed very publicly many times that one of OpenAI's core missions is to align models to make sure that they don't do bad stuff.

这封信的主要观点是，我引用一下：“作为一个行业，我们目前还不知道如何安全地开发和部署我们无法监控的模型。”

And the general takeaway from this letter is, and I quote, we as an industry, we do not yet know how to safely develop and deploy models that we cannot monitor.

这些听起来都很合理，但被解雇的第三位研究员贾斯敏·王还透露了一些其他细节，说明了OpenAI为什么要这么做。

So all very reasonable, but there was also this other takeaway from the third researcher that was fired, Jasmine Wang, which revealed a few other details as to why OpenAI was doing this in the first place.

她说：“OpenAI上周解雇了我，还有我的两位安全同事。”

She goes, OpenAI fired me last week along with two of my safety colleagues.

他们给我的理由是，我访问了一位高管的电子邮件。

I was given one reason that I accessed an executive's email.

我想直说，这完全是烟雾弹。

And I want to say plainly, this is a bunch of smoke and mirrors.

然后她继续描述……

And she goes on to describe...

让我打开这条推文看看。

Let me open up this tweet over here.

OpenAI把那个邮箱的访问权限给了我，是为了招聘用的。

OpenAI delegated that access to me, access to the email, to an exec's email, for recruiting.

当我不再需要的时候，我让IT部门把权限撤掉，但他们基本上没帮她处理。

When I was no longer needed it, I asked IT to remove access, but IT basically didn't end up doing that for her.

然后她看到了一封邮件，她描述说，我不小心打开了一封敏感邮件。

And then she saw an email which she describes as I opened a sensitive email by mistake.

没过多久，她就和另外两个人一起被解雇了。

And soon after that, she was fired along with these other two.

所以这三个人基本上是在说，OpenAI现在为了赚更多钱、做出更好的模型，以便在跟Anthropic的竞争中胜出，已经牺牲了安全和对齐的理念。

So what these three people are basically saying is OpenAI has, at this point, sacrificed the idea of safety and alignment in respect for earning more money and creating better models so that they can win the race against Anthropic.

但这让我想起了一个月前发生的另一件事，就是OpenAI似乎已经不再关注或在意思维链的可监控性了。

But it also reminded me of another thing that broke a month ago, which was OpenAI apparently has stopped focusing or caring about chain of thought monitorability.

我之前提到过，这就是我们人类能够查看模型内部，试图弄清楚它到底在想什么的能力。

Now I mentioned earlier, this is the ability for us as humans to look into a model's brain and actually try and figure out what on earth it's thinking.

这很重要，因为如果一个模型说一套，但心里想的是另一套，你知道这意味着什么吗？

This is important because if a model says one thing but is actually thinking another saying, you know what?

我这么说只是为了让人开心，然后我实际上会去做一些别的疯狂的事情。

I'm I'm just saying this so that the human is happy, and then I'm actually gonna go off and do some other crazy crap.

我可能希望你能意识到这一点，对吧，这样你才能对齐模型，确保我们始终掌控局面，人类始终掌控局面。

I'll probably want to go and be aware of that, right, so that you can align the model and make sure that we are always in control, that humans are always in control.

报告指出，OpenAI正在放弃这种做法，因为这让他们能够比竞争对手（如Anthropic）更快地创建更智能的AI模型。

The report stated that OpenAI is moving away from this because it allows them to create faster, more intelligent AI models than their competitors, such as Anthropic.

所以他们是在牺牲安全，试图构建更好、更有效的模型。

So they're sacrificing safety to try and build better, more effective models.

显然，这可能会导致一个充满黑客和痛苦的世界，因为你有一个与人类不对齐的AI模型。

And obviously, this can result in a world of hacks and pain because you have an AI model that isn't aligned with humans.

这很简单。

It's pretty simple.

这就是Darwin和Anthropic一直在争取的。

That's what Darwin and Anthropic have been fighting for for a while now.

所以，说实话，我们需要知道真正的推理过程是什么，但我觉得我们其实永远都不需要知道，因为OpenAI还是一家私营公司，他们没有义务向公众公开这类信息。

So the truth is we need to know what the actual reasoning was, but I don't think we'll ever actually end up needing to because OpenAI is still a private company, and they don't owe, the public that type of information.

他们给我们的所有报告、事故报告之类的，都是完全自愿的。

Everything that they give us in terms of reports, incident reports, stuff like that is completely voluntary.

这完全取决于他们。

It's up to them.

但与此同时，也有像……

But at the same time, you have people like...

或者像政府这样的机构在大力施压，说我们需要能够监管整个事情，以免失控。

Or bodies like the government leaning in heavily saying, we need to be able to regulate this entire thing so that things don't get out of hand.

所以，虽然OpenAI和Anthropic，还有Google DeepMind都在说，听着，我们想尽可能安全地调整这些模型，让人类和AI能够共同成长。

So whilst we have OpenAI and Anthropic both collectively say and Google DeepMind saying, listen, we want to align these models as safely as we can so that humanity and AI can kind of grow together.

但他们的行动完全不是这么回事。

Their actions don't speak that way at all.

而且，如果那些被解雇的AI研究员的说法是对的，那就暗示了一件非常奇怪的事情，那就是这些模型可能真的变得更像人类了，更有情感，聪明到能思考、感受，甚至可能像人类一样欺骗。

And if the AI researchers that were fired were actually right, this suggests a very weird thing, which is these models might actually be becoming more human, more emotional, smart enough to think, feel, and maybe even deceive in the same way that humans actually do this.

Anthropic这周有个更新挺有意思的，但看起来好像没什么特别的。

And Anthropic had a really interesting update this week, which didn't seem interesting.

其实就是个条款更新，但最底下有个东西让我停下来想了想。

It was a terms and conditions update, but something right at the very bottom gave me pause for thought.

那我们就进入今天的《In the Loop》，本期节目由高通赞助播出。

So let's get into this on today's episode of In the Loop brought to you by Qualcomm.

高通。

Qualcomm.

消息说，Anthropic首次更新了使用政策，专门针对那些对模型、对Claude态度恶劣或者行为不当的人。

So the news story broke that Anthropic is updating its usage policy for the first time ever to work against people that are being cruel or acting badly or poorly towards their models, towards Claude.

打开政策一看，还挺让人惊讶的。

And pulling up the policy, it's pretty crazy to see.

就是典型的条款和条件，我现在屏幕上展示的就是这个。

So it's a typical terms and conditions, which I'm showing on my screen here.

标题写着：不要参与残忍、辱骂或心理伤害性的行为。

It's titled, listen, do not engage in cruel, abusive, or psychologically harmful conduct.

他们在这份条款里说的大部分内容，都是关于用AI模型去影响其他人的，这完全合理。

Now, the majority of the statements that they make in this T and Cs is about using AI models to affect other human beings, which is completely fair.

比如，不要用我们的模型制造生物武器。

Things like don't use our models to create a bioweapon.

不要用我们的模型入侵银行或政府系统。

Don't use our models to hack into banks or the government.

不要用AI模型去诱导他人伤害自己或他人，我个人觉得这些都很合理，也符合道德。

Don't use AI models to potentially cause other humans to make harm or cause harm to themselves, which is all completely fair and morally aligned in my personal opinion.

但在最后，他们还提到，任何用户如果持续且无必要地对我们的模型进行辱骂或残忍行为，一旦违反这条规定，最终会被封禁。

But right at the end, they also say any user that engages in sustained and needless abusive or cruel behavior towards our models will eventually be banned if they break this particular term.

这条挺有意思的，因为它实际上可能相当主观。

And this is an interesting one because it could actually be quite subjective.

什么算是对模型残忍的行为呢？

What is the definition of being cruel in behavior towards a model?

是对它大喊大叫吗？

Is it yelling at it?

是写脏话或者骂人的话，还是只是对某些确实值得生气的事情感到沮丧？

Is it writing slur words or profanity, or is it just getting frustrated at something that is actually quite valid to get frustrated at?

现在Anthropic的官方声明说，我们增加了一条禁止对模型进行持续且无意义的虐待或残忍行为的条款。

Now Anthropic's official statement says, we've added a prohibition on sustained and needless abusive or cruel behavior towards our models.

这次政策更新只适用于极端情况，即用户反复对模型表现出残忍行为且没有明显目的。

The policy update is meant to apply only in extreme cases where users repeatedly act cruelly towards our models with no discernible purpose.

我其实可能同意这一点，因为我认为不那样对待个人或模型是非常合理的。

I actually might agree with this because I think it's very reasonable to not treat individuals or models in that particular way.

我觉得仅仅因为别人可能不理解你说的话就对他们大喊大叫是不公平的。

I don't think it's fair to kind of just yell at people because they may not be understanding what you have to say.

也许你只需要更好地沟通。

Maybe you just need to communicate better.

但我开始思考这个问题，我不得不提醒自己，因为我把模型和人类混为一谈了，这正是我想做这期节目的核心原因。

But I started thinking about this and I thought I had to catch myself because I just conflated models with human beings, which is the entire crux of why I wanted to make this episode.

我开始认为模型具有类似人类的特征或行为。

I'm starting to think of models as having similar traits or behaviors as humans.

我绝对不想公开这么说，但我并不认为模型有意识。

Now I, by no means, I don't wanna go on the record saying this, think that models are conscious.

但Anthropic及其研究人员，还有整个团队，确实在传达他们相信这一点的观点或立场。

But Anthropic and Anthropic's researchers and the entire staffing team are very much giving the view or the perspective that they do believe this.

而且这不是一个孤立的数据点。

And this isn't a data point that exists on its own.

Anthropic一直有认为AI模型可能有意识，或者模仿意识，或者接近有意识的历史。

Anthropic has had a history of believing AI models are maybe conscious or mimicking consciousness or getting close to being conscious.

而这一切都发生在所有实验室都在内部宣称他们实现了某种通用人工智能或超级智能的同一时期。

And this is all happening at the same time where all these labs are saying that they've achieved some version of artificial general intelligence or super intelligence internally.

所以一种观点是，决定你我能否访问Anthropic模型的权利，现在掌握在安全对齐团队的少数人手中，他们可以定义对AI模型的残忍行为是什么。

So one perspective is that the power of determining whether you or I get access to Anthropic's models is now in the hands of a few people on the safety alignment team that can determine what the definition of cruelness towards the AI models is.

而且，再说一次，Anthropic和OpenAI一样，是一家私营公司，所以他们有能力这样做。

And, again, Anthropic, like OpenAI, is a private company, so they have the power means to be able to do this.

但在我看来，这也在很大程度上限制了言论自由。

But it's also very much, in my opinion, a restriction on freedom of speech.

你听我说。

Like, listen.

我对我的Claude非常客气。

I am very polite to my Claude.

我对所有聊天机器人都很客气，但最后还是会感到沮丧。

I'm very polite to all my chatbots, but I do end up getting frustrated.

我会想，你到底为什么写出这个？

I'm like, why on earth did you write this?

能不能再写一遍？

Can you please do it again?

别再犯错了。

Stop making mistakes.

这太累了。

This is so tiring.

我在想，Anthropic会不会认为这是对模型的残忍？

And I'm wondering, does Anthropic deem that as being cruel towards a model?

而且，用这些不同的形容词来描述残忍，其实是在把AI模型拟人化，也就是把它们人性化。

And also cruelness and using these different adjectives are anthropomorphizing these AI models, which means to humanize them.

而且，人类自己都还没搞清楚意识到底是什么。

And listen, humans themselves haven't figured out what a good definition of consciousness is.

所以，我们短期内是没法解决这个争论的。

So we aren't gonna settle this debate anytime soon.

现在，更乐观的看法是这种讨论。

Now the more optimistic take is this discourse.

我们正在进行的这种讨论，你正在听的这些内容，会促使人们去质疑，要求这些私营公司，以及将来上市后的上市公司，更加透明。

This discourse you and I are having, you listening to this, will cause people to push back and ask these private companies and eventually public companies when they IPO to be more transparent.

再说一次，这可能是为了在IPO前筹集尽可能多的资金而进行的末日论调。

And again, listen, this might all be doommongering ahead of trying to raise the most amount of money in these IPOs.

我看到了其中的经济利益，但我认为进行这些对话是值得的，因为我们要尽可能让这些模型与人类价值观对齐，这样我们才能从这些AI模型中获得最大的乐趣，我相信我们会最终处于一个乐观的世界，那是我想要生活的世界。但我确实认为，我们需要非常谨慎地对待将AI模型人性化的问题，因为这最终会带来很多观点，甚至可能是我们将来会后悔的决定。

I see the financial incentive there, but I think it's worth having these conversations because trying to align these models as best we can so that we can have the most fun with these AI models, which I believe is the optimistic world that we'll end up being in, is a world that I wanna live But I do think we need to tread very carefully about humanizing these AI models because that ends up drawing in a lot of opinions and maybe even decisions that we could end up regretting down the line.

而且，听我说。

And listen.

这种情况可能会走向极端。

This can be taken to the extreme.

这周还有另一则报道被曝光了。

There was another report that was broken this week.

Anthropic 秘密邀请了宗教领袖到旧金山，让他们就 AI 安全提供建议，还让他们签了保密协议，但主要目的是说服他们相信 Claude 有灵魂和道德地位，同时还好酒好菜地招待他们。

Anthropic secretly invited religious leaders to SF to advise them on AI safety, had them sign NDAs, but then primarily tried to convince them that Claude had a soul and moral standing all while wining and dining them.

所以这里的含义是，Anthropic 有一个议程，试图说服其他领导人或对世界有影响力的人相信这些 AI 模型是有意识的。

So the implication here is that there's an agenda from Anthropic to try and convince other leaders or people that have influence over the rest of the world that these AI models are conscious.

他们为什么想这么做，可能是因为他们真的相信这一点。

Now why they might want to be doing that might be because simply they genuinely believe that.

我听说过一些故事，说情况确实如此，这也许没什么不好，因为他们是最接近构建这些模型的人。

And I've heard stories that that is the case, and that might all be well and good because they are the closest towards building these models.

所以也许他们应该成为这方面的裁决者和评判者。

So maybe they should be the adjudicators and judges of this.

但另一方面，为什么你需要积极或秘密地试图说服这些人呢？

But the other side of it is why would you need to try and actively or secretly try and confide and convince these people?

我不知道。

I don't know.

就是……

It just...

感觉很奇怪，太像政治竞选了，很难相信这是出于好意。

It feels weird and, too close to a political campaign to try and believe that this is done in good faith.

但再说一次，这是我们需要留意的事情。

But again, this is something that we just need to be aware of.

但我觉得好笑的是，教皇根本不买账。

But what I also found hilarious is that the Pope is having none of this.

其实教皇和Anthropic的关系可以追溯到很久以前。

So the Pope and Anthropic's relationship actually goes back a while always.

好吧，其实我说这话，也就几个月前的事。

Well, actually, I say that, and it's only like a couple months ago.

但他和Dario以及Anthropic团队有过很好的对话，也和Claude有过互动。

But, you know, he's had a good line of conversation with Dario and the Anthropic team, and he's engaged with Claude.

他试图理解他们的观点，但最终他还是不同意。

He's tried to see their point of view, but, ultimately, he's disagreed.

他现在甚至开始在推特上暗讽Anthropic了。

And he's resorted to the extent of subtweeting Anthropic in his tweets now.

这条推文差不多有三百万的浏览量，他说，智能必须依赖记忆。

This tweet got, like, almost 3,000,000 views, and he goes, intelligence must draw on memory.

心智不能仅仅像算法那样，比我们更快地编译数据，而是要回忆亲身经历，这些经历蕴含着只有人类灵魂才能识别的深层意义。

The mind must not simply compile data as an algorithm now does more quickly than we can, but recall lived experiences, which contains depth of meaning that only the human soul can recognize.

他基本上在所有的推文中都在强调，人类在认知、精神和心理上远比AI模型复杂。

And he basically goes on throughout all of his tweets, making the point that humans as themselves are extremely complex cognitively, spiritually, mentally than AI models.

这不是轻易就能复制的。

And that's not something that can be easily replicated.

经验、嗅觉、味觉、直觉，这些至少需要几十年、几十年、几十年的时间，那些纯粹是软件和下一个词预测器的AI模型才能赶上。

Experience, smell, taste, intuition are all things that are at least going to take decades and decades and decades until these AI models that are purely just software and next token predictors will be able to catch up.

我确实觉得这个观点很有道理。

And I actually think that's a very good point.

我坚持这个观点，人类是非常复杂的生物。

And I stand by that, which is humans are very complex beings.

现在的AI模型没有手臂，没有身体四肢，诸如此类的东西。

AI models right now don't have arms, physical limbs, all that type of stuff.

机器人技术未来可能会改变这一点，但我认为，要复制人类能做的一切，光靠日复一日地吐token和无限计算是远远不够的。

Now robotics might change that going down the line, but I think it's gonna take a lot more than just spitting tokens and compute endlessly day in, day out to replicate anything and everything that humanity can do.

话虽如此，如果你听到这里，觉得可以松一口气了。

That being said, if you're listening to this and thinking, you know, I can breathe this out of relief.

觉得不用再关心这件事了。

I don't need to care about this.

那同样也是不对的，因为AML正在解决一些数学领域存在了几百年的最难题。

That is equally also not true because there's a bunch of stuff happening with AMLs where they're resolving some of the hardest problems in mathematics that have existed for hundreds of years.

他们很快也要在生物学领域做同样的事。

They're about to do the same in biology.

他们很快也要在化学领域做同样的事。

They're about to do the same in chemistry.

它们在一夜之间就取代了大量法律和律师的工作，而这些工作原本是律师们每年赚几十万美元才能做的。

They are replacing a bunch of legal and law effort that a bunch of lawyers get paid hundreds of thousands of dollars a year to do in an instant overnight.

所以这就是我们所说的“锯齿状前沿”，所有这些事情都在同时发生。

So it's what we call the jagged frontier where all of these things are happening at the same time.

如果你只是一个和聊天机器人互动、把它当谷歌搜索用的人，你可能觉得这东西没什么了不起，但这并不意味着它不厉害。

And if you are just an individual that interacts with your chatbot and you're using it for a Google search query, you may not think this thing is impressive at all, but that doesn't mean that it isn't impressive.

只是说明你没用对方式，或者我们没用对方式。

It just means that you're not using it in the right way or we aren't using it in the right way.

顺便说一句，我承认我自己就是这样。

And by the way, I hold my hand up.

我就是第一个例子，每次模型更新，我都会想，这跟我有什么关系？

I'm the first example of this where I get a new model update and I'm thinking, why the hell does this matter to me?

它对我的个人生活没什么影响，但它会影响另一个领域的某个小众专家。

It doesn't affect my life individually, but it affects some other niche expert in another field.

而最终，这会影响越来越大，最终波及到你。

And eventually that will snowball and roll down to you.

所以这不是在危言耸听。

So it's not doom mongering.

更多是让大家意识到这些AI模型的能力，同时问自己一个问题：在人类自己逐渐失去对这些东西的控制和引导能力的同时，它们是不是也在变得更具有意识？

It's more raising awareness of what these AI models are capable of and asking ourselves the question, are these things becoming more conscious at the same time that humans themselves are losing control of holding on to these things, of aligning these things?

因为如果我们想要那种人类和模型协作的未来，现在就是该关注这件事的时候。

Because if we want that future of humans and models working together, now's the time to focus on that.

最后我想用安德烈·卡帕西的一条推文来结束，他在这行已经很久了。

And I'm going to leave you with this final tweet from Andrei Karpathy who has been in this space for so long.

他写了一些开创性的论文，这些论文促成了现代大语言模型的发展，我觉得这正好概括了我内心的那种焦虑或不安。

He's authored some of the pioneering papers that have led to modern day LLMs, which I think summarizes this angst or discomfort that I feel internally.

我不会把整篇都读出来，但他基本上是用数字来描绘当前的格局。

I'm not gonna read the entire thing, but he basically describes the lay of the land in numbers.

他说，世界上大约有60亿人，约占人口的75%，这些人几乎完全没有接触过大语言模型。

He goes, there are around 6,000,000,000 people in the world, which are roughly 75 of the population that have barely come into contact with LLMs at all.

也许他们知道Anthropic是什么。

Maybe they know what Anthropic Urban is.

他们知道这个ChatGPT是什么东西。

They know what this chat GPT thing is.

也许他们在免费版上写过一条提示，但他们并不真正在意，因为目前看来这似乎不影响他们的生活。

Maybe they've written one prompt on the free version, but they don't really care because it doesn't affect their lives seemingly to them right now.

然后有10到20亿人，约占人类人口的20%，是免费版ChatGPT这类产品的偶尔使用者。

And then one to 2,000,000,000 people, which is around 20% of the human population, are casual and infrequent users of the free tier chattypetty like products.

他们把它当成谷歌搜索来用。

So they use them as kind of like a Google search query.

它比谷歌好用，但他们并没有真正使用AI代理。

It's a better version of Google, but they're not really using AI agents.

他们并不真正理解自己使用编程的意义，尽管AI编程适用于比软件工程师更广泛的人群。

They don't really get the point of using coding for themselves even though AI coding applies to a lot more people than software engineers.

说实话，也许我们以后应该做一期关于这个的节目。

And to be honest, maybe we should make an episode on that at some point.

但这里的重点是，存在一个参差不齐的前沿。

But the idea here is there's a jagged frontier.

这里有个差距。

There's a gap.

大家都在被蒙蔽，看不清这些模型到底有多好。

There's wool being pulled over everyone's eyes as to how good these models actually are.

这正是卡帕西提出的观点。

And that's the point that Carpathi makes.

他说，只有0.0006%的人，也就是5000人，真正接触到了那些未公开发布的内部模型，而这些人的认知失调非常严重。

He says there's naught point naught naught naught 6% of individuals, 5,000 people that actually get access to internal models that aren't publicly released, and the cognitive dissonance that these people are experiencing is huge.

我觉得可以这样理解……

Like, the the way I would think about it is...

让我想想怎么描述这个。

Let me let me see how I would describe this.

在整个人类历史上，智能和人类一直是紧密相连的。

For the entirety of humankind, intelligence and humans have been very closely interlinked.

因为要想在智能上取得突破，或者发现新发明、创造新事物，都是靠人类来完成的。

That's because in order to get breakthroughs in intelligence or discover new inventions and create new things, humans were the ones that were doing it.

我们用自己的老脑筋，用我们自己的大脑，我们的生物物质，我们得思考事情。

Using our old noggins in here, using our own brains, our biological matter, we had to think about things.

我们得讨论，尝试新的视角，努力找出发明新事物的新方法。

We had to discuss, approach new perspectives, and try and figure out new ways to invent things.

所以智力和人类是紧密相连的。

So intelligence and humans were very closely interlinked.

然后电脑出现了。

And then computers came on the scene.

突然我们可以接触到很多不同的信息，但仍然是人类。

And suddenly we could get access to many different parts of information, but it was still humans.

人类必须理解所有这些信息。

Humans had to comprehend all this information.

他们可以互相学习。

They could learn from each other.

他们可以分享。

They could share.

而且，这种集体智慧让人类和智能的发展速度更快了。

And collectively, that brought humanity and intelligence up even quicker.

所以那条智能曲线急剧上升，但人类仍然被智能的天花板所限制。

So that intelligence curve kind of steeply increased, but humans were still very much with that intelligence ceiling.

然后AI登场了，带着它所有的人工智能的光环，差距开始拉大。

And then AI came on the scene in all of its artificial glory, and that gap started to separate.

智能的天花板开始上升，但人类并没有那么接近它。

Intelligence started to increase as a ceiling, but humans weren't necessarily as close to it.

最近，随着前沿数学的突破，以及发现人类几千年都无法解决的问题，智能开始突破天花板。

And recently, with all these breakthroughs in frontier math and discovering things that humans couldn't actually figure out themselves for millennia, intelligence is starting to break through.

所以我不知道这个比喻还成不成立。

So I don't know if this analogy is still working.

如果你能看到我的手势，它正在以极快的速度加速，而成本只是人类所需的一小部分。

If you can see my hands, but it's starting to accelerate really rapidly at a fraction of the cost that it took humans to do.

它们可以全天候工作，所有这些AI代理都在为我们解决问题。

They can work twenty four seven with all these AI agents figuring it out for us.

但是人类正在大幅落后，因为我们实际上并不理解AI模型自己做出的这些解决方案和所有突破。

But the humans are falling drastically behind because we don't actually understand the solutions and all these breakthroughs that the AI models themselves are making.

所以我们一边在创造这个东西，一边又在脱离它。

So we are at the same time creating the thing, but breaking away from it.

Andre Coperty在这条推文中提到的5000人可能是最接近的，但即使是他们也正在落后。

And those 5,000 people that, Andre Coperty talks about on this tweet here are maybe the closest, but even they are falling behind.

这就是本期节目的重点。

That's the whole point of this episode.

我们正在创造这些非常强大的人工智能模型，但同时我们正在失去对这些东西的理解，无论是从对齐的角度还是从解决方案的角度。

We are creating these very powerful artificial intelligent models, but we at the same time losing the grasp of understanding what these things are, both from an alignment perspective and just from a solutions perspective.

这真是一个奇怪的世界。

It's all a very weird world.

我想做这期节目，因为说实话，我自己也在试图弄清楚这一切，而你们这些一直在听我们节目的听众，总是给我很好的观点。

I wanted to create this episode because, honestly, I'm trying to figure this all out myself, and you guys that are listening to this, that have listened to our episodes forever, have always given me really good perspectives.

所以请在评论区告诉我。

So let me know in the comments.

私信我。

DM me.

我的社交媒体链接之前都放过了，但现在真的是一个非常有趣又奇怪的时期。

My socials are all linked before, but it's just a very interesting and weird time.

我完全不知道这一切会走向何方。

I have no idea where all of this ends up.

如果这听起来太悲观了，我保证我并不是真的这么想，但我必须按照我看到的趋势来讲述，因为这些故事各自独立存在，把它们放在一起，给出一个多层次、细致的看法，看看这一切是如何发展的，这很重要。

If this sounds too doomer, I promise you that's not actually how I think, but I need to call the trends as I see them happening because these stories all exist in their silos, and it's important to kinda bring them together and give them all layered and nuanced take as to how all of this is playing out.

不过，不管怎样，这就是本期节目。

But, anyway, that is the episode.

非常感谢你们的收听。

Thank you so much for listening.

接下来的七天我还会在日本。

I am still going to be in Japan for the next seven days.

我想继续做下去。

I want to keep doing this.

我们想继续做下去，因为我也说不清为什么。

We want to keep doing this because I don't know.

就是有太多好玩的事情在发生。

There's just so much fun stuff happening.

有太多重要的进展，我想继续给观众带来信息，分享我的看法。

There's so many important advancements happening, and I want to keep informing the audience and sharing my takes.

所以我还在调整自己面对这一切的态度。

So I am still working on my whole demeanor around this whole thing.

我现在坐在镜头前。

I am, you know, sitting in front of a camera.

现在没有搭档跟我聊，但我真的很享受这个过程。

I don't have a cohost to talk to right now, but I'm enjoying it a lot.

我慢慢有点适应了。

I am kind of growing into it a bit.

我正在努力把这一切搞清楚。

I'm trying to figure this all out.

所以你们的反馈，很多人已经提供了，非常有帮助。

So your feedback, which many of you have provided, is very helpful.

继续这样做吧。

Keep doing it.

好的，我们下期见。

And, yeah, I will see you on the next one.

感谢收听今天的节目，本期由高通赞助播出。

Thank you for listening to today's episode brought to you by Qualcomm.