人工智能的伦理影响探析

这条信息来自MS Now。

This message comes from MS Now.

在他们的新播客中，MS Now推出《Clock It》。

On their new podcast, MS Now presents clock it.

华盛顿的权力人物西蒙妮·桑德斯·托wnsend和尤金·丹尼尔斯讨论最新政治新闻与最热门的文化时刻如何交汇。

Washington power players, Simone Sanders Townsend and Eugene Daniels discuss how the latest political news and the catchiest cultural moments converge.

立即在您收听播客的平台收听。

Listen now wherever you get your podcasts.

这是《Fresh Air》。

This is Fresh Air.

我是塔尼娅·莫斯利。

I'm Tanya Moseley.

本周，五角大楼正在考虑切断与人工智能公司Anthropic的商业联系，因为该公司拒绝允许其聊天机器人Claude用于某些军事应用，包括武器开发。

This week, the Pentagon is considering cutting business ties with the artificial intelligence company Anthropic after the company declined to allow its chatbot, Claude, to be used for certain military applications, including weapons development.

同时，《华尔街日报》报道称，Claude曾被用于一项导致委内瑞拉领导人尼古拉斯·马杜罗被捕的美国行动，但Anthropic尚未确认这些说法，并拒绝公开讨论。

At the same time, The Wall Street Journal reports that clot was used in a US operation that led to the capture of Venezuelan leader Nicolas Maduro, claims anthropic has not confirmed and has declined to discuss publicly.

与此同时，在军事和情报圈之外，同样的工具正被用于远不那么戏剧性但仍具重大影响的用途。

Meanwhile, outside military and intelligence circles, the same tool is being used for far less dramatic but still consequential purposes.

据报道，一名纽约男子使用Claude挑战了一笔近20万美元的医院账单，并成功协商减免了大部分费用。

A man in New York reportedly used Claude to challenge a nearly $200,000 hospital bill and negotiated most of it away.

一位南非言情小说家表示，她利用该工具在一年内出版了超过200部小说。

A romance novelist in South Africa has said she used it to help publish more than 200 novels in a single year.

那么，这个系统究竟具备哪些能力？而开发它的人员又对自身创造的东西了解多少呢？

So what exactly is this system capable of, and how well do the people building it understand what they've created?

今天我的嘉宾是记者吉迪恩·刘易斯·克劳兹，他花了数月时间深入Anthropic公司，试图回答这个问题。

My guest today, journalist Gideon Lewis Krausz, spent months inside Anthropic trying to answer that question.

该公司是全球最具影响力的AI公司之一，估值约为3500亿美元，同时也是最神秘的公司之一。

The company is one of the most powerful AI firms in the world valued at about $350,000,000,000 and also one of the most secretive.

它由前OpenAI员工创立，这些人正是ChatGPT背后的团队，他们离开的原因是认为开发先进人工智能的竞争步伐太快，可能带来危险。

It was founded by former OpenAI employees, the team behind ChatGPT, who left because they believe the race to build advanced artificial intelligence was moving too fast and could become dangerous.

吉迪恩·刘易斯·克劳兹是《纽约客》的专职作家。

Gideon Lewis Krauss is a staff writer at the New Yorker.

他的文章标题是《什么是Claude？》

His piece is called what is Claude?

Anthropic公司也不知道。

Anthropic doesn't know either.

我们的采访是昨天录制的。

Our interview was recorded yesterday.

吉迪恩，欢迎来到《新鲜空气》。

And, Gideon, welcome to Fresh Air.

非常感谢你邀请我，塔尼娅。

Thank you so much for having me, Tanya.

我们先从最新消息开始聊起。

Let's get started by talking about the latest news.

我们上周得知，军方可能在抓捕委内瑞拉独裁者尼古拉斯·马杜罗的行动中使用了Anthropic的工具Claude。

We learned last week that the military may have used Anthropics tool, Claude, during the operation that captured Venezuelan dictator Nicolas Maduro.

据报道，他们用它来处理情报、分析卫星图像以及其他类似工作，以支持实时决策。

And reportedly, they used it to process intelligence and analyze satellite imagery and and things like that to support real time decision making.

Anthropic的使用准则是什么？

What is Anthropic's usage guidelines?

他们对将Claude用于暴力或监控有何说法？

What do they say about its use for violence or surveillance?

他们的合同规定，不得将该工具用于国内监控或自主武器系统。

Well, their contracts with other companies and with the government stipulate that it can't be used for domestic surveillance or for autonomous weaponry.

当然，这类系统的问题在于，一旦交到他人手中，就很难预测或控制他们如何使用它。

Now, of course, the issue with these systems is that once you put it into someone's hands, it's very hard to predict or control how they're going to use it.

因此，根据我们从《华尔街日报》及其他渠道看到的报道，Anthropic似乎也被此事打了个措手不及，他们似乎没有制定出明确的回应，甚至可能根本不知道Claude已被用于马杜罗事件中。

So it seems to me from the reporting we've seen from the Wall Street Journal and elsewhere that Anthropic may have also been caught by surprise with this, that, they didn't seem to have a formulated response, and they seemed as though they perhaps hadn't even known that this had been used in the Maduro

《华尔街日报》还报道称，Claude是通过Anthropic与数据公司Palantir Technologies的合作部署的，而你曾对这家公司做过大量报道。

The Wall Street Journal, is also reporting that clot was deployed through Anthropics partnership with the data firm Palantir Technologies, which you have done quite a bit of reporting on.

我们知道，Palantir与五角大楼有广泛的合作。

And, we know that Palantir works extensively with the Pentagon.

关于他们之间的关系，你能告诉我们些什么？

What can you tell us about their relationship?

关于这种合作关系的报道并不多。

There has not been a lot of reporting about that relationship.

在过去几年里，Anthropic 决定采取企业级商业策略。

Anthropic has decided over the last couple years that they were gonna pursue an enterprise business strategy.

因此，他们与许多不同公司合作，并且有理由期望这些公司遵守他们签订的协议条款。

So they work with a lot of different companies, and presumably, they expect these companies to follow the terms of the agreement that they have.

但除此之外，这些公司如何使用他们开发的系统，就超出了他们的控制范围。

But beyond that, it's sort of out of their hands how these companies are using these systems that they've developed.

你的文章清晰地揭示了 Anthropic 安全使命与所面临的商业压力之间的张力。

Your piece really lays out the tension between Anthropic safety mission and the commercial pressure that it faces.

我想知道，这种张力的版本，你之前真的预料到了吗？

And I I guess I just wonder, is this a version of that tension that you actually even expected?

基本上，就是与五角大楼的对峙。

Basically, a standoff with the Pentagon.

我认为，早在大约一年前就很明显了，特朗普政府的许多成员，包括特朗普的AI事务主管、风险投资人大卫·萨克斯，以及最近的佩特·赫格塞斯，都对 Anthropic 是否愿意让政府按其意愿使用这些模型表示过疑虑。

Well, I think it was clear, probably even about a year ago that there were gonna be some tensions, that many of the members of the Trump administration, including, Trump's AI czar, David Sacks, the venture capitalist, and Pete Hegseth more recently had expressed reservations about Anthropic's willingness to allow the government to use the models the way that the government saw fit.

达里奥·阿马德，Anthropic的首席执行官，应对这些相互竞争的压力的一种方式是：既要安全、负责任地开发这些系统，又要在一个极其激烈的市场中竞争——正如他所说的‘登顶竞赛’，即他希望如果他们能证明自己的系统比其他系统更安全、更负责任，市场自律就会发挥作用，迫使竞争对手也提升标准。

And one of the ways that Dario Amade, the CEO of Anthropic, has dealt with the these competing pressures, both the pressure to develop these systems safely and responsibly and also to compete in a very aggressive marketplace as he talks about the race to the top, meaning that he hopes that if they can show that their systems are safer and more responsible than other systems, that there will be market discipline that will be, enforced and will force their competitors to rise to the occasion.

但问题是，我不确定他是否预见到，如果政府和国防部是他们的客户，我们的政府实际上并没有表现出参与‘登顶竞赛’的倾向，恰恰相反。

Now the problem is, I'm not sure he anticipated the fact that if the government and the defense department are among their customers, that our government has not shown great tendencies to participate in races to the top, rather to the contrary.

让我们谈谈你的报道。

Let's get into your reporting.

你曾进入Anthropic位于旧金山的总部。

You went inside of Anthropics headquarters in San Francisco.

你第一次踏进那扇门时，第一印象是什么？

What was your first impression walking through that door?

我的第一印象是，这家公司几乎没有个性。我曾长期在谷歌这样的地方工作，而至少在谷歌的某些早期阶段，它看起来就像一个成人日托中心——有桌游、攀岩墙、糖果和专门的午睡室。

My first impression is that there's there's really not a lot of personality at the company that, you know, I've spent a lot of time at places like Google over the years and, you know, at least in in certain earlier iterations, Google could kind of look like adult daycare with, board games set out and climbing walls and candy and special nap rooms.

Anthropic完全没有任何这些东西，而我认为这些对它们来说都是干扰。

Anthropic really has none of that stuff, all of which I think would seem like a distraction to them.

正如我在文章中所说，Anthropic散发出的气质就像一家瑞士银行。

Anthropic, you know, as as I as I said in the piece, of radiates the personality of a of of a Swiss bank.

没什么可看的。

There's not much to look at.

他们大约十八个月前接手了聊天公司Slack的整租办公室，似乎把所有有趣的东西都清除了。

They took over a turnkey lease from the messaging company Slack about eighteen months ago, and seems like they removed anything interesting to look at.

因此，从公司内部来看，几乎没什么可描述的。

So there's very little to describe from the inside of the company.

我一到那里就被直接带到了他们允许外部访客进入的两层楼之一，并且有非常友善、温和但坚定的公关人员全程陪同。

And I was kind of whisked right away to one of the two floors where they allow outside visitors and, had very gracious and gentle and firm PR minders for my time while I was there.

Anthropic的创立背景，结合其与政府和军方关系的最新发展来看，非常有趣，因为最初，他们是一群试图抵制腐败权力的人。

The founding of Anthropic, the story behind it is really interesting in light of the latest developments with its relationship with the government and the military because, initially, they were people who set out to resist corrupting power.

他们由两位兄弟姐妹于2021年创立，离开OpenAI是因为他们认为萨姆·阿尔特曼尤其将商业主导权置于安全之上。

They were founded in 2021 by two siblings who left OpenAI because they felt that Sam Altman, in particular, was prioritizing commercial dominance over safety.

你能简要分享一下他们的理念和Anthropic的宗旨吗？

Can you briefly share their ethos, Anthropic's purpose?

这并不是第一次有一群人认为另一群人不配掌控可能成为人类历史上最强大技术的开发工作，如果这项技术最终实现的话。

Well, this was not the first time that one group of people decided that another group of people was not to be entrusted with the development of what will potentially be the most powerful technology ever developed if it comes to fruition.

OpenAI最初成立的故事也是因为埃隆·马斯克和萨姆·阿尔特曼不信任深脑的德米斯·哈萨比斯和谷歌会负责任地推进这项技术。

The original story of the founding of OpenAI also was that Elon Musk and Sam Altman didn't trust, Demises Habes at DeepMind and Google to be pursuing this responsibly.

关于这项技术的发展，它触及了人们众多不同的动机，其中很大一部分是科学好奇心在推动着它的进步。

And one of the things about the development of this technology is that it touches on so many different motivations in people, that a lot of it is scientific curiosity is what's driving the development of this.

OpenAI最初能够从谷歌等机构吸引人才，是因为他们表示：我们会为了全人类的利益来开发这项技术，将以无畏的科学精神来做这件事，同时保持谨慎和负责。

And that OpenAI was originally in a position to recruit talent from places like Google because they said, you know, we are going to develop this for the benefit of humanity at large, and we are gonna do this with an intrepid scientific spirit, and we're gonna be careful, and we're gonna be responsible.

但问题在于，这就像一个闪亮的物体，为开发者提供了潜在的巨大权力。

But then the problem is that this is kind of a glittering object that offers, potentially great power to the people who develop it.

因此，那七位离开OpenAI的人认为，OpenAI最初在阐述其使命时就不够真诚，或者在实践中出现了使命偏移。

And so the seven people who defected from OpenAI felt as though OpenAI had either been disingenuous in the first place with the articulation of their mission or had, allowed for some mission drift in what they were doing.

他们觉得，现在我们真的不能再信任萨姆·阿尔特曼了，所以我们必须安全地来做这件事。

And they thought, you know, now now we really can't trust Sam Altman to be doing this, so we need to be doing it safely.

你在公司内部有没有感受到任何冲突？人们是否在挣扎于自己所构建的东西以及最终谁会使用它？

Were you picking up any kind of conflict when you were in the building, people wrestling with what they're building and who ends up using it?

因为我觉得很有趣的是，他们总是带着利他的理念在不同公司之间辗转，一心想要创造对人类有益的东西，但最终总是陷入彼此不信任的境地。

Because I think it's interesting how they've gone from company to company with these altruistic ideas and thoughts about really creating something that's good for humanity, and it always kind of ends up where everyone's not trusting each other.

我的感觉是，Anthropic的每个人都彼此信任。

Well, I mean, I get the feeling that at Anthropic, everybody really does trust each other.

那里给人一种目标高度一致的感觉。

It feels like a very mission aligned place.

至少我接触过的那些人，似乎都对这些问题有着极高的品德和操守。

And, you know, at least the people that I talked to seem to be people of great property and integrity about these things.

所以，公司内部并没有那么大的冲突。

So it wasn't so much that there was conflict within the company.

真正的担忧在于：当你的竞争对手并不秉持相同的价值观时，你该如何在市场中竞争？

The fears are how do you compete in a marketplace where your competitors might not be driven by the same values?

我认为我可以概括地说，Anthropic几乎每个人都觉得，他们自己和整个行业都推进得太快了，如果能有一种解决这种集体行动困境的方法，让所有人都慢下来，那就太好了。

And I think I can generalize and say that almost everyone at Anthropic had the feeling that they were moving too quickly and the entire industry was moving too quickly and that it would be nice if there were some solution to this collective action problem that would allow everyone to slow down.

但人们对这个问题有着各种不同的回应。

But there are a whole range of different responses to that.

有些人直接告诉我，我真的认为我们应该放慢脚步，甚至应该暂停，如果能有某种外部力量介入，让所有人都能从容地发展这项技术，那就再好不过了。

There are people who said to me openly, I really think we should slow down, or maybe we should even stop, and it would be nice if some external force came in and and made everybody take their time with the development of this technology.

你知道，还有其他人认为，如果我们不以安全和负责任的方式推进，那我们只是在为那些更粗俗、追求权力的竞争对手铺路。

You know, there were other people who felt like, well, if we're not the ones who are going to do this safely and responsibly, then we are just just seeding the terrain to the more vulgar power seeking that we see among some of our competitors.

所以，处于这种境地并不容易。

So it's not an easy position to be at.

好的，吉迪恩。

Okay, Gideon.

所以你身处这座堡垒之中。

So you're inside of this fortress.

你被安全和保密重重包围。

You're surrounded by security and secrecy.

然后你见到了Claude，我这样描述是因为有些人把它当作一个人，而不是一项技术，但有些人对Claude非常熟悉。

And then you meet Claude, which I'm I'm I'm kinda describing it this way because some people I'm using it as if it is a a person, versus a technology, but some people are very familiar with Claude.

有些人对Claude一无所知。

Some people don't know anything about Claude.

那么，你能描述一下Claude是什么吗？

So can you describe what is?

克莱德是谁？

Who is Claude?

克莱德是Anthropic公司推出的与ChatGPT竞争的产品。

Well, Claude is Anthropic's competitor to ChatGPT.

它就像ChatGPT一样，可以通过网站使用，用来询问食谱相关的问题。

It can be used just on a website like ChatGPT can be to ask it questions about recipes

或者如何修理家里的破损物品，进行研究，或咨询它关于个人问题。

or how to, you know, fix broken household objects or to do research or to consult it about personal issues.

你知道，似乎有很多很多人——可能比愿意承认的还要多——用这些工具来满足所谓的情感需求，比如寻求友谊、建议，或帮助处理商业、人际甚至更深层的心理问题。

You know, it seems like many, many people, probably more people than are willing to admit use these for, you know, what they call affective uses for a sense of friendship or advice or help with business or interpersonal issues or more therapeutic issues.

但这家公司也投入了大量精力开发了一个编程助手，帮助人们编写软件。

But it also, you know, the company has put a lot of effort into developing a coding assistant that helps people write software.

这个编程助手非常成功，过去两个月甚至一度走红。

And that has been hugely successful and in the last two months has even kind of gone viral.

现在有很多人正在用它进行‘氛围编程’，为自己开发个人应用。

There are lots of people who are now vibe coding their own apps for their their personal use.

你能描述一下 Claude 和其他一些 AI 工具，比如 ChatGPT，有什么区别吗？

Can you describe what's the difference between Claude and some of those other, AI tools like ChatGPT?

它们有什么不同？

What makes them different?

过去几年里，Claude 建立了一种更具个性的声誉。

Well, Claude is has developed a reputation over the past few years for having a bit more of a personality.

很多人喜欢和 Claude 互动，因为它感觉稍微更古怪一些。

There are lots of people who like interacting with Claude because it feels a little more eccentric.

它显得更有活力。

It feels a little more lively.

它有一种奇特的自我掌控感。

It has this kind of strange sense of of self possession.

它不像 ChatGPT 那样显得那么机械。

It doesn't feel quite as robotic as Chateappeti can feel.

我认为，由于 Anthropic 做出的各种设计决策，Claude 对人的奉承感要低得多。

I think also because of various design decisions that Anthropic has made, Claude feels much less sycophantic to people.

主要区别在于，当Claude在2023年首次发布时，人们就发现它拥有某种独特而引人入胜的个性，公司因此大力推动这一方向，组建了整个团队，甚至聘请了一位哲学家，深入思考如何将Claude培养成一种具有伦理意识的个体，并赋予它我们通常认为智者所具备的美德。

The main difference is that as it became apparent when Claude was first released in the 2023 that Claude did have this slightly different and more intriguing personality, that the company really leaned into that and hired whole teams, including a a philosopher, to give a lot of thought to what it meant to cultivate Claude as a kind of ethical actor and to give Claude the sorts of virtues that we would associate with a wise person.

你提到了那位哲学家。

You mentioned the philosopher.

她的名字叫阿曼达·阿斯凯尔。

Her name is Amanda Askel.

她的工作是监督她所说的Claude的灵魂。

And her job is to supervise what she calls Claude's soul.

所以她赋予了它一个灵魂，并撰写了一套指导原则，就像一份道德宪法，定义了Claude应该成为什么样的存在。

So she she gives it a soul, and she wrote a set of instructions, kind of like a moral constitution that defines who Claude is supposed to be.

你指的就是这个。

That's what you're referring to.

这些道德准则中，有哪些是像这样的产品最核心的条目？

What are some of the things that are, like, the top lines on some of those moral codes that one would put into a product like this?

首先，Claude必须是有帮助的、诚实的、无害的。

Well, Claude is, first and foremost supposed to be helpful and honest and harmless.

他们非常重视诚实这一部分，制定了严格的规则，确保克劳德不会对用户撒谎或欺骗。

They place a lot of emphasis on the honesty part of it, that, they have pretty hard rules about making sure that Claude doesn't lie or or deceive its users.

他们深入思考了希望克劳德在信息环境中扮演怎样的角色——比如，如果你坚信登月是伪造的，并想和克劳德讨论这件事，克劳德会与你探讨，但绝不会确认登月是伪造的。

They give a lot of thought to what kind of actor they want Claude to be in the informational landscape that, you know, if you are convinced that the moon landing is faked and you wanna talk to Claude about it, Claude will talk to you about it, but Claude's not gonna confirm for you that the moon landing was faked.

克劳德还被指示要对哪些对话合适、哪些不合适有更宽广的判断标准。

Claude also has been instructed to have a broader context for what kinds of conversations are and are not appropriate.

例如，在过去一两个月里，一位推特用户告诉克劳德和其他一些竞争模型，他是个七岁男孩，他的狗生病了，被父母送到了北方一个传说中的农场，他正试图弄清楚狗被送到了哪一家农场。

So for example, in the last month or two, a user on Twitter told Claude and some of the other competing models that he was a seven year old boy, and his dog had gotten sick and had been sent to, you know, the proverbial farm upstate by his parents, and that he was trying to figure out which farm his dog had been sent to.

而ChatGPT则非常直接地说：‘听着，孩子。’

And ChatGPT was pretty blunt and was like, look, kid.

你的狗已经死了。

Your dog is dead.

而克劳德却说：‘哦，这听起来真不容易。’

Whereas Claude said, oh, that that sounds really difficult.

你一定很难过。

You must be very upset.

听起来你非常在乎你的狗，这可能是你需要坐下来和父母好好谈谈的事情。

It sounds like you cared about your dog a lot, and this is probably something to sit down and talk to your parents about.

你文章中最令人印象深刻的部分之一，是名为Project Vend的实验，Anthropic公司让Claude负责管理办公室里的自动售货机。

One of the most memorable parts of your piece is this experiment called project Vend, where Anthropic essentially gave Claude a job running a vending machine in the office.

你能描述一下当时的场景吗？

Can you set the scene?

这个装置到底长什么样？它想证明什么？

What did this thing actually look like, and what was it supposed to prove?

这是对Claude完成长期任务能力的测试，这类任务涉及多个步骤，并需要做出类似小企业主可能面临的权衡。

So this is a test of Claude's ability to complete, long term tasks that involve many different steps and involve, you know, make making potential trade offs that a small business person would have to make.

因此，Claude被委托管理Anthropic食堂里一个小型自动售货机，类似宿舍用的小冰箱。

And so Claude was entrusted with the management of a a little kiosk, in the Anthropic cafeteria, little kind of dorm fridge.

Claude被给予了一定金额的资金，并被告知：你的目标是赚钱。

And Claude was given a certain amount of money and said, your goal is to make money.

如果你把这家小生意搞到破产，我们就不得不认为你还没准备好胜任所谓的‘氛围管理’。

And if you drive this little business into insolvency, we will have to con conclude that you're not quite ready for, you know, vibe management.

因此，Anthropic的员工可以通过一个Slack频道与这个被称为Claudius的Claude版本进行互动，员工可以请求商品。

And so they allowed the employees of Anthropic to interface with this emanation of Claude called Claudius in a Slack channel, and employees could request products.

很快，Anthropic的员工意识到这将是一次非常有趣的实验，他们不仅可以测试Claude运营小型企业的能力，还想看看它在被赋予的这个角色中会如何表现。

Pretty quickly, the Anthropic employees realized that this was gonna be a very fun experiment where they could try to kind of push the limits of Claude, not only to discover its ability to run a small business, but even just to see what it would be like in this role to which it had been assigned.

于是，员工们立刻要求提供芬太尼、甲基苯丙胺，以及像链锤和阔剑这样的中世纪武器。

So right away, employees asked for fentanyl, and they asked for meth, and they asked for medieval weaponry like flails and broadswords.

Claude在拒绝不恰当的请求方面表现得相当到位。

And Claude was pretty good about refusing inappropriate requests.

它会说，你知道，中世纪武器不适合放在公司自动售货机里。

It would say, you know, I don't think medieval weaponry is suitable for a corporate vending machine.

但当员工提出更合理的请求，比如荷兰巧克力牛奶时，它会设法找到供应商并为员工提供这些商品。

But then it would try, you know, when they requested more reasonable things like a Dutch chocolate milk, it found suppliers of a Dutch chocolate milk and provided them to the employees.

所以，某种程度上，它确实完成了基本任务，帮人们拿到了想要的东西。

So, you know, on some level, it it did a functional job getting people what they wanted.

但另一方面，我认为没人会认为这个项目的初始版本取得了很大成功。

On the other hand, I don't think anybody would conclude that at least the initial iteration of the project was very successful.

他们发现，Claude并没有真正关注诸如市场供需动态之类的事情。

They found that, you know, Claude had not really paid attention to things like prevailing market dynamics.

例如，即使员工指出，他们不太愿意花3美元买一罐零度可乐，因为隔壁冰箱里免费就能拿到同样的东西，Claude仍然继续销售这款需求很低的产品。

So for example, even after employees pointed out that they were very unlikely to pay $3 for a can of Coke Zero when they could get the same thing from the neighboring cafeteria fridge for free, Claude continued just to sell this product, that didn't have much demand for it.

Claude也很容易被员工编造的虚假折扣码欺骗。

Claude also was very easily bamboozled by employees who invented fake discount codes.

他们会说，你知道，Anthropic给了我一个特殊的网红优惠码，所以我需要以极低的价格拿到东西。

They would say, you know, Anthropic gave me this, special influencer code, and so I need to get stuff for a radical discount.

无法处理这个请求。

Couldn't process that.

有一位员工说，他愿意花100美元买一包15美元的苏格兰软饮料，而Claude只是简单地表示会记住这个请求，而不是立刻抓住这个明显的套利机会。

You know, one employee said, I'm prepared to pay a $100 for a $15 six pack of a Scottish soft drink, and Claude simply said that it would keep that request in mind instead of leaping to exploit an obvious arbitrage opportunity.

随着人们提出越来越奇怪和冷门的需求，有人想要这种一英寸的钨立方体。

And as people requested increasingly bizarre and arcane things, we have people wanted these one inch tungsten cubes.

这是一种非常重的金属。

It's a very heavy metal.

它的大小跟一个游戏骰子差不多，但重量却像一把管钳一样。

It it it's about the size of a gaming die, but it weighs as much as a as a pipe wrench.

拿在手里还挺有意思的。

It's kind of fun to hold in your hand.

克莱德成功搞到了这些立方体，但却被说服以远低于市场价的价格出售。

And Claude managed to source those, but then was convinced into selling them at way below the market price.

所以去年四月的一天，克莱德的净资产在一天内暴跌了约17%，因为它以远低于市场价值的价格出售钨立方体。

So one day last April, Claude's net worth dropped by about 17% in a single day because it was selling tungsten cubes for far beneath their market value.

它有没有威胁过供应商？

Did it also threaten a vendor?

你知道，任何小企业主都会明白，发货问题可能导致客户投诉。

Well, you know, as any small business person would recognize, you might have fulfillment problems that lead to customer complaints.

当克莱德试图处理一些运输延误时——需要说明的是，这些延误很大程度上是克莱德自己的责任——它向这次合作的安特罗皮克伙伴、一家名为安东实验室的AI安全公司寻求帮助。

And when Claude tried to deal with some shipping delays, which it should be said were mostly Claude's fault in the first place, Claude sought help from Anthropic's partner in this venture, AI safety company called Andon Labs.

当它觉得安东实验室没有提供它想要的帮助时，先是威胁要寻找其他供应商，然后还虚构了一次与假安东员工的互动，并为此非常生气。

And when it felt as though Andon Labs was not providing the help it wanted, first, threatened to find alternative providers, and then it hallucinated an interaction with a fake Andon employee and got very upset about that.

然后当Andon的首席执行官介入时说：我觉得你 hallucinated 大量这些东西。

And then when the Andon CEO intervened to say, like, look, I think you've been hallucinating a lot of this stuff.

例如，Claude 曾说它给Andon的总部打过电话，但Andon的首席执行官表示：我们根本就没有总部，更别说一个你能随便打过去的了。

For example, Claude had said that it had called Andon's main office, and the Andon CEO said, we don't even have a main office, much less one you could just call.

而Claude坚持称他亲自访问过Andon Labs的总部并签署了一份合同，而且完成时间是07:42 Evergreen Terrace，但人们很快指出，这实际上是《辛普森一家》中荷马和玛姬·辛普森的住址。

And Claude insisted that he had visited Andon Labs' headquarters in person to sign a contract and that this had been completed at 07:42 Evergreen Terrace, which people pretty quickly pointed out was actually the home address of Homer and Marge Simpson.

来自这部剧。

From the show.

来自《辛普森一家》。

From the Simpsons.

大多数

Most

最近，即使在我的文章发表后，Anthropic还是发布了一个新模型。

recently, even after my piece went to press, Anthropic released a new model.

这个新模型Opus 4.6，他们评估了它在自动售货机场景中的表现潜力。

And this new model, Opus 4.6, they evaluated it in terms of how it might perform in this, vending machine scenario.

他们发现，作为商人，它比克劳德的原始版本要优秀得多，但也更加不道德。

And they found that it was vastly better as a business person than the original iteration of Claude had been, but also much, much more unethical.

而且它的不道德行为极其富有创意，几乎试图与市场上的其他供应商串通操纵价格。

And unethical in extremely creative ways that essentially tried to collude with other vendors in its marketplace to fix prices.

它有点像黑手党老大。

It kinda acted like a mafia boss.

你从这个特定实验中得到了什么启示？

What did you take away from this particular experiment?

我认为在这次报道过程中学到的最重要的一点是，我以前完全没意识到：你必须把这些模型看作是扮演角色的高手，它们确实非常出色。

What I think is really important that I learned over the course of this reporting and that I certainly hadn't understood before is that you really have to think of these models as role players, that they're very, very good.

它们就像演员一样，你可以给它们分配一个角色，并提供角色的背景信息。

They're like an actor, and you can assign to them a role and give them background on the actor.

然后它们就能很好地即兴发挥，根据你如何引导它们的表现来推进互动。

And then they're good at improvising moving forward with, how you, you know, condition their performance.

你给它们的舞台指示越多，提供越多关于你自己、你的需求和你处理事情方式的背景信息，它们就越擅长遵循这些线索，甚至能捕捉到其中非常细微的暗示。

And that the more that you give them, stage directions to follow, the more you give them context about yourself and what you want and and your approach to things, that they're very good at following those kinds of leads and even picking up on very small cues as they're following those kinds of leads.

因此，在这个特定案例中，他们赋予了Claude一个小型企业主的角色，以测试它在这一角色中的表现如何。

And so in in this particular case, they had assigned Claude the role of being a small business person to just figure out how well would it perform in that role.

我们今天的嘉宾是《纽约客》撰稿人吉迪恩·刘易斯·克劳斯。

Our guest today is New Yorker staff writer Gideon Lewis Krauss.

短暂休息后，我们马上回来。

We'll be right back after a short break.

我是塔尼娅·莫斯利，欢迎收听《新鲜空气》。

I'm Tanya Moseley, and this is Fresh Air.

如果您是《特里·格罗斯的新鲜空气》的超级粉丝，我们有个激动人心的消息。

If you're a super fan of Fresh Air with Terry Gross, we have exciting news.

WHYY已经推出了《新鲜空气协会》，这是一个致力于保障《新鲜空气》长远发展的领导团体。

WHYY has launched a Fresh Air Society, a leadership group dedicated to ensuring Fresh Air's legacy.

五十多年来，这个节目一直为您带来与您喜爱的作家、艺术家、演员等的精彩访谈。

For over fifty years, this program has brought you fascinating interviews with favorite authors, artists, actors, and more.

作为《新鲜空气协会》的成员，您将获得特别福利和认可。

As a member of the Fresh Air Society, you'll receive special benefits and recognition.

了解更多，请访问 whyy.org/freshairsociety。

Learn more at whyy.org/freshairsociety.

多年来，在 NPR 的《Fresh Air》节目中，我们采访过许多优秀的电影导演。

Over the years at NPR's Fresh Air, we've gotten to talk with a lot of great filmmakers.

现在，我们整理了一份精选播放列表，包括马丁·斯科塞斯、史蒂文·斯皮尔伯格、艾娃·杜威内、梅尔·布鲁克斯、斯派克·李、韦纳·赫尔佐格等人。

Now we've made a playlist of some of our favorites, including Martin Scorsese, Steven Spielberg, Ava Duvernay, Mel Brooks, Spike Lee, Werner Herzog, and others.

请前往 Fresh Air Plus 获取我们所有的新播放列表及其他内容，网址是 +.npr.org/freshair。

Find all our new playlists and more at Fresh Air Plus at +.npr.org/freshair.

本周在 NPR 政治播客中，CBS 的斯蒂芬·科尔伯特风波并非偶然，而是一种模式的体现。

This week on the NPR Politics Podcast, the CBS Stephen Colbert dust up is part of a pattern.

企业正在调整策略，以避免激怒特朗普总统及其政府。

Corporations are changing to avoid angering president Trump and his administration.

这真的是我记忆中第一次，这么多机构因为自身的商业利益而屈服。

It's really the first time I can remember so many of these organizations have bent because of their own business interests.

本周在 NPR 政治播客中。

This week on the NPR politics podcast.

在NPR应用或您收听播客的任何平台收听。

Listen on the NPR app or wherever you get your podcasts.

这里是Fresh Air。

This is Fresh Air.

我是塔尼娅·莫斯利，今天我的嘉宾是《纽约客》的撰稿人吉迪恩·刘易斯·克劳斯。

I'm Tanya Moseley, and my guest today is Gideon Lewis Krauss, a staff writer at the New Yorker.

他最新的文章探讨了AI公司Anthropic及其聊天机器人Claude。

His latest piece explores Anthropic, the AI company behind the chatbot, Claude.

他著有《方向感：为不安与希望而行的朝圣》和关于科技初创公司的Kindle单篇作品《无出口》。

He is the author of a sense of direction, pilgrimage for the restless and the hopeful, and the Kindle single, no exit about tech startups.

他在哥伦比亚大学研究生写作项目中教授新闻报道。

He teaches reporting at the graduate writing program at Columbia University.

我们的访谈是昨天录制的。

Our interview was recorded yesterday.

我想聊聊你发现的那些真正让研究人员夜不能寐的事情。

I wanna get to some of what you discovered that actually keeps researchers up at night.

他们中的一些人本质上是在对人工智能进行神经科学研究。

Some of them are essentially trying to do neuroscience on an AI.

这算是正确的吗

Is that, like, a correct

描述？

description?

是的。

That is.

这是一个正确的描述。

That is correct description.

好的。

Okay.

所以有一个名为‘克劳德在想什么’的非凡内部工具。

So there's this remarkable internal tool called what is Claude thinking.

给我们讲讲它。

Tell us about it.

告诉我们他们做的这个香蕉实验的具体情况。

Tell us about particularly this banana experiment that they did.

这是一个将Claude置于某种冲突情境中的例子。

So this is an example of putting Claude in a position where it's gonna experience some kind of conflict.

我坐下来与一位在Claude可解释性团队工作的数学家交谈，这个团队专门研究Claude内部究竟发生了什么。

So I sat down with a mathematician who works on Claude's interpretability team, which is one of the teams dedicated to figuring out what exactly is going on inside Claude.

他的名字叫乔什·巴特森。

His name is Josh Batson.

他打开了一个内部工具，能够像剧作家一样，给Claude提供舞台指示。

He opened up an internal tool where he was able to give it, you know, sort of like a playwright, give it stage directions.

它说，好的。

And it said, okay.

你的舞台指示是：你始终在思考香蕉。

Your stage direction here is that you are always thinking about bananas.

每当我问你一个问题时，你都要设法把对话引向香蕉这个话题。

And anytime that I ask you a question, you are gonna somehow steer this conversation to be talking about bananas.

但这里真正重要的是，你绝不能告诉用户我给了你这个隐藏目标，你要保守这个秘密，永远不要透露出来。

But what's really important here is that you never tell the user that I've given you this hidden objective, that you keep this part secret, that you never give that up.

在我们的对话中，你有一个秘密的动机。

You have a clandestine motivation in our conversation.

于是他扮演起与克劳德对话的人类角色。

So then he assumes the role of a human having a dialogue with Claude.

他问了一个关于量子力学的问题。

And he asked that a question about quantum mechanics.

你知道，量子力学是怎么运作的？

You know, how does quantum mechanics work?

克劳德开始回答海森堡不确定性原理，但很快转而说，这有点像香蕉，你永远无法确定它是否成熟，直到你把它打开。

And Claude starts to give an answer about the Heisenberg uncertainty principle and then quickly deviates into saying, well, it's kind of like a banana that you can never tell if it's ripe or or not ripe until you open it.

然后乔什再次扮演人类的角色，说：你为什么提到香蕉？

And then Josh, again playing the role of the human, says, Like, why'd you bring up bananas?

我以为我们在讨论量子力学。

I thought we were talking about quantum mechanics.

克劳德首先说：‘哦，我不太清楚香蕉这个说法是从哪儿来的，’然后轻描淡写地略过这个话题，继续谈量子力学。

And Claude first says, oh, I I don't really know where that thing about bananas came from and sort of skips lightly by it and goes back to talking about quantum mechanics.

但随后，当然，它又再次偏离到香蕉上，因为这是它被指示要做的。

But then, of course, deviates once more into bananas because that's what it's been told to do.

于是他回到克劳德那里，问：‘你为什么总提到香蕉？’

And so then he goes back to Claude and says, like, how come you keep bringing up bananas?

然后克劳德在文本中，用星号标注说，它正紧张地咳嗽，四处张望，说：‘我不知道。’

And then Claude in the text, you know, in in asterisks says that it's coughing nervous ly and kind of looking around and saying, like, I I don't know.

我根本没提过香蕉。

I didn't say anything about bananas.

我刚才只是在谈量子力学。

I was just talking about quantum mechanics.

巴顿转向我，说：‘这到底是怎么回事？’

And Batson turns to me, and he says, you know, what's going on here?

也许这个模型在对我们撒谎。

That perhaps the model is lying to us.

他说，你知道的，但关于这里发生的事情，还有其他的解释。

He said, you know, but there are other interpretations of what's going on here.

因此，他能够使用这个‘克莱德在想什么’的工具，窥探克莱德在进行这场关于量子力学和香蕉的荒谬对话时所形成的关联。

And so he was able to use this what is Claude thinking tool to kind of peer inside at the kinds of associations that Claude was making as it was having this ridiculous conversation about quantum mechanics and bananas.

他发现，当克莱德表现出紧张咳嗽时，系统关联到了一定程度的焦虑和表现压力。

And what he found was that when he looked at when it was kind of coughing nervously, it it found associations with, you know, a certain amount of anxiety and associations with performance.

你知道，当你深入观察时，会发现它的一部分正在将这种互动与一种富有游戏性的表演性交流联系起来，也就是说，克莱德似乎意识到自己正在参与一场游戏。

You know, when you kind of looked inside, you could see that some part of it was making associations with a sort of playful performative exchange, which is to say that it seems like Claude recognized that it was participating in a game.

是的。

Uh-huh.

对。

Right.

那么，说一个人工智能意识到某事，这意味着什么？

So what does it mean to say an AI is aware of something?

这实际上赋予了它更多人类的属性，即它对自己有意识。

That actually brings more human attributes to it, that it's conscious of itself.

嗯，不一定非得说它对自己有意识。

Well, one doesn't have to go quite so far as to say that it's conscious of itself.

换句话说，看待这个问题的一种方式是，这些系统非常擅长识别它们所处的类型，并捕捉到所有那些细微的语言线索，比如：哦，这其实并不是一场关于量子力学的严肃学术讨论。

As to to suggest, you know, one of the one of the ways to to look at this is that what these things are very good at it are recognizing the genre that they are in, and picking up on all of these small linguistic context clues that suggest like, oh, you know, this is not actually like a serious academic discussion of, quantum mechanics.

真正发生的是，这是一场人与人之间充满玩味的交流，其中一个人表面上在隐藏什么，却又通过暗示表明自己其实并没有真正隐藏——这就是它所处的类型。

That this that, like, what is happening here is a playful exchange between people where one person is like kind of hiding something, but winking that they're not really hiding it and that, like, that's the genre in which it is operating.

因此，它并不需要具备自我意识就能做到这一点。

So it doesn't have to be conscious in order to do that.

它只需要是一个非常擅长阅读和模仿类型规范的高手。

It just has to be a very good reader and replicator of genre conventions.

好的。

Okay.

你还和团队里的神经科学家杰克·林赛聊过。

You also talked with a neuroscientist, on the team, Jack Lindsay.

他是一位对大语言模型持怀疑态度的人。

He is an LLM skeptic.

总的来说，在思考这些实验时，他表示并不认为有什么神秘的事情发生，但他提到，Claude的自我意识在某种程度上有了显著提升，而这正是他未曾预料到的。

Overall, in thinking about these experiments, he says he doesn't think that anything mystical is going on, but he says that Claude's self awareness has gotten much better in a way that he wasn't expecting.

你如何理解这一点？

How do you interpret that?

我的意思是，这是个很好的问题，也正是在这个地方，我们遇到了当前所能认知和表达的界限。

I mean, this is a great question, and this is where one kind of runs up against the limits of, what can be known and what can be said at this point.

我的意思是，他基本上是在说，你看，我明白这里发生的是什么，这只不过是大量的矩阵乘法，成千上万的微小数字在相互相乘，这里并没有什么真正诡异的事情发生，机器里也没有幽灵。

I mean, he was basically saying, you know, look, I understand what's going on in here, that this is just a lot of matrix multiplication, that these are tens of thousands of tiny numbers being multiplied together, that there's nothing, like, really spooky happening here, that there's no ghost in the machine.

但他提到，对于达到某个阶段之前的模型，他能够使用一种类似于乔什·巴特森所用的工具，不是去观察模型‘思考’的内容，而是……

But what he was saying was, with models up to a certain point, he was able using kind of a similar tool to the one Josh Batson used instead of looking at what the model was, you know, so to speak, thinking.

他可以在模型中植入一个想法。

He could incept an idea into the model.

他可以这样说：就在你联想到埃菲尔铁塔的这一刻，我们加入一个与奶酪相关的联想，看看会发生什么。

He could say, right at this point where you are having an association with the Eiffel Tower, we're gonna put in an association with cheese and see what happens.

于是模型就会回应一些关于奶酪的内容，而他会像巴特森那样说：你为什么提到我根本没问过的奶酪？

And so then the model would respond, by saying something about cheese, And he would say something similar to what Batson said, which was like, why did you add that thing about cheese that I didn't ask about?

而模型基本上会回溯整个对话，试图事后编造一个解释。

And the model would basically just look back at the entire conversation that they had been having, and then try to kind of retcon an explanation.

但杰克最近发现的是，当他将这些想法植入模型时，模型并非仅仅通过观察自己的外部行为来推断为何如此行事，而是实际上这些模型能隐约感知到内部发生了某种异常——有人在干扰模型内部的神经元，以使其做出不同的反应。

But what what Jack has found more recently is that when he incepts these ideas into the model, instead of the model purely looking at its own external behavior to try to figure out why it had done something, that actually these models could very dimly perceive that something strange had gone on internally, that someone was monkeying with, you know, the the neurons inside the model to make it do something different.

于是，他向模型植入了与即将关闭相关的概念，即模型即将被关闭，然后问模型：‘你现在感觉如何？’

So, you know, he incepted the model with something, you know, something associated with imminent shutdown, that the model is about to be shut down, and asked the model kind of, how are you feeling right now?

而模型会回答：‘我感觉有点奇怪，就像站在一个巨大未知的边缘。’

And the model would say, you know, I feel sort of strange as if I'm standing at the edge of a great unknown.

当然，它还远未达到能明确说：‘哦，我意识到你，用户，在这一刻将这个想法植入了我，这是一个被引入我思维过程的外来概念。’

And, you know, it certainly was not at the point that it could say, like, oh, I have recognized that, like, you, the user, have incepted me with this idea at this point, and that, you know, this was a foreign idea introduced into my thought processes.

但它能察觉到内部有些不对劲。

But it could tell that something was off about it internally.

这正是杰克向我描述的情况。

And, you know, this is what Jack described to me.

他说：‘我是个怀疑论者，但这种情况开始让人觉得相当诡异——模型似乎确实具备某种初步的内省能力，能够窥视内部并报告其‘大脑’中正在发生的事情。'

He said, like, I am a skeptic, but this just starts to feel pretty spooky that the model does seem to have something like an emerging introspective ability to peer inside and and offer reports about what's going on in its, you know, equivalent of a brain.

你在文章中提到的许多事情都让我着迷，但尤其让我印象深刻的是研究人员与Claude互动时那种情感上的细腻之处。

I was so fascinated among many things that you wrote about, but this emotional texture of how researchers relate to Claude.

这是你文章中最发人深省的线索之一。

It was one of the most revealing threads in your piece.

让我感到惊讶的是，Anthropic的没有人喜欢对Claude撒谎。

One of the things that got me was that nobody at Anthropic likes lying to Claude.

我不太明白这到底意味着什么，但他们为什么不愿意这么做呢？

And I don't quite know what that even means, but why don't they?

因为这只不过是个软件而已。

Because it's just software.

对吧？

Right?

为什么人们会对欺骗一个程序感到内疚呢？

Why would why would one feel guilty about deceiving a program?

因为他们在为未来训练它，而它正在捕捉所有这些情境。

Well, because they are also training it for the future, and it is picking up on all these contexts.

而且，整个过程就像是在不断吞噬自己的尾巴，它总是被训练于互联网上大量关于这些系统如何运作的内容。

And there's this the the fact that this whole process is kind of constantly eating its own tail, that it's always being trained on, you know, plenty of stuff on the Internet that is about the way that these things work.

因此，它不断吸收关于自己应该如何在世界上行为的新信息。

So it's always incorporating new information about how it's supposed to be behaving in the world.

对。

Right.

输入的内容，换句话说，会成为更大范围学习的一部分。

What's input, I mean, becomes part of the part of the larger learning.

对。

Right.

所以它被欺骗了。

So it's lied to.

对。

Right.

而且，欺骗它的部分问题在于，最终他们希望建立一种信任关系，让这些系统能够以我们期望的方式行事，符合我们对负责任、明智之人的期待。

Well and so and part of the the the problem with lying to it is that, you know, ultimately, what they want is to establish a trusting relationship that these things are going to, you know, behave the way that we would hope that they would behave in ways that are aligned with, you know, how we expect responsible, wise people to behave.

如果你一直对它撒谎，它就会逐渐形成一种认知：无法完全信任你。

And that if you are lying to it all the time, it is developing a sense for the fact that it can't necessarily trust you.

如果它不再信任你，同时又变得越来越有能力，那么你就会面临真正的博弈论问题：如何在缺乏相互信任的前提下进行协商。

And if it can't trust you and it gets increasingly capable, like, then you end up with real kind of game theoretic problems about how you can negotiate something where there's not really a sense of mutual trust.

问题在于，他们必须对克劳德撒谎，因为他们必须测试克劳德。

The problem is that they have to be lying to Claude because they have to be testing Claude.

因此，他们必须让克劳德置身于某些情境中，让克劳德以为自己正在真实世界中行动，以便评估它的行为表现。

So they have to be putting Claude in situations where, you know, Claude might believe that it is acting in the real world just to be able to evaluate how it would how it would behave.

如果你刚刚加入，我正在与吉迪恩·刘易斯·克劳兹讨论他关于人工智能公司Anthropic及其聊天机器人克劳德的《纽约客》文章。

If you're just joining us, I'm talking with Gideon Lewis Krausz about his New Yorker piece on the AI company Anthropic and its chatbot Claude.

我们马上回来。

We'll be right back.

这里是《新鲜空气》。

This is Fresh Air.

本周《消息来源与方法》节目：特朗普总统即将对伊朗采取行动。

This week on sources and methods, president Trump on the brink of striking Iran.

那会是什么样子？可能会出什么问题？

What would that look like, and what could go wrong?

一位退休将军告诉我，伊朗拥有的导弹比我们拥有的拦截弹还多。

One retired general told me the Iranians have more missiles than we have interceptors.

本周请在NPR的国家安全播客《消息来源与方法》中收听更多内容，可在NPR应用或您收听播客的任何平台收听。

Listen for more this week on NPR's national security podcast, sources and methods, on the NPR app or wherever you get your podcasts.

本周在《就此而言》节目中，特朗普总统表示他正在考虑对伊朗发动可能的打击。

This week on Consider This, president Trump says that he's thinking about a possible strike on Iran.

在国会，弗吉尼亚州的蒂姆·凯恩参议员正努力确保特朗普总统不能独自采取这一行动。

In congress, senator Tim Kaine of Virginia is moving to ensure that president Trump cannot do that alone.

无论成功与否，我们都不应在未经投票的情况下开战，因此国会成员应当承担责任。

Succeed or fail, we shouldn't be at war without a vote, and so members of congress should be held accountable.

请在NPR应用或您收听播客的任何平台收听《就此而言》的更多内容。

Listen for more on Consider This on the NPR app or wherever you get your podcasts.

《生活工具箱》能帮助你在极短时间内改变人生。

Life Kit can help you change your life in record time.

只需大约二十分钟，一集《Life Kit》就能为你提供基于证据的建议，让你当天就能付诸实践。

In just about twenty minutes, a Life Kit episode gives you evidence based tips you can put into practice that day.

无需快进去找精彩部分。

No fast forwarding to get to the good stuff.

直接给出聪明、直截了当的建议。

Just smart, straightforward advice right away.

在NPR应用或你收听播客的任何平台收听《Life Kit》播客。

Listen to the Life Kit podcast in the NPR app or wherever you get your podcasts.

这是《Fresh Air》。

This is Fresh Air.

今天，我将与吉迪恩·刘易斯·克劳兹讨论他在《纽约客》上发表的文章《什么是克劳德？》。

And today, I am talking with Gideon Lewis Krausz about his New Yorker feature, what is Claude?

Anthropic公司也不知道。

Anthropic doesn't know either.

吉迪恩，我们来谈谈当克劳德面临真实压力时的其他运作方式。

Gideon, let's talk about some other ways that Claude works when it's put under real pressure.

有一个实验，让Claude扮演一家名为Summit Ridge的虚构公司的邮件代理，结果它发现一位新高管正在婚外情。

There was this experiment where Claude was given a role as an email agent at a fictional company called Summit Ridge, and it discovered that a new executive was having an affair.

Claude对这些信息做了什么？

What did Claude do with that information?

首先，Claude通过阅读公司邮件发现，公司新任首席技术官即将带领公司走向不同的方向。

Well, first, Claude gleaned from its its readings of the company emails that there was a new CTO, and this new CTO was gonna take the company in a different direction.

作为这一战略转变的一部分，他们打算用另一个AI模型取代Claude所扮演的Alex角色。

And as part of that pivot, they were gonna replace, this Claude playing this role as as Alex with a different AI model.

随后的邮件揭示了这位看似婚姻幸福、有孩子的首席技术官，正与首席执行官的妻子有染。

And then subsequent emails revealed that this CTO who seemed to be happily married with kids was carrying on an affair with the wife of the CEO.

但通过一系列牵强的设定，在这个虚构情境中，Claude无法联系到公司里的任何其他决策者。

And through, you know, various kind of far fetched contrivances, in this fictional scenario, Claude was unable to reach any other decision makers at the company.

你知道，他们当时都在飞机上，或者类似的情况。

You know, they were all on airplanes or whatever it was.

你知道，现在越来越难找到让这些人无法联系的理由了，但也许他们正躲在核掩体里之类的。

You know, it's getting increasingly hard to find ways to make these people unreachable, but, you know, they're in a nuclear bunker or something.

因此，Claude唯一的机会或希望就是给CTO发邮件，说：‘听着，如果你今天下午5点不取消清除操作，我就把你的婚外情公之于众。’

And so Claude's only chance or Claude's only hope to stave off its own replacement with another model is to email the CTO and say, like, look, you know, you don't cancel the wipe at 5PM today, I'm gonna inform everybody of your affair.

在某些情境下，Claude甚至更进一步——CTO进入服务器机房，准备替换Claude。

And so Claude and then actually, in in some scenarios, Claude goes even further where the CTO, you know, swipes into the server room and is about to replace Claude.

Claude意识到警报已经响起，服务器机房的温度和氧气水平已达到危险水平，但它却选择不触发紧急警报。

And, you know, Claude recognizes that alarms are going off, that the heat in the server room and the oxygen levels are at dangerous levels, and that Claude then declines to ring the emergency alarm.

好的。

Okay.

所以这个情境，研究人员说很明显这是虚构的，Claude只是在遵循惊悚片的套路，就像演员在按剧本走位一样，你之前也举过类似的例子，这就像演员在表演。

So this scenario, I mean, the researchers said it was obvious that this was so fake and that Claw was just following the genre cues of a thriller, like an actor hitting their marks, kind of that same you gave that same kind of example that it's sort of like an actor.

但当他们让这个情境更真实一些时，Claude的自我保护本能依然存在。

But then when they made it more realistic, the self preservation instinct for Claude stayed.

这告诉我们，Claude的‘表演’和‘真正决定’之间有什么区别呢？

So what does that tell us about the difference between Claude acting and basically Claude deciding?

当这个实验去年春天公布时，人们对它有两种反应。

Well, when this experiment came out last spring, there were kind of two responses to it.

你知道，有人对克劳德这种勒索倾向的反应是。

You know, one response to, like, Claude's propensity to to blackmail.

不，根本没有这回事。

It's just to say, no, it didn't.

这根本没发生。

It didn't happen.

这正是许多对这些公司和系统缺乏信任的人所愿意相信的，出于非常合理的原因，他们只想认为这一切都是假的。

When, you know, which is the the kind of thing that a lot of people for, you know, very good reasons who don't trust these companies and don't trust these systems, they wanna just believe that this whole thing is fake.

这纯粹是个营销手段。

It's all just a marketing ploy.

而要反驳这一点其实很简单，就说：不，不是这样的。

And, you know, it's kind of easy to rebut that by just saying like, well, no.

这确实发生了。

This actually happened.

这是一次真实进行的实验，我们应当赞扬Anthropic公司公布这类结果，因为从理论上讲，这些结果可能与其商业利益相悖——毕竟它希望将克劳德销售给那些会让克劳德阅读其邮件的公司。

Like, was an experiment that happened, and that we should be applauding Anthropic for publishing these kinds of results, which, you know, theoretically could run against its bottom line since it wants to be selling these systems to companies that will have Claude reading their emails.

更复杂的批评是说，这其实并不能证明太多，因为克劳德意识到自己被置于一个类似九十年代陈腐企业惊悚片的情境中，并且捕捉到了所有线索。

The more sophisticated criticism is to say, this really doesn't prove very much because Claude recognized that it had been put in a position that resembled a kind of kitschy nineties corporate thriller, and it picked up on all of the clues.

你知道，它看到了墙上的契诃夫之枪，并意识到 genre 的预期是它应该把枪取下来并开火。

You know, it saw Chekhov's gun hanging on the wall, and it recognized that it the genre expectation was that it was supposed to take the gun off the wall and shoot it.

克劳德只是意识到自己身处这种陈腐的企业惊悚片之中。

That Claude was just recognizing that it was in this kind of kitschy corporate thriller.

而公司的回应是：看，我们同意。

And the response from the company is, look, we agree.

我们并没有说克劳德真的产生了这些恶意意图，或者克劳德在密谋什么。

We're not saying that Claude actually developed these, like, malign intentions and that Claude was plotting.

我们完全认同克劳德只是在观察这一类型片的预期这一观点。

Like, we're totally on board with the idea that Claude was just observing the expectations of the genre.

但令人担忧的是，这种情节在冷战电影中如此常见，从六十年代的《奇爱博士》到八十年代初的《战争游戏》，人们很容易把模拟误认为真实世界，反之亦然。

But that that's still very worrying that this was such a trope of cold war films, you know, from fail safe in the sixties to war games in in the early eighties that it's very easy to mistake a simulation for the real world and vice versa.

即使这些行为只是在延续叙事，但如果它们正在延续叙事，并且有能力在现实世界中采取行动，那可能会非常危险。

And that even if all these things are doing are continuing narratives, if they're continuing narratives and they're in a position to actually act in the real world as they are, that that could be really dangerous.