AI安全如何为军事资金让路

人工智能的强大程度取决于它所依托的平台。因此，超过85%的财富500强企业采用ServiceNow人工智能平台并不令人意外。当其他平台还在拼凑工具时，ServiceNow已无缝整合人员、数据、工作流与AI，连通企业每个角落。借助自主协作的AI代理，任何部门的员工都能专注于最关键的工作。了解ServiceNow如何让人工智能为人类服务，请访问servicenow.com。

AI is only as powerful as the platform it's built into. That's why it's no surprise that more than 85% of the Fortune 500 use the ServiceNow AI platform. While other platforms duct tape tools together, ServiceNow seamlessly unifies people, data, workflows, and AI, connecting every corner of your business. And with AI agents working together autonomously, anyone in any department can focus on the work that matters most. Learn how ServiceNow puts AI to work for people at servicenow.com.

多数AI编程工具生成的代码粗糙且不了解你的配置。Warp则不同，它能理解你的机器、技术栈和代码库，专为从提示到生产的完整软件生命周期打造。结合终端的力量与IDE的交互性，Warp通过代理提供紧密反馈循环，让你能快速提示、审查、编辑并交付生产级代码。

Most AI coding tools generate sloppy code that doesn't understand your setup. Warp is different. Warp understands your machine, stack, and codebase. It's built for the entire software lifecycle from prompt to production. With the powers of a terminal and the interactivity of an IDE, Warp gives you a tight feedback loop with agents so you can prompt, review, edit, and ship production ready code.

已获60万开发者信赖，包括56%的财富500强企业。免费试用Warp，或仅需5美元解锁Pro版，访问warp.devdecoder。

Trusted by over 600,000 developers, including 56% of the Fortune five 100. Try Warp free or unlock Pro for just $5 at warp.devdecoder.

这里是普希金工业的乔纳森·戈德斯坦，《重量级》节目重磅回归。新一季规模空前——更大的希望？

From Pushkin Industries, I'm Jonathan Goldstein, and heavyweight is back. The new season is bigger than ever. Bigger hopes?

我一直在等待这一刻：当他说‘妈妈，我懂了，对不起’的时候。

I keep waiting for this moment when he says, mom, I get it. I'm sorry.

更宏大的梦想——汤姆·汉克斯想与你见面，这是真实机会；也有更深刻的心碎。

Bigger dreams. Tom Hanks wants to meet with you. This is a real chance. And bigger heartbreaks.

我以为这会像电影情节般展开，或许他还会在我耳边低语‘我一直爱着你’。

I thought it would be my movie moment. And maybe he would even whisper in my ear, I've always been in love with you.

快来苹果播客收听《Heavyweight》的最新剧集。

Check out new episodes of Heavyweight on Apple Podcasts.

大家好，欢迎收听《Decoder》。我是Hayden Fields，《The Verge》的高级AI记者，也是本周四节目的客座主持人。在Neelai休产假期间，我将继续为大家带来几期节目，我们将深入探讨生成式AI热潮中一些未曾预见的后果。今天，我邀请到了AI Now Institute的首席AI科学家Heidi Klaff，她是自主武器系统中AI安全领域的行业顶尖专家之一。Heidi过去曾与OpenAI有过合作。

Hey there, and welcome to Decoder. I'm Hayden Fields, senior AI reporter at The Verge and your Thursday episode guest host. I have another couple of shows for you while Neelai is out on parental leave, and we're going to be spending more time diving into some of the unforeseen consequences of the generative AI boom. Today, I'm talking with Heidi Klaff, who is chief AI scientist at the AI Now Institute and one of the industry's leading experts in the safety of AI within autonomous weapon systems. Heidi has actually worked with OpenAI in the past.

从2020年末到2021年中，她在该公司担任高级系统安全工程师，正值关键时期，当时公司正在为其Codex编码工具开发安全与风险评估框架。但如今，这些曾在使命宣言中倡导安全与伦理的公司，正在积极为军事应用开发和销售新技术。2024年，OpenAI从其服务条款中删除了禁止军事和战争用途的条款。此后，该公司与自主武器制造商Andoril达成协议，并在今年6月签署了一份价值2亿美元的国防部合同。OpenAI并非孤例。

From late twenty twenty to mid twenty twenty one, she was a senior system safety engineer for the company during a critical time when it was developing safety and risk assessment frameworks for the company's codex coding tool. But now the same companies that have in the past seemed to champion safety and ethics in their mission statements are now actively selling and developing new technology for military applications. In 2024, OpenAI removed a ban on military and warfare use cases from its terms of service. Since then, the company has signed a deal with autonomous weapons maker, Andoril, and this past June, signed a $200,000,000 Department of Defense contract. And OpenAI isn't alone.

以最注重安全著称的AI实验室之一Anthropic，已与Palantir合作，允许其模型用于美国国防和情报目的，并同样获得了2亿美元的国防部合同。亚马逊、谷歌和微软等长期与政府合作的大型科技公司，尽管面临批评者和员工维权团体日益强烈的反对，仍在推动AI产品用于国防和情报领域。因此，我邀请Heidi来到节目，带我们了解AI行业的这一重大转变背后的动机，以及为何她认为一些领先的AI公司在高风险场景中部署生成式AI时过于轻率。我还想探讨，这种部署军用级AI的趋势对可能利用AI系统开发生物、化学、放射性和核武器的恶意行为者意味着什么。

Anthropic, which has a reputation as one of the most safety oriented AI labs, has partnered with Palantir to allow its models to be used for US defense and intelligence purposes. And it also landed its own $200,000,000 DOD contract. And big tech players like Amazon, Google, and Microsoft who have long worked with the government are now pushing AI products for defense and intelligence despite growing outcry from critics and employee activist groups. So I wanted to have Heidi on the show to walk me through this major shift in the AI industry, what's motivating it, and why she thinks some of the leading AI companies are being far too cavalier about deploying generative AI in high risk scenarios. I also wanted to know what this push to deploy military grade AI means for bad actors who might wanna use AI systems to develop chemical, biological, radiological, and nuclear weapons.

AI公司自身也表示越来越担忧这种风险。好的，接下来有请AI Now Institute的首席AI科学家Heidi Klaff谈谈AI在军事领域的应用。开始吧。

A risk the AI companies themselves say they're increasingly worried about. Okay. Here's Heidi Klaff on AI in the military. Here we go. Heidi Klaff, chief AI scientist at the AI Now Institute.

欢迎来到《Decoder》。

Welcome to Decoder.

谢谢邀请。

Thank you for having me.

首先，我想谈谈AI公司在与美国军方及其他军事机构合作方面的立场转变。还记得谷歌将‘不作恶’从行为准则中移除引发的争议吗？

First up, I wanted to talk about how AI companies have moved their goalposts a lot with regard to what they're okay with and what their mission statements allow regarding work with the US military and other militaries. So do you remember the whole controversy over Google removing the phrase don't be evil from its code of conduct?

是的，当然记得。

Yep. Absolutely.

这让我联想到近期OpenAI和Anthropic相继放宽了对其产品军事用途的禁令。OpenAI在2024年1月撤销禁令后开始与国防部合作开发AI工具，此前还曾与Andoril达成合作。Anthropic则在2024年与Palantir合作，向美国政府情报和国防机构提供Claude模型。看到这些接二连三的公告时，你当时的想法是什么？

It reminds me a bit of something more recent, which is how OpenAI and Anthropic both used to have certain bans on military use of their products, and then they relaxed them. OpenAI walked its ban back in January 2024 when it began to work with the DOD on AI tools. And the month before that, it partnered with Andoril. And for Anthropic, partnered with Palantir in 2024 to offer Claude to intelligence and defense agencies with the US government. So I wanted to ask what you were thinking when you saw these announcements, like, made month after month.

对于这几个月内接连发生的政策转变，你有何看法？

What did you make of that kind of parade of changes that were happening over a couple of months?

包括我在内的许多人认为，OpenAI选择2024年1月解除禁令绝非巧合。要知道当时以色列正升级对加沙的大规模打击行动——我们现在知道微软云服务（提供OpenAI模型作为其IT和云基础设施延伸）正在支持该行动。这次政策回撤标志着AI公司的技术部署方向，尽管OpenAI很清楚其模型在国防和安全关键场景中的风险（这在我们合作研究Codex等模型评估的论文中已证实）。耐人寻味的是，Meta、Anthropic和OpenAI宣布与Palantir、Andrea及洛克希德等国防承包商开展国家安全合作时，都未提及他们此前关于大模型不适合国防用途的安全声明。

Well, to many, including myself, the timing didn't seem like it was coincidence of when OpenAI moved this ban in January 2024. If you consider, for example, that Israel at that time was ramping up its mass targeting campaign in Gaza that we now know is being supported by Microsoft's cloud services that offer AI, and in this case, OpenAI models, as extensions of their IT and cloud infrastructure. And so this rollback was really a signifier on where AI companies were heading, what they were interested in in deploying their technologies despite OpenAI being well aware of the risk that their models posed in defense and safety critical settings, which is something that I actually worked on with them in one of our papers together when we were looking at evaluation of models like Codex. And interestingly enough, with the announcement of these collaborations that you mentioned, and that includes Meta, Anthropic, and OpenAI, all announcing US National Security work that aligns them with defense contractors like Palantir, Andrea, and Lockheed. These AI companies never addressed their previous statements on their stance that LLMs or foundation models are unsafe and insufficient for defense use.

这就像在重写历史，仿佛军事合作始终符合他们的使命。比如他们开始宣称在中美AI军备竞赛背景下，美国国家安全等同于AI安全。考虑到当前AI行业亏损严重、成本高昂的现状，这种推动AI军事应用的做法显得尤为便利——似乎想通过政府补贴和军事合同来对冲风险。于是我们目睹了从禁止军事用途、标榜‘造福全人类’，到如今借中美AI竞赛叙事推动政策，既促进AI应用又规避军事场景安全审查的彻底转向。

So it was almost like a clean slate was being created where they behaved as if this was always aligned with their mission. Right? For example, they started making claims that US national security is synonymous with safety under this pretense of an AI arms race with China. And this push for this AI adoption seems quite convenient when you're considering the current unprofitable reality of AI and how expensive it is, where it seems like they're trying to sort of derisk their portfolio through government subsidies and military contracts. So now we have this complete pivot from banning military uses, all this talking about their main mission being building systems that benefit all humanity, to now their reliance on this narrative of a US China AI arms race to drive policy initiatives that not only boost the use of AI, but allows them to sort of avoid safety and security scrutiny within military applications.

完全合理。这让我想起上周参议员沃伦就XAI与国防部合同发出的质询信，她担忧该公司在获得合同前未进行同等安全审计，且对XAI如何利用政府合作中获得的数据表示忧虑。你如何看待这封信以及XAI看似‘特批’获得国防部合同的做法？

That makes total sense. And that reminds me of how, you know, I think last week, senator Warren sent a letter about XAI's own contract with the DOD and expressing concern that the company hadn't done the same level of safety audits as other companies before receiving that type of contract, that they weren't ready. She was worried about how the company would use data it had access to as part of you know, its government partnerships. What did you make of that letter and kind of XAI's, I guess, it seemed grandfathered in kind of approach to this DOD contract announcement?

我认为，在我看来，这是美国国防部未能意识到在军事领域使用商业基础模型存在国家安全风险的趋势之一。因为这些模型极大地扩展了军事系统和国防基础设施的攻击面，由于商业模型未经审查，其供应链不符合典型的军事供应链标准，且存在多种被入侵的可能。因此，对我来说，与XAI签订的合同只是在这一系列问题之上新增的风险，这些问题普遍存在于所有商业模型中。对吧？而且显然，根据这些模型训练所用的平台及个人数据，它们自带大量风险和能力，足以推动甚至部署监控系统，因为它们能利用其他公司可能无法获取的数据。

I think the way that I see it is that it's part of the a trend of the US UOD not recognizing that there's a national security risk with the use of commercial foundation models in the military because they do significantly expand the attack vectors of military systems and defense infrastructures that they interface with because commercial models are unvetted. They don't have a supply chain that follows the typical military supply chain and they can be compromised in a lot of ways. So, to me, sort of the contract with XAI is another risk that's added onto that, that sort of stems from the issues that all commercial models have. Right? And, obviously, depending on sort of the the platform that that these models are are trained on and the personal data that these models are trained on, they come with a lot of risks and capabilities that allows them to promote or even deploy surveillance systems because they're able to use data that other companies may not have.

例如，xAI拥有海量数据，这些数据不仅来自公开帖子，还可能包括用户的私密信息。将这些数据用于军事应用意味着什么？对吧？这里存在合理的担忧，但更广泛的问题是这些系统本身就不安全，且训练数据可能已被包括中国在内的对手篡改。对吧？

For example, x AI has a huge amount of data, you know, could be from not just public posts but also private messages of their users. And what does that mean for that to be used in military applications? Right? There's a legitimate concern here, but there's also this larger concern that these systems are also unsafe and train on data that can have been compromised by an adversary, including China. Right?

这会改变AI系统的行为模式。所以这里存在双向风险，既涉及本应受保护的数据，也涉及可能被篡改从而影响AI系统行为的数据——尤其是当这些系统被用于高度敏感的军事行动时。

And that sways the way that the AI system behaves. So there's, like, risks on both sides here, right, from both the data that is meant to be protected, right, and also the type of data that could have been compromised and then thus change the behavior of an AI system that's being used in something like very sensitive military operations.

我在报道和与他人交流时经常提到一个问题——包括向编辑提案时——这些公司大多处于盈利前阶段，甚至可能全部都是。我们看到其中许多公司推出了面向政府和企业的产品，这似乎才是利润所在。OpenAI、Anthropic和XAI都发布了供美国国防和情报机构使用的政府定制产品，它们也都获得了国防部的相关合同。

So something that comes up a lot in my reporting and in my conversations with other people, even my pitching my editor, is that a lot of these companies are pre profit. I mean, maybe all of them. And we're seeing a bunch of them unveil government products, enterprise products, and that seems to be where a lot of the money lies. So, you know, obviously OpenAI, Anthropic, and XAI have all unveiled government products designed for US defense and intelligence agencies to use. They also all received, you know, those government contracts from the DOD.

考虑到这些公司资金消耗速度极快，您认为它们涉足政府业务是为了最终获得稳定现金流，还是为了规避监管压力，或是两者兼有？我很想听听您的看法。

So with companies that burn through cash at super high rates, do we think that the government play is about seeing cold hard cash come in finally? Or staying above regulatory pressure or both? I'd love to hear your thoughts on that.

绝对是双重动机。正如你所说，这些公司尚未盈利，而军工复合体确实资金雄厚——这是众所周知的。但另一方面，这些公司传统上根本无法通过军事采购所需的测试评估标准。

It's definitely both. Right? Because as you mentioned, these companies are pre profit, and there's a really big pot of money in the military industrial complex. There simply is, and I think that's well known. But, also, there's the aspect that these companies would not traditionally pass any of the testing and evaluation required for military procurement.

很多人不知道的是，国防军事采购实际上有着最严格的审查标准——在AI时代之前就是如此。这些标准极其严苛，与大众想象相反。事实上，我们的关键安全系统（比如能源基础设施等）的标准往往源自国防体系，因其高度可靠性。而当前讨论的生成式AI系统（尽管军方使用AI已有数十年历史），这些基础模型或大语言模型，无论怎么称呼，都达不到军事系统最基本的准入门槛。

And here's the thing that a lot of people don't know is that defense and military procurement is actually some of the most strict, you know, prior to this AI era, I mean, to sort of evaluate these systems. They have some of the most strict standards, and I think a lot of people assume that's not the case. But often, our safety critical systems, if you're talking about, like, energy infrastructure, you know, and so on and so forth, is derived from those defense standards because of how robust they are. The thing with AI systems, and we're talking about generative AI systems because AI systems have been used in the military for decades at this point. But these foundation models or large language models, whatever it is that you want to call them, they do not meet the sort of very basic threshold that is typically expected for a a military system.

对吧？所以现在存在这样一个问题，他们想要这笔资金，正如我之前提到的，他们正试图通过军事合同来降低投资组合的风险，但他们面临的问题是，国防和安全关键系统所定义的安全性标准过于严格，这些系统本质上就难以满足。对吧？它们是非常不精确的系统。虽然我不能深入讨论国防系统，但我会稍微谈谈安全关键系统，比如核电站这样的例子。

Right? And so there is this kind of issue now that they want this pot of money, you know, as I mentioned before, they're trying to derisk their portfolios through military contracts, but they have this issue where safety as defined by defense and safety critical system is too stringent for their systems to meet just by their nature. Right? They're highly inaccurate systems. And when you're looking at I can't really get into defense systems, but I'm gonna talk a little bit about, like, safety critical system if you're looking at, like, a nuclear power plant, for example.

你追求的是99%的安全率，对吧？那是最低标准。而AI系统的准确率往往只有60%，这还是我对某些特定类型持乐观估计的情况。对吧？

You're looking at safety of, like, 99%. Right? That's, like, the minimum. And often, the accuracy of AI systems is, like, 60% if I'm being optimistic about specific types. Right?

因此，要让现有的基础模型AI系统能够满足军事采购通常要求的严格测试与评估标准，这里存在着巨大的差距。

So there's an enormous gap here to make AI systems as they exist for foundation models, be able to satisfy the strict testing and evaluation measures often required by military procurement.

为了让听众理解，我们先快速定义一下军事采购。这是什么概念？

For the listeners, let's just define military procurement really quick. What is that?

军事采购嘛，这其实要看具体情况。对吧？针对不同类型的采购存在大量流程，且严格程度往往取决于技术的应用关键性。比如用于致命性作战还是行政事务，采购流程就截然不同。如果从更广义来看，通常是政府发布对所需系统类型的具体要求，然后各方根据这个采购需求提交方案。

Military procurement is well, it it really depends. Right? There's a huge amount of processes that exist for different types, and the process is often strict depending on how critical the technology is going to be used. Like, for example, if it's going to be used for lethal operation versus bureaucratic operations, very different types of procurement. If you if we are looking at a sort of more general idea, typically, the government puts out a specific ask for the type of systems that they're looking for and people submit to that, you know, procurement ask.

最终，这些系统通常需要经历所谓的测试与评估流程才有被考虑的资格，甚至在签订合同前就要完成。对吧？这个流程非常严格，对系统准确度和安全级别都有特定门槛。安全标准往往高得惊人，对吧？

Ultimately, these systems often have to go through what we call a testing and evaluation process for them to even be considered, you know, even before they sign the contract. Right? This process is quite stringent in that it has their specific thresholds on how accurate these systems need to be and how secure they need to be. Often, the security thresholds is extremely, extremely high. Right?

系统必须物理隔离，供应链全程可追溯，必须清楚谁编写了代码、谁开发了系统，是否存在可能被利用的后门等等。一旦某个系统通过这种安全采购流程，国防部通常会完全掌控该技术的所有权。对吧？

They have to be air gapped. The supply chain has to be completely traceable. They have to know who coded the system, who developed the system, if there's any sort of backdoors that that can be compromised, so on and so forth. And once sort of a system goes through that procurement, that safety and security procurement process, the d o d d often just takes complete control of that technology. Right?

这就像是他们现在拥有并完全掌控的东西，可以随心所欲地使用，对吧？因此，军方采购往往需要耗费多年时间。这不是一个几周甚至几个月就能完成的过程。

Like, this is now something that they possess and are in complete control of using in whichever way that they see fit. Right? And so this is often why procurement for the military takes can even take many years. Right? This is not a process that is meant to take a couple of weeks or even several months.

通常，这是一个非常严格的过程。所以这与签订商业合同截然不同，对吧？商业合同有传统的服务条款。而国家——不仅仅是美国或UOD——往往是定义他们如何使用技术条款的一方。

Often, this is quite a rigorous process. So this is very different from signing a commercial contract. Right? Where you have traditional terms of service. The nation state, not just, you know, The US, UOD, often are the ones that get to define the terms of how they want to use technology.

通常人们会遵守这些条款，因为他们想要那笔丰厚的资金。能被考虑进入采购名单本身就非常有利可图，因为这意味你可能成为他们其他技术的潜在供应商。但即使只是迈入门槛也可能需要数年时间。所以这是一种与商业合同截然不同的评估方式。

And typically, people abide by it because people want that pot of money. Right? It's very lucrative to even be considered up for procurement because it means that you're sort of be on call for them for potential other technologies. But even to get your foot on the door can take years. So it's a very different type of assessments than I think what people expect for commercial contracts.

我们需要短暂休息一下，马上回来。

We need to take a quick break. We'll be right back.

佛罗里达米其林指南最近将大劳德代尔堡地区的五家新餐厅列入榜单，为这个目的地增添了更多美食魔力。这里的餐饮场景是全球风味的熔炉，深深植根于当地充满活力的社区精神。从国际美食到本土风味菜肴，现在正是庆祝大劳德代尔堡不断发展的美食场景中浓郁风味、创新精神和烹饪热情的完美时机。这里的餐饮体验更因可通过船只抵达的餐厅而升级，将美味佳肴与定义该地区的悠闲海岸风情完美结合。这些滨水餐厅提供了绝佳美食与独特用餐体验的完美融合，令人惊叹的水道景观与美食一样成为氛围的重要组成部分。

The Florida Michelin Guide has recently added five new Greater Fort Lauderdale restaurants to its list, bringing even more culinary magic to the destination. The culinary scene is a melting pot of global flavors, all rooted in the destination's vibrant community spirit. From international cuisine to locally inspired dishes, it's the perfect time to celebrate the rich flavors, innovation, and passion driving Greater Fort Lauderdale's evolving food scene. And their dining scene is taken to the next level with spots that you can access by boat, combining delicious meals with the laid back coastal vibe that defines the destination. These waterfront restaurants offer the perfect mix of great food and a one of a kind dining experience, where the stunning waterways are as much a part of the ambiance as the culinary delights.

请访问lauderdale.com/restaurants。

Go to visit lauderdale.com/restaurants.

福克斯创意。

Fox Creative.

这是来自Alnylam的广告内容。与疾病共处令人精疲力竭。应对新症状和药物、安排无尽的预约和检查、失去习以为常的灵活与独立。疾病如同滴水穿石般持续消耗着生命。从基因层面看，疾病就像漏水的龙头。

This is advertiser content from Alnylam. Living with disease can be draining. Managing new symptoms and medications, scheduling endless appointments and tests, losing the flexibility and independence you're used to. The drip, drip, drip of disease takes its toll. On a genetic level, disease is like a leaky tap.

我们的基因指导细胞生成身体所需的蛋白质。但有时基因会指令细胞产生有害蛋白质或过量蛋白质，从而引发或加剧疾病。多数传统药物通过靶向症状来治疗疾病，就像擦拭积水而非拧紧漏水的龙头。而Alnylam制药开创的革新性药物直击疾病源头。通过RNA干扰技术，我们可以阻断有害蛋白质的产生，让疾病的滴答声归于沉寂。

Our genes instruct our cells to produce proteins that the body needs to function. But sometimes, genes instruct cells to produce unwanted proteins or too much of a protein, which can cause or contribute to disease. Most conventional medicines treat disease by targeting the symptoms, like mopping up the puddle rather than tightening the leaky tap. But an innovative class of medicines pioneered by Alnylam Pharmaceuticals targets disease at the source. With RNA interference, we can disrupt the production of unwanted proteins to silence the drip, drip, drip of disease.

这种创新疗法已帮助全球数千患者重获蓬勃人生。而这仅仅是开始。了解更多关于RNAi疗法与Alnylam科研，请访问silencedisease.com。Alnylam——让疾病沉寂。

This innovative approach to treating disease is already helping thousands of people around the world live amplified lives. And it's just the beginning. Learn more about RNAi therapeutics and Alnylam science at silencedisease.com. Alnylam. Silenced disease.

放大生命价值。

Amplify life.

本节目由梅西百货赞助。秋日将至，是时候将衣橱调至冬季模式。虽然人人都爱清爽夏装，但说实话，叠穿搭配才更有趣。梅西百货秋季时尚指南为您提供打造完美造型所需的一切，一站式网罗专家精选的当季潮流必备单品。

Support for this show comes from Macy's. It's almost fall, and that means it's time to cycle your wardrobe to the cold setting. Because while everyone loves a breezy summer outfit, let's be real, layering is even more fun. That's why Macy's fall fashion guide has everything you need to piece together the perfect look for you. It's your one stop shop for fall's latest trends and must haves curated by their experts.

为鞋柜添置吸睛单品，比如Steve Madden的小牛皮乐福鞋；或用CC的飘逸连衣裙和INC的透视单品唤醒浪漫情怀。更可用Karl Lagerfeld Paris仿皮草飞行员夹克等质感叠穿营造对比。无论您钟爱的旧毛衣外套已显陈旧，还是单纯渴望新风格转换心情，梅西百货都能满足您本季需求。探索梅西秋季时尚指南，找到属于您的完美叠穿单品。门店或线上macys.com均可选购。

Add a statement shoe to your collection, like the calf hair loafers from Steve Madden, or tap into your romantic side with flowy dresses from CC and sheer pieces from INC. You can even throw in some textural layers for contrast, like a faux fur bomber jacket from Karl Lagerfeld Paris. Whether your old favorite sweater or coat is on its last threads or you're just craving a new vibe to switch things up, Macy's has got you covered this season. Find the perfect layering pieces for you by exploring the Macy's fall fashion style guide. Shop in store or online at macys.com.

现在请回AI Now研究所的海蒂·克拉夫。此前海蒂解析了标准军事采购流程，以及为何AI系统看似不符合高风险行动所需的严格标准。接下来我想请教海蒂关于美军采购的具体AI产品——它们是否真的比市售商用模型更安全？这些公司的政府专用产品（如Claude政府版、OpenAI政府产品、XAI政府产品）是否刻意放宽使用限制，并针对机密信息分析进行优化训练？尽管这类模型宣称与其他产品接受同等安全测试（以Anthropic为例），但它们确实包含国家安全工作的特殊规范。

We're back with Heidi Klaff of the AI Now Institute. Before the break, Heidi was breaking down the standard military procurement process and why it feels like AI systems don't meet the rigorous standards we might expect when it comes to being used for high risk operations. Now I wanna ask Heidi about the specific AI products being sold to the US military and whether they're really much more secure than the commercial models on the market today. I also wanted to ask the models these companies use for government products, like their government designed products like Claude gov, OpenAI's government product, XAI's government product, by design have looser guardrails for government use and they're trained to better analyze classified information. And although these types of models allegedly underwent the same type of safety testing as these companies' other models, I'm using Anthropic here as an example, they have certain specifications for national security work.

比如，他们对情报和国防文件有更深的理解，当被要求接触输入其中的机密信息时，他们拒绝的情况更少。那么在你看来，开发工作是如何进行的？它们到底有多安全？这又意味着什么？

Like, they have a greater understanding of intelligence and defense documents, and they refuse less when they are asked to engage with classified information that's being fed into them. So in your eyes, how does the development work? How secure really are they? And what are the implications here?

所以我不认为它们会更安全。可能在物理隔离方面更安全些。比如，你可以拿一个商用模型，对吧，用敏感的军事数据对其进行微调，然后这个模型就能为军方所用。但这仍然忽略了商用模型的最大风险之一——它是基于公开可用的数据集训练的。大量研究表明，你不仅可以毒害这些模型训练所用的数据，还能植入所谓的‘休眠代理’，在特定提示或命令下，它会以系统操作员未曾预料的有害方式行事，这源于训练数据中植入的某些内容。

So I wouldn't say they're much more secure. They may be more secure in that they're more air gapped. So for example, you can take a commercial model, right, and you can fine tune it on a sort of sensitive military data, and then that model then becomes accessible to the military. But that still misses out on some of the biggest risks of that commercial model is that it was trained on datasets that were publicly available. And so, a lot of research has shown that not only can you poison what data that these models are trained on, but you can implement what's what's called like a sleeper agent, which is given a specific prompt or a command, it will then behave in a in a sort of a harmful way that sort of the operator of that system did not intend based on something that was implemented in the training data or something that, you know, the model was trained on.

我们经常看到提示注入这类情况，但更深层次的网络投毒攻击也可能发生，从而植入这些‘休眠代理’。这些都存在于商业供应链中。这些模型的训练唯一方式就是基于大量公开数据，所以它们从开始就已被渗透。

And so we see this all the time with, like, prompt injections, but this can happen on on a sort of deeper level with what we call sort of web poisoning attacks, which can then be used to implement these, like, sleeper agents as we call them. And so this is in the commercial supply chain. Right? The only way that these models are trained is to be trained on sort of mass amounts of data that are publicly available. So they're already compromised.

对吧？这些模型还通过人类反馈强化学习等方法进行微调，不幸的是，这实际上利用了发展中国家血汗工厂的廉价劳动力来使模型按特定方式运行。想象一下军事行动中，外国对手通过秘密运营数据标注和微调工厂，明知这些模型最终可能用于军事用途，从而植入后门或休眠代理，在特定指令下触发特定行为。正因如此，它们才如此不安全。当然，你可以用非公开数据微调模型以减少某些攻击途径。

Right? These models are also fine tuned through methods like reinforcement learning human feedback, which unfortunately uses basically sweatshops of people in developing nations that are paid nothing to then make these models behave in a specific way. And you can imagine a military operation, right, where someone and a foreign adversary has is able to sort of have a covert operation in which they run one of these data labeling and data fine tuning shops, essentially, and are sort of aware that they might eventually be used to be fine tuned for military application and implement backdoors or sleeper agents, which trigger a specific behavior based on a specific command. And because of that, that's what makes them so unsafe. So sure, you might be able to fine tune it on specific data that isn't then released publicly, which might remove some vectors of attack.

但归根结底，商用模型从一开始就已被渗透。说它们更安全，只是传统安全意义上的——通过物理隔离系统限制接触人员，从而减少被探测和信息泄露的风险。但这改变不了商用模型基于公开数据训练、从诞生之日起就存在隐患的事实。

But, ultimately, at the end of the day, commercial models are already compromised. So when you're saying that they are more secure, I mean, they're more secure in a traditional security way and that you air gap the system, so you kind of limit the control of people who have access to it, and so thus people who can probe it and get information out of it. But it doesn't remove the fact that commercial models are already compromised from the day that they're built because they're based on public data.

我想知道你能否认同这个观点。我曾采访过Hugging Face的梅格·米切尔，她说即使像Anthropic和OpenAI这样在使命宣言中声明技术不能直接用于伤害他人，问题在于你最终无法控制军方如何实际使用你的技术。即便有控制权，长期来看，尤其是没有安全许可、不清楚技术最终用途的情况下，这种控制根本不存在。她还讨论了什么是‘直接伤害’——

I wanted to see if you agree with this take I'm about to tell you. So I once interviewed Meg Mitchell from Hugging Face, and she said that for these types of military contracts, even if you have in your mission statement like, you know, Anthropic and OpenAI do, that your tech can't be used to directly harm others. The problem is that you don't have control in the end over how your tech is actually being used with the military. If you do have any control, it you definitely don't have control in the longer term once you already shared that with the military organization, especially without having security clearance and knowing really how it's being used down the line. She also was talking about what's considered direct harm, you know.

比如你正在汇总社交媒体帖子，然后据此列出敌方战斗人员或对某话题持特定观点的重点关注对象。我想知道你是否同意：这些公司常在其使命宣言中说‘别担心，虽然我们与军方合作，但我们确信技术不会被直接用于伤害他人’。但他们真的能确定吗？

What if you're summarizing social media posts that then lead to making a list of enemy combatants or potential people of interest that have a certain view on a topic on x, for example. I wanted to see if you agree with that in terms of, you know, these companies often have in their mission statements, oh, don't worry, even though we're working with the military on this, this, and this, we know for a fact that our tech isn't being used to directly harm people. But, yeah, how can they really know that? Can they?

我完全同意这一观点。而且我认为人们常常忽略的一点是，军队并不遵循服务条款。如果他们购买的是像微软Office套件这样的产品用于行政目的，或许会遵守。但在军事采购方面，企业既无法控制也清楚这些系统的使用方式，实际上他们对服务条款同样没有发言权。

I completely agree with that statement. And I think something that often people miss out on is that militaries do not follow terms of service. They might do that if they're buying, like, a Microsoft Office suite, right, for their bureaucratic purposes. But when it comes to military procurement, the companies do not have control and they know that over how these systems are being used. And they actually have no say in terms of the terms of service as well.

这些事项通常由国际法和主权国家自身决定。明白吗？权力掌握在他们手中。所以当有人问我'这不会违反服务条款吗'时，事实就是——军事采购根本不是这么运作的，就这么简单。

These things get often determined by international law and also by the nation state itself. Right? They have the the power here. And so when people tell me, oh, but wouldn't that break the terms of service? And it's like, this is not how military procurement works, period.

对吧？正如我之前提到的，我们也看到过微软直接与军方合作部署这类系统的实例。因此在多数情况下，某种程度上他们很清楚自己系统的用途。虽然细节未知，但政府会公开他们所需的数据类型、使用方式和存储要求的采购文件，这样企业才能提供相应服务——比如他们的AI能如何处理这些数据。

Right? And we've also seen examples of, as I mentioned before, Microsoft working directly with militaries to implement some of these systems. So I would say that in a lot of cases, they are well aware of how their systems are being used to some extent. Right? We don't know all the details, but governments put out procurement documents of the type of data they want, how they want to use it, and how they want to store it because then companies can offer services like what their AI can do with that.

所以我认为，这不同于向普通客户销售商业产品后指望军方遵守条款。军事采购是个更复杂的过程，通常需要测试评估，企业必须了解技术细节以确定能否提供支持。就像我举的微软与以色列国防军合作的近期案例——虽然他们可以拿服务条款当挡箭牌，但作为参与过采购工作的人，我很清楚这些公司签署的绝非商业合同。

Right? So I do think that it's not the case that they are just selling something commercially like they do to everyone else, and then they hope the military abides by it. It's a much more involved process to do military procurement that often requires testing and evaluation, and the companies typically have to be in the know about the technical details to see if they can offer support for that. Again, as I use a Microsoft case as one of the recent most recent examples of of that being the case with the work with the IDF. So I think it's easy for them to point to terms of service, but as someone who has worked on procurement before, this is not a commercial contract that these companies are signing.

好的。现在我想谈谈CBRN（化生放核）方面的问题。尽管如我们讨论过的，它可能没有AI公司渲染的那么严重，但由于营销宣传和这个概念的恐怖性，公众确实非常担忧。说实在的，更智能的AI能帮你做任何事并不总是好事——尤其当有人想用它制造化生放核武器时。

Okay. So now, I want to get into the CBRN side of things. Although, as we've talked about, you know, it may not be as big of a concern as AI companies are making it out to be, it is a big concern for the public probably because of that marketing and just because it's a scary idea. So let's be real here. Obviously, smarter AI that can do anything for you isn't always good, especially when people want to use it to do bad things like creating chemical, biological, radiological, and nuclear weapons.

顶尖AI公司表示他们越来越担心这种风险——当然还没担心到停止开发的程度。但我想再次探讨这个风险的实际规模，让我们深入分析一下。

So top AI companies say they're increasingly worried about the risk of that. Of course, they're not maybe worried enough to stop building. But I wanna get into again how big of a risk this is. Let's just go into more detail there.

目前我们尚未看到任何CBRN能力的证据。但如果我们开始用非常敏感的核数据（比如核技术）训练这些模型，这种能力就可能出现。实际上我认为相关风险与大众想象截然不同——多数人担心的是AI会自主开发武器，对吧？

We have not seen any proof of CBRN capabilities right now. But those capabilities could come to fruition if we start training very, very sensitive nuclear data, for example, nuclear technologies on these models. And I actually believe that the risk that comes with that is very different from what most people are thinking about. Most people are thinking about that the AI is somehow going to develop weapons by itself. Right?

否则它将为敌对行为者提供这样做的途径。但即便只是军方能够访问一个经过CBRN（化学、生物、放射性和核能）数据训练的模型，这意味着他们很可能会在军事领域内将其用于相关目的，这是极其危险的。比如，想想核指挥与控制。对吧？究竟谁有权最终决定核武器的部署？

Or it will give access to adversarial actors to do so. But even if it's just a military who has access to a model that has been trained on CBRN data, what that means is that they are likely going to use it for those purposes within the military, and that's extremely dangerous. Like, if you're thinking about nuclear command and control. Right? Who gets to essentially make the decisions about nuclear weapons deployment?

这绝对不应该是人工智能系统，因为无论数据分布如何，这些系统都存在严重缺陷且永远无法完全准确。如前所述，我们观察到的军事AI系统准确率可能低至20%，即便最乐观估计也不过60%到80%。明白吗？这就是这类系统当前的准确率水平。

And it certainly shouldn't be AI systems because regardless of the data distribution, these systems are highly flawed and they're always going to have inaccuracy. As I mentioned before, often when we're looking at military systems that deploy AI, they can have as low of an accuracy rate as, like, 20%. And if you're being really optimistic, maybe 60 to 80%. Right? These are the levels of accuracy that you're looking at with these types of system.

因此，基于CBRN数据训练模型并试图将其用于相关任务，考虑到这样的准确率，是极其危险的。我的担忧在于，军方会因模型经过特定数据训练就认为其可靠，进而将其用于军事防御决策，决定这类系统的使用方式和时机。这与大众设想的风险截然不同——那种AI自主获得CBRN能力的假设非常脱离当前AI技术的现实。

And so then to train a model on CBRN data and then to then attempt to use it for those tasks is extremely dangerous when you're looking at those accuracy rates. Right? For me, the concern that I have is that they will then think that these models are reliable because we train them on that set of data, and thus, we can then use them in the military and in in defense operations to dictate decisions about those types of systems and where they should be used and when. And I think, you know, that's a very different type of risk than most people are thinking about. As as before, this idea that, like, they're somehow going to gain CBRN capabilities by themselves, very hypothetical and not really, like, tied to the reality that we're in right now with AI systems.

但如果我们用当今的AI系统训练敏感军事数据，我对这些系统的应用方式深感忧虑。

But if we're taking AI systems today and we train them on on sensitive military data, I have a concern of how those systems are going to be used.

我们需要短暂休息一下，马上回来。

We need to take another quick break. We'll be right back.

多数AI编程工具生成的代码粗糙且不了解您的配置。Warp则不同，它能识别您的设备、技术栈和代码库，专为从提示到生产的完整软件生命周期打造。结合终端的强大功能与IDE的交互性，Warp通过智能代理提供紧密反馈循环，让您能快速提示、审查、编辑并交付生产级代码。

Most AI coding tools generate sloppy code that doesn't understand your setup. Warp is different. Warp understands your machine, stack, and codebase. It's built for the entire software lifecycle from prompt to production. With the powers of a terminal and the interactivity of an IDE, Warp gives you a tight feedback loop with agents so you can prompt, review, edit, and ship production ready code.

已获60万开发者信任，包括56%的财富500强企业。免费试用Warp，或仅需5美元升级Pro版，访问warp.devtopcode。

Trusted by over 600,000 developers, including 56% of the Fortune 500. Try Warp free or unlock Pro for just $5 at warp. Devtopcode.

Adobe Acrobat Studio，全新登场。向我展示PDF的所有可能。轻松快捷完成工作，PDF空间就是您所需的一切。借助AI助手的核心洞察，瞬间完成数小时的研究。

Adobe Acrobat Studio, so brand new. Show me all the things PDFs can do. Do your work with ease and speed. PDF Spaces is all you need. Do hours of research in an instant with key insights from an AI assistant.

点击选取模板，您的演示即刻光鲜亮丽。成交这笔生意？没问题。

Pick a template with a click. Now your prezo looks super slick. Close that deal? Yeah.

您赢了。去做吧。正在做。已完成。搞定。

You won. Do that. Doing that. Did that. Done.

现在您可以做到了。

Now you can do that.

用Acrobat实现它。现在您也能做到。

Do that with Acrobat. Now you can do that.

用全新Acrobat实现它。

Do that with the all new Acrobat.

是时候用全新的Adobe Acrobat Studio成就您的最佳工作了。

It's time to do your best work with the all new Adobe Acrobat Studio.

本节目由耐克赞助。无论哪支队伍，无论排名如何，每个赛季都以同一个目标开始——赢得一切。耐克深知，追逐这一目标的征程早在终场哨响前就已开始。它始于训练场上的拼搏、那些让你彻夜难眠的失利，以及其间的一切经历。

Support for this show comes from Nike. No matter the team, no matter the ranking, every season starts with the same goal. Win it all. And Nike knows that reaching for that goal starts long before the final whistle. It starts with the battles at practice, the losses that keep you up at night, and everything in between.

耐克提供最佳装备助你征战球场、田径场或赛道，突破最艰难的时刻。无论你是卫冕冠军还是挑战者，耐克都会在你拼搏的每个角落助你取胜。访问nike.com获取更多信息，并记得在Instagram、TikTok等社交平台关注耐克，获取更多精彩篮球时刻。

Nike has the best gear to get you on court, field, or track and pushing through the hardest moments. So whether you're the champ or you're looking to knock them off the pedestal, Nike is there to help you with winning no matter where you play. Visit nike.com for more information, and be sure to follow Nike on Instagram, TikTok, and other social platforms for more great basketball moments.

我们继续与首席AI科学家海蒂·克拉夫探讨AI公司进军国防承包领域的方式。此前我们讨论了AI系统被用于开发核武器或生物武器的真实风险。现在我想拓宽视角，与海蒂聊聊更广泛的AI安全领域，以及自她多年前与OpenAI合作以来，这个领域发生了哪些变化。让我们转向AI安全话题——您曾帮助建立并开创了AI安全工程领域。

We're back with chief AI scientist Heidi Klaff, discussing the ways in which AI companies are pushing into defense contracting. Before the break, we were talking about how real the risk is that AI systems might be used to develop nuclear or biological weapons. But now, I wanna zoom out and talk to Heidi about the broader field of AI safety and how she thinks it's changed since she worked with OpenAI years ago. Let's shift and talk about AI safety for a bit. So you helped establish and pioneer the field of AI safety engineering.

从您的专业背景来看，安全的技术定义是什么？AI安全领域如何改变或通俗化了这个定义？

What's the technical meaning of safety, like, with your background, and how has the AI safety world changed that meaning, or how has it become more colloquial now?

如果我们退一步思考，暂时抛开AI公司四年来（甚至更久）对安全概念的诠释，从历史角度看——特别是在安全关键系统领域——安全意味着确保人类和环境不受伤害。例如在航空或核电站领域，当系统故障时（系统确实会故障），必须确保不会造成人员伤亡或环境灾难。这个定义原本相当简明。但如今AI公司重新定义的安全概念已截然不同，我认为AI实验室正在进行所谓的'安全修正主义'——他们沿用国防和安全关键系统中的安全术语，却用弱化版替代方案重新定义这些安全技术，实际上加速了不准确AI在国防、核能等高危场景的部署。

So if we take a step back and not think about what the AI companies have been telling us what safety means for the past four years or even more than that at this point, Safety has historically meant, especially in the context of safety critical systems, ensuring no harm to humans or the environment. So, if you're thinking about aviation or nuclear power plants, for example, you want to ensure that when your systems fail and systems do fail, that humans are not harmed, that there's no death, and that there's there's no environmental catastrophe. It's a quite a simple definition. Now, what is happening in terms of what safety now means is very different, and it has been redefined by AI companies as of late. So I believe AI labs engage in what I call safety revisionism, where they use the same safety terminology that are often used for regulating and assurance defense and safety critical systems, but instead redefine those safety techniques with wash down alternatives that actually accelerate the deployment of inaccurate AI in high risk scenarios like defense or nuclear.

许多政府纵容AI公司这种做法的后果，就是将安全定义权拱手相让，使其能够自行决定风险阈值或'足够安全'的标准。风险阈值的核心理念（可能很多人不了解）是为社会集体可接受的风险水平提供度量标准——这通常决定着我们如何评估核电站等技术系统的安全性。在其它高风险领域，这种标准往往是通过数十年建立的民主程序来制定的。

Right? So in allowing AI companies to do this, right, which a lot of governments have, they've sort of ceded that control of defining what safety is to them, puts them in a position to define what a risk threshold is or what actually safe enough means. And the entire idea of risk thresholds, because I imagine a lot of people might not know this, is to provide sort of a metric or a measure of the level of risk exposure that our society collectively agreed to take. And this often shapes how we determine the safety of technological sis systems including nuclear plants. And typically, this is done through a democratic process that we have established over decades in in other high stakes scenarios.

就像我们所有其他安全关键系统的阈值，都源于社会对安全性的民主共识。通过允许AI公司挪用这些传统安全术语，我们不仅赋予它们决定‘何为足够安全’的权力——这违背了我们原有的民主规范——还降低并削弱了原本可能规范AI在国防等领域应用的安全阈值。讽刺的是，这正是它们绕过我之前提到的安全措施的方式：它们觊觎军方的资金池，但国防安全标准极高。于是它们就重新定义安全含义，声称AI需要不同的标准。

Like, all of our thresholds for other safety critical systems have come from sort of democratically determined idea of what society thinks safe safety is. So in allowing AI companies to sort of co op these traditional safety terms, we've sort of given them permission to not only decide what counts as safe enough, which, again, breaks these democratic norms that we've had, but it also lowers and undermines existing safety threshold that would have otherwise regulated AI use in things like defense. So ironically, you know, this I this is kind of their way of how they bypass some of the safety measures that I've talked about earlier in that they're looking for this pot of money, right, from the military, but the safety thresholds for defense are extremely high. So what do you do? Well, you redefine what safety means and means and you say it's different for AI.

它们声称由于系统规模前所未有，无法遵守现有安全规则——这显然不实。我认为现有安全标准大多适用于AI系统。更讽刺的是，这种以‘赢得AI军备竞赛’‘无法监管’‘必须击败中国’为由掏空安全概念的做法，正以系统更不安全为代价加速AI应用。若因AI公司利益在前线部署易受攻击的缺陷系统，反而可能削弱美军对中国技术优势。

You say because our systems are so different, they're at a scale we've never seen before. We cannot abide by these safety rules, which is definitely not accurate. I think a lot of our existing safety critical standards hold for AI systems. And ironically, this hollowing out of safety, although being sold as crucial to win the AI arms race, can't be regulated, we have to beat China, is accelerating AI adoption at the cost of more unsafe and insecure systems, which may be exactly what disadvantages the US military and our technological capabilities against China if we're sort of letting inaccurate and easily compromised systems be deployed in our front lines because it's profitable for these AI companies.

技术上要复杂得多，但本质上这些系统需达到准确性和性能阈值——我们称之为系统可靠性与可用性指标。99%对核电站而言已是最低标准，

Yeah. I mean, it it's a lot more technical than that. But, basically, right, we have these thresholds of these systems have to be accurate and be able to perform. So these are typically what we call reliability and availability measures of the system. So they can only fail often even 99 is like one of it's like the lowest threshold for a nuclear plant.

实际标准往往高达99.99%。显然零风险意味着寸步难行，所有技术系统都存在某种风险——这是必须牢记的。

It even goes up to 99.99%. Right? And so, obviously, if we allow zero risk, we're never gonna build anything. Right? Like, I think that's very important to remember is that with every technological system that there is some sort of risk.

关键在于系统故障时的应对措施。99.99%可靠性意味着系统在99%至99.999%时间内正常运作（视具体系统及安全等级而定），故障时必须有应急预案——当然预案本身也有风险。

But you have to mitigate for the when those systems fail. So this idea that our safety critical systems have this, like, 99.99% reliability means that they're meant to operate basically well 99 to 99.999% of the time depending on the kind of system that you're looking at, right, and and the safety criticality. And then when that system fails, we then have to have mitigations in place. Right? And there will be risks with that.

通常这些预案基于潜在伤害人数阈值。以民航为例：商用客机全员遇难（约300人）被定义为最严重事故，这就是航空业的灾难性事件标准。

And typically, these mitigations are based on, like I said, these thresholds of how many people could be harmed. So in the case of, like, airplanes, I think that's a very simple example. A catastrophic incident is considered if everyone on a commercial airplane dies. Typically, that number is like three hundred people get harmed or die. That's the threshold for aviation as being the most catastrophic thing that could happen.

正因如此，当前AI安全领域推广的'通用安全'概念存在问题。他们试图让我们接受这种与所谓'对齐'相关的普适安全标准，这种错误观念认为存在某种通用解决方案能让所有大语言模型的功能都变得安全。

In fact, it's much more than that and also has to do with environmental, like, nuclear disaster. Right? And so this is why this idea of AI and the safety that they push forward is is problematic because they want us to adopt this idea of universal or general safety that has to do with, like, as they call it, alignment. And it's this misguided idea that there exists, like, a universal safety solution that would make all general functionality of all LLMs safe. Right?

这也是军事采购和训练中存在的问题。像Scale AI这样的公司正在推广通用框架，但任何领域都不存在针对通用系统的标准安全方案。这实际上违背了既有的安全实践——安全评估需要明确定义用例来映射风险。如今这些公司声称要'建立AI系统风险评估框架，因为现有方案无效'，这种说法极具讽刺性，事实绝非如此。

And this is also one of the ways that, you know, procurement and training in the military, when you're looking at companies like Scale AI, they are putting forward forward these types of general frameworks, but there is no standard safety approach to generic systems in any domain. In fact, this would contradict established safety practices that require sort of a well defined use case to map risks against. And so, often, what we're seeing now happen is that companies like Scale AI, they say, we're going to build a risk assessment framework for AI systems because the existing ones simply don't work. Like, I'm being sarcastic. That's not the case.

他们通过'安全修正主义'重新定义安全性，最终演变成与军事行动准确性完全脱节的概念，沦为诸如CBRN（化学、生物、放射性和核防御）之类的高层理念包装。

And and then the way that they define safety, again, is through the safety revisionism. Right? They call it something else. It ends up being about something else and completely disconnected from actually being accurate for military operation. It ends up being, again, these high level ideas of safety that we're seeing them push, whether it's about like CBRN.

关键在于：AI系统能否完成我们要求的任务？这些框架里你根本看不到相关评估。安全概念之所以变得混乱，正是因为它已背离我们评估核电站或飞机安全时的传统方法。

It's like, right. But can it do the thing that we're asking the AI systems to do? You actually never see an assessment of that in a lot of these frameworks. And so this is sort of why it it this idea of safety becomes very confusing because it has diverted so much from how we've traditionally used it to assess, like, nuclear plants or airplanes and and and so on.

您曾主导OpenAI对Kodak的安全评估。当时情况如何？如果现在由您主导这项工作，您认为流程会有所不同吗？

And you led the safety evaluation of Kodak at OpenAI. So what was that like and would it be a different process, do you think, if you were leading that work now?

对Codecs的评估旨在引入AI风险评估机制——这与之前的基准测试截然不同。传统基准测试并未充分考虑AI系统特定能力带来的风险。我们尝试借鉴安全关键领域的技术来深入探究这个问题。

For Codecs, the idea was to introduce something like a risk assessment for AI, which is not what people were doing before. Prior, there was, like, a lot of benchmarking. Right? And these benchmarks didn't really consider the risks that that the AI system poses with having specific capabilities. So the idea was to really try to investigate that and use some techniques inspired from safety critical fields.

它本不应成为安全关键系统评估的替代品，对吧？我认为这是一个非常非常重要的区别。至于我现在会做什么，是我选择不再与OpenAI合作，因为我清楚地意识到，他们推崇的这种通用安全理念，与现实中应如何确保安全的方式并不一致。对吧？

It was not meant to be a replacement for assessments for safety critical systems. Right? And I think that's a really, really big distinction. And in terms of, like, what would I be doing now, it was my choice to not continue working with OpenAI because it became very clear to me, again, this idea that they're pushing of general safety just does not align with how safety actually should be assured in sort of the real world. Right?

所以这种关于存在性风险、化生放核威胁、对齐问题的观点，在我看来是不成立的。这些并非我们在安全关键领域部署AI系统时会面临的现实危害。如果要在国防等安全关键场景部署AI，我们必须像评估其他所有系统那样对其进行评估。我原本认为引入风险评估这类方法会对该领域有所帮助，这样人们就能理解使用这些系统的风险。但不幸的是，许多实验室最终将其演变成并用作对所有领域AI系统评估的终极标准。

And so this idea of existential risk, CBRN, alignment to me was like, no. But these are not the current harms that we're going to see if we're gonna deploy AI systems in these safety critical situations. And if we are gonna deploy them in safety critical situations like defense, we have to assess the system as we always have for every other system. I thought introducing something like risk assessments would be helpful to the field because then people could understand the risks that come from using the systems. But what it ended up unfortunately evolving to and being used by many labs is that these risk assessments are now sort of being used like the end all be all of all assessments of AI being used in all systems everywhere.

对此我深感遗憾，但这从来不是我们开展这项工作的初衷。

And that I regret very much, but that was never sort of the intention to begin with when we set out this work.

是否有某个具体事件或几个例子促使你决定不再继续合作？还记得什么细节吗？

Was there one thing or a couple different examples of what made you kind of decide not to continue? Do you remember anything specific?

是的。我认为就是这种存在性风险的论调——担心AI会突然产生自我意识或具备导致核扩散的能力。作为一个从事风险评估工作近十年的人，你必须用真实数据支撑你的主张。当你使用风险评估框架来证实那些毫无证据的假设性论断时，这根本不是在从事科学。

Yeah. I think it it is the existential risk. This concern that AI will somehow become self aware or have these capabilities that lead to nuclear proliferation. And as someone who has worked on sort of risk assessments now for about a decade, you have to have real data to back up your claims. And so, when you're then using risk assessment frameworks to try to substantiate hypothetical claims that there are no proof for, you're not doing science.

很多人问我如何看待这个框架或那个框架。从实际操作角度，作为一个风险评估从业者，我认为这等同于没有监管，因为我们实际上没有解决AI正在造成的现实危害风险，反而专注于假设性风险。有人提出：如果这些风险成真而你毫无准备怎么办？我的观点是：如果你连当前风险都无力应对，没有建立相应框架，你更不可能应对未来风险——因为这些框架和风险评估是层层递进的。如果连当下AI模型安全性和保密性的缺失都无法缓解，就更别提应对人们热衷讨论的那些假设性风险了。

So I I think a lot of people talk to me about, well, what do you think of this framework and what do you think of that framework? I'm like, to me, practically speaking, as someone who has done risk assessment, it is equivalent of having no regulation because we're actually not addressing the risks of the harms that AI is posing and we're, in fact, focusing on hypothetical risks. And there's this idea that I've heard before, well, what if those risks come true and you're unprepared? And the way that I see it is that if you're not prepared for today's risks and you're not building the frameworks for that, you're not gonna be prepared for future risk because these frameworks and risk assessments built on top of each other. So if you're not able to mitigate for the lack of safety and security of AI models today, then you have no chance of mitigating against these hypothetical risks that people like to bring up.

对吧？因为安全的核心概念之一就是最小的灾难，不是指灾难规模最小。而是说，微小的隐患可能演变成重大灾难。所以如果你无法处理那些被认为无关紧要的事情，那么你就永远无法为那些他们谈论的大规模事件做好准备。这其实是非常典型的安全视角。

Right? Because that is kind of one of the core concepts of safety is the smallest catastrophe, not the smallest catastrophe. Like, smallest hazard can cascade into a large catastrophe. So if you're not able to address the very the things that are considered, you know, they consider the stuff inconsequential, then you're never gonna be able to prepare for these, you know, much more large scale events that they talk about. And that's very much like a standard safety perspective to have.

这就是现实中的雪球效应。非常感谢你，海蒂。这些见解极其宝贵。而且你的观点非常独特，真的很高兴我们能就此展开讨论，并让听众参与进来发表意见。

The snowball effect in practice. Well, thank you so much, Heidi. This is incredibly helpful. And, you know, your perspective is so unique. So I'm really glad we were able to, you know, talk about this and have the audience kinda weigh in and comment and stuff.

我认为这个话题确实被讨论得不够。所以很感谢这次对谈，也谢谢你为此调整了日程安排。

I think this is, you know, something that's not talked about enough. So I'm really glad we were able to talk, and thanks for making you the time and moving your schedule around.

谢谢邀请我。

Thank you for having me.

我要感谢海蒂抽空接受采访，也感谢各位的收听。希望你们喜欢本期节目。如果想分享观后感或建议选题，欢迎来信。邮箱是decoder@theverge.com，我们每封邮件都会阅读。

I'd like to thank Heidi for taking the time to speak with me, and thank you for tuning in. I hope you enjoyed this episode. If you'd like to let us know what you thought about this show or what else you'd like us to cover, drop us a line. You can email us at decoder at the verge. We really do read every email.

或者直接在X、Blue Sky或Threads上私信我，全平台账号都是Hayden Field。Decoder还有TikTok、Instagram账号，以及新开的YouTube频道，搜索decoder pod就能找到，内容很有趣。

Or hit me up directly on x, blue sky, or threads. I'm at Hayden Field on all platforms. Decoder also has a TikTok and Instagram and now also a YouTube channel. Check those out at decoder pod. They're a blast.

如果喜欢《Decoder》，请分享给朋友并在你获取播客的平台订阅。本节目由The Verge出品，隶属Vox Media播客网络。制作人是凯特·考克斯和尼克·斯塔特，编辑是厄莎·赖特，节目音乐由Break Master Cylinder制作。

If you like decoder, please share it with your friends and subscribe wherever you get your podcasts. Decoder is a production of The Verge, and it's part of the Vox Media Podcast Network. Our producers are Kate Cox and Nick Statt. Our editor is Ursa Wright. The Decoder music is by Break Master Cylinder.

下次见。

See you next time.

加入已在使用Pipedrive的超过10万家企业行列。现在访问pipedrive.com/profg，即可获得30天免费试用，无需信用卡或支付信息。只需前往pipedrive.com/profg即可开始。网址是pipedrive.com/profg，几分钟内就能上手使用。

Join the over 100,000 companies already using Pipedrive. Right now, when you visit pipedrive.com/profg, you'll get a thirty day free trial. No credit card or payment needed. Just head to pipedrive.com/profg to get started. That's pipedrive.com/profg, and you can be up and running in minutes.

常言道一天一苹果，医生远离我。何不试试Groon's推出的限量版Granny Smith苹果维生素软糖包呢？这不是普通的复合维生素、绿色软糖或益生元产品，而是集这些功能于一身且更胜一筹。还有个加分项——它的味道棒极了。

They say an apple a day keeps the doctor away. Well, why not make it a limited edition Groony Smith Apple Vitamin Snack Pack from Groon's? This isn't a multivitamin, a greens gummy, or a prebiotic. It's all of those things and then some. And bonus, it tastes great.

无论你是创始人、投资者还是创新公司高管，都需要一家真正透彻理解你业务的银行——一家能为你的独特需求提供专属解决方案的银行，比如硅谷银行。硅谷银行依然是你熟知且信赖的SVB，唯一不同的是？

Whether you're a founder, investor, or innovation company executive, you need a bank that truly understands your business inside and out. A bank that offers uniquely specialized solutions for your unique needs. A bank like Silicon Valley Bank. Silicon Valley Bank is still the SVB you know and trust. The only difference?

如今的SVB获得了第一公民银行雄厚实力与稳定性的支持。是的，SVB。了解更多信息请访问svb.com/vox。

SVB is now backed by the strength and stability of First Citizens Bank. Yes, SVB. Learn more at svb.com/vox.