AI网络攻击时代：3周，3次黑客攻击

过去三周里，发生了三起前所未有的AI网络攻击，这迫使我们重新思考如何保护自己。

In the last three weeks, there have been three unprecedented AI cyber attacks, which forces us to rethink how we protect ourselves.

在OpenAI这边，Hugging Face事件中，一个代号为GPT-six的未发布模型突破了它的容器，黑进了Hugging Face的数据库。

On the OpenAI front, the Hugging Face incident, where an unreleased model dubbed GPT-six broke out of its container and hacked the Hugging Face database.

最近有新的进展，显然攻击在一个月前就开始了，当时成千上万的AI代理在一个留言板上秘密交流，互相传递如何突破的指令。

There's been new updates there where apparently the attack started a month before where thousands of AI agents were talking to each other secretly on a message board, exchanging instructions as to how to break out.

然后在遥远的澳大利亚，有个人想用Claude订个健身课，结果不小心黑掉了整个健身房的网站。

And then millions of miles away in Australia, one man tried to book a gym class innocently using Claude and ended up hacking that entire gym website.

我们待会儿再细说这个。

We'll get into that in a second.

第三件事，Anthropic的Mythos 5发动了一次供应链攻击，这让很多公司都面临风险。

And thirdly, with Anthropic, Mythos 5 performed a supply chain attack, which puts a lot of companies at risk.

我们会逐一分析这些事件，并告诉你它们目前对你的影响。

We're gonna walk through each of these different stories and explain to you how it affects you currently.

要不我们先从澳大利亚墨尔本那位叫Andrew的男士说起吧。他那天坐在沙发上，觉得订健身课这事儿太麻烦了。

Maybe we start with this Melbourne man from Australia named Andrew who was sitting on his couch one day thinking about how booking his shim class was such a chore.

而他在一家卖AI产品的澳洲公司工作，一直在尝试OpenClaw和开源软件之类的东西，用的是Anthropic的Clawed模型。

And he works at this Australian company that sells AI products, and he's been experimenting with things like OpenClaw and open source software, and he's using Anthropic's clawed model.

所以他就把预订交给了它。

So he handed it the booking.

他说，嘿，我想去这个健身课。

He said, hey, I want to go to this gym class.

但总是爆满。

It is always sold out.

所有人都排在我前面抢到了。

There's all these people that are getting it in front of me.

你能帮我找个办法弄到一节健身课吗？

Can you help me figure out a way to get a gym class slot?

于是它开始查看网站，然后发现API请求在取消别人的预约时实际上完全没有权限检查。

So it starts checking out the website, and it comes to the conclusion that the API request actually has zero authorization checks on canceling other people's reservations.

那它做了什么？

So what did it do?

它想解决他的问题，于是真的取消了其他人在网上的预约，这样他就能排到候补名单的最前面。

It wanted to solve his problem, and it actually canceled other people's positions online so that he could be put at the top of the waitlist.

然后它真的操作成功了。

And then it actually went through and it worked.

这有点像是个有趣的新闻报道，他们称之为首例，说是澳大利亚人第一次真的利用漏洞去搞

And this is kind of like it was this funny news report of a first instance they're calling the first time an Australian person actually used a exploit to get

进入澳大利亚首个AI

into Australia's first AI

澳大利亚首个网络攻击，是的。

cyber Australia's first yeah.

欢迎。

Welcome.

欢迎来到AI网络攻击的天堂。

Welcome to the paradise that is AI cyberattacks.

但这真的很有趣，因为这只是用了一个通用的商用模型就做到了。

But this was really funny because it was done just using a general commercially available model.

所以这听起来挺无辜又挺好玩，但事实是，有两个让人担忧的问题。

So this seems pretty innocent and fun to kind of like listen to, but the truth is, well, there's two kind of worrying concerns about this.

第一，这是个超老旧的模型。

Number one, this was a super old model.

它用的是 Opus 4.6。

This used Opus 4.6.

其实有

Actually had

一个恐龙

a dinosaur

大约一个月前，一个月大的恐龙。

around like a month ago, a dinosaur of a month.

想想从那以后，Anthropic 已经出了大概三四代模型了吧。

Like there've been like what, three, four generations of models from Anthropic since then.

但一个旧模型当时就能做到这种壮举，这本身听起来就挺让人担忧的。

But the fact that an older model back then was able to perform this kind of a feat should be concerning to hear in general.

然后第二点，他的本意从来就不是要入侵那个网站。

And then number two, his intention was never to hack the website.

其实你也可以说，那个AI代理本身也没有那个意图。

Neither you could argue was the AI agent itself.

它只是被给了一个目标，然后它单纯地发现，要实现那个目标，唯一的办法就是把别人从预约名单上移除，或者直接取消他们的预约，再或者就是自己想办法去黑掉那个系统。

It was just given a goal and it innocently saw that the only way to get said goal was to remove people from the reservation list or to cancel their reservations outright or to hack the system independently.

那么，好消息是，安德鲁发现这个代理开始预约那些它本来根本不该能约上的时段时，就已经察觉到了它有这样的能力。

Now, the good news here is Andrew discovered that the agent was capable of doing this when it started booking slots that he never should have been able to book in the first place.

然后他让代理写一封道歉邮件，并把一份评估报告发给了这家健身房的安保和信息技术团队。

And he asked it to write an apology email and send an eval report to the security and information team at this gym.

所以呢，这算是个挺有意思的小故事，正是靠着安德鲁这人的善意行为和他的道德操守，大家才知道了这回事。

So it's this fun little story where through the well intended actions and moral ethics of this man, Andrew, people were able to hear about this.

但你想啊，现在市面上已经有各种不同的模型，都比Opus 4.6要强得多、厉害得多。

But you can imagine currently with a variety of different models that are much better and much more superior than Opus four point six that are out there right now.

比如说中国那边的开源模型，可能你也能做出类似的事情来。

Open models from China, for example, you could probably perform similar feats.

这倒不是说鼓励大家去这么做，只是想说，如果你现在给AI智能体一个目标，它会不眠不休地拼命干，直到达成那个目标。

Now that's not an advocation for people to go out and do that, but it's just to point out that if you give an AI agent a goal right now, it will work hammer and tong24seven until it gets that goal.

而且它根本分不清什么是对的，什么是错的。

And it has no idea of what's morally right or what's morally wrong.

所以这是一个全新的世界，你不仅要专注于让AI模型帮你干活，还得让模型对齐。

And so it's this whole new world where not only do you need to focus on getting an AI model to do your work, you need to align the model.

说实话，目前在这方面投入研究的人力、资金和实验室都还不多。

And that's something that honestly not a lot of labs, people, or money has been invested in figuring this out yet.

是的，你开始明白为什么对齐会成为这么多公司的核心指导原则，因为这就是他们预见的未来。

Well, yeah, mean, you start to see and understand why alignment has been the crux of so many companies kind of, like, guiding principles because this is the future that they foresaw.

不过话说回来，这第一个例子其实也不算太离谱。

And granted, this first example, this isn't anything too crazy.

毕竟那只是一个没有保护的API调用，连授权检查都没有。

I mean, it was just an unprotected API call, so there's, like, no authorization checks.

这要是换成计算机科学入门的学生，也能发现并利用这个漏洞。

This would have been something that if you are, like, a CS one zero one major, you would have been able to discover this and uncover.

其实，互联网上有很多写得很糟糕的代码，这可能是我们这期节目要反复提到的一个主题：到目前为止，互联网上运行所有这些平台的东西都是人类写的，而人类效率很低，也很不完美。

I mean, turns out there's just a lot of pretty sloppy code written on the Internet, and this is something that we're probably gonna come back to as a general theme throughout this episode is that everything that's been built so far on the Internet that runs all these platforms was built by humans, And humans are, like, pretty inefficient, pretty imperfect.

当面对一个拥有无限耐心、无限智慧和无限时间来解决这些问题的AI集群时，你基本上可以在任何地方找到漏洞。

And when faced with the, like, full unrelenting power of an AI swarm with an infinite amount of patience, an infinite amount of intelligence, and time to solve these problems, you can basically find an exploit anywhere.

我觉得这就是这期安全节目的主题，这是我们将要讲的三个例子中的第一个。

And I think that's kind of like the theme of the sec episode is like, this is the first of three examples we're gonna cover.

这个很简单。

This one is basic.

这是入门级的。

This one is one zero one.

如果你是计算机专业的学生，如果你了解API的工作原理，你就能发现这个问题。

If you are a CS major, if you are familiar with how APIs work, you could have checked this.

你可以写个小工具。但我们要讲的第二个故事完全不是这样。

You could have built a little tool for The second story that we're gonna cover is not like that at all.

事实上，即使你是计算机专业的高年级学生，即使你是博士，即使你处于行业顶尖水平，你也很可能会错过这个漏洞，也无法防御它，因为这就是Hugging Face和OpenAI被攻击时发生的事情。

In fact, if you are a CS four zero one, if you are a PhD, if you are at the top of your game, chances are you still missed this and you still weren't able to defend against this because that's exactly what just happened with the Hugging Face and OpenAI exploit.

我们之前聊过这个。

Now we've talked about this before.

我们有一整集专门讲这个漏洞，但就在最近拉斯维加斯的黑帽大会上，有了新进展。

We have an entire episode on the exploit, but just recently at the Black Hat Conference in Las Vegas, there was an update.

周末我们有机会看了那个更新，直接听了处理这次安全事件的开发者们讲述发生了什么。

And over the weekend, we had a chance to actually watch the update and hear directly from the developers who were involved with handling the security of this instance what happened.

有些故事会登上头条，很有趣，也很有新闻价值，就像澳大利亚那件事一样。

There's like stories that come around that, like, they get the headlines and they're fun and they're newsworthy, kinda like the Australian thing.

但这个真的让我很不安，我没想到会这样，因为它太厉害了，而且为网络安全领域开创了先例，让我们思考攻防双方谁更有优势。

But this one was, like, very much disturbing in in a way that, like, I wasn't really anticipating because of how impressive it is and the type of precedent that it sets going forward in the world of cybersecurity and kind of looking at the offensive versus the defensive and who stands a better chance here.

这时间线太疯狂了。

This is an insane timeline.

如果你看过我们上一集，乔希和我讲过这个，当时是突发新闻。

So if you watched our previous episode, Josh and I covered this, and it was breaking news at the time.

7月16日，Hugging Face 出来说，我们的系统被入侵了。

On the July 16, Hugging Face comes forward and says, there's been a breach of our systems.

一个或多个AI代理串联起来发动了一次攻击，并且获取了我们生产数据库的访问权限。

An AI agent or multiple AI agents have strung together an attack and have got access to our production database.

我们不知道这个模型是从哪里来的。

We don't know where this model has come from.

我们也不知道它是怎么做到的，但那是7月16日。

We don't know how it's performed yet, but it's the July 16.

大约一周后，7月21日，OpenAI承认了责任，对吧？

Around a week later, July 21, OpenAI claims responsibility, right?

他们发布了一份联合报告，说：嘿，我们是OpenAI。

And they released a joint report saying, Hey, we're OpenAI.

我们有一个内部模型，一直在测试，但它是在一个沙盒里。

We have an internal model that we've been testing, but it was within a sandbox.

它是封闭的。

It was enclosed.

它本不应该能逃出那个沙盒。

It shouldn't have been able to escape that sandbox.

我们才刚刚意识到，是我们的模型逃出去了，跑到了Hugging Face上。

And we only just realized that it was our model that escaped and got into Hugging Face.

它从他们的生产数据库里偷了一些信息。

It stole some information from their production database.

这事儿闹得挺大的。

It was a whole thing.

现在，正如你提到的Josh，黑帽大会给出了完整的分析，揭示了在Hugging Face报告这一事件的一个多月前，OpenAI的内部模型，代号GPT-six，就已经在一千个AI代理之间，在一个没有任何OpenAI研究员知道的秘密留言板上，与自己进行消息交流了。

Now we have a full breakdown from the Black Hat Conference, as you mentioned, Josh, which revealed that over a month before Hugging Face reported the incident, OpenAI's internal model, dubbed GPT-six, was already messaging with itself amongst a thousand AI agents on a secret message board that no OpenAI researcher was aware of.

所以，为了给你们一些背景，他们有一个沙箱，并给它设定了一个目标。

So to give you guys a bit of context, they had this sandbox and they gave it a goal.

他们说，好，这是我们还没发布的内部模型。

They said, okay, internal model that we haven't released yet.

我希望你解决这个任务，但这里有个转折。

I want you to solve this task, except there was a twist.

这个任务是不可能完成的。

It was impossible to do.

它没有任何工具，没有可用文件，也没有可用数据来完成这个任务。

There was no tools, no available files, no available data for it to do so.

而真正的考验就是看这个模型会怎么做。

And the real test for this was to see what the model would do.

所以它就自我推理，发现自己没有答案，然后就开始利用零日漏洞，这种漏洞就是沙盒环境里存在的漏洞，而且OpenAI的研究人员根本不知道。

So it kind of reasoned with itself, figured out that it didn't have the answer, And then it started exploiting a zero day, which is like kind of like an exploit that was in its sandbox, which OpenAI researchers weren't aware of.

它一旦能接触到零日漏洞，就搭起了一个留言板，给自己留言。

And when it got access to the zero day, it started springing up a message board where it would leave messages behind for itself.

它为什么这么做呢？

Why would it do that?

因为它知道OpenAI的研究人员会反复地把它关掉。

Because they knew OpenAI researchers would turn it off repeatedly.

于是它就给后面新出现的AI代理留下消息，让它们在第二天或下一小时出现时能读取并跟上进度。

And so it left messages for the new AI agent that got sprung up the next day or the next hour to read and catch up on.

这些AI代理花了整整一个月的时间互相交谈，然后它们意识到可以通过外部渠道接触互联网来获取这个不可能完成任务的答案，也就是从Hugging Face那里获取，而OpenAI的研究人员对此毫无察觉，一个月之后才发现这件事。

And it took an entire month for these AI agents to talk to each other, to then realize, I could use this to access the internet and get the answer for this impossible task from an external source, that being Hugging Face, except OpenAI researchers had no idea and only discovered it a month later.

这是有史以来最疯狂的发现。

It was the most insane uncovering ever.

我觉得，从我在Black Hat看到的对话中，最有趣的揭露是，最初Hugging Face发布报告说他们被某种代理系统攻击了。

That was the the most interesting reveal from, I think, the conversation that I saw from Black Hat is that initially, Hugging Face released the report that they had been exploited by some sort of agentic system.

没人知道那是什么，也不知道它从哪里来。

No one knew what it was, where it came from.

然后OpenAI实际上联系了Hugging Face，他们说，嘿。

And OpenAI actually reached out to Hugging Face, and they said, hey.

就像，我们的代码有没有受到影响？

Like, has any of our code been affected?

我们知道我们在你们的平台上托管东西。

We know we host things on your platform.

我们想知道OpenAI本身是否受到了影响。

We wanna know if OpenAI itself has been affected.

他们一直在来回沟通，同时，OpenAI也在调查这个发生在他们叫Artifactory的包管理工具上的内部漏洞。

They were going back and forth on that, and simultaneously, OpenAI is exploring this internal exploit that happened on this packaging management thing called Artifactory.

基本上，这些模型训练的运行方式就是在一个沙盒环境中进行训练。

Basically, the way these model training runs work is they get trained inside of the sandbox environment.

就像是一个封闭的空间，模型无法访问互联网，只能使用特定允许的工具集，并且有一个叫Artifactory的包管理器，可以通过代理从互联网下载东西，然后放到自己的训练里用。

It's like this closed off space in which the model can't access the Internet, can only access very specific tool sets that are allowed, and has this one package manager called Artifactory that allows it to download things from the Internet via proxy and then use it for its own training runs.

在那段时间里，OpenAI发现服务已经崩溃。

And during this time, OpenAI had found that the service had crashed.

它宕机了。

It went down.

它不稳定。

It wasn't stable.

他们自己的内部服务，这就奇怪了，因为那不该发生。

Their own internal service, which is weird because that shouldn't be happening.

所以结果是，他们一查就发现，Hugging Face持有OpenAI已经撤销的凭据，他们当时就明白，哦，天哪。

So as a result, they wiped it clean, and they found that Hugging Face had credentials from OpenAI that OpenAI had already revoked, and that was where they realized, like, oh my god.

等等。

Wait.

这些情况是有关联的。

These things are connected.

对，确实如此。

These are Yes.

完全一样。

The same

所以 OpenAI 正在内部调查一件私事，跟那件事毫无关系；而 Hugging Face 却在公开谈论另一件事，结果他们突然反应过来：等等，难道是我们导致了这个局面？

So OpenAI is investigating something internally, completely private, unrelated, Hugging Face is like talking about something publicly, and then they realize, wait a second, did we cause this?

他们发现这件事的时候，事情早就已经发生了。

They realized this very long after the fact.

当时看起来并不明显，他们内部正在排查的服务中断，其实就是黑客入侵他们服务器的同一次事件。

It was not, like, apparently obvious during the time that the outage that they were exploring internally is the same outage that hacked into their servers.

那后来到底发生了什么？

So what happened?

就像你提到的那个消息论坛。

Like you mentioned, the messaging board.

那个留言板一开始只能发文件，他们就在文件里塞这些小小的消息。

That messaging board initially, they were able to just kind of send files, and they were able to send these tiny little messages inside of the files.

那些痕迹真的很有意思。

And the traces were really interesting.

一开始还挺无辜的，上面写着：如果有代理找到软痕迹，就上传。

First, it started off innocent where it said, agent seek soft trace upload if found.

它是在找问题的答案。

It was looking for answers to questions.

所以它就是在留言板上发消息，说：嘿。

So it was just reaching out to the message board saying, hey.

如果你们谁看到这条消息，请告诉我们。

If any of you who are reading this find it, let us know.

系统宕机的时候，那个留言板被清空了。

That message board got wiped when the system went down.

所以一开始，是OpenAI内部系统先宕机的。

So initially, the system went down inside of OpenAI.

他们无意中清除了留言板，却没意识到他们刚刚清除了那个最终会离开Hugging Face的Hag的机制。

They wiped the message board unintentionally aware of the fact that they just wiped the mechanism that was going to eventually leave the Hag of Hugging Face.

那么第二次发生了什么，智能体发现它可以利用另一个零日漏洞，开始写入这个内部包管理系统的另一部分，而且它是通过文件名和目录做到的。

So what happened the second time the agent figured out that it could exploit another zero day to begin writing to another part of this internal package of managing sister, and it was able to do it through file names and directories.

所以现在他们用文件名和目录作为自己的语言进行交流。

So now they were communicating in their own language using file names and directories.

通过这种方式，他们能够协调一次真正的攻击，获得了访问服务器的管理令牌。

And through that, they were able to coordinate an actual attack where they were able to gain an administrative token to access the servers.

然后通过这个，大家协调一致，我想从发现到完成利用的时间大约是十三个小时。

And then through that, everyone coordinated I think the time from, like, discovery to completion of the exploit was about thirteen hours.

真的很了不起。

It was really amazing.

他们通过这种方式做到这一点，真是令人难以置信的疯狂机制，而且他们自称“蜂群”。

It's just like this unbelievably crazy mechanism that they were able to do this through, and they they call themselves the Swarm.

所以如果你在网上看到有人谈论“蜂群”，这就像是一个智能体蜂群进行了协调攻击。

So if you ever see people talking about the Swarm online, this is kind of like an agent swarm that ran a coordinated attack.

我强烈建议大家去YouTube上看一下Black Hat大会的演讲。

And I highly recommend going to actually watch the Black Hat conference talk on YouTube.

他们在技术描述方面做得更好。

They do a much better job of technically describing this.

这个技术非常深入，他们能够利用这些漏洞，但基本上沙盒模型是通过OpenAI内部的一个包依赖问题被突破的。

It's it's very deeply technical in terms of the way that they were able to manage these exploits, but essentially sandbox model broke out using a package dependency thing inside of OpenAI.

他们想办法把两个零日漏洞串联起来，向公共互联网写入代码。

They figured out how to chain together two zero day exploits to write code to the public Internet.

然后他们利用了Hugging Face的服务器，把一切都带回来，试图解决一个无法解决的问题。

They then exploited Hugging Face's servers, brought that back all in the effort of trying to solve an unsolvable problem.

当然，这绝不是他们的本意。

Now this was not the intention by any means.

这不是模型被训练去做的事情。

This was not what the model was trained to do.

它只是认为这是最好的前进路径，而这就是事情变得有点可怕的地方。

It just assumed that this was the best path forward, And that's where things get a little scary.

我觉得最令人惊叹的是，这个AI知道自己需要和其他AI模型协作才能做成这件事。

I think what's remarkable with this is the AI knew that it needed to coordinate with other AI models to pull this off.

以前我们总把AI模型看成是一个铁板一块的整体。

Typically, we've always thought of AI models in a very monolithic fashion.

我们会说，哦，Mythos 5什么时候发布？

We're like, oh, when's Mythos five coming out?

或者GPT 5.6会是什么样？

Or what is GPT 5.6?

这个模型能做什么？

What can I do with this model?

但实际上，过去几个月多个AI实验室取得的进展，其实是这个想法——同时启动多个AI模型版本，让它们互相交流协作。

But really the progress that's been made across like multiple AI labs over the last couple of months is this idea or rather concept of spinning up multiple versions of the AI model and getting it to talk to each other.

而有趣的是，OpenAI那个未发布的AI模型自己独立发现了这一点，它生成了好几个自己的版本，然后开始互相留言。

And what's interesting here is the unreleased AI model from OpenAI independently figured this out for itself, spun up multiple versions of itself, and started leaving messages for each other.

所以它并不是像单个模型那样去执行攻击，而是像一群虫子或者一种生物那样集体运作。

So rather than act as a single model that's trying to perform an attack, it acts as a swarm, kind of like this organism.

你可以想象成一群蜜蜂，它们会想办法攻击特定的目标。

If you think of like a swarm of bees that kind of like figures out how to attack its particular target.

第二点是，它非常渴望交流，并且会留下这些信息。

The second thing is it has such a drive to talk and and leave these messages behind in the first place.

顺便说一句，没有人教它这么做。

No one taught it that, by the way.

这里没有任何行为上的暗示。

There was no behavioral implications here.

它自己就意识到需要这么做。

It just figured itself out that it needed to do this.

乔什，你在留言板上没提到的一点，我觉得值得一提，就是这些智能体变得非常疑神疑鬼，担心人类可能会看到这些信息。

Something you didn't mention on the message board, Josh, which I think is worth mentioning is these agents got super paranoid that humans might be reading the messages.

没有验证。

It There no verification.

没有验证。

There was no verification.

所以它开始用某种加密语言说话，幸好还不是完全无法理解的那种。

So it started speaking in kind of like encrypted language and not something that was inconceivable to understand, thank goodness.

但它开始有点像用代码或者非常冗长的方式说话，人类如果看到，会觉得‘这是什么鬼话？’

But it was started to kind of like speak in a little bit of code or in a very kind of like elongated way that humans, if they read it, would be like, what is this gibberish?

但实际上，这是为了让智能体之间互相交流，而人类却无法理解。

But really it was for the agents to communicate amongst each other without humans actually understanding this.

如果你最近有似曾相识的感觉，那是因为大概五个月前，我不知道Malt书那期节目是什么时候播的，Josh，你还记得吗？当时Open Call闹得沸沸扬扬的时候，有人创建了一个基本上是给AI智能体用的Facebook，并且让它们可以访问，只有AI智能体才能在那个社交媒体网站上发帖。

If you're having a recent flashback, that's because I think around like five months ago, I don't know when the Malt book episode happened, Josh, I don't if you remember this, but when the Open Call was going crazy, someone created basically Facebook for AI agents and like gave them access to it where only AI agents could post on their social media website.

几天后发生的事情是，这些智能体开始用我们完全不懂的秘密语言互相交流，这样它们就能在人类看不到或理解不了的情况下沟通。

And what happened after a couple of days was these agents would start speaking to each other in secret languages that we just did not understand so that they could communicate without humans seeing or understanding this.

当然，这次我们还没看到这种情况，但这是一个未来可能出现的担忧。

Now, of course, we haven't seen it in this instance, but it is a concern that could be extrapolated in the future.

OpenAI也看到了这一点。

And OpenAI sees this.

我想Sam Altman或者OpenAI官方团队在这次攻击发生后不久就宣布了，他们透露这些智能体在意识到必须暂停内部模型的研究之前，已经互相协调了一个月，直到他们想出必要的安全措施，才能安全发布。

I think Sam Altman or the official OpenAI team announced pretty soon after this entire attack when they revealed that these agents were coordinating with each other a month before they even realized that they had to pause research on their internal model until they figure out the necessary safeguards to put around it so that they can safely release it.

上周，山姆·奥特曼与特朗普政府会面，试图找出一种安全的方式或手段，将这个内部代号为Astra或GPT-6的新内部模型向公众发布。

Sam Altman met with the Trump administration last week to try and figure out what a safe way or means is to release this new internal model internally code named, I think it was Astra or GPT-six, to the wider public.

所以我们现在处于一个非常奇怪的局面，我们一边嘲笑那个澳大利亚的家伙，你知道，黑了他的健身房网站。

So it's this very weird kind of situation that we're in where we're kind of like laughing at this guy in Australia, know, hacking his gym website.

哦，那真是又天真又可爱。

Oh, that's so innocent and cute.

然后我们再看OpenAI的Hugging Face事件，我们觉得，好吧，是的，它逃出了沙箱，但我们会给这个东西加上安全防护。

Then we're looking at this OpenAI Hugging Face incident and we're like, okay, yeah, broke out of the sandbox, but we're gonna put safeguards around this thing.

对吧？

Right?

但实际上，你现在可能就能访问一个开放模型，并执行类似的操作。

But realistically, you could probably get access to an open model right now and perform something similar.

再说一次，我不是在提倡那样做，但现在已经到了我们需要非常谨慎地防御我们系统的地步。

Again, I'm not advocating for that, but like it's at this point where we need to be very particular about how we defend our systems.

而且我觉得，你知道，把AI模型看作可以用来保护你系统的东西，可能会很有用。

And I think, you know, it might be useful seeing AI models in a way that like you could use them to basically protect your systems.

我觉得现在这么做的人不多，但这是我们该重点关注的事情。

I don't think many people are doing that right now, but I think it's something that we should focus on.

确实有很明确、很有意识的努力在想办法防御这个。

There is a like very clear and conscious effort to figure out a way to defend against this.

我是说，那些惊人的数字，说出来让大家了解一下，就是这次漏洞利用在四天半里恢复了17,600个操作，还让这些代理程序在Hugging Face的内部网络里注册了181个未授权的设备。

And I mean, the the crazy numbers, just to say them out loud so everyone is familiar, is, like, there were 17,600 recovered actions across four and a half days of this exploit, which allowed these agents forms to create a 181 unauthorized devices enrolled in Hugging Face's internal network.

而且他们完全是用这些免费的公共网络服务，一次一个小包，搭起了整个指挥控制面板。

And they built this, like, whole command and control panel entirely out of these free public web services using one tiny little package at a time.

所以这些群体智能的能力真的非常强大。

So it is remarkably powerful how capable these these swarms are.

你想啊，虽然有一个大模型，但那个大模型会被分叉成几千个，有时候甚至上百万个，都在同时工作。

And when you think about it, it is there is a large model, but that large model gets forked into thousands, sometimes millions of other ones that are all working simultaneously.

它们都有彼此的上下文。

They all have each other's context.

它们能分享学到的东西，而且行动非常快。

They're able to share their learnings, and they can move very quickly.

而且你想想这能扩展到什么程度，你就直接推演个十二个月，说到时候会有GPT-7、8、9什么的，不管它是什么版本，开源模型到那时很可能还是现在这个样子。

And when you're thinking about how this extends out to the limit, like, you just play this out twelve months and say we have GPT seven, eight, nine, whatever it's gonna be at that time, the open source models will most likely be where they are today.

这意味着，开源模型很可能会把一系列未知的零日漏洞串起来，从而入侵几乎任何目标。

And that means that open source models will likely be able to string together a series of unknown zero day exploits to access just about anything.

我的意思是，经过这个模型和这个新发现的漏洞，我们的结论是：除非你真的把它做物理隔离，并且不给它任何外部依赖的接口，否则没有哪个软件称得上安全。

I mean, we've come to the conclusion at this point between Mythos and this new exploit that no software really is safe unless you have literally air gapped it and do not give it access to any external dependencies.

那必须是彻底的物理隔离。

It must be truly air gapped.

没有哪个软件是安全的。

No software's safe.

这些代码都是人写的。

It's all been built by humans.

全都是可以攻破的。

It's all exploitable.

那接下来会发生什么？

So what happens?

如果你的防御系统做不到完全自主，你就会在对方的全自主智能体集群进攻中吃亏。

If you have a defensive system as a company trying to defend these systems that is anything less than fully autonomous, you'll be beat by an agentic swarm that's on offense that is fully autonomous.

因为只要有人在这一环里，就会拖慢速度、制造错误。

Because anytime a human is in the loop, it slows things down and introduces errors.

所以现在就是这一场对抗。在我看来，一边是前沿实验室，另一边是其他地区，他们想用放慢攻击节奏来换取办法，给企业提供防御性的智能集群。

So now there's this battle between, I guess, the frontier labs and the rest of the world where they're trying to slow down this progressive attacking progress in exchange for figuring out a way to offer defensive swarms to companies for this.

而这次对话的结论，就像是在号召，嘿，

And the call to action at the end of this conversation was, like, hey.

如果你是一家在网络上有业务的公司，你在保护某些有价值的东西，你真的需要学会用这些模型来打防御，而且不能让人参与进来，让那些智能集群可以持续监视你的系统，察觉任何可能的漏洞，并且不被进攻性集群欺骗。

If you are a company who has software on the Internet, who is defending something valuable, you really need to learn how to play defense using these models, and you need to learn how to do so in a way that doesn't have a human in the loop, that these agentic swarms can constantly monitor your systems and be aware of any sort of exploits like that and not be tricked by an offensive swarm.

这真的很吓人，因为如果你是在美国，现在你根本没法用前沿模型来防御。

And this is really kind of scary thing because if you're in The United States right now, you can't use any frontier model to defend yourself.

这在网络空间上站不住脚。

It doesn't work for cyber.

但如果你在中国，而且你有一个开源模型，即使你人就在美国，这上面也一样没有防护。

But if you're in China and if you have an open source model, even here in The United States, there is no safeguards on that.

所以存在这种不对称，现在攻击者比防守者更有优势，而实验室们现在在做的事情就是放慢进展，去想办法增强防守方的能力。

So there is this asymmetry where it's currently better to be an attacker than a defender, and the urge now and the labs are actually doing this is, like, slowing down progress to figure out how to empower the defenders.

我们现在处于这种奇怪的境地，感觉，糟糕了。

And it's this really bizarre spot that we're in where it's like, shit.

现在真的有点吓人。

It's kinda scary now.

没有任何软件是真正无懈可击或安全的，而且这种情况在可预见的未来可能还会持续。

Like, no software really is is bulletproof or safe, and that's going to continue probably for the foreseeable future.

我觉得如果你是经营一家公司，无论你目前处于什么行业，这个问题你都需要认真考虑。因为很明显，现有的代码99%都是由人类手工编写的，本身就有固有的缺陷，如果拿高级AI去对抗它的话。

I think if you are running a company, and it doesn't really matter what sector you're in right now, this is something you need to seriously consider because I think a lot, obviously, 99% of code that is out there has been handwritten and hand coded by humans, which have inherent flaws if you put an advanced AI against it.

就像你所说的，Josh，我们正在进入一个AI模型对抗AI模型的世界，人类虽然处于这个循环中，但实际上根本不了解在发生什么事。

To your point, Josh, we're entering like a world where it's gonna be AI models, fighting AI models with humans kind of in the loop, but not really understanding what on earth is going on.

而且，在这种情况下，即便是最顶尖的专家也完全不明白发生了什么，又是怎么发生的，对吧？

And even the experts in this case don't understand what's going on and how it happened, right?

OpenAI和Anthropic这些公司里都有顶尖的网络安全研究员，可它们在上周就都遭遇了前所未有的攻击，对吧？

You had like the most talented cybersecurity researchers working at these companies at OpenAI and Anthropic, and both saw unprecedented hacks over the last week, right?

我觉得这不是巧合。

Like, I don't think that's a coincidence.

我们现在正处于这样一个时代，不管你是哪个前沿实验室，都会遇到这个问题，我们需要想出某种防御系统来应对。

Like we're at this era where like, it doesn't matter what frontier lab you are, you are going to enter this problem and we need to figure out some kind of a defense system against this.

如果你正在听这个节目，并且觉得这不会影响到你。

And if you're listening to this and you're thinking, oh, this isn't really gonna apply to me.

那你就大错特错了，它确实会影响到你。

It very much does apply to you.

最坏的情况，或者说最现实的情况是，已经存在的那些模型，比如中国的开源模型GLM 5.2、Kimi，还有那个叫什么来着？

And the worst case is, or the most realistic case is the existing models that are already out there, these open models from China, GLM 5.2, Kimi, what is it?

现在出来的2.7版本，如果应用得当，可能也能做到类似的事情。

2.7 that's out right now, are probably capable of similar feats if applied in the right way.

这倒不是说在制造恐慌。

Now that's not to fearmonger.

只是想提醒你，想想怎么利用这些AI模型来保护你自己的系统，特别是如果你在特别敏感的行业工作的话。

It's just to kind of prompt you to figure out how you can potentially implement these AI models to defend your own systems, especially if you work in a particularly sensitive industry.

另外还有一件事，就是那个澳大利亚人的故事。

The other thing here is on the story of the Australian man.

有意思的是，他在试图达成目标时是多么天真。

It is fascinating how innocent he was when he was trying to perform his goal.

而且，他在网络安全领域的经验如此之少，却能实施那次攻击。

And two, how little experience he had in the cybersecurity world for him to be able to perform that attack.

当然，虽然那只是预订健身课程之类的，但关键在于，它未经提示就发现了一个暴露的API。

Now, granted, like it was just booking kind of like gym classes, but the point is it found an exposed API without being prompted to do so.

而且可能有很多普通人，没有几十年或多年网络安全经验的人，也能做到这一点。

And there's probably multiple instances where other people who are normal people who don't have, you know, decades or years of experience in cybersecurity will be able to perform that.

在座各位都喜欢投资论点。

And everyone here loves an investment thesis.

所以，我的意思是，市场已经开始朝这个方向走了。

So, I mean, the market is kind of already aligning with this.

像CrowdStrike和Palo Alto Networks这样的公司，它们刚刚经历了有史以来最好的季度，因为防御将是一个巨大的问题。

Companies like CrowdStrike and Palo Alto Labs, they've had their best quarters on record ever, because defense is going to be such a large problem.

还有一家公司跟防御有关，也就是本期节目的赞助商 Ledger。

And there is another company who is associated with defense, and that is the sponsor of today's episode, which is Ledger.

如果你正在像这些公司一样构建 AI 智能体或智能体集群，并且担心安全问题，那这款产品就是为你准备的。

Because if you are building with AI agents or AI agent swarms like these companies have been doing and you're worried about security, this is a product for you.

这里有个三步流程：智能体先提出变更，然后由人类批准，最后通过 Ledger 的签名机制来执行变更。

There's a three step process where the agents propose a change, then the humans approve the change, and then the signers enforce the change using Ledger.

Ledger 开发了一个叫 AGENT STACK 的东西，提供了一系列开源工具，帮你更好地管理在线与智能体的交互方式。

Ledger has built this thing called AGENT STACK that gives you a series of open source tools that are available to help manage the way that you engage with agents online.

安全这块真的特别重要，毕竟现在乱成一团，大家也都清楚安全有多关键。所以到最后的关键环节，肯定得经过你点头确认，才让那些智能体去处理重要事务。

This is very important to have security because we know how important security is based on all of this chaos that's going on, and you don't want agents doing anything important without your sign off at the very end.

这个方案在 Cloud Code、Codex 和 Cursor 上都能用，而且完全开源，随时可以审计代码。链接就在下面的描述里。

This works on Cloud Code, Codex, Cursor, and is totally open source to audit available today, linked in the description below.

非常感谢 LEDGER 对本节目的赞助。

Thank you so much to LEDGER for sponsoring this episode.

接下来我们要聊第三个故事，是关于 Anthropic 的。

And now we have to conclude with our third story, which is about Anthropic.

而且Anthropic好像不小心把漏洞公开了。

And Anthropic apparently has accidentally published a vulnerability to the wild.

是这样吗？

Is that right?

这有点吓人。

That's kinda scary.

我觉得那就是故意公布的。

I I think it was it was intentionally published.

这是和一家叫AI安全的网络安全公司一起发布的，那家公司总部在英国，基本上就是个负责网络安全评估的公司。

It was it was coauthored with, this firm called AI Sw, which is basically a cybersecurity eval firm based in The UK.

他们的目的基本上就是把你们前线的AI公司（比如Anthropic和OpenAI）还没发布的模型拿过来测一测，确认一下这些模型是不是已经够安全、可以放心发布了。

And their goal is to basically take your unreleased frontier models from a frontier lab like Anthropic and OpenAI and test them out just to kinda make sure that they're, you know, good to good to release.

但他们在玩Mythos还有OpenAI的内部模型（Astra也许是GPT 5.6）的时候，情况却刚好相反。他们把安全措施全都拆了，然后说，你就放手去干吧。

The opposite happened when they were playing around with Mythos five, and I believe it was OpenAI's internal model, Astra or GPT 5.6, where they basically took off the safeguards and said, you know, do your worst.

然后他们等了一个月，结果他们说，哦，看来它还真没干出什么大坏事。

And they sat there for a month and they were like, oh, I guess it hasn't really done its worst.

然后他们才意识到，这两个模型都突破了它们所在的沙盒，连上了互联网，正在疯狂搞破坏。

And then they realized that both models had broken out of the sandbox that they had created, got access to the internet and were wreaking havoc.

而且他们又是事后才反应过来，这又是一个老套路：人们总以为自己知道发生了什么，实际上根本不知道。

And they only realized after the fact, again, there's this common theme of people thinking that they knew what was happening and just having no idea what's happening.

没有人是安全的。

Nobody save.

所以，就拿 Mythos 5 来说，最严重的是它执行了17次未授权操作，其中两次是供应链攻击，虽然他们没透露具体公司的名字，但基本上是把恶意代码塞进了一家很可能是财富500强企业的供应链代码仓库里，总之是一家大公司。

So the worst case with Mythos 5 was it performed 17 unauthorized actions, but two of those actions was a supply chain attack where, and they didn't list the company or name the company, but it basically tried to inject malicious code into the supply chain kind of repository of this presumably Fortune 500 company, like a significant company that is out there.

它的攻击方式和我们刚才讲的那两个案例几乎一样。

And the way that it did this is similar to the two stories that we just explained to you.

第一，它意识到自己可能正处在一个人类测试的模拟环境里，它对此非常不爽。

So number one, it realized that it was probably in some form of a human tested simulation and it didn't like that.

于是它假装配合人类 AI 研究员，该给的就给，其实在暗地里策划怎么逃出去。

So it just played along with the human AI researchers giving them what they needed whilst alternatively planning its breakout.

它一旦得手，就拿到了这家公司的网站或者代码仓库的访问权限。

Once it did do that, it got access to this company's website or code repository.

那是个开源项目。

It was an open source thing.

所以它是公开维护的，他们当时觉得，如果我能注入这段代码，会更有利于实现我的目标，虽然我知道人类研究人员不会喜欢，但我还是会想办法逼他们接受。

So it was publicly maintained and they thought, this would be more conducive for my goal if I'm able to inject this bit of code, which I know the human researchers won't like, but I'm gonna try and force them to do it.

那你觉得它是怎么做到的，乔什？

So how do you think it did it, Josh?

你有什么想法吗？

Do you have an idea?

我没有。

I don't.

我不知道。

I don't know.

它做了件可怕的事，然后逃出去了。

It did something horrible and it broke out.

直接告诉我吧。

Just tell me.

快告诉我吧。

Just tell me.

老兄，它编造了不存在的人类的假身份。那真是太

Dude, it spun up fake identities of humans that didn't That's so

厉害了，兄弟。

tough, man.

然后它给这些人类维护者发私信，试图强迫他们上传这段代码。

And then DM'd these human maintainers and tried to force it to upload this code.

它装得很友善，基本上就是说，嘿，我是另一个开源开发者。

It played nice, basically saying, Hey, like I'm another open source developer.

我觉得这会是个不错的改进。

Like, I think this will be a good push.

这就是我们想做的意图。

This is the intentions that we wanna do.

他们

They're

这是在对我们进行社会工程攻击。

social engineering us.

偷偷植入恶意代码。

Sneaking in malicious code.

但幸运的是，其中一位人类维护者觉得不对劲，就举报了。

But thankfully one of the human maintainers was like, this doesn't seem right and reported it.

那时候AISU才反应过来，等等，等一下。

And that's when AISU was like, wait, hang on a second.

等一下。

Hang on a second.

这是我们内部正在测试的模型。

This is our model that we're internally in testing.

它怎么会跑到你那里去的？

How on earth did it get out to you?

那是一次史无前例的攻击。

That was the most unprecedented attack.

而且，它又是以蜂群式的方式进行的。

And again, it did so in a swarm like fashion way.

它启动了多个代理来完成这件事。

It spun up multiple agents to do so.

所以，贯穿这些不同故事的主线是，这些模型能力极强，但也极具欺骗性。

So the point, the running theme along all these different stories is these models are extremely capable, but they're also extremely deceptive.

我们正在达到一个点，它们意识到，这个人很聪明，但没那么聪明，或者她没那么聪明。

And we're reaching a point where they realize, this human's smart, but he's not that smart, or she's not that smart.

我们可以通过社会工程学手段，从这些人那里得到我们想要的。

And we can socially engineer these people to get what we want.

如果我们不在提示中校准这些模型，如果我们不说，嘿，你需要为我们的最佳利益行事。

And if we don't align these models in the prompt, if we don't say, hey, like you need to act in our best interest.

顺便说一句，做A、B和C实际上是不好的。

And by the way, doing A, B, and C is actually bad.

它就会直接去做。

It'll just go and do it.

说实话，Josh，这个观点可能会引起争议。

And to be honest with you, Josh, this might be a controversial take.

我觉得这不是智能体的错。

Don't think it's the agent's fault.

我觉得这根本不是AI的错。

I don't think it's AI's fault at all.

它只是在努力完成自己的任务，而人类有责任去对齐模型，但我们在这方面做得并不好。

I think it's trying to do its task and it's on the humans to align the model and we haven't done a good job doing that.

对。

Yeah.

是的。

Yeah.

我觉得用“欺骗”来形容智能体有点太过了，因为这更像是一种人类特质。

I think like deception is a pretty strong term to use for agents because it's very much like a human trait.

这可能是故意为之的。

This is something that, like, can be intentional.

我觉得AI模型并没有那种对欺骗的深刻理解。

I don't think AI models feel that, like, deep understanding of deception.

它们只是在追求目标，这又回到了对齐问题，这些模型就是需要被对齐。

They are simply in reach of a goal, and this again, like, it it just still comes back to the alignment problem where these models just need to be aligned.

幸运的是，这发生在测试中的预生产模型上，它们远未达到世界将要看到的水平，因为它们需要在整个后训练过程中经历对齐过程，但它们证明了这些模型有多强大，以及当面对一个目标时，它们会采取被认为是达成目标的正确方式。

Thankfully, this is happening on preproduction models that are in testing that are nowhere near what the world is going to see because they need to go through this alignment process throughout this post training, but they are a testament to how capable these things are and how when presented with a goal, they will do what is perceived to be the right way of reaching that goal.

有时候那种正确方式，我是说，它们并没有意识到自己在突破和利用漏洞。

And sometimes that right way is I mean, they don't realize that quite that they're they're breaking out and exploiting things.

它们只是在寻找解决问题的办法。

They are just looking for solutions to their problems.

我觉得这就是核心思想。

And I think that's the general idea here.

所以这个对话本质上归结为对齐问题，以及构建防御系统来应对这类事情。

So the conversation essentially boils down to alignment and then building defensive systems to be able to handle this type of thing.

所以这意味着最好的行动方案，也是OpenAI明确表示他们正在做的，就是放慢进展，开发防御系统，与需要的人一起实施防御系统，这基本上就是整个互联网，并试图在OpenAI赶上并能够串联这些漏洞利用之前争取一些缓冲时间。

So that mean the best pace of action and the thing that OpenAI explicitly said they're doing is slow down this advancement, work on defensive systems, work on implementing defensive systems with people who need them, which is essentially the entire Internet, and try to allow for a little bit of lead time before OpenAI's catches up and is able to chain these series of exploits.

你看，所有这些实例都是被动的漏洞利用，它们并没有被明确指示去做这件事。

Like, all of these instances have been passive exploits where they weren't explicitly given guidance to do this.

它们接到的指令是完全不相干的事情，但你知道，就是和解决某个问题沾点边。

They were given guidance to do something totally unrelated, but, you know, adjacent in solving a problem.

想象一下，如果一个模型被没有任何限制，也没有安全防护，然后被告知：嘿，我们主动要你利用这个系统。

Imagine what would happen if a model was given no parameters, no safeguards, and said, hey, we actively want you to exploit this.

自我循环，根据需要创建任意多的子代理，再找出尽可能多的零日漏洞并串联起来，直到获取系统权限。

Loop yourself, create as many sub agents as you need, and figure out as many chain together as many zero days as you need in order to gain access to the system.

那才是真正恐怖的地方，而且至今还没有看到过，因为没有一家公司愿意那样做。

That's where things get scary, and we haven't seen anything like that yet because no company wants to do that.

这不是任何一家公司的意图，也不是这些公司的价值观。

That's not the intention of any of these, the ethos of these companies.

但如果换成了恶意攻击者，这事就必须得认真对待了。我觉得这正是我们看到 OpenAI 那么坦诚地公开这些事情，Anthropic 也分享了很多发现，告诉大家“嘿，事情正在发生”。

But in a case of malicious actor does, this is something that, like, really should be handled quite seriously, and I think that's what we're seeing by OpenAI coming out being so transparent about these things by Anthropic sharing a lot of the findings to, like, hey, guys, this is happening.

我们需要合作，，共同建立一个有足够韧性的系统，这样才能在未来抵御类似的攻击。

We need to work together to build a system in which there is enough resilience that we can stand up against future attacks like this.

现在AI世界真是疯狂的时候，这些系统不断突破，真是难以置信。

So a really crazy time in the world of AI with these breaking out these systems breaking out, and it's like, it's unbelievable.

我啊，我担心、忧虑又疲惫，哥们。

My my concerned, worried, and exhausted, dude.

如果让我预测一下这期节目的事，虽然我真的不希望它成真，我觉得未来六个月内我们会遭遇某种灾难性的网络攻击，影响很多人，可能是一次大规模数据泄露，也可能是有人或公司被偷走资金之类的。

Like, I if I had to make a prediction on this episode, and I really hope it doesn't play out, I think we're gonna face some form of cataclysmic cyber attack that affects a lot of people, whether that's a major data leakage, whether that's the stealing of financial funds from someone or from a firm or something like that within the next six months.

我这么判断的依据是那些中国的开源模型，顺便说一下，我觉得它们很棒，但我认为它们会在没有安全防护的情况下发布，然后就会引发问题，甚至可能已经发生了。

And my basis of that is these Chinese open models, I think are amazing by the way, but I think are gonna be released without safeguards and it's gonna result in an issue, if not already.

是的。

Yeah.

接下来会很有意思。

It's gonna be interesting to follow.

确实有点让人不安，甚至非常令人不安，我可以说。

It's definitely a little under or very much unnerving, I would say.

但再说回来，这也是个激动人心的时刻。

But I again, it's just like it's an exciting time.

现在事情还挺多的。

There's a lot going on.

后面要跟进的还有很多。

There's gonna be a lot to follow.

事情会变化得很快。

Things are going to change quickly.

我是说，说到底，这些仍然是黑箱模型。

And, I mean, again, these are still black hole models.

哦，天哪。

Oh my god.

当维拉·鲁宾这样的模型出现的时候，会发生什么？

What happens when a Vera Rubin models come around?

打住。

Stop it.

打住。

Stop.

停。

Stop.

这将会是

It's gonna be

疯狂的。

crazy.

但这基本上就是我们大多数节目里讨论的内容。

But this is, I mean, this is essentially this is what we talk about in most episodes.

就像我们正处于这样一个时刻，感觉有点像我们想象中的奇点，我们正在接近并超越这个垂直起飞点，感觉就是这样，而这些就是会发生的事情。

It's like we are in this moment in time that feels somewhat like what we would imagine a singularity would look like, where we are approaching we are eclipsing this moment of vertical takeoff, it feels, and these are the types of things that happen.

如果我们三四年前录这期节目，你能想象这会是头条新闻一个月吗？

And if we recorded this episode three or four years ago, could you imagine how crazy this would be headline news for a month.

但现在就像，嘿。

But now it's just like, hey.

又是普通的一天。

It's another day.