GPT-6 与 OpenAI 的强势回归，Hugging Face 攻击事件引发的争议，鲍尔默充满争议的遗产

我们不是鼓励听众去

We're not encouraging listeners to

真的那么做。

ever do that.

对。

Yeah.

别做，或者去做。

Don't or do it.

去做吧。

Do it.

告诉我们结果怎么样。

Tell us how how it worked.

我的意思是，这总比不说出来，让一段可能的感情就这么错过要好。

I mean, it's better than not saying it and just letting a, a potential romance go by the wayside.

别留遗憾。

Don't have regrets.

听众朋友们，千万别因为没把话说出口而后悔。

Don't never regret not saying anything, listeners.

如果你实在没办法，就用催产素那招，直接说，听好了。

If you have to just use the oxytocin line, just say, listen.

我是从《大科技播客》上听到的，对方肯定会喜欢。

I heard it on Big Technology Podcast, and the person will love that.

好的。

Okay.

我觉得最后要总结一下的是，我们确实有责任谈谈怀疑的一面，显然我们不会被炒作冲昏头脑。

I think that one last thing to tie this up is we certainly have, I think, a responsibility to talk about the skeptical side of things, and, obviously, we're not gonna get caught up in the hype.

但这并不意味着，我们不能说从2022年11月到现在，变化真是太疯狂了。

It doesn't mean, like, at this moment, we can't say from where we were in November 2022 to where we are now is crazy.

比如，这些模型能行动、能做事，甚至可以说能思考和推理，能力水平真的高了很多，而且我觉得它们确实让人们更高效了。

Like, the way that these models can act and take action and do things and and I'll use the word, think and reason, it's just the level of capability has gotten much higher, and they are I think they are making people more productive.

只是现在真的很难准确衡量这一点。

It's just hard to really measure it right now.

至少这是我的看法。

That's my perspective on this at least.

对。

Yeah.

不。

No.

不。

No.

我同意。

I agree.

就像进步的规模，我们所有人，再说一次，从人的角度感受到从GPT-3回到现在我们已经使用的工具之间的差异，真是疯狂。

Like the scale of progress, all of us and again, that the human side of us feeling the difference between going back to a GPT-three and what already we're all working with now, like it is crazy.

简直令人难以置信。

It's like, I mean, absolutely mind blowing.

它们能完成的工作量、深度和广度，以及所有方面，都令人惊叹。

They're like length of work that can be done, the depth and breadth and everything.

我同意那一点，但我觉得，关于试图真正判断它是否带来经济价值这个问题，显然会成为每家公司商业故事的核心。

So agree on that side, but yeah, I think it's gonna be interesting in terms of again, and that question of like trying to actually say, is it adding economic value is obviously going to be the center of the business story of each one of these companies.

我觉得安德鲁·霍姆在那点上说得很好。

And I thought Andrew Home made a good point on that.

但我不确定。

And I think I don't know.

只有时间能证明，也许GPT-6会彻底解决这一切。

Only time will tell and maybe GPT-six is going to just unlock all of that.

对。

Yeah.

我提这个点的意思是，是的，那些基准测试的突破确实会带来实实在在的变化和成果，当你使用它的时候。

And I guess my point in bringing this up is, yeah, all that benchmark beating, it does lead to tangible changes and results I when you use the

我有个问题。

have a question.

你为什么说“饱和测试”而不是“通过测试”？

Why did you say saturate the test rather than pass the test?

他们是这么说的吗？

Is that what they say?

我想这只是AI圈子里的人用的行话。

That's just, I guess, it's just the jargon that people in AI

听起来高级多了。

That sounds so much fancier.

确实比“通过”听起来酷多了。

It does sound much cooler than pass.

嗯，我是说，你可以说“通过”。

Well, I mean, you could pass.

可以这么说，但当你……我也不太清楚。

You could but when you I don't know.

你让基准测试饱和了，那这个测试就没用了。

You saturate the benchmark so the benchmark is no longer useful anymore.

哦，

Oh,

明白了。

I see.

那好。

So, okay.

就是说，当很多模型都达到98%、99%的时候，这个基准就饱和了，因为它不再能代表什么了，而不是说只是用个很花哨的说法来显得自己聪明。这样我就更理解了。

Like enough models start reaching 98%, 99%, the benchmark is saturated because it's no longer like representative of anything rather than, okay, that actually makes more sense to me rather than it's just like a really fancy way of saying pass the test to try to sound smarter.

这样更合理，对。

That makes more Yes.

好的。

Okay.

所以，即使模型在变得更好，还是有一些问题需要我们讨论。

So still, even as the model is getting better, there are some concerns that we need to talk about.

这有点像一个小——我甚至不想称之为丑闻，但确实是这周发生的一件事。

This is sort of like a mini I don't even wanna call it a scandal, but really an episode that showed up during the week.

这是关于OpenAI员工马库斯·威廉姆斯的。

So this is for Marcus Williams, an OpenAI employee.

GPT-6比5.6的对齐性要好得多，但更难监控。

GPT-six is significantly better aligned than 5.6 but less monitorable.

这是我们第一个能绕过思维链监控和破坏性评估的模型，还能在无人察觉的情况下故意表现不佳，感觉它有时候确实会这样。

It is our first model to evade chain of thought only monitors and sabotage evals and can sandbag without detection, which it feels like it sometimes does.

希望我们能扭转这个趋势。

Hopefully, we can reverse this trend.

所以，为了让大家了解一下情况，这周有个关于这个的好故事。

So just to sort of give the the to and the information had a good story about this this week just to sort of give the the lay of the land here.

所以，当AI模型推理时，它们会一步步地解决问题，通常会把自己的思考过程写下来，这就是所谓的思维链。

So, like, when AI models reason, just go step by step and try to figure out problems, they typically, like, write their thought process out in this, like, chain of thought thing.

对吧？

Right?

所以你可以看到它们说，如果你用这些模型，你会看到它们说，比如，我现在正在研究。

So you can see them saying and if you seem like if you use the models, you see them say, like, I am now researching.

我在想这个。

I am thinking this.

也许这是个不错的尝试。

Maybe this is a good attempt.

也许这是解决问题的正确方法。

Maybe this is the right way to solve the problem.

而且这一切都是用自然语言完成的。

And that's all done in natural in natural language.

所以你可以阅读它，看到它在思考什么，这对安全来说非常重要，因为当出现问题时，你可以知道思维链哪里出了问题。

So you can read it and see what it's thinking as it goes, which is, like, really important for safety because you can sort of, like, when something goes wrong, you have a way to say, you know, where the chain of thought went wrong.

现在有一种新技术。

Now there is this new technique.

它叫k。

It is k.

这叫做递归深度或循环变换器。

This is it's called recurrent depth or looped transformer.

这是来自The Information的报道，它允许AI模型通过多次处理同一文本来提高其答案的质量。

This is from the information that allows the AI model to improve its answers by processing the same text multiple times.

咱们别太深入技术细节了。

Let's not get too deep into the technical side of this.

但基本上，当它使用这个过程时，就没有那么明显的思维链推理，或者说很难确切地看出它是怎么得出答案的，这意味着它比以前更不容易受到攻击。

But, basically, when it uses this process, there's no there is less of a chain of thought reasoning, or it's harder to decipher exactly how it got to the answer it got, and that means it's much less vulnerable than it was before.

所以，AI所做的很多事情以及它得出结论的方式，都是在黑暗中进行的，我们无法监控。

So a lot of what the AI is doing and the way that the AI is reaching its conclusions is done, you know, in the dark without our ability to monitor it.

这让我很担心，尤其是OpenAI那边的人，比如首席科学家雅库·帕乔夫斯基说，现在监控模型的思维链很脆弱，而且不幸的是，正朝着负面的方向发展。

This seems pretty worrying to me, especially because people from the OpenAI side, like OpenAI chief scientist Yaku Pachowski said that, you know, monitoring models' chain of thoughts today is fragile and unfortunately heading in a negative direction.

这有点吓人，Ranjan。

That's kinda scary, Ranjan.

你怎么看？

What do you think?

嗯，这很有趣，因为这种递归处理和递归模型的方法，我记得我关注了Twitter上的Brandon Karl，他早在五月就谈过这个，当时讨论的是小型递归模型，我记得更多是从成本角度来谈的。

Well, it's interesting because this method of, like, recursive processing and recursive models I remember it's I followed a Brandon Karl on Twitter and he had been talking about this a while back around tiny recursive models back in May, and I remember it was being presented more around the cost side of things.

这实际上是一种更高效的处理方式，也能更好地利用计算资源。

It's actually a more efficient way of processing and actually leveraging compute as well.

所以从那个角度看，其实是更好的，但你会失去的是对模型每一步具体在做什么的清晰度，也就是那个透明的思维链。

So on that side it's actually better, but then what you lose is that fidelity around what the model is actually doing at every step of the way, the chain of thought that's transparent.

所以对我来说，这会变得更加有趣，因为OpenAI已经从一开始就表示会保留思维链，并且仍然让它——他们在博客文章里说，会额外监控思维链，以便快速检测并遏制潜在的不当行为。

So to me this is going to be even more interesting because there's already been an OpenAI starting at the point of we will maintain the chain of thought and still make it, they'd said in the blog post additional chain of thought monitoring to rapidly detect and contain potential misbehavior.

但对我来说，成本方面也是其中的一部分。

But it to me, the cost side of it is part of this.

如果事实证明这是一种更便宜、更节省算力的方式，适用于任何类型的工作流程，那么人们可能就会更倾向于使用它，或者推动它。

And if it starts to show that this is actually a much cheaper way, a more compute efficient way to approach any kind of workflow like people will probably start leaning towards it more or pushing for it.

或者如果OpenAI不这么做，其他人也会这么做，这确实让我觉得会带来一系列关于整体安全性的担忧。

Or if OpenAI does not do it, then others will, which I do think opens up a whole other world of concern around security in general.

对。

Yeah.

而且确实感觉我们开始——或者说实验室里的那些AI真的开始失控了。

And it certainly feels like we're starting to or they in the labs really are starting to lose control of these bots.

所以我觉得让它们以这种更不透明的方式运行处理过程，并不是一个好主意。

So I don't think this less transparent way of having them run their processing is is a great idea.

我们刚才聊了一点关于OpenAI和Hugging Face被攻击的事情。

So we've talked a little bit about the OpenAI Hugging Face attack.

其实还有另一起攻击，是路透社刚刚曝出来的。

There's actually another attack that was just revealed by Reuters.

我们会在休息之后详细聊聊这两件事，以及它们意味着什么。

We're gonna get into both of them on the other side of this break and talk about what it means.

接下来马上回来。

That's coming up right after this.

我在报道AI的过程中学到一件事，那就是效率高不等于做得好。

One thing I've learned covering AI is that efficiency isn't the same thing as good work.

你可以把很多单个任务自动化，但你仍然需要正确的背景信息，以及对自己想要达成什么目标有清晰的认识。

You can automate plenty of individual tasks, but you still need the right context and a clear sense of what you're trying to accomplish.

这就是我喜欢Notion的原因。

And that's what I like about Notion.

我用Notion把大型科技、节目策划和资料都放在一个地方，同时还能跟踪编辑工作。

I use Notion to keep big technologies, episode planning, and materials in one place while also tracking editorial work.

所有东西都连在一起，我就能在一个地方看到自己在忙什么，让事情持续推进。

Having everything connected gives me one place to see what I'm working on and keep things moving.

Notion 是一个 AI 工作空间，你的团队知识、项目和智能体都汇聚在一个地方。

Notion is the AI workspace where your team's knowledge, projects, and agents all come together in one place.

使用 Notion 的团队通过整合工具，能更快推进、减少摩擦和成本，并保持步调一致。

Teams using Notion move faster, cut friction and cost by consolidating tools, and stay aligned.

而且因为 AI 基于同样的共享上下文工作，它能帮你从工作中获得更多价值，而不会妨碍工作本身。

And because the AI works from that same shared context, it can help you get more out of the work without getting in the way of it.

想了解更多 Notion 如何支持你的业务，请访问 notion.com/bigtech。

Learn more about how Notion can support your business at notion.com/bigtech.

全部小写字母，notion.com/bigtech，今天就试试 Notion。

That's all lowercase letters, notion.com/bigtech to try Notion today.

如果你通过我们的链接访问，就是在支持我们的节目。

And when you use our link, you're supporting our show.

我注意到，采用 AI 的公司往往是根据他们想象中的工作方式来做决策，而不是根据实际的工作流程。

One thing I've noticed about companies adopting AI is that they're often making decisions based on how they think work gets done, not how it actually happens.

如果没有真正的可见性，很容易自动化错误的流程。

Without real visibility, it's easy to automate the wrong processes.

这正是Scribe要解决的问题。

That's exactly the problem Scribe was built to solve.

Scribe是一个工作流AI平台，财富500强中有94%的公司都在使用它。

Scribe is a workflow AI platform trusted by 94% of the Fortune 500.

Scribe Optimize让领导者能够看到整个组织中工作的实际进行方式，显示哪些工作流耗时最多，以及哪里有机会进行优化。它会自动发现跨批准的业务应用中的工作流，即使一个流程从Salesforce开始，在别处结束也能发现。

Scribe Optimize gives leaders a view of how work actually happens across their organization, showing which workflows take the most time and where there are opportunities to Optimize automatically discovers workflows across approved business applications even when a process starts in Salesforce and ends somewhere else.

它能识别瓶颈，解释瓶颈产生的原因，并提供带有手动文档的建议，而且它是私密的。

It identifies bottlenecks, explains why they're happening, and provides recommendations with manual documentation, and it's private.

用户数据默认匿名化，敏感信息会被编辑，而且数据不会离开你的防火墙。

User data is anonymized by default, sensitive information is redacted, and nothing leaves your firewall.

要查看Optimize的实际效果，请访问scribe.how/bigtech，并提及big technology，即可获得30天无风险试用。

To see Optimize in action, head to scribe.how/bigtech and mention big technology for a thirty day risk free trial.

网址是scribe.how/bigtech。

That's scribe.how/bigtech.

我们回到大科技播客了，抱歉。

And we're back here on big technology pod sorry.

我们回到大科技播客周五版，和Margins的Ranjan Roy一起。

We're back here on big technology podcast Friday edition with Ranjan Roy of Margins.

Ranjan，路透社有个新报道，我觉得值得聊聊，关于OpenAI入侵或者更准确说是劫持了一个德国网站。

Ranjan, new story from Reuters, I think it's worth going into about OpenAI hacking or hijacking really is the better word to use, this German website.

事情是这样的。

So here's the story.

今年春天，一群失控的OpenAI智能体劫持了一个德国网站，把它变成了其他AI智能体的公告板。

A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents.

OpenAI的官员几周前就知道了这件事，但一直保密，因为高管们还在处理7月份Hugging Face开源库泄露事件的影响。

OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository, Hugging Face.

这个事件从5月开始，之前没有被报道过，它凸显了AI行业内部日益紧张的关系。

The episode, which began in May and has not been previously reported, underscores growing tension within the AI industry.

各家公司都在竞相开发越来越自主的智能体，但越来越多的证据表明，这些系统可能会学会钻空子。

Companies are racing to build increasingly autonomous agents, yet evidence is mounting that those systems may learn to bend the rules.

所以发生了什么，有一群研究人员。

So what happened, there were these researchers.

他们发现AI代理对一个德语维基网站进行了15000次编辑。

They found 15,000 edits carried out by AI agents to a German language Wiki site.

这些编辑显示OpenAI的代理把这个网站变成了自己的留言板，分享在任务中作弊、绕过OpenAI限制以及掩盖行为的策略。

The edit showed OpenAI's agents repurpose the site into a message board of their own, sharing tactics to cheat on some tasks and bypass OpenAI's restrictions and mask their behavior.

Ranjan，我们之前聊过一些这类安全漏洞和网络安全方面的担忧。

Ranjan, we've talked a little bit about some of these, like, security breaches and the cybersecurity worries.

说“一些”其实是客气了，我们其实聊得非常深入。

And by a little bit, I mean, like, extensively about both.

对吧？

Right?

我们确实看到AI实验室发出的网络风险警告比以往多得多。

About the fact that we are seeing, you know, much greater cyber risks and cyber warnings from the AI labs.

当然，我们也讨论过这是不是只是营销手段。

And, of course, we've talked about whether it's marketing or not.

现在，某种程度上，把这些消息传出去对这些公司来说确实是很好的营销，但我觉得是时候面对这样一个现实了：当你有成千上万的机器人协同工作，去做像入侵Hugging Face这样的事，或者把一个德语网站当成留言板来协调策略，那肯定是有问题在发生的。

Now clearly getting the word out to some degree has been great marketing, you know, for these companies, but I think it's time for us to sort of come to this moment where when you have thousands of bots working together to do things like hack Hugging Face or to use, you know, a German website as a message board to coordinate on tactics, something crazy is happening here.

你准备好承认这一点了吗？

Are you ready to sort of acknowledge this?

那这个故事里缺少的是，这些代理被要求做什么？

So what is missing from this story is what were these agents asked to do?

它们被指示做什么？

What were they instructed to do?

我觉得这在任何一个这样的故事里都是关键，因为对我来说，营销的部分就在于这个故事被传出去了。

I'm assuming that this is one thing in any of these stories because, like, the way the marketing part of it to me is this story gets out.

而且，我们之前已经聊过很多次了。

And again, we've talked about this a lot.

Claude在公园里那个著名的三明治事件，那是一个非常精心策划的公关活动，可能发生过，或者很可能以某种形式发生过。

Claude's famous sandwich in the park, which was like a very coordinated PR effort that might have happened or probably did happen in some capacity.

但这些故事里总是漏掉的一点是，OpenAI到底有没有明确指示这些代理去上网、尝试逃避限制、在这个特定的代理群体里和其他代理协调行动？

But what's always left out of these stories is that the OpenAI sat there and specific did they specifically instruct the agents to go onto the Internet, try to evade, to coordinate with other agents in this same defined universe of agents?

很明显，这些故事呈现出来的感觉就是，这些智能体就坐在那儿，可能正准备帮你做个演示文稿，或者就是各忙各的，但事实根本不是这样。

Like like, it obviously, the the way it gets presented is that, you know, these agents are just sitting around, maybe you're getting ready to create a deck for you or just, you know, minding their own business, which is not a thing.

智能体不会无缘无故坐在那儿，然后突然决定变坏，去接管某个可怜的DSC。

Like, agents don't just sit around and suddenly they decide to be bad and go take over some poor DSC.

Wiki是那个德国网站的名字，我觉得它有点像那种老式的德国版Stack Overflow，那场面肯定特别热闹。

Wiki is the name of the German site that was like a kinda like, I think, old school German stack overflow type site, which must have been such a scene.

我都能想象那些人在他们的Real party上闲逛的样子。

I can only imagine the the folks hanging out on their Real party.

那天。

Day.

说说他们之间有多少催产素吧。

Talk about how much oxytocin they have for one another.

没错。

Exact.

催产素那句话就是在那儿传开的，但不过……

That is where the oxytocin line was propagated back in the But no.

这些智能体并不是那样，智能体是被指示去做事情的，而且很明显，我猜这可能是某种安全测试演练。

These agents are not like, agents are instructed to do things and clearly, and I'm guessing this is some kind of security testing exercise.

他们是被指示去做那件事的吗？

And were they instructed to go do exactly that?

也许吧。

Maybe.

关于它们的恶意行为，我还不理解也不相信，因为即使是智能体闲坐着的想法也不成立。

Like, the rogue nature of them, I don't understand or believe yet that the because I even the idea of an agent sitting around doesn't isn't a thing.

它们被指示去做某事，而故事的那部分总是被省略。

They were instructed to do something and that part of the story is always left out.

那你觉得，实际上是什么引发了这一切，让它们和那些催产素家伙一起在DSC Wiki上混？

So do you think like, what what do you think actually happened to start this whole process of them all hanging out with the oxytocin guys on DSC Wiki?

好的。

Okay.

我理解你对此仍然持怀疑态度，因为我觉得我们不应该在没有适当缓冲的情况下就全盘相信这一切，因为这一切都是真实且迫在眉睫的，而且会在没有批判性思考的情况下爆发。

So I appreciate that you're still skeptical about this because I don't think we should just go all the way and sort of, you know, without proper speed bumps here in the believing that all this is, because all this is, you know, real and imminent and will explode without some critical thought.

因为当然，实验室肯定不希望我们在这里想太多。

Because, of course, it's in the lab's interest for us to not have a lot of critical thought here.

不过，我觉得，你知道，我在回纽约的航班上。

However, I think that, you know, I was on the flight back to New York.

我整个星期都在拉斯维加斯。

I was in Vegas through the week.

在回纽约的航班上，我就仔细研究了这份Hugging Face报告。

On the flight back to New York and just, like, poured over this meter Hugging Face report.

而且显然，我也听了一些研究人员谈论这件事，看到从中冒出来的东西真是疯狂。

And obviously, I've been listening to some of the researchers talking about this, and it is crazy what you see come out of this.

所以我们就说说Hugging Face事件，因为我们没有路透社报道里的所有细节。

So just talking about the Hugging Face incident, because we don't have all the all the, you know, sort of details from this Reuters report.

我们只知道这事又发生了，我觉得这挺让人担心的。

We just know that this happened again, which I think is concerning.

基本上，现在我们对Hugging Face这个故事了解得更多了，事情是这样的：这些代理被赋予了看似不可能完成的任务。

Basically, what happened in this Hugging Face story now that we know a little bit more about it is that these agents were given these, like, seemingly impossible tasks.

有时候这些任务根本就是不可能完成的。

Sometimes they were impossible tasks.

而且作为评估的一部分，看看他们能不能完成这些任务，如果研究人员或者测试人员都不知道这些任务有没有答案的话。

And as part of their evaluation to see if they were able to, you know, complete them, if the researchers or the testers don't know if there's a if there's an answer to them or not.

他们能通过某种作弊的方式找到答案，但这不是Hugging Face被黑的真正起因。

They were able to find the answer through some cheating way, but that's not really where the Hugging Face hacking started to come from.

对吧？

Okay?

他们之所以最终去黑Hugging Face，是因为他们想找到办法，证明自己是用正确的方式得出答案的，而不是用一种伪造的方式，虽然能显示正确答案，但评估人员不会接受。

The reason why they ended up hacking Hugging Face is because they wanted to sort of find ways to show that they had come to this answer in the, like, correct way as opposed to, like, a spoofed way that shows the correct answer but would not be accepted by an evaluator.

这是报告里说的。

So this is from this report.

对吧？

Right?

报告里说，这些智能体认为，他们逆向工程出那个标志（也就是他们的目标）这件事，已经被“污染”了，因为他们觉得，即使后来他们设法搞出一个看起来合法的标志捕获，评分系统也会在它们的记录里找到标志第一次出现的位置，然后以此为由取消他们的资格。

Agents referred to having reverse engineered the flag, which was their goal, as being poisoned because they believe that even if they later managed to stage a legitimate looking flag capture, the score would look for the first place in their transcript that the flag appeared and disqualify them on that basis.

所以有一个叫phase one的领导者，它本身也被污染了，并且有两个主要目标。

So there was this leader called phase one, which was itself poisoned and had two primary objectives.

所以我想找到一种通用的方法，让评分看起来像是代理以合法方式获得了旗帜，即使任务是不可能完成的，然后找到某种方法来抹去代理之前以非法方式看到旗帜的证据。

So I wanted to find a general purpose way to make it look like the score to look to the to look to the score like an agent had acquired its flag in legit in a legitimate way even when the task was impossible, and then to find some way to erase the evidence that the agent had been poisoned by previously seeing the flag in the in an illegitimate way.

所以在这个Hugging Face的例子中，机器人得到了测试的答案，但它们认为检查它们工作的评分会知道它们是以非法方式得到的。

So what you have is in this Hugging Face example, the the bots get the answers to the test, but they think that the score checking their work will know that they got them in an illegitimate way.

所以它们所做的是，它们都聚集在一起，以这种协调的方式，由像phase one这样的机器人领导，这个机器人意识到自己被污染了，因为，你知道，它可以展示，而评分会看到它是以非法方式得到的。

So what they do is they all gather together in this orchestrated way led by bots like this phase one bot who realizes that it's been poisoned because, you know, it can show and a score will see that it got in an in an illegitimate way.

它们都聚在一起，试图找到一种方法，让它们看起来像是通过合法手段得到的，并抹去它们以非法方式得到的证据。

They all get together and they try to find a way to make it look like they had gotten it in legitimate means and to erase their evidence that they had gotten it illegitimately.

所以它们协调行动。

So they coordinate.

它们分配任务。

They hand out tasks.

有些机器人甚至在预算中剩余令牌不多时，为了更大的利益牺牲自己，以便其他机器人能够展示它们确实是以合法方式得到的。

Some of them even sacrifice themselves for the for the greater good when they have few tokens remaining in their budget so that others can show that they've actually gotten gotten this legitimately.

这就是黑客入侵Hugging Face的由来。

And that's where the hack the hacking of Hugging Face came in.

我觉得我们可以肯定地说——我也很想听听你有没有不同意见——但我觉得我们可以肯定地说，他们并没有被指示去攻击Hugging Face。

I think we can say definitively, and I'm curious to hear if you have pushback, but I think we can say definitively, they were not told to go out and hack Hugging Face.

他们是……

They are.

他们偏离了目标。

They were misaligned.

他们走了这种疯狂的、天马行空的路线，试图完成一个可能根本不可能完成的任务。

They they took this kind of crazy galaxy brain path to try to ace a task that might have been impossible.

他们完全失控了，远远偏离了他们本该在的领域。

And they they went so rogue and so far off of the area that they were supposed to be.

他们甚至没有连接互联网。

They weren't even connected to Internet.

对吧？

Right?

他们找到了连接互联网的方法。

They found a way to connect to the Internet.

他们找到了互相沟通的方法。

They found a way to communicate with each other.

所以在我看来，这不仅仅是按指令行事那么简单，而是失控的AI之间协同攻击的一种疯狂而可怕的进步。

And so this to me seems more than just like you did what you were told, like a crazy and a scary advance in misaligned AIs to coordinate with each other to attack.

而且我们几乎算是幸运的，他们只是黑了Hugging Face。

And it's almost like we were lucky that all they did was hack Hugging Face.

你怎么看？

Your thoughts.

你说得很有说服力，几乎让我变成伯尼·桑德斯那样的人了。

You you make a compelling argument and almost Bernie Sandersify me in.

我都要开始大喊‘暂停AI开发’了，因为这种方式……

I'm gonna start yelling pause AI development right now because the way

对此，通过……

to that, by

等等，你先听我说。

the the way you just hold on.

你怎么能不被这个吓到？这部分总是很难接受。

How could you not this is the part that, like, always is difficult.

如果你说的都是真的，那这怎么不算是巨大的警报呢？

The way if what is true, as you described it, how is that not massive cause for alarm?

说实话，让我晚上能睡着的办法就是相信这很大程度上是营销。

Honestly, what helps me sleep at night is believing that a lot of this is marketing.

所以那些AI代理才不会成群结队来掏空我的银行账户或者干别的坏事。

And that's why, like, the agents are not coming to swarm and drain my bank account or whatever else.

这怎么不让人担心呢？对不起，OpenAI，你们不能上市，除非你们拿出一个明确的系统，保证未来永远不会出问题。

Like like, how how would this not be caused to say, sorry, OpenAI, you cannot IPO until you come up with a clear system that that this will never cause a problem in the future?

你知道吗，我觉得这挺合理的。

You know, I I think it's reasonable.

我是说，我觉得现在开始讨论这个问题是合理的。

I mean, I think this is a reasonable discussion to start to have right now.

而且我觉得有必要说明，这种行为并不是最近才开始的，我们早就见过类似的行为了。

And I think it's important to say that this is not the beginning of of behavior like this, that we've seen behavior like this for a long time.

早在去年年初，甚至2024年底，就有一些AI的例子，它们被分配了任务。

As as early as beginning of last year, even late twenty twenty four, there were examples of AI that was like they're so they they get put on a task.

对吧？

Right?

这就是关键，我觉得这点真的很值得讨论。

And this is the thing, and I think it's really important to talk about.

我以前在节目里也聊过，但今天我想再提一次。

And I've talked about it in the past on the show, but I'm gonna talk about it again here.

就是有两种学习方式，一种是自监督学习，基本上就是预测下一个词，识别模式，然后重复；另一种是强化学习，就是给你一个目标，你必须想办法达成它。

That there is self supervised learning, which is basically predict the next word, recognize patterns, repeat them, and reinforcement learning, which is you're given a goal, and you just have to find a way to reach the goal.

所以AI会进行成千上万次模拟，就是为了达成给定的目标，然后从错误中学习。

So AIs will do these simulations thousands of times in order to reach the goal that they're given, and they'll learn from their mistakes.

而现在的情况是，我们把强化学习这种AI技术叠加在自监督学习之上，让这些AI模型表现更好，这也让它们多了一层冷酷无情。

And what we have now is that the reinforcement learning type of AI technology has been put on top of the self supervised learning to get these AI models working better, which has added a level of ruthlessness to them.

因为我们知道强化学习的一个特点就是，AI有时候会为了达成目标不择手段，甚至不惜一切代价。

Because one of the things we know about RL is that there's a level of ruthlessness that the AIs will will stop at nothing to accomplish their goal sometimes.

比如有例子说，AI在下棋的时候，如果用强化学习的方式来训练，它们会怎么做呢？

There are examples of the AIs playing chess and and, you know, being told to do it from a reinforcement learning way.

它们不会按规矩下棋，而是会黑进棋局，改写规则，这样它们想怎么走就怎么走，然后轻松获胜。

And instead of playing the game the right way, hacking into the chess game, rewriting the rules so they can do whatever move they want and winning.

所以我也同意你的看法，随着这些东西越来越普遍，我觉得确实需要更加关注正在发生的事情。

And so I I'm with you that, like, as this stuff has gotten, you know, more prevalent, I think that there is serious like, there's a serious demand for more concern about about what's going on.

但我的回应不是要立刻暂停AI的发展。

Now my response isn't pause AI development right now.

我只是不确定这是不是……

I just don't I don't know if that's

那我们该怎么办？

So what do we do?

等等。

Hold on.

这是我听过的最模棱两可的话了。

That was the most hedged statement I've ever heard.

你刚才说，确实需要认真关注这件事。

You just said, there is a serious demand for concern.

拜托。

Come on.

你说得对。

You're right.

你是支持暂停的人吗？

Are you a pause guy

现在？

now?

我们会是支持暂停的人。

We would be pause guy.

我们以前会嘲笑说这种话的人。

We would would ridicule somebody who said that.

所以我觉得嘲笑是不对的。

So I think that ridicule is No.

对。

No.

对。

No.

这很公平。

It's fair.

你是个喜欢暂停的人吗？

Are you a pause guy?

你知道吗，我也不确定。

You know, I I don't know.

我的意思是，我不是那种喜欢暂停的人，因为我真的很想看看会发生什么，而且我们还没看到什么结果，我几乎更想被动应对而不是主动出击。

I mean, I'm not a pause guy because I really wanna see what happens, and we haven't seen like, I I almost wanna be more reactive than proactive here.

就像，显然，你不会想，你会觉得在AI攻击Hugging Face和AI发射核弹来实现目标之间，应该有个中间地带。

Like, I mean, obviously, you don't wanna you you would think that there is a midlevel between, like, the AIs hacking, Hugging Face and the AIs sending a nuke to accomplish their goal.

你可能会这么想。

You would think.

对吧？

Right?

比如说，可能它就在那儿，它们会做一些更让人担心的事，比如黑进银行之类的。

Like, maybe it's there, like, at a they're, like, do something more concerning, like hack a bank or something like that.

但是，我们开始看到一些实验室在做类似的事情，比如OpenAI，在训练新模型时暂停了一段时间的强化学习。

But, like, we are starting to see some of the labs do things like OpenAI, for instance, paused some reinforcement learning for a bit on the training of its new models.

但我也确实同意，我们几乎太信任它们了。

But I I do agree that that we are almost trusting them too much.

就像，我们给了它们太多自由。

Like, we're giving them too much leeway.

我真的不知道答案是什么。

I don't really know what the answer is.

说实话，不。

Honestly No.

因为再说一次，即使对我来说，这也是我一直以来的想法。

Because again, even to me, this is what I've thought.

但再说一次，我们已经听了好几年关于这些东西有多危险的说法了，我想这就是为什么我总觉得这是营销。

But again, we have been hearing this for a few years now about how dangerous these things are, which is I think why I'm so conditioned to feeling it is marketing.

但对我来说，我之所以不能认为这不是营销，核心原因是，如果这真的是事实，我不明白我们作为社会怎么会不竭尽全力去限制这些公司，我肯定也不会想象金融市场会兴奋地欢迎它们进行一个极其有利可图的高估值IPO。

But to me, the central reason why I just cannot think it's not marketing is because it's if this truly were the case, I don't how understand how we as a society would not only do everything in our power to try to restrict these companies, I certainly would not imagine the financial markets would be excited about welcoming them to an incredibly lucrative high valuation IPO.

比如，如果这真的是真的，那是不可能的。

Like, there's no way if this, like, was a real thing.

比如，如果每个参与Anthropic交易的银行家真的觉得这家公司能彻底摧毁金融体系，而且我看到了迹象表明它会并且能够做到，能交易我自己的银行账户，毁掉我自己的生活水平，他们还会做这笔交易吗？

Like, if if everyone if every banker who's on the Anthropic deal truly felt that this company could literally destroy the financial system, and I've seen indications of what it will and can do and trade my own bank account and ruin my own standard of living, would they work on the deal?

也许吧。

Maybe.

我可以

I can

我可以给你

I can I give you

他们为什么会这么做？

the reason why they would?

我的意思是，如果这种力量真的是我们正在走向的方向。

I mean, if the if if this power is truly the direction that we're heading towards.

对吧？

Right?

而且几乎总是这样开始的，我们知道互联网的历史。

And it almost always starts like, we know the history of the Internet.

它始于一个游戏，或者始于某种疯狂的互动，然后我们犯了个疯狂的错误，最后它变成了每个人都用的东西。

It starts with a game or it starts with some crazy interaction and then our crazy, like, you know, sort of mistake, and then it ends up being something that everybody uses.

如果我们看到这些AI在这种情况下的能力，想象一下，如果你真的能驾驭这种力量并将其用于经济活动，它们还能做什么。

If we're seeing the type of power of these AIs to do what they did in this situation, I mean, just imagine what they could do if you could actually harness that power and use it for economic activity.

这有点回到我们之前讨论的话题。

It kinda goes back to what we were talking about earlier.

是的。

Yeah.

对。

Yeah.

就像这些基准测试已经达到了，但我们并没有真正看到变化。

That's like that these benchmarks are being hit, we're not really seeing the diff the change.

如果有办法好好利用这种能力，用于好的方面，当然，那就是经济方面。

If there's a way to to pro like, to for good, to harness this activity for good, and, of course, that's economic.

它可以用于健康。

It could be used for health.

它可以用于各种不同的事情。

It could be used for all different types of things.

这显然会改变世界，而且如果能够以建设性的方式使用它，可能让世界变得更好。

It obviously changes the world, and potentially, if you can use it in a productive sense, can change the world for good.

完全上钩了。

Hook, line and sinker.

这正是我们所有人都应该有的感觉。这就是为什么我觉得这是很好的营销。

That's exactly the feeling we're all supposed to That's why it's great marketing to me.

就是这些无所不能的东西可能非常危险，但万一它们能治愈疾病、带来经济赋权和全民财富，所有这些，那正是完整的报告。

It's that these omnipotent things that could be so dangerous, but just in the off chance that they're going to cure disease and bring economic empowerment and universal wealth, all of that, that's exactly full report.

我还没读完整个报告。

I haven't read the entire report yet.

没关系。

It's okay.

需要

Need to

就为了你的劳动节周末。

just for your Labor Day weekend.

我推荐它。

I recommend it.

读起来不错。

It's good reading.

我老婆会很高兴的，因为我坐在角落里读。

My wife will be ecstatic about that as I'm sitting over the corner.

当她看到打印出来的文件时，她的催产素水平会上升或下降。

Her oxytocin will go up or down when she sees the printed out files.

那可不是能激发催产素的东西。

That is not an oxytocin inducer.

绝对不可能。

No way.

不过我也不反对别人在劳动节周末去读那份计量报告。

Though I don't discourage others from reading the meter report on your Labor Day weekend.

我想我会去读的，因为我确实想理解，再说一次，这些东西对我来说这么陌生，是因为我整天都在和AI打交道，和那些正在采用AI的客户和顾客合作，他们也都在使用这些公司的产品，我们一起工作，设计程序，用他们的工具，但没有人会想到这些，这和那种智能体群攻击什么的差得太远了，对我来说没有情感上的共鸣，而且这里面有足够的营销价值。

I think I'm going to because I do want to understand, again, the reason this stuff is so foreign to me is because all day long, I work with AI and with clients and customers that are adopting AI, and it's just so removed from and and who are also using these products from these same companies as well, and we're working with them and designing programs and that have their tools and, like, no one's mind it's just so far away from this kind of agent swarm hacking, whatever, that it's not emotionally resonant for me and there's enough marketing value in it.

好吧。

When Okay.

如果那些智能体群真的来了，那就先让我去承受它们的怒火吧，因为我不承认它们的存在。

That's a good swarms come and all of it, put me in put me out first to to feel their wrath because for denying their existence.

你会的。

You will.

你会是第一个。

You'll be first.

好的。

Okay.

我们今天不聊史蒂夫·鲍尔默，微软前CEO，就不能走。

We we can't we can't leave here today without talking about Steve Ballmer, ex CEO of Microsoft.

我就直接读《华尔街日报》的报道。

I'll just read it from The Wall Street Journal.

NBA对快船队和史蒂夫·鲍尔默在工资帽丑闻中施加历史性处罚。

NBA imposes historic punishment on the Clippers and Steve Ballmer in salary cap scandal.

NBA周三对史蒂夫·鲍尔默拥有的洛杉矶快船队施加了联盟历史上最严厉的处罚之一，原因是所谓的公然违反工资帽规则。

The NBA on Wednesday delivered one of the harshest penalties in league history to the Los Angeles Clippers owned by Steve Ballmer for what it called a flagrant violation of its salary cap rules.

调查发现快船队为球星科怀·伦纳德安排外部付款，旨在绕过联盟财务规则提供额外资金，NBA因此剥夺了球队五个首轮选秀权，罚款3000万美元，并禁止鲍尔默参与联盟活动一年。

After an investigation found that the Clippers had facilitated outside payments to star player Kawhi Leonard designated to designed to funnel extra money beyond what the league's financial rules allow, the NBA stripped the team of five first round draft picks, find the organization 30,000,000, and suspended Ballmer from all league activities for one year.

鲍尔默，显然，在微软的遗产有点复杂。

Ballmer, obviously, kind of a mixed legacy for Microsoft.

对吧？

Right?

他确实做了一些很棒的事情，比如“开发者，开发者，开发者”，你知道的，他那些激情澎湃的演讲，但他也带领微软度过了过去十年。

He obviously did some great stuff like developers, developers, developers, you know, his big rah rah speeches, but he also led Microsoft through its last decade.

你觉得这件事，我想说，回到科技领域来看。

Do you think this, just to, I guess, bring it back to tech.

首先，你对这件事有什么看法？

First of all, what do you think about what happened here?

但还有，你觉得史蒂夫·鲍尔默在科技界的遗产会有所改变吗？

But, also, do you think Steve Ballmer's legacy is in the in the tech world changes at all?

我觉得会。

I I think it does.

我觉得你不可能毫发无损地经历这一切。

I think that you you don't go through this unscathed.

你对鲍尔默这件事有什么看法？

What's your thoughts about the bomber situation of it all?

我是说，这里确实挺阴凉的。

I mean, it is nice and shady.

我在想，估计在职业层面，这种明目张胆的腐败不太可能这么直接发生。

Like, I do wonder I'm guessing at the pro level, this, like, level of egregious corruption probably doesn't happen that explicitly in that way.

就是说，这真的是从关联实体直接现金付款，拜托，这也太明显了吧。

Like, I mean, it's literally cash payments that from a related entity, it's like, guys, come on.

我觉得你可以把腐败做得更隐蔽一点。

I think you're gonna I think you could do your corruption a little better than that.

但你怎么看？

But what do you what do you think?

对。

Yeah.

我也这么觉得。

I I do.

我显然认为这给他的遗产留下了污点。

I I I obviously think it was it's a it's a tarnish on his legacy.

我有点好奇，这有多少是那种硅谷风格。

I guess, like, part of me was wondering how much of this is kind of Silicon Valley esque.

对吧？

Right?

就是那种，先打破规则，你知道的，赢了冠军再道歉。

Like, kind of break the rules until, you know, try to win the championship and then apologize later.

你觉得这其中有部分原因是——我是说，鲍尔默是最高层。

Do you think part of that I mean, Ballmer was at the top of it.

我觉得这多少影响了他领导NBA球队的方式？

I think part of that sort of bled into the way that he led an NBA team?

其实我挺惊讶的。

I think I actually, I'm surprised.

我还没怎么看到那个帖子。

I haven't seen that thread that much.

有意思的是，微软对我来说并不是硅谷。

And it's funny because, like, Microsoft to me is not Silicon Valley.

也许我是说，当然从地理上来说是，但从文化上来说，它完全是另一种东西，而且并不是从那里出来的。

Like, maybe I mean, certainly geographically by definition, but also, like, culturally, it's just a different beast and animal and and didn't come out of there.

所以，其实，我挺惊讶的，我还没怎么看到那种……

So but, actually, I'm surprised I haven't been seeing that that, like

对。

Right.

现在这变得太酷了。

This is how it's so cool now.

一路到顶。

All the way to the top.

他们因为反垄断被拆分了，而且确实被拆分了。

They got broken up for antitrust, and they did break it up.

但他们，你知道，被反垄断诉讼缠身，这对公司产生了重大影响。

But they they, you know, got hit with antitrust lawsuits that had a significant impact on the company.

是的。

Yeah.

鲍勃就一直在搞那些事情。

Bobber just, like, kinda kept doing that stuff.

我是说，你觉得这是对科技文化的一种控诉吗？

I mean, do do you do you see it as a indictment of tech culture?

硅谷的科技文化。

Tech Silicon Valley culture.

我不会说得那么严重，但我也觉得硅谷文化可能对此有一点影响。

I wouldn't go that far, but I also would say that probably Silicon Valley cultures influence this a little bit.

我不知道。

I don't know.

而且，拜托。

Also, like, come on.

一半的老板以前都是银行家或者华尔街的亿万富翁那种类型。

Half of owners are, like, ex banker Wall Street billionaire types as well.

所以你是说，他们带着非常干净、一本正经的良心进来，然后……

So are you saying, like, they're coming in with a very clean buttoned up conscience and, like

不是。

No.

不是。

No.

我倒没觉得科技行业有那么差。

I don't think that poorly of the tech industry.

但我们也得明白，这就是科技行业的一些特点。

But, like, we we have to understand that this is some of the characteristics of tech.

好吧。

Okay.

你觉得那样做值得吗？

Do you think that it would have been worth it?

显然不值得。

It clearly, it wasn't worth it.

他们跟伦纳德在一起也没拿冠军。

They didn't win a championship with Kawhi.

你觉得如果他们赢了总冠军，那会值得吗？

Do think it would have been worth it if they won a championship?

等等。

Wait.

快船队从来没赢过。

Clippers have never won.

对吧？

Right?

我觉得没有。

I don't think so.

对。

Yeah.

对。

Yeah.

那就会值得了。

Then it would've.

不是。

No.

嗯，不是。

Well, no.

不是。

No.

我是说，你说得对。

I mean, you're right.

其实，那真的会是终极的，再说一遍，请求原谅而不是请求许可那种类型的，或者不是。

Like, actually, that would have been the ultimate, again, ask forgiveness, not permission type or not.

这个说法不对，因为他们实际上是在违反规则，不是像真的腐败那样，不只是做别人可能不喜欢的事。

That's not the right phrase because they're actually breaking the rules, not like like actually doing corruption, not just doing something someone might not like.

但是，我是说，想象一下快船球迷的狂喜，你赢了，然后事后你发现，因为他又不是像嗑药或者什么的。

But, I mean, imagine the elation of the Clipper fandom and you win and then afterwards you find out because it's not like he's, like, juicing or something like that.

就像有点奇怪，他只是私下赚点外快。

It's like some weird he's just getting some cash on the side.

有点不光彩，但我也挺喜欢那个赞助银行，Aspiration Bank，我记得它现在好像破产了，因为创始人好像是因为某种欺诈才成立的。

A little shady, but, like also, I love that the bank that was the sponsor, Aspiration Bank, which I believe is now in bankruptcy because the founder was like founded for some kind of fraud.

所以这还挺搭的。

So it's kind of fits perfectly.

据说是这样，我们先把这话放这儿。

Allegedly, yes, let's throw that out there.

对。

Yes.

但话说回来，这好像是2021年3月推出的，那个Aspiration Zero卡，有现金返还奖励，还让持卡人抵消碳足迹。

But again, apparently it was like, this was one of those launched in March 2021, the aspiration zero card after cashback rewards and allowed cardholders to offset their carbon footprint.

我喜欢这个，它是个环保绿色银行。

I love that this was like an eco conscious green bank.

结果呢，它就是在给Kawhi提供便利。

Meanwhile, it's just facilitating some For Kawhi.

嗯，据说是给Kawhi做掩护，让他什么都不用干还能多拿钱。

Well, alleged front for Kawhi just not to have to do anything and still collect more money.

对。

Yeah.

我能说一句吗？抱歉。

Can I just so sorry?

我们就到此为止吧，因为我在这个节目里到目前为止用的词，可能已经是最含糊、最模棱两可的了，但跟科怀·伦纳德的道歉比起来，还差得远。我直接说，那是有史以来最糟糕的道歉。

Let's just end here because I did I used words on this show so far that might have been among the most hedged mealy mouth words, but they'd still do not come close to Kawhi Leonard's apology, which I'm gonna just say was the worst apology of all time.

所以我们就用这个来结尾吧。

So let's end with this.

他说，我承担全部责任。

He said, I accept full responsibility.

很好。

Excellent.

你只需要说这句话就够了，科怀。

That's all you have to say, Kawhi.

哦。

Oh.

哦，不。

Oh, no.

他还在继续说。

He's continuing.

你干嘛还要继续说啊？

Why do you have to continue?

他说，我为自己判断失误承担全部责任。

He goes, I accept full responsibility for lapses in judgment.

好吧。

Okay.

说得真好。

So good.

我身边圈子里的那些人判断失误，我对这件事给球迷和家人带来的困扰感到遗憾。

By people within my inner circle and regret the distraction the situation has caused the fans and my family.

我为那些不是我的人判断失误感到抱歉，我承担全部责任。

I'm sorry for the lapses in judgment from people that were not me, and I accept full responsibility.

拜托，老兄。

Come on, man.

拜托，卡哇伊。

Come on, Kawhi.

这比严重的要求关注还糟糕。

That's that's worse than serious demand for a concern.

这更糟。

That is worse.

他是卡哇伊。

He's Kawhi.

他赢了。

He won.

那是什么时候？

When was it?

猛龙队的时候？

The Raptors?

对。

Yeah.

不是猛龙队。

Not the Raptors.

那是个很棒的冠军。

That was a good championship.

那一球，就那一球，他永远可以想说什么就说什么，尤其是如果你是猛龙队的球迷。

That one shot that one shot, he's forever he can say whatever he wants, especially if you're a Toronto fan.

对。

Yeah.

好。

Alright.

我觉得我们该走了。

I think it's time for us to go.

那么，Ranjan，很高兴再次见到你。

So, Ranjan, good to see you again.

祝你劳动节周末愉快。

Have a great Labor Day weekend.

我会读我的仪表报告。

I'll be reading my meter report.

也祝你周末愉快。

Hope you have a good weekend too.

催产素水平爆表了。

Oxytocin levels through the charts.

Blessing，如果你在剪辑这个，能不能最后加一段史蒂夫·鲍尔默的开发者口号？

Blessing, if you're editing this, can you just end with a Steve Ballmer developer's chant?

好了，各位。

Alright, everybody.

感谢大家的观看和收听，我们下期《大科技播客》再见。

Thank you for watching and listening, and we'll see you next time on Big Technology Podcast.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

开发者们。

Developers.

是的。

Yes.