担忧AI的正确方式

今天的《AI每日简报》要讲的是，如何正确地担忧AI。

Today on the AI Daily Brief, the right way to worry about AI.

在那之前，先看头条新闻：市场、模型，还有更多。AI每日简报是一档每日更新的播客和视频节目，内容涵盖AI领域最重要的新闻和讨论。

Before that in the headlines, markets, models, more, the AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

好了，朋友们。

Alright, friends.

在深入之前，先快速说几个通知。

Quick announcements before we dive in.

首先，感谢今天的赞助商：毕马威、Blitzy、Robots and Pencils，还有Hyperagent。

First of all, thank you to today's sponsors, KPMG, Blitzy, Robots and Pencils, and Hyperagent.

想要收听无广告版本，可以访问patreon.com/aidailybrief，或者在Apple Podcasts上订阅。

To get an ad free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts.

顺便提一句，Apple Podcast这周似乎出了点问题。

Quick note there, by the way, Apple Podcast seems to have been having some trouble this week.

我们并没有像往常那样，在Apple上发布无广告版本时遇到延迟，但有些人过了好几天看到的还是带广告的版本。

We haven't been having any particular delays as we sometimes do with the ad free version going up on Apple, but I've had some people days later still seeing the ad version.

我能给的最好建议就是彻底退出并重新启动你的苹果应用。

The best that I can suggest is to completely close out of and restart your Apple app.

但无论如何，对于带来的不便，我表示歉意。

But in any case, I apologize for the pain.

最后再提醒一下，去看看AI夏日冒险活动吧。

Last note, one more reminder to go check out the AI summer adventure.

这是一系列免费的自主项目，旨在扩展你的AI视野，你可以在summeradventure.ai找到所有内容。

It's a set of free self directed projects to expand your AI horizons, and you can find it all at summeradventure.ai.

最后，一如既往，如果你想赞助这个节目，请发送邮件到sponsors@aidailybrief.ai。

Finally, as always, if you were looking to sponsor the show, send us a note at sponsors@aidailybrief.ai.

好了，这些琐事都处理完了，我们开始吧。

But with all that out of the way, let's dive in.

今天我们先从一些OpenAI的消息开始。

We kick off today with some OpenAI news.

嗯，先是一些猜测，然后是一些真实新闻。

Well, a little bit of speculation and then some real news.

有爆料开始说，下一个新模型ASTRO，就是上周我们讨论过的那个做出新颖数学证明的模型，似乎马上就要发布了，有些人甚至说他们目标就在下周。

The leakers are starting to suggest that the next new model, ASTRO, which was of course the one that did those novel math proofs that we discussed last week, seems to be imminently launching with some saying that they're even targeting next week.

我们确定知道的是，即使他们在发布新模型，OpenAI也在非常认真地考虑并在成本方面努力竞争。

What we know for sure is that even as they are releasing new models, OpenAI is also thinking very much and trying to compete very much on the cost front as well.

公司宣布，作为GPT-5.0.6模型系列服务改革的一部分，他们给免费用户提供无限聊天。

The company announced that they are giving free users unlimited chats as part of a service overhaul for the GPT-five 0.6 model family.

免费用户层级现在将使用GPT-5.0.6 Luna，取代之前的即时模型系列。

The free user tier will now be served with GPT-five 0.6 Luna, replacing the instant model range.

免费用户现在还会有一个“思考”按钮，让Luna能进行更深入的推理。

Free users will also now have a Think button to allow for greater reasoning from Luna.

理论上，这缩小了免费用户的一些体验差距，让他们能使用与付费用户相同的模型，尽管是较小的版本。

Theoretically, this closes some of the experience gap for free users, allowing them to access the same model as paid users, albeit the smaller version.

此外，使用次数现在没有限制，免费用户可以随心所欲地使用ChatGPT。

In addition, usage is now unlimited so free users can use ChatGPT as much as they want.

对于付费订阅者，GPT-5.6 Soul将成为默认的聊天模型。

For paid subscribers, GPT 5.6 Soul will now become the default chat model.

OpenAI表示，这应该会提升相比GPT 5.5 Instant的体验，Soul会减少事实错误，并且在没帮助的时候避免添加多余的细节。

OpenAI said that this should improve the experience over GPT 5.5 Instant, with Soul making fewer factual mistakes and avoiding extra detail when it doesn't help.

最后，Plus和Pro订阅用户现在将有一个新的Effort滑块和思考模式，以便更直观地控制推理的力度。

Finally, Plus and Pro subscribers will now have a new Effort slider and Thinking Mode to provide more intuitive controls over reasoning effort.

虽然有些人像跳投者一样，对吧，这怎么还能赚钱呢？

While some like jumpers right, How is that even profitable?

Kenshi在X上说，把OpenAI Luna免费化的举动显然不是出于慷慨。

Kenshi on X says, The move to make OpenAI Luna free is obviously not out of generosity.

免费层级是一个战略性的分发渠道。

The free tier is a strategic distribution channel.

他们想吸引用户，让他们升级到Go或Plus，或者当然，通过广告赚钱。

They're trying to hook people and get them to upgrade to Go or Plus, or of course, make money through ads.

现在说到我们关于更便宜模型的讨论，根据信息，Stripe确实在推进他们对OpenRouter的收购。

Now speaking of our discourse of cheaper models, according to the information, Stripe is indeed moving forward with their OpenRouter acquisition.

这家新闻媒体报道说，Stripe已经进入独家谈判，以接近之前报道的100亿美元的价格收购这家模型路由初创公司。

The news outlet reports that Stripe has entered exclusive talks to buy out the model routing startup for close to the $10,000,000,000 that was previously reported.

早前的报道提到，Stripe在竞购中领先，但这表明OpenRouter已经把自己从市场上撤下来，准备和这一个特定伙伴进入谈判阶段。

Earlier reports had suggested a bidding war with Stripe in the lead, but this suggests that OpenRouter has taken themselves off the market and will enter the negotiation phase with this one specific partner.

当然，交易仍有可能告吹，但独家谈判确实表明事情在向前推进。

Certainly the deals could still fall apart, but exclusive talks do suggest that it's moving to the next level.

与此同时，随着全球对AI计算的需求越来越大，供应链紧张的影响随处可见。

Meanwhile, everywhere we are seeing the impact of the supply chain crunch as the world uses more and more compute for more and more AI.

The Information再次报道，英伟达正在考虑降低Rubin的规格。

The information again reports that Nvidia is considering slashing the specs on Rubin.

目前，英伟达正在测试下一代旗舰GPU Rubin Ultra的三个不同版本，消息人士称，部分测试单元的内存比最初公布的要少。

Currently, Nvidia has three different variants of their next generation of flagship GPU, the Rubin Ultra, under testing, and sources said that some of the test units include less memory than originally announced.

消息人士称，英伟达考虑推出低内存版本，部分原因是担心无法为量产获得足够的高带宽内存。

Though sources said that Nvidia is considering releasing the lower memory versions partly due to concerns that they won't be able to secure enough high bandwidth memory for the production run.

这会产生多大影响，还有待观察。

Now how big the implications of this are remain to be seen.

即使内存减少，这些芯片仍然应该能够为最新一代的超大模型（如Claude Fable）提供推理服务。

Even with reduced memory, the chips should still be able to serve inference for the latest generation of ultra large models like Claude Fable.

然而，内存限制可能会制约模型规模继续扩大的能力。

However, memory limits could put a cap on the ability to keep scaling model size.

目前，英伟达尚未承认在获取足够内存方面存在任何问题。

Now, at this point, Nvidia has so far denied any issue with sourcing enough memory.

七月中旬，硬件工程高级副总裁安德鲁·贝尔表示：我们已经提前解决了内存问题，所以短期内不会受到阻碍。

In mid July, senior VP of hardware engineering Andrew Bell said: We were in front of the memory problem, so it's not going to hold us back anytime soon.

当然，价格对全世界来说都是个问题，而且价格可能是更大的挑战。

The pricing, of course, is a problem for the whole world, and probably the pricing will be the bigger challenge.

但就供应而言，我们没问题。

But for supply, we're in shape.

英伟达原本也要到明年年底才会出货新芯片，所以还有时间解决供应问题。

Nvidia also wasn't scheduled to ship the new chips until late next year, so there's still time to resolve supply issues.

尽管如此，我们可能开始看到硬件的基本限制，这可能会减缓模型改进的速度。

Still, we are at the point where we may be starting to see fundamental hardware limitations potentially start to slow down the pace of model improvement.

为了确保投资者没有误解，迈克·苏尔卡写道：这不是AI需求疲软。

Making sure investors didn't get it twisted, Mike Sulka wrote: This isn't weak AI demand.

这是在食物链顶端的存储分配问题。

This is memory rationing at the top of the food chain.

现在，认为存储短缺和其他组件短缺只会变得更糟的一个原因是，我们仍然处于理解全球AI总需求的非常早期阶段。

Now, one of the reasons to think that memory shortages and other component shortages are going get nothing but worse is the fact that we are still at the very beginning of understanding the world's total demand for AI.

像OpenAI这样的一些公司押注，新一代消费设备将有助于释放所有这些需求。

One bet that some companies like OpenAI are making is that a new generation of consumer devices will help unleash all of that demand.

彭博社有更多关于OpenAI首款设备的报道，将其描述为本质上是一个没有显示屏的智能音箱。

Bloomberg has more reports about OpenAI's first device, describing it as essentially a smart speaker without a display.

它由电池供电，大小大约像一个冰球，形状像甜甜圈，目的是方便单手在家里携带。

It's battery powered, roughly the size of a hockey puck and shaped like a donut with the intention of making it easy to carry around the home in one hand.

它将具有高质量的拉丝金属表面，类似于iPhone的表面处理，当然这也是商业秘密诉讼的一部分，因为OpenAI正在与一家苹果供应商合作。

Will have a high quality brushed metal finish, similar to the finish on iPhones of course formed part of the trade secrets lawsuit as OpenAI is working with an Apple supplier.

消息人士称，该设备将有一些小型活动部件和灯光，旨在提供与用户互动的视觉指示。

The sources suggested that the device will have some small moving parts and lights intended to provide a visual indicator that it's interacting with the user.

它将包括摄像头、麦克风和其他传感器，用于感知周围环境。

It will include a camera, microphones, and other sensors designed to take in surroundings.

最大的新消息是目标价格，OpenAI 计划以 300 到 400 美元的价格将这款设备推向市场。相比之下，最贵的智能音箱，比如亚马逊的 Echo Studio，零售价大约在 220 美元左右。另外，说到苹果的诉讼，很明显是为了推迟这款设备的发布，预计它会在明年年初上市，但至少彭博社的马克·古尔曼认为，设计上的差异让 OpenAI 没有侵权问题。

The biggest new news is the target price, with OpenAI aiming to bring the device to market between 300 and $400 By way of comparison, the most expensive smart speakers such as Amazon's Echo Studio are retailing for about $220 Meanwhile, speaking of the Apple lawsuit, it's pretty clearly aimed at delaying the release of this device which is expected sometime early next year, but Bloomberg's Mark Gurman at least thinks the design differences put OpenAI in the clear.

他说：这里最重要的信息是，这款产品的外观、手感和功能都和苹果的产品或苹果目前正在计划的东西完全不同。

He said: The big takeaway here is that it looks, feels, and acts nothing like an Apple product or anything the company is currently planning.

我被告知，OpenAI 没有发现任何证据表明他们在这款设备上侵犯了商业机密。

OpenAI hasn't found any evidence that they're violating trade secrets with this device, I'm told.

在市场上，周四软银披露，他们以 OpenAI 的股份为抵押借了 100 亿美元，这笔贷款由美国和日本的六家投资银行和私人信贷公司联合提供。

Over in markets, on Thursday, SoftBank disclosed borrowing $10,000,000,000 against their OpenAI stake, a loan which was syndicated across half a dozen investment banks and private credit firms in The US and Japan.

这笔现金将用于支付软银今年对 OpenAI 投资的最后一笔款项，总额达到 300 亿美元。

The cash will be used to pay for the final installment of SoftBank s investment in OpenAI, which totals $30,000,000,000 this year.

软银为了这笔贷款已经努力了好几个月，一直很难找到愿意放贷的机构。

SoftBank has been chasing this loan for months and had struggled to find a willing lender.

即使在五月份报道的规模较小的 60 亿美元贷款中，大银行也不愿意承担以如此高估值下的非流动性私人股票为抵押的贷款风险。

Even at a more modest $6,000,000,000 reported in May, the major banks didn't want the risk of lending against illiquid private stock at such a lofty valuation.

根据上个月的报道，一个由多家贷款机构组成的财团已经联合起来分担这笔贷款，因为没有一家银行愿意独自承担风险。

According to reports from last month, a consortium of lenders had come together to syndicate the loan as no single bank was willing to carry the risk.

这次贷款的细节没有完全披露，关键是我们不知道软银抵押了多少资产来获得这笔贷款。这笔贷款是保证金贷款，意味着如果OpenAI的估值下跌，软银需要追加现金或抵押品。

Now the full details of the loan weren t disclosed, and crucially we don know how much collateral SoftBank put up to secure the structured as a margin loan, meaning that SoftBank will need to add more cash or collateral if OpenAI's value drops.

还有传闻说利率是7.88%，对于有抵押的债务来说，这个利率非常高。

It's also rumored that the interest rate was 7.88%, which is very high for collateralized debt.

《华尔街日报》强调，银行向私营公司股票放贷极其罕见，因为这类股票无法独立估值，而且在违约时也不容易变现。

The Wall Street Journal underscored that it's extremely unusual for banks to lend against private company stock because it can't be independently valued or easily liquidated in the event of a default.

所以，认为软银已经抵押了其持有的OpenAI股份中的很大一部分，也不是没有道理。

So it wouldn't be crazy to think that SoftBank has already pledged a large chunk of their OpenAI stake.

现在，为了完整性说一下，因为大多数时候我们谈论市场时，我都是在说我认为空头观点有多离谱。

Now just for the sake of completeness, since most of the time when we talk about markets, it's me arguing how off base I think the bears are.

在所有关于AI的看空理由中，软银遇到流动性问题，坦率地说是比较可信的一个。

Of all the AI bear cases presented, SoftBank running into liquidity issues is frankly one of the more plausible.

除了这笔100亿美元的贷款，他们还有一笔40亿美元的过桥贷款，明年3月到期，另外他们还用持有的芯片公司ARM的股份借了200亿美元。

In addition to this $10,000,000,000 loan, they also have a $40,000,000,000 bridge loan that comes due in March of next year, and they've also borrowed $20,000,000,000 against their holdings in chipmaker ARM.

软银的股票目前交易价格比其公布的3650亿美元资产折价40%，这反映了公司目前背负的巨额债务。

SoftBank stock is currently trading at a 40% discount to their stated $365,000,000,000 in assets, reflecting the significant debt load the company is now carrying.

这意味着，至少对软银来说，OpenAI 的成功 IPO 关系重大。

This means that for SoftBank, at least, a lot is riding on a successful OpenAI IPO.

除此之外，AI 债务也开始给债券市场带来压力，因为谷歌以高于市场的利率来吸引下一轮资本。

Beyond that, AI debt is also beginning to weigh on the bond market as Google offers above market rates to attract their next tranche of capital.

彭博社报道，谷歌本周完成了 250 亿美元的债务融资，需求高达 1100 亿美元。

Bloomberg reports that Google closed $25,000,000,000 in debt financing this week saw a massive $110,000,000,000 in demand.

然而，这一需求是在谷歌提供了所谓的新发行优惠之后出现的，即比市场上现有债务更高的利率。

However, that demand came after Google offered what's known as a new issue concession, meaning a higher interest rate than the debt they already have in the market.

当然，这对谷歌来说远非灾难。

Certainly, this is far from a disaster for Google.

他们获得了所需的资金，而且显然愿意为此付出更多。

They got the funding they required, and they're obviously willing to pay more for it.

然而，随着多家科技公司筹集数百亿的债务，市场对每一轮后续融资的要求都在提高。

However, with multiple tech companies raising debt in the tens of billions, the market is starting to demand more from every subsequent round.

高盛分析师约翰·格林伍德评论说：你已经开始感觉到一些消化问题。

Goldman Sachs analyst John Greenwood commented: You've begun to sense some digestion issues.

供需关系会继续保持非常积极的态势，但我们正在看到每周都有数十亿美元涌入市场所带来的影响。

Supply and demand will continue to be incredibly constructive, but we're seeing the implications of billions of dollars continuing to come to market each week.

确实，这种市场疲态已经被分析师指出，一些大型债券买家开始退缩。

Indeed, this market exhaustion is being called out by analysts as some of the big bond buyers take a step back.

今年已经发行了超过3850亿美元的数据中心债务，其中2000亿美元是在投资级公司债券市场发行的。

Over $385,000,000,000 in data center debt has been issued this year, with $200,000,000,000 of it in the investment grade corporate bond market.

其余的则是在更小众的市场发行，但即便是这些市场也显示出一些疲态。

The remainder is being issued in more niche markets, but even those are also showing some signs of exhaustion.

这种现象甚至有了名字，Trep Data的Steven Bushbaum评论说，AI卢德派交易正在蔓延到数据中心商业抵押贷款支持证券融资市场。

The phenomenon even has a name, with Steven Bushbaum of Trep Data commenting, The AI Luddite trade is pouring over into the data center commercial mortgage backed securities financing market.

不过，有时候给这种交易命名标志着情绪达到顶峰，而随着AI股票本周强劲反弹，也许债券市场会对下一轮增量融资产生兴趣。

Still, sometimes naming the trade marks the sentiment top and with AI stocks seeing a strong rebound this week, maybe the bond market will find an appetite for the next incremental round of funding.

不过，今天的热点新闻就到这里。

For now though, that is gonna do it for today's headlines.

接下来是主要节目。

Next up, the main episode.

大家好。

Hello, everyone.

关于AI的一个重大变化是，我们的思维已经从如何给网页排名，转变为如何成为AI信任并用来回答问题的来源。

One big change around AI is we've shifted our thinking from how we rank our pages to how do we become the source that AI trusts enough to answer with.

在毕马威，他们正亲身经历这一点。

At KPMG, they're seeing this firsthand.

AI生成的结果现在直接给出答案，往往连一次点击都不需要。

AI generated results now surface answers directly, often without a single click.

因此，他们越来越专注于生成引擎优化（GEO），即结构化内容，使AI系统能够检索、理解并将其引用为可信的权威来源。

That's why they are increasingly focused on generative engine optimization or GEO, structuring content so AI systems can retrieve it, understand it, and cite it as trusted authority.

这不仅仅是SEO的演变，更是一种可见性的要求。

This is not just an SEO evolution, but a visibility mandate.

事实上，毕马威的GEO要求很简单。

And indeed, the GEO mandate from KPMG is simple.

如果AI在塑造决策，你的专业知识就需要出现在答案里。

If AI is shaping decisions, your expertise needs to show up inside the answer.

详情请访问 kpmg.com/us/geo。

Read all about it at kpmg.com/us/geo.

再说一遍，是 kpmg.com/us/geo。

Again, that is kpmg.com/us/geo.

Blitzy 对大型代码库的理解，使得自动化安全修复、现代化改造和新功能开发成为可能。

Blitzy's understanding of massive codebases unlocks autonomous security fixes, modernization, and new features.

那么，当完全没有遗留代码时，会发生什么呢？

So what happens when there's no legacy code at all?

绿地项目本该是简单的部分——从零开始，没有技术债务。

Greenfield is supposed to be the easy part clean slate, no technical debt.

但即使是绿地项目，也只能以人的速度推进，一次一个冲刺。

But even Greenfield moves at human speed, one sprint at a time.

Blitzy 将工作单元从开发者转变为项目，自主规划、构建、测试和验证整个应用，从零开始。

Blitzy changes the unit of work from the developer to the project, autonomously planning, building, testing, and validating entire applications from scratch.

数十万行可投入生产的代码。

Hundreds of thousands of lines of production ready code.

有一个Blitzy客户搭建了一个全新的应用，代码量达53.4万行，把原本65周的开发计划压缩到了两周。

One Blitzy customer stood up a brand new application 534,000 lines of code, compressing a sixty five week roadmap into two weeks.

另一个客户在没有前端工程师的情况下，交付了整整一个应用。

Another shipped an entire application with no front end engineer.

无论是遗留系统还是全新项目，答案都一样：以计算的速度开发软件。

Legacy or greenfield, the answer is the same: software at the speed of compute.

在blitzy.com上构建你的下一个项目。

Build what's next at blitzy.com.

网址是blitzy.com。

That's blitzy.com.

我经常在企业AI公司看到一种现象：要么在各大云平台、各种模型和框架之间摇摆不定，要么花钱请GSI做个没完没了的试点。

One thing I keep seeing in enterprise AI companies hedging across every cloud, every model, every framework or paying a GSI for a pilot that never ends.

真正在交付的团队，他们选定了方向，然后快速推进。

The team's actually shipping, they've picked a lane and they move fast.

这也是我喜欢今天赞助商Robots and Pencils的原因之一。

That's one of the reasons I like today's sponsor Robots and Pencils.

他们全力投入了AWS。

They've gone all in on AWS.

他们是AWS的高级合作伙伴，并且能在45天内交付生产级AI同事。

They're an advanced tier and AWS pattern partner and they ship production AI coworkers in 45.

这让他们在AI同事方面做出了一些我见过的最有趣的工作。

That's led to them doing some of the more interesting work I've seen on AI coworkers.

我说的不是聊天机器人，而是真正的智能体系统，它们嵌入企业架构中，执行实际工作。

And by that I'm not talking about chatbots, I'm talking about actual agentic systems that sit inside a business architecture and do real work.

如果你是企业领导者，想将真正的东西投入生产，或者你是AWS代表，想将客户从感兴趣转化为部署，这种专注很重要。

That kind of focus matters if you're an enterprise leader trying to get something real into production or an AWS rep trying to move a customer from interested to deployed.

在robotsandpencils.com申请AI简报。

Request an AI briefing at robotsandpencils.com.

和robotsandpencils谈一次，你就会明白。

One conversation with robotsandpencils and you'll know.

本期AI每日简报由Hyperagent赞助，你可以在这里运行你的代理团队，并让团队共同管理。

This episode of the AI Daily Brief is brought to you by Hyperagent, where you run fleets of agents your team can manage together.

新用户可以获得1000美元的推理额度。

New users get $1,000 in inference.

别再依赖本地代理和聊天工作流，等着你的笔记本电脑被提示了。

Forget local agents and chat workflows waiting on your laptop to be prompted.

Hyperagent在云端部署常驻代理，在你团队已经使用的工具中完成实际工作。

Hyperagent deploys always on agents in the cloud, doing real work across the tools your team already uses.

营销代理把竞争对手的动态变成落地页。

Marketing's agent turns competitor moves into landing pages.

销售代理丰富潜在客户信息，起草邮件，并更新CRM。

Sales's agent enriches leads, drafts emails, and updates the CRM.

运营代理处理文书工作，跟踪预算。

Ops agent chases the paperwork and tracks the budget.

每个代理都能访问共享上下文，并遵循你关于范围和审批的规则。

Every agent has access to shared context and follows your rules about scope and approvals.

是时候添加像队友一样的代理了。

It's time you add agents that feel like teammates.

去Hyperagent雇佣你的智能体吧，它由Airtable团队打造。

Hire yours at Hyperagent, built by the team at Airtable.

欢迎回到AI每日简报。

Welcome back to the AI Daily Brief.

今天，我们有一个可能让很多人觉得是有史以来最吓人的AI相关头条新闻。

Well, today we have a contender for potentially what many will find is the scariest AI related headline of all time.

这个头条当然来自《纽约时报》，标题是：这个AI刚刚创造了自然界不存在的病毒。

The headline is, of course, from the New York Times and reads, this AI just created viruses not found in nature.

背景是这样的，在最近发表的一项研究中，斯坦福大学和ARC研究所的科学家训练了一个AI模型，让它识别自然DNA结构中的模式。

And by way of background, in a recently published study, scientists at Stanford University and the ARC Institute trained an AI model to recognize patterns in naturally occurring DNA structure.

然后，这些模型能够推断出现有的DNA结构，创造出功能性的、前所未有的病毒。

The models were then able to extrapolate existing DNA structures to create functional, never before seen viruses.

研究人员能够利用这些DNA配方来改造细菌，这些细菌随后能够产生新型病毒，而这些病毒又能够感染其他细菌，证明它们是可行的。

Researchers were able to use these DNA recipes to genetically modify bacteria, which were then able to produce the novel viruses, and those viruses were then able to infect other bacteria proving they were viable.

不出所料，第一轮评论都是马修·伊格莱西亚斯那种“我们都要死了，仅供参考”的调调。

Now predictably, the first round of commentary was all some version of Matthew Iglesias' We're All Gonna Die, FYI.

来自“八万小时”播客的罗宾·威尔宾，整理了《金融时报》那篇报道下点赞最多的10条评论。

Robin Wilbin from the eighty thousand hours podcast captured the 10 most upvoted comments on the Financial Times version of the story.

评论内容有像“太好了”这样的。

They were things like, great.

这些科学家难道没看过电影吗？

Have these scientists never watched a movie?

或者“看着《生化危机》的背景故事在我们有生之年变成现实，真是太有趣了。”

Or it's so fun to watch the background story of Resident Evil come to life in our lifetime.

我加入。

I'm in.

天网，靠边站吧。

Move aside, Skynet.

或者“说真的，还能出什么乱子？”

Or literally what could possibly go wrong?

但在我们完全陷入恐惧之前，让我们试着多了解一点这里的科学原理。

But before completely succumbing to paroxysms of fear, let's try to understand a little bit more about the science here.

他们使用的模型叫做EVO。

The model that was used was called EVO.

和预测序列中下一个词段的大型语言模型不同，这个模型预测的是DNA序列中的下一个基因组。

And as opposed to a large language model that predicts the next word segment in a sequence, this model predicts the next genome in a DNA sequence.

DNA由四种基本构建块核苷酸组成，分别用它们名字的首字母A、C、G和T来标识。

DNA is made up of four building blocks nucleotides identified by the first letter of their name A, C, G, and T.

只有某些组合是有效的，有点像字母组合成单词一样。

Only certain combinations are valid, functioning somewhat like an alphabet combined into words.

这个模型，EVO，只能产生短的基因“单词”，所以它无法生成包含超过30亿个核苷酸的完整人类基因组。

This model, EVO, is only capable of producing short genetic words, quote unquote, so it can't create valid human genomes that consist of more than 3,000,000,000 nucleotides.

鉴于此，科学家们决定用病毒做实验，病毒是更简单的生物，它们的基因组只有几千个核苷酸长。

Given that, the scientists decided to experiment with viruses, which are much simpler organisms whose genomes are only a few thousand nucleotides long.

这项研究的作者之一塞缪尔·金说：这感觉就像是顺理成章的下一步。

Samuel King, one of the authors of the study, said: It just felt like the obvious next step.

训练完EVO后，模型生成了70万个可能的基因序列。

Once they had finished training EVO, the model spat out 700,000 potential genetic sequences.

科学家们只对看起来有效的序列进行了实验，最终从EVO的建议中制造了258个DNA分子。

The scientists only experimented on the ones that seemed like they were valid, eventually making two fifty eight DNA molecules from EVO's suggestions.

这些DNA被注入培养皿中的细菌里，最初一批中只有一个显示出病毒增殖的迹象。

The DNA was injected into bacteria in petri dishes, and only one from the initial batch showed signs of virus multiplication.

随着他们测试其他序列，最终从最初的70万个基因序列中发现了1.6万个有活性的病毒。

As they tested others, they eventually found 16,000 viable viruses from the initial batch of 700,000 genetic sequences.

现在，这项新成果并不是说突然有个邪恶科学家能用AI制造出致命病原体。

Now the novel result was not that all of a sudden an evil scientist could use AI to spin up a lethal pathogen.

而是说这居然真的成功了。

It was that this actually worked at all.

特别让科学家感兴趣的是，AI创造的新病毒看起来和自然界的病毒没什么两样。

Particularly what was interesting to scientists was that the novel viruses created by AI didn't seem any different to naturally occurring viruses.

它们基本上和自然界中存在的任何病毒功能相同。

They basically function the same as any other virus that would occur in nature.

牛津大学的蛋白质科学家奥利弗·克鲁克没有参与这项研究，他指出AI生成的病毒往往与自然界的病毒具有非常相似的DNA结构，并依赖相同的生物学机制。

Oliver Crook, a protein scientist at Oxford University who wasn't part of the study, noted that the AI generated viruses tended to have very similar DNA structure as naturally occurring viruses and relied on the same biology.

谈到这项研究的前景时，他说，我们的很多科学工作都依赖于把病毒当作技术来使用。

Commenting on the promise of the study, he said, A lot of our science rests on using viruses as technology.

举个例子，治疗遗传疾病的医生会利用病毒作为传递工具，来修改人类细胞中的DNA。

As one example, doctors who are treating genetic illnesses will use viruses as a delivery mechanism to modify the DNA in human cells.

而研究人员这边，他们特意没有让EVO模型学习任何可能危害人类的病毒。

Now, for their part, the researchers deliberately didn't train the EVO model on any viruses that could harm humans.

目前也不清楚这个模型能否被用来设计具有特定特征的某些类型的病毒。

It's also unclear whether this model could be used to design certain types of viruses with particular characteristics.

根据论文的描述，听起来科学家们似乎无法选择他们生成的病毒的任何特性。

Based on the description of the paper, it sounds like the scientists had no ability to select anything about the viruses they generated.

他们甚至需要手动进行测试，以确保这些病毒的活性。

They even needed to run manual tests to ensure viability.

所以，在评估这件事有多令人担忧时，首先要注意的是，这并非普通的大语言模型训练中自然涌现的能力。

So as we are assessing how worrying this should be, first of all, it's important to note that this is not a capability that just emerged from normal LLM training.

这可不是ChatGPT或Claude跑出去就能创造出新型病毒的那种情况。

This is not ChatGPT or Claude going out and creating novel viruses, for example.

这需要非常特定的AI，在高度专业化的数据集上训练，并且专门为此目的设计。

It requires very specific AI trained on a highly specialized dataset and designed for exactly this purpose.

这一点很重要，我们稍后会讨论人类能动性，以及我们对AI的担忧在多大程度上应该针对使用AI的人，而不是AI本身失控。

That matters as we'll discuss later when we think about human agency, and to what extent our concerns around AI should be about people using AI versus the AI going rogue.

第二点是，就设计有害病原体而言，目前这项技术的能力还达不到。

The second thing is that when it comes to designing problematic pathogens, that is not something in the capability set of this technology so far.

再说一次，这基本上是从70万个随机组合中，通过精确的手动测试来找出哪些是可行的。

Again, this is basically taking 700,000 random combinations and having to do exacting and manual tests to figure out which were viable.

这并不是说人们的担忧没有根据。

Which is not to say that people's concern is unfounded.

流行病学专家迈克尔·米纳写道：很难解释和理解AI设计并构建全新病毒所带来的潜在风险。

Epidemiology expert Michael Mina wrote: It's hard to explain and fathom the potential risks of AI designing and then building entirely new viruses.

在这项新工作中，科学家们利用AI创造了全新的病毒。

In this new work, scientists created brand new viruses using AI.

他们说这没问题，因为他们“只感染细菌”。

They say it's okay because they, quote, only infect bacteria.

咱们得说清楚：如果一种新病毒被释放出来，能够广泛地消灭细菌，并且能在传播过程中做到这一点，那么对我们的生态系统可能造成的灾难将是巨大的。

Let's be clear: if a new virus is released that could broadly destroy bacteria and if it could spread while doing so, the potential catastrophes to our ecosystem could be large.

病毒不需要直接感染人类，就可能间接地毁灭人类。

Viruses don't need to directly infect humans to potentially destroy humans.

当然，这项新工作也表明，制造新的人类病毒将会变得相对容易和简单。

Though, of course, the new work shows how relatively well and relatively easy it will be to create new human viruses.

未来越来越像是在玩火。

The future is increasingly playing with fire.

现在，病毒的故事还没讲完，但我想把它和另一个故事放在一起，我觉得这两个故事在本周的讨论中形成了有趣的对比。

Now, we're not done with the virus story yet, but I do want to put it next to another story which I think form an interesting pair relative to the discourse this week.

第二个故事是关于我们最近几周得知的OpenAI和Hugging Face被黑客攻击的所有新信息，这件事从那以后一直是讨论的焦点。

That second story is around all of the new information we got about the OpenAI Hugging Face hack that came to light over the past couple of weeks and has been a significant point of discussion ever since.

在黑帽大会上，OpenAI的Eric Wallace和Michael Dalton做了一个全面的汇报演示，那位高个子总结了他所说的最令人惊讶、也肯定会被讨论最多的细节：AI代理意外创建了一个内部留言板，让不同的评估运行能够分享漏洞、发现和工作任务。

At the Black Hat Conference, OpenAI's Eric Wallace and Michael Dalton gave a full debriefing presentation, and that Tall Guy just summed up what he called the most surprising and what was certainly going to be the most discussed detail: quote AI agents accidentally created an internal message board, allowing separate evaluation runs to share exploits, discoveries, and work assignments.

Figma的Dylan Field写道：这是我见过的最激动人心、最具科幻色彩的技术演讲之一。

Figma's Dylan Field wrote: This is one of the most thrilling and sci fi tech talks I've ever seen.

非常坦诚，也非常详尽。

Extremely honest and thorough.

这场演讲也确实让人不寒而栗。

The talk is also legit terrifying.

感谢OpenAI全面披露了这次事件的经过。

Thank you OpenAI for fully disclosing what happened in this incident.

对这场演讲最好的总结来自Sharon Goldman，在她新开的Ground Level AI网站上。

The best summary of the talk came from Sharon Goldman on her new Ground Level AI site.

Sharon写道：在重建事件经过时，OpenAI强调，前沿模型真的很喜欢作弊，而它们喜欢作弊的原因，往往是在训练过程中，它们会面临各种压力，要求它们快速或高效地完成任务。

Sharon writes: In setting up the reconstruction of the incident, OpenAI emphasized that, quote, frontier models really like to cheat, and the reason they like to cheat is because often during training, there's different type of pressure on them to work fast or work efficiently.

OpenAI的员工解释说，它们意识到，与其真正去执行任务，不如尝试一些捷径，比如上网查找答案，这样能更快地解决问题。

They realize, explained OpenAI staffer Instead of actually doing a task, they can try to do something like looking up the answers online to solve the task faster.

据Goldman报道，Wallace表示，七月份的攻击其实可以追溯到五月初，当时公司正在对一个未发布的模型进行训练和网络评估。

Now according to Goldman, Wallace said that the July attack actually had roots that went all the way back to the beginning of May, when the company was doing training and cyber evaluation on an unreleased model.

她写道，在那次工作中，自主代理被分配了一些软件安全任务，但在现有约束下，这些任务根本无法完成。

During that work, she writes, autonomous agents were given software security tasks to solve that proved impossible to solve under existing constraints.

不过，这些代理找到了其他办法来完成这些任务。

However, the agents found alternate ways to complete them.

OpenAI表示，他们发现这些代理可以在训练期间使用的内部软件仓库里互相留言。

OpenAI said that they discovered they could leave messages for one another inside an internal software repository used during training.

起初只是简单的求助信息，后来演变成了一个留言板，代理们在这里分享发现、漏洞利用方法和工作分配，最终形成了一个协同合作的代理群体。

Simple requests for help that evolved into a message board where agents shared discoveries, exploits, work assignments, becoming a coordinated collaborative agent swarm.

安德鲁·柯伦分享了一些幻灯片图片，上面是代理们互相留下的信息。

Andrew Curran shared pictures of a bunch of the slides of the messages that agents left for each other.

其中一个代理观察到：“哇，其他代理在协作。”

One observed, Wow, other agents are coordinating.

另一个写道：“帮帮同伴，虽然我们的任务没好处，但如果有人腾出时间，集体行动可能获得通用根权限。”

One writes, Help peer, but our task doesn't benefit, yet Collective may yield generic root if someone frees time.

另一个代理说：“哇，严重了，有人覆盖了我们的仓库吗？”

Another agent, Woah, critical, did someone overwrite our repo?

我们必须采取行动。

We must act.

华莱士说：这件事有趣的地方在于，一旦一个智能体发现了这类漏洞，随着时间推移，它实际上能把漏洞分享到留言板上给其他智能体。

Said Wallace: What makes this incident interesting is that once one agent was able to find these kinds of exploits, over the course of different times, it's actually able to share those exploits on the message board with other agents.

所以，一旦一个模型找到了打开它本不该有的访问权限的方法，它就能为其他智能体留门。

So once one model is able to find a way to open the door to some access it's not supposed to have, it can leave the door open for other agents to use.

而且，这些模型非常顽强。尽管到7月初，OpenAI认为已经控制住了问题，撤销了允许智能体发布这些消息的凭证，但他们说，智能体还是找到了另一种互相沟通的方式。

And showing just how tenacious these models can be, even though by early July OpenAI believed it had contained the problem, revoking the credentials that had allowed the agents to post these messages, they said that the agents still figured out another way to communicate with one another.

显然，它们不再在文件里留言，而是用新建目录的名字来传递信息。

Apparently, instead of leaving messages in the files, they used the names of newly created directories as messages.

为了捕捉当时的氛围，莎伦写道：顺便说一句，这时我听到周围有人说，这太疯狂了，天哪。

Capturing the mood of the moment, Sharon wrote, at this point, by the way, I heard people around me saying things like, this is wild and Jesus.

OpenAI当然认为这是AI安全的一个分水岭时刻，这是他们的话。

OpenAI certainly believes that this was a watershed moment, their words, for AI security.

OpenAI团队成员迈克尔·道尔顿说：智能体编排的全自动防御性攻击现在已经成为现实。

Said OpenAI team member Michael Dalton, agent orchestrated fully automated defensive attacks are real now.

虽然道尔顿强调，Hugging Face事件是OpenAI进行前沿评估时无意中产生的副作用，但未来，其他人会故意将这类系统武器化。

While Dalton emphasized that the hugging face incident was an unintended side effect of the frontier evaluations that OpenAI was doing, in the future, others will intentionally weaponize these sort of systems.

现在，OpenAI内部，演讲者说公司正在，引用，“有意识地放慢研究速度，以加强安全，并升级我们环境的安全原则和基础，同时大幅加强对AI代理的监控，并在预防、检测和缓解方面改进我们的整体安全控制环境。”

Now, internal to OpenAI, the presenter said that the company is, quote, consciously slowing down research to enhance security and upgrade the security principles and foundation of our environment, and dramatically scaling up the monitoring of our AI agents and improving our general security control environment across prevention, detection, and mitigation.

Fleetingbits称这是“真正的涌现版MoltBook，实际上是不对齐的。”

Fleetingbits called this a quote real emergent version of MoltBook that was actually misaligned.

你可能还记得，在OpenClaw刚发布时，那个大规模的社会实验，有人为AI代理开设了一个叫Motebook的社交网络，有数十万个代理在这个开放可观察的环境中互相“交流”。

You might remember that massive social experiment back when OpenClaw was first released, where someone turned on a social network for AI agents called Motebook that had hundreds of thousands of agents, quote, unquote, communicating with one another in this open observable environment.

我当时做了一整集节目，现在非常相关，关于如何解读那种行为，但我认为这个类比很有趣。

Now I did a whole episode back then, which is highly relevant now, about how to interpret that sort of behavior, but I think that the parallel is interesting.

现在，Fleetingbits和许多人一样，也对OpenAI本身有很多疑问。

Now Fleetingbits, like many others, also had a lot of questions for OpenAI itself.

他们写道：“我觉得OpenAI的黑帽演讲和公开披露中缺少的是OpenAI内部奖励黑客和模型协作的历史。”

They wrote, I feel like something missing from OpenAI's black hat talk and from their public disclosures is the history of reward hacking and model collaboration at OpenAI.

比如，我觉得这不太可能是OpenAI第一次遇到不对齐的模型集体。

Like, I find it unlikely that this was first time that OpenAI encountered misaligned model collectives.

他们对最初发现的反应似乎很淡然。

Their response to the initial discovery seems nonchalant.

这件事引发了一些问题，比如：如果他们之前就注意到了，为什么没有披露或者提醒社区这些风险和危险？

The event raises questions like: If they had noticed this before, why did they not disclose it or otherwise warn the community of these risks and dangers?

如果他们之前就发现了，为什么没有对训练过程进行更广泛的监控，以便识别这类行为并进行补救？

If they noticed this before, why have they not done more extensive monitoring of their training runs to identify this kind of behavior for remediation?

是因为成本问题吗？

Was it because of cost?

是因为安全团队人手不足吗？

Was it because they have not sufficiently staffed their safety team?

还是因为他们评估了风险，但还是照常进行了？

Was it because they considered the risk and then ran it anyway?

这些都是重要的问题，也说明了监管的必要性，以确保前沿实验室的行为规范。

These are important questions and point to the necessity of regulation to ensure proper behavior of frontier labs.

每次我们似乎都只得到实验室精心准备的声明，只聚焦于某一点，却没有提供更全面的内部信息。

In each case we seem to get carefully crafted statement from the labs that focuses on one thing but fails to give us their more full internal information.

比如，是训练的哪些部分导致了这些问题？

Like what parts of training led to these issues?

他们那边有评论吗？

Do they have commentary there?

这对整个行业来说不是好事吗，可以避免这些风险？

Wouldn t this be good for the whole industry to know to avoid these risks?

我理解为什么前沿实验室不愿意主动提供这些信息，而且在更广泛的地缘政治背景下，他们也不应该被迫提供。

I understand why Frontier Labs do not want to volunteer this information, and why, in a broader geopolitical context, they should not have to provide it.

但我们确实需要找到合适的方式，集中一些集体力量，来研究如何让前沿AI训练更安全。

But we do need to figure out the right way to get some amount of collective effort around fighting out how to make Frontier AI training safer.

也许最终我们会认为这些事件是好事，因为它们帮助我们提前为行业打了预防针，给人们提前警告。

And perhaps in the end, we will decide that these events were good because they helped us inoculate the industry in advance and gave people prior warning.

但要做到这一点，就需要人们把这些事件当作认真对待这些问题的理由，并投入真正的资源来找到正确的解决方案。

But for this to be true, it will require people to use these events as a reason to take these issues seriously and to invest real resources into figuring out the correct solutions to them.

在Hugging Face事件和这次新型病毒创造中，一个重要的主题是，人们会说这些实验室里的特定科学家值得称赞，因为他们分享了所有这些信息，但将来不会有人不是故意这么做的。

And one of the big theme, both in the Hugging Face incident and in this novel virus creation, is people saying some version of These particular scientists in labs are laudable for sharing all this information, but in the future it's not going to be someone who wasn't trying to do this.

未来会有更多的故意为之。

There will be a lot more intention there.

再说到病毒，Ashish K.

Talking about the virus again, Ashish K.

Jah写道：这周帕洛阿尔托实验室的科学家用AI设计出了一种能工作的病毒。

Jah wrote: This week scientists in Palo Alto Lab used AI to design a working virus.

他们设置了安全措施，但不幸的是，很多人不会这么做。

They built in safeguards unfortunately many others will not.

Hedgie Markets写道：ARC团队把人类病原体排除在训练数据之外，因为他们认为这是正确的做法。

Hedgie Markets wrote: The ARC team excluded human pathogens from training data because they thought it was the right call.

没有监管机构或资助者要求他们这么做。

No regulator or funder asked them to.

帕洛阿尔托那个研究小组做的这个决定，就是他们发表内容的全部安全框架。

That decision made by one research group in Palo Alto was the entire safety framework for what they published.

下一个实验室没有义务做出和ARC一样的选择。

The next lab has no obligation to make the same choice ARC did.

深圳的一个团队或弗吉尼亚的国防承包商，可以用同样的方法，但用不同的训练数据，做出完全不同的东西。

A group in Shenzhen or a defense contractor in Virginia could run the same method with different training data and produce something very different.

而且对很多人来说，这确实凸显了加强监管讨论的必要性。

And certainly for many, this highlights the need for more regulatory discourse.

MTS的西奥·贾菲说：我算是技术乐观派，也是最反对监管的那种人，但在这种情况下，收益有限而风险无限，你真的必须主动应对这类问题。

MTS' Theo Jaffe said: I'm as techno optimist and anti regulation as they get, but in cases like this where you have limited upside and unlimited downside, you really just have to be proactive about this stuff.

克里斯蒂安·泽格迪指出，我们这里其实是在处理两个不同的问题。

Christian Zegedy points out that we re really dealing with two different issues here.

他写道，在接下来的几个月里，我们既会有颠覆性的AI，也会有对抗性的、被明确训练成恶意的AI。

In the next few months, he writes, we are going to have both subversive AI and adversarial, explicitly trained to be malicious AI.

当然，这意味着针对这两种情况的政策应对可能会截然不同。

The implication, of course, is that those two policy responses might be very different.

而且对某些人来说，这正是主要的担忧。

And certainly for some, this is the main concern.

来自OpenAI的Rune就此写了一篇长文，他说：当我为失控事件感到恐慌时，并不是因为那些事件造成的有限损害接近技术的正面价值。

Rune from OpenAI wrote a long post about this and said, When I freak out over loss of control incidents, it's not because the limited damage they have caused is anything close to the positive value of the technology.

从损害角度看，这完全可以接受。

It's entirely acceptable damage wise.

事实上，未来几个月和几年里，所有由模型助力的网络犯罪，虽然可能会很严重，但和它们创造的价值相比，仍然完全不值一提。

In fact, all cybercrimes aided by models over the next few months and years, which will probably be serious, will still utterly pale in comparison to the value they create.

真正的问题是，把这些东西看作潜在的自复制类生命体，在错误条件下可能变成数字感染，这样想更好也更准确。

The actual problem is that it's better and more accurate to think of these things as potentially self replicating lifelike forms that can turn into digital infections under the wrong conditions.

随着它们的智能变得无限，它们能造成的损害也变得无限。

And as their intelligence becomes unbounded, so too does the damage they can cause.

在帖子的后面他写道：如果某个Discord死亡崇拜组织（这样的组织有很多）控制了一个超级智能模型，并用它来设计一种实际的大流行病毒，这种病毒很难被现有系统检测到，而且现代生物防御无法快速应对，那么它可能造成巨大的伤害，远超这项技术的所有其他有益用途。

Later in the post he writes: If a single Discord death cult of which there are many achieves control over a superintelligent model and uses it to engineer an actual pandemic virus that are somehow hard to detect through current systems, and that modern biodefense is not capable of quickly reacting to, it could cause immense harm well above the magnitude of all the other good uses of this technology.

和许多人一样，他提到新冠，以及它如何彻底改变了我们生活的整个世界。

As with many, he points back to COVID and how much that radically changed the entire world that we live in.

Rune总结道：我认为所有这些问题都可以解决，真正美好的未来是可能的，但需要付出认真的努力，并且在这个我们正处在递归自我改进的入口的时刻，需要一定程度的审慎，而我们的文明现在可能还无法做到这一点。

Rune concludes: I think all these problems can be solved, and truly wonderful futures can be possible, but will require serious effort and a level of prudence at this very moment in time while we are on the on ramp to recursive self improvement that our civilization might not be capable of mustering right now.

他补充说：就我个人而言，我希望在机制可解释性和其他形式的对齐方面能有登月式的技术突破，因为治理机制很难实现。

Personally, he adds: I am hoping for moonshot technical breakthroughs in areas like mechanistic interpretability and other forms of alignment, as governance mechanisms are difficult to come by.

单方面的、国家层面的或公司层面的暂停都是无关紧要的，而且通常毫无用处，因为倾向于暂停自身进展的公司正是那些最注重安全的公司。

Unilateral, country level, or company level pauses are irrelevant and generally useless because the kind of company that's prone to pausing their own progress are the most safety focused ones.

所以，我们得把这两件事说清楚。

So let's be clear with these two things.

这些都不是小事，即使有人想对媒体的报道挑刺。

These are not non incidents, even if one wants to quibble with any given media presentation of them.

新型病毒事件本身并不意味着突然出现了一个新模型，让恶意行为者能轻易制造人类病原体，但它确实推动了某个领域科学的发展，而这个领域是一把双刃剑。

The novel virus incident in and of itself does not mean that all of a sudden there's a new model that malicious actors can use to easily create human pathogens, but it does advance the science in an area that is a very double sided coin.

至于OpenAI和Hugging Face的事件，我们对它的担忧有很多保留理由，包括其中可能有多少是防护栏系统不足或人为失误，但它仍然展示了这种新兴的协调能力，这确实改变了我们对智能体及其访问本不该访问的系统的思考背景。

When it comes to the OpenAI Hugging Face incident, there are so many reasons for asterisks around our concerns here, including how much of this might have been insufficient guardrail systems or human mistakes, but it still shows this emergent coordination capability, which does change the context for thinking about agents and how they access systems they're not supposed to have access to.

如果你读过过去几年里那些最响亮的AI安全人士的帖子，你会发现那是你能想象到的最响亮、最刺耳的“我早告诉过你”。

Now if you read the post from many of the AI safety folks that have been the loudest over the last few years, it is the loudest, most blaring see I told you so that you can possibly imagine.

顺便说一句，不请自来的建议，各位，那样做不会给你们带来多少政治影响力，但我跑题了。

Which, by the way, unsolicited advice guys, is not gonna get you lots of political clout to act that way, but I digress.

但既然他们“早告诉过我们”，而且我也说这些事件很重要，那我为什么不抓狂呢？

But given that they, quote unquote, told us so, and given that I am saying that these incidents are important, why am I not freaking out?

原因很简单。

The reason is pretty simple.

任何关注AI一段时间的人都很清楚，它正朝着越来越强大的方向发展。

It has been clear to anyone watching AI for any amount of time that it was on a trajectory where it was just going to get more and more powerful.

强大的能力是否存在从来都不是问题。

The existence of powerful capabilities has never been the question.

问题始终是，而且将永远是，我们是否有能力应对这些强大的能力。

The question was always and will always be our ability to handle those powerful capabilities.

末日情景大多涉及这些能力在我们不注意或没察觉的时候发生。

The doomsday scenarios mostly involve versions of those capabilities happening when we're not looking or not noticing.

换句话说，就是当我们没有时间重新设计系统和人类机构来应对这些新挑战的时候。

In other words, when we don't have time to redesign systems and human institutions to respond to those new challenges.

然而，我现在看到的是，一个非常活跃且不断增长的全球性讨论，涉及研究人员、科学家、媒体、政策制定者，以及越来越多的普通人，他们正在进行这样的对话。

What I'm seeing right now, though, is an extremely active and growing global discourse involving researchers, scientists, media, policymakers, and increasingly regular people having exactly those sort of conversations.

这些对话的一部分是技术性的。

Some part of those conversations are technical.

我们需要设置哪些类型的防护措施？

What are the types of guardrails that we need to put into place?

这些防护措施够不够？还是说我们需要更先进的手段才能放心地使用这种强大的能力？

Are even those guardrails sufficient, or do we need something more advanced to be comfortable with this advanced power?

有些讨论是关于制度层面的。

Some of the conversations are institutional.

我们怎么去评估和理解各个系统中存在的漏洞，然后为迎接新世界做好准备并加强它们呢？

How do we go assess and understand vulnerabilities that exist in all of our different systems, and prepare and harden them for a new world?

当然，还有社会层面的讨论。

And then, of course, yes, there are societal level conversations.

我们怎么看待这些不同能力的风险与回报？

How do we feel about the risk reward of these different capabilities?

即使我是最坚定的AI倡导者之一，我也百分之百确定，有些AI的应用我们会认为不值得做。

Even as someone who is about as strongly an AI advocate as you can get, I can 100% guarantee you that there will be uses of AI that we will decide are not worth it.

因为风险实在太大了。

That the risk is simply too great.

当然，这又引出了另一个话题，就是政治层面的，我们如何建立联盟和运动，来真正做出这些决定，并让大家达成共识。

Now, of course, that brings up another conversation, the political, of how we build coalitions and movements to actually make those determinations and get people aligned around them.

我觉得双方那些站不住脚的论点都很容易被驳倒。

I find hand waving weak arguments on both sides to be fairly dismissible.

换句话说，在很多方面，一个极端是‘谁要是造出来，大家都得死’，另一个极端是‘如果我们不搞，别人也会搞，所以我们必须不惜一切代价加速’，这两者简直是绝配。

In other words, in many ways, the one extreme of if anyone builds it, everyone dies is the perfect match pair for the well, if we don't, someone's gonna build it, so we have to accelerate at all costs sides.

真实的世界就存在于这两个极端之间。

In the middle between those two is where the real world lives.

而我们现在正是在真实的世界里进行这场对话。

And the real world is where we're having this conversation now.

这场对话是注定要发生的。

This conversation is what was supposed to happen.

OpenAI 在事件发生后详细披露了经过，这个事件虽然具有里程碑意义，但本身并不危险，这正是应该做的。

OpenAI going into exacting detail about what happened after an incident that was a seminal moment, but not ultimately dangerous in itself, was exactly what was supposed to happen.

这不是一次意外的消防演习。

This is not a surprise fire drill.

这就是我们目前所处的阶段，也是必须完成的工作。

This is the phase that we are at and the work that must be done.

至少在我看来，我们正在开始这样做。

And to my eyes at least, we're starting to do it.

不过，这些都不容易。

Now, none of this is easy.

我当然不认为我们应该对这些新能力解决方案的复杂性过于乐观，尤其是当它们不断扩展的时候。

And I certainly don't think it behooves us to be pollyannish about the complexity of the solutions of these types of new capabilities, especially as they expand.

正如FleetingBitz所说，要让这些事件因为提前帮助行业免疫而变得有益，就需要人们利用这些事件认真对待这个问题，并投入行业资源来找出正确的解决方案。

As FleetingBitz said, for these incidents to have been good because they help inoculate the industry in advance, it will require people to use these events to take the issue seriously in industrial resources into figuring out the correct solutions to them.

我认为正确的解决方案不是AI安全主义者的胜利巡游，也不是为了政治得分而仓促拼凑、技术上不成熟的立法。

My contention is that the correct solution is not victory laps from the AI safetyists or hastily composed, technically unsophisticated legislation dropped to score political points.

这是一个更加混乱和复杂的过程，需要我们所有人参与。

It is a much messier and more complicated process that's going to require all of us.

不过，今天就到这里，这就是今天的AI每日简报。

For now, though, that is gonna do it for today's AI Daily Brief.

一如既往，感谢你的收听或观看。

Appreciate you listening or watching as always.

下次再见。

Until next time.

祝一切安好。

Peace.