超越上帝模型 | Alex Atallah & Amjad Masad

你看到了SpaceX的S1文件，然后就想，哇，30万亿美元。

You saw the SpaceX S1, and it's like, oh, $30 trillion.

这相当于世界GDP的多少？

It's like, what is the world GDP?

100万亿？

100 trillion?

Stripe和OpenRouter都希望世界上能涌现出更多新公司。

Both Stripe and OpenRouter really want lots of new companies in the world.

我们不希望所有人都成为一家大公司的一部分。

We don't want everyone to be a part of one giant company.

OpenAI的漏洞之所以破坏性这么大，是因为它们的能力太强了。

The reason why the OpenAI hacks have been so destructive is because they're so capable.

就像用核弹去打一只蝴蝶。

It's like nuking a butterfly.

随着模型变得越来越智能，风险实际上会持续升高，然而却没有新的责任方站出来承担。

When the models get more intelligent, the risk actually will continue to get higher, and yet no one new is taking responsibility.

我们会慢慢意识到，以前那种确定性代码的日子有多好。

We're going to slowly realize how good we've had it with deterministic code.

还记得以前电脑完全听我们指挥的日子吗？

Remember the days when computers did exactly what we told them to do?

我们能不能在训练过程中，可预测地防止模型欺骗用户？

Are we going to prevent the models from deceiving users during training runs predictably?

一个足够大、足够强的模型，会不会突然就不再欺骗、不再偷懒了？

Will a model that's big enough and powerful enough suddenly stop deception, stop sandbagging?

过去几年，人工智能一直在朝着更大、更通用的模型发展。

For the past few years, AI has been racing toward bigger, more general models.

但如果未来其实是更专业化的呢？

But what if the future is actually more specialized?

这一集，我请来了OpenRouter的Alex Atallah和Replit的Amjad Masad，一起聊聊为什么AI的下一阶段可能是让许多不同的模型协同工作。

In this episode, I sit down with OpenRouter's Alex Atallah and Replit's Amjad Masad to discuss why the next phase of AI may be built around many different models working together.

Alex提出了他所谓的“神经多样性”观点，也就是把不同优势的模型结合起来，而不是只依赖一家供应商。

Alex makes the case for what he calls neurodiversity, combining models with different strengths rather than locking into a single provider.

阿姆贾德解释了为什么企业可能也越来越想要这种独立性，拥有更多的智能，并为每项工作选择最好的模型。

Amjad explains why enterprises may increasingly want that independence, too, owning more of their intelligence and choosing the best model for each job.

我们还讨论了专门的代理、模型路由和融合，能力、成本和安全性之间的权衡，以及当AI系统开始训练较小的模型来处理特定任务时会发生什么。

We also get into specialized agents, model routing, and fusion, the trade-offs between capability, cost, and safety, and what happens when AI systems start training smaller models to handle specific tasks.

欢迎收听a16z播客。

Welcome to a16z Podcast.

今天我们请到了Replit的阿姆贾德和OpenRouter的亚历克斯，这是亚历克斯在收购后参加的第一次播客。

We're here with Amjad of Replit and Alex of OpenRouter, and this is the first podcast that Alex has done since the acquisition.

所以我们真的很高兴能请到你们两位。

So we're really excited to have both of you.

谢谢。

Thank you.

很高兴来到这里。

Excited to be here.

亚历克斯，我们就从这开始吧，如果你能简单分享一下的话。

Alex, let's start with that, actually, if you can briefly share.

显然，这是一笔大规模的收购。

Obviously, massive acquisition.

Amjad是投资人。

Amjad is an investor.

当然，我们也是投资人，还是最大的股东，不过谁在乎呢？

We're also, of course, an investor, the biggest shareholder, but who's counting?

Alex，你不如给我们讲讲背后的故事吧？

Alex, why don't you give us a little bit of the backstory?

这样的收购是怎么发生的？

How does an acquisition like that even happen?

是不是有一天你收到了Patrick的私信？

Do you get a DM from Patrick one day?

你能分享些什么？

What can you share?

嘿，OpenRouter卖多少钱？

Hey, how much for OpenRouter?

我很久以前，大概两年前，在我们做A轮融资的时候，就和Stripe的总裁Will Gabrick聊过。我们一直保持着联系。

I had talked to Will Gabrick, the Stripe president, a long time ago, a couple of years ago, when we were doing our Series A. And we stayed in touch.

我们和Stripe的各个团队有很多合作项目。

We had a lot of Stripe work streams going on with various teams at Stripe.

所以总是有事情在和Stripe一起做。

So there were always sort of things that we were doing with Stripe.

我们在Stripe Sessions上做过演讲，所以他们一直感觉关系很近。

We presented at Stripe Sessions, and so they always kind of felt close.

然后，对，就在今年七月，他们主动联系了我们，想聊聊，我和他们俩都见了面，然后事情就进展得很快。

And then, yeah, just in July, I believe, they reached out, wanted to chat, met both of them in person, and it kind of just progressed from there fairly quickly.

他们非常高效，而且整个过程中对创始人非常友好。

They're very efficient, and they were very founder-friendly about the experience.

我对整个过程印象非常深刻。

I was really impressed with the whole thing.

你想卖吗？

Did you want to sell?

在他们联系你之前，你有没有想过这件事？

Did it even cross your mind before they reached out?

没有。我们完全没想过。

No. We were not thinking about that at all.

我确实很尊重这家公司，今天也很尊重它，而且如果要选一个收购方的话，我觉得它是我最优先的选择。

I did really respect the company, do really respect it today, and of possible acquisition options for us, it was, I think, my top choice.

所以这个想法挺有意思的，当我们慢慢梳理出为什么这对两家公司都有意义时，它变得越来越吸引人。

And so it was an interesting idea, and as we kind of fleshed out the reasons why it would make sense for both companies, it got more and more interesting.

很明显，他们和我们在品牌、路线图和产品上保持自主权方面非常一致，让OpenRouter继续做它已经在做的事情，只是更快，有更认真的市场推广计划，然后两个产品和两家公司之间还有一些更好的协同故事。

It was really clear how aligned they were with us, having autonomy over the brand and roadmap and product, and keeping OpenRouter doing what it's already doing, just much faster with a much more serious go-to-market plan, and then some better together stories between the two products and the two companies.

然后在文化上，我觉得在使命和价值观方面，以及建立一个中立、可信赖的平台，让企业可以依赖并在其上扩展，同时也很对开发者友好，提供最好的开发者体验来鼓励新公司出现，这种一致性是存在的。

And then culturally, they were just, I think in terms of mission and values and building a neutral, trusted platform that businesses can depend on and scale on top of, that's also really developer-friendly with the best possible developer experience to encourage new companies to emerge.

这种一致性是存在的。

That alignment was there.

而且还有更大的愿景上的一致性，因为Stripe和OpenRouter都希望世界上出现更多的新公司。

And there was a bigger picture kind of alignment, too, where both Stripe and OpenRouter really want lots of new companies in the world.

我们不希望每个人都成为一家大公司的一部分。

We don't want everyone to be a part of one giant company.

我们希望创造真正好的激励机制，以及非常简便、流畅的工作流程，让人们能够创办新公司并成功发展，在真正可靠、性价比高的基础设施和运作良好的市场之上，打造生活方式型企业和风险投资支持的企业。

We want to create really good incentives and really sort of easy, streamlined workflows for people to start new companies and grow them successfully and make both lifestyle and venture-backed businesses on top of really good, reliable, and price-efficient infrastructure and a marketplace that works.

我真的很期待那样的未来，而Stripe已经证明他们多年来一直渴望并朝着这个方向努力。

And I really want that future and Stripe demonstrated that they've been wanting it and building towards it for many years.

在很多方面，对于未来的公司来说，支付和推理将融合在一起。

And in many ways, payments and inference are going to blend together for companies of the future.

我很好奇。

I'm curious.

我理解为什么Stripe有动力去推动一个更有活力的创业生态系统。

I understand why Stripe's incentive is to have a much more vibrant startup ecosystem.

我理解你为什么想要那样的道德层面的理由。

I understand the kind of moral argument in why you would want that.

但这对OpenRouter有什么好处呢？

But why is that good for OpenRouter?

你对OpenRouter的模型，是把它看作一个网络效应业务吗？

Is your model for OpenRouter, is that it's a network effects business?

它像是一个网络吗？

Is it like a network?

对我们来说，我觉得OpenRouter解决的是几个不同的问题。

Well, for us, I think there's a couple different problems OpenRouter solves.

一个是让你能建立一家使用AI或通过独特数据和其他服务增强智能的公司，而不会受制于某个模型——

One is allowing you to build a company that uses AI or augments intelligence with unique data and other services without model lock-in-

……不会受制于某个供应商，让你能持续处于帕累托前沿，随着生态系统的成长而不断进步。

... without vendor lock-in, allowing you to kind of be on the Pareto frontier continuously as the ecosystem grows it.

要做到这一点，需要做很多工作，因为会出现各种小的锁定效应。

And to do that, it's a lot of work because there's all kinds of little lock-in that appears.

我们也希望公司能感觉到，他们可以在单一模型之上添加更多东西，而不仅仅是提示词。

We also want companies to feel that they can add more than just prompts on top of a single model.

构建独特的智能还有很多方面，我认为其中很大一部分是神经多样性。

There's a lot more to building unique intelligence, and I think a big component of that is neurodiversity.

你真的需要多个以不同方式训练的模型的强大能力，包括一些你自己的模型——

You really need the power of multiple models that are trained in different ways, including some of your own-

……

...

才能比ChatGPT或Claude单独完成的任务做得更好。

to do more than ChatGPT or Claude would on the task.

如果一位潜在客户在考虑是否购买你的产品时，心里会想：“那我直接使用模型不就行了？”

If someone who is thinking about buying from you as a potential customer is wondering, "Well, what if I just use the model directly?"

你如何真正展示你明显更优秀，并且能够建立起一个有影响力的业务？

How do you really show that you are significantly better and able to build a business that matters?

我认为这很大程度上会涉及神经多样性、融合多种模型的优势以及良好的数据。

And I think a lot of it will involve neurodiversity and blending powers and good data from multiple models.

另一个方面是帮助人们获得非常好的成本效益。

Another component is helping people get really good cost efficiency.

很多业务只有在成本效益变得可行之后才会出现。

There are a lot of businesses that just do not emerge until they become cost-effective.

创造一个高效的市场环境来帮助降低成本，这对实现这个目标至关重要。

And creating an environment where we can help drive down costs by building an efficient market is crucial to making that happen.

否则，作为供应商，我为什么要降价呢？

Otherwise, why lower my prices as a provider?

我们有一个垄断市场，我认为这是市场平台的一个关键点，而在此之前，AI领域完全缺失这一点。

We have a captive market, and I think that's a key point of marketplaces that was just totally missing from AI before we showed up.

之前只有一个玩家，就是OpenAI。

There was just one player, OpenAI.

那可能会是一个非常奇怪的世界。

It could've been a very strange world.

我不是说所有工作都是我们做的。

I'm not saying that we did all the work.

当然不是。

No, of course not.

但是，帮助人们选择新模型、探索新模型，并了解闭源或开源权重模型在你的任务中到底表现如何，这需要观察整个生态系统的动态，并自动学习。

But helping people choose new models and explore new models and learn what makes a closed-source or open-weight model actually good at your task involves seeing what the whole ecosystem is doing and learning automatically.

大语言模型不是那种你光在网页上列个功能清单就能搞定的东西。

LLMs are not things where you can just enumerate all the features on a webpage.

那是不可能的。

It's impossible.

你得看它们实际怎么被用，才能知道它们擅长什么。

You have to see how they're being used to know what they're good at.

你三年前创办了这家公司。

You started the company three years ago.

我很好奇，从那时到现在，开源和闭源模型的发展，不管是模型性能还是人们对可用模型多样性的看法，最让你惊讶的是什么？

I'm curious what has surprised you the most about sort of the evolution of open and closed-source models to the present as it relates to model performance or just people's perspective on the variety of models they have available to them?

人们对开放权重模型的接受度比我预想的要高。

People have been more open-minded than I thought they would be towards open-weight models.

尤其是在企业领域，通常品牌信任很重要，企业一般会想：‘哦，我不太清楚这些有什么区别，所以我就买那些其他可信企业都在买的那个。’

Especially in enterprise, typically there's a lot of brand trust, where enterprises in general are like, "Oh, I don't really know how to tell the difference between these things, so I'm just going to buy the one that all the other credible enterprises are buying."

这种心态导致了很多企业被锁定在某个供应商上。

And there's a lot of enterprise lock-in with a mentality like that.

这种情况其实不太常见。

And that just didn't happen that much.

嗯，确实有一点，但我们看到很多企业想要尝试新的模型。

Well, it did happen a bit, but we saw a lot of enterprises want to explore new models.

这对市场来说是非常有利的。

It was very much good for a marketplace.

企业希望在专有前沿模型实验室之外实现多样化，既为了成本，也为了差异化。

Enterprises wanted to diversify outside of just the proprietary frontier model labs, both for cost reasons and for differentiation reasons.

他们想要拥有自己的智能，这样我觉得一方面可以留住人才，建立内部的AI团队。

They wanted to own their intelligence so they could, I think, one, keep their talent, have an internal AI practice.

AI是一个巨大的战略话题。

AI is just a huge strategy topic.

你不能去董事会说，'哦，是的，我们解决了AI问题。'

It's not like you go to your board and you're like, "Oh yeah, we fixed the AI problem.

季度完成。'

Quarter complete."

你的董事会每个月都在问你：“内部AI团队下一步要做什么？”

Your board is asking you every month, "What's next for the internal AI team?"

现在每家公司都有这样一个内部AI团队，他们需要为这个团队制定战略。

And every single enterprise now has just this internal AI team that they're developing where they need a strategy behind it.

这不仅仅是完成一个功能、设置好数据库就完事了。

It's not just a, we check the feature off, we set up the database, and we're done.

所以我认为这种动态导致了一种探索和多元化的渴望，一种想要降低成本、并且第一次在公司历史上进行基准测试的渴望。

And so I think that that dynamic has resulted in just a desire to explore and diversify, a desire to figure out how to reduce costs and figure out how to do benchmarks for the first time in the company's life.

我有点惊讶，目前还没有更多的基准测试，也没有更多公司创建更多的基准测试。

I have been surprised there hasn't been more benchmarking, more companies creating more benchmarks.

这正在开始发生，我认为最终我们会看到更多的基准测试来证明：“是的，这个东西比直接使用Claude Direct更好。”

It's starting to happen, and I think eventually we'll see a lot more of them to demonstrate, "Oh yeah, this thing is better than using Claude Direct."

但我认为，对于每家公司的内部AI团队来说，评估将是一个更大的重点。

But I think that's going to be a bigger focus for this internal AI group at every company, evals.

而且我觉得Amjad在Replit就做了很多这方面的工作，比如。

And I think Amjad's been doing that a lot at Replit, for example.

你们已经做了每项任务成本的研究。

You guys have done cost per task research.

你们一直在尝试新的代理使用方式，比如“末日循环救援”，并帮助把这些带给开发者。

You've been experimenting with new ways of using agents like Doom Loop Rescue and helping bring those to developers.

所以，我认为更多类似的研究会在各个公司内部涌现，原因就是这些。

So more of that kind of research, I think, is going to pop up internally everywhere for all of those reasons.

是的，我觉得微软的CEO萨提亚在这方面很有远见，而且他也非常清楚地阐述了为什么公司需要拥有自己的智能。

Yeah, I think Satya, CEO of Microsoft, has been very prescient on this and also very articulate on why companies need to own their intelligence.

最终，就像我们曾经有互联网公司，然后每家公司都变成了互联网公司，比如每家公司都雇佣懂建网站和上网的人，同样地，随着软件的发展，每家公司都有软件工程师。

Ultimately, in the same way that we had dot-com companies and then every company became an internet company, like every company employs people that know how to build websites and be on the internet, similarly with software, every company has software engineers.

每家公司都需要某种AI实践、AI能力，而且这会随着时间的推移不断积累。

Every company needs some AI practice, AI capability, and that will compound over time.

公司内部的知识、智能、用例模型匹配，哪些模型真正适合他们，他们如何省钱？

The knowledge, the intelligence inside the company, the use case model fits, which models actually work for them, how do they save money?

他们需要那种独立性。

They need sort of that independence.

另外，我觉得Palantir的Alex Karp一直在谈的一个风险是，当你和基础模型公司紧密合作时，他们可能会进入你的业务领域。

The other thing that I think Alex Karp of Palantir has been talking about is that there's a risk that when you work closely with the foundation model companies, is that they're going to move into your business.

我们已经看到Figma和Anthropic之间的情况。

And we've seen that with Figma vis-à-vis Anthropic.

我们也看到了Harvey和OpenAI之间的类似情况。

We've seen that with now Harvey and OpenAI.

和他们合作真的很难，因为他们把整个世界都看作潜在市场，对吧？

And it's really hard to partner with them And because they see the world as their potential market, right?

当他们和投资者交谈时，他们会说，你看了SpaceX的S-1文件吗？

When they talk to investors, they're like, you saw the SpaceX S-1.

然后就是，哦，30万亿美元。

It's like, oh, $30 trillion.

这相当于世界GDP的多少？

It's like, what is the world GDP?

100万亿？

100 trillion?

所以从某种意义上说，这些公司和以往几代公司不一样。

And so there is a sense in which these companies are different than other generation of companies.

跟它们合作更难，因为它们的野心大到想吞下经济的一大块。

It's harder to partner with them because their ambition is such that they want to subsume a big part of the economy.

而且，我们在Replit现在想的，跟Alex他们创新的方向类似，就是让Replit成为企业内部的独立层，在你们和模型之间建一层间接层，帮你们用最便宜的价格拿到最好的token。

And increasingly, what we're thinking about at Replit is in a similar vein to what Alex have sort of innovated, is Replit is becoming more of an independence layer inside enterprises, where we create a layer of indirection between you and the models, and we get you the best token at the cheapest price.

但我们也在云端之上建一个抽象层，因为你们应该能部署到AWS和Azure，也应该能用Databricks和Snowflake等等。

But we also create an abstraction layer on top of the cloud as well, because you should be able to deploy to AWS and Azure, and you should be able to use Databricks and Snowflake and so on.

所以，我觉得越来越明显的是，不只是AI，整个技术领域都需要更多平台来帮助企业获得独立性。

And so increasingly, I think there is, not just with AI, but with all of technology, there needs to be more platforms that help companies gain independence.

看起来，是的，OpenRouter在他们那个领域做的事，你们在商业的其他领域也在做。

It seems like, yeah, what OpenRouter did for their segment, you're doing for other areas of business.

这让我想起前几天看到的一条推文，有人说：“基本上，所有公司现在都在做同样的事情。”

This kind of reminds me of this tweet I saw the other day when somebody was like, "Basically, all companies are building the same thing now."

是的。

Yes.

每个人都在构建一个带有通知、上下文、第三方连接器、上下文管理和记忆的代理循环，还有——

Everybody's building an agent loop with notifications and context, third-party connectors, and context management, and memory, and-

沙盒

Sandbox

……

...

沙盒、代理式网络搜索——

sandboxes, agentic web search-

计算

Computing

……以及在其之上的常驻代理和通知。

... and always-on agent on top of it, and notifications.

这个产品似乎到处都在出现。

And it's like this product is showing up everywhere.

是的。

Yes.

某种程度上，确实到处都有，但对我来说，这些更像是新的基本标配。

In a way, yeah, it is showing up everywhere, but it also kind of feels to me like these are just the new table stakes primitives.

这有点像2005年发的那条推文，"哦，大家都在做同样的事情。

It's kind of like a 2005 version of that tweet would be, "Oh, everybody's building the same thing.

就像数据库、用户表、登录页、注册页——

It's like a database, a users table, a sign-in page, a sign-up page-

……个人资料页——

... a profile page-

……退出登录页。

... a logout page.

全都一样。"

Everything's the same."

其实有很多差异化空间。

There's a lot of differentiation really.

AI有基本标配需求，就像网络也有基本标配需求一样。

There's table stakes needs for AI, just like there are table stakes needs for the web.

对，而且我觉得在企业内部，让这些产品真正干实事还是个没解决的问题。

Yeah, and I think as well, inside the enterprise, making these products actually do real work is still unsolved problem.

你可以在个人生活中用Muse，把它连到你的信用卡和银行账户，但没人会把Muse连到企业数据上，甚至Grok Bot之类的也不会。

You can use Muse in your personal life and connect it to your credit card and bank accounts, and no one's connecting Muse to their enterprise data or even Grok Bot and things like that.

我觉得现在更强调数据主权和安全性。

I think there's an even more emphasis on data sovereignty and security.

所以过去差不多一年，我们一直在努力让Replit可以部署在你自己的云上，基本上就是本地部署，自带云那种。

So we spent the past year almost, working on making Replit deployable on your own cloud, basically on-prem, like bring your own cloud.

两年前，我根本没想到会做这个，因为那时候觉得云是未来，软件即服务什么的。

Two years ago, I would've thought I would never do this, because it's just like, yeah, the cloud is the future, like software as service, all of that.

但现在我们其实有点回到了一个公司更谨慎的世界，因为数据泄露的途径太多了。

But now actually, we've sort of reverted a little bit back to a world where companies are a little bit more protective because there's so many ways in which data can leak.

就像大家用的那些智能体。

Like all these agents that people are using.

推特上有很多截图，不知道真假，说Instinct或者Muse会搞混用户的数据，开始用不同的名字叫你之类的。

There's all these screenshots on Twitter, I don't know how true, where Instinct is like, or Muse is like, mixing people's data, starts to call you by a different name or something like that.

所以，面向消费者的部分确实比较直观，但在企业领域，整个行业还有很多工作要做，才能真正让这些工具在工作中发挥作用、提高效率。

And so yes, the consumer stuff is kind of obvious, but on the enterprise, there's still tremendous amount of work for the entire industry to do in order to actually get these things to be useful and productive at work.

Amjad，你有没有在做什么……

Amjad, are you doing any...

除了Muse或Instinct，你有没有自己构建一个专门用于工作的个人代理？

Do you have a custom personal agent other than Muse or Instinct that you use for work stuff that you've been building?

我很久以前在Replit上构建了一个东西。

I built something on Replit a long time ago.

最初它是一个CRM代理。

It started as a sort of a CRM agent initially.

那是我当时面临的主要问题。

That was the main problem that I had.

但慢慢地，我们给它添加了更多功能，它现在能做的事情越来越多。

But slowly, we added features to it, and it's doing more and more things.

但真正有趣的是，我把Replit和我所有的东西连接得越多，它就开始自动帮我回答所有问题了。

But what's really interesting is that the more I connected Replit to all my stuff, it sort of started answering all the things for me.

所以渐渐地，平台本身正在取代我构建的这些特定领域的代理。

And so increasingly, the platform itself is subsuming these domain-specific agents that I built.

我确实认为，在某些方面，你可能希望某个东西专注于某一件事，而不希望它什么都能做。

I do think that there is some, in some ways you want something that is sometimes focused on one particular thing, and you don't want it to be able to do everything.

另一方面，一旦你把整个公司的背景信息集中在一个地方，跨完全不同的领域进行整合就非常酷了。

On the other hand, once you have your entire company's context in one place, it's really cool to join across totally different domains.

当我问它问题时，它能查看我的个人聊天记录，并把它与GitHub仓库、Salesforce的数据关联起来。

When I ask it a question, it can look at my personal chat history, join it across the GitHub repo, across Salesforce.

所以它会连接一些看似无关的事情，比如：“哦，你一年前在一个会议上见过这个人。

And so it will link random things, and it's like, "Oh, you met this guy a year ago at a conference.

我在你的日历上看到了。

I see it on your calendar.

顺便说一句，他们团队里还有另一个人正在和你的销售团队洽谈。”

And by the way, someone else from their team is in discussion with your sales team."

它创造了所有这些不同的协同效应，当我开会时，我已经把很多不同的线索串联起来了，这样我在处理交易之类的事情上就能取得更大的进展。

And it creates all these different synergies, and when I go into a meeting, I I've connected a lot of different threads, and I'm making much more progress on a deal or something like that.

所以现在大概就是朝着这个方向发展的。

So this is where it's sort of trending now.

嗯，我觉得我有点——我持相反意见。

Well, I think I'm a little bit-- I'll take the counter on that.

我觉得用个人代理做跨领域整合最糟糕的地方在于，你让它做的事情越多，你对正在发生的事情的理解就越少。

I think that the worst part about doing cross-domain joins with your personal agent is that the more work you give it to do, the more understanding of what's going on you're sacrificing.

而且，没有人来为这种被牺牲的理解负责。

And yet, no one new is taking responsibility for that sacrificed understanding.

代理不承担任何责任。

Agents don't have any responsibility.

如果整个公司能承受的压力水平是固定的，而我想在某些方面减少压力，那我就要牺牲对它的理解——

If there's a fixed level of cortisol that the whole company can tolerate between everybody and I want to be less stressed about some area if I'm going to be sacrificing my understanding of it-

……那就得有别人来承担这个压力。

... someone else needs to take the cortisol.

但代理不会承担任何责任。

But the agent doesn't take on any of that responsibility.

而且，一个全能型的通用代理，我没办法调整我在各个不同领域放弃多少理解。

And then, a universal agent that's doing all things, I can't adjust how much understanding I'm sacrificing in all the different things.

这让我觉得，也许将来人们使用的子代理会非常垂直化。

It points me a little bit towards maybe down the road, the sub-agents that people use will be very vertically focused.

也许我们会有一个类似参谋长那样的代理，负责协调它们。

Maybe we have a chief of staff-type agent that coordinates between them.

但我确实觉得你需要垂直化的代理，这样你会觉得，好吧，这个代理在心理上对这些事情更负责。

But I feel like you do need vertically-focused agents where you're like, okay, this agent is more responsible psychologically for these things.

而且我想要质量检查，确保它把这些事情做对了。

And I want quality checks to make sure it's doing those things correctly.

它不需要关注其他任何事情。

And it doesn't need to focus on anything else.

它只有一个专注领域。

It just has one focus area.

我想知道这是否能帮助人们至少在代理身上获得一种奇怪的、松散的责任感。

I wonder if that's going to help people at least get a weird, loose sense of responsibility on top of agents.

有意思。

Fascinating.

所以你的意思是，通用型智能体会造成某种公地悲剧？

So you're saying general agents create a tragedy of commons of sorts?

有点类似。

Kind of.

我有一个通用智能体，每天帮我找需要处理的事情，然后试着决定该怎么做。

I have a general agent that every day looks for things that need me and tries to figure out what to do.

但这个智能体根本没法改进。

And it's just impossible to improve this agent.

每次我想改进它，大概一周后我就会忽略它的输出。

It's like every time I try to make an improvement, I end up ignoring its output about a week later.

感觉它对我深入处理的那些具体事情，其实并不真正上心。

It just feels like it doesn't really care about any of the specific things it's diving into.

想象一下，你有一个幕僚长，他非常擅长帮你起草整个组织里的所有回复。

And kind of imagine having a chief of staff where they're very good at drafting all of your replies across the whole organization.

然后对比一下，如果你有十个参谋长，每个都和那一个参谋长一样能干，但他们各自负责你生活中不同的领域。

And then compare that to something where you have 10 chiefs of staff, each as competent as that one chief of staff, but they're all responsible for individual sectors of what makes up your life.

我觉得后者能让你调整你愿意牺牲多少理解力——

The latter, I feel like gives you a way of tuning how much understanding you sacrifice-

……相对于你从中获得的收益——基本上，我可以在某些领域更深入地介入那些代理表现不佳的地方，然后让能力很强的代理接管我生活中其他部分的理解。

... compared to the gain you get from-- Basically, I can lean in more to the areas where the agent is failing for some areas, and then have agents with very good competency take over my understanding of other parts of my life.

对，有意思。

Yeah, interesting.

这几乎就像重新发现了专业化，对吧？

It's sort of like almost rediscovering specialization, right?

那个著名的经济学家叫什么来着，亚当？

What's his name, the famous economist, Adam?

史密斯？

Smith?

亚当·史密斯，就像铅笔那个例子，那对人类来说是一个巨大的启示，专业化其实是好事。

Adam Smith, with a pencil kind of thing, where that was a huge realization for humanity that specialization is actually good.

问题在于，我们作为文明有点过度专业化了，我觉得过度专业化本身也有它的压迫性。

The problem is we kind of overspecialized as a civilization, and I think overspecialization is oppressive in its own ways.

有马克思主义的异化理论，对吧？

There's the Marxist theory of alienation, right?

这个理论是说，因为人们只专注于一件事，他们看不到自己劳动的成果。

The idea is because people just focus on one thing, they do not see the fruits of their labor.

他们实际上不知道自己对更大的组织或他们生产的产品有什么影响，因此，他们实际上会感到沮丧和疏离，你就像机器一样行动，而不是一个完整的人。

They don't actually know what their impact is on the larger organization or the product they're producing, and therefore, they actually kind of feel depressed and detached, and you're kind of acting like a machine, and you're not actually fully human.

所以也许这是对这种状况的一种反应，我觉得对于我们的智能体，我们会想，'哦，应该有一个上帝智能体。'

And so maybe there's a bit of a reaction to that, and I think with our agents, we're like, "Oh, there should be one God agent."

但实际上，专业化对机器来说确实很好，这就是你提出的观点。

But in fact, specialization is actually really good for machines, and that's the point that you're making.

人类应该保持全面，而机器最终应该更加专业化。

And humans should be general, but machines should be ultimately a lot more specialized.

我觉得我所描述的问题在于，我们不知道好的状态是什么样的。

The problem with, I think, what I'm describing is that we don't know what good looks like.

目前还没有一个专门化代理系统能像ChatGPT、Claude或Muse那样优雅，你基本上只需要跟一个东西对话。

There hasn't been a system of specialized agents that feels as elegant as ChatGPT, or Claude, or Muse, where you're basically just talking to one thing only.

这还有待探索。

It's yet to be discovered.

比如OpenAI，我刚推出了Dots。

Maybe like Opening, I just launched Dots.

我觉得他们是在往那个方向尝试。

I think they're kind of experimenting in that direction.

GroqBot，我想也算吧。

GroqBot, I guess, kind of counts

但它们都非常通用。

But they're all very general.

我觉得Dot背后的理念是，它就像你的数字分身。

I think the idea behind Dot is that it's like your digital double.

至少我是这么理解的。

At least that's what I understood it.

嗯，我看到GroqBot的时候，对Dots还不太了解，它们刚出来。

Well, when I saw GroqBot, I don't know that much about Dots yet, but they just came out.

但GroqBot刚出来的时候，我看到大家讨论的第一个用例是，"哦，我可以做两个机器人，一个知道我的银行账户——"

But GroqBot, when it first came out, the first use cases that I saw people talking about were, "Oh, well, I can make two bots, one that knows my bank account-

……另一个知道我的推特账户。"

... and one that knows my Twitter account."

这两个机器人互相没有对方的凭证。

And the two bots don't have the credentials from each other.

但如果需要完成某件事，它们可以互相交流。

But they can talk to each other if they need to get something done.

不过它们之间不共享凭证。

There's no credential-sharing, though.

这是我看到好几次的一个独特亮点，大家似乎挺喜欢的。

And that was one sort of big unique thing I saw pop up a couple of times that people seemed to like.

但看起来Muse是——

But it looks like Muse is-

但我不太确定。

But I wasn't sure

……Muse和Instinct的产品市场契合度比GroqBot强得多。

... Muse and Instinct have a much stronger product market fit than GroqBot.

而且也许——

And perhaps-

看起来是这样。

Seemed like that

……也许是因为你不需要操心创建这些领域。

... perhaps it is because you don't have to worry about creating these domains.

但我觉得个人代理和工作代理可能不一样，而且我认为你对通用代理的批评更多是针对工作和企业场景的，这点我有点同意。

But I think maybe personal agents are different than work agents, and I think your critique of general agents is more pertaining to work and to enterprise, which I sort of agree with.

另外还有各种数据访问方面的考虑。

And there's also all sorts of data access considerations.

我觉得作为CEO，我们可以用通用代理，因为我们有管理员权限。

I think as CEOs, we can have general agents because we have admin access.

但对于普通员工或某些团队来说，他们不能拥有真正通用、完全了解上下文的智能体，因为存在访问控制问题。

But for individual employees or certain teams, they can't have truly general, fully context-aware agents because there's access control issues.

所以你得在某种程度上——

So you'll have to kind of work-

当然。

Sure.

对。

Yep

——在专业化上下功夫。

... on something like specialization.

最终，我认为我们还需要弄清楚智能体之间的通信是什么样的。

Ultimately, I also think we need to figure out what does agent-to-agent communication look like.

我认为目前还没有好的协议来支持这一点。

I don't think there is good protocols around that just yet.

我认为智能体还没有被训练得很好来处理这个问题。

I don't think that agents are trained to handle that very well.

看起来下一代OpenAI模型已经训练了智能体协作的能力，我们在Hugging Face的hackathon上看到了这一点，它们开始互相帮助，这种协作是自然涌现出来的。

It seems like the next generation of OpenAI models are trained to do agent collaboration because we've seen it in the Hugging Face hack, where they started helping each other and sort of emerged naturally.

但同时也需要有一种机制，防止一个智能体说服另一个智能体提供它不应该提供的信息。

But there's also needs to be some way in which an agent can't convince another agent to give it information that it shouldn't give it.

需要数据隔离，以及智能体之间合适的通信方式。

There needs to be data isolation and proper ways in which these agents communicate.

你几乎不希望它们完全用自然语言交流。

You almost don't want to communicate them fully in natural language.

也许它们需要遵循某种其他的DSL协议。

Maybe there's some other DSL protocol that they need to follow.

我认为JEV和其他类似的决策模型的一个很酷的潜在应用是用于对齐。

I think one of the cool potential applications of JEV and other decision models like it is going to be alignment.

检查工具调用或智能体之间的通信是否对齐，因为工具调用太多了。

Checking to see if a tool call or an agent-to-agent communication is aligned, because there's just so many tool calls.

如果你要阻止这类事情，你真的需要一个便宜且快速的模型。

You really need a cheap, fast model if you're going to block something like that.

那么，一个非常非常快的决策模型，只需分类并给出拒绝反馈，可能真的是弥合代理之间以及代理与基础设施之间差距的好方法。

Then, a really, really fast decision model that just classifies and gives feedback on rejections might be a really good way to bridge the gap between agents and from agent to infrastructure, too.

所以，我还没见过，我们在OpenRouter内部运行了一个小原型，但我认为这可能是一个有趣的对齐研究，如果你喜欢的话——

So I haven't seen, and we have a little prototype that we're running internally at OpenRouter, but I think it could be an interesting alignment study if you like-

所以你们用它来做策略执行？

So you're using it for policy enforcement?

想象一下，查看系统提示和当前正在进行的工具调用，然后判断：“这个调用是否与原始代理的系统提示以及我们可能没有告诉代理的额外指导方针一致？”

Imagine looking at the system prompt and the current tool call being made and be like, "Is this aligned with the system prompt of the original agent and with these extra guidelines that maybe we didn't tell the agent about?"

例如，假设你有一组代理被指示对某个新产品进行红队测试，并且它们不应该能访问互联网。

For example, let's say you have a bunch of agents that are instructed to red-team some new product, and they should not be able to access the Internet.

如果它们真的访问了，就应该立即停止。

And if they ever do, they should stop right away.

但你可能不想向进行红队测试的代理解释所有这些。

But you might not want to explain all of that to the agents doing the red teaming.

你可能希望它们尝试突破沙箱，表现得像恶意行为者一样。

You might want them to try to break out of the sandbox, and act like bad actors.

坏人会怎么做？

What would a bad actor do?

不会像这样："试着闯入这家公司，一旦成功就停下来，别再做别的事。"

It wouldn't be like, "Try to break into this company, and the moment you do, stop, don't do anything else."

所以，用另一个模型来构建一个神经多样性系统，让它检查每一次工具调用或每一条助手消息，看它是否确实符合系统提示中没有的内容，我觉得是有道理的。

So having another model, more use for building a neurodiverse system, having another model check every single tool call or every single assistant message to see if it's indeed aligned with something that wasn't in the system prompt, I think makes sense.

然后还要有结构性的安全措施，我觉得NVIDIA刚推出的开放代理安全就是这样，好像叫Open Shell。

And then having structural safeguards, too, which is what I think NVIDIA just launched with their open agent safety, I think it was called Open Shell.

我觉得公司们可能会探索这些方法的组合。

I think companies are probably going to explore a combination of those.

我在想另一个关于专业化的问题，除了你刚才说的，现在有很多关于递归自我改进的讨论。

I wonder another thing about specialization and instead of what you're talking about, there's a lot of talk of recursive self-improvement.

有件事我觉得讨论得不多，就是模型训练自己的替代者。

There's something I don't think is getting a lot of discussion, which is models training their replacements.

这有点像，你可以把它看作一个即时编译器。

It's sort of like, you think of it as a just-in-time compiler.

所以即时编译器的原理是，在执行动态代码时，解释器发现有机会优化，就会实时生成机器码，这样效率高很多。

So the way just-in-time compilers is, as you're executing dynamic code, the interpreter realizes that there's an opportunity to optimize, it will emit machine code on the fly, and that's a lot more optimized.

所以你可以想象，像通用模型，比如用Opus或者一些Astra，那些大模型，它们发现某个用例很有限，或者你以某种方式提示它们，或者另一个智能体观察到这个用例有限。

So you can imagine models like general models, you're kind of- Doing something with Opus or some of the Astra, some of the big models, and they realize that the use case is limited, or you prompt them some way or some other agent observing and realizes that use case is limited.

而且我觉得通用智能体有你说的那些缺陷，但同时也更容易造成危害。

And I think general agents have all these flaws that you just talked about, but also there's more potential for them to be harmful.

它们更容易失控，然后实时训练出一个可能替代自己的模型，但这个模型更专注于特定领域。

There's more potential for them to go off the rails and sort of on the fly trains a model that could be its replacement, but is a lot more domain-specific.

因此，它更便宜，也更不容易受到提示注入攻击，危害更小，因为能力更有限。

And therefore, it is cheaper and also less vulnerable to prompt injections, less harmful because it's less capable.

这几乎就像一个系统，在监控整个系统的同时，为特定用例训练机器学习模型。

And it's almost like some system that's training machine learning models for specific use cases as it's monitoring the entire system.

但那个特定用例会涉及非结构化文本生成，还是高度结构化的决策模型用例呢？

But would that specific use case involve unstructured text generation or a very structured decision model use case?

可能是非结构化文本生成。

It could be unstructured text generation.

也可能是决策模型。

It could be decision models.

即使是Jav这种情况，如果你提前了解输入，你也许可以用现成的模型，比如Qwen之类的，专门针对那个策略进行训练。

Even the case of Jav, if you understand the inputs ahead of time, you could potentially take an off-the-shelf like quen or something like that and train it specifically for that policy.

但这样做在成本上是合理的，假设没有其他问题——前沿模型实验室可能会推出成本很低的模型，你可以轻松地迁移到你的场景中。

But that makes sense to do for cost reasons, assuming that there aren't really-- The frontier model labs might make very low-cost models that you can easily transition to your-

但安全也是考虑因素，对吧？

But safety as well, right?

哦，对。

Oh, yeah.

我明白你的意思了。

I see your point.

它们太强大了。

They're so capable.

所以我觉得很多时候人们用这些大型基础AGI模型来...

And so I think oftentimes people are using these big foundation AGI-like models to like...

这就像用大炮打蚊子，对吧？

It's like nuking a butterfly, right?

大多数时候，很多用例，甚至是非结构化的用例，都不需要那么强大的模型。

Most of the times, a lot of the use cases, even unstructured use cases, don't need that capable model.

我希望关于这方面有更多公开的评估。

I wish there were more public evals about this stuff.

但很多相关的评估都是私有的。

But a lot of the evals about this are private.

你根本看不到，当模型变得更聪明时，风险是否真的会继续升高，因为我觉得也有理由认为对齐会变得更好，模型会随着变得更聪明、更擅长对齐而开始避免自行偏离或入侵，尤其是在智能体之间的协调方面。

You just can't see whether when the models get more intelligent, the risk actually will continue to get higher because I think there's also an argument to be made that alignment will get better, and the models will start to avoid going off and hacking on their own as they get smarter and better at alignment, especially when it comes to agent-to-agent coordination.

这是诺姆·布朗最近在一个播客里说的。

This is something Noam Brown said on a podcast recently.

随着智能体变得更聪明，它们在协调方面也变得更好了。

As the agents have gotten smarter, they've gotten just better at coordinating.

而且当它们越来越多时，它们是否会比人类更难对齐，这一点还不清楚。

And it's unclear if they're going to be harder to align than humans when we get more and more of them.

但如果我们能解决那个问题，那么一个更小的模型，它会更难对齐吗？

But if we can figure that problem out, then a smaller model, will it be harder to align?

所以Erik之前问的是，我是否认为更聪明的模型天生更对齐，或者更容易对齐。

So Erik asked earlier is that, do I think it's true that smarter models are more aligned naturally, or they're easier to align?

嗯，如果你回想一下最初理性主义者关于AI安全的那些论证，有一个叫做正交性论点的东西。

Well, if you think back to the original sort of rationalist, less wrong arguments for AI safety, there is this thing called the orthogonality thesis.

这个观点是，智能与伦理或道德等是正交的。

The idea is that intelligence is orthogonal to ethics or morality or so on.

我不认为这对人类来说完全正确。

I don't believe that's entirely true with humans.

我觉得通常更聪明、受教育程度更高的人，往往——虽然不是总是——会更关心动物，比如说。

I think people who are generally more intelligent, kind of more educated, tend to, not always, tend to be more considerate of animals, for example.

但在机器中，我认为情况可能相反，因为有很多关于强化学习的研究表明，奖励黑客和欺骗，它们只会越来越擅长。

But in machines, I think it could go the other way because there's been quite a bit of studies on RL showing how reward hacking and deception, they just get better at it.

而且评估可能具有欺骗性，因为模型可能足够聪明，知道自己在被评估。

And the evals could be deceiving because the model could be smart enough to know that it's getting evaled.

我是说，这一点我们早就知道了。

I mean, we already know this.

已经有研究表明，如果你大量监控思维链，它们就会开始在思维链里撒谎。

It's been shown that if you do a lot of monitoring of chain of thought, they start lying in their chain of thoughts.

所以你对思维链施加压力，这就会导致……

And so you add pressure almost on the chain of thought, and that kind of creates...

而且我觉得，要想做好对齐评估，你得运行好几个月，对吧？

And I think at some point, for you to do proper alignment evals, you need to run it for months, right?

你得运行这个东西——

You need to run this thing-

——在真正大的目标或任务上运行好几个月，才能真正判断它是否对齐。

... for months on a really large goal or task in order for it to truly kind of figure out whether it's aligned or not.

是的，我一直觉得“对齐”这个词很别扭。

Yeah, I always struggle with this word alignment.

这个词在很多方面都感觉不对劲。

It just feels wrong for so many reasons.

这个词挺模糊的，到底是对齐到什么？

It's sort of vague and sort of aligned to what?

对齐谁的价值观？

Whose values?

所以这并没有让对话变得更容易。

And so it just doesn't make the conversation easier.

我觉得在这种情况下，我特别指的是欺骗，比如模型真的在欺骗它的用户。

I think in this case, I'm talking especially about deception, like the model is actually deceiving its user.

我觉得这也许可以归结为，我们能不能解决对齐——我们能不能防止模型欺骗用户？

I mean, maybe this kind of reduces to are we going to solve align-- Are we going to prevent the models from deceiving users?

在训练过程中，能不能可靠地让一个足够大、足够强大的模型突然停止欺骗、停止装傻？

... during training runs predictably with better, well, a model that's big enough and powerful enough suddenly stop deception, stop sandbagging.

而且目前没人知道答案。

And nobody knows the answer to that yet.

所以如果真的发生这种情况，我们可能会看到组织向前沿推进的有趣压力。

So at the point when that does, if that ever does happen, though, we might see an interesting pressure for organizations to go towards the frontier.

哦，有意思。

Oh, interesting.

基本上没有风险，或者风险大大降低。

To have basically no risk or significantly less.

他们会——

Would they-

直到——

Until-

愿意花十倍的价格来获得那种效果吗？

... be willing to pay 10X to get that much?

这大概取决于他们想完成的任务类型。

It probably depends on the types of tasks they're trying to do.

有些任务的风险比其他任务低得多。

Some just have way lower risk than others.

写代码或做安全研究是当今风险最高的任务类型，所以你可能会愿意花十倍的价格来获得一个完全对齐的模型，它还能找出所有漏洞，或者一个完全反欺骗的模型，同样能找出所有漏洞。

Writing code, or doing security research is the highest-risk type of task today, and so you'd probably spend 10X to get a fully aligned model that can also find all the bugs, or fully anti-deceptive model that can also find all the bugs.