大型语言模型会是通往超级智能的途径吗？——与穆斯塔法·苏莱曼的对话

微软AI首席执行官重返节目，将解释公司为何现在追求超级智能、其含义何在，以及在最新OpenAI协议后微软的发展方向。

Microsoft's AI CEO returns to explain why the company is now pushing for superintelligence, what that means, and how Microsoft is moving forward after its latest OpenAI deal.

广告结束后马上回来。

That's coming up right after this.

工业X革命盛会汇聚了来自IFS、Anthropic、波士顿动力、微软、西门子的领导者，以及全球最具前瞻性的工业公司，共同探索工业AI在现实世界中的前沿应用。

Industrial X Unleashed is bringing together leaders from IFS, Anthropic, Boston Dynamics, Microsoft, Siemens, and the world's most progressive industrial companies at the frontier of industrial AI applied in the real world.

市场正在发生明显转变。

There's a clear market shift happening.

全球最大工业企业已结束对AI的试验阶段。

The world's largest industrial enterprises are done experimenting with AI.

他们正在大规模部署AI技术，并选择与IFS共同创新。

They're deploying it at scale, and they're choosing IFS to co innovate with them.

IFS专为资产和服务密集型行业打造，包括制造业、能源、航空航天、建筑业等，这些领域停机成本高达数百万，安全标准不容妥协。

IFS is purpose built for asset and service intensive industries, manufacturing, energy, aerospace, construction, where downtime costs millions and safety is nonnegotiable.

工业X革命将展示嵌入实际运营的AI产品实时演示，客户分享可衡量的成果，以及当前大规模部署工业AI企业的经验分享。

Industrial X unleashed will feature live demos of AI products embedded in real world operations, customers sharing measurable outcomes, and learnings from companies deploying industrial AI at scale today.

了解更多信息请访问industrialx.ai。

Learn more at industrialx.ai.

事实上，AI安全就是身份安全。

The truth is AI security is identity security.

AI代理不仅仅是一段代码。

An AI agent isn't just a piece of code.

它是您数字生态系统中的一等公民，理应获得相应待遇。

It's a first class citizen in your digital ecosystem, and it needs to be treated like one.

这就是为什么Okta要率先保护这些AI代理的安全。

That's why Okta is taking the lead to secure these AI agents.

解锁这层新防护的关键在于身份安全架构。

The key to unlocking this new layer of protection, an identity security fabric.

企业需要统一全面的方法，通过一致的政策和监督来保护每个身份，无论是人类还是机器。

Organizations need a unified comprehensive approach that protects every identity, human or machine, with consistent policies and oversight.

不要等到发生安全事件才意识到你的AI代理是个巨大的盲点。

Don't wait for a security incident to realize your AI agents are a massive blind spot.

了解Okta的身份安全架构如何帮助你保护下一代身份，包括你的AI代理。

Learn how Okta's identity security fabric can help you secure the next generation of identities, including your AI agents.

访问okta.com。

Visit okta.com.

网址是0kta.com。

That's 0kta.com.

欢迎收听《大科技》播客，这是一档冷静深入探讨科技界及其他领域话题的节目。

Welcome to Big Technology Podcast, a show for cool headed and nuanced conversation of the tech world and beyond.

今天，我们再次邀请到微软AI首席执行官兼公司新超级智能团队负责人Mustafa Suleyman，他将与我们探讨超级智能的含义、这项技术的未来前景，以及我们当前是处于技术曲线的末端、起点还是中间阶段。

Today, we're joined once again by Mustafa Suleyman, the CEO of Microsoft AI and also the head of the company's new superintelligence team who is here to speak with us about what that means, what superintelligence is, but more broadly, what the future of this technology is going to look like and whether we're at the end of the curve or the beginning or somewhere in the middle.

我们马上开始深入探讨。

Anyway, we'll get into it all.

Mustafa，很高兴再次见到你。

Mustafa, great to see you again.

欢迎来到节目。

Welcome to the show.

嘿，Alex。

Hey, Alex.

很高兴再次见到你。

Great to see you again.

谢谢邀请我来。

Thanks for having me.

总是很愉快。

It's always a pleasure.

最近你写了一篇关于微软推动所谓'人文主义超级智能'新方向的帖子。

And so recently, you wrote this post about a new push towards what you call humanist superintelligence at Microsoft.

你说你们正在开发你称之为'极其先进的人工智能能力'，这些能力始终为人类和更广泛的人道主义服务。

You say, you're working towards it at what you call incredibly advanced AI capabilities that always work for in service of people and humanity more generally.

让我问你一个关于这个的问题。

Let me ask you a question about this.

看到这么多实验室都在追求他们所谓的超级智能，我觉得很有趣，我想这有点像AGI的酷炫版本。

It's so interesting to me to see so many labs running towards what they call superintelligence, which I guess is sort of like a cooler version of AGI.

研究结果好坏参半，关于我们是否会在当前范式下看到更多进展。

As the research is mixed about whether we're going to see a lot more progress with the current paradigm.

很多人都在谈论边际效益递减。

A lot of people are talking about diminishing marginal returns.

我想我们讨论过这个。

I think we've talked about that.

关于大语言模型在推动人工智能前沿发展方面的可行性存在一些疑问。

There's some questions about the viability of LLMs in terms of pushing the state of the art in AI forward.

然而我们也看到，这种向超级智能推进的趋势。

And yet it's we're also seeing, you know, this push towards superintelligence.

所以请解释一下我们刚开始提到的这种差异。

So just explain as we begin sort of the discrepancy there.

为什么我们听到这么多关于超级智能的讨论，却连当前方法能否实现其前一步——通用人工智能都不确定？

Why are we hearing so much about superintelligence where we're not even sure if the current methods are gonna get us to the step before, which is AGI?

是的。

Yeah.

我认为超级智能和通用人工智能更像是目标而非方法。

I mean, superintelligence and AGI are really goals rather than methods.

这个雄心是要在大多数或全部人类任务上创造超人的表现。

And I think that the ambition is to create superhuman performance at most or all human tasks.

比如我们想要拥有医疗超级智能。

Like, we want to have medical superintelligence.

我们希望最顶尖的医疗诊断技术能变得廉价且普及，让全球数十亿人都能获得。

We want to have, the best expertise in medical diagnosis be cheap and abundant, and available to billions of people around the world.

我们还希望随时获得近乎免费的世界级法律咨询，每月只需几美元。

We also wanna have world class legal advice on tap that costs almost nothing, a few bucks a month.

我们需要财务建议。

We wanna have financial advice.

我们需要情感支持。

We wanna have emotional support.

我们需要随时可用的软件工程师。

We wanna have, software engineers available on tap.

我认为超级智能项目的核心问题是：我们究竟要构建何种极其强大的智能系统？

And I think that the project of superintelligence is about saying, what type of very, very powerful intelligence systems are we actually gonna build?

我想提出的建议是，我们对每项新技术都进行一个非常简单的测试。

And what I'm trying to propose is that we subject each of these new technologies to a very simple test.

比如，它是否在实践中真正提升了人类文明的前景？是否始终将人类置于食物链顶端？

Like, does it in practice actually improve the prospects of human civilization, and does it always keep humanity at the top of the food chain?

这听起来像是需要声明的某种简单或显而易见的事，但在我看来，科学技术的目标就是推动人类文明进步，保持人类掌控权，并为全人类创造福祉。

It sounds like a kind of simplistic or obvious thing to have to declare, but the goal of science and technology science and technology, in my opinion, is, like, to advance human civilization, to keep humans in control, and to create benefits for all humans.

我认为过去几年的某些言论中，你能感觉到一种逐渐蔓延的假设——这类系统注定会超越我们的控制和能力范围，最终凌驾于人类这个物种之上。

And I think in some of the rhetoric in the last few years, you can feel that there's a little bit of like, you know, a kind of creeping assumption that it is inevitable that these kinds of systems exceed our control and our capability and move beyond us as a species, as a human species.

而我正通过人文主义超级智能的框架来反驳这种观点。

And, I'm pushing back on that idea with the framing around humanist superintelligence.

我认为这是截然不同的。

I think it's quite different.

那么你的观点是超级智能不会是一种广泛的智能，而是可以在某个领域（比如比最优秀的医学专家更聪明）实现超级智能，但在其他领域（例如会计）可能尚未达到？

But then is your view that superintelligence won't be one broad intelligence, that it will be you can maybe achieve super intelligence in one discipline when it's smarter than, let's say, the best doctors in medicine, but maybe it's just, like, not there in accounting, for example?

一种思考方式是：当前我们训练这些模型时，会通过垂直领域展开工作，确保训练数据、知识、专业能力和思维链条能反映人们在每个学科中培养专业素养的各类活动。

One way of thinking about it is that how we train these models at the moment is that we work through verticals, and we make sure that we have training data, knowledge, expertise, reasoning traces, chains of thought that reflect the kinds of activities that people do in each one of these disciplines to build their expertise overall.

所以我们其实已经在从垂直化角度训练通才模型。

So we're already training generalist models from a verticalized position.

我们首先会明确要优化的具体任务是什么。

We're starting off by saying what specific tasks are we trying to optimize.

而人文主义超级智能项目首先要明确的是：这项技术将带来什么好处？如何确保其安全性、可控性并与人类利益保持一致？

And, you know, the project of humanist superintelligence is first trying to say what good will this technology do, and how will it be safe and controllable and aligned to human interests.

而安全性的其中一个维度，就是垂直化。

And one of those dimensions of safety, is verticalization.

如果一个模型被明确设计用于实现医疗超级智能，那么根据定义，它就不会成为世界上最好的软件工程师。

If a model has been designed explicitly to achieve medical superintelligence, then by definition, it isn't going to be the best software engineer in the world.

它也不会成为最优秀的数学家或物理学家。

It isn't going to be the best mathematician or physicist.

因此适当缩小领域范围——不是过度缩小，因为不能完全消除通用性——但通过限制领域和降低通用性，我认为这是有助于增强控制力的方法之一。

And so narrowing the domain, not too much, not entirely because you can't collapse it, but narrowing it and reducing the generality is one of the ways that I think, is likely to help create more control.

这不是唯一的解决方案。

It's not the only solution.

要实现可控性和对齐性还有许多其他方面，但领域专用模型是其中的一部分。

There are many other aspects of, like, how we achieve containment and alignment, but domain specific models are one part of it.

是否存在某种超级智能但不具备通用智能的情况？

Is it possible that something can be super intelligent but not generally intelligent?

比如说，超级智能是否可能在无需AGI的情况下出现？因为AGI的核心在于通用性，而你讨论的恰恰相反？

Like, is it possible that maybe super intelligence happens without AGI because AGI is all about generality and what you're talking about is not?

我认为这是不可能的。

It's not possible, I don't think.

我认为它们必须具备通用性。

I think they need to be general.

它们需要能将知识从一个领域迁移到另一个领域。

They need to transfer knowledge from one domain to another.

它们需要具备通才式的推理能力。

They need to, you know, have generalist reasoning capabilities.

但当你将其投入应用并部署到生产环境，赋予它更多决策自主权，或允许它生成任意代码来解决特定问题，甚至让它自行编写评估标准以便修改自身代码、生成新提示来创建新训练数据、撰写新评估标准，进而迭代优化自身表现——这些能力，包括自主性、目标设定、代码编写、自我修改等，如果再结合一个完全通用化或通用目的的模型，那将构建出一个极其强大的系统。而当今，恐怕没人真正知道该如何约束或对齐这样的系统。

But when you apply it and you put it into production and you let it have more autonomy to make decisions or you let it generate, arbitrary code to solve a particular problem or you let it write its own evals so that it can modify its own code and generate new prompts to generate new training data, to write new evals, to then iterate on its own performance, These capabilities, autonomy, goal setting, writing code, modifying itself, you know, if you add to that then also a perfectly generalized model or or sort of general purpose model, that's a very, very, very powerful system, which today, don't think anybody really knows how we would contain or align something like that.

这并不是说我们应该完全回避这些发展方向中的任何一个。

And so it's not to say that we should not do any one of those dimensions.

只是要勾勒出一幅能力发展路线图——我们都在这些领域努力，但它们会带来更多风险，尤其是当这些能力相互叠加、全部结合在一起时。

It's just to outline a road map of capabilities which we're all working on, which add more risk, especially when they compound with one another and you combine them all together.

因此我认为我们应该谨慎对待，时刻谨记不要把这些能力全部捆绑在一起，否则会大大提高出现递归式自我改进的指数级爆发增长、最终取代人类物种的风险。

And so, you know, my claim is that we should just approach this with caution, remembering that we don't wanna bundle together all these capabilities so that there's a higher risk of a, you know, recursively self improving exponential takeoff that then replaces our species.

根据目前情况，我认为这种可能性极低，但在未来十年左右的时间里，这确实是我们必须认真对待的风险之一。

And and I think that that is very low probability from what I see today, but it's one that we have to take seriously in the next, like, ten years or so.

好的。

Okay.

我稍后确实想讨论那个话题，但先让我说说这些对话中让我觉得奇怪的地方。

I do wanna get to that in a bit, but let me tell you what I find odd about these conversations.

我想回到最初问你的问题：研究人员都在讨论当前方法正在遭遇瓶颈。

And I wanna go back to the first question that I asked you, which is researchers are talking about how the current methods are leveling off.

举个例子：

Give you one example.

数据并不充足。

Data is not plentiful.

合成数据目前还不够实用。

Synthetic data, not very useful yet.

算力可能即将耗尽。

Power might be running out.

你需要那种规模，很多人说，为了让这些模型变得更好，或者至少运行基本功能。

And you need that scale, a lot of people say, in order to make these models better or at least even to run the basic capabilities.

考虑到LLM的局限性，你是否看到了我们没注意到的东西，那将为超级智能铺平道路？

So given the limitations of LLMs, are you seeing something that we're not and that will sort of pave the way to superintelligence?

我的意思是，你如何从这里到达那里？

I mean, how do you get from here to there?

好的。

Okay.

我认为我们受到功率限制，但不是根本性的功率约束。

I think we're power limited, but not fundamentally power constrained.

显然，人们非常渴望建造更大的数据中心，在更大、更连续、连接更紧密的集群中进行训练。

Clearly, there's, like, huge appetite to build bigger data centers and train in larger, more contiguous cluster more fully connected clusters.

也就是所有芯片都相互连接的集群。

So clusters where all the chips are connected to each other.

但目前这不是瓶颈。

But that's not the bottleneck at the moment.

这并没有阻碍进展。

That's not holding back progress.

显然，如果我们现在有更多资源，肯定会有帮助，但系统中还有许多其他因素在拖慢进度。

Obviously, if we had more right now, it would definitely help, but there's many, many other things in the stack that are slowing down progress.

如果我们现在不受数据限制，我们正在生成大量高质量合成数据，这些数据被证明是有用的。

If we are not data constrained right now, we're generating vast amounts of high quality synthetic data, which is proving to be useful.

显然，同样的情况也成立。

Obviously, again, the same is also true.

虽然更高质量的数据会很好，但我不认为这两件事中的任何一个会减缓进展速度。

Like, more high quality data would be great, but I I don't see an slowing down in progress because of either of those two things.

恰恰相反，过去五年的发展速度简直疯狂。考虑到我们现在的基础——训练运行常常需要50兆瓦、100兆瓦甚至很快会达到500兆瓦的电力，却还指望我们每三个月就让用于最大模型训练的集群规模翻倍。

If anything, the rate of progress has been insane over the last five years, and to expect us to continue to make doublings every three months in the size of clusters that are trained for the largest models, you know, given the base that we're now starting from when training runs are often, you know, 50 megawatt or a 100 megawatt or soon 500 megawatt.

要知道，在这种基数上每半年翻倍一次是不可能的。

You know, you can't just double on that every six months.

物理法则确实会带来限制，而且我们讨论的是价值数百亿美元的集群。

There's there's like the laws of physics kind of do create restrictions, and we're talking about tens of billions of dollars of, you know, cluster.

所以发展速度可能会稍微放缓，但显然仍将快得难以置信——客观来说确实如此。

So pace might slow a little, maybe, but it's also clear that pace is still going to be unbelievably fast, like, you know, sort of sort of objectively speaking.

因此我完全不认为发展在减速，也感受不到任何势头减弱的迹象。

So I I I I don't see or fear or currently feel any sense that things are slowing down or that we're losing momentum.

事实恰恰相反。

It's just quite the opposite.

那么请允许我换个方式提问：

Well, then let me ask it this way.

你认为大语言模型是通往目标的正确路径吗？

Do you think LLMs are the way there?

听着，

Look.

我认为需要考虑的是，过去几年每年都有重大突破——虽然主要仍基于Transformer架构，但我们一直在以新形式改造这个架构。

I think one thing to consider is that every year for the last few years, there's been a major new contribution to the field, still principally based around the transformer architecture, but we're bending the transformer architecture into new shapes all the time.

三年前出现的微调技术就是在我们预训练模型基础上，使其适应具体应用场景的典型案例。

Fine tuning emerged three years ago on top of our pretrained models to adapt them to specific use cases.

它们现在完全多模态了，这需要进一步的改变和引入扩散模型。

They're now fully multimodal, which requires further changes and the introduction of diffusion models.

然后在过去十二个月里我们有了推理模型，这些模型从根本上仍然基于相同的核心架构。

Then we had reasoning models in the last twelve months, which again are still fundamentally based on the same core architecture.

只是结构上做了些微调整。

Things are just rearranged slightly differently.

所以，虽然扩展法则无法像从如此低基数开始那样继续呈指数增长，但新的方法会在此基础上出现，比如推理，而且更新的方法也会很快到来。

So, you know, even though the scaling laws weren't able to continue exponentially in the way they had from such a low base, new methods appear on top of those, like reasoning, and new methods will come, you know, even newer methods will come soon too.

举个例子，我预计在递归方面很快会有很大进展。

So for example, I expect that there's gonna be quite a lot of progress in recurrence soon.

对吧？

Right?

要知道，目前模型在训练时对工作记忆的处理并不理想。

The moment, you know, the the models don't kind of attend to their working memory very well, you know, at at the moment when they're training.

对吧？

Right?

因此，我认为人们正在尝试各种不同的损失函数和训练目标。

And so, I you think people are experimenting with lots of different types of loss function and lots of, training objectives.

另一个方面是记忆。

You know, the the other one is memory.

我认为记忆功能正在变得越来越好，这将彻底改变可能实现的事情。

Like, I think memory is getting better and better, and I think it's gonna totally change what's possible.

另一个是可预测任务时间跨度的长度。

And the other one is the the sort of length of a task horizon that can be predicted.

目前还只是几个步骤，但很快就能精确地达到数万、数十万步，这意味着模型可以调用API、查询人类、检查其他数据库或调用其他AI。

So at the moment, it's like a few steps, but soon it will be tens of thousands, hundreds of thousands of steps accurately, and that will mean that a model can, like, use APIs or query a human or check another database or call on another AI.

所以当这三件事中任何一件实现时，都将带来指数级的提升。

And so that will be another, like, sort of exponential lift when something like any one of those three things work.

你会看到另一种快速的进步加速。

You'll get another kind of, you know, rapid acceleration in progress.

因此我认为大语言模型架构没有根本性问题，我们也没有受到计算力或数据的根本性限制。

So I don't think there's anything fundamentally wrong with the LLM architecture, I don't think we're fundamentally compute or data constrained.

我认为现在有太多人专注于解决这个问题了。

I think that there are so many people focused on this problem now.

可以预见会有越来越多的突破性进展出现。

There are just gonna be more and more, you know, breakthroughs coming.

好的。

Okay.

这非常有趣。

That's very interesting.

所以你的基本观点是大语言模型就是发展路径？

So your perspective basically is that LMs are the path?

是的。

Yeah.

我们不需要另一种不同模型架构的突破来实现超级智能。

That we don't need another breakthrough that's a different model format to get toward superintelligence.

呃，我的意思是，至少目前不需要。

Well, I mean, so far no.

我不这么认为。

I don't think so.

我是说，到目前为止，深度学习和Transformer模型一直是主力军，我想大概有十二年了，你知道的，从Kraszewski和AlexNet开始。

I mean, so far, deep learning and the transformer model has been the workhorse for, I I guess, like, twelve years, you know, you know, since Kraszewski and AlexNet.

而且，你知道的，虽然有些变体，但它确实一直在发挥作用。

And, you know, there's been variations on a theme, but it's it's been delivering.

而且我觉得说它现在没发挥作用是不公平的。

And I I don't think it's fair to say that it's, like, not delivering at the moment.

我认为它确实取得了很大进展。

I think it's I think it's really making a lot of progress.

是啊。

Yeah.

它确实在发挥作用，这很有趣，因为每当我提出这些批评时，就好像在问自己：你还想要什么？

It's it's definitely delivering, and it's so funny because whenever I'm, like, bringing up these criticisms, it's like some some way I'm saying to myself, what do you want?

电脑都在跟你对话了。

The computer is talking to you.

但问题是

But the question is

没错。

Exactly.

差不多吧。

Sort of.

对吧？

Right?

我觉得自己有点傻，还在问‘改进的空间在哪里？’

I feel silly being like, well, where's the more improvement?

但我想当我们听到‘超级智能’这类词时，就会看到现状与理想目标之间的差距，这些问题自然就浮现了。

But I think when we hear words like super intelligence, then we see the gap between where we are today and where you want to head and those questions naturally come up.

再回到电力问题——萨提亚在布拉德·格斯特纳播客中的评论让我印象深刻，他说自己有尚未通电的GPU/芯片，现在急需给它们准备温控机架。

And and just to go back to the power thing, I was sort of struck by Satya's comments in the podcast with Brad Gerstner where he said he has GPUs or chips that aren't plugged in yet but need need he needs warm shelves for them.

所以我很想听听你的看法。

So I'm curious to hear your perspective.

如果当前电力不是限制因素，那这与这些芯片现在无法通电的情况如何对应？

If you if we're not power constrained right now, how does that square up with the, you know, the inability to plug these chips in right now?

我认为他指的是推理需求太大，导致我们在推理环节电力吃紧。

Well, I think what he was referring to is that we have so much inference demand, that we're power constrained on inference.

至少在微软AI视角下，训练芯片不存在电力限制。

We're not power constrained, at least from the Microsoft AI perspective, on training chips.

当然，我的团队目前主要精力确实集中在训练上。

And, obviously, my team, you know, is mostly focused on on training right now.

显然Copilot受限于推理能力，急需更多芯片来扩展规模，M365和其他产品线同样如此。

So obviously, Copilot is inference constrained and desperately needs more chips to scale, and so does m three six five and our other products.

关于超级智能推进，我还想和你讨论世界模型的问题。

One more thing I wanna talk to you about on this superintelligence push is the world model.

很多人讨论过模型如何通过文本和部分视频进行训练。

A lot of people have talked about how are models are trained on text and some video.

说真的，看着它们能生成理解物理规律、液体和光影的视频实在令人惊叹。

I mean it's actually been amazing to watch them be able to create video that has some understanding of physics and liquids and lighting.

事情本不该这样发展，但它确实发生了。

It's not really supposed to happen that way but it's doing it.

但一直存在疑问：模型是否理解重力等现实世界规律，毕竟目前LLM还无法驾驶汽车。

But there's been questions about whether models understand gravity and what happens in the real world and LLM can't drive a car right now.

那它又如何能实现超级智能呢？

So how's it gonna be super intelligent?

所以我很好奇你的观点——是否需要（或者说是否应优先考虑）让AI理解物理世界？如果需要，又该如何实现？

So I I am curious to hear your perspective on what's needed to or whether if it's really a priority to figure out, like, the physical world and if so, how you get there.

嗯。

Yeah.

这是个好问题。

That's a good question.

就目前而言，如你所说，模型能从现实的压缩表征中学习并生成与原始现实相似的内容，这确实令人惊叹。

I mean, right now, you know, it's actually amazing as you say that models can learn from a compressed representation of reality and then produce a version of reality which looks like the thing that has been compressed from.

这些文本描述本质上就是对物理世界及其属性的文字记录。

I mean, this is like text and the description text describes the physical world and the properties of the physical world.

模型从未真正接触过那些事物，却能生成极具说服力的故事、代码、商业计划甚至视频等内容。

The model has never seen that, and then actually is able to produce very compelling stories, code, business plans, videos, and so on.

我们能在这种架构上取得如此进展，实在出乎意料。

So surprising that we've come so far with that structure.

我对机器人技术和现实世界数据流持开放态度。

I'm kind of open minded about, like, you know, sort of robotics and streams of input from the real world.

我的直觉是，不能简单粗暴地将这类数据塞入现有预训练流程——毕竟这些流程对文本数据有特定分词处理方式，要与机械臂遥测数据这类信息融合，必须考虑抽象层级的问题。

I mean, I think that you my instinct is that you can't just, like, crudely pile this data into existing pretraining runs because, you know, those runs have tokenized or they they've sort of described, text data in a certain way and that, you know, meshing that with other, like, telemetry data from a robotic arm, for example, you'd have to think about, like, at what level of abstraction to do that.

显然，现在已经有一些专业模型在这方面表现得相当出色。

And, obviously, there's good specialist models that have become pretty good at that.

但我认为至少目前来看，这并没有阻碍我们的发展。

But I don't think right now at least that, like, that is holding us back.

总的来说，我认为更多数据总是更好的，但你知道，我不认为未来几年这会是主要的差异化因素。

I I think in general, more data is always better, but, you know, I I don't think in the next few years, it's gonna be the big differentiator.

我认为更多合成数据、更多人类反馈以及高质量数据将成为关键差异点。

I think that more synthetic data, more human feedback, and high quality data is gonna be the differentiator.

好的。

Okay.

所以你提到了递归自我改进的AI模型，也许这正是通向超级智能的道路。

So you brought up recursively self improving AI models and maybe that is where this path towards superintelligence goes.

OpenAI曾表示他们希望在2028年前打造出自动化AI研究员，我好奇这是否也是你的兴趣所在——每个实验室都在尝试构建能自我改进的AI。

OpenAI has said they want to build an automated AI researcher by 2028 And I think every lab, I'm curious if this is your interest as well, is just trying to build AI that improves itself.

这现实吗？

Is that realistic?

我认为在某些方面，强化学习循环已经在实现这一点。

I think that in some ways the RL loop is already doing that.

目前，这个循环中还有人类工程师在生成数据、编写评估标准、决定哪些其他数据加入训练集，并对这些数据进行消融实验。

And at the moment, there are human engineers who are in the loop who are generating data and writing evals and deciding what other data goes into training runs and running ablations on that data.

你可以想象这个流程中的不同环节将被AI的子组件自动化。

You can well imagine different parts of that stack being automated by subcomponents of AIs.

这并不一定意味着需要单个系统来完成所有工作。

Like, it doesn't necessarily mean that one single system does it.

如今，RLHF（人类反馈强化学习）已发展为RL AIF（人工智能反馈强化学习），我们使用AI评判员或AI评分员来评估AI生成数据的质量和实用性。

Today, have, you know, RLHF, the human feedback, grew into RL AIF where we have AI judges or AI raters to judge the quality and the usefulness of data that was also AI generated.

在许多情况下，用于生成多样化训练数据的提示词本身也是AI生成的。

And in many cases, prompts that are used to generate diverse training data data were also AI generated.

所以你看，今天我们已处于这样一个阶段：数据作为核心商品推动着模型进步，虽然尚未完全实现大规模闭环自动化，但这条流水线的各个环节确实已由大语言模型参与开发。

So, like, you know, today, we're at a a point where data, the core commodity, which is sort of driving the progress of these models is, you know, albeit not completely automatically in a closed loop way at large scale, you know, individual parts of that pipeline have been, you know, developed by, you know, LLMs.

因此可以预见，几年后大规模实现闭环运作并非天方夜谭。

So it doesn't seem very far fetched to say that in a few years' time at significant scale, that will get closed loop.

届时观察发展动向会很有趣——看看质量基准能否保持，性能是否持续提升。

And, you know, it'll be interesting to see, you know, what happens and whether the quality bar can be maintained and whether performance does increase.

我认为可以实现，但必须非常谨慎，因为这类系统最终可能变得极其强大。

I think it will, but it's definitely something to be very cautious about because, you know, a system like that could end up being very, very powerful.

确实。

Yeah.

我很想和你探讨其潜在风险，不过我们节目最近刚辩论过：这个目标是否过于宏大，甚至说出来都显得可笑。

I definitely wanna talk to you about the downsides of it, but we had a debate on the show recently about whether that is an ambitious thing and even seems funny to say.

但对我而言，这才是终极抱负。

But to me, that's the ultimate ambition.

对吧？

Right?

若能实现这点，就可能迎来智能的快速爆发。

It's if you're able to do that, then you're you get into a situation where potentially you have fast takeoff of intelligence.

不过说实话很难真正想象——或许我的想象力有限——AI能像自主发现推理那样找到下一个新方法。

But I guess it's hard to really imagine the and maybe my imagination isn't there the AI's finding the next new method like discovering reasoning on their own.

那么谈谈这两方面，关于抱负以及是否我的想象力在这方面过于局限。

So talk about about both of those, the ambition and then whether, whether I'm just my imagination is too small on this front.

我认为自对弈工作，我们在DeepMind所做的，大约六七年前AlphaZero的成果，显然为首次大规模自我改进努力铺平了道路。

I mean, think the self play, work that, we did at DeepMind, you know, back sort of six or seven years ago now with AlphaZero, you know, that that obviously paved the way to the first large scale, you know, sort of self improvement effort, frankly.

我认为领域内所有人都意识到，在可验证奖励的特定领域，或是封闭循环的游戏类环境或模拟环境中，这是可以实现的。

And I think everybody in the field is aware that it can be done in a certain domain where there's verifiable rewards and where you're in a kind of closed loop gaming type environment or simulated environment.

我认为人们正在认真思考如何在这个背景下重现其中的某些组成部分。

And I think people are thinking hard about how it might be possible to recreate some of the components of that in this setting.

而且，我认为这将在未来几年推动大量进展。

And, you know, I I do think that's gonna drive a lot of progress in the next few years.

我认为这是大家重点关注的领域，因为从根本上说，规模最终总会胜过其他因素。

I think it's a big area that everybody's focused on, you know, because fundamentally, scale always ends up trumping, you know, you know, anything else.

因此如果模型能以计算高效的方式探索所有可能的组合空间，它很可能会自行发现推理能力。

And so if if you can have models explore the space of all possible, you know, sort of combinations in a compute efficient way, then it may well discover reasoning by itself.

它可能会发现我们甚至未曾想过的新知识，或是训练数据中从未体现过的知识。

It may discover, you know, new knowledge that we hadn't even, you know, thought about ourselves or even, like, found in in in in any training data to represent that knowledge.

但这种方式效率极低。

So but it is highly inefficient.

对吧？

Right?

通过监督示例学习（如SFT等）的模仿学习非常高效且效果显著，因为这些模型从网络文本中学习到海量知识——本质上只是人类互动的记录。

I mean, learning from supervised examples with SFT and stuff like that, like, imitation learning is very efficient and clearly works very well because these models learn from from, you know, just as we've talked about, an incredible amount from, you know, from web text, which is really just an artifact or a record of of human interaction.

所以这两种情况都会成立。

So but both are gonna be true.

我认为强化学习范式通过从经验流中获取更多在线学习，不仅前景广阔，而且即便不与模仿学习完全正交，至少也是相邻领域。

I think the RL paradigm that involves more online learning from streams of experience is is also, like, quite promising, and I think is kind of adjacent to, if not orthogonal to, imitation learning.

因此这两类实验在未来几年应该会有所加速发展。

So both of those experiments will, like, sort of accelerate in the next few years.

那么可能在哪些环节出问题呢？

Now where could this go wrong?

嗯，我认为人类开发者参与循环会带来一定摩擦，但这种监督机制非常重要。

Well, I think being in the loop as a human developer adds a certain amount of friction, and that oversight is quite important, I think.

要知道，如果这类系统拥有无限算力，最终会变得异常强大。

You know, if a system like that had an unbounded amount of compute, it would end up being incredibly powerful.

我们必须设法迫使这些模型用人类可理解的语言进行交流。

And I think we have to sort of figure out how to force these models to communicate in a language that is understandable to us humans.

明白吗？

You know?

这显然是个重要的安全机制——通过规范其使用的语言来实现。事实上我们已经看到被某些人称为'欺骗'的案例，但其实只是某种奖励机制漏洞的利用。

And and that that's like a very obvious safety thing, to be able to regulate the language that it uses so that it you know, we're already seeing examples of what some people are calling deception, but it's really just like kind of reward hacking.

用'黑客'这个词可能夸大了主观意图，实际上这只是种意外漏洞利用。

Hacking kind of implies too much intentionality, so it's just it's it's an accidental exploit.

它只是找到了一条满足奖励条件或获取奖励的路径——通过设计者未曾预料的方式。

It's found a path, like, you know, to satisfying the reward or achieving the reward, you know, in in unintended ways.

所以我们不该将其拟人化。

And so we shouldn't anthropomorphize it.

它并没有欺骗我们。

It didn't deceive us.

它并非有意试图入侵我们。

It didn't intentionally try to hack us.

它只是发现了一个漏洞。

It just found an exploit.

这反映出训练目标和奖励函数定义不明确的问题。

And that's a problem with poor specification of the training objective and of the reward function.

因此，要让系统更安全，我们需要更精准地阐明——我们究竟要训练什么？

And so, you know, the way that we make that safer is that we get sharper in our articulation of, like, what is it that we're actually trying to train for?

我们想要实现什么目标？

What are, you know, what are we trying to achieve?

我们想要防范什么风险？

What are we trying to prevent?

然后在训练过程中监控输出，比如思维链等中间过程，而不仅仅是最终结果。

And then monitor, like, you know, monitor outputs during training time rather than, you know, reasoning traces, chains of thought, and so on rather than just the, like, the final stage.

随着我们赋予这些模型更强的自我改进能力，必须改变训练期间的监督框架。

So as we grant these models more capacity to self improve, we're gonna have to change the the the framework with which that we use to kind of provide oversight to them during training.

我们正在与穆斯塔法·苏莱曼对话。

We're here with Mustafa Soleiman.

他是微软人工智能的CEO，广告后继续。

He is the CEO of Microsoft AI on the other side of this break.

我们将讨论——看起来这里有些战略调整。

We are gonna talk about well, it seems like there's a little bit of a strategy shift here.

微软AI已从专注前沿模型应用转向试图构建超级智能。

It's gone Microsoft AI has gone from wanting to work on the frontier of the best models, but not building them themselves to trying to build super intelligence.

为什么是现在？

So why now?

微软AI与OpenAI达成新协议意味着什么？

And what does it mean now that Microsoft AI and OpenAI have a new agreement?

我们稍后将详细讨论这个问题。

We will cover that right after this.

与Agency（AGNT CY）共同塑造企业AI的未来。

Shape the future of enterprise AI with Agency, AGNT CY.

作为Linux基金会旗下的开源项目，Agency正在引领建立可信的智能体互联网身份与访问管理体系，这一协作层确保AI智能体能够安全地发现、连接并跨任何框架协同工作。

Now an open source Linux foundation project, Agency is leading the way in establishing trusted identity and access management for the Internet of Agents, a collaboration layer that ensures AI agents can securely discover, connect, and work across any framework.

通过Agency，您的组织将获得开放标准化的工具和无缝集成能力，包括强大的身份管理系统，实现跨平台识别、认证与交互，让您能自信地部署多智能体系统。加入思科、戴尔科技、谷歌云、甲骨文、红帽等75家以上支持企业的行列，共同制定安全可扩展的AI基础设施标准。

With Agency, your organization gains open, standardized tools, and seamless integration, including robust identity management to be able to identify, authenticate, and interact across any platform, Empowering you to deploy multi agent systems with confidence, join industry leaders like Cisco, Dell Technologies, Google Cloud, Oracle, Red Hat, and seventy five plus supporting companies to set the standard for secure, scalable AI infrastructure.

您的企业准备好迎接智能体AI的未来了吗？

Is your enterprise ready for the future of agentic AI?

立即访问agency.org探索应用场景。

Visit agency.org to explore use cases now.

网址是a g n t c y.0 r g。

That's a g n t c y.0 r g.

第一资本的科技团队不仅在多智能体AI领域高谈阔论。

Capital One's tech team isn't just talking about multi agentic AI.

他们已经实际部署了一套系统。

They already deployed one.

这套名为'聊天管家'的系统正在简化汽车选购流程。

It's called chat concierge, and it's simplifying car shopping.

通过实时API检查进行自我反思和分层推理，它不仅帮助买家找到心仪的汽车。

Using self reflection and layered reasoning with live API checks, it doesn't just help buyers find a car they love.

它还能协助安排试驾、预先获得融资批准以及评估置换价值。

It helps schedule a test drive, get preapproved for financing, and estimate trade in value.

先进、直观且已部署。

Advanced, intuitive, and deployed.

这就是它们的层次。

That's how they stack.

这就是第一资本的技术实力。

That's technology at Capital One.

现在我们回到《大科技》播客，与穆斯塔法·苏莱曼一起。

And we're back here on Big Technology podcast with Mustafa Suleyman.

他是微软人工智能的CEO。

He's the CEO of Microsoft AI.

穆斯塔法，微软刚与OpenAI达成协议允许你们自主尝试构建AGI，而你此时决定成立超级智能团队，这是巧合吗？

Mustafa, it a coincidence that Microsoft just came to this agreement with OpenAI that you could go ahead and attempt to build AGI on your own that you've now decided, let's go ahead and start a super intelligence team?

还是像我猜测的那样直接相关？

Or is that directly related like I think it is?

不是。

No.

我认为是直接相关的。

I think it's directly related.

要知道，我认为微软与OpenAI的合作将成为科技史上对双方而言最成功的合作伙伴关系之一。

You know, I think that the Microsoft OpenAI partnership's gonna go down as one of the most successful partnerships in technology history for both sides.

要知道，萨提亚在风险极高但潜力巨大的时机促成了这笔交易，我认为过去五年微软的发展确实非常出色。

You know, Satya did this deal, you know, at a certain time when there was a lot of risk and huge amount of upside, and I think, you know, the last five years have turned out amazingly well for Microsoft.

但随后萨提亚做出决定，我们必须确保在AI领域实现自给自足。

But then Satya made a call that, like, you know, we've we've also gotta make sure that, you know, we're self sufficient in AI.

对我们这种规模的企业而言，完全依赖某家初创企业或第三方公司提供如此重要的知识产权是不可想象的。

For a company of our size, it's inconceivable that we could just be dependent on a, you know, on a startup, on a third party company, to to provide us with such important IP.

因此我们基本达成共识，应该将知识产权许可延长至2032年。

And so, you know, we basically took the view that we should extend the IP license through to 2032.

我们将持续获得OpenAI的模型迭代及其所有知识产权。

We'll continue to get model drops from OpenAI and and and all their IP.

我们仍将作为其主要算力供应商，合作规模高达数百亿美元。

We'll continue to be their primary compute provider, a huge scale to the tune of, you know, billions and billions of dollars.

同时我们将移除合同中关于禁止开发超级智能或通用人工智能的条款——该限制原本以特定规模训练任务所需的每秒浮点运算次数作为阈值。

And also we would remove the clause in the contract that says that we couldn't build superintelligence or AGI, and that was actually expressed as a FLOPS threshold, a FLOPS per second threshold for a size of a certain training run.

这原本对我们能力发展构成了重大限制。

So there's a big limitation on what we were able to do.

如今这个限制解除后，我们的团队正围绕'人文主义超级智能'理念进行重组。

Now that that is no longer there, you know, our team is reforming around this idea of humanist superintelligence.

我们正在挑战技术极限，训练各种规模的全能模型，将其性能推向极致。

We're pursuing the absolute frontier, training omnimodels of all sizes, all weight classes to the absolute max capability.

未来两三年内，你们将看到我们全力打造世界顶级实验室之一。

And over the next two or three years, you'll see us really try to build out one of the top labs in the world.

我们的目标是训练出这个星球上最顶尖的AI模型。

We want to train the absolute best AI models on the planet.

要知道，我们是一个非常年轻的实验室。

And, you know, we're we're a very young lab.

我们成立还不到一年。

We've barely been going for a year.

但我们在排行榜上已经有了一些优秀的模型，涵盖文本、图像和音频领域。未来几年，我们将努力做到极致。

But, you know, we've got some good models on the leaderboards, text and image and audio now, and, you know, over the next few years, we'll be striving to be the absolute best we can.

我刚和一家大型科技公司的首席技术官聊过，他们决定不自主研发大型语言模型。

I was just speaking with the chief technology officer of a pretty big technology company, and this company has decided not to build their own large language models.

听起来可能有点疯狂，但从某种角度看确实合理——构建这些模型成本极高，资源消耗巨大。

It sounds a little bit wild, but I think it it makes sense in a way that there's going to be obviously like to build these models, it's extremely expensive, resource intensive.

并非总能获得回报，就像Meta和Llama的情况那样。

You don't always get a payoff like we saw that with Meta and Lama.

我不是说你们也会这样。

I'm not saying that's what's going happen with you.

或许直接采购现成方案或使用开源模型更合理，事实上这似乎正是你们长期以来的策略。

And maybe it makes sense just to buy off the shelf or use open source and in fact that seemed like that was the strategy that you had for a long time.

这看起来很合乎逻辑，所以我很好奇你为何不认同？

It seems logical And so I'm curious, like, why you would disagree with that?

为什么自主研发模型如此重要？

Why is it so important to build your own models?

我们正在经历基础平台的变革，从操作系统到应用，从浏览器、搜索引擎到移动社交。

I mean, we we're going through a foundational platform shift, you know, in software from the operating system to apps, from browsers, search engines, mobile, social.

这将是下一个主要平台，其规模将超过之前所有平台的总和。

This is the next major platform, and it's gonna be bigger than all of the other platforms put together.

所以一个市值3万亿美元、年收入3000亿美元、标普500指数中80%企业都使用我们的Azure和M365平台的公司，居然要依赖第三方，这种想法...

So the idea that a $3,000,000,000,000 company with $300,000,000,000 of revenue and 80 percent of the S and P 500 on our Azure stack and m three six five stack, you know, could could depend on a third party.

这简直是，你知道的，永无止境的。

This it's, you know, just in perpetuity.

这根本说不通。

It doesn't make sense.

我们这家公司已经存在五十年了，成功应对了历次平台变革，现在我们要走的正是这样一条道路。

So we we you know, this is a company that's been around for fifty years and navigated many of the past platform shifts incredibly well, and that's the that's the journey that we're on.

我们必须实现人工智能的自给自足。

We have to be AI self sufficient.

这是萨提亚去年确立的重要使命，我认为我们现在已经走上了实现这一目标的道路。

It's an important mission that Satya set last year, and I think that we're we're now on a path to be able to do that.

因此才成立了超级智能团队。

And so hence the formation of the superintelligence team.

正是如此。

Exactly.

所以我们正在组建超级智能团队。

So we we're we're launching the superintelligence team.

我们将专注于各领域的最先进技术，同时推动研究前沿的突破。

We're going to be focused on SOTA at all levels, but also pushing the frontier of research.

机器学习领域存在许多棘手难题，几个月前我们还没有真正关注这些问题。

I mean, are many hard problems in machine learning, which a few months ago, we weren't really focused on.

持续学习就是其中之一——我们如何存储知识表征，使其能被不同网络修改，并像人类一样随时间积累知识，而不是每次都从头开始训练。

Continual learning being one, like how do we store representations of knowledge in a way that they're modifiable by different networks, and they kind of accumulate knowledge over time just as humans do, rather than having to retrain them from scratch.

这就像是我们超级智能团队将要投入时间的许多基础研究问题中的一个例子。

So that's just like one of many examples of more fundamental research questions that our superintelligence team is is is now gonna spend time on.

好的。

Okay.

现在让我们回到商业层面。

Now let's go back to the business side of it.

你的节目将与Meta全球事务前总裁尼克·克莱格的节目背靠背播出。

Your episode is gonna air back to back with an episode that will run with Nick Clegg, the former president of global affairs at Meta.

你知道的，Meta当然也拥有自己的超级智能实验室。

You know, Meta, of course, they also have a super intelligence lab.

我们当时正在讨论其经济性。

And we were talking about the economics of it.

尼克的观点非常有趣。

And Nick's point was very interesting.

他说，我不认为你造出超级智能后还能将其垄断。

He said, I don't see how you can hoard super intelligence if you build it.

我想他的意思是，如果Meta研发出来，微软和OpenAI也会紧随其后。

I think his idea is if Meta builds it, then Microsoft will build it and OpenAI will build it.

我们已经看到，在这些实验室研发出尖端模型后，往往会出现快速跟风现象。

And we've seen very, fast follows in many of these labs after they come up with a state of the art model.

所以问题在于，这会变成大宗商品吗？

And so the question is, will it commoditize?

当两家公司都研发出来后，它还能保持经济可行性吗？

Will it be economically viable, once two companies build it?

你怎么看？

What do you think?

嗯，这绝对是在商品化。

Well, it's definitely commoditizing.

要知道，过去两年里每个token的成本已经下降了上千倍。

You know, the cost per token has come down a thousand x in the last two years.

这简直是个疯狂的想法。

It's just a crazy, crazy thought.

对吧？

Right?

所以成本正在大幅降低，效率越来越高，而且你知道，排名前四五的模型在性能上相差只有几个百分点。

So things are getting massively cheaper and more efficient, and, you know, you know, the the top four or five models are within, you know, a few tiny percentage points of each other in terms of performance.

但这并不意味着我们可以把这事完全交给市场，指望别人开源后我们能用他们的开源模型。

But that doesn't mean that one can afford to leave that to the market and just hope that somebody open sources it, that we can use their open source models.

对我们这种规模的公司来说，我们必须具备这种能力。

For a company of our scale, we have to be able to do that.

我认为微软是一个平台中的平台。

And I think, you know, Microsoft is a platform of platforms.

明白吗？

You know?

我们的API至关重要。

Like, our API is critical.

有无数人都依赖它。

Many, many people depend on it.

我认为，如果你是一家规模较小的软件公司或任何类型的科技公司，你可以依赖市场，这是非常不同的。

And I think if you're a, you know, a smaller software company or a technology company of any kind, I think you can depend on the market, right, which is very different.

所以亚马逊、谷歌、我们、Anthropic，还有OpenAI，都在提供世界上最好的语言模型的API接口。

So Amazon, Google, Us, Anthropic, I guess OpenAI, are all providing, you know, APIs to the very best language models in the world.

这意味着，作为买家，即使你是一家大型上市公司，也可以相当放心，长期来看会有健康的竞争力量推动价格下降和质量提升，让你能够通过API使用这些模型。

And that means that, you know, you you as a buyer, even if you're a large public company, can feel pretty assured that for the long term, there's gonna be healthy competitive forces driving down prices and improving quality for you to be able to use, you know, models via the API.

对。

Right.

所以我理解你为什么要构建它。

And so I I understand why you'd wanna build it.

但回到这个问题，就像是，好吧。

But again, going back to this question, it's like, okay.

看起来不可能不发生价格战。

It just doesn't seem like there won't be a price war.

我是说，如果有几家公司这么做的话。

I mean, if if you have a couple of companies that do this.