构建AI的现实世界基础设施，携手谷歌、思科与a16z

好消息是基础设施又变得性感了，这还挺酷的。

The good news is infrastructure's sexy again, so that's kinda cool.

这就像是互联网建设、太空竞赛和曼哈顿计划的结合体，涉及地缘政治影响、经济影响、国家安全影响，还有速度影响——这种影响相当深远。

This is, like, the combination of the build out of the Internet, the space race, and the Manhattan Project all put into one, where there's a geopolitical implication of it, there's an economic implication, there's a national security implication, and then there's, just a speed implication that's pretty profound.

我是说，我觉得这话说起来容易。

I mean, I think it's easy to say.

我从未见过这样的景象。

I've seen nothing like this.

我很确定没人见过类似的情况。

I'm fairly certain no one's seen anything like this.

九十年代末、两千年代初的互联网很宏大，我们当时觉得，天哪，不敢相信这种建设速度和规模。

The Internet in the late nineties, early two thousands was big, and we felt like, oh my gosh, can't believe the, build out, the rate.

相比之下，这次...我是说，用10倍来形容都是保守的。

This makes it I I mean, 10 x is an understatement.

这是互联网规模的100倍。

It's a 100 x what the Internet was.

AI热潮不仅仅在改变软件行业。

The AI boom isn't just changing software.

它正在改造支撑AI运行的实体基础设施。

It's transforming the physical infrastructure that runs it.

今天你将听到来自谷歌的Amin Vadat、思科的Jitu Patel以及a16z的Raghu Raghuram的对话，探讨如何构建支撑大规模AI的现实系统——从芯片、电力到数据中心和网络。

Today, you'll hear a conversation with Amin Vadat from Google, Jitu Patel from Cisco, and Raghu Raghuram from a sixteen z on what it takes to build the real world systems behind large scale AI from chips and power to data centers and networking.

他们讨论了当前建设规模、算力与互连的新限制，以及硬件和架构专业化如何重塑行业格局和全球地缘政治。

They discuss the scale of the current build out, the new constraints on compute power and interconnect, and how specialization in hardware and architecture is reshaping both the industry and global geopolitics.

地缘政治。

Geopolitics.

这是对基础设施本身如何在AI时代被重塑，以及未来发展趋势的务实观察。

It's a grounded look at how infrastructure itself is being reinvented for the AI era and what comes next.

我们开始吧。

Let's get into it.

还有什么比现在更适合讨论基础设施的呢？

What better time and place to talk infrastructure?

好的。

Alright.

我们又回到了休息室。

We were back in the green room.

就在第一个问题得到回答时，我被中断了。

And just as the first question was getting answered, I got cut off.

所以据我所知，这可能是完全重复的。

So this could be an entire repeat for all I know.

不过无论如何，我们继续吧。

So but, anyway, let's go.

对吧？

Right?

第一个问题很相似。

The first question is similar.

首先欢迎两位的到来，感谢你们的参与。

So both of you, firstly, welcome and thank you for being here.

非常感谢

Thank you so

也祝你们度过愉快的一天半时间。

And hope you'll have a great day and a half as well.

两位在行业里都已有相当资历。

Both of you have been in the industry for a while.

你们都经历过多次基础设施周期。

And both of you have lived through many infrastructure cycles.

对吧？

Right?

那么从你的角度来看，你见过这样的周期吗？

So have you seen anything like this cycle from your vantage point?

不是从投资者的角度，而是从你内部负责构建事物、规划事物等的角度来看？

Not from an investor vantage point, but from your internal vantage point where you are responsible for building things and planning for things and so on?

你们谁想先开始？

Anyone of you, where do you wanna start?

阿明，你想先开始吗？

You wanna start Amin?

阿明，你先说吧。

Go ahead, Amin.

我觉得这很容易说。

I mean, I think it's easy to say.

我从未见过这样的情况。

I've seen nothing like this.

我很确定没人见过类似的情况。

I'm fairly certain no one's seen anything like this.

九十年代末、两千年代初的互联网很庞大，我们当时觉得，天啊，难以置信这种建设速度和规模。

The Internet in the late nineties, early two thousands was big, and we felt like, oh my gosh, can't believe the build out, the rate.

相比之下这个...我是说，说10倍都是保守估计。

This makes it I I mean, 10 x is an understatement.

这是互联网规模的100倍。

It's a 100 x what the Internet was.

我认为其上升空间和互联网一样巨大。

I think the upside is as big as the Internet was, same thing.

10倍乃至100倍。

10 x and a 100 x.

是的。

Yeah.

无与伦比。

Nothing like it.

我同意。

I'd agree.

我认为在规模、速度和范围上都是前所未有的。

I don't think there's any priors to the size the speed and scale.

我想说好消息是基础设施又变得有吸引力了，这挺酷的。

I'd say the good news is infrastructure is sexy again, so that's kinda cool.

有很长一段时间它都不受关注。

It was a long time where it wasn't sexy.

我认为真正有趣的是，这就像是互联网建设、太空竞赛和曼哈顿计划的结合体，具有地缘政治影响、经济影响、国家安全影响，以及相当深远的速度影响。

The thing I would say that's really interesting is this is like the combination of the build out of the Internet, the space race, and the Manhattan Project all put into one, where there's a geopolitical implication of it, there's an economic implication, there's a national security implication, and then there's just a speed implication that's pretty profound.

所以，是的，我们从未见过如此规模和范围的景象。

So, yeah, none of us have ever seen it at this size and scale.

另一方面，我认为我们严重低估了。

On the other hand, I think we are grossly underestimating.

比如，我现在最常问的问题是，是否存在泡沫？

Like, there's the most common question I ask right now is, is there a bubble?

我认为我们严重低估了建设规模。

I think we're grossly underestimating the build out.

我认为实际需求将远超我们当前的预测。

I think there's gonna be much more needed than what we are putting the projections towards.

那么接下来的问题是，你认为我们现在处于资本支出周期的哪个阶段？

So that's the follow on question is, where are we, do you think, of the CapEx spend cycle?

但更重要的是，你们内部使用哪些信号来指导决策？

But more importantly, what are the signals that you guys use internally, right, in your thinking?

我是说，数据中心这类设施必须提前四五年规划。

I mean, you have to plan data centers, whatever, four, five years in advance.

还得采购核反应堆之类的设备。

You have to buy nuclear reactors and whatnot.

那么你如何看待需求信号和技术信号？

So how do you think about the demand signals as well as your technology signals?

Jeetu，对你来说也是同样的问题，但要从企业和新云等角度考虑。

And, Jeetu, the same thing for you, but from the point of view of enterprise and neo cloud, etcetera.

Amit？

Amit?

我认为我们尚处于周期早期，尤其是相对于当前观察到的需求而言。

We're early in the cycle is what I would say, certainly relative to the demand that we're seeing.

我们的内部用户——我们研发TPU已有十年，目前第七代产品已投入内外部使用。

Our internal users are we've been building TPUs for ten years, so we have now seven generations in production for internal and external use.

我们七八年前生产的TPU仍保持100%的使用率。

Our seven and eight year old TPUs have 100% utilisation.

这充分说明了市场需求。

That just shows what the demand is.

当然每个人都想用最新一代产品，但能拿到什么就用什么。

Everyone would of course prefer to be on the latest generation, but whatever they can get.

这说明需求非常旺盛，同时也反映出我们不得不拒绝的客户和应用场景。

So this tells me that the demand is tremendous, but also who we're turning away and the use cases that we're turning away.

不是那种'哦还行吧'的程度。

It's not like, oh yeah, that's kinda cool.

而是'天哪，我们真的不打算投资这个'，而且因为优先级问题根本没有选择余地。

It's, oh my gosh, We're actually not going to invest in this and there's no option because that's where we are on the list.

在座各位很多人也面临同样情况。

Same with many of you in the room.

我们正在与在座的各位合作，许多人直接向我表达了感谢。

We're working with many of you in the room and many of yours are telling me directly and thank you.

我们需要更早行动。

We need more earlier.

对吧？

Right?

正如你所说，当前的挑战在于我们受限于电力供应。

Now the challenge here though is, as you said, that we're limited by power.

我们受限于土地改造。

We're limited by transforming land.

我们受限于审批流程。

We're limited by permitting.

我们还受限于供应链中许多物资的备用交付能力。

And we're limited by backup delivery of lots of things in the supply chain.

所以我担心的是，供应实际上无法像我们期望的那样迅速赶上需求。

So one worry I have is that the supply isn't actually going to catch up to the demand as quickly as we'd all like.

我在之前的会议中听到关于我们将花费数万亿美元的讨论，我认为这个数字是准确的。

I heard in the previous session some of the discussions of the trillions of dollars that we're gonna be spending, which I think is accurate.

我不确定我们能否兑现所有这些支票。

I'm not sure that we're gonna be able to cash all those checks.

换句话说，实际上你们有些钱无法按理想速度全部花出去。

In other words, literally, you all have some money you can't spend it all as fast as you want.

我认为这种情况会持续三到五年。

I think that's going to extend for three, four, five years.

哇。

Wow.

那你们如何处理其中涉及的折旧周期问题呢？

And how do you deal with the depreciation cycles that are involved there?

需求曲线与折旧周期曲线是否匹配？

Does the demand curve and the depreciation cycle curves match up?

幸运的是，我们采用的是准时制采购。

Well, fortunately, we buy just in time.

但好处是硬件设备也正好赶上这个时机。

But the nice thing is just in time for the hardware.

太空能源设备的折旧周期大约在25到40年之间。

The depreciation cycle for the space power is more like somewhere between twenty five and forty years.

所以我们享有优势

So we have benefits I

我认为，如果从网络角度考虑，同时观察企业用户、超大规模运营商以及新兴云服务商，情况会大不相同。

think if you think of on the networking side and you look at both enterprise and the hyperscalers as well as neo clouds, I think the story is quite different.

企业市场还相当初级，是基于真实基础设施构建的。

So the enterprise is pretty nascent, and it's built out of true infrastructure.

我完全不认为数据中心会像假设的那样——所有数据中心都将在某个时间点需要重新架设机架，而且每个机架所需的电力水平将与传统数据中心有根本性不同。

I just don't think that the data centers like, if you assume that 100% of the data centers at some point in time will need to get re racked, and you will need a very different level of power requirement per rack that's gonna be there compared to what used to be there in the traditional data centers.

我只是觉得企业用户的发展程度还远远不够。

I just don't think that the enterprises are far enough along.

也许少数超大规模企业可能达标，但绝大多数企业还远未达到这个阶段。

Maybe the few enterprises that are at super high scale might be there, but I don't think the enterprises are far enough along.

超大规模运营商和新兴云服务商则是完全不同的情况。

Hyperscalers and Neo Clouds is a completely different story.

关于阿明提到的电力、计算和网络资源稀缺性这三大制约因素，我认为当前由于单一地点电力供应不足，数据中心正转向电力资源丰富的地区建设，而非将电力输送到数据中心所在地。

And to Amin's point on this notion of scarcity of power, compute, and network being the three big kind of constraints in this thing, I would say right now that because there's not enough power singularly in one location, data centers are being built where the power is available rather than power being brought to where the data centers are.

这就是为什么我们看到全球各地都在开展大量建设项目。

And that's why you're seeing a lot of projects that are being built out all throughout the world.

不过另一个关键点是，我们面临的大部分制约因素在未来很长时期内都将持续存在。

The other point though is the lion's share of the constraints that we're gonna have, I think are gonna be sustainable for a long period of time.

随着数据中心建设得越来越分散，首先，扩展网络将面临巨大需求，以便机架能获得越来越多的网络资源进行纵向扩展。

And as you have data centers that are being built farther and farther apart, one, there's gonna be a huge demand for scale up networking so that you can have a rack that gets more and more networking for scale up.

其次，横向扩展也将有很大需求，需要将多个机架和集群连接在一起。

The second is you're gonna have a lot of demand for scale out where you have multiple racks and clusters that need to get connected together.

但我们刚刚推出了一款新的硅片、芯片和系统，用于跨网络扩展，可以让两个相距800-900公里的数据中心作为一个逻辑数据中心运行。

But we just launched a new piece of silicon as well as a new chip and a system for scale across networking, where you might have two data centers that act as a logical data center that could be up to 800, 900 kilometers apart.

你会看到这种情况，因为单个地点的电力资源将无法满足集中需求。

And you will see that just because there's not gonna be enough concentration of power in a single location.

因此必须构建不同的架构体系。

So you'll just have to have different architectures that get built out.

实际上，这正好引出了我想讨论的下一个话题——系统和网络的未来发展等等。

Actually, that brings us to the next topic that I wanna discuss, the future of systems and networking and so on and so forth.

谷歌最早（至少是大规模）为网络革命采购了横向扩展的商用服务器。

So Google bought the first or at least large scale, scale out commodity servers in production for the web revolution.

而现在，英伟达正以不同形式让大型机回归。

And now, NVIDIA is bringing back the mainframe in a different form.

那么你认为接下来会发生什么？

So what do you think happens next?

这是否意味着我们需要这种新型的集群范围一致性计算模式，将实现共享内存等功能？

I mean, is this the new style of coherent cluster wide computing that we need and there's going be shared memory and all sorts of things?

还是你认为这种模式会再次改变？

Or do you think the pattern changes again?

我认为我们并未完全回归大型机时代，因为人们仍然在这些资源池上运行横向扩展架构。

I don't think we're quite too back to mainframes in that it is still the case that people are running on scale out architectures across these pools.

换句话说，你拥有GPU或TPU时，并不是在说'那是我的GPU超级计算机'。

In other words, you have GPUs or TPUs, you're not necessarily saying, hey, that's my GPU supercomputer.

而是在说'我拥有16,384个GPU'。

You're saying I've got 16,384 GPUs.

是的。

Yep.

或许我要去获取一些子集。

And maybe I'm going to go grab some subset.

现在我在许多情况下实现了全面的统一连接，这太棒了。

Now I've got uniform all to all connectivity in many cases, which is fantastic.

TPU也是如此。

Same with TPUs.

并不是说我有一个9000芯片的集群，就必须让我的任务适配它。

It's not like I say I have a 9,000 chip pod and I have to make my job fit on that.

也许我实际上只需要256个。

Maybe I actually only need 256.

也许我需要10万个。

Maybe I need a 100,000.

所以我确实认为这种软件横向扩展仍将存在。

So I do think that actually this software scale out is still going to be there.

不过我要指出两点。

I'll note two things though.

第一，你说得完全正确，大约二十五年前在谷歌和其他地方同时发生的，确实是计算基础设施的一次变革。

One, you're absolutely right that say about twenty five years ago at Google and other places simultaneously, was really a transformation of computing infrastructure.

就像那种理念——实际上你会在商用PC上进行横向扩展，那些你可以直接买到的运行Linux系统的机器，这就是你用于磁盘、计算和网络的方式。

Like the notion that actually you would scale out on commodity PCs essentially, those same ones that you could buy off the shelf running a Linux stack and that's what you would do for disk, that's what you would do for compute, that's what you do for networking.

我的意思是你们都认为这是理所当然的，但这在当时是相当激进的。

I mean you all take it for granted that this is sort of it was radical.

有很多人认为这是个糟糕的主意，根本行不通。

There are many people who thought that this was a terrible idea, that wasn't going to work.

我认为当前这个时刻令人兴奋的地方在于，我们将要重新发明——我不是说谷歌——我们将要重新发明计算。

I think the exciting thing about this moment right now is actually that we're going to be reinventing I'm not saying Google, we are going to be reinventing computing.

五年后，无论从硬件到软件的计算堆栈会变成什么样，都将变得面目全非。

And five years from now whatever the computing stack is from the hardware to the software, it's going to be unrecognizable.

顺便说下，这种协同设计的存在是有道理的——以我最熟悉的谷歌为例：Bigtable、Spanner、GFS、Borg、Colossus这些系统都与硬件集群的横向扩展架构进行了深度协同设计。

And by the way there was this co design because if you think about it, I'll use Google examples because I know those best: Bigtable, Spanner, GFS, Borg, Colossus they were hand in hand co designed with the hardware cluster scale out architecture.

如果没有横向扩展软件，你们也不会去开发横向扩展硬件。

And if you wouldn't have done the scale out hardware if you didn't have the scale out software.

没错。

Yep.

同样的情况即将在此刻重演。

Same thing is gonna happen in this moment.

所以我认为大型机的形态将会变得非常非常不同。

So I think actually the mainframe is gonna look very, very different.

好的。

Okay.

是的。

Yeah.

我确实认为市场会对集成系统产生极大需求——目前思科很幸运，我们的业务覆盖了从物理层到语义层的全栈。

I do think there'll be, like, this extreme demand for an integrated system because right now we are very fortunate at Cisco where we do everything from the physics to the semantics.

你要考虑从芯片到应用的全链路。

You think about the silicon to the application.

除了功耗之外，关键约束在于这些系统的集成度如何，以及它们是否真能在全栈范围内实现最低损耗的协同工作？

And other than power, one of the constraints is how well integrated are these systems, and do they actually work with the least amount of lossiness across the entire stack?

因此这种深度集成的水平将变得至关重要。

And so that level of tight integration is gonna be super important.

这意味着行业必须进化到这样的状态：即使由多家公司分别负责不同组件，我们也必须像单一企业那样协同工作。

And what that means the industry will have to evolve into is we will have to work like one company even though we might actually be multiple companies that actually do these pieces.

所以当我们与谷歌等超大规模企业合作时，在正式签约前就会开展长达数月的深度设计合作。

And so when we work with hyperscalers like Google or others, there's a deep design partnership that actually goes on for months and months together ahead of time before we actually even do the deal.

交易一旦完成，当然会有巨大压力确保他们进展迅速。

And then once the deal is done, of course, there's a tremendous amount of pressure to make sure that they're moving pretty fast.

但我认为行业的力量在于确保你在开放生态系统中运作，而不是成为封闭花园，这在技术栈的每一层都将变得至关重要。

But I think the industry's muscle of making sure that you operate in an open ecosystem and not be a walled garden is gonna get important at every layer of the stack.

非常棒。

Really great.

那么让我们稍微拆解一下技术栈。

And so let's talk about the disaggregate the stack a little bit.

最有趣的话题之一是处理器。

One of the most interesting topic is processors.

对吧？

Right?

显然，有一家出色的供应商生产了目前市场份额巨大的优秀处理器。

Clearly, is an amazing vendor producing an amazing processor that has massive market share today.

对吧？

Right?

而且我们不断看到初创公司尝试各种处理器架构。

And we see startups all the time doing all sorts of processor architectures.

你的堡垒里有一个出色的处理器。

You've got an amazing processor inside your fortress.

你认为处理器领域接下来会发生什么？

What do you think happens next in processor land?

是的。

Yeah.

我们是NVIDIA的超级粉丝。

We're huge fans of NVIDIA.

我们销售大量NVIDIA的产品和芯片。

We we sell a lot of NVIDIA products and chips.

客户非常喜欢它们。

Customers love them.

我们也是自己TPU的超级粉丝。

We're also huge fans of our TPUs.

我认为未来其实非常令人兴奋，实际上我们并没有达到那种'好吧，有TPU、有GPU、有训练芯片或其他什么'的阶段。

I think the future is actually really exciting, and actually we're it's not that I don't think that we've hit the point of, okay, there's TPUs, there's GPUs, there's whatever, trainiums or something else.

我们正亲眼见证专业化的黄金时代，这是我的观察。

We're really seeing the golden age of specialization, And that's my observation.

换句话说，以TPU为例（我再次用它举例因为我最熟悉），对于某些计算任务，它的每瓦效能比CPU高出10到100倍，而瓦数才是关键指标。

In other words, if you look at it, a TPU, I'll use that example again because I know it best, for certain computation is somewhere between ten and one hundred times more efficient per watt, and it's this watt that really matters than a CPU.

这种优势让人难以忽视，对吧？

That's hard to walk away from, right?

10到100倍的差距。

10 to 100x.

但我们知道还有其他计算任务，如果能构建更专用的系统——不只是小众计算，而是谷歌经常运行的那些计算任务。

And yet we know that there are other computations that if you built even more specialized systems for, but not just a niche computation, computations that we run a lot of at Google.

比如服务型任务，或者能受益于更专用架构的代理型工作负载。

Like for example maybe for serving, maybe for agentic workloads that would benefit from an even more specialized architecture.

所以我认为真正的瓶颈在于：设计和投产一个专用架构到底有多难、需要多长时间？

So I think that actually at one bottleneck is how hard is it and how long does it take to turn around a specialized architecture?

目前这个过程遥遥无期。

Right now it's forever.

对全球顶尖团队来说，从概念到实际投产，最快也要两年半——这已经是光速了。

For the best teams in the world, really from concept to live in production, speed of light is two and a half years.

我是说在一切顺利的情况下。

I mean that's if you nail everything.

对吧？

Right?

确实有少数团队能做到这一点。

And there are a few teams that do.

但你要如何预测两年半后的未来，来构建专用硬件呢？

But how do you predict the future two and a half years out for building specialized hardware?

所以第一，我认为必须缩短这个周期。

So A, I think we have to shrink that cycle.

但第二，当发展速度稍缓时（这是必然的），我们必须构建更多专用架构，因为其省电、节约成本和空间的优势实在不容忽视。

But then B, at some point when things slow down a little bit, and they will, I we're gonna have to build more specialized architectures because the power savings, the cost savings, the space savings are just too dramatic to ignore.

这实际上也会对地缘政治结构产生非常有趣的影响。

And this will actually have a really interesting implication on geopolitical structures as well.

因为想想中国的现状，他们其实并不生产两纳米芯片。

Because if you think about what's happening in China, China actually doesn't make two nanometer chips.

他们制造的是七纳米芯片。

They make, you know, seven nanometer chips.

但关键在于他们拥有无限的电力供应。

And and so if you think about what but they have unlimited amount of power.

他们还拥有无限的工程人力资源。

And they have unlimited amount of engineering resource.

因此他们可以在工程层面进行优化，保持七纳米芯片的同时确保无限供电。

And so what they can do is do the optimization on the engineering side, keep the seven nanometer chips, and make sure that they give people unlimited amount of power.

我们可能采用不同的架构设计——虽然电力资源有限，工程师数量也不及中国充裕。

We might have a different architectural design where you have to get extremely You power don't have as many engineers as you might enjoy in China.

但我们可以实现两纳米芯片的突破。

And you can actually go to two nanometer chips.

这些芯片在某些方面可能更节能，但在其他方面可能存在热损耗问题。

But and those might be power efficient in some ways, but they might have thermal lossiness in other ways.

需要综合考虑各种因素，未来架构将根据地域特点变得更加专业化。

Like, there's a whole bunch of things that have to get factored in, on the architecture that will get more specialized even by geo and by region.

然后根据监管框架的演变情况，你知道，那个地理范围会如何扩展。

And then depending on how the regulatory frameworks evolve, you know, how that that geo then expands.

就像如果中国扩展到世界不同地区，与美国扩展到世界不同地区相比，会出现非常不同的架构。

Like if China expands to different regions in the world, you will have a very different architecture that plays out than if America expands to different regions in the world.

所以这是一个非常有趣的博弈论练习，可以预测未来三年科技领域的总体发展趋势。

So this is a very interesting kind of game theory exercise to go through on what happens in the next three years in in tech in general.

而目前没有人知道答案。

And no one knows right now.

是啊。

Yeah.

这就是我们所处世界的魅力所在。

That's the beauty of the world that we live in.

没错。

Yeah.

是的。

Yeah.

所以我们很快就要用工程师数量除以代币数来衡量系统，就像用瓦特除以代币数一样。

So we'll soon be measuring systems by engineers per token in addition to watts per token.

好吧。

Alright.

那么让我们转向另一个话题，这个非常

So let's turn to another topic, which very

工程师每千瓦。

Engineer per kilowatt.

工程师每千瓦。

Engineer per kilowatt.

在美国。

In The US.

网络，对吧？

Networking, right?

显然，你提到了扩展规模，横向扩展。

Obviously, you alluded to it, scale up, scale out.

在你的案例中，你提到了跨规模扩展。

In your case, you mentioned scale across.

所以在我看来，网络也将以一种相当显著的方式被重新定义。

So it seems to me that networking is also gonna get reinvented in a fairly significant way.

那么你看到的主要迹象是什么？或者网络将朝着什么方向发展？

So what are the leading signs that you're seeing that and the signals that you're seeing in or the direction networking is gonna take?

是的。

Yeah.

网络确实需要转型。

Networking is going to need a transformation for certain.

换句话说，建筑物内部所需的带宽规模简直惊人。

In other words, the amount of bandwidth that's needed at scale within a building is just astounding.

我的意思是，需求还在增长。

I mean, it's going up.

网络正在成为主要瓶颈，这很可怕。

The network is becoming a primary bottleneck, which is scary.

因此更高的带宽直接转化为更好的性能。

So more bandwidth translates directly to more performance.

考虑到网络最终实际上是一个小型电力消费者，每瓦特提供的效用。

And then given that the network winds up actually being a small power consumer that delivered utility you get per watt.

这就像是一种超线性收益。

Like it's a super linear benefit.

就像在这里投入一点，就能在那里获得更多。

Like spend a little bit here, get way more there.

因此我认为那一方面确实存在。

So I think that that side is absolutely there.

我要在此强调，对于这些工作负载，我们实际上清楚网络通信模式是什么，这是我们的优先事项。

I'll put in a plug here in that for these workloads we actually know what the network communication patterns are, our priority.

所以我认为这是一个巨大的机遇。

So I think this is a massive opportunity.

换句话说，当你已经知道大致电路走向时，是否还需要数据包交换的全部功能？

In other words do you then need the full power of a packet switch when actually you know what the rough circuits are going to be.

我并不是说需要构建电路交换机，但这里存在优化空间。

I'm not saying you need to build a circuit switch but there is an optimization opportunity.

另一个关键点是这些工作负载具有极强的突发性。

The other aspect of this here is these workloads are just incredibly bursty.

是的。

Yeah.

正如我们曾撰文所述，电力公司都清楚这点——当我们在进行涉及数十至数百兆瓦规模的计算与网络通信时。

And to the point where, and we've written about this, power utilities know this when we're doing network communication relative to computation at the scale of tens and hundreds of megawatts.

对吧？

Right?

比如电力需求激增时突然中断，转而进行网络通信，随后又迅速切回计算模式。

Like massive demand for power, stop all of a sudden and do some network communication, and then burst back to computing.

那么如何构建一个需要在极短时间内满负荷运行，随后又进入闲置状态的网络？

So how do you build a network that needs to go at 100% for a really short amount of time and then go idle.

同样道理适用于跨用例场景——我们明确认为，没有人会全年12个月在所有广域数据中心站点持续进行大规模自由训练。

And then same actually for the scale across use case which we're absolutely saying, you don't run large scale free training across all your wide area data center sites twelve months out the year.

这其实是我经常思考的问题：假设你在这三个数据中心站点部署了最新最强的芯片。

So and then you're gonna this is a problem I think about a lot is let's say you build the latest greatest chips in these three data center sites.

在迁移到另外三个站点更新的芯片之前，这些设备能服役多久？

How long are you gonna be there before you migrate to the latest latest chips in three other sites?

那么你如何处理遗留下来的网络呢？

And then what do you do with the network that you left behind?

人们会在上面运行任务，但你根本不需要像大规模训练（特别是预训练）那样庞大的网络容量。

People are going to run jobs on them but you're not going to need nearly the network capacity that you did for large scale training, pre training anyway.

所以这种只需要在5%时间使用大规模网络的转变，我不知道该如何构建这样的网络。

So the shift of needing massive networks for like 5% of the time, I don't know how to build a network like that.

如果你们有人知道，请告诉我。

So if any of you do, let me know.

我是说，你都不知道怎么构建，现在根本没人知道该怎么建

I mean, you don't know how to build this, there's nobody that knows how We're to build

正在努力解决这个问题。

trying to figure it out.

这确实是个引人入胜的难题。

It actually is a fascinating problem.

是啊。

Yeah.

我确实认为，如果把电力视为限制条件而算力作为资产，那么网络将成为力量倍增器。

I do think, like, if if you think of if power is the constraint and if compute is the asset, I think network is gonna be the force multiplier.

嗯。

Mhmm.

因为你看，如果数据包具有低延迟、低性能和高能耗，那么每节省一千瓦的传输功耗，就能把这一千瓦电力分配给GPU使用。

Because, you know, if a if a packet if if you have low latency and low performance and high energy inefficiency, then the pack the every kilowatt of power you save, moving the packet is a kilowatt of power you can give to the GPU Yeah.

这非常关键。

Which is, you know, super important.

另外，当你考虑纵向扩展、横向扩展和跨域扩展时，特别是在推理与训练之间，需要优化的目标也不同。

The the other thing is, you know, when you think about scale up versus scale out versus scale across, you also need, especially on inference versus training, there are different things that get optimized.

比如在训练过程中，你可能需要更优先优化延迟。

Like, you might optimize for latency much more on training runs.

你可能会在推理过程中更注重内存优化。

You might optimize much more for memory on inferencing.

从架构角度来看，我认为网络的发展方向将不再是先建立训练基础设施再应用于推理，而是会逐渐构建专为推理设计的原生基础设施。

There there's there's architectural and so I I also feel like the way that networking will evolve is rather than it being a training infrastructure that then gets applied to inferencing, you might have inferencing native infrastructure that gets built over time.

因此需要重点考量的是所有架构组件是如何演进的。

And so there's there's good considerations to look at on, like, how all of the architectural components are are moving.

但在我看来，从战略角度而言，网络领域正在发生的最重大变化之一是：如果你只是博通的封装商，那么你将面临一个极具掠夺性的垄断局面。

But in in my mind, like, if if I were to say strategically one of the biggest things that's happening in networking from our vantage point is if you're just a wrapper around Broadcom, then you've got a monopoly that's gonna be a very predatory one.

思科之所以至关重要，正是因为你不必生活在一个所有人都只是封装博通芯片的世界里。

And so one of the big reasons where Cisco is super relevant is you don't just have a Broadcom world with people just wrapping Broadcom.

虽然他们的系统基于博通，但实际上你会有芯片选择权。

I mean, kind of, their systems are on Broadcom, but you will actually have a choice of silicon.

这种芯片选择的多样性将变得极其重要，特别是对于高吞吐量的消费模式而言。

And that choice and diversity of silicon is gonna be super important, especially for high volume, you know, kind of consumption patterns.

既然提到系统，最后问个相关问题，然后我们转向用例讨论。

So last question on the systems since you brought that up, we'll move to use cases.

推理环节——你们两位都提到了，你刚才是在处理器背景下谈的。

Inference, both of you have mentioned I mean, you talked about it in the context of the processors.

你刚刚开始讨论架构问题。

You just started talking about the architecture.

你们目前是否在部署专门的推理架构？

Are you deploying today's specific architectures for inference, I mean?

还是仍在采用共享工作负载模式？

Or is it still shared workloads?

我们正在部署专为推理设计的架构。

We are deploying specialized architectures for inference.

我认为软件和硬件同样重要，但硬件也会以不同配置形式部署——这是我的表述方式。

And I think as much software as hardware, but the hardware is also deployed in different configurations is the way I would say it.

推理中另一个变得非常有趣的方面是强化学习，尤其是在服务关键路径上，因为延迟变得绝对关键。

And then the other aspect of inference that is becoming really interesting is reinforcement learning, especially on the critical path of serving because latency just becomes absolutely critical.

我认为如何构建系统以及如何相互连接，当然网络在其中扮演关键角色，正变得越来越有趣。

And I think that so how you would build your system and how you would connect it up to one another, and of course networking plays a key role there, becomes increasingly interesting.

是否存在某些单一瓶颈点，如果移除它们就能加速实现我们所需的推理成本千倍降低，还是说这只是我们正在记录的自然曲线？

And are there singular choke points that if removed would accelerate the thousand fold reduction in the cost of inference that we need, or is this just a natural curve that we are writing down?

我们的规模非常庞大。

So we're massive.

我是说，这里的事情。

I mean, things here.

第一点，可能你们很多人都熟悉这个。

One, again, maybe many of you are familiar with this.

推理中的预填充和解码阶段看起来非常非常不同。

Prefill and decode on inference look very, very different.

所以实际上，理想情况下应该使用不同的硬件。

So actually, ideally, you would have different hardware, actually.

平衡点是不同的。

The balance points are different.

所以这是一个机会。

So that's one opportunity.

它也有缺点。

It comes with downsides.

我们可以讨论这个问题。

We can talk about that.

不过我想说的是，可能有些人没有意识到，我们实际上正在推动推理成本的大幅降低。

What I would say though is that maybe something people don't realize is that we're actually driving massive reductions in the cost of inference.

我的意思是10倍甚至100倍的降低。

I mean 10 Xs and 100 Xs.

问题或机遇在于社区，用户群体不断要求更高的质量，而非更高的效率。

The problem or opportunity is the community, the user base keeps demanding higher quality, not better efficiency.

因此，就在我们实现所有预期的效率提升后，新一代模型又出现了，其单位成本的智能水平大幅提升，但价格仍比上一代更高、成本更贵。

So just as soon as we deliver all the efficiency improvements we're looking for, the next generation model comes out and it is the whatever intelligence per dollar is way better, but you still pay more and it costs more relative to the previous generation.

然后我们又重复这个循环。

And then we repeat the cycle.

这几乎就像你的推理链条越长，市场就越没耐心。

And it's almost like the longer the reasoning that you have, the more impatient the market gets.

对吧？

Right?

例如，如果你有二十分钟的推理周期，比如深度研究时，可以实现约二十分钟的自主执行。

So for example, if you have a twenty minute reasoning cycle, like for example, with deep research, you could have autonomous execution for about twenty minutes.

这很有趣。

That was interesting.

现在大多数编程工具已经可以实现七小时到三十小时的持续自主执行。

Now you have, you know, most of the coding tools that can go up to seven hours to thirty hours of, you know, duration of autonomous execution.

当这种情况发生时，市场反而更强烈要求压缩时间。

When that happens, there's actually a greater demand for saying compress the time down.

于是这就形成了一个自我实现的预言：正因为你能实现更长时间的自主执行，反而需要更强的性能。

And so you lack it's it's kind of a self fulfilling prophecy where you need to have more performance because of the fact that you've been able to go out and do things for a longer autonomous amount of time.

这几乎是个永无止境的循环，你将永远需要更强的推理性能。

And so it's almost a never ending loop where you'll you'll need to have more performance for inference Yeah.

永无止境。

In perpetuity.

是啊。

Yeah.

不过，单位成本的智能水平是商业模式指标，而不仅仅是处理器能力的体现。

Though, intelligence intelligence per dollar is a business model metrics metric, so it is not just a processor capability.

不。

No.

这是端到端的。

It's end to end.

绝对是的。

Absolutely.

对。

Yeah.

那好吧。

So okay.

那我们换个话题，聊聊实际应用吧。

So let's change topics and talk about actual usage.

对吧？

Right?

你们两位都管理着庞大的组织。

So both of you have massive organizations.

在运用现有AI技术方面，你们目前取得的关键性成果有哪些？

Where are the key wins that you're getting today with with applying all the AI that's available to you?

然后我们再谈谈客户的应用，但我其实更想知道你们内部的实施情况。

And then we'll talk about what your customers are doing, but I'm actually curious about what you're doing internally.

在团队内部吗？

Within the teams?

是的。

Yeah.

我是说，编码是最明显的应用领域，而且这方面的应用正在获得越来越多的关注和能力提升。

So I mean, coding is the obvious one, and that's actually picking up increasing traction and increasing capability.

实际上就在前几天，我们刚发表了一篇论文，展示了如何运用AI技术实现指令集迁移。

We just actually in the last couple of days published a paper that showed how we applied AI techniques to do instruction set migration.

换句话说，我们实际上经历了从x86到ARM的大规模迁移，使得我们整个代码库——在谷歌这是一个非常庞大的代码库——变得与指令集无关，并且能够适应未来可能出现的第五代风险或其他任何情况。

So in other words we actually had a fairly massive migration from x86 to ARM, making our entire code base, and at Google it's a very very large code base sort of instruction set agnostic and including to you know future risk five or whatever else might come along.

数以万计，数十万的独立个体

Tens and thousands, hundreds of thousands of individual Your

整个代码库，你要让它变得与平台无关？

entire code base, you're going make it agnostic?

整个代码库，因为我们希望需要所有代码库都能

Entire code base because we we want want to need all of our code base to be

老兄，这项目也太疯狂了。

Man, that's a crazy ass project.

是啊。

Yeah.

所以，我们确实这么做了。

So so we we it it was.

不过这个计划的动机其实源于几年前。

And the the motivation, though, for this actually was a few years ago.

我们当时有一个了不起的旧系统叫Bigtable，后来又开发了更出色的新系统Spanner。

We had this amazing legacy system called Bigtable and then a new amazing system called Spanner.

于是我们决定告诉全公司：所有人都需要从Bigtable迁移到Spanner。

And we decided to tell the company, hey everyone needs to move from Bigtable to Spanner.

Bigtable在其时代非常出色，但Spanner更优秀。

And by the Bigtable was amazing for its time but Spanner was better.

谷歌估算完成这次迁移需要七千人年的工作量。

The estimate from doing that migration for Google was seven staffed millennia.

多少？

How much?

怎么

How

多少？

much?

七千年的员工配置。

Seven staffed millennia.

当时我们有个新部门，必须实际考察情况，结果发现并不是员工偷懒编造的借口。

That that we we had a new unit that we had to actually to see what and and it was it wasn't, like, made up people being lazy.

就像是，这就是我

It's like, this is what I

在做的事。

was doing.

不过他们能想到这点还挺可爱的。

It's endearing that they came up with that though.

你猜我们最后怎么决定的？

And you know what we decided?

Bigtable万岁。

Long live Bigtable.

我决定什么？

I decided, what?

老实说就是觉得不值得。

It just wasn't worth it, honestly.

机会成本太高了。

Like the opportunity cost was too high.

所以我们做了这类迁移，比如从TensorFlow转到JAX。

So and we have these sorts of migrations, TensorFlow to JAX.