EP 189: NVIDIA and Marvell Earnings, Hot Chips Hot Takes

比如CXL这些，都是。

Like, CXL, all the things.

哦，我本来还想说，因为我们之前聊了很多这个。

Oh, the other thing I was gonna say, I because we talked about this a lot.

所以我很高兴他们终于正式确定了XPU attach这个方向。

So I'm glad that they have finally officially chosen their lane of XPU attach.

而且他们说得非常非常清楚。

And so they were very, very clear.

你知道吗？

You know?

就是说，如果单片计算tile没来，我们也不会不做，但那不是重点。而两年前，大家都觉得你必须做全套，不然就完了。

Like, we're not gonna not do monolithic compute tiles if they don't come, but, like, that's not gonna be whereas, like, two years ago, everybody's assumption was you gotta do the whole thing or you're dead.

对吧？

Right?

或者你必须在计算方面做得更多，不然博通就把你吃了。

Or you gotta do more in compute or Broadcom eats you.

然后他们一直说XPU连接、XPU连接、XPU连接。

And then they kinda kept saying XPU attach XPU attach XPU attach.

现在呢，好像就明确下来了，我们的方向就是XPU连接。

And now it's, like, firmly, our lane is XPU attach.

而且我确实觉得这是件好事。

And and I do think that's a positive.

这样你就能更好地融入整个AISC生态，不用非得拿下整个项目，只要在自己的领域里做好，每个项目都能有你的份。

It makes you a much more companion to the entire AISC ecosystem not having to win the whole program, but you can do well with your lane of everybody's program.

我觉得这个定位挺好的。

I think that's a good positioning.

虽然更难预测，但我就是很高兴他们终于选定了方向。

Tougher to model, but I'm just I'm I'm happy that that they've picked a lane.

不过，除此之外，也就那样吧。

But, yeah, other than that, it's like yeah.

总之，说说你的看法吧？

Anyway, give me give what was what was your take?

我觉得，你知道，今天股价几乎跌了10%。

I I think the, you know, the stock is down 10% today almost.

我觉得大家不满意的简单总结就是，他们不幸在英伟达之后一天发布财报，英伟达说下一年至少增长70%，而他们的数字是60%，从绝对值来看，这对Marvell或任何公司来说都是惊人的增长，60%的增长。

And I think the big the the very simple summary of what people didn't like was they have the unfortunate position of reporting the day after NVIDIA, where NVIDIA is talking about 70% at least 70% growth next year, and their their number was 60%, which, in absolute terms is fantastic growth for Marvell, for any company, see 60% growth.

但从数学上看，他们还是在失去市场份额给英伟达。

But it's still just mathematically they're losing share to NVIDIA.

这真的非常非常难。

It's just very, very hard.

你说得对，我确信他们十月份会有投资者日，他们会展示一个更积极的前景。

To your point, I'm sure they're gonna have their Investor Day in October, and they're gonna they're gonna lay out a much more positive picture.

但我觉得目前的情况就是，英伟达增长70%，他们增长60%。

But I think for the moment, it's like just NVIDIA is growing 70, they're growing 60.

然后你还得处理所有复杂的问题，比如这些人做什么，他们在哪里销售，谷歌的交易会是什么样子，这真的……

And then you have to deal with all the complexities of, like, what do these guys do, and where are they selling, and what is this Google deal gonna look like, and it it's just it's

对。

Yeah.

对。

Yeah.

不。

No.

我明白。

I I get that.

就像我说的，这很难，因为英伟达有明确的执行时间表，而你们这边，听起来不错，但我需要确切知道怎么把这些时间表转换成模型。

Like like I said, it's it's hard because you you have the clear execution timeline position of of NVIDIA, and then you have, like, it it actually sounds good on paper, but I need to know exactly how to, like, convert and model these timelines for you guys.

所以这更像是信心，看起来更乐观，但还不是保证，不像英伟达那样，就像你说的，他们说要增长70%。

So it's like conviction, like, looking more optimistic, but not, like, a guarantee at this point versus where, like, NVIDIA tells you, like you said, they're gonna grow 70.

他们至少增长70%，可能还会更多。

They're gonna at least grow 70 and and probably more.

我们不知道，对吧，对Marvell没有同样的信心。

We we don't know, right, the same confidence level for Marvell.

但就像我说的，我喜欢他们有自己的定位。

But but I do like I said, I like that they have their lane.

我喜欢CXL进入视野，虽然，再说一次，CXL不会给他们带来巨额收入，你知道，可能就几十亿美元，不算小，但也还行。

I like that CXL's come into the picture even though, again, right, CXL's not gonna add a gigantic amount of revenue revenue to them, you know, maybe couples of billions of dollars, not tiny, but but decent.

但是，对，谷歌那个才是大头。

But, yeah, the the Google one's kind of the big one.

对吧？

Right?

如果那1200亿美元能更快地转化更多，那他们的业务形态就完全不同了，但我们不知道，他们也没透露细节。

If that can convert more on of that $120,000,000,000 in a in a quicker timeline, that's a very different shape of their business, but we don't know, and and they didn't provide color on that.

对。

Yeah.

他们确实非常兴奋。

They were certainly very excited.

我的意思是，他们的语气非常积极。

I mean, they they they their tone was very positive.

我觉得马特在谈到他们在谷歌面临的那个机会时，非常乐观。

Matt was very bullish, I would say, in his comments about that opportunity they face at Google.

我的建议是，如果他们听的话，我们知道英伟达下个季度哪天发布财报。

My advice, if they're listening, is we know what day NVIDIA's holding earnings next quarter.

别这么做。

Don't do it.

对。

Yeah.

Marvell还没公布日期。

Marvell hasn't announced it yet.

把你的日期定在那之前两天。

Put your date two days before that.

你永远不想跟着来。

You never wanna follow.

你永远不想跟着英伟达。

You you never wanna follow NVIDIA.

跟着英伟达后面太难了。

It's tough it's tough act to follow.

这倒是真的。

That's that's true.

好的。

Okay.

关于Marvell，我们还有什么遗漏的吗？

Anything anything else on Marvell we're missing?

我知道那有点，我是说，就像我说的，我们肯定会在他们的投资者日之后讨论这个，然后试着分析一下市场机会和经济方面的东西。

I know that was kind of I mean, like I said, it's we we will, I am sure, talk about this after their investor day and kinda land try to land parse out some the market opportunity in in economics.

不过——

But

好的。

okay.

行。

Alright.

我们来聊聊Hot Chips吧。

Let's let's talk Hot Chips.

你知道吗，这挺有意思的。等等。

You know, it was interesting Hold on.

好的。

Okay.

我要说点什么吗？

Am I gonna say something?

对，Hot Chips。

Hot yeah.

对，Hot Chips。

Hot Yes.

Hot Chips 是一个学术会议。

Hot Chips is is an academic conference.

就像，斯坦福大学电气工程系教授们创办的那种。

Like, professors set up by the electrical engineering department at Stanford.

他们有教授，你知道，就是那种有学者、博士们展示学术论文的场合。

They have professors, they they, you know, that's the kind of thing where you have you have academics, PhDs presenting academic papers.

外面有海报。

There are posters outside.

但这是一个在硅谷中心举办的学术会议。

But it's an academic conference held in the heart of Silicon Valley.

所以来参加和演讲的人往往是来自大公司、做最前沿工作的人。

And so the the people who attend and speak at it tend to be people from the major companies doing the most advanced work.

这非常有趣，但代价是，在学术环境中，你可能会展示你最好的工作，但当所有大公司都来展示时，即使有技术负责人，他们可能也会有所保留。

And that is super interesting, but it comes at the the trade off of, you know, in an academic context, you're probably gonna present your best work When you have all the big corporates presenting, even though you have the technical leads, you they're probably holding stuff back a little bit.

但这是一个非常有趣的会议。

But it's a super interesting conference.

我最近听到的最大的抱怨是，来的投资者太多了，这让大家都有些不满，但这是一个非常有趣的会议，而且非常技术性。

The biggest complaint I've heard about it lately is that it's the the amount of investors who show up is sort of has everybody frustrated, but it's it's very it's a very interesting conference, but it's very, very technical.

对。

Yeah.

对。

Yeah.

所以，你知道，我最喜欢的一点一直是，Hot Chips的人总是这样定位它，就是，你知道，工程师对工程师的展示，展示他们是怎么做事的，怎么解决问题的，他们面临的问题是什么，他们不知道的事情，还有那些尚未解决、正在努力攻克的问题。

So, you know, my my favorite part about this is always, the way, you know, the Hot Chips guys have always sort of framed this, like, whenever I talk about it with them is, you know, it's engineers presenting to engineers, showing how they did things, how they solved problems, what the problems they face, things that they don't know, things that are unsettled that they're trying to solve.

说实话，我特别喜欢的一点是，很多问题都来自竞争对手。

And to be honest with you, one of the things I really like is a lot of the questions come from competitors.

你知道，比如你有一个谷歌的活动，然后有来自Marvell或者D Matrix的人会说，嘿。

You know, you would have like a Google event and you had somebody from Marvell or D Matrix be like, hey.

你们是怎么解决这个问题的？

How'd you solve this problem?

我很好奇你们是怎么想的。

I'm curious how you think about this.

这真的很了不起。

Like, it's it's amazing.

你会觉得，这太棒了。

Like, you're like, this is great.

而且，或者说

And and Or

最精彩的部分是华为的人每年都会出现，向每一位演讲者提出极其敏感的问题。

or the the the best part is the Huawei guy shows up every year and asks incredibly sensitive questions to every single presenter.

对。

Yeah.

而且有时候他们还会回答这些问题，比如……

And sometimes they answer them, like

其实我今年没怎么听到华为的人发言。

I actually didn't hear the Huawei guys much.

今年基本上被一个D Matrix的员工包场了，他几乎每次都会站起来提问。

This year was was owned by a d matrix employee who, like, literally just got up almost every time and and asked questions.

大家都在想，这是不是D Matrix在做营销啊？

And people are like, is this marketing stuff for for d matrix?

另一个有趣的事情是，Etched作为赞助商也来了，这挺酷的，因为我可以问他们一些更详细的问题。

Or the other the other the other fun thing was is was etched was there as a sponsor, which was cool because, like, I got to ask them some more detailed questions.

但他们带来了一个机架，一个黑色的机柜，而且锁得严严实实的。

But they had this this rack, this black box cabinet, and it was locked up.

大家都好奇，这玩意儿里面到底装了什么？

And everybody's like, what the crap's in this thing?

我觉得里面应该是空的。

Like, I'm assuming it was empty.

对吧？

Right?

因为很明显，他们迟早要卖机柜的。

Because, obviously, they they wanna sell racks at some point.

他们现在卖的是单个的盒子，不是机柜。

They sell individual boxes at this point, not not rack fabrics.

但我坐在那儿就想，为什么里面会有个霍迪尼？

But I'm, sitting there, I'm like, why why is there a why is Houdini in there?

霍迪尼是要从里面逃出来吗？

Is Houdini gonna bust out of there?

给我点东西看看啊。

Like like, give me some something.

所以有几家初创公司。

So there's a couple startups.

三星也在那里，我确实想在谈到内存和定制芯片时聊聊它。

Samsung was there, which I do wanna talk about when we talk about memory and custom based dies.

不过，让我先大概讲一下整体情况。

But but let let me give kinda, like, a high level a high level view.

我确实觉得这是很长一段时间以来最好的Hot Chips之一，不管别人怎么看。

I do think it was one of the best hot chips in a long time despite what others thought.

那里有特别多的投资人。

There was an inordinate amount of investors there.

我和他们两拨人分别吃了两次团体晚餐。

I had two group dinners with with with with both of them.

说实话，他们所有人，我都不太懂技术。

All of them, admittedly, I'm not technical.

我主要就是来随便看看的。

I'm kinda just here to check it out.

比如，给我解释一下这是什么。

Like, explain to me what this is.

但是因为有些演讲者选择了非技术性的路线，我觉得他们可能是在面对更多元化的观众，想讲个故事，而另一些人则深入到了非常专业的细节里。

But be but because there is some presenters who kinda took a nontechnical route, and I think because they were just trying to tell a story knowing maybe it was a more diverse audience, and then there were others who went super deep in the weeds.

整个技术栈上都有非常酷的东西。

Really, really cool stuff across the stack.

每年都不一样。

Each each year, it varies.

你知道，去年也有内存互连ASIC和CPU。

You know, last year, there was also memory interconnect ASICs and CPUs.

今年这三样都有。

This year was all three of those.

但像往年，主要是晶体管设计。

But like in years past, it was transistor design.

还有下一代代工厂。

It was next generation foundry.

所以，你知道，它确实会变，但今年和去年一样，很多互联网络、CPU、还有内存，基本上就是核心。

So, you know, it it kinda changes, but this year, like last year, lots of interconnect networking, CPUs, a six memories was essentially the the core.

但我的看法，我也写了文章，这是我给出的高层框架，就是大家对问题有很多共识。

But but my take, and I wrote about this, and this is sort of the high level framing that I gave is there was a a lot of agreement on the problem.

大规模推理和在大规模下服务推理，是一个计算上非常复杂的问题。

Inference at scale and serving inference at scale is a computationally complex problem.

ASIC怎么工作，内存怎么工作，软件怎么在这两种工作负载之间切换，大家都同意。

How the ASIC works, how the memory works, how the software moves between those two workloads, everybody agrees.

但真正有趣的是，你听那些D矩阵的人、SambaNova的人，甚至NVIDIA和AMD，还有所有演讲的人，甚至内存公司的人，每个人对怎么最好地解决这个问题都有不同的看法。

But what was really interesting was you listen to, you know, the the d matrix guys or the SambaNova guys or even NVIDIA and AMD and everybody who presented, even the memory guys, everybody kinda had, like, a different point of view on how they think is best to go solve this problem.

所以，在怎么做这些事情上，大家是有分歧的。

And so there was disagreement in in how we go and and do these things.

所以我这周写了这个，因为有一个学术术语叫“发酵期”，我给大家简单解释一下，它来自1990年的一篇论文，叫《技术不连续与主导设计：技术变革的循环模型》，作者是Anderson和Tushman。

And so I I wrote about that this week because there's an academic term called the the era of ferment, which I will sort of just briefly explain, right, for everybody, but it comes from a 1990 paper called technological discontinuities and dominant designs, a cyclical model of technological change by Anderson and Tushman.

基本上，发酵期就是在一个重大技术突破之后的一段时间，在这个时期，各种竞争方法被尝试，直到一个共同的架构出现。

And essentially, it is an era of ferment is when a period after a major technical break, when an entry tests competing approaches before one common architecture emerges.

这就是为什么你会看到很多不同的方案。

And this is why you're seeing, you know, a lot of diversity.

有意思的是，那些技术深入的人、其他工程师，或者对某个问题有偏好的人，就会说，嘿。

Now interestingly, like, people who were either technical in the weeds or other engineers or have a bias to a particular one of these problems was sort of like, hey.

你只在SRAM里做这个，或者

You're only doing this in SRAM or,

你知道，在

you know, on

HBM上做，因为你拿不到HBM。

an HBM because you can't get HBM.

所以他们就说，你这是在绕弯子，不是因为这是最好的方法，而是因为你没得选。

So they're like, so you're you're you're hacking around this not because it's the best way to do it, but because you don't have a choice.

对吧？

Right?

这可能是真的，也可能不是。

And and that may or may not be true.

但实际情况是，你在推理、内存、ASIC、设计，甚至某种程度上在互连方面，都有各种相互竞争的方法。

But the reality was, like, you have all of these kind of competing approaches on inference, on memory, on ASICs, on the design, even some to some degree, the interconnects.

我只是觉得这很有趣。

And I just thought that was interesting.

这是很长一段时间以来，我第一次看到这种动态：我们都有强烈的意见，而且对于如何大规模解决推理问题，我们的意见各不相同。

This is the first time in a long time I've seen this this dynamic of we all have strong opinions and they differ about about how we should go solve inference at scale.

总之，这就是我大概的看法。

And, anyway, so that was, like, high level kinda kinda my take.

就像我说的，我很久没见到这种情况了，这对我来说挺有意思的。

And like I said, I haven't seen this in a while, which just to me was kinda interesting.

我也是这么看的，即使是那些初创公司也一样。

And the way I frame this too is they're all even the startups.

对吧？

Right?

即使是那些做内存的人也一样。

Even the memory guys.

你可以清楚地看到，他们展示的是明年的东西。

You could you could be clear that, you know, they're showing next year.

他们展示的是两年后的东西。

They're showing two years away.

他们展示的甚至是一些他们还没解决的问题。

They're showing things that they haven't even solved yet.

比如，概念上我们打算这么做，但我们知道散热是个问题。

Like, conceptually, we're gonna go do this, but we realize heat's a problem.

我们打算这么做，但我们知道良率有问题，或者我们造不出来。

We're gonna go do this, but we realize yield's a problem, or we can't manufacture this.

对。

Yeah.

他们承认这些挑战，但他们现在就在为两年后的某些架构做技术押注，而我们完全不知道两年后模型或任何东西会是什么样子。

Like, they would acknowledge those changes, but they're making technical bets right now in some of these architectures for the future in two years, and we have no idea what the future of models or anything's gonna look like in two years.

所以，看看他们下的赌注，再跟别人比比，挺有意思的，尤其是在这个非常不稳定的环境里。

So it's just interesting to look at bets they're making and compare and contrast those with others, again, in an environment that is highly unsettled.

所以我们也不知道。

And so we don't know.

你知道，我们不知道标准化的推理架构会是什么样子。

You know, we don't know what what the standardized inference architecture will look like.

我觉得它会逐渐出现，然后如果一些初创公司能成功证明这一点，那我担心的是，如果真是这样，一些大厂商会直接采用那些设计并自己来做。

I think it will emerge and then again, you know, say that some of the startups are successful in proving that out, then, you know, my concern is one of the bigger vendors just takes those that design and does it, if if that's true.

不过，不管怎样，技术路线的多样性让我觉得特别有意思，非常吸引人。

But, anyway, the diversity of technical approaches, I I I found fascinating and thought was super interesting.

对。

Yeah.

看到好几家公司，大概四五家，展示了他们的CPU架构，这确实很有意思。

It was it was very interesting to see a number of companies four or five companies presented their CPU architecture.

而且，坦白说，我今年因为家里的事没能参加，但我读了所有的论文和相关的报道，让我印象很深的是，展示的CPU和GPU数量之多，而且它们的架构差异真的很大。

And as I so I in full disclosure, I couldn't attend this year for family stuff, but I was reading through all the papers and reading all the following content, and it was really striking to me the number of CPUs that were presented and GPUs that were presented and just how different their architecture was.

NVIDIA有大核心，ARM的核心小得多，而且它们包含的细节功能也各不相同。

NVIDIA has these big cores and ARM has much smaller cores and just the sort of number of nitty gritty features they're including.

现在有很多实验在进行中。

There there is a lot of experimentation taking place.

而且，我的看法跟你说的差不多，就是我们还不知道推理架构会是什么样子。

And and, you know, my read on that is is sort of what you're saying, which is we don't know what inference architecture is gonna look like.

你说它会标准化，但我不确定我们会走那条路。

And you say it's gonna standardize, I'm not I'm not sure that we go down that path.

我觉得它可能会保持非常分散和多样化。

I think there's it could remain very fragmented and very different.

这些技术方案之间的差异之大，真的让人印象深刻。

It just it is it is really striking how sort of radically different some of these technical approaches are.

种类太多了。

So much variety.

那是个很有趣的展会。

It was it was a fun show.

真希望我能去参加。

I wish I could have gone.

对。

Yeah.

对。

Yeah.

而且不只是新颖，有些方案简直是前所未有。

Well, and and not just like like, some of them were, like, novel.

你跟那些工程师聊过就会发现，他们这次真的是豁出去了，放手一搏。

Like, you talk to the engineers and you realize they're they're really swinging for the fences with this one.

你懂吧？

You know?

有可能成功，但难度也真的很大。

Like, it could work, but it's also really hard.

你明白吗？

You know?

所以，我同意你的看法。

And so so, you know, I I I agree with you.

我不知道。

I don't know.

我觉得至少我们是一致的。所以我的看法，再说一遍，我尽可能和所有人聊过。

It I think at least an agree so so my take, again, just and I and I talked to everybody I could.

我和六家领先的初创公司开过会，挑战在于所有这些不同的方法都有软件层面的额外开销。

I had meetings with six of the leading startups in this was, the the challenge is there's a software tax to all these different approaches.

对吧？

Right?

如果你要把KV缓存放到内存或存储的其他地方，或者把它移出芯片，软件就得做不同的工作。

If you're gonna put the KV cache somewhere else in memory or storage or you're gonna move it off die like, software has to do different work.

对吧？

Right?

所以因为这不是一个简单的事情，从算法的角度来看可能很高效，但像其他代理编排之类的，就涉及更多的工作。

And so because it's not just a clean it might be efficient from an algorithm standpoint, but, like, some of the other agent orchestration and whatnot, like, there's a little bit more work involved.

所以有一种干净利落的做法。

And so there's a clean way to do this.

有些供应商会这么做，但软件方面就变得非常复杂，如果一切按计划进行的话，这可能是对的。

Certain vendors do it, and the software's then there's a really hard way to do which, again, might be right if x, y, and z goes according to plan.

但计划往往赶不上变化。

X, y, and z, right, might not go according to plan.

不过，我觉得大家都在做技术押注，这一点真的非常非常有意思。

So but, again, like, the idea that everybody's making technical bets, I think, is is super, super interesting.

而且我猜，我们会看到定制超大规模云服务商推出一些定制化的方案，还有几家商家也会跟进。

And and I'm and I'm assuming we will have some bespoke variety by the custom hyperscalers and then a couple of merchants.

但你知道，我想你大概也同意我的看法。

But, you know, I think you probably agree with me.

那三四十家初创公司可能撑不到最后。

There there 30 or 40 startups don't survive the day.

但也许当整个行业从他们的尝试中学到东西，并把那些理念应用到自己的架构中时，因为那些理念在哲学上或技术上可能是合理的，只是他们自己没能实现。

But but maybe when the industry learns from what they tried and apply it in their architectures because maybe it was philosophically or or technically sound, they're just not gonna be the ones that pull it off.

会有别人来实现的。

Somebody else will.

对。

Yeah.

我的意思是，这确实是个两难的问题。

I mean, this this is dilemma.

这又回到了我们之前聊的，英伟达卖的是机架。

It goes back to what we're talking about before about NVIDIA sells racks.

对吧？

Right?

他们不卖服务器。

They don't sell servers.

也不卖芯片。

They don't sell chips.

他们卖的是机架。

They sell racks.

整套系统。

Systems.

系统。

Systems.

平台。

Platforms.

他们希望我们说平台，Jay。

They they would like us to say platforms, Jay.

我可不说。

I'm not saying it.

系统，我就说系统。

Systems, I'll go with systems.

我不会让步说平台的。

I'm not giving into platforms.

他们卖的是系统。

They sell systems.

而且如果你是个初创公司，卖东西的话，你能做的也就那么多。

And if you're if you're a startup that sells, like, you can only do so much.

对吧？

Right?

所以你们最大的差异化能力就在于芯片层面。

And so your biggest ability to differentiate is gonna be at the chip level.

是不是？

It's right?

那你怎么去设计整个机架、整个系统呢？

And how do you then design a whole a whole rack, whole system?

我觉得这挺有挑战性的，因为你得考虑网络这些因素。

And I think that's that's challenging because then you have to factor in, you know, networking.

对吧？

Right?

如果你是芯片公司，你可能有全世界最棒的XPU AI加速器，但如果你用错了网络架构，你的总拥有成本优势就全没了。

If you're a chip company, you sell you you could have the world's greatest XPU AI accelerator, and you lose all your TCO advantage because you're using the wrong networking stack.

我觉得确实是这样。

And I I think that's right.

所以对初创公司来说会很难。

And so it's it's gonna be tough for a start up.

我觉得这会让他们很难卖出去。

And I think it makes it very hard for them to sell.

而且更别提，现在大家都在拼资产负债表，初创公司显然做不到。

And and then let let alone, like, everyone's competing with balance sheet now too, which you obviously can't do as a start up.

所以，是的，我觉得会很艰难，但我确实认为其中一些会被收购，我不会给出具体的估值数字，但我认为它们被收购的可能性是存在的。

So, yeah, I think it's gonna be tough, but I do think I do think some of them will get acquired for for I won't I won't put a number on the valuation, but I think some there there is a a likelihood that they these get acquired

对。

Yeah.

同意。

Because Agree.

他们确实在推出一些非常有创新性的东西。

They are coming up with some really innovative things.

同意。

And Agree.

市场这么热，我觉得大公司会意识到他们需要快速上市，然后把这些东西收购过来。

With the market as hot as it is, like, I think the big the big players are gonna recognize that they need rapid time to market and pick these things up.

但是，是的。

But Yep.

但不会是70家，或者不管有多少家。

But not 70 of them or however many there are.

对。

Yep.

这些不是买家。

These aren't buyers.

不是。

No.

没错。

Correct.

好的。

Okay.

总之，我觉得那挺有意思的。

Anyway, I thought that was interesting.

内存这块，我想花点时间聊聊，因为所有内存供应商的主题，也是未来内存峰会的主题，就是定制HBM基片。

Memory, I wanna spend some time on because the theme of all memory providers, which which again was also the theme of future memory summit, so it's just carrying on with detail, is custom custom HBM base base dyes.

而且，你知道，这挺有意思的。

And, you know, it was it's it's interesting.

比如，你听了，我也听了这个。

Like, you listened I listened to this.

美光没有透露任何关于定制基片的平台细节，但三星和SK海力士都提到了。

Micron didn't give any platform details on custom based dye, but both Samsung and SK Hynix did.

这挺有意思的。

And it was interesting.

他们有点指出，这就是为什么从逻辑上讲，转向定制基片是合理的，但这也是为什么这很难。

They kinda point out, like, this is why it logically makes sense to go to custom based dies and but this is why it's hard.

对吧？

Right?

这是个散热问题。

It's a thermal problem.

他们得处理热点问题。

They gotta deal with hot spots.

一旦ASIC的功耗达到那个水平，事情就变得更难了。

It's once you have that level of a of of an ASIC in terms of of of wattage, it it gets harder.

所以他们正在努力解决这个问题。

So they're trying to solve, right, that problem.

但三家公司都反复提到的一点，实际上我的现任CEO在会议上也说了，我想是这周早些时候或者上周，我记不清了，他们觉得他们在定制基础晶圆上的工作更像是，你应该把它看作定制ASIC。

But but the thing that all three of them kept mentioning, in fact and and my current CEO said this at the conference they were at, I wanna say earlier or in in the week or or last week, I can't remember, that they think about, like, their work in custom based dye more like they like, you should think about custom ASICs.

那是为那个客户量身定制的独特设计。

That it is a unique design customized for that customer.

正因为如此，他的观点是——这让我在和三星和SK的人交流时产生了一些疑问——当你在基础晶圆上达到那种定制程度，结合那家公司的内存和你在他们基础晶圆平台上的定制，一个供应商要支持三家内存厂商，甚至可能两家，都会变得更难。

And because of that, his point was, and I this raised some questions to me when talking to the Samsung and the SK guys, is that it's it's harder for one vendor to support three memory players, maybe even two memory players, when you've gone down that degree of customization in in base die with that person's memory and your customization on their base die platform.

这让我不禁思考，好吧。

And that just got got me thinking like, okay.

那如果美光的基片平台效率更高，跟美光的内存搭配起来效果更好，而三星的不行，那英伟达就会想——我不是说这一定会发生。

So what if Micron's based I platform is more efficient, leads to better efficiencies with Micron memory than Samsung's does, and now NVIDIA is sitting there going I'm not saying this is happening.

大家别紧张。

Like, nobody freak about this.

我只是举个例子。

I'm just saying this is an example.

但这样一来，同一个加速器在不同供应商那里性能就会有差异。

But then you have performance differences between vendors for the same accelerator.

这是有可能的。

Like, that's possible.

我不知道怎么把这个跟我想说的关于英伟达的观点调和起来。

And I don't know how to reconcile that with then the point I was gonna make with the NVIDIA one.

而且，如果你真的要定制基片平台，那这会不会也增加内存的成本？

How does that also not increase the cost to do this for memory that you're actually gonna customize a base type platform?

他们肯定会为此多收费的。

They're gonna charge you more for this.

从优化和定制的角度来看，这么做完全合理，但成本会更高。

It makes all the sense in the world in optimizations and customizations to do this, but it's gonna be more expensive.

所以，在定制基础平台的世界里，内存价格不会下降。

So so so in a custom baseline world, like, memory doesn't go down.

实际上，内存价格可能还会上涨，而且每个客户可能只会选择一到两家内存供应商，因为我也同意，为一个GPU支持三种定制基础芯片确实很难。

In fact, it probably goes up, and it's gonna behoove one to two vendors memory vendors per customer because I agree it would be very hard to support three custom based dies for one GPU.

我觉得这种期望不太合理。

I I think that's an unreasonable thing to expect.

仔细想想，这个问题确实有点棘手。

This is a this is a tricky one when you think about it.

如果你是内存公司，你知道，人们对内存公司的评价就是，哦，这只是个标准品。

If you're a memory company, you know, the knock on memory companies is, oh, it's just a commodity.

可以互换的。

It's interchangeable.

对。

Yeah.

是可以互换的。

It's interchangeable.

对。

Yep.

没错。

Right.

所以内存公司可能觉得定制化很有吸引力，这样内存就不是完全同质化的了，而且更难替代。

And and so the memory companies probably see a lot of appeal to some form of customization, so that memory isn't totally fungible, and it's much harder.

这样就能锁定客户。

You get you get customer lock in.

但与此同时，内存的这种商品化对他们来说也有价值，因为可以互换，因为是标准化的。

But at the same time, like, that that sort of commoditization of memory is is is valuable to them because it's interchangeable, because it's standard.

对吧？

Right?

一切都是标准化的，但如果他们开始搞全定制化，就可能失去这个优势。

Everything's standardized, and you they risk losing that if they start going everything custom.

但我理解这种吸引力。

But I I understand the appeal.

我觉得这会导致后续的复杂性，他们很难想清楚，因为定制的吸引力太大了。

It's I think it it leads to complexity down the line that it's it's hard for them to think through because the appeal of being custom is so big.

同意。

Agree.

我也不是说每个人都用定制芯片，但我是说，我敢保证英伟达和AMD可能会用，也许还有其他人。

I also I'm not saying everybody uses custom based dye, but I'm saying, like, I guarantee you NVIDIA, and and AMD probably will, and maybe somebody else.

但就拿你最大的客户来说，对吧，甚至可能是你的XPU供应商，他们这么做是有道理的。

But but just take your biggest customers, right, maybe even your XPU providers who might do this for for good reason.

我的观点正是你所说的，就是它变得更像是设计的一部分。

My point is exactly what you said is that it becomes more design in.

它变得更像是全栈协同优化，但同时也更不通用。

It becomes more co optimized before the full stack, but it's also less fungible.

而且我认为可能会有性能上的差异，而且这种差异会变化。

And I would argue might have performance variations that vary.

我再强调一下，我就是觉得，如果说我的GPU用美光的内存或者三星的内存，表现就不一样，性能比SK海力士的好，那这个世界就太奇怪了。

Like, I again, I just don't like, it would be it's a weird world to say my GPU with Micron memory or with Samsung memory behaves differently, performs better than my SK one.

对吧？

Right?

如果真的是这样的话，那这个环境也太诡异了。

Like, that's a weird environment to be in if if that played out even in that fashion.

对吧？

Right?

所以，不管怎样，大家都明白了，定制化基板芯片肯定会成为趋势。

So, anyway, everybody walked away like, custom based die is gonna be a thing.

我们理解。

We get it.

是的。

Yes.

这对某些厂商有好处，但我的重点是，这无论如何都不会帮助降低内存成本。

It benefices offenders, but but my point is in no world does that help memory costs.

这确实会改变一些东西，但对内存厂商来说，肯定没那么容易替代了。

It it it changes it, but it's certainly less fungible for the memory guys.

所以我觉得大家都在讨论的就是，这种定制基片技术会非常厉害。

So I just thought that was, like, the talk of the everybody was like, this custom base type stuff's gonna be incredible.

我当时就想，是啊。

And I was like, yeah.

不过这里面也有一些很有意思的权衡，你知道的，如果它真的会普及的话，看起来确实会。

There's also some interesting trade offs, you know, with it, you know, if it's gonna be a thing, which it seems it will.

所以，反正就是这样。

So, anyway, that was that.

太疯狂了。

Crazy.

对。

Yeah.

还有其他的，比如三星展示了ZHBM，然后还有更多的3D堆叠技术。

And the other stuff like like Samsung was showing ZHBM and and then and then, you know, more three d stacking.

我当时就觉得，那挺好的。

I was like, like, that's great.

如果有一天你们能做到，并且能商业化、规模化，那就太好了。

If you get there someday and you can commercialize this and scale it, wonderful.

但那还远着呢。

But that's that's way far out.

其实那也不是，我就不多说了，但有个家伙，我记不清他是哪儿的了。

That's not in fact, I won't get into it too, but, like, a guy I can't remember where he was from.

我不知道他是来自一家高带宽闪存公司，还是只是做了一个关于高带宽闪存的演讲，但有些部分听起来挺积极的，实际上却有点消极。

I don't know if was from a high bandwidth flash company or if he was just giving a presentation on high bandwidth flash, but, like, parts of it sounded positive, but then it was actually kinda kinda kinda negative.

比如，他的幻灯片上写着，嘿。

Like, he had these slides like, hey.

我们得解决这些问题，你有没有想过这样处理？

We gotta solve these problems, and did you think about it this way?

然后大家就觉得，我觉得他并不是在展示高带宽闪存的积极面。

And people were like, I don't I don't think he's showing positivity on high bandwidth flash.

这确实在Slack上引发了一些不错的讨论。

This definitely got some good some good conversation in the Slack.

不过，总之，那场演出很棒。

But but, anyway, yeah, it was a great show.

就像我说的，那里人山人海，是我见过最热闹的一次。

There like I said, there was it was it was about as busy as I've ever seen it.

我的意思是，主礼堂都坐不下了。

I mean, people couldn't sit in the main auditorium.

人们只能去楼上。

They had to go upstairs.

外面还有很多人站着。

There was overflow outside.

是的。

Yeah.

场面很大，而且明年肯定会更大，我敢肯定。

It was it was big, and it's just gonna get, you know, bigger next year, I'm sure.

不过，总之，这是一场很棒的展会。

But, anyway, it was a it was a good show.

最后我想说的是，我很喜欢这些展会，这更加坚定了我关于这个“发酵时代”概念的观点。

And and and and lastly, I just say, like, I like these shows, and and and I'm and and this just doubles down on my point of this era of ferment concept.

我一直以来的标准看法是，如果你了解半导体公司的路线图，你就能预测未来，因为软件性能直接取决于计算能力的发展方向。

Like, my my standard view has always been if you just understand semiconductor company road maps, you can predict the future because software performance is directly aligned to where compute goes.

这也是我喜欢去展会的原因。

And and that's why I like going to the show.

有趣的是，所有模型都必须以某种特定的方式发展，这些不同的方法才能取得一定成功，而这很难预测。

It was just interesting that, like, models all have to go in a very certain way in order for all these different approaches to have some degree of success, and that's very hard to predict.

所以，这种混乱并没有帮助我预测，因为情况太多样了。

So it's like my this this confusion doesn't help my prediction because it's so diverse.

我不确定这能帮我预测什么，但这就是我喜欢去展会的原因——它能让你一窥未来计算的可能性。

I'm not sure what it helps me predict, but that's why I like going to the show is it's a glimpse of what's computationally possible in the future.

你听了Waymo的演讲吗？

Did you listen to the the Waymo presentation?

是的。

Yeah.

对。

Yeah.

我听了Waymo的演讲。

I was in the Waymo presentation.

因为我觉得特别有意思，它涉及了我们刚才在你提到的‘动荡期’里讨论的内容，也就是Waymo详细讲了他们为自动驾驶汽车采用的芯片方案。

Because I I thought that was that was super interesting because it sort of talks about touches on what we're talking about before in your age of ferment, which is Waymo had a big discussion on on their approach to silicon for their autonomous cars.

很明显，他们的推理负载跟很多人，尤其是前沿实验室正在处理的情况，差别非常大。

And it's very clear that their inference workload is really, really different than what a lot of other people are, certainly what the frontier labs are are dealing with.

你知道的，现在有个大趋势是转向FP4精度。

Like, you know, they use, like, you know, there's this big move to f p four.

也就是降低精度，因为这样效率更高，而且损失也不大。

So sort Less precision because it's more efficient and you don't lose much.

但在汽车上可不行，因为你需要极高的精度，毕竟车上那些传感器调校得都非常精细。

But you can't do that in a car because you actually need super high degrees of precision because you have all these different sensors that are very, very fine tuned.

这意味着他们为工作负载设计的芯片会非常非常不同。

That means that their their silicon for their workload is gonna be really, really different.

而且他们是在逆着一种相当普遍的潮流而动。

And it's they're moving against a trend that's been pretty common.

其他所有人都在往低精度方向走。

Everyone else is moving down.

他们却不想这么做。

They don't wanna do that.

对。

Yep.

而且我觉得，这其实说明以后会有更多这样的情况。

And and I think that, you know, that that sort of speaks to there are gonna be more more of those.

我们之前没聊到，但记得在他们公布财报的时候，AMD收购了Thales，那是一家初创公司，号称能为每个单独的模型定制芯片。

We didn't we didn't talk about it, but like, back when they announced earnings, AMD acquired Thales, is a startup which is a startup that sort of claimed to be able to do custom silicon for every individual model.

他们的方法很有意思，基本上在芯片设计里用了很多标准化的金属层，只需要把最上面的一两层换掉，就能适配不同的模型。

And they had a really interesting approach where they sort of had, you know, lots of different metal layers in their chip design that were standardized and just the top layer or two, they could switch around for each each model.

所以理论上，这样可以非常快速地迭代定制芯片。

So in theory, allowing very, very rapid iteration of custom silicon.

而且，你知道，他们的宣传就是：把你的模型给我们，我们为你优化芯片。

And for, you know, if you their their pitch was give us your model and we'll we'll optimize the chip for you.

这确实是个很有挑战性的做法，原因很多，我们就不细说了。

And that's a real that's a really challenging approach for a lot of reasons we don't have to get into.

但AMD收购了他们。

But AMD acquired them.

我们得假设，这不仅仅是为了得到一批工程师，我觉得他们意识到了这一点，他们得做更多半定制的工作，为了——

We have to assume for some for more than just maybe they just wanted a bunch of engineers, but I think that they're they're attuned to this, that they're gonna have to do more semi custom work for

对。

Yeah.

为了——

For

为了那些每个公司都有不同模型的情况。

for companies that everyone's gonna have a different model.

你提的这点真的很到位，而且Waymo那位老兄，其实聊到了，这其实是这次对话里最精彩的部分，因为我之前完全没想过这些。

You you bring up a really good point, actually, and the Waymo guy, like, kinda talked through, like actually, this was the most fascinating part of this conversation because I had not thought about any of this.

对吧？

Right?

但他当时在说，一辆自动驾驶出租车需要做什么、需要知道什么，跟你自己开的那辆、停在停车场里的自动驾驶汽车，是完全不一样的。

But but he was making points about, you know, what a what a what an autonomous taxi has to do and know is different than your autonomous car that you're operating and parking in a parking lot.

而且他展示的是，比如，这辆车需要知道，它能不能停在这儿？

And and and what he showed was, you know, this this car needs to know, for example, can it park right here?

这是一个带杆子的标志牌，上面大概有八种不同的、什么时候能停车的标志。

And this is a sign with a pole, and it had, like, eight different when do you could park their signs.

周二不行，周三早上8点以后也不行。

Not on Tuesday, not from 8AM on Wednesday.

然后他说，它还得知道能不能在这儿停下来。

And he was like, and it needs to know if it can stop here.

我当时就想，哇，真没想到。

And I was like, dang.

你看，这需要大量的图像处理，要知道今天是星期几，现在几点，下一趟行程有多远。

Like, you got that's a lot of image processing, what day of the week it is, like, what time is it, how far is my next ride.

这跟你的私家车要解决的问题完全不一样，对吧？

Like, has to solve different things, right, than a you know, your your passenger car.

然后我就想，但这又是在说他们的问题很独特。

And I just thought but but that's, again, that's them saying our problem is unique.

他们的问题也有点特殊，需要他们用独特的方式去解决。

Our problem is also somewhat bespoke, and we need to go solve it a unique way.

所以就像你说的，我觉得那特别棒。另外他们还展示了一样东西，他们展示了车的后备箱。

And so exactly like you said, I just thought that was super the other thing they showed was they showed the back.

他们展示了他们早期一辆车的后备箱。

They showed the back of one of their cars from, like, really early on.

那是一个架子。

Like, it was a rack.

那后备箱里有一架子计算设备，就像一个小型数据中心，外面用木头或金属盒子装着。

There was a there was a rack of of of compute of, like, a little data center in the back of this car and, like, a wooden slash metal box.

作为那个产品的工程师，感觉就是，对，没错。

It's like to be an engineer on that product, be like, yeah.

你是怎么做到的？

How are you doing this?

嗯，你知道，我们得想办法在后座放一台电脑。

Well, you know, we gotta figure out how to get a a computer in the back seat.

我们要装这个架子。

We're gonna put this rack.

它连着所有这些乱七八糟的光纤。

It's connected with all these crazy fibers.

总之，这真的非常非常有意思。

Like anyway, it it was super, super interesting.

但这也解释了为什么他们觉得，就像你说的，他们需要自己做模型，需要用自己的ASIC来算，因为他们要解决的问题在他们的领域里相对独特，而且他们还得处理其他各种事情。

But it also makes sense why they feel like, like you said, they need to do their own models, and they need to do their own compute with their own ASIC, which they outlined because, again, they're they're trying to solve a problem relatively unique to their vertical, and you've got all these other things they gotta deal with.

对吧？

Right?

安全、合规。

Safety, compliance.

你明白吧？

You know?

总之，我还能继续讲，另一个演示也可以讲，但我们还是先到这里吧。

Anyway, I could go and the other presentation was I could go, but we'll we'll we'll stop.

不过你说得对。

But you're right.

目前来看，还是需要一些新颖的方法。

Novel approaches for the time being.

好的。

Alright.

这个不错。

That's a good one.

感谢大家的收听。

Thanks for listening, everybody.

我们下周再聊，希望到时候不会像现在这样，我觉得我可能有点疯，但谁知道呢。

We'll talk to you next week when, hopefully, it won't be I guess, I was crazy, but you never know.

秋天会非常忙乱的。

Fall's gonna be crazy.

感谢收听。

Thanks for listening.

感谢大家收听。

Thank you for listening, everybody.

给我们留个评论吧。

Leave us a review.

告诉你的经纪人给我们留个评论。

Tell your agents to leave us a review.

告诉你的朋友们。

Tell your friends.