加文·贝克——人工智能、半导体与机器人前沿 - [像最佳投资者一样投资，第385期]

但你的PUE确实很重要，因为你的电力成本非常关键。

But your PUE then really matters because your cost of electricity really matters.

所以，你为优化SFU所做的某些事情，实际上可能会以非常负面的方式提高你的PUE。

So things you do to optimize SFU might actually increase your PUE in a really negative way.

因此，我想，如果我今天早上刚想到这一点，也许这太明显了，每个实验室都已经在这么做了。

And so I think if you're I just thought of this this morning, maybe it's super obvious that every lab already is doing this.

但我只是觉得，如果你把它全部代入一个公式，再计算出所需的成本，就能真正做出权衡。

But I just think if you leak it all into an equation and then you do dollars that it takes to get it, you can really make trade offs.

哦，天哪。

Oh, wow.

如果我在网络上多花两倍的钱，我会获得高得多的SFU，这很好，但我的PUE也会大幅上升，这就不好了。

If I spend 2x more on networking, I get much higher SFU, which is great, but then I have a much higher PUE, which is bad.

而我们最终想要的不仅仅是每秒多少exaflops，而是每美元资本支出、每瓦电力所获得的每秒exaflops。

And what we ultimately want is not just actual exaflops per second, but we want exaflops per second per dollar of CapEx per watt of electricity.

而我刚才描述的这个公式——至少我会把它写下来——涵盖了所有这些因素。

And the equation I just described, which I'm going to write out at least for myself, captures all of that.

每个人都会做出不同的设计决策。

And everybody's going to be making different design decisions.

数据中心架构以及你所做的这些设计决策将变得极其重要。

And data center architecture and these design decisions you make are going to be immensely important.

我认为你会看到，特别是在这些十万级集群中，一些公司将在每美元资本支出每瓦特功耗的exaflops方面获得超过100%的优势。

And I think you're gonna see, particularly with these 100,000 clusters, some of these companies are going to have greater than a 100% advantage in exaflops per dollar of capex per watt consumed.

这时，这些实验室之间的真正差距就会显现出来——这决定了GPT-7或8的成本是1万亿美元还是5000亿美元，下一代的成本是2000亿美元还是4000亿美元。

And this is when you're gonna really separate these labs because that is the difference between GPT seven or eight costing a trillion dollars or 500,000,000,000, the next class costing 200,000,000,000 or 400,000,000,000.

对于推理来说，情况也是一样的。

And then for inference, it's the same.

完全一样。

It's the exact same.

这不仅影响服务成本，还直接影响用户体验，比如每秒处理的token数，而我们知道，这对谷歌的搜索用户体验来说是最重要的因素之一。

Not only is it cost to serve, but it directly affects user experience in terms of tokens per second, which we know from Google is one of the most important things for search UX.

因此，所有这些也都会应用于推理场景。

So all of this is also gonna apply in inference.

推理要简单得多，因为它实际上完全取决于内存带宽和片上内存。

Inference is just a lot easier because it really, really just comes down to memory bandwidth and on chip memory.

这让我想起了瓦茨拉夫·斯米尔关于能源历史的论述，他称之为原动机，比如化石燃料、风能等等。

This reminds me so much of the Vaclav Smile history of energy stuff where you would get a new he called them prime movers, some source of energy, fossil fuels, wind, whatever.

然后你会经历一个漫长时期，所有的进步都来自于效率的提升。

And then you get this long period of the gains all coming from the efficiency.

所以在煤炭时代的早期，当你转动涡轮机时，它只能捕获煤炭本身10%到15%的可用能量。

So if you're spinning a turbine or something in the early days of coal, the turbine only captured 10 or 15% of the available energy from coal itself.

而现在，我们已经达到了大约98%的水平。

And now we're at, like, 98 or something like that.

这听起来和你在这里描述的情况完全一致。

And basically, it sounds like that's the exact same thing that you're describing here.

GPU就像煤炭，它们会不断改进，这在技术上很酷，因为煤炭不会。

The GPU is the coal, and those will keep getting better, which is cool in technology unlike coal.

但听起来这基本上就是整个故事，实在太有趣了。

But it sounds like that's basically the story, which is so interesting.

这就像通过不同的化石燃料机制和发动机将阳光转化为可用能源。

That is like turning sunlight into usable energy via different fossil fuel mechanisms and motors.

而这是将阳光直接转化为计算能力。

And this is turning sunlight into compute, literally.

现在，这种阳光可以以真实的阳光形式出现。

And now that sunlight can come in the form of actual sunlight.

它也可以以人工阳光的形式出现。

It can come in the form of artificial sunlight.

那就是核能。

That's nuclear.

它还可以以储存的阳光形式出现。

It can come in the form of stored sunlight.

那就是化石燃料。

That's fossil fuels.

但关键是将阳光转化为计算能力的效率。

But this is the efficiency at which you turn sunlight into compute.

太酷了。

And So cool.

就看你为这个比率支付了多少钱。

In the dollars you pay for that ratio.

嗯。

Yeah.

顺便说一下，这真的很有趣。

And by the way, it is just interesting.

比如，这就是我以前对风能不太感兴趣的原因，因为涡轮机已经非常高效了。

Like, it's one reason, like, I was never that excited about wind because turbines were really efficient.

而十年前或十五年前，太阳能的效率还非常低。

Whereas you could look ten or fifteen years ago, solar was terribly inefficient.

我真的觉得这太棒了。

I actually think it's awesome.

这让我有点难过。

It's a little sad to me.

所有这些孩子都非常担心全球变暖，而你经常听到关于二十多岁的人说，我不希望把孩子带到一个变暖的世界。

All these kids are really worried about global warming, and you have all these things about 20 year olds, oh, I don't want to bring children into a warming world.

全球变暖是个大问题。

Global warming is a big problem.

但这个问题已经解决了。

It is a solved problem.

它之所以被解决，是因为光伏电池的效率每年以接近10%的速度复合增长，而电池效率则比这个数字低200个基点。

And it is solved because photovoltaic cell efficiency is compounding just under 10% a year, and battery efficiency is compounding 200 bps less than that.

如果你将这种增长趋势长期延续下去，甚至不需要太长远的未来，世界将直接依靠太阳能运行。

And if you compound that out, over the long term, not even the long term, the world is going to run on sunlight directly.

化石燃料将彻底消失。

Fossil fuels are just going to go away.

这是因为经济因素。

It's because of economics.

因为太阳能和储能的成本将低于任何其他发电方式。

And it's because sunlight and storage is going to be cheaper than every other way of providing power.

下一代现在不会再面临这个问题了。

It's just not going to be a problem for the next generation now.

也许我们达到了某个临界点，导致情况不可逆转，诸如此类。

Maybe we hit some tipping point and it's irreversible and this and that.

在我的有生之年，排放量将急剧下降。

Emissions are going to collapse in my lifetime.

真的会急剧下降。

Literally collapse.

显然，像前工业时代的人类一样，我们实际上是巨大的污染者，因为每单位木柴的污染非常高，而你必须生火，否则就会被剑齿虎吃掉。

Obviously, like pre industrial age humans, we're actually massive polluters because the pollution per unit of firewood is really high, and you needed fires, otherwise you'd get eaten by a saber toothed tiger.

但那时候人类数量并不多。

Now, there weren't that many of them.

在我的有生之年，我们有可能回到新石器时代的排放水平，前提是我能因为人工智能而多活几年。

It is possible that we're going to be back to neolithic levels of emissions in my lifetime, assuming I could live a little bit longer because of AI.

所以这是一个令人振奋和鼓舞的想法。

So that's an awesome and encouraging thought.

这很好。

That's good.

我只是希望有人能大声疾呼这一点。

And I just wish somebody was shouting that from the rooftops.

好吧，我们就在那儿。

Well, here we are.

我很想继续深入探讨，因为每一层都十分有趣。

I'd love to keep going up the stack here because every single level is interesting.

半导体本身以及其中潜在的创新让我很感兴趣。

So the semiconductors themselves and the potential innovations there are interesting to me.

从数据层面一直到应用层，我都觉得非常有趣。

The data piece is really interesting to me all the way up to the application layer.

我只是好奇你对这一切有什么看法。

And I'm just curious what you think about all of this stuff.

我们在这所有讨论中都假设，到了GPT-5、6和7时，我们将拥有更多数据来训练它们。

So one of the things that we're assuming in all this is that we're gonna have more data to train these things on at GPT five, six, and seven.

我认为关于未来可用数据是什么、我们如何获取更多数据或如何创建合成数据，以及这些方法是否能帮助打造更好的模型，存在一些有趣的讨论。

And I think there are some interesting discussions about what available data there will be or how we'll get more data or how we'll create synthetic data and whether or not that will work to create a better model.

比如，我对您如何看待这一切很感兴趣，因为如果我们真要展开您之前所描述的这场大规模竞争，这些条件都是必需的。

Like, I'm interested in what you think about all of this stuff that is required if we're gonna have the big arms race that you were describing earlier.

我认为九个月前这确实是一个悲观的情景。

So I do think this was a real bear case maybe nine months ago.

但我在Cloud 3.5的技术论文中看到过一些暗示，而在NVIDIA Nebotron的技术论文中则表达得更加明确。

But I think it was hinted at in the Cloud 3.5 technical paper and maybe a little more explicit in the NVIDIA Nebotron technical paper.

你知道，这太棒了。

You know, it's awesome.

NVIDIA，随便吧。

NVIDIA, whatever.

我觉得AI的发展可能会遇到一个瓶颈因素。

It feels like there's gonna be a rate limiting factor for AI.

他们解决了这个问题，而Nebotron正是做到了这一点。

They solve it, and that's what Nebotron did.

但我认为，由于一些没人理解的原因，没人真正理解这些模型。

But I do think for reasons that no one understands, and no one understands these models.

没人明白它们是如何工作的，为什么能工作，为什么存在缩放定律。

No one understands how they work, why they work, why scaling laws.

有各种各样的理论，也许我们对它们的理解正在一点点进步，但没人真正理解它们。

There's all sorts of theories, and maybe we're getting a little better at understanding them, but no one understands them.

所以，第一点，没人理解，但看起来合成数据是有效的。

So no one understands, point one, but it does look like synthetic data works.

没人明白为什么，但看起来它确实有效。

No one understands why, but it looks like it works.

那么，再次强调，它会持续有效吗？

Now, again, will it continue working?

我不知道。

I don't know.

没人知道。

Nobody knows.

没人知道。

No one knows.

见过凯文·斯科特的人刚做了一个播客，他基本上说：看。

People who have seen Kevin Scott just did a podcast, and he basically said, look.

我看过一些GPT-5的早期检查点，扩展定律仍在继续。

I've seen some early checkpoints of GPT five, and scaling laws are continuing.

我认为，在很多方面，这是目前我们拥有的最好的迹象，表明它们确实在扩展。

I think in a lot of ways, that's the best indication we have that they're scaling.

实际上，就XAI而言，我认为OpenAI结合了XAI所做的工作和Blackwell的延迟。

And I actually think largely in respect to XAI, I do think OpenAI is the combination of what XAI is doing and Blackwell delay.

Blackwell的延迟意味着，如果你在等待Blackwell并试图组建一个10万台规模的集群，XAI将可能拥有一年的优势，这对其他实验室来说是不可接受的。

The Blackwell delay means that if you're waiting for Blackwell and you're trying to get a 100,000 cluster, XAI is gonna have arguably a one year advantage, which is untenable to these other labs.

所以现在他们都急于启动自己的10万台规模集群，但他们没有埃隆从第一性原理出发来设计和重新设计数据中心。

So they're all now frantically working on stating up their own 100,000 cluster, but they don't have Elon designing the data center, redesigning the data center from first principles.

数据中心架构以前一直只是锦上添花的东西。

Data center architecture was always like kind of nice to have.

现在这已成为必需品。

Now it is must have.

这关乎生死存亡。

It's existential.

我认为合成数据看起来确实会有效。

I think synthetic data does look like it's gonna work.

这引出了一个问题：这些模型的价值将从何而来？

This goes to where will the value for these models come from?

为什么Meta对开源如此坦然？

Why is Meta so comfortable open sourcing?

你最终可能会看到所有人都开源。

You may ultimately see everyone open source.

XAI已经开源了GrokWOD。

XAI has open source, GrokWOD.

但原因在于，价值可能并不来自模型本身。

But the reason is the value may not come from the model.

现在想想，如果你在运行我描述的这个方程，并且在每美元资本支出和每瓦特的exascale算力上拥有1到200%的优势，且规模定律成立，那么你在模型质量上将获得巨大的优势，以至于你永远不会开源它。

Now look, if you're running that equation I described and you have a one to 200% advantage on exaflops per CapEx dollar per watt and scaling laws hold, you're going to have such a massive advantage of model quality that you're never going to open source it.

你仍然可以有效地窃取模型并进行蒸馏。

You could still effectively kind of steal models, distill them.

如果你拥有规模定律带来的计算优势，天啊，你将拥有极其宝贵的东西。

If you have that compute advantage of Scalie Laws whole, wow, you're gonna have something very valuable.

价值显然来自于分发渠道和独特数据。

The value clearly comes from distribution and unique data.

Meta已经开源了。

Meta is open sourced.

但他们并没有开源所有数据，这些数据只会用于他们自己的LABA版本。

Haven't open sourced all their data, and that data is just gonna be for their version of LABA.

所以它肯定会更好。

So it'll for sure be better.

谷歌拥有YouTube，以及他们在试图构建知识图谱时开发的各种其他数据源——那些在你搜索时出现的小小知识面板。

Google, they have YouTube, and then all sorts of other data sources that they've developed when they tried to boil the ocean to create the knowledge graph, which are those little knowledge panels that kind of appear when you do searches.

因此，结合YouTube、知识图谱项目以及谷歌地图，他们拥有海量数据。

So between YouTube and the work they did for the knowledge graph and Google Maps, they have crazy data.

他们并不在意，因为他们可以用这些数据来为自己的模型变现。

They don't care because they could use that data to monetize their model.

XAI无论是否开源，都肯定能以其他人无法比拟的方式访问xData，因为x拥有XAI 25%的股份。

XAI, whether they open source or not, they will always have, for sure, access to xData in a way no one else does because x owns 25% of xAI.

而且我认为，随着时间推移，XAI将成为贯穿埃隆旗下整个公司生态系统的智能层。

And then I would think over time, xAI will kinda be an intelligence layer that cuts across Elon's ecosystem of companies.

顺便说一下，我们应该谈谈人工智能与机器人技术，这可能是我们这一生中最大的颠覆，堪比人工超级智能和这些数字神明。

And by the way, we should talk about AI and robotics, which I should think may be the biggest disruption in our lifetime, comparable to artificial superintelligence and these digital gods.

如果这些模型没有人在BAMAF、SFU、检查点频率和PUE方面形成复合优势，从而在每瓦每美元资本支出的exaflops性能上实现显著差异，我认为它们最终都会收敛到大致相同的智能水平。

If these models, unless someone develops a compounded advantage of BAMAF, SFU, checkpointing frequency, and PUE, such that they have a dramatic difference at exaflops per dollar of CapEx per watt, I think they're probably all gonna converge to roughly the same intelligence.

事实上，以我认为至少谷歌和Meta的思维方式来看，即使其他人领先他们很多且效率更高，他们也会试图用金钱来解决这个问题。

And the reality is, given the way that I think at least Google and Meta are thinking, even if someone else is way ahead of them and more efficient, they will try to solve the problem with money.

哦，天哪。

Oh, wow.

他们只花了三千亿美元？

It only cost them 300,000,000,000?

没问题。

No problem.

我们很乐意花一万亿美元。

We're happy to spend a trillion.

不过，最终经济规律还是会起作用。

Although, ultimately, economics will apply.

这些可能会影响股价。

These may impact stock prices.

比如，我认为这七个巨头中的一些公司，这种事并非不可想象。

Like, and I think it's not inconceivable, some of these magnificent seven.

别管股票回购和股息了。

Forget buybacks and dividends.

他们可能会取消股息，停止股票回购，转而发行新股来资助这项事业。

They may eliminate their dividends, stop buybacks, and start issuing stock to fund this.

多年来，我们在很多方面都可能处在一个极其疯狂的世界中。

For years, we could be in a really wild world in a lot of ways.

但这些智能体很可能最终会趋同于某种相似的‘智商’。

But these intelligences will probably converge on kind of a similar what I'll call IQ.

而关键就在于谁拥有最独特的实时世界数据。

And then it's just who has the most differentiated real time data about the world.

这些独特的数据源，加上你每次对这些模型的回答进行评分，都在帮助它们改进。

It's these unique data sources coupled with every time you rate an answer from one of these models, you're helping it improve.

因此，如果你能把独特数据与互联网规模的分发结合起来，你就拥有了制胜公式。

And so if you can couple unique data with Internet scale distribution, then you're going have a winning formula.

但只有少数几家公司具备这种能力。

And there's only a few companies that have that.

比如xAI、谷歌、微软，尽管在互联网规模方面可能有所不同。

You know, it's xAI, it's Google, it's Microsoft, although Internet scale maybe.

而OpenAI则通过微软获得这种能力。

And OpenAI gets that through Microsoft.

可能是亚马逊和Anthropic通过这种方式获得优势，当然还有Meta。

Probably Amazon and Anthropic gets it through that, and then, you know, obviously, Meta.

然后苹果是一个巨大的变数，我们应该谈谈这一点。

And then Apple is the big wild card, and we should talk about that.

因为我认为最重要的一件事是，推理将在哪里进行？

Because I think one of the biggest things is where is the inference going to happen?

计算资源往往在集中化和去中心化之间循环。

And compute tends to cycle in between centralization and the decentralization.

我们现在正处于一个长期集中化的末期，大量计算都在云端的大数据中心运行，这是因为这些大型数据中心能实现更高的效率。

And we're at the end of, like, a long period of centralization in the cloud, where a lot of compute ran in the cloud in these big data centers is just because you could get much higher efficiency in these big data centers.

显然，对于交易来说，这将发生在云端的巨型数据中心中。

Now clearly for trading, that is gonna happen in giant data centers that will be in the cloud.

不过，我认为人们对这些AI数据中心的一个被低估的方面是，我们不必长期担心它们会压垮电力需求，因为它们可以建在任何地方。

Although something that I think is underappreciated about these AI data centers and why we shouldn't worry about them over the long term crushing power demand is you could put them anywhere.

你可以把它们建在怀俄明州。

You put them in Wyoming.

它们不需要靠近大城市。

Like, they don't need to be near a big city.

我认为我们最终会看到巨大的数据中心建在页岩气田上，旁边配有发电厂，远离任何人类，这可能是美国短期内缺乏核能的解决方案。

I think we'll eventually see giant data centers in shale gas fields with power plants on those fields long away from any humans that will probably be an intermediate term solution to the lack of nuclear power in America.

美国确实拥有所有最优秀的模型。

The US, we do we have all the best models.

无论好坏，美国都希望这些模型在美国境内进行训练。

And The US wants these models trained in America for better or worse.

显然，Mistral 是法国的。

Obviously, Mistral is French.

我不知道这些模型是在哪里训练的。

I don't know where the models were trained.

它们会根据在哪里能以最高利用率获得最低成本的计算资源，经历集中化与去中心化的循环，这类似于我前面描述的方程的一种变体。

They have these cycles of centralization and decentralization based on where can you get the lowest cost compute at the highest utilization, kind of a variant of the equation I described earlier.

我认为这个方程的各个组成部分，不仅对实验室有帮助，对投资者也大有裨益。

And I do think the subcomponents of that equation, not only are they maybe helpful to labs, but they're really helpful to investors.

如果你能将SFU提升20%，一切就归结于几毫米的硅片。

If you can improve SFU by 20, it all comes down to millimeters of silicon.

仅仅多出几毫米的硅片，甚至可能更少，他们就会惊呼：天啊，因为硅片的毫米数才是成本的终极决定因素。

For only a few more millimeters of silicon or maybe even less millimeters of silicon, they're like, oh my god, because millimeters of silicon are the ultimate in cost.

哇，你简直拥有一个极其成功的公式。

Wow, you have like a massively winning formula.

所以，你可以几乎逐个审视这条漫长链条中的每个环节，哪里的每平方毫米硅片成本最高？

And so you can just almost go look at each step of that long chain and where do you have really high cost per square millimeter of silicon?

而那里就是真正可以优化的机会。

And then that is the opportunity to really optimize.

无论是在基础设施软件、数据中心硬件，还是半导体层面，机会都存在。

And whether it's at the infrastructure software, the data center hardware, the semiconductor layer, it's all there.

但我认为，推理任务将越来越多地在手机上完成。

But I think you're gonna see inference increasingly done on phones.

这显然是苹果的布局。

So this is clearly Apple's play.

这是他们处于如此有利地位的原因之一。

It is one reason that they are in such a advantaged position.

所有其他公司都像囚犯一样被困住了。

All these other companies are trapped when these prisoners deliver.

我确信，考虑到成本如此之高，他们一定也希望如此。

I am sure they would like, given how much it costs.

他们都是经济动物。

They're all economic animals.

如果他们能达成一致，说：你知道吗？

If they could just reach an agreement and say, you know what?

别杀了杰夫。

Not killing Jeff.

在2026年之前，没人会搭建Blackwell集群，他们可能都会签字同意。

Nobody is gonna stand up a Blackwell cluster until 2026, they'd probably all sign it.

但这简直就是经典的囚徒困境，而那将是一个纳什均衡。

But it is literally a classic prisoners to live, and that would be a Nash equilibrium.

但在国家处于生存危机的情况下，我们不可能在创造数字上帝的竞赛中达成纳什均衡。

But we're not going to reach a Nash equilibrium in the race to create digital God when the states are existential.

所以这里存在囚徒困境，但苹果并不在其中。

So there is prisoners to live, but Apple's not.

同样地，谷歌在资本支出、运营支出等各方面累计投入了数百亿美元用于搜索。

By the same way, Google spends collectively, cumulatively spend, I don't know, hundreds of billions of search between CapEx, OpEx, all this stuff.

而苹果凭借其在iOS系统中的分发垄断地位，几乎和谷歌一样有效地实现了变现。

And Apple monetizes it almost as well as Google because they have this distribution chokehold, Tollbooth, in iOS.

他们显然会像谷歌在安卓手机上那样，对Apple Intelligence采取同样的做法。

They're clearly gonna do the same thing with Apple intelligence that Google will obviously do that with their phone, the Android.

因此，你的手机上将运行一个小型模型，用于处理简单问题，可以把它想象成一个拥有100智商、具备极其丰富知识的模型。

And so you're gonna have a small model running on your phone that for simple questions, think of it as like a 100 IQ model that with, like, really, really sophisticated knowledge.

可能会有两个这样的模型。

There'll probably be two of them.

它们会相互核查。

They'll check each other.

因此，大量推理将在边缘端进行。

And a lot of inference will then happen at the edge.

如果推理在边缘端进行，我认为我们将看到超级手机。

And if inference is happening at the edge, then I do think we're gonna see superphones.

因为对你作为人类而言，限制推理的主要因素通常是内存，而如今iPhone的售价正是基于其闪存容量。

Because the advantage to you as a human, what bounds inference at most times is memory, and you will be willing to pay today iPhones are sold based on how much flash storage they have.

你最关心的将是手机上的DRAM容量，因为它将决定你能本地运行的模型参数规模。

What you will most care about is the amount of DRAM you have on your phone because that will determine the perimeter count of the model you can run locally.

而这正是你本地智能的质量，它能以隐私安全的方式访问你所有的数据。

And that is therefore the quality of your own local intelligence that has access to all of your data in a privacy safe way.

当你向它提问时，如果能在本地完成，它就会在本地处理。

And when you ask it for something, if it could be done on a compute, it will.

我认为在网络领域，一个真理是：能路由时就路由，必须时才交换。

And I think for networking, like a truism is route when you can, switch when you must.

但我觉得，对于推理来说，下一步将是：能本地处理时就本地处理，必须时才用云端。

But I think the next thing for inference will be local when you can, cloud when you must.

所以，如果你能在手机上本地进行推理，你总会选择这样做，因为它是免费的。

So if you can inference locally on your phone, you will always do that because it's free.

而云端推理肯定是要花钱的，因为你正在消耗GPU小时。

And cloud inference definitely costs money because you're burning GPU hours.

现在虽然已经非常高效了，但在手机上进行推理实际上是免费的。

Now it's very efficient, but the inference on your phone is effectively free.

然后，作为人类，我们的竞争力正变得越来越明显，这很有趣。

And then increasingly, our competitiveness as humans you know, it's very funny.

很多非常聪明的人对人工智能持一点怀疑态度。

A lot of really smart people are a little skeptical about AI.

如果你的智商是125、135或者更高，我能理解。

I get it if, like, you have a 125 IQ or a 135 or whatever it is.

人工智能，尤其是在你的领域，并不让你觉得有多了不起。

AI, particularly your domain, isn't at all impressive to you.

然后你就说，好吧。

And it's like, okay.

对我来说，这有点像通过搜索来了解其他领域。

It's a little bit for search for me learning about other domains.

但老天，很多人的智商并没有达到120。

But, man, a lot of humans don't have one twenty IQs.

人们忽视的是，有很多人的智商，比如说，我不知道。

And what people are missing is that there's a lot of humans with, like, I don't know.

我完全不知道。

I have no idea.

100的智商。

A 100 IQs.

最多的是100智商的人。

The most with a 100 IQs.

是的。

Yeah.

那就假设是100智商吧。

So let's just say it's a 100 IQs.

他们可以使用AI，突然间，他们的智商就像变成了115。

And they could use AI, and all of a sudden, they're like a one fifteen.

这简直让人震惊。

And it's like, holy shit.

太不可思议了。

It's incredible.

而这种情况会持续下去，直到这些ASI的智商达到一千，那时人们就会想：作为人类，我为什么还要做其他事，而不只是和AI一起创作艺术呢？

And that's just gonna keep going until these ASIs have IQ of a thousand, and it's like, why, as a human, am I bothering to do anything other than work with an AI to create art?

不管怎样，如果我能拥有一部价值3000美元的iPhone，它的DRAM是千元iPhone的四倍，存储空间也更大，以便本地模型能运行RAG。

Anyways, if I could have, like, a $3,000 iPhone that has four times the DRAM on it that the thousand dollar iPhone has, it did maybe more storage so that local model could do rag.

这是我的模型。

And this is my model.

它喜欢我。

It likes me.

我选择用来和我对话的声音。

I pick the voice that talks to me in.

它了解我。

It knows me.

它是我的朋友。

It's my friend.

它是我的代理。

It's my agent.

在改进这些模型的过程中，所有这些都同时为我点赞。

All at once is thumbs up for me in that process of improving these models.

我认为最终会有一种方式，让它们能够按需为个人提供RLHF支持，但这需要大量突破才能实现。

I think they will ultimately be there will be a way that they could be RLH staffed on an individual basis on the fly, which we're gonna need a lot of breakthroughs to make that happen.

它会是我的模型，喜欢我、了解我，并知道我想要什么。

And it's gonna be my model that likes me and knows me and knows what I want.

然后，这就是大家所谈论的代理未来：我手机上有一个代理。

And then this is where you get this agent future that everybody's talking about, where it's like I have an agent on my phone.

如果我手机上有一个代理，而我的超级手机拥有115或120的智商，而别人的代理只有100的智商，那么作为人类，我会拥有显著的优势。

And if I have an agent on my phone, because I have a super phone that has, you know, an IQ of a 115 or a 120 and someone else's agent only has an IQ of a 100, I'm gonna be profoundly advantaged as a human.

然后这种趋势会一直持续下去，因为我相信苹果最终会从中盈利。

And then that continues all the way up because I think eventually Apple will monetize this.

苹果实现盈利的方式显然是因为设备上的LLM还不够智能。

And the way Apple will monetize this clearly is with the odd device LLM isn't smart enough.

它们会将任务发送到云端。

They're gonna send it to the cloud.

它们会使用一种叫做路由器的技术，这是每家AI应用公司都在使用的。

They're gonna use something called a router, which is what every AI application company uses.

你只想根据每美元的成本，将查询路由到最合适的模型。

You just wanna route to the best model for the query per dollar.

苹果会向公司收费，以让它们进入自己的路由器系统。

Apple will charge companies to be in their router.

然后，如果你想被路由到更高级的模型，最终你不得不支付越来越高额的费用。

And then, oh, if you wanna be routed to, eventually, you're gonna have to pay a bigger and bigger toll.

每个人都会像谷歌那样支付这笔费用。

And everybody will pay that toll the same way Google paid that toll.

但如果我作为一个普通人，能在本地拥有一个更智能的模型，然后我可以选择让苹果让这个过程变得很简单。

But if I, as a human, I could have a smarter model locally, and then I could opt in into maybe Apple will just make it easy.

他们会说：好吧，你可以选择云超级智能，或者选择云智能。

They'll say, okay, you can have cloud superintelligence or you can have cloud intelligence.

而我每月支付60美元来获得云超级智能。

And I pay $60 a month for cloud superintelligence.

不管它是每月1000美元、10000美元，为了这个你愿意付多少钱？

Now, or whatever it is, dollars 1,000 a month, dollars 10,000 a month, what wouldn't you pay for that?

所以我的手机比很多人多出了20点智商，而我每月支付10000美元来获得云超级智能、超智能。

And so I have 20 points of IQ on my phone relative to a lot of people, and then I'm paying $10,000 a month for cloud superintelligence, hyperintelligence.

超级智能每月只要1000美元，而普通智能每月20美元。

Superintelligence is just $1,000 a month, and then regular intelligence is $20 a month.

作为一个人，我会获得巨大的优势。

Like, I'm going to be really advantaged as a human.

这在我看来很反乌托邦，但我也很难不认为它会真的发生。

And that seems dystopian to me, but it's also kind of hard for me to not see that happening.

许多投资结论都由此衍生，就像说，嘿。

And many, many investment conclusions flow from this in the same way that they flow from like, hey.

好的。

Okay.

猛犸象公司达到了90%。

Mammoth is at 90%.

好的。

Okay.

所以这可能不是一个好的投资地点。

So that's probably not a good place to invest.

SFU达到了30%？

SFU is at 30%?

哇。

Wow.

这是个绝佳的投资地点。

That's a great place to invest.

PUE为1.8，理论上可以达到1.3。

PUE is at 1.8, and it could theoretically be at 1.3.

在这个等式中，这是个绝佳的投资地点。

That's a great place to invest in that equation.

你最好投资于效率最低的那条链。

You kind of want to invest at the most inefficient chain.

推理发生的位置会带来大量投资影响。

A lot of investment implications flow from where does inference happen.

而这对人类也会产生许多深远影响。

And then a lot of implications for humanity flow from that as well.

我很想聊聊这个等式中更普通的一面。

I'd love to talk a little bit about the commoner part of this whole equation.

我们谈到了奥林匹斯山，以及那些为争夺主导权而相互竞争的王者游戏。

We talked about Mount Olympus and the game of kings fighting each other for dominance.

但我们还没有讨论过，一个普通的新公司如何利用这些超级智能在应用层开展业务，或者做其他事情？

We haven't talked at all about what about just a normal new company that's being started to take advantage of these super intelligences at the application layer or to do something else?

之后我们再谈机器人。

Then we'll get to robotics after that.

但如果我迫使你只能作为初创公司的早期A轮或B轮投资者，而它们没有这些卓越七家公司及其他巨头所拥有的资源，你会如何看待在GPTX持续扩展和提升的世界中，哪些公司和机会最具吸引力？

But if I force you to just be a early stage series A, series B investor in the world of startups and they don't have this kind of resources that all these magnificent seven and others have, how would you think about the aspects of companies and opportunities that would get most attractive in the world where GPTX keeps scaling and getting better?

我首先想说的是，关键始终在于人们在做什么。

First thing I would just say, it's always what are people doing?

我目前专注于那些改进我前面提到的方程要素的公司，因为我认为……

What I am doing is really focusing on companies that improve that equation elements of that equation I described earlier because I think

这是一个瓶颈。

That's a choke point.

这是最有可能成功的方向。

That's the highest likelihood of success.

在科技领域，只要你发现了一个瓶颈，如果能投资于缓解该瓶颈的领域，通常都会取得不错的结果。

And in tech, whenever you find a constraint, if you can invest in something that alleviates that constraint, you usually do well.

而目前，我深信瓶颈在于SFU、检查点，这关系到可靠性，以及PUE。

And right now, I profoundly believe the constraint is an SFU, checkpointing, which goes to reliability, and then PUE.

因此，这就是我投资的方向。

So that is where I am targeting my dollars.

我认为应用层真的非常困难。

I think the application layer is really, really hard.

有些人在这方面做得非常出色。

There are people who are killing it.

我觉得你播客里的莎拉，我虽然从未见过她，但非常钦佩。

I think you have Sarah, who I've never met, but I admire on your podcast.

从我的角度看，她在应用层上做得简直太棒了。

And she's, like, absolutely, from my perspective, crushing it at the application layer.

但说实话，今天投资应用层对我来说真的很难。

But just, geez, investing there today feels really hard to me.

我只是觉得，任何对这一点充满信心的人都应该记住查斯·科尔曼那1%的数据。

And I just think anyone who has a lot of conviction about that needs to be reminded of that 1% Chase Coleman stat.

是的。

Yeah.

我刚才也在想这一点。

I was just thinking that.

所有这些风投都对这样一种观点非常坚定：那些在1995年、1996年、1997年获得融资的公司，似乎都抓住了互联网的机遇。

All these VCs had really strong views of, like, there are all these companies that were funded in '95, '96, '97, and they seemed like they had got in the net.

但这需要时间。

But it just takes time.

但我想，他们当时确实是为风投而进入这个领域的。

But I guess they did go into that for the VC.

他们上市了，并且成功套现了。

They went public, and they were able to cash out.

但他们实际上并没有真正深入理解互联网的长远价值。

But they actually didn't go into the net with the fullness of time.

所以我只是觉得，在应用层面上保持极大的谦逊非常重要。

So I just think it's important to have a lot of humility at the application layer.

也许只是因为我对基础设施层非常熟悉。

And maybe it's just that I am so comfortable with this infrastructure layer.

到目前为止，我一直专注于这一领域。

That is where I have concentrated to date.

实际上，只有一个应用公司让我感到兴奋。

There's really only been one application company that I've been excited about.

我们见过很多这样的公司。

We've seen a lot of them.

也许我太了解蔡斯·科尔曼的数据了，因此在对待这一应用层时过于谨慎。

Maybe I'm, like, too aware of the Chase Coleman stat, and and I'm being too careful in how I approach this application layer.

但我可能会错过很多像CMGI那样了不起的公司。

But I'm going to miss out on a lot of CMGI was incredible.

除了雅虎之外，还有哪些公司？

What were some other besides Yahoo?

Lycos呢？

What were Lycos.

还有许多我们今天甚至想不起来的公司，但它们都是非凡的风险投资成果。

There are all these companies that we don't even think of today, but they're incredible venture outcomes.

也许我确实需要更加留意这一点，但显然有一些人，无论是Benchmark还是Cerusfirm，都在取得成功。

Maybe I do need to be a little more mindful of that, but they're clearly people, whether it's Benchmark, whether it's Cerusfirm, who are succeeding.

我在谈到应用层时提到的这一点，是听Vishria说的，但真的深深引起了我的共鸣。

And what I was saying in the application there, I heard this from Vishria, but it really resonated with me.

我认为这也涉及到整个AI技能投资回报率的争论。

And I think it also goes to this whole skilling AI ROI debate.

这不仅是一个关于创造数字神明的投资回报率争论，我认为还存在一个非常明确的投资回报率，而且你实际上可以通过数学计算来证明。

Not only is it an ROI debate about creating a digital god, but I think there's a super clear ROI, and you can actually do math to show that.

很多人将资本性支出和运营性支出混为一谈，这种方式毫无帮助。

And a lot of people are conflating CapEx and OpEx in ways that are just not helpful.

Meta的股价下跌了约80%，部分原因是他们在元宇宙上过度支出，但部分原因也是因为苹果通过IDFA剥夺了他们的定向投放能力。

Meta went down, like, 80% partially because they were overspending on the metaverse, but partially because Apple took away their ability to target through IDFA.

Meta的估值是五倍，其收入增长正在重新加速。

Meta's a five x, and the revenue growth is reaccelerated.

为什么它的增长会重新加速？

Why is it reaccelerated?

因为他们投入了大量资金用于人工智能，以研究如何精准投放广告。

Because they spent a vast amount amount of money on AI to figure out how to target.

也许仅凭元宇宙的回报就足以证明所有支出的合理性。

It's like maybe the meta return alone justifies all of the spending.

他们称之为元宇宙优势，但元宇宙优势只是其中一部分。

They call that meta advantage, but meta advantage is just part of it.

作为广告主，你可以让元宇宙平台来负责定向投放。

That's just where as an advertiser, you can let meta do the targeting.

你还需要谷歌的性能最大化广告。

You need and then Google has performance max.

所有这些都曾是广告创意产生的地方。

And all of that is that's where the it used to be ads were created.

我们稍后会回到应用场景。

We're gonna come back to applications.

广告创意曾经是由人类来决定的：我们需要向波士顿一位40岁的白人男性展示这种创意，向这个城市中特定人群展示另一种创意，并在一天中的特定时段投放。

But ads were created, we're gonna pay some humans to figure out that we need to show this creative to white 40 year old guy in Boston, and then this type to this type of demographic category in this city, and we're gonna show them at this time of day.

不，我们要在本地体育队赢球且天气晴朗时展示广告，因为那是看广告的最佳时机。

No, we're gonna show them after the local sports team is won and it's sunny because that's actually the best time to see an ad.

但现在AI可以完成所有这些工作。

But now AI can do all of that.

我们将拥有一百万种创意，并且会实时优化。

And we're gonna have a million creatives and it's gonna be optimized on the fly.

这就是AI正在帮助谷歌、Meta和其他公司实现的目标。

That's what AI is enabling Google and Meta and other firms to do.

而且可能仅凭这一点

And probably just on that

仅这一点就足以证明其价值。

Just that justifies it.

是的。

Yeah.

AI已经带来了巨大的投资回报。

There has been a massive ROI on AI.

我们在谈什么？

What are we talking about?

关于这场AI投资回报率的争论，另一件让我觉得特别有趣的是：好吧，我们要进行一场关于投资回报率的晦涩辩论。

And then the other thing that's really funny to me about this whole AIROI debate is, okay, we're going to have this abstruse debate about ROI.

这些公司都是上市公司，还有一种叫做资本回报率的指标。

These companies are all public, and there is something called return on invested capital.

自从这些公司增加资本支出以来，它们的资本回报率就一直在上升。

And ROIC has gone up for all of these companies since they ramped CapEx.

我们在谈什么？

What are we talking about?

如果你对AI的投资回报率持怀疑态度，为什么这些公司的资本回报率会上升？

If you're an AI ROI skeptic, why has ROIC got up at these companies?

是的，资本支出确实增加了。

Well, yeah, CapEx is up.

笔记本的支出增加得更多。

Notepad is up more.

为什么会这样？

Why is that?

因为他们正在做你在AI世界中预期会发生的事情。

Because they're doing exactly what you would expect to happen in the world of AI.

他们正在用GPU小时替代人力劳动。

They're trading off human labor against GPU hours.

这就是为什么他们的ROIC会上升，而这些GPU确实非常高效。

That's why their ROIC has gone up and these GPUs are really efficient.

让我们来讨论一下AI的投资回报率吧。

Let's have an AI ROI debate.

当那些大规模投入AI的公司的ROIC开始下降时，才值得讨论；在此之前，这纯粹是智力上的荒谬。

When the ROIC at the big AI spenders starts to go down, until then, it's just the height of intellectual ridiculousness.

我的意思是，荒谬至极。

I mean, ridiculous.

维什里亚提到的一点，我觉得对应用层非常有力：有这么多SaaS指标。

What Vishriya said that I thought was so powerful for the application layer, there's all these SaaS metrics.

第一年后，你应当达到百万级别才能跟上节奏。

After your first year, you ought to be at million to be on pace.

第二年后，你希望达到五百万。

After your second year, you want to be at 5,000,000.

第三年后，你希望达到大约一千万。

After your third year, you want to be at like 10.

如果你的现金消耗合理且超出这个水平，你很可能能跑通这些曲线，成为一家成功的SaaS公司。

And if you're above that with reasonable cash burn, you could probably run these curves, you're going be a successful SaaS company.

所有这些曲线，大家都知道，正是SaaS估值倍数被推高的原因之一，因为它几乎变得像量化指标一样。

And all those curves, which everybody knew is one reason SaaS multiples got so inflated because it became almost like quantitative.

哇，这家公司在第三年就达到了150亿美元？

Oh, wow, this company is at $15,000,000,000 in year three?

这意味着我们可以预测它在第八年将达到十亿美元。

That means we can pencil them in for a billion dollars in year eight.

A，这导致估值倍数被疯狂推高。

A, that led to multiples expanding ridiculously.

B，这导致整个行业融资过度，使得每个垂直领域的竞争加剧，导致这些曲线不再成立。

B, it led to the industry being overfunded such that the degree of competition in each vertical went up such that the curves no longer held.

这也是为什么如果你在2021年左右进行了大量SaaS投资，你现在会陷入困境的原因之一。

And this is one reason why if you made a lot of SaaS investments, particularly in 2021, you're in a world of pain.

此外，AI出现了，它从根本上改变了应用软件的范式，因为应用软件的核心作用是提升人类效率。

And then on top of that, here AI comes along, and it fundamentally changes the paradigm for application software because what application software fundamentally does is makes humans more efficient.

而如今，我们正处于AI时代，它确实提升了人类效率，但你不再需要围绕它构建那么庞大的系统。

And today, we're in state with AI where it's making humans more efficient, but you don't need that big of a wrapper around it.

这就引出了我们关于渔业的评论。

That's where we're gonna come to the fishery comment.

最终，如果AI开始取代人类，而你以每用户席位模式销售应用软件，当用户席位开始减少时，你就面临问题了。

And then ultimately, if it's replacing humans, if you're selling application software on a per seat model and those seats start to go down, you have a problem.

这些公司对此非常清楚。

These companies are super aware of it.

它们曾经一路狂奔，但现在却不得不在每个垂直领域应对新一代以AI为核心的公司，这些公司通常只是围绕GPT构建的极其轻量的封装。

They were going fast in one direction, and now they have to contend almost in every vertical with this next generation of AI first application companies, which are just these often very thin wrappers around GPT.

你会称它为一个LLM包装器。

You'll call it an LLM wrapper.

渔业部门告诉我一件特别有趣的事：所有这些AI公司都在彻底颠覆传统的SaaS指标。

And what Fisheries said to me that was so funny is like all these AI companies are blowing these traditional SaaS metrics out of the water.

但这些情况并没有在那些AI投资回报率论文中被提及。

And all this isn't being touted in those AI ROI papers.

现在有这么多公司能在九个月内从零做到三千万美元的收入。

Just that there's all these companies that are going like 0 to 30,000,000 in nine months.

尽管AI的边际成本其实很高，但它们相对于传统软件公司却更注重现金流效率。

And even though AI is really high marginal cost, they're being more cash flow efficient relative to software company.

因此，在几乎每一个垂直领域，都有多家AI优先的公司，正如Vishri所说，它们只是围绕你选择的任何LLM做的极其轻量的包装。

So in almost every vertical, there's multiple companies that are AI first that are just really I think what Vishri said was they're just paper thread wrappers around pick your LLM of choice.

然后，是的，它们会使用路由机制来选择最佳的LLM，但对客户来说，它们就像魔法一样。

Then, yeah, they use a router to find the best LLM, but they're like magic to their customers.

它们并不是在争夺软件预算。

They aren't going after software budgets.

他们瞄准的是人力预算。

They're going after labor budgets.

观察这一切对我来说非常有智力上的吸引力。

It's very intellectually interesting to me to watch.

你如何获得这样的信念：这些公司中的一家能够利用最初对客户来说如同魔法的功能来建立护城河，同时承认大型模型将继续以极高的速度持续进化？

How do you get conviction that one of these companies can use what is initially magical to their customer to build defensibility in, all while allowing for the fact that the big models are going to continue to compound at a really high rate?

这可能涉及到找到一种非常有效的销售方式。

And this goes to things like maybe you just find a really good sales motion that works.

也许你找到了一个非常容易集成的切入点。

Maybe you find a really easy integration point.

也许你在这种集成基础上建立了某种护城河。

Maybe you build some defensibility around that integration.

也许你让小企业能够在非常原始的系统上轻松实现RAG。

Maybe you make it really easy for a small business to do rag on a really unsophisticated system.

你如何在这一层封装中建立差异化？

How are you building differentiation into that wrapper?

而且希望你不要进行微调，因为深度微调会把你锁定在某个模型生成上。

And hopefully, you're not fine tuning because that does extensively fine tuning because that locks you into a model generation.

你如何构建一个复合型AI系统，使得每次查询都能以最低成本完成，并且从一个小模型开始，逐步过渡到大模型？

How are you creating a compound AI system such that you're serving each query at the lowest cost possible and you're starting with a small model and treating it to a big model?

你可以做很多事情，而这些都非常重要。

There's a lot of things you can do, and all of those are really, really important.

但我只是觉得，在应用层面上，似乎任何你能想象到的类别，都有多个初创公司迅速崛起，完全颠覆了传统的SaaS指标。

But I just think at that application layer, there's just it seems like for any category, any category it feels like that can be imagined, there's multiple startups that have exploded, that are defying all traditional SaaS metrics.

到目前为止，我还不清楚它们每一个究竟有多少竞争优势。

And it's not clear to me at this point how much defensibility each one of them has.

其中一些可能会在我说的某个维度上拥有中等程度的竞争壁垒，但我认为很多可能并没有。

Some of them are going to have a mid defensibility around one of the axes I described, but I think a lot of them may not.

我认为这正是这个领域如此难以把握的原因，也是我为何如此谨慎地对待这个领域。

And I think that's what makes it so hard and why I've been approaching that space so carefully.

你知道最令人惊叹的是什么吗？

You know the amazing thing?

我认为关于劳动力的这种说法，举个最简单、最普通的例子来说吧。

I think the framing around labor I mean, just take me as a stupid, simple, tiny example.

我有一个由三个人组成的团队在开发这样一个系统。

I have a team of three people building one of these.

我们可以把它称为一个轻量级的LLM包装器，用于对私营公司进行研究。

We'll call them like a really light LLM wrapper for doing research on private companies.

如果你想想我每天使用这个工具的频率，就像我以前使用分析师那样，一天好几次，而且它返回的结果质量相当。

And if you just think about how often I use this thing in a way that I normally would have used an analyst, it's multiple times a day, and it's returning work as good.

所以，在每一个白领劳动市场中，都有大量没有差异化的繁重工作，我想这是其中一个主要的发现。

So there's so much undifferentiated heavy lifting in every white collar labor market, I guess, is like one major learning.

100%。

100%.

如果你

If you

能立即获得答案，而且几乎是免费的，那你会多么频繁地使用这些工具，简直难以置信。

can have the answers instantly and for effectively free, it's crazy how much you use these things.

我们才刚刚开始。

We're just getting started.

是的。

Yeah.

在我的公司，我也有一样的情况，用了一个大语言模型。

And I have the same thing with, like, an LLM at my firm.

这其实非常有趣。

It was actually very interesting.

我们有一个非常有才华的实习生。

We had an intern who was really talented.

他说：听着，我觉得这个内部AI工具太棒了。

And he said, Listen, I think this internal AI tool is amazing.

我每天都用，而且使用频率越来越高。

And I use it every day and more every day.

今年夏天，它变得好太多了。

It's gotten much better this summer.

我们为此投入了很长时间。

We work at it for a long time.

然后我说，给我看看你是怎么用的。

And then I was like, Show me how you use it.

因为我一直用的是某种方式。

Because I'd been using it a certain way.

结果这个21岁的孩子说，嗯，我就是这样用的。

And then this literally, this 21 year old kid was like, Well, this is what I do.

突然间，我使用这个工具的时间变成了每天两小时、三小时、四小时。

And now all of a sudden, my usage of this tool is two hours a day, three hours a day, four hours a day.

我认为杰克·韦尔奇有个术语叫‘琐碎工作’，就是在白领环境中必须做得很好的那种辛苦、不愉快的工作。

And I do think Jack Welch had this term called scut work, just like hard, unpleasant work that had to be done really well in a white collar setting.

我认为这些所谓的‘小助手’或‘迷你助手’——不管我们怎么称呼它们——将取代大量这类琐碎工作。

And I think it's all these little rappers or mini rappers, whatever we're gonna call are gonna replace a lot of that scutt work.

最初，它们会与人类协同工作。

And initially, it's gonna be in combination with humans.

在科幻小说中，尼尔·阿舍的世界里有一种叫做‘高频带’的东西，是一种人机混合体。

With science fiction, Neil Asher's world, there's something called a high band, which is a human AI hybrid.

它们通过类似Neuralink的装置连接到一个由外部结构支撑的计算机服务器上，这些外部结构是机器人外骨骼，人们可以靠它行走。

And it's where they're linked through something like a Neuralink to, like, a computer server that is supported by, like, an exo structure, robotic exo structure that they walk around on.

这就像某种融合象棋，你怎么叫都行。

There's, like, fusion chess, whatever you want to call it.

我们会有一段时间处于这种状态。

We'll have that for a while.

然后，100智商的人表现得像120智商的人，接着表现得像130智商的人，再之后130智商的人表现得像160智商的人。

And then that goes to the 100 IQ humans performing like 120 IQ humans, and then performing like 130 IQ humans, then 130 IQ humans performing like 160 IQ humans.

但最终，只要规模定律持续下去——这本身就是一个巨大的未知数——最终就只会是AI了。

But then eventually, it feels like as long as scaling laws continue, which is a big F, it's just gonna be the AIs.

也许我们可以聊聊机器人技术。

Maybe we could talk about robotics.

我曾在四月份与一位投资者进行了一次非常有趣的对话，他长期私下和公开地投资于这些领域，持有我们今天讨论的许多公司的大量股份。

I had a really interesting conversation, call it April, with an investor that has been investing in lots of these same things for long periods of time, privately and publicly, and has big positions in lots of the companies that we've been talking about today.

他向我指出，在未来五年左右的时间里，人们严重低估了机器人与我们今天讨论的全部技术相结合所发挥的作用。

His observation to me was the big underestimation that's happening over, let's say, five years is the role that robotics and robots will have combined with all of this technology we've spent all of today talking about.

我很想听听你对这一点的看法，因为短期内，感觉这些公司融资热潮有些过热，你根本不清楚这些机器人到底设计来做什么，基本上都是通用人形机器人。

And I would love to hear you riff on that because in the near term, it feels like a little bit quite a bit of frothiness, like some crazy funding rounds for these companies that you don't really know what the robots are being designed to do, sort of general purpose humanoid type robots.

还有很多更专业的机器人也很有趣。

There's all sorts of interesting, more specialized ones that are cool too.

但你对这一切有什么看法？

But what do you think about all this?

相对于基础模型和半导体领域，机器人似乎被讨论得少得多。

It does seem kind of under discussed relative to just all the foundation model and semiconductor stuff.

我同意。

I agree.

我认为，机器人可能带来的短期颠覆性影响，会超过我们刚才讨论的白领劳动自动化。

I think it may end up being a bigger near term disruption than what we were just discussing, the automation of a lot of white collar labor.

我认为，第一个真正会对世界产生影响的机器人，将是每辆配备特斯拉所谓AI四代硬件的特斯拉汽车。

I think the first robot the first robot that's really gonna impact the world is every Tesla car with what they call their AI four hardware.

从我的角度来看，公开的无干预行驶里程数是存在的。

Because from my perspective, there's a publicly sourced miles between disengagement.

所以你必须记住，对于特斯拉来说，特斯拉的无干预行驶里程数也会是一样的。

So you have to remember for Tesla, Tesla is gonna get the same miles between disengagements.

比如，如果你在火星上建了一座新城市，那里全是外观不同、街道布局各异的汽车，你把一辆特斯拉放进去，它在那座城市的无干预行驶里程数，也会和在其他任何城市一样。

Like, if you built a new city on Mars and it was populated by entirely different looking cars and streets and everything, you could drop a Tesla in that city, and it would have the same miles between disengagements that it gets in any other city.

而像小马智行这样的系统则是有地理围栏限制的。

Whereas something like Weibo is geofenced.

我们实际上只在那些拥有良好网格道路和良好天气等条件的城市中使用它。

We're really only using it in cities that have, like, nice grids and good weather, etcetera, etcetera, etcetera.

通过观察不同版本FSD的众包数据中的无干预行驶里程数，我清楚地看到，当他们切换到12.3版本时——这本质上是完全基于深度学习的系统，我认为几乎消除了所有人工编写代码——进步的速度发生了显著变化。

It is clear to me looking at the crowdsourced data, miles between disengagements with different versions of FSD, that when they cut over to twelve three, which is effectively all deep learning, I think eliminated almost all human code, something dramatic changed in the rate of progress.

然后当他们切换到12.5版本时，这个版本在AI四代硬件上运行效果最佳，也就是以前被称为HW四代的硬件。

And then when they cut over to 12.5, which runs best on the AI four, which used to be called the HW four.

这其实就是特斯拉的本地计算机。

It's just like the local computer of the Tesla.

现在它正在推广到AI3，这是另一个跃升。

It is now rolling out to AI three, was another step function.

根据同样的扩展规律，这些跃升性的改进是在远少于特斯拉现在在奥斯汀超级工厂数据中心公开部署的算力下实现的。

And these, going to that same scaling law, those step function improvements were made with a fraction of the compute that Tesla is now installing publicly in their data center at the Gigafactory in Austin.

他们实际上已经申请了一些专利——有时作为投资者，我希望他们别申请这么多专利——但这些专利在数据中心冷却方面非常创新，与他们在做的工作相关，据公开消息称，他们将部署超过5万台H100或H200芯片。

And they've actually filed sometimes I wish they as an investor, I wish they couldn't file so many of these patents, but they've filed some really innovative patents for data center cooling related to what they're doing with what looks I think it's been publicly said it's gonna be, you know, over 50,000 h one hundreds or h two hundreds.

FSD现在遵循同样的扩展规律，甚至可以说遵循更快的扩展规律，因为他们需要追赶GPT已经走过的路。

FSD is now on the same scaling law and arguably on a faster scaling law because they have a lot of catch up to do that GPTs have been on.

所以我认为12.5版本就像GPT-3，它能稳定地在大多数地方自动驾驶而无需人工干预。

So I think 12.5 is like GPT three, and it can consistently drive me most places with no interventions.

我是个季节性驾驶员。

I'm a seasonal driver.

我只在夏天开我的特斯拉。

I really only drive my Tesla in the summer.

实际上，我妻子贝基开车比我多，因为她比我更喜欢开车。

And, actually, my wife, Becky, tends to do most of the driving because she likes it more than me.

所以我们基本上能看到一种季节性的变化。

So we kinda get, like, a seasonal look.

几乎每到五月，我们就会检查一次进展。

It's almost like every May, we check-in.

一直以来，进步都是持续不断的。

There was just always continuous progress.

今年，当我们启用12.3版本时，感觉就像自从我第一次拥有那辆特斯拉以来，过去十年的所有进步都浓缩在了这一个版本里。

This year, it's like when we turned on 12.3, it was like all of the progress over the last ten years was in that one release from the first time I had that Tesla.

当我从12.3升级到12.5时，又出现了同样的情况。

And then we had that again when I went 12.3 to 12.5.

他们目前的算力大概处于GPT-2级别。

And they're at probably a GPT-two level of compute.

我认为他们将迅速达到GPT-4.5级别的算力，这意味着按照这些数量级，你将很快获得百倍的提升。

Think they're going to go really fast to GPT-4.5 compute, which means you're going to get, using these orders of magnitude, you're going to get a 100x improvement really fast.

所以我认为，那些一直持怀疑态度的人。

So I think there's all these people who have been skeptical.

他们都等着彻底丢脸。

They're all in for abject humiliation.

他们就是如此。

They just are.

而且与GPT-2不同，只有特斯拉能访问基于行驶里程的视觉训练数据集。

And then unlike GPT-two, only Tesla has access to a visual trading dataset that is based on miles driven.

我们可以争论这个数据集比第二大训练数据集微博大了100倍、1000倍还是10000倍。

We could argue whether it's a 100 x, a thousand x, 10,000 x bigger than the second biggest trading dataset, which is Weibo.

所以人们会问：他们怎么赚钱呢？

So it's like people, oh, how are they gonna make money?

但在自动驾驶领域，从我的角度看，他们欠的是YouTube、Meta的所有平台、开放互联网和X，而其他人却像用雅虎一样在尝试做这件事。

Well, it's like in this case, in the world of self driving, from my perspective, it's like they owed YouTube, they owed all of Meta's properties and the open Internet and X, and then other people are, like, trying to do it using Yahoo.

是的。

Yeah.

用雅虎。

Using Yahoo.

祝你好运。

Like, good luck.

谁会赢呢？

Like, who's gonna win?

当然，这种情况可能会改变。

Now, obviously, that could change.

保持谦逊很重要。

Important to have humility.

可能会出现算法上的突破，降低对这个交易数据集的依赖。

There may be an algorithmic breakthrough that reduces the importance of that trading data set.

毫无疑问，微博会试图用蛮力解决，不惜投入任何所需的资金来获取数据以竞争。

And for sure, Weibo is gonna try and brute force it and just throw whatever amount of dollars they need to get the data to compete.

他们采用了不同的方法，使用激光雷达。

And they have a different approach using LiDAR.

特斯拉没有。

Tesla doesn't.

我们走着瞧。

We'll see.

我觉得这并不是板上钉钉的事。

Like, I don't think it's a foregone conclusion.

关于未来，没有什么是确定的。

Nothing about the future is certain.

但只要我看看十二点五在AI四硬件上的表现，再想想它是在多么少的算力下训练出来的，而他们在奥斯汀正在用现有技术搭建的超级集群，我认为我们会直接跳过12.5到GPT-2。

But just if I look at how amazing twelve dot five is on AI four hardware and think about the tiny amount of compute that that was trained on and the mega cluster that they are standing up in Austin using known techniques, We're gonna skip, I think, 12 dot five to GPT-two.

我们会迅速跳到GPT-4。

We're gonna skip really quickly to GPT-four.

我知道Weibo肯定会加强它。

And then look, know, I'm sure Weibo will reinforce it.

可能会有算法上的突破，让其他人也能参与进来。

There may be algorithmic breakthroughs such that there are other people.

我们走着瞧。

We'll see.

但另一件大事是使用大语言模型来实现全自动驾驶。

But then the other big thing is just using an LLM for FSD.

X 上最好的关注者之一是范博士。

One of the best followers at X is Doctor.

他是英伟达机器人部门的负责人。

Jim Fan, who's NVIDIA's head of robotics.

他发布了很多帖子，谈到他和埃隆在 X 上的互动非常有趣。

And he's had a lot of posts about how there's a fascinating shape between him and Elon on X.

人工智能在 X 上发生的程度真是惊人。

It is amazing the extent to which AI happens on X.

谷歌的 Jax 团队和 Meta 的 PyTorch 团队爆发了一场激烈的争执，甚至闹到了 MAMAF。

The Jax team at Google and the PyTorch team on Meta got into this bitter fight, and it went to MAMAF.

哪个框架更适合 MAMAF？

Which framework was better for MAMAF?

最终，两个实验室的负责人不得不在 X 上公开出面和解。

Eventually, the heads of each lab had to step in publicly on X and make peace.

但天啊，真的。

But like, wow.

你知道吗，仅仅关注这场争论，你就能学到很多东西。

You know, you learn so much just following that fight.

每个AI研究员都在X上活跃。

And like every AI researcher is active on x.

AI在X上发生。

AI happens on x.

这是一个绝佳的平台来使用它。

And it's such a great forum for for using it.

但吉姆·法耶特与埃隆有过一次有趣的交流，吉姆·法耶特谈到LLM如何大幅改善FSD。

But Jim Fayette had this fascinating exchange with Elon where Jim Fayette talked about how LLMs could massively improve FSD.

埃隆回复说：是的。

And Elon replied, yes.

只有两种数据来源能够无限扩展：合成数据和真实世界视频。

The only two data sources that will scale infinitely are synthetic data and real world video.

我觉得这很有趣。

And I thought that was interesting.

然后这就引出了我认为的、我刚才描述的特斯拉自动驾驶未来观点可能出错的最大风险。

And then that goes to, I think, maybe the biggest risk in which this view that I just described of Tesla's autonomous future is wrong.

那就是，合成视频数据是否能像合成数据那样被使用。

It's just if synthetic video data can be used in the same way that synthetic data can be.

我们知道合成文本数据是有效的。

We know that synthetic written data works.

但我们不知道合成视频数据是否有效。

We don't know if synthetic video data works.

没人知道。

Nobody knows.

显然，监管机构的要求非常高。

And obviously, there's a very high bar for regulator.

我认为大概是，我忘了具体数字，全球每年有五万或十万人都死于车祸。

I think it's something like, I forget whatever it is, like fifty thousand or a hundred thousand people die in car crashes every year globally.

甚至可能达到一百万。

It might even be a million.

显然，我们可以用人工智能大幅降低这一数字，但人们对人工智能造成的交通事故的容忍度远低于对人类造成的交通事故。

Obviously, we could take that down dramatically using AI, but we're much less willing to tolerate traffic fatal accidents from AIs than humans.

就是这样。

That is what it is.

所以，这将会受到严格监管。

So, you know, it's gonna be heavily regulated.

但范博士提出，LLibs之所以能真正帮助实现全自动驾驶，原因如下。

But doctor Jim Fan posited that the reason LLibs are gonna be able to really help with FSD is because of the following.

这是我对于一些从事这些问题研究的人的看法，以我相对较低的智商来理解的方式。

This is the way my relative to some of the people working on these problems, my comparatively low IQ brain conceptualizes it.

任何基于真实世界数据训练过的系统，都知道在那种真实情境下，一个优秀的人类司机该怎么做。

Anything that's been trained on real world data just knows what to do, what a really good human driver would do in that real world situation.

如果遇到全新情况，它可能就不知道该如何应对。

If there's a novel situation, it may not know what to do.

而从我的角度来看，大语言模型正是在这里能真正发挥作用的地方。

And that's where, from my perspective, the LLM can really help.

因为GPT-4的一个涌现特性是——我们或许可以争论这究竟是真正的涌现特性，还是仅仅属于上下文学习——它具备所谓的世界模型。

Because one of the emergent properties of GPT-four, and we could debate whether or not it actually is an emergent property or just in context learning, It has what's called a world model.

这意味着，我相信你了解这一点：如果你问GPT-3，‘如果你把一瓶香槟倒过来放，再在上面放一个涂满肥皂的篮球，会发生什么？’

And that means, I'm sure you know this, but if you ask GPT-three, hey, what happens if you stand a champagne bottle upside down and put like a basketball covered in soap on top of it?

GPT-3完全不知道答案。

GPT-three, no idea.

GPT-4通常能正确回答这类问题。

GPT-four will often get questions like that right.

我其实应该去试试看它是否能正确回答这个问题。

I should actually see if he gets that exact question right.

一个三岁的小孩会说：‘这会掉下来的。’

A three year old human will say, That's going to fall.

香槟瓶会摔碎的。

The champagne bottle is going to shatter.