Groq首席执行官乔纳森·罗斯 - 下一代AI硬件

这就是我的优势。

And that's my advantage.

现在来为我写软件吧。

And now come write software for me.

来为我构建网络吧。

Come build networking around me.

来为我构建一个系统吧。

Come build a system for me.

来把你们所有的框架都带过来。

Come bring all your frameworks.

但没人愿意这么做，因为成本太高，而回报太小。

And no one wants to do that because it's just too high of a cost for too little of a game.

在我们的情况下，我们只是让它完全兼容PyTorch，因为所有人都在用PyTorch开发。

In our case, what we did was we just made it completely compatible with PyTorch, which is what everyone develops in.

所以，你知道，PyTorch模型可以直接在我们的硬件上运行。

So, you know, the PyTorch model, it just works on our hardware.

不需要任何努力。

There's no effort required.

而且我们的 API 与 OpenAI 兼容，你只需将指向改为 Grok 即可。

And also, our API is compatible with OpenAI, so you just change to point to Grok.

所以基本上，人们已经写好了这些代码。

So basically, people have already written this code.

它可以直接运行，因此他们不需要重新编写或做其他事情。

It just works, so they don't have to redo it or something.

所以你实际上可以立即吸引开发者加入。

So you've kind of already can bring in the developers immediately.

没错。

Exactly.

所以我们火了。

So we went viral.

实际上，现在已经快一个月了。

Actually, it's been almost a month right now.

我们已经有七万名开发者。

We already have 70,000 developers.

相比之下，英伟达花了大约七年时间才达到十万开发者。

For comparison, it took NVIDIA about, I think it was seven years to get to 100,000 developers.

我们预计在七周内就能达到这个数字。

We're on track to get there in seven weeks.

哇。

Wow.

苹果通常不会参与关于生成式AI的这些讨论，但也许你认为，他们自研芯片的劣势，反而可能成为优势，或者你认为这是劣势？

It doesn't seem like Apple is usually in these conversations about generative AI, but that disadvantage, maybe you think it's advantage or maybe you think it's a disadvantage that they design their own chips.

你对苹果持乐观还是悲观态度？

Are you like bullish or bearish on Apple?

我持乐观态度，但正因为我持悲观态度。

I'm bullish, but because I'm bearish.

我认为苹果落后其他人太多，这反而会促使他们做出更明智的决策。

I think Apple is so far behind everyone else, it's going to embolden them to make some smart decisions.

他们实际上会与其他公司合作。

They're actually gonna partner with others.

他们会做一些事情。

They're gonna do things.

而我认为，谷歌面临的挑战最大，因为它们领先其他公司太多，以至于难以意识到这一点。

Whereas, I think Google has it the hardest because Google's so far ahead of most others that it's hard for them to recognize.

他们有自己的芯片。

They have their own chips.

他们有自己的软件。

They have their own software.

我的意思是，他们写了《Attention Is All You Need》这篇论文。

I mean, they wrote the attention is all you need paper.

他们领先太多了。

They're so far ahead.

他们没有意识到的是，他们的策略已经失效，需要采取其他措施。

And what they're not realizing is that their strategy isn't working and they need to do some other things.

我认为其他一些超大规模云服务商也是如此。

I would say same for some of the other hyperscalers.

坦白说，所有这些公司都一样。

Frankly, all of them.

这听起来可能有点奇怪。

This is going to sound a little weird.

我认为苹果拥有最大的优势，因为他们可能是唯一意识到自己尚未完全掌控局面的公司。

I think Apple has the best advantage because they're probably the only ones who realize that they haven't locked it down.

我认为微软、Meta 和谷歌都认为，基于他们目前的行动和对话，自己已经是赢家了。

I think Microsoft, I think Meta, I think Google, all think that they're the winners at this point based on what they're doing, based on conversations.

但我觉得亚马逊意识到自己稍微落后了，我认为这会给他们带来更多的灵活性。

But I would say Amazon realizes they're a little behind, and I think that's going to give them a little more flexibility.

这种现象你经常能看到。

And you see this all the time.

有时候，落后的那方反而能后来居上，因为他们并不觉得自己的做法有多么珍贵。

The folks who are the most behind sometimes end up getting ahead of everyone else because they don't find what they're doing that precious.

苹果刚刚解散了他们的自动驾驶团队，并表示你们现在转去做生成式AI了。

Apple just canned their self driving team and said, you are now working on generative AI.

过去，创新最大的开支是工程师的工资。

In the past, the biggest expense for innovation was salaries of engineers.

而未来，最大的开支可能是硬件和基础设施。

And potentially in the future, the biggest expense is hardware plus infrastructure.

这将如何改变这个行业？

How does that change the industry?

这正是Grok启动的部分原因，因为我们希望在AI时代保护人类的自主性。

This is a little bit of why Grok got started because we want to preserve human agency in the age of AI.

担忧在于，如果一小群人掌握了所有的计算资源，他们就会拥有全部的发言权。

And the concern is that if a small group of people has all of the compute, then they will have all to say.

我们希望确保每个人都能获得计算资源。

We wanna make sure that everyone gets access to compute.

这是一个真正的担忧，因为通过为员工、合作伙伴以及其他所有人提供更多的计算能力，你可以让他们变得更加高效。

This is a real concern because you will be able to make your employees and your partners and everyone else more efficient by giving them more compute.

这跟搜索不一样。

This is not like search.

搜索是一种信息时代的科技，你建立索引，然后从中检索信息。

With search, which is an information age technology, you build your index and then that index is retrieved on.

你可以通过在索引中更深地搜索来略微提升质量，但这在质量上并不是颠覆性的改变。

You can improve the quality a little bit by searching deeper in the index, but it's not a game changer in terms of quality.

你为所有人建立这个索引，所有人都使用相同的索引。

You're building that index for everyone and they all get the same index.

现在有了大语言模型，你可能也是为所有人构建这个模型，因为构建成本太高了，但事实上，你提供的计算资源越多，结果就越好。

Now with LLMs, you're probably building that model for everyone because it costs so much to build, but actually the more compute you give, the better the results get.

这种关系有多线性？

How linear is that?

要获得两个额外的结果，你得提供100倍的算力吗？

To get two extra results, do you have to give a 100 x computer?

这到底是怎么运作的？

How does it work?

我不确定这一点是否已经得到充分证实，但让我换种方式说一下。

I don't know that that's well established yet, but let me also put it another way.

如果你要找一位顾问，而一位顾问比另一位好10%，你会只多付10%的费用吗？

If you were going to work with a consultant and one consultant was 10% better than this other consultant, would you only pay 10% more?

我会付超过10%的额外费用。

I'd pay more than 10% more.

对于认知类任务来说，这通常是成立的。

And that's generally true of cognitive tasks.

所以，即使成本线性增长得非常厉害，你还是会想要更好的结果，因为你很可能把很多赌注押在上面。

And so even if it gets super linearly more expensive, you're gonna want that better result because you're probably banking a lot on it.

你会做出一些战略性的决策，因此它带来的回报可能会远超投入。

You're gonna be making some strategic decisions, and so it really can have an outsized return.

在芯片领域，市面上只有少数几种极其昂贵的芯片。

In the chips world, there's a small number of super expensive chips out there.

其中一些可能用在你的手机里，还有一些是像GPU之类的高端芯片，但绝大多数芯片数量庞大，其中很多甚至是在二三十年前制造的。

Some of them may be going to your phone, some of them are these, let's say, even the GPUs or whatever it might be that are out there, and then there's a very large number of chips, many of which were made even twenty, thirty years ago, that's the vast majority of things.

我们会看到类似的情况吗？比如，这些模型会被用于处理那些极其重要的问题，而我会在这里用超便宜的模型处理其他事情？或者你认为人们会怎么使用这些模型？

Are we gonna see something similar where you're gonna bifurcate these models will be used for these very, very important questions I can use, but then I'll use the super cheap models over here for other stuff, or how do you think people are gonna be using these things?

我确信，更困难的问题会分配更多的计算资源，而简单的问题则会使用较少的计算资源。

I definitely believe that more difficult questions will get more compute applied to them and easier questions will have less compute applied to them.

这种情况的运作方式很像下棋。

The way that this is going to work is a lot like playing chess.

当你下棋时，你可以下快棋，也可以下慢棋，慢棋需要深思熟虑，但耗时更长。

So when you play chess, you can play speed chess or you can play normal chess where you think a lot, but it takes a lot longer.

如果你按秒计费硬件成本（实际上就是这样），那么只要可能，你都会想下快棋。

And if you are charged by the second for the hardware, which is effectively what you are, then you're gonna wanna do speed chess wherever you can.

你会倾向于直接采用模型流畅输出的词元序列。

You're gonna wanna just go with the stream of consciousness output of the tokens.

你不会希望它深入思考，试图找出更好的答案。

You're not gonna wanna have it think really deeply and try and come up with a better answer.

此外，正如你所暗示的，还存在规模较小和较大的模型之间的区别。

There's also an element of, as you were alluding to, smaller and larger models.

这也会对成本和质量产生影响。

That has an effect on the cost and the quality as well.

但大模型的作用是真正提升了模型的直觉。

But what bigger models do is bigger models actually improve the intuition of the models.

即使你使用较小的模型，只要给它更多计算资源，让它深入搜索，有时也能得到更好的答案。

Even if you run a smaller model, if you run more compute on it, if you have it search deeper, you can end up sometimes getting better answers.

因此，对于某些任务，你可能更倾向于使用较小的模型，尤其是当你第一次尝试回答某个领域的问题时，其实并不需要大型模型。

And so for some tasks, you may want a smaller model, especially if it's an area you've never tried to answer something in before, you don't really need a large model.

你真正需要的是更多的计算周期来找出答案。

What you really need is a lot more of those compute cycles to figure out the answer.

但另一方面，模型越大，产生的幻觉就越少。

But then there's another side to this, which is the larger the model, the fewer hallucinations you get.

这是因为，你有没有过这样的经历：你去了某个地方，心想：天啊，导航把我带到这里了。

And the reason that that happens is, have you ever gone somewhere and you're like, gosh, the GPS took me here.

这根本不是我想去的地方。

This is not where I meant to go.

这是因为模型中包含了一些与驾驶无关的额外信息，比如关于周边环境和你打算去的地方，你会觉得这不对。

And it's because there's just some extra information unrelated to driving about the neighborhood, about where you're intending to go, and you're like, this is wrong.

模型越大，维度越高，出现这种错误的可能性就越低。

Well, the bigger the model, the higher dimensional, the harder it is to have one of those mistakes.

因此，在一段时间内，你可能会看到这些模型继续变大，以降低这些幻觉发生的概率。

And so you're probably for a while going to see these models continuing to get larger in order to reduce the probability of these hallucinations.

所以，对于任何需要减少幻觉的场景，这都是最佳方式。

And so for anything where you need to reduce hallucinations, that'll be the way.

但你也可以通过增加计算量来实现这一点。

But you can also do that by applying more compute.

但这需要投入更多的工作和努力。

It just takes work and effort.

为什么增加计算量能减少幻觉呢？

Why does the compute reduce the hallucinations?

我给你举个例子。

I'll give you an example.

我接下来要说的是哪个词

What is the next word that I'm about to

说。

Say.

好的。

Okay.

所有听这段话的人都在脑子里想到了同一个词。

Everyone listening to this had the same word in their head.

如果我让你补全这句话：双曲正切函数平方的二阶导数是

If I ask you to complete this sentence, the second derivative of the square of the hyperbolic tangent is

我不知道。

I don't know.

你不知道。

You don't know.

那你怎么会不知道呢？

So how is it that you don't know?

这不是人们经常谈论的事情。

It's not a common thing that people talk about all the time.

大型语言模型很像下棋。

Well, large language models are a lot like playing chess.

这里是一系列的词元，而不是一系列的走法。

There's a sequence of tokens instead of a sequence of moves.

在每一步，模型都会为所有可能的词元分配一个概率分布，然后按概率从高到低排序。

And what happens is at each point, the model assigns a probability distribution across all the potential tokens and then sorts them into highest probability first and lowest probability last.

每个听众都熟悉类似自动补全这样的功能。

Everyone listening is is familiar with, like, an autocomplete or something.

然后，通常算法会选择其中一个最高概率的词元。

And then, typically, the algorithm will pick one of the top tokens.

它并不总是选择概率最高的那个，因为有时候你并不想总是选择最显而易见的答案，但它会往这里走。

It doesn't always pick the top one because there's reasons why you don't wanna always just pick the most obvious answer, but it goes up here.

当你这么做时，就像下棋时直接下出第一个冒出来的招数。

Now, when you do that, it's like playing the first move that comes to mind when you're playing chess.

但如果你仔细想想，模拟一下其中一盘棋，你最终会想出更好的着法。

But if you think about it a little bit and you play out one of those games a little bit, you end up coming up with better moves.

这就像AlphaGo第二局中的肩冲手。

This is like the shoulder hit in the second game of AlphaGo.

那其实是一个非常低概率的着法。

It was actually a very low ranked move.

在一万盘棋中才出现一次，之所以能下出这步，是因为运行它的TPU有足够的算力进行更深入的搜索。

It was played one in 10,000 games, and it was only because the TPUs that it was running on had enough compute to go deeper and find it that it played it out.

这就是算力发挥作用的地方。

That's where the compute comes in.

对。

Yeah.

因为你能够探索更深的搜索空间。

Because you can search a deeper space.

对于这些词元来说，情况也是一样的。

The same is true of these tokens.

如果我问你，双曲正切函数平方的二阶导数是什么，并强迫你给出答案，你会胡言乱语。

If I ask you, what is the second derivative of the square of the hyperbolic tangent and I force you to give an answer, you'll give me gobbledygook.

但如果你能回头尝试大量其他可能性，突然间，其中一种就会开始变得合理。

But if you can go back and try a whole bunch of alternatives, then all of a sudden, one of them is gonna start to make sense.

这就像当你听到一个合理答案时的感觉，比如如果我告诉你答案，你会说，听起来没错。

And sort of like when you hear something that makes sense, like if I told you the answer, you'd say, that sounds right.

你可能并不真正懂，但听起来就是对的。

You may not know, but it sounds right.

而这种辨别好答案的能力，正是你进行搜索所必需的，因此这些模型在直觉判断方面非常出色，再叠加搜索机制，它们的表现就会更好。

And that ability to detect a good answer is really what you need in order to be able to search, and so these models are very good at that sort of intuitive part, and then you layer on that search part and they get better.

这叫做束搜索。

It's called beam search.

这是一种技术。

That's the technique.

现在你的芯片是在美国制造的。

Now your chips are built in The United States.

能详细说说其中的经济优势吗？你们为什么要这么做？

Walk us through the economic advantage, why are you doing that?

不只是因为你是爱国者。

Besides just the fact that you're a patriot.

当时我们做出这个决定是出于很多原因。

At the time, we made the decision for a lot of reasons.

其中一个原因是当时对台湾有些担忧。

One was there was a little bit of concern about Taiwan at the time.

但这并不是最主要的因素。

That wasn't the top reason that factored in.

另一个原因是，我们能够组建一个更积极的团队，因为他们更渴望与我们合作。

The other was we were able to get a better team because they were hungrier for the business to work with us.

我们与格罗方德合作，他们在纽约州北部为我们制造芯片。

We work with GlobalFoundries and they fab our chips in Upstate New York.

芯片的封装则在加拿大完成。

They're packaged in Canada.

我们也在美国制造我们的系统。

And we build our systems in The US as well.

我们正努力打造一个完整的北美供应链。

And we're trying to do this fully North American supply chain.

这里的优点在于，为什么我会建议其他人也这么做，尤其是当你刚开始起步、试图成为一家成功企业时，这并不是关于在出现问题时提供解决方案，而是因为事情可能会发生；如果我与你合作，我就无需担心这种可能性。

And the advantage here and why I would recommend that everyone else do this, especially if you're starting to come up and trying to become a successful entity, it's not about being the solution if something happens, it's something could happen And if I work with you, then I don't have to worry about that eventuality.

我降低了风险。

I reduce my risk.

因为大家都处于相同的时间区，所以更方便。

It's easier because everyone's in the same time zones.

你可以更快地迭代。

You can iterate faster.

在某些情况下，比如从1到n，把事情交给相隔一万英里远的地方做更有意义，但从0到1的阶段，你肯定希望尽可能靠近。

Some ways, like a one to n makes a lot more sense to do something that's 10,000 miles away, but a zero to one, you're gonna want it as close as possible.

此外，目前还存在地缘政治方面的担忧。

On top of that, there is an element right now of concern geopolitically.

当时情况并非如此。

This wasn't the case at the time.

人们在这里下的是象棋，而大多数供应链都横跨众多国家。

People are playing chess here on this, and most supply chains are stretched across so many different countries.

这些国家中的任何一个都可能否决你的供应链，这是一个巨大的风险。

Any one of those countries could just veto your supply chain, and that's a huge risk.

此外，许多大型芯片设计公司似乎都在竞争争取台积电的制造产能，彼此竞相抬价或利用自身影响力争取名额，我推测这意味着台积电的产能有限，因此必然会有赢家和输家。

Also, seems like a lot of these big chip designers are competing essentially to get slots for TSMC to manufacture them, and so they're kind of like bidding up each other to get the slot or using their own leverage to get there, and I assume that means TSMC only has a certain amount of capacity, so it means there's going to be winners and losers there.

在某种程度上，尽管台积电对任何人来说都不是瓶颈。

To some extent, although TSMC is not the bottleneck for anyone.

真正限制我们的并不是GPU，或者在我们的情况下，也不是LPU本身。

It's not the GPU or, in our case, LPU itself that's the limiter.

真正受限的是这种叫做HBM的技术，HBM是用于GPU的内存。

It's this technology called HBM, and HBM is the memory that is used in GPUs.

对我来说，这是个新术语。

That's a new term for me.

HBM代表什么？

What does HBM stand for?

它代表高带宽内存。

It stands for high bandwidth memory.

全世界的HBM产量几乎都来自韩国。

Pretty much the entire world's supply of HBM comes out of Korea.

两大制造商是SK海力士和三星。

Two major manufacturers are SK Hynix and Samsung.

现在，美国的美光公司非常希望成为大规模生产HBM的厂商，但通常被认为位居第三。

Now Micron in The US would love to become a manufacturer of scale for that, but they're generally considered the distant number three.

因此，NVIDIA的做法为我们带来了可能是他们取得巨大成功的第三点：他们实际上是一个买方垄断，也就是垄断的反面。

And so what NVIDIA has done this brings us into probably a third thing that they've done that's made them very successful, is they're effectively a monopsony, which is the opposite of a monopoly.

他们不是单一的卖家，而是众多尖端零部件的单一买家。

Instead of being a single seller, you are a single buyer of a lot of the cutting edge parts.

所以他们可以与三星等公司签订长期协议。

So they can sign, like, these long term deals with Samsung and these other companies.

在SK海力士以及其他所有这些公司。

In SK Hynix and and all these others.

它们买下了全部的供应量。

And they buy up all of the supply.

但这不仅仅是HBM。

But it's not just the HBM.

还有被称为互连基板或CoWoS的东西，HBM就安装在上面，而这种东西的供应也很有限。

It's also this thing called an interposer or co ops, which is what the HBM goes on, which is a limited supply of.

英伟达是全球最大的超级电容器买家。

NVIDIA is the largest buyer of super capacitors in the world.

所以它们也牢牢掌控了这一领域。

And so they've got a lock on that too.

还有这么多其他东西。

And there's all of these things.

实际上，这挺有意思的。

And actually, it was interesting.

前几天有人告诉我，AMD下调了他们的出货量预测。

Someone noted the other day to me that AMD had revised their volume projections down.

这很奇怪，因为他们的需求其实更高。

And that's weird because their demand is higher.

因为他们根本搞不到这些东西。

Because they can't get this stuff.

他们根本搞不到这些东西。

They can't get this stuff.

所以他们实际上根本生产不出来。

So they can't actually produce any.

所以无论你是从AMD还是NVIDIA购买GPU，你实际上都是在购买三星或SK海力士的HBM。

So it doesn't matter whether you buy a GPU from AMD or Nvidia, you're really buying the HPM from Samsung or SK hynix.

从一开始，我们就做了一些非常不寻常的事情，并且基于此做出了实际的设计决策：我们彻底摒弃了所有那些高端技术，因为我们知道根本无法获得这些材料。

Of the things that we did that was very unusual from the beginning, and we made actual design decisions on this, we eliminated any of the exotic technology because we knew that we would never be able to get access to this stuff.

你希望我尽可能多地使用普通、容易获取的材料。

You wanted me to build as much commodity stuff as possible that's easy to access.

对。

Correct.

事实上，我们的下一代芯片，其中一个版本原本设计中包含了HBM。

In fact, our next generation chip, one version of it actually had HBM in the design.

我们确实花了一百万美元购买了HBM，因为你必须提前很久下单，而它本打算用于生产批次。

We actually bought a million dollars of HBM because you have to buy it way in advance, that and was gonna be part of the production run.

当我开始意识到你需要提前那么久采购时，发现它的价格远高于芯片本身。

And when I started seeing you have to buy it that far in advance, it's way more expensive than the chips are.

我当时就想，维托，不行。

And I'm just like, Vito, no.

我们把它删掉了。

We took it out.

它的成本和风险并没有带来相应的价值，但我们的架构非常独特。

It really didn't add that much for the cost and the risk, but we have a very unusual architecture.

GPU需要HPM。

GPUs require HPM.

它们是围绕HPM构建的。

They're built around HPM.

是的。

Yeah.

我的意思是，你的芯片采用的是14纳米技术，这已经落后好几代了，但在某些方面，这反而是个优势，而不是缺陷。

I mean, your chips are like 14 nanometer technology, which is several generations old, but in some ways that's a feature, not a bug.

还有另一个原因。

And here's the other one.

所有人都关注FLOPS，也就是每块芯片能提供的计算能力。

So everyone's focused on FLOPS, which is the amount of compute that each chip is capable of.

但实际上，限制因素往往是芯片之间的互连，而不是FLOPS。

But, actually, the limiter tends to be the interconnect between the chips, not the FLOPS.

如果你能实现更快的通信，那其实就没那么重要了。

If you can have faster communication, then it doesn't really matter.

对吧？

Right?

你可以直接把它们加起来。

You can just add them up.

要理解这一点，最好的方式是用汽车工厂来打个比方：为什么GPU在运行大语言模型时比我们的LPU慢得多。

And the best way to think about this, the reason that GPUs are so slow versus our LPUs when they're running a large language model, let's use a car factory's analogy.

如果你需要一百万平方英尺的装配线空间来生产汽车，但你只有十分之一大小的仓库，那么你只能搭建出十分之一的装配线，让汽车通过，然后把它们停在停车场里。

If you need a million square feet of assembly line space for the cars to be produced, but you only have a warehouse that's one tenth of that size, then what happens is you set up the one tenth of the assembly line that you can fit, you run the cars through, and then you park them in a parking lot.

然后你拆掉这条装配线。

You tear down the assembly line.

再搭建下一个十分之一的装配线，再次让汽车通过。

You set up the next one tenth and then run them through again.

这叫做批处理。

That's called batching.

这就是GPU所做的事。

That's what GPUs do.

这是因为它们在等待高带宽内存（HBM）提供数据。

That's because they're waiting for that HBM, that high bandwidth memory to feed them.

因为我们有同步互连，并且拥有数百个芯片，这实际上就像一条完整的生产线。

Because we have the synchronous interconnect and we have hundreds of chips, it's actually like that full assembly line.

所以一个令牌就像一辆汽车，它可以直接从头到尾运行，而无需等待内存加载。

And so a token is like the car and it just goes from beginning to end without ever having to wait for a memory load.

因此，我们不仅消除了对HBM的需求及其供应链问题，还让整个过程更快了。

And so not only did we get rid of the HBM and our supply chain issues, we actually made it faster.

现在大多数人担心的是，他们看到后会说：要实现这个需要792个芯片。

Now the concern that most people have, they look at it and they're like, you need 792 chips to do this.

而其他人只需要八个。

Others only need eight.

是的。

Yeah.

但它们的价格只有百分之一左右。

But they're one one hundredth of the price or whatever.

对吧？

Right?

但实际上，每个芯片只负责计算中非常小的一部分。

But actually, each chip is only doing a very small part of the computation.

因此它能非常迅速地继续下去。

So it then moves on very quickly.

这有点像说，天啊，这座工厂的成本真高。

And it's a little bit like saying, gosh, the factory costs so much.

从工厂里出来的汽车会比手工打造的汽车贵得多。

Cars that come out of it be much more expensive than hand built cars.

这是一种人们很难理解的直觉。

And that's an intuition thing that people are really struggling with.

现在很多人认为我们已经达到了摩尔定律的极限，或者正在接近这个极限。

Now a lot of people think we've reached the limits of Moore's Law or we're reaching those limits.

你同意吗？这对芯片开发和其他类似的技术有什么影响？

Do you agree, and how does that affect things like chip development and some of these other types of things?

摩尔定律是一个了不起的推测，我们都把它当作现实来遵循，并竭尽全力维持它的延续。

Moore's Law was an amazing suggestion that we all followed and treated as real and did heroics to keep going.

我认为我们应该稍微调整一下这个定律，让它仍然成立。

I think we should tweak the law a little bit so that it can still be true.

与其把它看作是通过缩小晶体管尺寸来容纳更多元件的经济定律，我们不如开始说三维空间中的单位体积，这样我们就能开始堆叠芯片，继续提升密度。

Instead of it being about shrinking the size of the transistor so you could fit more and being an economic law, we should start saying unit volume in three d of space so that now we can start stacking chips and continue to get that density.

一旦我们完全填满了立方体而不是二维空间，我们就会找到这个定律的下一个漏洞，以继续推进进展。

And then once we fully fill up cube instead of a two dimensional space, then we'll find the next loophole in that law to continue the progress.

但从功能上讲，这还没有结束。

But functionally it is not done.

发生变化的不再是芯片本身，而是芯片所在的封装方式，以便实现规模化。

And what's moved, instead of it being about the chip itself, it's become about the packaging that the chip is in, in order to scale that up.

这可以说是接下来每个人都在玩的游戏。

And that's sort of the next game that everyone is playing.

总的来说，作为这些大语言模型的消费者和使用者，它们的进步速度确实非常快。

Just in general, as a consumer and a user of these LMs, they do seem like they're improving at like a super fast rate.

你预计明年会发生什么？

What do you expect to happen, let's say, the next year?

今年年初我做了一些预测，我的首要预测是，到年底时，会出现一些实际应用，其中几乎不存在幻觉。

So I made some predictions at the start of this year, and my top prediction was that by the end of the year, there would be some deployments where there's effectively no hallucination.

幻觉总是会存在，因为你不可能做到完美。

There's always gonna be a hallucination because you can't be perfect.

在某些情况下，幻觉将不再成为问题，因为它会被解决得非常好。

Hallucinations, in some cases, won't be a thing because it will be solved so well.

并不是每个人都能接触到这种技术，但因为它会被某个人解决——可能是以更高的价格，或者因为他们在使用更多计算资源、更大的模型——所以它会被视为一个可解决的问题。

Not that everyone will have access to this, just that it will be considered a solvable problem because someone will have solved it for maybe a higher price, maybe whatever, because they're doing more compute, they've got a bigger model.

这将极大地改变现状。

That's gonna change things a lot.

我的意思是，即使今天，你也可以通过某种方式来定义问题。

I mean, already today, you could define the question in a way.

你可以找到合适的提示词，从而常常消除幻觉。

You could find the prompt so you can often eliminate the hallucination.

如果你是个优秀的提示词编写者，你可以大大减少幻觉。

If you're a good prompt writer, you can reduce hallucinations quite a bit.

很多人已经开始用大语言模型自动将提示重写为更好的提示，因为事实证明，大语言模型本身是非常出色的提示工程师。

A bunch of people have actually taken to rewriting the prompt into a better prompt and automatically doing that with LLMs because it turns out LLMs are really good prompt engineers themselves.

完美。

Perfect.

是的。

Yeah.

那么事情就这样了。

So then there you go.

你也可以把这个做得更好。

You can make that better as well.

这会带来延迟问题，因为每多一步操作，都会延迟给出答案。

Now this is a latency issue because every time you do another step, it delays giving an answer.

说到我们自己的书，我们的延迟要低得多。

Speaking of our own book, we are much lower latency.

我们非常推崇这类技术，尽管目前它们已经被广泛应用，并且带来了很大帮助。

We're a big fan of these sorts of techniques, but they are currently used and they help a lot.

事实上，有一种叫做反思的技术，你只需问模型：‘对于这个输出，你怎样才能做得更好？’

In fact, there's a technique called reflection where you just ask the model, hey, on this output, how could you have made it better?

很好，现在就去做吧。

Great, now do that.

哦，有意思。

Oh, interesting.

然后再做一次，再做一次，再做一次。

And then do it again, do it again, do it again.

通常的经验法则是，每三次反思相当于一代模型的提升，但这是指数级的。

And typically the rule of thumb is every three reflections is a generational model improvement, but it's to the power.

所以如果你想获得两代提升，就需要九次。

So if you wanna get two generations, it's nine.

如果你想获得三代提升，就需要二十七次。

If you wanna get three generations, it's 27.

这就是为什么你在许多展示前沿成果的论文中会看到里面有一个数字。

This is why you see in a lot of these papers showing state of the art results, they'll have a number in there.

我忘了他们怎么称呼它了，可能是几次提问之类的。

I forget what they refer to it as, but it's shots or whatever.

多少次迭代？

How many iterations?

多少次提问？

How many shots?

他们会运行大量实例，然后不断改进结果。

They do a whole bunch of instances, and then they just improve the results over and over again.

所以速度对此至关重要。

So speed really matters for that.

当你自己设计提示时，有没有什么方法让你今天获得了更好的结果？

When you're doing your own prompts, is there something you've done in a way to get better results today?

对人有效的方法，对大语言模型也有效。

Things that work with people work with LLMs.

如果你有几个员工，给他们明确的目标通常会非常有帮助。

So you have a few employees, giving a very clear objective is generally very helpful with your employees.

所以如果你说‘写一个故事’，它就会写一个故事，而这会让你失望。

So if you say, write a story, it'll write a story and that's gonna disappoint you.

但如果你说‘写一个令人兴奋的故事’，或者定义什么是英雄之旅——即故事中存在紧张感和障碍，到结尾时，你最初面临的紧张和障碍其实并不重要，而出现了一个新的、更重要的障碍，这就是英雄之旅的结构——然后你明确说‘现在创作一个英雄之旅’，它就会表现得好得多。

But if you say, write a story that is exciting, or if you define what a hero's journey is, which is something where there's tension and there's an obstacle, by the end of it, the tension and obstacle that you started with turns out isn't important, and there's a new one that's important, that's kind of the hero's journey arc, and define that and you say, now make a hero's journey, it'll do that much better.

但此外，先让它列出大纲，再让其将大纲转化为最终内容，这也会有很大帮助。

But then also just asking it to do an outline before it gets started and then having it turn that outline into output, that helps a lot.

始终思考：我能做些什么来让人更好地完成这件事？

Always think what could I do to get a human being to do this better?

这会有帮助。

And that'll help.

一个非常有效的方法是，让它想象一下答案会是什么样子。

One that helps a lot, asking it to imagine what an answer would look like.

突然间，答案的质量就会大大提高。

And all of a sudden, the answers get much better.

有意思。

Interesting.

我也得试着用在我合作的这些人身上。

I gotta try that with the humans I work with too.

没错。

Exactly.

我本来想说的是，我从大语言模型身上学到了这一点，然后我把这个方法应用到了人身上，效果非常好。

So what I was about to say is I learned that with LLMs, and then I back ported that to people, and that actually works very well.

现在你作为CEO，处于一个非常有趣的位置。

Now you're in a very interesting position as a CEO.

你的员工是全球最抢手、最受激烈争夺的人才。

Your employees are some of the most in demand, most aggressively recruited people in the world.

在这种情况下，你该如何经营公司？

How do you run a company when you're in that situation?

从一开始我们就一直面临这种情况，所以我们已经习以为常了。

We've always been in that situation from the start, so we're just used to that.

而且这又回到了汉密尔顿·赫尔默的七种竞争优势。

And again, it goes to Hamilton Helmer's seven powers.

如果你只关注经济因素，把它当成一种商品，认为只要多付钱就能留住人，那你会失败。

If you focus purely on economics and it's a commodity thing, and it's like, I'm gonna pay you more, you're gonna lose.

所以，我们招聘的任何一个人，我们都会尽量把薪资定得低于他们从其他地方拿到的最高报价。

So one thing is anyone that we hire, we always try and be lower than the highest offer that they get elsewhere.

因为否则我们就得不到足够的信号。

Because otherwise, we don't have enough signal.

你不想要一个唯利是图的人，你想要真正相信你所做事业的人。

You don't want the mercenary, you want the person who really believes in what you're doing.

没错。

Exactly.

接下来一点是，拥有顶尖人才时，人们通常更愿意留在顶尖人才身边。

The next thing up is by having really great talent, people generally wanna stick around really great talent.

我觉得每个人都认为自己的人才很出色。

I think everyone thinks that their talent is great.

我记得曾和一位CEO交谈，他特别自豪于自己团队的人才密度，但后来他有机会见到我们的一些人，他说：‘我以为我懂什么是人才密度了。’

I remember talking to the CEO who was particularly proud of their talent density, and then they had an opportunity to meet some of our people, and they're like, I thought I knew what talent density was.

现在我明白了。

Now I know.

当你只招聘非常优秀的人时，其他非常优秀的人也想和非常优秀的人共事。

When you just hire really amazing people, other really amazing people wanna work with other really amazing people.

要挖走他们真的很难。

It's really hard to pull them away.

过去七年左右，这些员工的市场薪资水平大幅上涨。

The market rate for these employees has gone up dramatically over the last, let's say, seven years.

每年可能都上涨了20%以上，而相比之下，普通软件工程师的薪资——至少在过去三年里——我认为已经下降了，但大多数与AI相关的工程师，即使他们愿意接受较低的薪资来为你工作，你仍然需要不时加薪以保持竞争力。

It's probably gone up over 20% a year every year, whereas like the average software engineer, at least in the last three years, I believe it's gone down in the last three years, most of these more AI related engineers, even if they're willing to take a reduced salary to come work for you, you still have to do some sort of raises and match it over time too.

另一点是我们不做AI。

The other thing is we don't do AI.

我们是卖铲子和铁锹的。

We're picks and shovels.

实际上，在Grok公司，真正做AI的人非常少。

We actually have very few people who actually do AI at Grok.

我们是一个赋能者。

We're an enabler.

你们更多是在和英伟达这样的公司竞争。

You're more competing with, like, the NVIDIA or these other types of places.

没错。

That's right.

所以我们这里确实有这些人，但其中一个机会是，当你身处其中并帮助所有人让这些技术运行起来时，你就能学到东西。

And so we do have people here, but one of the opportunities is when you're near this stuff and you're helping everyone get it to work, you get to learn.

所以，如果你作为一名软件工程师或硬件工程师加入，你将有机会以其他方式无法获得的方式与这些技术互动，因为我们实际上能与最顶尖的用户和客户合作。

And so if you come in as a software engineer or whatever or hardware engineer, you get to interface with this in a way that you wouldn't otherwise because we actually get to work with the best of the best as users and customers of our stuff.

我们可能比其他人更近距离地看到他们真正关心什么，以及他们的工程实践是什么样子。

We might actually have more of a front row seat to what matters to them and what that engineering looks like than a lot of others.

但我们自己并不亲自去做这些，因此这大大减轻了竞争压力。

But we're not the ones doing it ourselves, and so that takes a lot of the competitive pressure off.

好的，我们最后问两个问题。

Alright, two last questions we ask all of our guests.

第一个问题是：你相信什么样的阴谋论？

First is, what is a conspiracy theory that you believe?

我是世界上最不擅长相信阴谋论的人。

I am the world's worst conspiracy theory believer.

我原以为你在这方面还挺在行的。

I would have thought you'd be pretty good at it.

关于阴谋论，大多数狂热者往往能同时相信两件相互矛盾的事情。

The thing about conspiracy theories is most people who are conspiracy theory junkies, they can believe two things that are incongruent.

我本人和我们在Grok招聘时所看重的一点，是我们称之为‘现实指数’的东西。

One of the things that I do and that we hire for at Grok is what we call reality quotient.

我们有一整套提升现实指数的层级。

And we've got a whole bunch of levels of how you improve on reality quotient.

但最基本的是我们所说的‘可塑性思维’，也就是当事实发生变化时，你的想法也会随之改变。

But the start is what we call a malleable mindset, or when the facts change, your mind changes.

这与阴谋论格格不入，因为阴谋论一旦被证伪，你就会觉得：‘哦，我错了。’

That doesn't work well with conspiracy theories because conspiracy theories get debunked and then you're like, oh, I was wrong.

但我想说，我有一些奇怪的信念，而我们的一些奇怪信念竟然成真了。

But I would say it's more that I have some weird beliefs, and we have some weird beliefs that have come true.

我不确定是否存在什么阴谋论。

I don't know that there'd be conspiracies.

其中一个令人惊讶的是，我们当时觉得显而易见的是，推理将在市场中占据越来越重要的地位。

And one of them shockingly was that we thought it was obvious that inference would start to become a bigger part of the market.

所有人都觉得我们疯了。

Everyone thought that we were nuts.

他们觉得会是训练主导。

It's going be training.

但我们说，你花钱在训练上，却靠推理赚钱。

And we're like, But you spend money on training and you make money on inference.

当然，推理的规模会越来越大。

Of course, inference is going to get larger.

我不确定。

I don't know.

我对阴谋论不太在行。

I'm bad at conspiracy theories.

对不起。

I'm sorry.

那太好了。

Well, that's great.

我们问所有嘉宾的最后一个问题。

Last question we ask all of our guests.

你认为哪些普遍接受的智慧或建议其实是糟糕的建议？

What conventional wisdom or advice do you think is generally bad advice?

我不喜欢给人建议，因为人们不喜欢接受建议。

I hate giving advice because people don't like to take advice.

所以我要说的是，对我自己和我认识的其他人最有帮助的是尽量变得更加无畏。

And so what I will say is the thing that has been most advantageous for myself and others that I've known is to try and be more fearless.

问题不在于不够无畏，而在于你没有意识到自己已经变得恐惧，而这正是阻碍你的原因。

And the problem is not being more fearless, it's recognizing that you've become afraid and that's what's stopping you.

我经常参加一些会议，有人会说，我们不应该因为这个原因做这件事。

I'm in meetings all the time where someone says, we shouldn't do this for this reason.

但事实上，他们不想做是因为害怕。

But the reality is they don't wanna do it because they're afraid.

也有一群人完全不害怕，总是惹上麻烦。

There are also groups of people who have no fear and get in trouble all the time.

但你我经常接触的那些人，往往是比较胆怯的类型。

But the folks that you and I interact with a lot are the kind who are more afraid.

真的吗？

Is that true?

还是你觉得典型的创业者性格可能

Or you think maybe a typical founder personality might

创业者没那么害怕。

A founder is less afraid.

但我是说，工程领域，因为如果你犯了错，所以你会更谨慎，想把一切都做对。

But I mean, engineering, because if you make a mistake and so you're more cautious, you wanna get it all right.

这有点像纳西姆·塔勒布的观点，即高风险事物的风险价值其实并未被充分定价。

Sort of like a Nassim Talebism of the value of the risk isn't really priced in on high risk things.

因此，你真正应该追求的是更高风险的事物，因为生活中那些低风险事物背后真正潜藏的风险，其实也未被定价。

And so you should really be going after higher risk things because the low risk things in life, that real risk that's under there for everything isn't priced in.

我常举的一个例子是，每个人租用房产时都必须购买火灾保险，但没人需要购买疫情保险。

An example I like to give is everyone has to have fire insurance on any property that they rent or whatever, but no one needs pandemic insurance.

然而，在过去两百年里，普通上班族因疫情而无法上班的次数，远多于因火灾而无法上班的次数。

Yet over the last two hundred years, the average person working in an office building has been out of an office more often because of pandemics than because of fires for buildings.

你得去关注那些风险更高的事情，因为那样的话，风险与回报的比值会更好一些。

You got to look at things that are riskier and do those because then the price to value is a little better.

创业是你能做的最好、风险最低的选择之一，因为如果你在大公司，随时可能被裁员。

Startups are one of the best, lowest risk things you can do because if you're in a big company, you could be in a layoff.

你的成长速度不会那么快，还有其他各种因素。

You don't grow as fast, you know, all these other types of things.

所以，去找那些别人都害怕去做的事。

So, look for the things that everyone else is afraid of.

去做那些事，然后对那些别人都不害怕的东西，稍微保持一点警惕。

Go do those things, and then all the things that no one else is afraid of, be a little fearful of those.

我觉得这是个门格主义，这样你会好很多。

I think that's a mongerism, and then you'll be much better off.

这太棒了。

That's great.

感谢乔纳森·罗斯加入我们的《DAS世界》节目。

Thank you, Jonathan Ross, for joining us on World of DAS.

这真的非常有趣。

This has been really interesting.

顺便说一下，我在Twitter上关注了你，JonathanRoss321。

By the way, I follow you JonathanRoss321 three two one on Twitter.

我强烈建议我们的听众去和你互动。

I definitely encourage our listeners to engage with you there.

我学到了很多，这真的非常有意思。

I learned a ton, so this has been super interesting.

谢谢，谢谢邀请我。

Thanks, and thanks for having me.

我很感激。

I appreciate it.

如果你是个数据狂热爱好者，去

If you're a super data nerd, go

worldofdaas.com。

to worldofdaas.com.

就是daas，worldofdaas.com，注册我们的每周数据即服务简报。

That's daas, worldofdaas.com, and sign up for our weekly data as a service roundup newsletter.

谢谢收听。

Thanks for listening.

如果你喜欢这期节目，不妨给这个播客打分并留下评价。

If you enjoyed this show, consider reading this podcast and leaving a review.

想了解更多World of DAS，DAS是d-a-a-s，你可以在Spotify、Apple Podcasts或任何你收听播客的平台订阅，也可以前往YouTube观看视频。

For more World of DAS, and DAS is d a a s, you can subscribe on Spotify or Apple Podcasts or anywhere you get your podcasts, and also check out YouTube for videos.

你可以在Twitter上找到我，用户名是@Orin。

You can find me at Twitter at at Orin.

拼写是u-r-e-n，Orin，我们非常期待你的反馈。

That's a u r e n, Orin, And we'd love to hear from you.

《World of DAS》由Safegraft赞助播出。

World of DAS is brought to you by Safegraft.

Safegraft提供实体地点的地理空间数据。

Safegraft is geospatial data for physical places.

访问safegraft.com了解更多。

Check it out at safegraft.com.

以及由Flex Capital赞助。

And by Flex Capital.

Flex Capital投资于我们曾在《World of DAS》中讨论过的数据公司。

Flex Capital vests in data companies like those we talk about at World of DAS.

访问flexcapital.com了解更多。

Check it out at flexcapital.com.