开源能否阻止AI权力集中？

人工智能越来越强大，但构建它所需的资源却越来越集中。

AI is becoming more powerful, but the resources needed to build it are increasingly concentrated.

这种情况非得一直持续下去吗？

Does it have to stay that way?

MTS主持人Sophia Dew前往旧金山参加开源AI峰会，向AI全栈的研究人员和创始人提问：开源能否为人工智能创造一个更加去中心化的未来。

MTS host Sophia Dew heads to the Open Source AI Summit in San Francisco to ask researchers and founders across the AI stack whether open source can create a more distributed future for AI.

《Attention Is All You Need》这篇里程碑式论文的共同作者Lukasz Kaiser认为，当前的资源集中现象可能是现有技术的一种特性，而非人工智能发展的必然趋势。

Lukasz Kaiser, co author of the landmark Attention Is All You Need paper, argues that today's concentration may be a property of our current technology, not an inevitable feature of AI.

Transformer 模型依赖海量的数据和算力，但它们问世还不到十年，下一个突破可能会彻底改变这种经济格局。

Transformers thrive on enormous amounts of data and compute, but they're less than a decade old, and the next breakthrough could change those economics.

从开源模型和算力获取，到企业自主掌控智能，Sophia 探讨了 AI 权力究竟集中在哪里、开放性能带来哪些改变，以及哪些问题仍未得到解决。

From open models and access to compute to companies owning their own intelligence, Sophia explores where AI power is actually concentrating, what openness can change, and what it still leaves unsolved.

和我连线的是 Lukasz Kaiser，他是一位 AI 研究员，也是《Attention Is All You Need》这篇论文的合著者。正是这篇论文引入了 Transformer，彻底改变了现代 AI 的发展轨迹。

Joining me is Lukasz Kaiser, an AI researcher and the co author of Attention Is All You Need, the paper that introduced the transformer that completely changed modern day AI.

那么，现代 AI 的现状如何？当前的权力又集中在哪些地方呢？

Where is modern day AI and where is power currently concentrating?

去年年底到今年年初，有一个非常重要的时刻，就是编程代理开始真正发挥作用了。

There was a very important moment sometime around end of last year, start of this year, where the coding agents started to work.

这种AI的本质是，你需要在非常昂贵的数据中心上训练它，并且需要大量的数据。

The nature of this AI is that you need to train it on very expensive data centers and on a lot of data.

所以大公司来做这件事，然后他们收费，对吧？

So there's big companies that do this, and then they charge for it, right?

这就导致了集中化，因为在我看来，即使是现在的模型也还不够聪明。

That concentrates it in the sense to get the, like, to me, even the current models are not smart enough.

所以还有可能变得更聪明。

So there still could be much smarter.

可能需要一些研究上的突破，才能用更少的数据让它们变得更聪明。

There may be some research breakthroughs that are needed to make them smarter with less data.

但研究突破嘛，你知道，有时候会来，有时候不会，它们不像商业计划那样有保障。

But since research breakthroughs, know, sometimes they come, sometimes they don't, they're not like a business proposition.

但大公司觉得，好吧，没有研究突破也能做事，那就是不断加大规模，越来越大。

But the big companies are like, okay, but you can do things without research breakthrough, which is to go bigger, bigger, bigger.

现在，这非常令人专注。

Now, that is very concentrating.

你做得更大、更大，你需要数十亿美元。

You go bigger, bigger, you need billions of dollars.

你需要从互联网的每个角落抓取数据。

You need to scrape data from every corner of the internet.

所以当前的技术状态有点令人专注，但我们应该记住这只是当前状态。

So the current state of the technology is a bit concentrating, but we should remember that it's just the current state.

当然，变压器很棒，但它们甚至不到10年历史。

Sure, transformers are great, but they're not even 10 years old.

我确定会有其他东西。

I'm sure will be other things.

也许我们可以用更少的数据、更多样化的模型来实现训练。

Maybe we'll make it possible to train with less data, more diverse models.

就像变压器真的很好，如果你在整个人类互联网上训练它们，然后它们就相当聪明了。

Like transformers are really good if you train them on the whole Internet, and then they're reasonably smart.

但如果你试着说，你只是刚学到这个，那它们就变得很笨，对吧？

But if you try to say, you know, you just learned about this, then they're just stupid, right?

效果并不好。

It doesn't work that well.

但效果不好，可能是因为我们需要一个研究突破才能让它起作用。

But it doesn't work because maybe we need a research breakthrough that will make it work.

但Transformer必须包揽一切。

But the transformer needs to be everything.

这是个问题，但这是个研究问题，希望最终能解决。

And that's a problem, but it's a research problem that hopefully gets solved.

对于Transformer，从架构上看，你需要大量资源。

With the transformer, architecturally, you need a lot of resources.

作为公司或模型，你拥有的资源越多，你就越强大。

And the more resources you have as a company, as a model, the more powerful you can be.

但你觉得有没有一种算法突破或某种研究突破，能让小玩家真正有竞争力？

But do you think there is a type of algorithmic breakthrough or some type of research breakthrough that will allow smaller players to really compete?

嗯，我是说，我们知道它是存在的。

Well, I mean, we know it exists.

我们自己就是证明，对吧？

We are the proof, right?

人类并不是什么都擅长。

Humans are not experts in everything.

对。

Yeah.

但在某些领域，确实有专家。

But in the field, there are experts.

他们非常厉害，有时候比我们那些超大型模型还要强。

They're very good, sometimes better than our super huge models.

所以它是存在的。

So it exists.

我觉得我们还没有从技术上找到它。

I don't think we have found it technically.

它甚至可能在某个地方有注意力层，对吧？

It may even have attention layers somewhere, right?

也许是因为不仅仅是模型，还有损失函数、数据以及训练方式。

Maybe because it's not just the model, it's also the loss and the data and how it's trained.

就像有很多因素，所以我们真的不知道具体该看哪里。

Like, there's a lot of factors, so we don't really know exactly where to look.

这就是研究中的问题。

That's the thing in research.

但另一方面，我觉得越来越多的人在关注这个，我希望如此。

But on the other hand, I think more and more people are looking, I hope.

我觉得实验室有点不，嗯，不完全是。

I think the labs have a little bit no.

OpenAI，当我加入的时候，是一个非常研究型的实验室。

Open A, when I joined, was a very research lab.

现在它有点研究实验室，但也是一个非常大的公司，需要做产品之类的。

Now it's a research lab a bit, but it's also a very big company, like, that needs to do products and and stuff.

所以它对研究的关注就少了。

So it has less focus on researching.

但这给了开源运动一个机会。

And But this gives the chance to do open source movements.

这对学术界，对大学来说是个机会。

That's a chance for academia, to universities.

我最近也开始自己做一些事情，挺有意思的。

I recently started doing things on my own also, and it's funny.

所以我买了一块5090 RTX显卡。

So I bought a 5,090 RTX GPU.

它的算力比我们团队当年做Transformer研究用的那八块GPU机器还要强。

It has more power than the eight eight GPU machines we used as a team to do the Transformers research.

所以你可以买一块GPU，就拥有我们当年设计Transformer时所有的算力。

So you can buy and have one GPU that has all the power we used to design transformer.

所以，你知道的，你可以用它来做很多事情。

So, you know, so you can do things with it.

比如说，你没法训练一个大型语言模型，但你可以做研究和实验。

Like, you can't train a big LLM, but you can research and experiment on things.

对。

Mhmm.

我觉得很多人都应该这么做。

And I think many people should.

我相信那时候机器学习研究还有很多可以探索的。

I believe there is still much more in machine learning research then.

我最后一个问题是，是什么让你对AI的未来感到乐观，觉得它会朝着真正开放、广泛赋能每个人的方向发展？

My last question for you is what makes you feel optimistic that the future of AI is going in a direction that will be truly open and broadly empowering for everyone?

你看看这个世界，人类是世界上最神奇的计算机，对吧？

If you look at the world, humans are the most amazing computers in the world, right?

我们的大脑仍然是模型无法比拟的，而且我们不是通才，也没有一个所谓的人类超级大脑。

Our brains are still unmatched by the models, and we are not generalists, and there's not like one big brain of humanity.

我觉得从根本上来说，可能给定固定数量的数据，最好的学习方式就是有很多分布式的模型，它们各自很强，但组合起来更强。而且如果你看一些基础研究，确实有理由相信这一点。

I think fundamentally it might be that given a fixed amount of data, the best way to learn from it is to have a lot of distributed models that are strong on its own, but even stronger when they're And if you look at like some basic research, there are actually reasons to believe it.

就像我刚才说的集成学习之类的。

Like I was just talking about ensembles and things like that.

所以，是的，当你把整个互联网的数据喂给Transformer时，它们表现很棒，但我们知道还有更好的算法，它可以从更少的数据中学到东西，而且学得非常好。

So, yes, transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better, and it can learn from much smaller data and be amazing at the things it's learning.

我们就是证明。

We are the proof.

我们还没找到如何在计算机上完美实现这一点，但我们一定会找到的。

We have not found it yet, like, how to do this in computers very well, but we will.

我的意思是，机器学习的发展并没有在2020年停止。当然，因为它太出色了，大家都把注意力放在了大数据上。

I mean, progress in machine learning has not stopped in '20 Of course, because it's so good, everyone has focused on the big data.

但现在既然成本这么高，我们可能会重新把重点放在更基础的研究上。

But now that it's so expensive, maybe we will focus back more on more fundamental research.

是的，我完全相信这能行，只是需要研究和努力。

Yeah, I'm absolutely convinced it can work and it just needs research and effort.

我觉得很多人有时候会看当前AI的状态，然后有点悲观，觉得它就像，你知道的，只是大公司、大数据中心，一切都那么庞大，然后你订阅一个服务，却对它无能为力。

I think many people sometimes look at the current state of AI and get a little pessimistic that it's like, you know, it's just big companies, big data centers, everything's so big, and then you get a subscription to something you cannot do much about.

对我来说，这就像是，好吧，但这只是当前的状态。

And to me, it's like, okay, but that's just the current state.

你知道，这只是今年或明年的事。

You know, it's just this year or next year.

这是技术的一步，但它是朝着更有趣的方向迈出的一步，也许每个人都能拥有自己的模型。

It's a step in the technology, but it's a step towards something much more fun where you can have everyone can have their own model maybe.

我们将从更小的数据中学习。

We'll learn from much smaller data.

模型会像我们一样成为不同领域的专家，也许，你知道，也许一个会这样开玩笑，另一个会那样开玩笑。

The models will be experts in different domains like we are, and maybe, you know, maybe one will be making jokes this way and another another way.

就像现在，当你问一个大语言模型一个笑话时，它总是同样的，对吧？

It's like, currently, when you ask a big language model about a joke, it's always the same, right?

关于原子之类的。

Something about atoms.

这会改变，因为这是技术的结果，当研究跟上时，技术也会改变。

This will change because it's an outcome of a technology, and the technology will change when the research catches up.

而且，对于研究人员来说，这里也有巨大的机会去赶上这个潮流。

And also, for researchers, there's like just huge opportunity to catch up with this.

所以，你知道，别担心，这已经集中起来了，因为那是方便的做法，而且它展示了伟大的事情是可以做到的。

So, you know, like, just don't worry about, okay, it got concentrated because that was a convenient thing to do, and it showed that great things can be done.

那很好。

That's good.

但前方还有事情，我们只需要一些研究上的突破。

But there is things ahead that that we just need to, just need some research breakthroughs.

感谢收听本期a16z播客。

Thanks for listening to this episode of the a16z Podcast.

如果你喜欢这期节目，记得点赞、评论、订阅，给我们留下评价，并分享给你的朋友和家人。

If you liked this episode, be sure to like, comment, subscribe, leave us a rating or a review, and share it with your friends and family.

更多节目，请前往YouTube、Apple Podcast和Spotify。

For more episodes, go to YouTube, Apple Podcast, and Spotify.

在X上关注我们@a16z，并在Substack上订阅我们的a16z Substack。

Follow us on x at a sixteen z, and subscribe to our Substack at a sixteen z dot Substack dot com.

再次感谢你的收听，我们下期再见。

Thanks again for listening, and I'll see you in the next episode.

本信息仅供教育用途，不构成任何买入、持有或卖出投资或金融产品的建议。

This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product.

本播客由第三方制作，可能包含付费推广广告、其他公司提及以及与a16z无关的个人。

This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with a16z.

此类广告、公司和个人均未获得Ah Capital Management LLC、a16z或其任何关联方的认可。

Such advertisements, companies, and individuals are not endorsed by Ah Capital Management LLC, a16z, or any of its affiliates.

信息来源于发布之日被认为可靠的来源，但a16z不保证其准确性。

Information is from sources deemed reliable on the date of publication, but a16z does not guarantee its accuracy.