Is AI video just a prequel? Runway's CEO thinks world models are next

大家好，欢迎回到《Equity》——TechCrunch旗下聚焦初创企业商业动态的王牌播客。

Hello, and welcome back to Equity TechCrunch's flagship podcast about the business of startups.

我是丽贝卡·贝兰，本期节目我们邀请行业专家，一起深入探讨科技界的一个热门趋势。

I'm Rebecca Bellan, and this is the episode where we bring on industry experts to help us explore a trend in the tech world and dive deep.

AI生成视频从新奇玩意儿迅速变成创作工具，快得就像眨一下眼，而最懂这一点的，莫过于Runway公司。

AI generated video has gone from novelty to creative tool in what feels like the blink of an eye, and nobody understands this better than Runway.

这家公司总部在纽约，但发布的模型实力远超其规模，已经能和谷歌、OpenAI这些资金雄厚的实验室一较高下。

They're a startup that's based in New York, and they're punching way above their weight in terms of their model releases, coming out with models that are competing with some of the most well funded labs like Google and OpenAI.

随着Runway不断升级视频生成模型，他们发现这些技术的应用远不止于创意领域。

As Runway scaled its video generation models, it learned that its applications are extending way beyond creativity.

他们正朝着通用世界模型迈进，应用场景涵盖游戏、机器人，甚至可能通向更广泛的智能形态。

They are advancing into general world models with applications in use cases like gaming, robotics, and perhaps a more generalized intelligence.

今天和我们聊这个话题的，是Runway的联合CEO兼联合创始人克里斯·瓦伦祖拉。

Here to talk about this today with us is Runway's co CEO and co founder, Chris Valenzuela.

克里斯，欢迎来到节目。

Chris, welcome to the show.

谢谢邀请我来。

Thank you for having me.

是的。

Yeah.

很高兴又能和你聊聊。

Really excited to chat to you again.

这周我们一直在交流。

We've been chatting all week.

我一直在和你的团队讨论Runway在做的事情。

I've been chatting with your team about what you guys are doing at Runway.

我想先提一下你最近说过的一句话：AI能帮好莱坞拍出50部电影，而不是砸一亿美元做一部大片。

But I wanted to kick things off by bringing up something you said recently, that AI could help Hollywood make 50 films instead of $100,000,000 blockbuster.

我知道，像这种吸引眼球的标题，容易让人误解，以为你们这些做AI的就想颠覆甚至取代好莱坞和电影制作。

Now I think with, you know, catchy headlines like that, people might misconstrue or they get attached to something like, oh, these AI guys are just trying to disrupt and displace, you know, Hollywood and filmmaking.

但我真想知道，你到底想表达什么？

But I wanna know what you're actually proposing.

对吧？

Right?

除了把电影制作当成内容流水线，更细致的观点到底是什么？

Like, what is the more nuanced argument than just treating filmmaking like a content factory?

是的。

Yeah.

这是个好问题。

That's a that's a good question.

我同意。

And I agree.

有时候，标题确实会显得更吸引眼球一些。

Sometimes, like, headlines do tend to be a bit more cake baity.

我是在一场会议的多种讨论背景下说那番话的，当然，这种说法更容易被拿来报道。

I said that in the context of a lot of other things within a conference, and I think, of course, that feels like much more interesting to publish.

这一点我也认同。

And it which which I agree.

其实这和我原本想表达的观点也没差多少。

It's actually not not that far from what I was actually, like, arguing.

说实话，我的理由很简单。

And and my reasoning here is kind of actually pretty simple, to be honest.

这跟过去几十年来媒体、电影，或者说内容整体的制作方式有关。

It has to do with the way media and, like, feature films or just content in general has been made for the last couple of decades.

整个流程充满了各种技术细节、程序、预算限制和时间压力。

And it's a process that's filled with a lot of technicisms and processes and budget constraints and time constraints.

这个过程很大程度上依赖于审批，依赖于说服对的人。

It's a process that's driven a lot by approvals, by convincing the right people.

我觉得整个行业，尤其是好莱坞和电影制作，处境挺艰难的，主要是因为流程太复杂，想通过审批、拍出东西实在太难了。

The industry itself, I would say, is in a tough spot, the Hollywood and and filmmaking is, mostly because it's good and just way too complex to approve and make anything.

我并不是第一个提出这种看法的人。

And and I'm not the only one who's argued this.

其实很多人都早就说过类似的话。

I think many people have said this before.

拍一部电影，至少得花上几百万，还得再花几百万去宣传。

You need, like, at least a couple millions just to, like, make a movie and then at least a couple more millions to, like, promote the movie.

所以，除非你有本事筹到五千万到一亿美元，不然想拍电影基本没戏。

And so unless you're willing and able to fundraise or convince someone to give you at least 50 to $100,000,000 to make a movie, then it's not probably gonna happen.

当然，这里面也有不少水分，这也不是人人都适合的选择，但总的来说，大多数人应该都认同这种看法。

And and, of course, there's deceptions, and and I'm not this is not a decision for everyone, but, like, I would say in the large part, I think most people agree with that sentiment.

你去洛杉矶走一圈，到处都能看到这种情况。

Just go to LA and you'll see that all over the place.

所以现在有趣的作品少、电影类型单一，部分原因就在这儿。

And the reason there's a lack of interesting projects or a lack of diversity of films is in part because of that.

对。

Yep.

我觉得现在出现的新变量是，当有一种技术能让你快速推进这些环节时，会发生什么。

What I think is happening now on the new ingredient in the mix is, well, what happens when you have a technology that allows you to fast track many of those stages and steps.

对吧？

Right?

这样能让你在预可视化和前期制作上快速迭代，当然也包括电影实际画面的制作。

So it will help you iterate way faster on pre visualization and preproduction, and, of course, on making the actual frames for the movie.

这种趋势现在其实已经能看到迹象了。

And that's already are seen today.

AI正在大规模应用于影视公司，也包括独立电影人。

Like, AI is getting deployed at scale in studios and also people and independent filmmakers.

如果你有一个真正出色的想法、一个好剧本，或者一个世界想听的故事，你很快就能把这部作品拍出来。

And and where you start to see kind of trajectory wise is a place where if you have a really good idea, if you have a really good script, if you have something the world wants to hear, you might be very soon in a position where you can tell that story.

而且制作成本只需要原来的一小部分——换作以前，这种做法在那个阶段简直让人觉得疯狂。

And you can tell it for a minimal fraction of the budget that otherwise will require to you something else that feels like probably at that stage, like, insane that we were, like, right that way.

对。

Yeah.

所以我的总结是，如果有五千万到一亿美元的预算，你或许能拍出上千部电影，甚至更多。

And so my my summary of that was like, well, yes, with hopefully, with $50,000,000 or $100,000,000, you're gonna be able to make a thousand films, like, or or hopefully many more.

真正的限制，不会再是技术问题。

Like, constraint would not be a technical one.

这将是一个关于故事讲述的问题。

It will be a storytelling one.

你有什么想讲的好故事吗？

Do you have something good to tell?

如果有，技术会首先帮你把它实现出来。

If you do, then, like, technology will help you do it in the first place.

是的。

Yeah.

你知道吗，很多人担心这只会带来更多低质的AI内容。

You know, I think that, like, the concern is among many people is that this will just create more AI slop.

对吧？

Right?

作为一名编剧，我在电影片场工作过，确实有种担忧。

And as someone who's a writer, I've worked on film sets, There's like a fear.

大家心里都有一种强烈的恐惧，觉得很多东西会因此消失。

There's a visceral fear that a lot of that will go away.

很多电影制作中那种协作的氛围都会消失。

A lot of that industry will go away, that collaborative nature of creating films.

另一方面，我不知道你有没有注意到，今天我收到了一个广告。

On the other hand, I'm not sure if you saw, I got an ad for this today.

有一部新的恐怖片要上映了，类似那种讽刺类型的恐怖片。

There's a new scary movie coming out, like this like the the satire film, scary movies.

《魔法保姆麦克菲》要出续集了。

There's a second Practical Magic.

《魔法保姆麦克菲》是我最爱的电影。

Practical Magic is my favorite movie.

但我真不一定想再看一遍。

I don't know that I need to see it again.

我的意思是，现在有这么多现成的IP不断被翻拍，而很多精彩的新故事却被搁置了，原因大概是行业机制觉得必须不断用老演员重拍老IP，这样才能保证投资回报。

My point is that there's all this existing IP that keeps getting produced, and other amazing stories are being sidelined because, I don't know, whatever industry mechanisms say that we need to keep producing existing IP with the same actors you've seen for thirty years, and that's the only way we're gonna get a return on our investment.

我期待像你这样的工具能改变这种状况。

I'm hopeful that tools like yours can change that.

不过在深入之前，我想先退一步说，Runway并不是唯一一家做这个领域的公司。

But, you know, before we go into it, I wanna take a step back because Runway, there are other companies in this space.

对吧？

Right?

竞争对手比如Opening Eyes，还有谷歌，都在做视频生成模型。

Is a competitor, opening eyes making video generation models, so is Google.

我觉得你们特别有意思的一点是，你们没有典型的硅谷背景。

And one thing that I find interesting about you guys is that you don't have that typical Silicon Valley background.

对吧？

Right?

你们是2018年一起创办了Runway。

You cofounded Runway in 2018.

你和联合创始人亚历杭德罗、阿纳斯塔西斯，都是纽约大学艺术学院出来的。

You and your cofounders, Alejandro and Anastasis, came out of NYU's art school program.

能聊聊这段经历吗？

Can you talk a little bit about that?

当然可以。

Sure.

对。

Yeah.

我很乐意谈一谈。

I'm happy to.

说到内容数量这个问题，其实我们以前就见过类似的情况。

Also, to to a point before, I think on on, like, the the quantity of content, I I think that we've seen that before.

有时候人们容易忘记这一点。

I think people sometimes forget.

这其实有个说法，叫斯图尔格定律。

This is actually there's a law for this called Sturgeon's Law.

我们看到的大部分内容质量都不高，只有很小一部分是优质的。

It's like the most of the things that we see are low quality, then small percent of it are good quality.

但我想说的是，这种现象以前就出现过。

But my point is that we've seen that before.

这个领域其实可以追溯到上世纪四十年代，甚至更早，但在2016到2018年我们刚开始做Runway这些想法的时候，深度学习和神经网络根本还没成气候。

The field goes back to the forties and maybe even before that, but deep learning and neural networks working weren't really a thing in 2016, 2017, and 2018 when we started working on some of these ideas of Runway.

所以当你谈到竞争对手和别人也在做类似的东西时，我们起步时根本没什么竞争对手，因为那时候连个像样的行业都还没有。

And so in a way, when you speak about competitors and others building here, when we started, there was no competitors because there was no industry altogether.

那时候我们根本找不到可以对标的人，也没法从别人那儿获得灵感，更谈不上觉得自己在和谁竞争。

And so there's no one we could, like, compare to or look or inspire or think that we were, like, competing with.

我觉得正因如此，我们才慢慢形成了这种独立的思维方式，也更清楚自己最初为什么要做这件事。

And I think that helped us shape a little bit of this independence and unique way of thinking about why we do this in the first place.

你说得对。

And you're right.

我们一开始就是一群艺术家和工程师。

We started this as, I would say, artists and engineers.

我们三个人的背景都是艺术和科学的结合。

We come from a background, the three of us, from a background that mixes art and science.

要么是搞工程的对艺术感兴趣，要么是搞艺术的想变成工程师。

We were either engineers with an interest in art or an artist who wants to become an engineer.

说实话，很长一段时间里，我们总觉得格格不入，在很多场合都找不到自己的位置，因为我们很难被归类。

And for a long time, if if I'm honest, like, it was kind of strange and weird, and we didn't feel at place in in many different, like, times and moments because it was hard to classify us.

你们在艺术学院上学，可又是软件工程师。

What are you guys were in art school, but you're software engineers.

你们一边发表人工智能的论文，一边谈论印象派。

So you're publishing papers on AI, but also you're, like, speaking about, like, impressionism.

这根本说不通。

Like, it just doesn't fit.

但我想，正因如此，时间久了反而成了我们最大的优势——没错，就是因为它不合常规。

And and I think that in a way has become maybe our biggest strength over time is that, yes, it doesn't fit.

这正是关键所在。

That's exactly the point.

对。

Yeah.

如果你真想做出优秀的产品和技术，想掌握自己的命运，那就别被标签束缚，干脆自己定义一个标签。

The point is that if you wanna make good products and good technologies and you wanna create your own destiny, maybe don't live your life by labels, but, like, create on your own label.

而且我觉得，随着时间推移，公司本身已经体现了这种文化。

And I and I think we've the company itself has has embodied that culture over time.

所以我们做这一切，是出于真心的热爱，这正是我们多年来一直梦想要做的事。

And so it comes from from a place of care, what we this is what we have dreamt about doing for years.

能把它当成一份工作来做，感觉就像实现了梦想一样。

And so the fact that we can do it as a job feels also like a dream.

是的。

Yeah.

还有一件事我觉得很重要，你们总部设在纽约。

And, you know, another thing that I think is important to point out is that you're based in New York, headquartered in New York.

你们当然在全球都有办公室，但纽约是根基。

You have offices everywhere, of course, but New York strong.

作为纽约人，我特别欣赏这一点。

Really appreciate that as a New Yorker.

对我来说，这总是特别有意思。

Always fun for me.

这意味着我可以去你们办公室找你，我确实去过了。

It means I can visit you in the office, which I did.

顺便说一句，你们的办公室真不错。

Lovely office, by the way.

谢谢。

Thank you.

我逛了一圈，观察大家怎么工作，发现整个氛围特别平等。

One thing I noticed while I was walking around, looking at the way people work is everyone seems to be I mean, it seems really flat.

对吧？

Right?

你的办公桌就在大家中间，跟其他同事挨着，还能看到做电影的人和工程师坐在一起。

Like, your your desk is amongst it with, you know, all of your other workers' desks, and it feels like there's filmmaking people next to engineers.

那你们是怎么定位自己的？

And, like, are you how do you consider yourselves?

你们是创意工具公司，还是人工智能实验室，或者两者都是？

Are you a creative tool, an AI lab, both?

是的。

Yeah.

你其实没必要太纠结公司该用什么标签来定义。

I mean, you think too much about the best way to describe, like, label of a company.

如果从实际角度看，我们就是一个做深度研究的AI实验室，专注于研发和构建大型模型。

But if you look at it, like, very practical, we are a deep research lab, an AI research lab that's pioneering and building, like, large models.

所以，没错，你可以把我们看作一个大型研究机构。

So, yes, you could think of us as a large research lab.

同时我们也在开发产品，打造令人兴奋的工具，某种程度上也是一家软件公司。

We also create products and create, like, exciting tools, and I think we're a software company in some domain.

我们拍电影。

We make movies.

我们也喜欢制作剧集。

We like making shows.

我们热衷于创作各种有意思的内容。

We like making just enter interesting pieces of content.

我真不知道该怎么把我们归类为工作室、创意机构，或者艺术家团体。

I don't know how you can classify that as a studio, creative agency, sort of artist.

再说一遍，我觉得我们最独特的地方，就在于把这些东西全都放在同一个地方。

Again, I think, like, the the sweet spot for us is, like, what happens when you have all of those things in the same place.

你说得对，你走进办公室，就能看到一个在好莱坞做视觉特效十年的人，和一个专攻大型自回归模型预训练的研究员，坐得近得几乎能碰着胳膊肘。

To your point, you can go into the office, and you literally have someone who's working Hollywood and visual effects for the last ten years sitting literally, like, centimeters apart from, like, a research scientist working on, like, pretraining on large autoregressive models.

有意思的是，这两个人居然能聊得来，这可不常见。

Like and those people, funny enough, can speak the same language, which is not that common.

他们不仅能用同一种技术语言交流，价值观上也相通——因为我们的研究员和工程师里有艺术家、音乐家、摄影师，也拍电影；而我们的创意人员里，也有不少是工程师。

And they can speak that same language technically and from a value perspective because some of our researchers and engineers are artists, they're musicians, they're photographers, they they also make films, and some of our artists and creatives are also engineers.

所以这种组合挺棒的。

So it's a nice combination of things.

我觉得，从文化上来说，我们确实打造了一个很特别的地方。

And I think, yeah, it's a I think we've built a pretty special place from a cultural standpoint.

你知道吗？你们早期的一位投资人告诉我，你们一开始根本不愿意招有博士学位的人，理由是觉得这儿做事的方式不一样。

You know, one of your early investors told me that you guys refused at the beginning, least, I don't know if you're still doing this, to hire people with a PhD at first, just because you were like, no, we're doing things differently here.

真的吗？

Is that true?

倒也不是我们故意不招有博士学位的人。

Maybe it wasn't like that we refused to, like, hire people with PhDs.

更准确地说，我们不太像别人那样看重学历背景。

It's more just really I think we we tend to not look at credentials as much as maybe others might have.

我们一些最出色的人，可能根本没有任何传统意义上的头衔，这正是关键所在。

Like, some of our best people might not have any of the credentials you traditionally look for, and that's kind of the point.

我觉得人才在全球是平均分布的，你是不是进了斯坦福，其实并不能说明你有多能干、多会执行、能做出什么，或者学得有多快。

Like, I think talent is equally distributed around the world, and if you happen to get into Stanford somewhere else, like, might not necessarily mean a lot about how much you can do and how much you can execute and build or learn.

我们想践行这一点。

I think we wanna embody that.

这其实是个凭实力说话的地方。

It's like, it's it's a meritocracy.

我们看你做的成果，你能创造出什么，而不是你出身哪里、上过什么学校、跟谁度假、认识谁。

Like, you we we judge you by the outcomes of and what are the things you can make, not by who you are or where you went to school or who you vacation with or who you know.

也许是因为，我们当初就是从那儿起步的。

Maybe because, like, that's where we started.

如果我们只看自己认识谁、出身哪里、姓氏或头衔，那我们根本不可能走到今天。

If we if we judge ourselves by who we knew and where we were and our last names and our titles, then we wouldn't be here.

所以，这其实也体现了我们内部建立的这种靠实干赢得认可的机制。

And so it's a representation, I would say, of this very hard work driven meritocracy that we've built internally as well.

我知道，这也让公司不得不保持精简，很早就得想办法赚钱。

And I know that that has also had its effects on on, you know, how you've had to be lean as a company and how you've had to, you know, start revenue generating pretty early.

所以，赚钱和研究其实是同步进行的。

So, like, really the revenue generation was happening alongside the research.

不过我想聊聊研究这块。

But I do wanna talk a little bit about the research.

对吧？

Right?

当然。

Because Sure.

我当初在研究你们的时候，看过你们最早推出的那些视频生成模型。

I was you know, in the course of researching you, was looking at some of your you know, the first video gen models that came out.

当然，以当时的技术水平来看，这些模型已经很厉害了，不管是两年前还是四年前。

And, you know, of course, for the time, they were impressive for the time, like, two years ago, whatever, four years ago.

这让我想起了威尔·史密斯吃意大利面的那个视频。

And it just reminded me of that Will Smith eating spaghetti thing.

现在视频生成技术已经进步太多了。

Like, we've come a long way in video generation.

对吧？

Right?

我还记得，Runway 出过一项关于双过程图像生成的研究。

And one piece of research that I believe came out of Runway was about dual process image generation.

对吧？

Right?

就是用一个视觉语言模型实时评估 AI 图像生成器的效果。

Having this kind of like judge vision language model assessing AI image generators in real time.

你们之前有在做这方面的研究吗？

Is that something that you guys were working on?

有几个方面，我觉得主要还是看我们走到了哪一步。

A couple of things, I guess, the on how far we've come.

是的。

Yeah.

我们确实进步巨大。

We've we've come very far.

我说的是，我们最早发布的那个视频模型，你可能看过一些输出，大概是2023年，也就是两年半前，在AI领域里，那简直像三十年前的事了。

I mean, I think the first ever video model we put out that maybe you've seen some of the outputs was around 2023, so, like, two and a half years ago, which in AI landfill, like, three decades ago.

对。

Yeah.

当时那可是大新闻，登上了《纽约时报》的头版。

And it was such a big deal that it was in the front page of the New York Times.

我们还有一整篇专题报道，要是有人想查，会发现当时所有人都震惊了——居然能生成这种视频，以今天的眼光看，分辨率低得可怜，画质差得不行。

We had a, like, a whole article, if if people wanna search for it, it's like everyone was, like, amazed that you could render this, like, incredibly low res, incredibly bad by today's standards, like, video output.

但那时候这东西是全新的。

But it was it was it was new.

完全是前所未有的。

It was totally new.

当时没人觉得这能做成，可我们还真做到了。

Like, no one thought you could do it, and so there we are.

而今天，我们能输出超逼真的4K视频，画质和镜头语言的水准，两年前根本不敢想。

And today, we're able to output, like, incredibly hyper realistic four k footage with a level of realisms and cinematography that you'll, like, have never imagined by just two years ago.

我觉得整个行业和这个领域确实进步巨大。

I thought, yeah, I think the industry and the field has gone really far.

对我们来说，希望再过三年，回看今天，现在的模型会像两年前的模型那样落后。

And the idea for us will be like, well, hopefully, in, like, three more years, we'll look back at today, and the models we have today will feel like the model we had two years ago.

我觉得我们正迅速朝着这个方向前进。

And I think we're heading towards that trajectory very soon.

从研究的角度来看，我们做了很多有意思的科研工作。

From a research perspective, we do a lot of research interesting research.

我们会发布一些研究成果。

We publish a couple of things.

但大部分内容我们不会公开，主要是出于竞争考虑。

We don't tend to publish all of it for mostly competitive reasons.

我们在视频和图像领域都有研究。

We do have research in in the video domain and image domain.

我们也会发布一些关于偏见问题的研究。

We publish also things in, like, in biasing.

那你怎么确保这些模型——你可能指的是这个——

So how do you make sure that the models maybe that's the one you're referring to.

如何让模型更清楚地意识到自己在现实世界中的表现方式呢？

It's how do you make sure that the models can become a bit more self aware of how they're depicting things in the real world?

因为模型的训练方式决定了它们可能会反映人类看待世界的方式。

Because, like, the models are are trained in such a way that they may reflect how humans see the world.

所以，如果你想让模型的输出更多元，就得改进训练方法。

And so if you wanna had a more diverse sort of outputs of a model, then you need to improve the way you train the model.

我们投入了大量时间和资源，确保这些模型不仅在产品层面，也在公平性和多样性层面足够吸引人，让人愿意真正使用。

And that's something we've spent time and and and spend, like, significant amount of resources to make sure the models are from both from a product, but also from a graded standpoint and diverse standpoint, like, interesting enough for people to actually use.

我们稍微跳出来看一眼。

Zooming out a little bit.

Runway 现在已经成为电影制作人和视频编辑们的首选工具。

So Runway has become, you know, the go to for filmmakers, video editors.

你们已经筹集了数亿美元的资金。

You've raised hundreds of millions in funding.

PitchBook 的数据显示，你们的投后估值接近8.6亿美元。

PitchBook says somewhere close to 860,000,000 at a 5.3 post money valuation.

我记得你们上一轮融资时，我报道过，当时的重点是你们想在世界模型上做更深的探索。

Your last round, I remember covering that, and it was with the angle that you're you're trying to go deeper on world models.

你能解释一下吗？为什么现在从视频模型转向世界模型的趋势这么明显？

Can you explain why there's this push we're seeing just generally from video to world models?

这两者之间是怎么连接起来的？

Like, how do those two meet?

因为谷歌是分开训练他们的模型的。

Because Google trains theirs separately.

他们把VO和Genie分开训练，所以我很好奇你是怎么看它们未来融合的。

They train VO separately than Genie, so I'm kind of curious how you see them moving together.

对。

Yeah.

就像我之前说的，不是非要自己掌控一切。

To the point of, like, not having, like, a necessarily, like, our own your own destiny that I was speaking before.

我们早在2022到2023年就讨论过通用世界模型这个概念了。

Like, we spoke about this idea of general world models back on 2022, 2023.

那时候，AI领域还没形成现在这种普遍共识，我们就开始思考：到底该怎么构建这样的通用世界模型？

And so relatively early within the, like, lifetime of what we now consider, like, a common trope of AI, which is how do you build this general world models or world models?

我们慢慢形成了自己对这个概念的理解，以及怎么去看待它。

And I think we build our own intuition as to what that actually means and how we think about them.

我觉得，现在‘世界模型’这个词已经变成一个泛化的说法，几乎什么都能往里装了。

Maybe an observation is I think, generally, world models have become, like, a cultural term to describe anything.

我最近看到一家法律公司把他们的工作称为世界模型，我当时就想，你这说法有点牵强了。

I I saw a a legal company recently described their work as a world model, then I'm like, okay, you're kind of stretching it a little bit.

那你怎么定义世界模型呢？

So how do you define a world model?

对我们来说，世界模型一直是指那种能像人类一样理解物理世界的模型。

So so so for us, it's always been around this idea that you could train models that can understand the physical world in perhaps a similar way that we humans do.

有人认为，实现这一点的最佳方式就是给模型提供足够多的像素或视频数据。

And there's an argument to be said that the best path for that is you need to show the model or models, like, enough pixel or video data.

如果模型能理解视频在现实中是如何生成的，它们自然就会领悟现实世界中的各种现象——比如物理规律、重力、连贯的反射、因果关系。

And if the models can understand how videos should be or how videos are created in the real world, they're naturally understanding or becoming aware of things that occur and happen in the real world, physics, gravity, cohesive reflection, cause and effect.

这些都不是我们特意告诉模型要去学的东西。

Those are things that we don't tell our models to learn from.

我们不会跟模型说：‘当你扔一个物体时，它应该以这个速度、这个方向运动，原因是什么。’

We don't tell a model, hey, when you throw an object, like, the object should move at this velocity and with this set of vectors, and this is why.

模型是自己默默学会这些的。

The model just implicitly learns about it.

所以某种程度上，视频已经成为最直观的方式，让我们训练出能像人类一样理解、行动和应对世界的AI系统。

And so in a way, video has become the most obvious representation of how you can train AI systems that can understand, act, and behave in the world in similar ways that we do.

归根结底，我认为长期追求的终极目标，不是让模型仅仅能描述现实。

And that's ultimately, I would say, lot of the holy grail over time is you don't want models that can describe reality.

你看现在的语言模型，它们能描述现实，是因为它们靠文本训练，而文本本身是我们人类对现实的一种抽象表达。

So you think about language models as models that can describe reality because they train on text, and text is representation by kind of obstruction of, like, reality that we humans have created.

单靠视频来训练，反而更纯粹，因为你直接模拟了现实。

Training on video alone, is a bit more pure because you can simulate reality.

你不需要去描述它。

You don't have to describe it.

你只需要模拟它，然后从模拟中学习和修正。

You can just simulate it and unlearn from that simulation.

所以对我们来说，这条路和这种研究理念，比单纯盯着语言要有效得多。

So that I think for us has been a much better path and philosophy of research than just fully focusing on language alone.

现在的语言模型能做到的一些事，你觉得有没有可能完全用视频来替代？

Are there certain things that LMs do today that you think could kind of be replaced with just with video?

是的

Yeah.

我是说，视觉推理和视觉规划这类能力。

I mean, like visual reasoning and like visual planning.

所以世界模型最有意思、最有吸引力的应用场景之一，就是在机器人领域。

So that's why some of the most interesting compelling use case of world models is in the robotics side.

让系统学会如何在现实世界中完成极其复杂的动作。

Having a system understand how to navigate a very particularly complex action in the world.

你可以这么想：用一个语言模型，只靠视觉输入，再用语言来推理。

The way you can think about it is I'm gonna like have a language model, like just do like vision and try to reason about it with language.

另一种方式则更接近我们人类的做法。

Another one could be more like we do.

当你在现实世界中执行任务时，并不会一直在脑子里自言自语地分析每一步是怎么回事。

Like, when when you're tasked to do things in the real world, you're not you don't have an internal monologue all the time about how things work.

比如开车的时候，你有时候根本记不清自己是怎么开的，就是开过去了。

Like, sometimes you drive and you don't even remember how you drive, you just, like, you drove.

你懂的吧。

You know?

对。

Mhmm.

但你内心其实对这个世界和当前情境有一种潜移默化的理解。

But you have an internal implicit understanding of the world into the situation.

对吧？

Right?

因为你见过太多次了，开过太多次车，看过太多次，你的大脑已经自动形成了这种能力。

So but because you've you've seen enough times, you've drove in enough times, you've seen it, that your brain is kind of hardwired to do it.

同样地，我们也可以训练人工智能系统，让它们通过观看视频，而不是只靠语言，来习得类似的行为。

And there's there's not going to be said that there's a similar way to train artificial networks and systems that can do that same kind of behavior, but now you feed it videos and instead of language alone.

你有没有开始训练能同时处理语音、文字和视频等多种模态的单一模型？

Have you started training single models on multiple modes, like voice, text, video all at the same time?

有。

Yes.

我们训练的是所谓的全模态模型。

We we train like what we call Omni models.

全模态模型就是把这些不同能力整合在一起的模型。

So Omni models are models that have kind of the combination of those things together.

对。

Yes.

明白了。

Okay.

关于世界模型，眼下大家讨论最多的实际应用，比如业内其他公司，我之前提过谷歌。

So for world models, the, you know, near term use cases that everyone's talking about, I mean, others in the field, you know, I mentioned Google before.

还有World Labs这家公司，他们主要就在做世界模型。

There's World Labs, you know, company, they're pretty focused on on world models.

看起来发展方向首先是娱乐场景。

It seems like the trajectory is entertainment use cases first.

对吧？

Right?

比如游戏、互动娱乐，然后是机器人。

Like gaming, interactive entertainment, and then robotics.

我知道Runway也已经开始发力机器人领域了。

And I know Runway has has started doing a robotics push as well.

再往后还有什么吗？

Is there something beyond that?

有的。

There is.

我们主要有三个产品方向。

I mean, we have we have like three main kind of like product, I would say, focus.

第一个，我叫它线性媒体。

One is, I call it linear media.

线性媒体就是视频、电影、短片、广告，还有各种内容。

So linear media is videos and films and short stories and ads and just content generally.

然后是非线性媒体。

Then there's, like, nonlinear media.

非线性就是实时的。

Nonlinear is real time.

所以，当你能制作这类视频，却不必等待的时候，会发生什么？

So what happens when you can create these kind of videos, but I don't have to wait?

我可以直接实时渲染出来。

I can just, like, render them in real time.

这就是实时交互。

So this is real time.

你正在问我问题。

You're asking me a question.

我回复你，而且我了解你的视频内容。

I reply back, and I'm aware of your your video.

你也了解我的视频。

You're aware of mine.

你会让我去做一些事情。

I you're gonna ask me to do things.

你懂的。

You know?

对。

Yeah.

在继续之前，你再详细说说这个，刚才你提到过。”

So you actually, before you move on, talk a little bit more about that because you talked up yeah.

你们最近推出了 Characters，对吧？作为一款实时视频产品。

You guys recently released Characters, right, as a Correct.

我最近看到这方面的应用越来越多了。

As a product for real time video, and I'm seeing this crop up a lot.

我最近刚参加了 Human X。

I recently was at Human X.

我采访了 Pika。

I interviewed Pika.

他们正在做实时视频代理。

They are doing real time video agents.

对吧？

Right?

我就不细说了，但本质上这是为网红打造一个实时视频助手，像是他们的数字分身，能让其他人跟它互动。

So I won't go into it in too much detail, but it's essentially a way to create, I think for influencers, a real time video agent that's like a a maybe a twin of an influencer that other people can interact with.

挺有意思的。

Interesting.

不过我也采访过Descartes，他们用实时视频做虚拟试穿这类应用。

But then, you know, I also interviewed Descartes, and they're doing you know, they're using real time video for things like virtual try ons.

所以电商场景特别多，我想知道你们对Characters目前的观察怎么样。

So there's a lot of ecommerce use cases, and I'm curious what you're seeing for Characters so far.

首先，很高兴看到越来越多公司开始关注这个领域。

So first of all, it's great that more companies are, like, paying attention to to this.

我觉得这有助于整个生态更聚焦。

I think it helps the ecosystem, like, focus more.

就像世界模型一样，大家对它的定义和用途其实还很难达成共识。

I think similarly with world models, there's like this it's hard to, like, agree on what it means and why how it can be used.

这毕竟是你自己的目标，每个人的想法可能都不一样。

It's it's your own purpose, and people might have different ideas and things.

如果你想围绕它打造网红，我觉得这也不是什么坏事，但那只是一种方式。

If if you wanna build influencers around it, like, that I don't think that's necessarily bad, but, like, that's one way.

还有很多别的做法。

There's many other ways of doing things.

对我们来说，主要还是在做软件或者模型，尽可能让人感觉像真人，做到我们能做的最好程度。

I think for us, it's mostly been about building software or models that can feel as human as possible in the best possible way we can.

比如学习这件事，你就可以想想。

And so you can think about learning, for example.

目前模型能给任何年龄段、任何形式的学习者提供的帮助，还存在一个巨大的缺口——那就是个性化辅导。很多研究都证明，学生能否成功，最重要的因素之一就是有个导师。

One of the things that is a huge gap in terms of what models can offer students of any type of age or form or or kind of content in terms of helping you learn sort of two things, like the ability, and this is proven in in many studies that one of the highest predictor of, like, success of a student is having a tutor.

有个能随时陪着你、根据你的情况调整教学内容的人，这当然特别重要，但要大规模实现根本不可能。

Having someone that can be there or personalizing kind of your your education, and, of course, like, it's impossible to do that at scale.

除非你特别有钱，不然怎么让全世界每个人都有个专属家教呢？

Like, how are you gonna get everyone in the world a tutor unless you're, like, extremely rich.

对吧？

Right?

没错。

Right.

这就是技术的作用。

Well, that's the role of technology.

比如，有一条路可以让每个人随时都有个家教，了解你的偏好，永远不会累，能一遍又一遍地重复讲解。

Like, there's a path where anyone can, like, have a tutor that's there twenty four seven that understands the preferences, that's never gonna get tired, that can resay the things over and over again.

我觉得我们现在看到的角色和一些模型，正朝着这个个性化方向发展。

And I think a lot of what we're seeing right now with characters and and some of our models are heading towards that direction, you know, of this personalization aspect.

当然，个性化还可以有其他很多种方式。

And, of course, you can think about personalization in many other different ways.

所以我称之为非线性的，因为它不是从一个预设的起点走到终点。

And that's why I call it nonlinear because, like, it's it's it's not you don't begin and end in a very specific predetermined way.

这是一个开放的世界。

It's an open ended world.

这场对话能往任何我们想的方向发展。

This conversation can go anywhere we wanna take it.

它是开放式的。

It's open ended.

它是非线性的。

It's nonlinear.

直到最近，这种能力在技术上还根本做不到。

That same ability wasn't really possible with the technology until very recently.

以前你不可能打开电脑就进入一个开放的世界，因为这在不久前还根本不可能。

You couldn't just open your computer and navigate an open ended world because that was not possible until very recently.

所以我觉得，当这项技术越来越普及，越来越多的人能用上时，各种有趣的应用才会真正涌现出来。

And so I think a lot of the interesting use cases and applications start to emerge as this becomes like more accessible to more people.

而且，你们那个，叫什么来着？

And, you know, you guys have your what is it?

创新实验室，还有你们的开发者计划。

The Innovators Lab and also your Builders Program.

你正在把这推向整个世界。

You're you're you're putting this out in the into the world.

所以我觉得初创公司可以去尝试，帮我们发现实际的应用场景。

So I think startups can experiment and kind of show you where the use cases are.

对吧？

Right?

是的。

Yeah.

没错。

Correct.

我觉得关键在于认识到这还是一种相当新的技术。

I think it's recognizing that it's a fairly new technology.

我的意思是，真正的发展才刚刚几周、几个月而已。

I mean, it's really thinking like weeks, months hold.

从艺术和创意的角度来看，对我来说是这样。

And this is for me from an artistic creative standpoint.

如果你看看人类历史上那些重大的文化与科学革命，就会发现它们往往源于人们不断摆弄各种东西。

If you look at the history of some of the, like, humanity's largest and biggest, like, cultural and scientific revolutions have been people tinkering with things.

对吧？

Right?

你把东西交给别人，把技术交出去，别让创作者自己去穷尽所有可能的用途，而是交给普通人去探索。

You you give them stuff, you give them technology, and you have, instead of the creator figure out all the things that could do, you give it to someone.

对吧？

Right?

就拿相机来说吧。

So think about, I mean, cameras.

相机最初是作为科学工具，用来研究人体运动的，可有人觉得用它拍电影挺有意思。

Cameras were a scientific breakthrough supposedly meant to be used to study human motion until some people just thought it was good to make theater with it.

结果呢？就这么发展到了今天。

And then, like, it didn't then here we are.

对吧？

Right?

这其实就是随意摆弄带来的结果。

It did, like, it's a consequence of just tinkering.

所以我们希望鼓励更多这样的实验和摸索，尤其是在新模型的早期阶段。

And so we wanna promote more of that experimentation and tinkering specifically with early stages of new models.

Characters 能通过 API 使用吗？

Is Characters available via API?

可以。

Yes.

你可以通过 API 来使用它。

You can use it via an API.

有没有谁你不希望他们使用这个功能？

Is there anyone you wouldn't let use it?

我第一个想到的，有点悲观的用法，就是把它当个陪伴工具。

Like I imagine my first use case that comes to my dystopian brain is this is like a companion thing.

对吧？

Right?

所以人们可以随便找个虚拟女友，这种事其实已经有人在做了。

So people can like just have their virtual girlfriend that they you know, people are already doing this.

对吧？

Right?

现在他们还想要一个真实的面容。

So like now they wanna face.

但你为什么觉得这一定就是反乌托邦呢？

But why do you think that's dystopian necessarily?

我明白有人会觉得这可能不太妥当，但我觉得……

I could see the I could see the argument of it being like, I guess, more of a bad thing, but like

我觉得科技在很多方面让人类变得更加孤立了。

I think technology has isolated humans in a lot of ways.

对吧？

Right?

现在很多人都说我们正面临孤独症危机，把原因归咎于社交媒体，这挺讽刺的，因为社交媒体本来是想把大家连在一起的，结果却让不少人感觉更孤单了。

We have the loneliness epidemic that a lot of people point to social media as the cause, which is ironic because social media was meant to bring us all together, but really it's made a lot of us feel more isolated.