本集简介
双语字幕
仅展示文本字幕，不包含中文音频；想边听边看，请使用 Bayt 播客 App。
在通往空间智能的道路上，生成真正具有空间上下文和空间基础的像素，这是Atlas所迈出的非常艰难的一步。
On the path to spatial intelligence, generating pixels that are truly spatially contextualized and grounded, that is the very hard step that Atlas has taken.
你知道，大语言模型是基于下一个词元预测构建的。
You know LLMs are built on NEXT token prediction.
我们看到视频模型是基于帧预测构建的。
We've seen video models as being built on frame prediction.
Atlas实际上是新视角预测。
Atlas is really new view prediction.
这是人工智能真正能够为人们及其流程释放巨大价值的地方。
This is the real place where AI can actually unlock a ton of value for people and their process.
我们说的是，比如，50倍到100倍的缩减。
We're saying, like, 50, a 100 x reduction.
在《黑客帝国》第一部里有一个著名的镜头，尼奥正在往下掉。
There was a famous shot in the first Matrix movie where Neo is, like, falling down.
没错。
Exactly.
他们用了几百台摄像机，在绿幕前从那个角度拍摄。
They had hundreds of cameras viewing that angle on a green screen.
在Atlas上，我们只需要三台摄像机就能做到。
On Atlas, we can do this with just three cameras.
不需要影棚拍摄，不需要绿幕，也不需要昂贵的校准。
No studio capture, no green screen, no expensive calibration.
这个结果从来没有人见过。
No one has ever seen this result.
你开始做这件事的时候，知道它会成功吗？
When you set out to do this, did you know it was gonna work?
我相当确定。
I was pretty sure.
每次我们把模型做大，每次训练时间更长，效果都显著提升。
Each time we made the model bigger and each time we trained it for longer, it got significantly better.
那是不是意味着我们很快就能看到四维视频了？
Does that mean we're gonna get four d video?
我可以到处走走吗？
Do I go walk around?
语言模型的核心是预测下一个词元。
Language models are built around predicting the next token.
如果模型转而学习预测世界的下一个视角，会发生什么？
What happens when a model instead learns to predict the next view of the world?
本期节目中，Martin Casado 与 World Labs 的联合创始人李飞飞、Justin Johnson 和 Ben Mildenhall 一起，讨论 Atlas 以及构建能够理解物理空间的 AI 这一更广泛的挑战。
In this episode, Martin Casado sits down with World Labs co founders Fei Fei Li, Justin Johnson, and Ben Mildenhall to discuss Atlas and the broader challenge of building AI that can reason about physical space.
他们解释了 Atlas 如何将生成与三维重建结合起来，团队为什么选择新视角预测作为其底层基础，以及他们在尝试扩展一种未经测试的方法时学到了什么。
They explain how Atlas combines generation and three d reconstruction, why the team chose new view prediction as its underlying primitive, and what they learned trying to scale an approach that hadn't been tested before.
他们还谈到了目前仍然缺失的部分，包括更丰富的动态和交互，以及世界模型最终如何将模拟与机器人和规划联系起来。
They also get into what's still missing, including richer dynamics and interaction, and how world models could eventually connect simulation with robotics and planning.
这一切背后是一个更大的假设。
Underlying it all is a bigger hypothesis.
新视角预测能否在空间智能中扮演类似角色，就像下一个词元预测在语言中所扮演的那样？
Could new view prediction play a similar role for spatial intelligence that next token prediction has played for language?
昨天可是个大日子，你们发布了一个新的前沿模型，反响特别好，而且还在持续升温。
So big day yesterday, you launched a new frontier model, which has got amazing reception, which is still coming in.
我觉得咱们可以这样来聊，先说说昨天具体发布了什么，然后再回顾一下历史，一步步往上聊。
I think maybe a good way to structure this conversation, let's just talk about exactly what that was, and then we'll go back to history and work our way back up.
那么，Justin，你要不要先讲讲昨天发布的内容，以及它的重要性？
So maybe, Justin, you wanna talk about what was launched yesterday, why it's significant?
好的。
Yeah.
Atlas是我们新一代的世界模型。
So Atlas is our new next generation world model.
它有三个基本功能。
It has three basic things.
它能够生成、重建和模拟世界。
It can generate, reconstruct, and simulate the world.
在这其中，有几个主要的能力。
So within that, there's a couple different major capabilities.
它的相机条件生成能力非常强。
It has really good camera condition generation.
你可以输入一张图片和一条相机轨迹，控制模型生成视频帧，从任何你想要的角度来呈现。
So you can input an image together with a camera trajectory and steer the model and have it generate video frames, mark any perspective you want.
它在稀疏三维重建方面表现很出色。
It's really good at sparse three d reconstruction.
你可以输入一张或多张，最多一百张真实世界的视图，用它们来重建真实世界，这个重建结果既可以生成一段穿越空间的新视频，也可以生成该空间的显式三维重建。
You can input one or multiple up to a 100 frames that are views of the real world and use those to reconstruct the real world, and that reconstruction can take the case either of novel a video flying through the space or an explicit three d reconstruction of the space.
最后，它还可以用于模拟。
And finally, it can be used for simulation.
为此，我们展示了那些很酷的子弹时间视频，在网上引起了很多关注，还有机器人模拟。
And for this, we show off these awesome bullet time videos, which got a lot of attention online, and then also robotic simulation.
什么是子弹时间视频？
What's a bullet time video?
子弹时间视频就是……
A bullet time video.
这来自《黑客帝国》。
This comes from The Matrix.
第一部《黑客帝国》里有个著名的镜头，尼奥在往下掉。
There was a famous shot in the first Matrix movie where Neo was, like, falling down.
对。
Yes.
没错。
Exactly.
是的。
Yeah.
所以记得那个著名镜头里，他正在往下掉。
So then remember in that famous shot, he's falling down.
那是慢动作，镜头绕着转了一圈。
It's in slow motion, and the camera flies all the way around.
他们拍那个镜头的方式是，周围摆了一圈上百台摄像机。
The way that they did that shot is they had a ring of hundreds of cameras.
所以他在工作室里摔倒了。
So then he fell over in the studio.
他们用几百台相机从那个角度拍摄绿幕，然后用这几百台相机制作了《黑客帝国》里那个著名的镜头。
They had hundreds of cameras viewing that angle on a green screen, then they used those hundreds and hundreds of cameras to make that famous shot in Matrix.
但现在有了Atlas，我们只需要三台相机就能做到。
But now with Atlas, we can do this with as few as three cameras.
所以不需要工作室拍摄，不需要绿幕，也不需要昂贵的校准。
So no studio capture, no green screen, no expensive calibration.
我们真的可以把三台iPhone绑在iPod上，用它们拍摄一些事情发生的视频，比如有人投篮，或者有人把草莓掉进一碗牛奶里。
We can literally stick three iPhones onto iPods, use these to take sort of a video of something happening, like someone shooting a basket or someone dropping a strawberry into a bowl of milk.
然后从这三段iPhone视频中，我们可以重新构图，想象一下，比如冻结时间，当牛奶溅起时，让相机飞过去，得到这些令人惊叹的冻结时间视角。
And then from those three iPhone videos, we can then reframe the shot and imagine, like, freeze time, have the camera fly in as the milk is splashing up and get these amazing frozen time views.
对。
Yeah.
而且我们只需要几台相机就能做到。
And we can do this with just a couple cameras.
Atlas到底是做什么的？最简单的说法是什么？
What is the simplest description of what Atlas does?
输入是什么，输出又是什么？
What goes in and what comes out?
对。
Yeah.
Atlas最核心的原理，最根本的一点，就是做新视角预测。
So one of the really core principles of Atlas, the most fundamental thing, is it does new view prediction.
这是一个非常基础的能力，我们觉得特别令人兴奋，是基础模型里一个全新的基础能力，以前从来没人做过。
And this is a really fundamental primitive that we think is super exciting, a super new primitive for base models that no one's ever done before.
对吧？
Right?
我们知道大语言模型是基于下一个词预测的。
So we know LLMs are built on next token prediction.
我们也看到视频模型是基于下一帧预测的。
We've seen video models as being built on next frame prediction.
Atlas 本质上就是新视角预测。
Atlas is really new view prediction.
对吧？
Right?
也就是说，给定一个场景的若干视角，或者对场景的一段描述，这些信息会进入我们所说的空间上下文，它隐式地描述了我们要讨论的那个世界是什么样的。
That given some number of views of a scene or a description of a scene, those go into what we call a spatial context that describes implicitly what is the world that we wanna talk about.
然后你可以把虚拟相机放在空间和时间的任意一个点上，Atlas 就能理解从那个时空位置看，这个世界应该是什么样子。
Then you can point a virtual camera at at an arbitrary point in space and time, and Atlas will understand what that world is supposed to look like from that position in space and time.
Ben，现在市面上的视频模型都自称是世界模型，也都声称能生成新视角，你能不能更具体地拆解一下，这和之前那些五花八门的模型到底有什么不同？
Ben, the video models out there all claiming to be world models and all claiming to have novel views, and can you maybe tease apart kinda more concretely how this is different from the myriad models that have come before?
嗯。
Mhmm.
对。
Yeah.
我觉得 Justin 刚才说的空间上下文这一点非常关键。
I think what Justin was saying about the spatial context aspect is super important here.
所以现在有很多视频模型。
So there's many video models.
很多视频模型其实是因为单张图片输入或者首尾帧插值而出名的。
A lot of video models actually got their claim to fame from their single image input or their start to last frame interpolation.
现在开始出现一些模型，能够处理二十、三十、五十张图片这样的全方位参考。
Now we're starting to see models that can do this kind of omni referencing with twenty, thirty, 50 images.
但Atlas的关键在于，它对你输入的每一帧都有一个空间上的、有实际意义的位置。
But what's key with Atlas is that it actually has a kind of, like, spatially grounded meaning to every frame you put into it.
所以它不仅仅是一张图片，模型想怎么解读就怎么解读，或者你可以试着通过文字提示跟它争论，让它做特定的事情。
So it's not just an image that the model's gonna interpret whatever way it wants, or you can kind of try to argue with it in the the text prompting and get it to do something specific.
而Atlas呢，每张图片都关联了一个三维相机位姿，这意味着你可以用极高的精度来完成重建任务。
With Atlas, every image actually has an associated three-dimensional camera pose, and that means that you can perform this task of reconstruction with an extremely high degree of precision.
对吧？
Right?
所以如果我们有四个视角看这个房间，每个角落一个，你可以把这些输入模型，然后得到这个房间里所有东西的精确复制。
So if we had four views of this room, one in each corner, you can put those into the model and then get an exact replication of everything you see in this room.
而且它不会去猜测其他角落有什么，或者物体之间的关系。
And it's not going to guess what's in the other corner, like a relationship between things.
它只会精确地重现你给它的内容。
It's just gonna reproduce exactly what you give it.
你也可以在创意或想象的层面上做到这一点。
And you can also do that in a kind of creative or imaginative sense too.
如果你从不同的AI生成或真实世界场景中取两张照片，你实际上可以定位和布置它们，构建出这些有意的、定向的飞越效果，这些效果完全由你放置内容的确切位置以及相机的观看和移动路径所决定。我认为这与那种更随机的、像老虎机一样的效果非常不同，那种效果需要你一遍又一遍地重新生成，只能通过视频模型那种更高层次的文本控制来实现。
If you take two photos from different AI generations or real world locations, you can actually position and stage those to build these kind of intentionally directed fly throughs that are really governed by exactly the precise place that you put the content you want and where the camera's gonna look and travel, which is very different, I think, than the kind of, like, more slot machine effect you get of having to retry generations over and over with just that kind of higher level of text control you get with video models.
这只是传统视频模型的一个明显升级版，还是一种新的架构？
Is this just kind of an obvious scaled up version of a traditional video model, or is it a new architecture?
我认为这在几个方面都是相当新的东西。
I think it's a pretty new thing for a couple different reasons.
好的。
Okay.
我们谈到的一点是，它在同一个模型中同时进行生成和重建。
One that we talk about is it does both generation and reconstruction jointly in the same model.
就像Ben说的，这个东西只需要看这个房间的几个视角，就能把房间里的一切都精确地重建出来，跟你看到的一模一样。
Like Ben was saying, this thing can take a couple views of this room and then reconstruct everything in this room exactly as you see it.
以前，重建在计算机视觉里是一个独立的子领域，有自己专门的任务和专门的模型。
And historically, reconstruction has been its own subfield in computer vision with its own specialized task, its own specialized models.
而生成则是所有文本转视频模型最擅长的，比如我们过去几年看到的那些大型扩散模型。
And generation is what all the text to video models are really good at, like, what all the big diffusion models we've seen the last couple years.
这些模型在创意应用上非常出色。
And those are great for creative applications.
我想想象一些从未存在过的东西。
I wanna imagine something that's never been there before.
但现在有了Atlas，我们第一次把视觉智能的这两个不同部分整合到一个模型里。
But now with Atlas, for the first time, we're putting these two different parts of visual intelligence together in one model.
所以它能在同一个架构中同时进行三维重建和生成。
So it can do both three d reconstruction and generation together in one architecture.
为了实现这一点，我们不得不做一些改变。
So to do that, we had to make a couple changes.
一个改变是，我们从一开始就让它支持多模态。
One is we had to make it multimodal from the start.
所以这个东西天生就能处理文本。
So this thing natively works on text.
它能处理图像。
It works on images.
它能处理视频。
It works on videos.
它还能把相机位姿作为模型的原始输入，我觉得以前从来没有人把相机位姿用在预训练阶段。
It also works on camera poses as a native input to the model, which I don't think anyone's ever done at the pre training phase before.
而且它把三维作为它处理的一种原生模态。
And it uses three d as a native modality that it works on.
所以这个东西从一开始就是为原生多模态而设计的，
So this thing from the beginning was designed to be natively multimodal in a
这种方式是别人想不到的。
way that no one else can think.
抱歉。
Sorry.
我对这个领域不太熟。
I just have I don't know the space super well.
说到三维，是指深度那种吗？
By three d, is this, like, depth or Yeah.
还是模型之类的？具体是什么意思？
Models or, like, what does that mean?
对。
Yeah.
我们目前用的形式是深度图。
So the formulation we used so far is depth maps.
好的。
Okay.
对吧？
Right?
所以现在，当你有一个带有虚拟相机、能确定其在三维空间中位置的帧时，那个相机位置和相机参数就是模型的原生输入。
So right now, when you have a frame that has a virtual camera telling its position in three d space, that camera position and camera parameters are a native input to the model.
然后，与相机位置相关联的，你可以同时有RGB信息，告诉你那个空间位置看起来是什么样，以及一个深度图，告诉你那个位置在三维空间中的空间结构是什么。
And then what attached to that camera position, you can have both, like, RGB telling you what does that position in space look like, and you can have a depth map that tells you what is the spatial structure of that position in three d space.
所以，文本、图像、视频、三维相机，这些都是这个模型以多模态方式联合处理的模态。
So then text, image, video, three d cameras are these modalities that this thing all does jointly in a multimodal way.
我想补充一点，因为我觉得Justin刚才说的真的很重要，还有Ben说的，这一点被低估了。
I wanna add something because I think what Justin just said is actually so important, and also what Ben said, that it's underappreciated.
这是第一次我们将像素生成和像素重建统一起来。
It's the first time we have a unification of pixel generation and pixel reconstruction.
在计算机视觉领域，这个领域已经存在了半个多世纪。
In the world of computer vision, this field has been around for more than half a century.
我在这里，在这个领域待了几十年，我无法告诉你有多少博士论文是关于重建或新视角合成问题的。
Sitting here, having been in this field for decades, I cannot tell you how many PhD theses have been written on the problem of reconstruction or novel view synthesis.
而且，我们的领域传统上也有多个方向。
And also our field traditionally have multiple tracks.
你去参加计算机视觉会议，会有像素生成方向、识别方向，还有三维重建方向。
You go to a computer vision conference, you have the pixel generation track, you have some recognition track, and you have three d reconstruction track.
这是一个很优雅的模型，它通过锚定视角和视角估计，把重建和生成这两个问题统一起来了。
This is an elegant model that combines or unifies the problem of reconstruction and generation by anchoring on viewpoints and the viewpoint estimation.
这真的非常强大。
And that's just incredibly powerful.
我们能退一步，你先给我们补充一下背景吗？
Can we take a step back and then maybe you just fill something out?
你创办公司的时候，我记得你说过要攻克空间智能，对吧？
So when you started the company, I remember you saying you wanna tackle spatial intelligence, right?
现在我们有了这个新模型，作为外行，我觉得它非常通用，你有下一个完美预测，这是下一个新视角预测。
And now we have this new model, and so as a lay person it feels very general to me, you've got next perfect prediction, and this is next new view prediction.
新视角预测，所以你可以从一个视角得到另一个新视角。
New view prediction, And so like you can get one viewer instead of views, and you get a new view.
你能大概解释一下，这是如何朝着空间智能这个普遍问题迈出的重要一步吗？
Can maybe you pencil out how this is a significant step to this general problem of spatial intelligence?
要不你先说说什么是空间智能吧。
Maybe by starting to describe what spatial intelligence is.
空间智能最终要让我们既能生成空间本身，又能在其中推理，还能在空间里进行编辑和交互。
Well, spatial intelligence eventually must enable us to both generate what the space is, reason within it, and being able to edit and interact within it.
现在我们可以争论，这是三维还是四维的？
Now we can argue is it three d or four d?
归根结底，加上时间维度就是四维的。
Ultimately, it's four d with the time dimension.
但即使只是三维，这些也是空间智能必须完成的基本任务。
But even just three d, these are the fundamental tasks that one has to do or spatial intelligence has to enable.
然后我们再说，有了这些，你就能渲染、模拟，还能规划行动。
And then we talk about with that, you can render, you can simulate, and you can plan actions.
但要做到这些，一个根本问题就是要理解空间的几何结构、物理特性。
But to do that, a fundamental problem to solve is to understand the geometry and structure and the physics of the space.
对。
Yeah.
而且我确实认为Atlas是一个重要的进步，因为现在每一帧都能生成并估算出一个关键信息，那就是视角，也就是相机的位置。
And I do believe Atlas is a significant step forward because now with every single frame, you can generate and estimate an important piece of information, which is the viewpoint, the camera pose.
而这是了解空间几何结构所需的最关键信息。
And that is the most critical information one needs about the geometry of the space.
这就能带来我们在博客中展示的模型下游出现的各种涌现行为。
And that can lead to all the emergent behaviors we see in the downstream of the model, which we showed in the blog.
所以在通往空间智能的道路上，生成像素肯定是一个早期步骤，我们已经看到了所谓的海量模型。
So on the path to spatial intelligence, generating pixels is definitely an early step, which we have seen with what you call gazillions of models.
但是生成真正具有空间上下文和空间基础的像素，绝对是另一个重大步骤。
But generating pixels that are truly spatially contextualized and grounded is absolutely another major step.
而Atlas正是迈出了这非常艰难的一步。
And that is the very hard step that Atlas has taken.
我们当然可以继续深入探讨，对吧？
We definitely have you know, we can just keep going here, right?
比如，还有第四维度——时间，这会带来动态效果，还有更高保真的模拟和空间描绘。
Like, there is the fourth dimension of time, which will bring in dynamics, there is more higher fidelity simulation and delineation of the space.
所以这是空间智能路线图的一部分。
So this is part of the roadmap of spatial intelligence.
太好了。
Great.
对。
Yeah.
我确实很想深入聊聊这个方向的发展。
I mean, I definitely wanna, like, dig into, like, where this is going.
但首先，我们也许可以谈谈是怎么走到这一步的。
But first, maybe let's talk about getting here.
明白。
Mhmm.
World Labs 成立多久了？
How long has World Labs been in existence?
两年半。
Two and a half.
两年半。
Two and a
所以你们之前其实已经发布过一些模型了。
And so you you've actually released models before.
那为什么你们不直接跳到Atlas呢？
So what why didn't you just jump right to Atlas?
好。
Great.
因为
Because
它太神奇了。
it's so magic.
对吧？
Right?
是的。
Yeah.
贾斯汀的团队需要很多芯片。
Justin's team needs a lot of chips.
对。
Yeah.
要把这个东西真正做大，确实需要很多GPU。
You need a lot of GPUs to actually scale this thing up.
是的。
Yeah.
所以去年我们发布了我们的Marble世界模型，那是我们推出的第一个大型世界模型。
So what one of the last year, we we released our marble world model, and that was the first kind of big major world model that we put out.
它支撑了我们现在的Marble产品。
That that powers our current marble product.
而且Marble真的很酷。
And mar and marble is really cool.
Marble可以处理图像。
Marble can take images.
它还能处理视频。
It can take videos.
它可以接受文本提示，并用这些生成三维世界。
It can take text prompts and use these to generate three d worlds.
但Marble和Atlas之间最大的区别之一，正是输出的模态。
But one of the biggest differences between Marble and Atlas is exactly what is that output modality.
对。
Yeah.
所以Marble主要专注于高斯溅射作为输出表示。
So Marble was really focused on Gaussian splats as an output representation.
无论你输入什么，它都会输出一个以高斯溅射表示的三维世界。
So whatever you're inputting, it's gonna output a three d world represented as a as a Gaussian splat.
高斯溅射真的很有用。
And Gaussian splats are really useful.
对吧？
Right?
它们真的很棒。
They're really nice.
渲染起来很容易。
They're easy to render.
它们可以在移动设备和VR设备上高效渲染。
They are they they can they can render efficiently on mobile devices, on VR devices.
它们可以和游戏引擎、模拟引擎等无缝协作。
They can interoperate with other without with with game engines, with simulation engines.
所以高斯溅射有很多优点。
So there's a lot of nice things about Gaussian splats.
但是，你知道，我觉得这在之前的Marvel模型中是个瓶颈。
But, you know, I think that was kind of a bottleneck in the previous Marvel model.
所以我们在Atlas中做的是稍微重新设计了一下，我们意识到需要更早地分离这些模态，并让所有这些模态在模型中更统一地工作。
So what we did with Atlas is redesign the thing a little bit, and we realized that we need to bifurcate these modalities earlier and actually have these things all these modalities working in a more unified way in the model.
所以现在有了Atlas，基本的原始元素不再是生成一个高斯溅射世界。
So now with Atlas, the the the fundamental primitive is not like a could generate a Gaussian splat world.
就像我们说的，基本的原始元素是预测新视角。
The fundamental primitive is, as as we said, new view prediction.
对。
Yeah.
它可以生成RGB帧。
And that can generate RGB frames.
它可以生成三维，而且当我们需要的时候，可以用这些来生成漂亮的Gaussian splat世界。
That can generate three d, and we can use those to generate beautiful Gaussian splat worlds when you need them.
但当我们不需要的时候，我们不必让输出受限于Gaussian splats。
But we don't need to bottleneck our our our outputs through the Gaussian splats when we don't need to.
这实际上花了很多心血和汗水才弄明白，这些不同的表示方式各有什么优缺点。
And that was a that actually took a lot of, you know, blood, sweat, and tears to understand, like, what are all the pros and cons of these different representations.
对。
Yeah.
这是其中的一部分。
That that's one part of it.
另一部分就是，你得一步步往上爬。
The other part is you gotta, like, climb the scaling ladder.
对吧？
Right?
你得从小实验、小模型做起，慢慢积累信心，搞清楚什么能行、什么能扩大规模。
You gotta, like, work your way up and, like, do smaller experiments, do smaller models, like, to build your conviction on what's gonna work and what's gonna scale.
而且，如果你能一下子就知道什么能扩大规模，那当然直接去做就好了。
And there's there's, you know, if you could instantly know the right thing that's gonna scale, you know, you should just do that.
但当我们刚创办公司的时候，
But when when we started the company, left.
世界完全不一样。
The world was a very different place.
比如
Like
嗯，其实没有所谓的空间智能的规模法则。
There's well, there's no scaling law of spatial intelligence.
对。
Right.
所以，当我们创办公司的时候，世界的情况很不一样。
So, like, when we started the company, like, the world was in a very different place.
技术的情况也很不一样。
The tech was in a very different place.
我们有很多雄心壮志，想把它带到某个方向，但经过几次迭代，我们才找到这个我们认为真正能规模化扩展的方案。
We had a lot of ambitions about where we wanted it to go, but it took a couple it took a a couple iterations for us to hit upon this formulation that we thought is actually is actually, like, this is the one this this is the one that can scale up.
你知道，Ben，作为NERF的创造者，做了很多三维重建的工作，所以对我来说，并不是那么显而易见，如果你有多个视角，最终就能得到一个三维的东西，但你却以最终得到三维物体为职业。
You know, Ben, you know, being the creator of NERF and doing a lot of three d and reconstruction, so it's not so obvious to me that, like, if you have multiple views that you actually end up with a three d thing, but, like, you've kind of, like, made a career of ending up with a three d thing.
所以也许可以谈谈这一步。
So maybe talk a little bit about, like, kind of that step.
是的。
Yeah.
是的。
Yeah.
对。
Yeah.
就像你说的，我职业生涯中很多很多年，实际上大部分时间，都在做从图像生成三维物体的事情。
Mean, as you said, I've spent many, many years of my career, the vast majority of my career, actually working on producing three d things from images.
而且这其实是我们公司早期经常讨论的话题，比如，这种方法会不会是生成三维的方式？
And it's actually something we we talked a lot early on in the company even of, like, is this gonna be the approach that produces three d?
对吧？
Right?
我们要合成多个视角，然后从中构建三维吗？
Are we going to synthesize multiple views and then build three out of that?
还是直接尝试生成三维？
Are we gonna try to go direct to three d?
这个领域一直有很多不确定性，关于哪种方法最终会胜出，或者哪种方法能更早地发挥出最大的优势。
Like, there's been a lot of, uncertainty in the field around, like, which of those approaches kind of will win out or will kind of, like, reap the best advantages earlier on.
但我确实很有信心，因为看到了那种我几乎可以称之为暴力扩展的力量，虽然是在非常非常非常小的婴儿规模上，不是真正的模型扩展，而是我们过去三年里看到的密集重建的扩展。
But I did have a lot of conviction just from seeing the kind of power of what I would almost call the brute force scaling scaling in a very, very, very small baby scale, not like a real model scaling, but the scaling of dense reconstruction that we had seen happening over the past three years before.
所以，基本上我们
So, basically, we
为什么是密集重建？对。
Why is dense reconstruction Yeah.
密集？
Dense?
对。
Yeah.
密集，密集。因为我知道我们接下来要讨论稀疏，我想确保大家明白什么是密集，什么是稀疏。
Dense dense Because I know we're gonna talk about sparse, and I wanna make sure that people understand what is dense and what is sparse.
所以我觉得这其实即使在商业和业务层面，也是三维重建技术产品化的一个根本性挑战。
So I think this is actually even on the kind of, like, business and commercial side, I I think been one of the challenges of productizing three d reconstruction technology, like, a fundamental level.
对吧？
Right?
人们通常不会，随便说，比如我拍了三张这个物体的照片，或者我拍了六张这个房间的照片。
People kind of don't in in a in a casual sense, like, you think I took three photos of this object or I took six photos of this room.
比如说，我看着这些照片。
Like, I I look at the photos.
我脑子里能理解它们是怎么拼在一起的，能大概填补空白，然后明白。
I can understand in my mind, like, how those piece together, I can kinda fill in the gaps and get it.
但这两者之间从来没有真正的协调，一边是数据驱动的先验，另一边是那种暴力密集重建，其实密集重建更像科学或医学成像。
But there's just never been really any kind of reconciliation between those, like, really data driven priors and then the kind of brute force dense reconstruction, which it actually is much more akin to almost like scientific or medical imaging, what we did in dense reconstruction.
对吧？
Right?
基本上，你必须在重建中出现的每一个东西，都需要至少三到四个视角。
You basically have to say every single thing I wanna appear in this reconstruction, I need at least three or four views of it.
如果你想想，就算在这个房间里，比如麦克风下面、桌子下面、植物叶子的每个缝隙和角落。
And if you think about that, like, even just in this room, right, there's, like, under the microphone, under the table, between every different crack and crevice on the plant leaves.
对吧？
Right?
要真正拍到覆盖所有这些地方的画面，就得在房间里走来走去，非常繁琐和费力。
To actually truly get a picture that covers every one of those spots, it's just, like, very tedious and exhaustive effort to walk around the room.
我想你们都见过我在各个地方跑来跑去，拍这些东西。
I think you've all seen me running around various places, like, capturing them.
对。
Yeah.
这需要，你知道，对于训练有素的人来说，可能只需要几分钟。
It takes you know, for for someone who's well trained per se, like, it can take minutes.
但如果你给一个普通消费者，甚至是一些第一次尝试做这个的专业人士，一个普通的手机摄像头或拍摄设备，那他们大概要花一个小时。
But if you hand a casual consumer or even some, like, kind of professional trying to do this for the first time, an average, like, cell phone camera or capture device, like, it's gonna take them probably an hour.
我见过有人第一次尝试扫描一个多房间的环境，花了两个小时在里面走，才获得足够的覆盖。
I've seen someone for their first time trying to scan a multi room environment spend, like, two hours walking through it and get enough coverage.
那真是个非常非常累人又繁琐的过程。
And that's just just, like, very, very exhaustive and tedious loop.
所以，是的，当我们说密集，我们真的是指密集。
And so, yeah, when we say dense, we really mean dense.
就像这个房间一样。
It's like this room.
需要，比如，几百张照片。
Want, like, hundreds photos.
照片。
Of photos.
很多照片。
Many photos.
对吧？
Right?
我想，比如，拍几百张，200、300张照片来捕捉这个房间。
I want, like, hundreds, 200, 300 photos of this room to to capture it.
而我们想做的就是把它减少到，比如，三张。
And what we're trying to do is bring that down to, like, three.
三张。
Three.
对吧？
Right?
我们说的是，大概50到100倍的缩减。
Well, we're saying, like, 50, a 100 x reduction.
到了那个规模，就完全颠覆了之前的计算方式，比如你要重建什么样的场景。
And then that's at that scale where it just completely, like, flips that calculus on its head of, like, what type of captures you reconstruct.
你可以回到你已有的图像资料。
You can go back to existing imagery you have.
你可以用网上找到的东西，甚至能从中构建场景。
You can go to stuff you find on the Internet and even build scenes out of that.
你可以用随手的视频，挖掘出很多过去从未想过能重建的素材，然后回去把它们变成三维的，让它们活起来。
You can go to casual videos and, like, kind of unearth a lot of footage in the past who would never have treated as reconstructible, and go back and, like, bring it to life as three d potentially.
这是我们和Atlas一起玩了很多的东西。
This is something we've been playing around with a lot with Atlas.
对吧？
Right?
比如，拿一些老片段。
Like, taking old clips.
比如，我拿了很多以前从来没成功过的旧照片，把它们放进系统里，然后第一次看到了重建效果，或者拿旧照片，扔掉95%我拍的照片，然后想象出那些传统NeRF或Splat重建根本不可能得到的角度。
Like, I've taken a bunch of my own old captures that never worked before and then put them through the system and and then kinda seen a reconstruction over the first time or taken my old captures and thrown away 95% of the photos I took, and, you know, imagine angles that I never would have gotten from a traditional kind of like nerf or splat type reconstruction.
网站上演示中有一个被低估的地方，就是斯坦福的演示，Ben展示了从3到25张图片不等的情况。
One thing that's underappreciated on the website of the demos is the Stanford demo where Ben showed anywhere between three to 25 images.
你可以重建整个斯坦福广场。
You can reconstruct that entire Stanford quad.
但问题是，我们不得不从空中视角来展示。
But the thing is, we had to show it from an aerial view.
但每一张输入图片都是Ben站在地面上，从地面拍的。
But every single input image is Ben standing on the ground, taking a picture from the ground.
所以你看到的一切都是生成的，但遵循重建的规律。
So everything you see are generated, but according to the laws of reconstruction.
这真的很神奇。
And this is really magical.
这就是生成和重建需要从根本上相互配合，才能解决这个问题的地方。
And this is where, like, generation and reconstruction need to interplay in a really fundamental way to solve this problem.
因为按照Ben之前讲的传统方法，之所以需要那么多视角，是因为我需要多张图像，然后在三维空间里对这个点进行三角测量，从多个视角去看它。
Because under the classic kind of stuff that Ben was talking about, like, the reason you need so many views is because I need, like, multiple images, and I need to triangulate this point in three d space and see it from multiple viewpoints.
所以这意味着，传统方法必须这么做。
So that means, like, that's required in the traditional version.
反过来，任何在这些视角里没被拍到的部分，比如某个像素在输入视角里不可见，那它在三维重建里就会是个空洞。
And on the flip side, anything that wasn't captured in these views, like, any pixel that was not visible in one of the input views will be a hole in the three d reconstruction.
因为从根本上说，如果某个东西在输入视角里看不到，你就得靠想象去填补这些空白，这本质上就是一个生成的过程。
Because fundamentally, like, if a thing wasn't visible in the input views, you know, you need to imagine it to fill in the gaps, and that's fundamentally a generative process.
所以即使在这个房间里，就算让Ben拿着单反相机随便拍，让他拍几百张这个房间的照片，就算是做这种密集拍摄的世界级专家，也还是会漏掉一些地方。
So even in this room, even if we set Ben loose with a DSLR and, like, let him, like, capture, like, hundreds of views of this room, even the world expert on doing these dense captures is still gonna miss some spots.
比如他不可能拍到所有麦克风的底下，或者所有桌子的下面，或者所有椅子灯之间的缝隙。
Like, he's not gonna get, like, underneath all of the microphones or underneath all the tables or, like, in between all the chair lights.
不管拍多少视角，你总会漏掉一些东西。
You're always gonna miss something no matter how many views you get.
所以这就是为什么需要把生成作为模型里的另一种机制，因为你永远不可能拍到所有东西。
So that that's where you need generation as another mechanism in the model because you're never gonna get everything.
所以模型需要具备一定的生成能力，来想象——根据我看到的，先对能看到的部分进行三角测量，然后填补那些不可避免没被捕捉到的空白。
So you need to have some generative capacity for the model to imagine, oh, based on what I'm seeing, then, like, first triangulate what I what I can see, but then fill in the gaps of the stuff that inevitably inevitably was not captured.
对。
Yeah.
而且这里有个特别酷的点，就是大语言模型其实很早就明白了这一点。
And there there's something, like, super cool about this that LLMs have really understood this for for a long time.
对吧？
Right?
当时几乎就像是一场上下文长度的竞赛。
There was kind of almost these, like, context wars.
最初几年，就像是从128到256，再到512。
Like, the first couple of years, I it was like, oh, we got to one twenty eight to two fifty six to, like, five twelve.
我们做到了百万级别。
We got a million.
对吧？
Right?
现在大家都比较直观地理解了上下文长度的重要性，比如用编程模型的时候，你会把上下文长度调得很高。
And everyone kind of understands now at a pretty tangible level the value of, you you crank your context length to high when you're using your coding model.
这是个难题。
It's a hard problem.
大家都有这种感觉，但在图像和视频模型方面，还没有人用同样系统的方法去推动这一点。
Like, everyone has a feel for that, but, like, no one has pushed that at all on the image and video model side in the same kind of, like, principled way.
没有人会尝试把一小时的视频放进去，然后像大海捞针一样去检索第37分钟的那一帧。
Like, no one's out there trying to, like, put a, like, an hour long video through and do a needle in the haystack retrieval of, like, a frame at the thirty thirty seven minute mark.
而在重建和生成方面，其实也有同样的情况——重建就像是带长上下文的生成，你往里面放很多东西。
Whereas with reconstruction and generation, you actually have the same exact thing of, like, reconstruction is just, like, generation with a really long context, and you put a lot of stuff in it.
对。
Yep.
对吧？
Right?
就像，这就是那种方式。
Like, that's the way to
实际上就是构建一个连续体，把这两者连接起来。
actually build this continuum where you kinda bridge between those two things.
而且像Atlas，能做到这一点，是Marvel永远做不到的。
And, like, Atlas, like, being able to like, this is something we can never do with Marvel.
Marvel有个根本性的限制，就是实际上你塞不进超过几张图片。
Marvel had this kind of fundamental blocker of, like, you couldn't really jam more than honestly, like, a couple images in.
但Atlas，我可以拿64张图片的采集数据，然后对整个房子进行飞越浏览，所有内容都基于实际看到、几乎看到或稍微从缺失部分推断出来的信息。
But, Atlas, I can go and I can actually take, like, a 64 image capture and do, like, a a fly through of an entire house, and everything is grounded by being, you know, seen or, like, almost seen or, like, slightly extrapolated from what's not there.
但你得到的这些——你知道，我拿2000张图片采集的多房间房子，把它缩减到三四十个输入，飞越效果看起来基本一样，这在以前是完全不可想象的，而这全靠构建一个能优雅扩展的上下文窗口，你可以往里面塞东西。
But you're just getting these you know, I'm I'm taking captures I did with 2,000 images of a a multi room house and taking it down to, like, thirty, forty inputs, and the fly through looks like basically the same, and this is just like totally inconceivable before, and it's all enabled by building this gracefully scaling kind of context window that you can dump stuff into.
那么，是不是可以这样理解，就是你实际拍的那些稀疏的图片，嗯，
And and so is the way to think about it is like the the sparseness or the pictures that you physically took Mhmm.
然后Atlas作为模型生成其余的视角，接着你用经典的重建技术？
And then Atlas as a model creates the rest of the views, and then you use classic reconstruction techniques?
大致可以这样理解吗？
Is that roughly the way to think about it?
或者
Or
从某种意义上说。
In some sense.
对。
Yeah.
对。
Yeah.
我的意思是，那是Atlas的视频。
I mean, that's a video of Atlas.
就像你可以把任意多的输入，甚至减少到单一视角，然后几乎可以把Atlas当作一个渲染引擎，生成你想要的任何其他内容。
It's like you can take however many inputs you use you have down to, like, a single view, and then you can almost use Atlas as this this rendering engine to produce anything else you want.
对吧？
Right?
你可以像操作虚拟摄像机一样在其中导航。
You can you can navigate it Like the virtual camera.
对。
Yeah.
没错。
Exactly.
你可以直接说，比如，好。
Can just you can say, like, okay.
我这里有
I have
一张图片。
a picture here.
我想
I want
把图片放在那里、那里、那里。
the picture there, there, there.
你可以做几个这样的，然后就能编排一个密集的穿越飞行。
You can make a couple of those, then you can stage a dense fly through.
你可以按顺序来做，因为这是一个自回归模型。
You can do this in sequence because it's an autoregressive model.
这取决于你，对吧，在生成过程中，你可以选择性地把内容添加到上下文里。
It's up to you, right, to kind of pick and choose what you add interactively into the context as you generate.
我觉得最让我震惊的是，你看，我的思维模型很简单。
I mean, thing that I just blows my mind is, listen, I just have a very simple mental model.
我有四张图片，然后需要模型在它们之间进行推断，而且当你控制时，它必须匹配。
I have four pictures, and then I've gotta have the model extrapolate between them, and then it has to fit when you're in control.
比如，它必须是三维的，因为
Like, it's gotta be three d because
对。
Yeah.
就是
It's
而且我总觉得这些扩散模型，视觉上很棒，但不够精确。
Like, and I always think of these diffusion models as, like, being visually great but not accurate.
展开剩余字幕（还有 369 条）
所以，我甚至不知道这算不算一个问题，但，它到底是怎么做到的？
And so, like and and I don't even know if there's a question here, but, like, how is it?
就是，整个房间都能对得上。
Like, the room fits.
它是怎么做到在三维空间里保持一致的？
Like, how is it that it's three three d consistent?
就只是靠大量的数据吗？
Is it just lots of data?
或者说，对。
Or Yeah.
我觉得，部分原因是对规模假设的信念。
I mean, it's a part partially, it's a belief in the scaling hypothesis.
对吧？
Right?
就像，你知道的
Like, you know
这就是命运。
It's fate.
顺便问一下，但我必须得问。
Did you by the way, but I have to ask.
你当初开始做这件事的时候，你知道它会成功吗？
When you set out to do this, did you know it was gonna work?
我相当确定。
I was pretty sure.
那太好了。
That's so so okay.
什么
What
你确定吗？
were you sure?
我觉得我们三个人对扫描定律都深信不疑。
I think three of us have total conviction about the scanning law.
我觉得我们确实是这样。
That that I think we do.
我确实认为，具体的架构选择和数据混合才是细节中的魔鬼。
I do think the exact architecture choices and data mixtures is where the devils are in the details.
我亲眼看着贾斯汀和他的团队从‘我们真不知道这要花多久’到‘哦，也许有生命迹象了’，再到‘哇，这真的能行’。
I have watched Justin and his team going from, we really don't know how long this is gonna take to, oh, maybe sign of life to, wow, this is gonna work.
所以还没有人做到过。
So no one has done it.
但我认为有两个假设。
But I think the hypothesis two hypothesis.
一个是缩放定律假设。
One is scaling law hypothesis.
另一个是下一视角预测。
The other one is next viewpoint prediction.
我们主要对这两点有信心。
We had conviction of these two things primarily on.
所以我当时非常确信这会成功。
So I I think I was very convicted that it was going to work.
但我没想到会这么快就达到这么好的效果。
I was not sure it was gonna work this well at this fast.
对吧？
Right?
我当时觉得，我们有可能做到。
Like, I thought there's a chance that we do this.
但可能不太确定的是，第一次用新架构和新范式预训练一个新模型。
Maybe it's not clear that, like, the first cycle of pretraining a new model with a new architecture and a new paradigm.
第一次就能成功，这简直不可思议。
Like, the first cycle of that working is insane.
所以我当时觉得，可能需要多轮预训练才能达到我们想要的质量水平。
So I thought there was a chance in which we had to we might have had to do a couple more turns of of that pretraining cycle before we got to the level of quality we were we we wanted.
我们现在是不是已经接近这种架构方法的扩展极限了，需要另一个突破，还是说还没有？
Is there is it are we kind of, like, at the end of, like, the scaling for this architectural approach when we need another breakthrough, or is there No.
不是。
No.
我们才刚开始。
We're at the beginning.
真的吗？
Really?
是的。
Yes.
在不改变架构的情况下。
Without changing without changing the architecture.
对。
Yeah.
我们基本上才刚开始。
We're basically at the beginning.
对。
Yeah.
我觉得我们基本上还在起步阶段，目前主要受限于算力。
I think we're basically at the beginning, and we're basically limited by compute at this point.
对吧？
Right?
就像李飞飞喜欢指出的，数据非常重要，但每件事都有瓶颈，而我认为继续扩大规模的主要瓶颈实际上是训练算力。
Like, data is very important as Fei Fei likes to point out, but, like, everything has a bottleneck, and I think the main bottleneck on continuing to scale this thing is actually training compute.
对吧？
Right?
比如在开发过程中，我们训练了一系列模型。
Like, during development, we trained a sequence of models.
我们在博客文章里稍微读到过，但我们训练了几个模型，就像是扩展阶梯的前几级。
We read about this in the blog post a little bit, but we trained a couple models that, like, the first couple rungs of the scaling ladder.
每次我们把模型做得更大，每次训练时间更长，每次用更多的芯片，模型都明显变得更好了。
And each time we made the model bigger and each time we trained it for longer, each time we put it on more chips, like, it got significantly better.
对。
Right.
我们喜欢的模型大小，就是博客里展示的那个模型，显然是我们训练过的最大最好的一个，但限制它的不是规模、数据或类似的东西。
And the model size that we like, the model that we showed in the blog post is obviously the biggest and best one that we trained, but the thing that was limiting it was not the scale or the data or anything like that.
实际上就是，我们有一个想发布这个东西的截止日期，所以我们就根据那个截止日期，倒推我们能训练多少。
It was literally, like, we had a deadline of when we wanted to release this thing, and therefore we backed up what we could afford to train in after that deadline.
但这里有个内幕故事，对吧？
But here's a little bit of an insider story, right?
就像Justin和团队，他们从较小的模型开始训练，然后稍微大一点，你知道，他们有这些路线图。
Like Justin and team are training from the smaller and slightly bigger, you know, having these roadmaps.
然后夏天有一天，初夏的时候，那还不是现在的Atlas模型大小，是一个更小的模型。
And then there was one day in summer, early summer, that it's not even the current Atlas model size, it's a smaller model.
然后Ben和Justin，Ben把它输入到视角生成中。
And then Ben, Justin, Ben feed it into the viewpoint generation.
还记得那个著名的桌子，就是NERF论文和很多论文里的花园桌子，一夜之间我收到了Slack消息。
And remember that famous table, the garden table for NERF paper and many papers that overnight I got a slack.
我的意思是，大家都看到了Ben发的Slack，说摄影师从桌子底下飞过去了。
I mean, all saw the slack from Ben that the cameraman flew through under the table.
带着一个足球。
With a soccer ball.
是的。
Yes.
带着一个足球。
With a soccer ball.
足球是自然出现的，还是
Is soccer ball emergent or is
那是关系里的？
that in the relationship?
哦，没关系。
Oh, that's okay.
那是真的。
It was real.
对吧？
Right?
好的。
Okay.
那天早上，我们三个人对视一眼，说：就是它了，我们要把这个做出来。
That morning, the three of us looked at each other in the eyes and say, That's it, we're gonna build this.
我们五秒钟之内就做了决定。
We made a decision within five seconds.
这个结果，从来没有人见过。
This is just a no one has ever seen this result.
Ben，你能具体讲讲使用场景吗？
Ben, can you talk through maybe more specifically the use cases?
World Labs 一直以来有很多创意工作者用户，他们用它来保持一致性，比如在二维图像、电影、三维和游戏等方面。
So World Labs has historically had a lot of users that were creatives, and they use it for consistency in, you know, like, whatever, two d images and for movies and for three d and for games, etcetera.
那么，你能谈谈这如何扩展了现有的使用场景吗？
And so maybe can you talk about how this extends the use cases that are catered to the existing ones?
然后，我其实想聊聊机器人技术。
And then we'll I would like to talk about robotics, actually.
明白。
Mhmm.
对。
Yeah.
当然。
Sure.
是的。
Yeah.
我是说，这还挺有意思的。
I mean, it it's kinda funny.
其实，我们看到的很多人使用Marvel的主要方式之一，正好就契合了这个新的视图预测场景。
Actually, one of the kind of main ways we even saw people using Marvel plays exactly into this new view prediction case.
比如，一个
Like, a
我们很多
lot of our
Marvel是之前的那个。
Marvel being the previous.
抱歉。
Sorry.
对。
Yeah.
Marvel，我们之前的产品。
Marvel, our previous product.
之前的Marvel。
Previous Marvel.
比如，人们会拿那个产品，放一张图片进去，得到一个完整的三维场景，作为高斯泼溅，然后从不同视角截几张图就走了。
Like, people would take that product, put an image in, get a full three d scene as a Gaussian splat, take a couple screenshots of it from different points of view and leave.
对吧？
Right?
然后我们就想，
And we're like,
我们可以直接把这些数据做成图像
we can just make those images of that data
在数据专业版里。
in data pro.
对吧？
Right?
所以我觉得，而且你知道，有很多
So so I think, like and, you know, there's a lot
质量损失在那里。
of degradation there.
他们就说，嗯，
They're like, well,
这个高斯泼溅效果可以更好。
the splat could look better.
然后我们就说，好吧。
And it's like, okay.
如果我们直接生成那些视角，而不需要精确的控制方式呢？
What if we just generatively model those viewpoints without exact modality of control?
所以我觉得，即使是像视图合成这样的核心能力，长期以来也一直是个学术问题。
So I think, like, even that core capability of just, like, view synthesis, it's sort of been this academic problem for a long time.
但如果说要做这种非常密集的采集，那生成式视图合成其实是个相对较新的问题。
But in the sense of, oh, you're gonna do this really dense capture, like like, generative view synthesis is a relatively quite a new problem.
而且我们看到很多人在创意流程中，对吧，他们有多阶段的工作流程。
And we just see so many people who, in this creative pipeline, right, people have a multistage workflow.
对吧？
Right?
我觉得没有人会只用一个大一统的模型来完成整个任务，甚至都不会去看一眼答案什么的。
I don't think there's a single person out there using one monolithic model, not even see the answer or whatever for for their entire, task.
人们会有那种，你知道的，一堆故事板和情绪板，里面是从他们最喜欢的图像模型集合里挑出来的图片。
People will have this, like, you know, kind of bunch of storyboards and mood boards of images they pull out from, like, their favorite collection of image models.
然后他们会去用不同的视频工具，把这些图片作为关键帧组合起来。
And then they'll go to different video tools and, like, build those together as key frames.
然后他们之后会去剪辑和编辑这些内容。
And then they'll go on, like, clip and edit those later.
对吧？
Right?
所以我们看到，虽然是个比较小众但非常具体的用例，就是让Marble提供一种保证，让你的生成内容能基于一个三维一致的世界。
So we were seeing this, like, sort of, you know, niche, but very specific use case for marble as just providing that, like, sanity that you can ground your generations in some kind of three d consistent world.
对吧？
Right?
你知道，我试过用各种图像模型，让它们给我一个房间的不同视角。
People, you know, I don't I've I've thought was with various image models to ask them to, like, give me different viewpoints of a room.
每次你都能看到，哦，东西好像动了。
And every time you can just look and see, oh, things kinda moved around.
就是不稳定。
Like, it's not stable.
而且，哪怕只是这一个用例的苗头，我觉得也暗示了表面之下隐藏着价值。
And, like, even that one seed of a use case, I think, kinda signals that there's this value, and there's hiding under the surface there.
就是说，几十年来人们已经习惯了持久的三维状态，比如在虚拟环境中模拟他们在现实世界会做的事情，有舞台、道具和各种元素，不管是拍电影、做节目、拍广告，还是搭建游戏场景。
Like, there's just decades of people being used to persistent three d state, like, virtually modeling what they would be doing in the real world and having, you know, a stage and props and, like, elements there, whether it is for a movie or a show or marketing shot or, like, building out game environments.
这种状态性和持久性对于人们如何进行空间推理以及随时间推移开发环境来说非常关键。
Like, this this statefulness and persistence is so key in how people think about spatial reasoning and, like, developing an environment over time.
人们不会以那种转瞬即逝的方式来思考，比如生成一个东西，随便改改，然后扔掉，只保留文字提示。
Like, people don't think in this ephemeral, like, generate a thing, turn anything, like, just throw it away, keep my text prompts.
人们想要构建一个资产集合，并以此方式建模一个世界。
Like, people wanna build this, like, collection of assets and, like, model a world in that way.
所以我们试图通过这种面部上下文机制和其他方式，提供那种级别的控制和精确度，以及接收不同模态输入的能力，从后置图像开始。
So we're we're trying to provide, like, again, with with this facial context mechanism and other things, like, we're trying to provide that level of control and precision and the ability to ingest different modalities of input starting with the post images.
但你知道，我们想给人们更多控制权，让他们能控制场景中元素、编辑、交互等，随着我们不断前进。
But, you know, we wanna give people more control over the elements of the things in the scenes they're looking at and editing and interaction and all that as we go forward.
对。
Yeah.
我认为这不仅能解锁我们已经看到的那些领域的更多用例，还能扩展到任何人们想要创建虚拟复制或预想现实世界空间的地方，比如建筑和施工领域。
And I think that that, it unlocks, like, further use cases in those areas we're already seeing, but, also expanding out into kind of any place people want to create a virtual replication or or, like, you know, a pre imagination of a real world space they need to build, right, for architecture and construction.
比如，我之前跟一个做展会摊位的人聊过。
Like, I talked to a guy at some point building booths for conferences.
对吧？
Right?
世界上有很多东西你可能没想过需要制造出来，而每一个基本上都要经过一个相当费劲的虚拟设计阶段。
There's just so many things in the world you don't think about need to be fabricated, and every single one of those basically goes through this, like, pretty painstaking virtual design phase.
在这个过程中，进入三维软件的那部分，现在算是比较辛苦、费力的环节之一。
And of that process, like, the part where you go into three d software is kind of one of the most, like, arduous and, like, labor intensive parts right now.
比如说，把创意总监、设计总监或者建筑师给的文字评论、草图或者非常快速的反馈，再映射回三维模型里，这大概占了工作量的95%。
Like like, taking feedback on a three d design from kind of, like, verbal commentary or sketches or really, really quick stuff you got from like a creative director or like a design director or an architect or whatever, like mapping that back into the three d representation is like 95% of the work.
对吧？
Right?
你可能开个会，拿到反馈，然后回去改一个星期的版本。
You can have a meeting, get feedback, and then you go back and do a week of revisions.
这主要是因为我们的软件到现在已经用了好几十年了，一直都没变得像玩乐高、捏陶土、用手做东西或者用铅笔画草图那么直观。
And that's just because, like, our software is kind of decades old at this point, and it's it's just never became as intuitive as, you know, playing with Legos or, like, pottery or doing this stuff with your hands or sketching with a pencil.
对。
Mhmm.
而这里正是AI真正能为人们的工作流程释放巨大价值的地方，无论是创意应用，还是更偏工业或设计领域，或者其他什么。
And this is the real place where AI can actually unlock a ton of value for people in their process, whether it's a creative application or something more industrial or design or or whatever.
而这真的激励我去打造不同版本的模型，来满足这些不同类型人群的需求。
And that, like, really motivates me to kind of build different flavors of of our model to cater to those kind of people.
是的。
Yeah.
我能理解这对创意工作者有多大帮助，因为漫威就是这么做的，而且这也能延伸到设计或建筑等领域。
Can understand how it helps with the creatives because like, Marvel did that, and also how that extends to things like design or architecture.
但是飞，你收购了一家机器人公司，所以我们……
But Fei, you acquired a robotics company, and so it's We
我们刚才也提到了这个。
less just talked about it too.
我知道，我知道。
I know, I know.
但对我来说，尤其是在Atlas的背景下，这跟机器人技术怎么联系起来，还不太明显。
But it's less obvious to me, especially in the context of Atlas, like how that maps to robotics.
所以如果你不介意的话，能不能简单解释一下。
So if you wouldn't mind just penciling that out.
是的，实际上Atlas是这个难题的关键部分。
Yeah, actually Atlas is a key part of the puzzle.
我们收购了一家以前叫SYNNEX的公司。
So we acquired this company that was formerly known as SYNNEX.
他们的核心技术是什么？
And what is their key technology?
目前他们的核心技术是一套从现实到模拟，再从模拟到现实的系统。
Right now their key technology is a system that goes from real to SIM and then SIM to real.
这在机器人领域意味着什么呢？
And what does that mean in robotic situation?
比如说，你想训练一个机械臂在工业环境中完成布线工作。
You want to train a robotic arm to figure out how to do cabling, let's say, in an industrial setting.
首先，你需要大量的数据来训练机器人策略，让它学会做这些布线工作。
Well, you need a whole bunch of data to first train a robotic policy to do these cables cabling activity.
然后你要评估这个机器人策略是否做得好。
And then you wanna evaluate if the robotic policy is doing a good job.
接着把机器人部署到布线环境中。
And then you deploy the robot into the cabling environment.
对。
Yeah.
为了训练，这家公司，Cynix，以及我们现在的机器人团队，以前做的就是Ben说的那种密集重建。
In order to train, what you what this company, Cynix, and now our robotics team used to be doing is doing exactly what Ben was saying, dense reconstruction.
你拿——哦，有意思。
You take Oh, interesting.
所以现在，拍下环境的照片，然后尝试重建那个环境，这个过程极其痛苦，耗时又费力，而且真的严重拖慢了机器人从仿真到现实的速度，对吧？
So now pictures of a situation and then try to reconstruct that environment is excruciatingly painful, takes a long time, laborious, and it really blocks the velocity of robotic simulation real to sudden, right?
所以Atlas真的是这方面的下一代技术。
So Atlas really is the next generation technology for for that.
而且这不只是针对机器人布线或者别的什么。
And this is not just for robotics cabling or anything.
我们应该把视野放宽，认识到现在机器人领域最大的问题其实是数据。
We should zoom out and recognize the biggest problem right now in robotics is actually data.
总有一天会是芯片的问题，因为现在收集机器人在真实环境中运行的数据太难了。
One day it'll be chips, for Because now it's it's so hard to collect real world data where robots are operating.
为了做到这一点，你不仅需要收集数据，比如布线场景或者洗碗场景之类的，还有一个非常重要的步骤叫做随机化，就是你必须把同一个环境拿来，然后随机化它的条件。
In order to not only you need to collect the data of, let's say, the cabling situation or dishwashing situation or whatever, there is also a very important step called randomization, is that you have to take the same environment and then randomize the condition.
这样线缆就不会只能朝一个方向弯，它可以朝不同的方向弯，或者盒子可以有不同的大小、颜色、不同的盖子等等，或者出现在场景的不同位置。
So the cable doesn't literally only bend this way, it can bend a different way, or the box can have different sizes, colors, different lids and all that, or in different parts of the scene.
所以你必须经历一个从真实到模拟的过程，才能获得足够的数据，再加上你能从互联网上获取的其他数据。
So you have to go through a real to sim situation in order to get enough of that data in addition to other data you can get from internet.
所以这个真实到模拟的步骤会大大受益于Atlas。
So this real to some step will be really helped by Atlas.
这只是第一步，满足当前技术下的机器人需求，因为我们还没有一个足够强大的前沿基础模型来支持机器人技术。
That's just the first part of this, is meeting the robotics needs in the current technology because we don't yet have a frontier foundation model that's robust enough for robotics.
但Atlas是一个全能模型，是一个多模态模型。
But Atlas is an omni model, is a multimodal model.
它能接收不同类型的输入，并生成不同类型的输出。
It takes different kinds of input and generates different kinds of output.
你完全可以想象下一步是Atlas接收动态数据。
You can totally imagine the next step is Atlas taking in data that's dynamical.
这真的可以开始弥合动作规划、机器人与Atlas输出之间的差距。
And that can really start to bridge the gap between action planning and robotics and the Atlas output.
所以这是路线图上的下一步。
So that's on that roadmap.
你打算
You gonna
说点什么吗？
say something?
我想说的是，训练机器人策略与之前见过的任何AI应用都有根本性的不同。
I was gonna say there's something fundamentally different about training a robotics policy compared to really any other application in AI we've seen before.
这就像，如果你在生成一段代码，或者生成一张图片，或者生成一段视频，模型本质上是在创造这个产物。
And that's like, if you're generating a piece of code, like you're generating an image, you're generating a video, I'm I I the model is fundamentally creating this this artifact.
对。
Yeah.
而且这个产物，网上或者别的地方有很多这样的例子可以收集。
And that artifact, like, there's a lot of examples of artifacts that you can go out on the web or somewhere and collect.
对吧？
Right?
你想生成图片？
You wanna generate images?
网上有很多图片。
There's lot of images out there.
你想生成视频？
You wanna generate videos?
网上有很多视频。
There's a lot of videos out there.
你想生成一个代码库吗？
You wanna generate a code base?
网上有很多代码库可以学习。
There's lot of code bases out there you can learn from.
对。
Yeah.
机器人策略从根本上来说是不同的。
A robotics policy is something fundamentally different.
它不是产生一个静态的东西。
It's not producing a static thing.
相反，它是一个策略，要进入现实世界，采取行动，并努力实现一个目标。
It's instead a policy that's gonna go out into the world, make actions, and, like, try to achieve a goal.
而世界并不总是按照你预期的方式回应。
And the world's not always gonna respond the way you expect.
对吧？
Right?
意外的事情总会发生。
Unexpected stuff is gonna happen.
所以，机器人策略本质上就是一个在现实世界中与真实环境互动的智能体，过程中会发生各种事情。
So a robotics policy is, like, fundamentally an agent that is out in the real world interacting with the real world, and stuff happens.
因此，关键的一点是，这些策略在训练时需要接触到部署过程中可能出现的所有问题。
So you need like, a critical part of that is the those those policies during training need to be exposed to every possible thing that could go wrong during their deployment.
这就是为什么仿真对机器人技术如此重要。
And that's where simulation is really key for robotics.
对吧？
Right?
所以，这里有两个角度。
So there's the and then there's two angles on that.
一个是传统的仿真方法。
Like, one is the kind of classical simulation.
你可以去用你喜欢的物理引擎，然后像人类设计师那样创造性地想象，在完成这个任务时可能会遇到哪些场景？
You can go out you can go and, like, go to your favorite physics engine and, like, try to imagine creatively as a human designer, what are all the scenarios that might happen in this in this in this, when achieving this task?
然后试着写明确的代码来模拟所有这些情况。
And then try to write explicit code that models them all.
这是一种思路，而且对编码智能体来说是个有趣的思路。
That that's one angle, and that's an interesting angle with coding agents.
实际上，这也会被大大增强。
Like, that actually gets supercharged too.
但还有另一种思路，就是尝试更数据驱动的模拟。
But there's another angle, which is try to more data driven simulation.
对吧？
Right?
比如，我们能不能有一个学习模型，能理解环境，理解世界会如何对动作做出反应？
Like, maybe we can can we have a learned model that can understand how the environment, how the world is gonna respond to actions?
而且有时候它可能会以意想不到的方式反应。
And maybe it might respond in unexpected ways sometimes.
那么，我们能不能构建这些神经模拟器，用尽可能多的数据来训练它们，然后用这些神经模拟器，这些学习到的神经模拟器，作为模拟平台来训练机器人策略呢？
Then could we build these neural simulators that are trained on as much data as we can, then use these neural simulators, these learned neural simulators, you know, as a simulation bed to train robotic policies?
这确实是Atlas未来一个非常有趣的发展方向。
And that's that's a really interesting future direction of Atlas.
但事情并不止于此。
But but then it doesn't stop there.
对吧？
Right?
所以，我知道。
So I I know.
所以，一旦你有了这个学习模拟器，它其实已经，嗯，怎么说呢，是的。
So so but once you have, you know, this learn simulator, like, this learn simulator kind of already has in its, like Yeah.
就像有了一个心智模型。
Mental brain.
它理解这个世界。
Like, it understands the world.
它理解世界会对动作做出怎样的反应。
It understands how the world is gonna respond to actions.
那为什么模拟器本身不能成为规划器呢？
And why doesn't the simulator itself become the planner?
对。
Yep.
对吧？
Right?
就像，这也是我们围绕世界模型及其通用性所持有的核心论点，那就是模型应该理解一些核心内容，比如生成世界、模拟世界、理解它们在不同搜索情境中的表现，而且，你知道，理解世界如何对行动做出反应，与想象我需要采取什么样的行动来让世界以特定方式做出反应，是高度相关的。
Like, same and that's that's kind of the core thesis that we've had around world models and their generality, that there are some core stuff that a model should understand around generating worlds, simulating them, understanding how they appear in different search situations, and, you know, understanding how the world's gonna respond to an action is highly related to imagining what kind of action I need to take to make the world respond in a particular way.
对。
Yep.
顺便说一句，我收到的一条反馈是，恭喜你们发布，反响非常积极，我觉得这可能是今年最重要的模型发布，而且，你知道，我一直说好话，但有一位这个领域的专家，我发短信问他，你觉得怎么样？
One piece of feedback that I got, by the way, congrats on the launch, it was overwhelmingly positive, I think it was probably the most significant model launch this year, and know, I've always said glowing things, but one person who's an expert in this space who I texted was like, What'd you think?
他说，太棒了，太神奇了，动态效果惊人。
And it's a great person, it's fantastic, it's amazing, dynamics.
所以至少在机器人领域是这样，但总的来说，拥有一个会动的世界似乎是理想的。
And so it would seem to be at least the robotics case, but generally, it was kind of ideal to actually have a world that moves.
那能不能稍微聊聊这个，然后还有其他你愿意分享的未来方向，你觉得值得聊一聊的。
And so maybe talk a little bit about that and then any other future directions that, a, you're comfortable sharing but you're thinking worth talking through.
对。
Yeah.
我是说，动态这块肯定会实现的。
I mean, like, dynamics is clearly gonna happen.
其实，我们已经有初步的动态了。
Like, actually We have baby dynamics.
我们确实已经有初步的动态了。
We actually do have baby dynamics already.
我觉得这一点大家可能没太注意到，因为博客文章里没有特别强调。
And this is something I think people didn't quite appreciate if they didn't really highlight in the blog post.
但是，之前的Marvel世界模型，基本上是静态的。
But, like, the previous Marvel world model, it was, like, fundamentally static.
对。
Yeah.
就是，这个模型完全处理不了任何动态变化。
Like, the the the model just, like, could not handle any dynamics at all.
而且这从一开始就融入了模型架构和训练过程中。
And that was just, like, baked into the model architecture, baked into the training.
整个东西本质上就是静态的。
Like, the whole thing was fundamentally static.
对。
Yeah.
我们早就知道这是Marvel之后的一个大问题，而且在Atlas中已经解决了。
We already knew that that was a big problem post Marvel, and we already fixed it in Atlas.
对吧？
Right?
Atlas的架构从根本上是支持动态的。
Like, Atlas architecture is already fundamentally supports dynamics.
Atlas的训练数据本身就包含动态内容。
The Atlas training data fundamentally has dynamics.
而且如果你仔细看我们发的一些视频，实际上，
And if you look carefully in some of the videos that we even posted Actually,
我看到了你发的那个视频
it I saw you see the movie
有点快。
a little fast.
对。
Yeah.
波浪。
The waves.
水波。
The water waves.
对。
Yeah.
所以，比如有些
So, like, some
比如有些例子，水里有波浪，就像那些航拍视角里看到的那样。
of the examples, there's, like, waves in the water, like, in some of the, like, air generated aerial views.
还有小车在动来动去。
There's, like, little cars moving around.
所以，动态效果其实已经包含在这个模型里了。
So, like, dynamics is actually already in this model.
但我说过，顺便提一下，如果你要从多个图像重建三维结构，动态效果对我来说似乎很有问题。
But I said I said, by the way, dynamics seems very problematic to me if you're trying to reconstruct three d from multiple is.
是的。
Is.
对吧？
Right?
所以，就像，
And so, like,
这些东西是不是互相矛盾？
are these things, like, at odds?
或者不是。
Or No.
所以，实际上我们这里的一个观点是，如果你要做基础的三维重建，你其实希望没有动态。
So so actually, one of our theses here is that, like, you know, if you're gonna do fundamental three d reconstruction, you actually wanna have no dynamics.
就是说，你希望能够建模场景的精确视图，时间完全冻结。
Like, you wanna be able to model, like, exact views of the scene with exact frozen time.
对。
Right.
然后，这其实是我们之前那个大理石方法的一个问题。
Then, like, this is actually kind of a problem with our previous marble approach.
对吧？
Right?
在那里，你可以尝试找完全静态的数据，但那真的很难扩展，也很难获得更多。
Like, there, like, you can try to find data that's fully static, but that's really hard to scale and really hard to get more of.
我们意识到的是，即使最终我想要静态输出，获得它的最好方式其实是让模型接触动态。
And the thing we realized is that even in the case where I want static output in the end, the best way to get it is actually expose the model to dynamics.
对吧？
Right?
就是说，把模型暴露在尽可能多的动态和静态数据里，让模型自己学会怎么把动态的部分分离出来。
Like, expose the model to as much dynamic stuff as you got, as much static stuff as you got, and let the model figure out how to factor out the dynamic stuff.
所以，尤其是在……哦，有意思。
So in in especially in, like, the Oh, interesting.
对。
Yeah.
这个，实际上，在Atlas预训练阶段，它就已经见过大量的动态数据了。
This like and this is actually so, again, in that Atlas pretraining already, like, it it saw a ton of dynamics in the pretraining already.
然后，我们针对这个检查点和这次发布做的后训练，更侧重于静态内容，更侧重于空间移动，而不是时间变化。
Then the post training that we did specific to this checkpoint and this release was focused a lot more on on static stuff, focused a lot more on spatial movement and less on temporal.
但是，我们其实已经……我相当确定，这个预训练检查点里已经蕴含了很多潜在的动态信息。
But, like, we already we already like, I'm pretty sure this the the the pre trained checkpoint already has a lot of latent dynamics in it.
对。
Yeah.
这是我们未来要大力改进的地方。
And this is something we're gonna improve quite a lot going forward.
那么，Ben，这是不是意味着我们会得到四维视频？
So so, Ben, does that mean we're gonna get four d video?
就是可以到处走动的那种？
Just gonna like go walk around?
我是说，
I mean,
我，你可以看到他们脸上的
I You can see the smile on their
笑容。
face.
所以，我其实能看到，如果你现在就停下来，只做更大、更快、更好的东西，你几乎可以建立一个完整的行业。
So I I actually can see if, like, you just stopped now and you only did kind of bigger, you know, faster, better, you could build almost an entire industry.
这感觉像是一个非常基础的模块，然后如果你什么都不做的话。
It feels like a very horizontal primitive, and then if you did nothing else.
但是，除了那种更大、更快的东西之外，还有没有其他让你感到兴奋的应用方向？你关注的那些应用，好像更偏向于……对。
But are there other things that are not just kind of bigger, faster that you're excited about for the applications you're focused on, which tend to be kind of more on the Yeah.
就是那种内容创作、三维方面的？
Kind of content creative three d side?
对。
Yeah.
我真的很兴奋能推动这种多模态的发展。
I'm really excited about pushing that kind of multimodal aspect.
我觉得不同的控制方式在这里非常关键。
I think different modes of control is so critical here.
我觉得，尤其是在学术界，大家非常低估了给这些模型添加控制条件的重要性，这样才能把模型内部的东西提取出来。
I think, like, it's super underappreciated, especially in the academic community, how critical it is to add control conditioning to these models to kind of get out what's inside.
说实话，这种动态精确静态……我不太
I mean, honestly, this dynamic precise static is I'm not
确定我甚至理解那些……对。
sure I even understand what those Yeah.
用通俗的话说，就是不同的实例。
Mean, in layman's language is Different instance.
可编辑性。
Editability.
对。
Yeah.
我觉得可编辑性才是关键。
I think editability is is the key here.
对。
Yeah.
所以，这个我们在单图模型里已经见过了。
So, I mean, this is something we've seen in, like, sort of single image models.
今年开始，视频模型也开始解锁这个能力，就是那种多轮交互的感觉，或者说很直观地理解我的意思，比如我想要这个人、这个物体、这件事发生，然后把这些都组合在一起，生成一个混合体，而不用手动去操作很多步骤。
And starting this year in video models is starting to be unlocked in terms of, oh, like, getting that flavor of, like, multi turn or really, like, intuitively interpreting, like, I want, like, this person and this object and this thing to happen and kind of combining those all together in, one pastiche without having to do a lot of, like, manual work with the system.
就像它直接理解你的意图，前沿的图像模型在编辑方面已经差不多能做到这一点了，对吧。
Like, it just interprets it, like, kind of frontier image models are kind of there, right, for in terms of editing.
但我们在视频领域还没看到这种能力那么强地推广开来，更不用说世界模型了。
But we haven't seen that propagate out as strongly into video yet and then into world models.
对吧？
Right?
我们确实看到了一些比较玩具级的例子，比如输入一句话，然后一只恐龙就出现了，这些实时模型能做到这样。
We've seen some really kind of toy examples of, oh, I can, like, put in a sentence and, like, you know, a dinosaur appears or something with these, like, sort of real time models.
但我想把它提升到真正的工业级强度，因为关键在于，你得增加控制，但又不能牺牲模型的质量，否则它基本上就只是个花招。
But I wanna, like, turn that up to really industrial strength and make that because, like, the the trick here is you gotta add control but not compromise the quality of the model, or it just becomes a a party trick, basically.
就是说，如果你给用户额外的控制选项，但质量下降了，没人会认真考虑用你的模型去替代他们正在用的前沿视频模型。
Like, it's like no one is going to seriously think about swapping their, like, cutting edge frontier video model usage for your model if you give them extra knobs, but the quality degrades.
所以我觉得这真的是一场博弈，怎么在保持我们当前模型输出质量的高标准的同时，加上人们要求的有趣功能，比如与场景互动、控制布局、控制物体和场景中事物的身份，或者控制时间。
So I think it's really that game of, like, how can we maintain, like, the high bar we've set was the outputs we're able to get in the current model and then add all kinds of interesting stuff that people ask us for in terms of, like, I want to interact with the scene or control the layout or control, like, the identity of the objects and the things that we're seeing within there or control time.
对吧？
Right?
我觉得这正是一个切入点，能开启大量有趣的产品和界面设计工作。
And I think that's, like, an access where it opens up like a ton of really interesting product and interface work.
你在那里添加的复杂性和丰富性，真的能让你几乎从零开始重新设计人们与计算机中有状态的三维世界互动的方式。
The more complexity you add there and richness in terms of kind of like enabling you to really think about, like, redesigning almost from scratch the way people interact with sort of, like, stateful, you know, three d worlds in the computer.
这真的是最终目标，就是获得构建那种系统所需的所有能力。
Like, that's that's really the end goal here is getting, like, all the capabilities you need to build that kind of system.
太棒了。
Awesome.
除了更大更好之外，你还有什么新功能会让你感到兴奋的吗？
Anything you'd add to that as far as new functionality that you'd be excited about that's not just bigger, better?
对我来说，让我们回到智能的第一性原理。
I think for me, let's go back to the first principle of intelligence.
智能不是坐在那里卡住，只是看到或解读空间和物理空间中的事物。
Intelligence is not sitting there stuck and just seeing something or interpreting something when it comes to space and physical space.
对吧？
Right?
它实际上是感知与体验互动之间的闭环。
It's really this, closing the loop between seeing and experiencing an interaction.
所以，沿着那个梯子往上走，正是本所说的。
So just thinking about going up that ladder is exactly what Ben said.
酷。
Cool.
我觉得这里有个有趣的概念，就是AI完备性。
I think one interesting notion there is this notion of AI completeness.
你
Have you
之前读过这个吗？
read this before?
是的，是的，我读过。
Yeah, yeah, I have.
所以，我听到的关于AI完备性的说法，就LLM而言，就是说你必须基本上是最聪明的语言模型，才能回答最聪明的LLM需要回答的问题，或者你必须解决通用智能。
So like everyone I hear about AI complete, by way, in terms of LLMs, which is like you have to be basically, you know, like, the smartest LM to answer the question what the smartest LLM will need to answer, or you have to solve general intelligence.
不。
No.
不是。
No.
这基本上和图灵完备性有关。
It's basically it's a it's a connection to Turing completeness.
对吧？
Right?
嗯。
Mhmm.
就是说，如果一个任务在图灵完备的意义上，就像经典复杂性理论里说的，任何问题都能归约到那个问题上。
Like, the idea being that, like, a a task is Turing complete, like, in classical complexity theory, if, like, I can take any class and any problem in this category reduced to that one problem.
对。
Yeah.
对。
Yeah.
对。
Yeah.
对。
Yeah.
对。
Yeah.
对。
Yeah.
对。
Yeah.
所以呢，AI完备性的一个宽松定义就是，存在这样一个基本原语。
So so then, like, the the kind of, like, the soft definition of AI completeness is, like, there's this fundamental primitive.
那是一个AI任务。
That's that's an AI task.
但如果我能以最广泛的通用性解决这个AI任务，
But if I could solve this AI task in its full broadest generality
你就能解决所有问题。
You solve all that.
那就能解决任何智能问题。
It would solve any intelligence problem.
对。
Yeah.
比如，LLM的经典例子就是，下一个词预测是AI完备的，因为我可以，你知道，有个经典例子，我记得是伊利亚提过的，就是有一本悬疑小说，然后模型得读完整本悬疑小说。
And, like, the classic example at LLMs is, like, next token prediction is AI complete because I could, like you know, there's the classic example, I think, from Ilia, where, like, I there's a mystery novel, and, like, the thing has to read the whole mystery novel.
而悬疑小说的最后一句话是，凶手是——预测下一个词。
And the final sentence of the mystery novel is, like, and the killer was predict the next token.
对。
Yeah.
所以基本上，你可以把任何智能任务都归结为这个。
So, like, you could basically, like, frame any kind of intelligence task in terms of that.
所以很明显，词预测是大家认为AI完备的东西。
So clearly, token prediction is something that people believe is AI complete.
对。
Yeah.
对。
Yeah.
但我觉得这是我们正在意识到的事情，而且Ben今天早些时候也谈到了，就是新视角预测，这是我们在Atlas中拥有的一个基础能力，尤其是生成式的新视角预测。
But I think that's something we're kinda realizing, and Ben was talking about this earlier today, is, like, new view prediction, this primitive that we have in Atlas, especially generative new new view prediction.
对。
Yeah.
这也是AI完备的。
This is also AI complete.
对吧？
Right?
因为我可以拿一些东西，比如
And because I could take something like
你可以有一部电影，然后你处理电影的所有帧，然后凶手走出来，你就能准确预测出
You can you can have the the movie and you do all of the frames of the movie and then, like, the killer walks out and then you predict exactly
谁走出来。
who walks out.
没错。
Exactly.
还不止这样。
Not just that.
你可以说，我想要一个世界，比如马丁在后门上写黎曼猜想的证明。
You could say like, I want because I wanna have a world where like Martin is like writing a proof of the Riemann hypothesis on the back door.
然后镜头一转，
And then the camera buttons over
转到下一块白板。
to the next whiteboard.
他解决了问题，好吧。
He solves the so okay.
所以从进化的角度来看，对吧，新视角预测正是进化通过让动物移动来解决的问题。
So to take a evolutionary view, right, That new viewpoint prediction is exactly evolution had to solve by making animals move.
大自然给了动物眼睛，但没给树眼睛。
Nature gave animals eyes, but nature didn't give trees eyes.
为什么？
Why?
因为当你移动时，你会看到一个新的视角。
Because when you move, you see a new viewpoint.
这就是你所说的AI完整或智能完整。
And that is whether you call it AI complete or intelligence complete.
所以我们非常坚信，下一个视角预测等同于下一个词预测。
So we do believe very strongly that next viewpoint prediction is is the equivalent of next token prediction.
太棒了。
Amazing.
那么，恭喜你们成功发布了这个出色的模型。
Well, with that, congratulations to all of you on a phenomenal model launch.
我们对未来的模型发布充满期待，感谢你们的到来。
We're very excited for future model launches, and thanks for coming.
谢谢。
Thank Thanks
非常感谢。
so much.
感谢收听本期a16z播客。
Thanks for listening to this episode of the a 16 z podcast.
如果你喜欢这期节目，记得点赞、评论、订阅，给我们评分或写评论，并且分享给你的朋友和家人。
If you like this episode, be sure to like, comment, subscribe, leave us a rating or a review, and share it with your friends and family.
想听更多节目，可以去YouTube、Apple Podcast和Spotify。
For more episodes, go to YouTube, Apple Podcast, and Spotify.
在X上关注我们@a16z，并订阅我们的Substack：a16z.substack.com。
Follow us on x at a sixteen z, and subscribe to our Substack at a16z.substack.com.
再次感谢收听，我们下期见。
Thanks again for listening, and I'll see you in the next episode.
提醒一下，这里的内容仅供参考，不应视为法律、商业、税务或投资建议，也不应用于评估任何投资或证券，并且不针对任何a16z基金的投资者或潜在投资者。
As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a sixteen z fund.
请注意，a16z及其关联公司可能也持有本播客中讨论的公司的投资。
Please note that a sixteen z and its affiliates may also maintain investments in the companies discussed in this podcast.
更多详情，包括我们的投资链接，请访问a16z.com/disclosures。
For more details, including a link to our investments, please see a 16z.com forward /disclosures.
关于 Bayt 播客
Bayt 提供中文+原文双语音频和字幕，帮助你打破语言障碍，轻松听懂全球优质播客。