将8位AI转化为16位电影品质的Topaz AI工作流程

好的。

Alright.

欢迎回到《DeNoised》的又一期节目。

Welcome back to another episode of DeNoised.

我们将回顾一下Infinity Fest的一些内容。

We are gonna focus on a bit of a recap of Infinity Fest.

我在那里主持了一个关于40高斯、溅射、AI和虚拟制作的小组讨论。

I hosted a panel there on 40 gauss and splats and AI and virtual production.

Adi无法到场，所以我会在过程中向他和大家做一下说明。

Adi cannot make it, so I'm gonna fill him in and fill all of you in while we're doing it.

然后我们会聊聊AI电影制作工作流中其他一些最新的进展。

And then we're gonna talk about some of the other things, the updates we've been seeing in AI filmmaking workflows.

准备好了吗，Adi？

You ready, Adi?

听起来不错吧？

Sound good?

太棒了。

Awesome.

对。

Yeah.

好的。

Alright.

所以是Infinity Fest。

So Infinity Fest.

那是上周四和周五。

It was last Thursday and Friday.

非常棒的活动。

Super cool event.

有很多很好的论坛和演讲。

A lot of really good panels, lot of good talks.

重点讲一下我主持的论坛。

Focus on the panel that I've moderated.

我们举办了一个关于人工智能创新和虚拟制作的小组讨论，但我们真正深入讨论的是高斯点云和3D Nice。

We had a panel on AI innovations and virtual production, but that was the the real thing that we were talking about a lot was Gaussian splatting and three d Nice.

4D高斯点云。

Four d Gaussian splatting.

我们请到了保罗。

We had Paul

看你主持这么大的小组讨论啊。

Look at you moderating big panels.

大型小组讨论。

Big panel.

是的。

Yeah.

卡瓦利尔本人，乔伊。

Boy cavalier himself, Joey.

那很不错。

That was good.

我也很好奇。

Was I was curious as well.

我当时就想，我就直接问你问题吧。

I was like, I'm just gonna ask you questions.

我也想知道。

I wanna know as well.

而且结尾还有一些不错的预告，关于一些我希望能有更多时间深入探讨的内容，比如三角形高斯溅射。

And there were, like, some good teasers at the end too of things that I I wish we had more time to get into because I'm also still curious about them, like triangle Gaussian splats.

是的。

Yeah.

我想提醒一下观众，AI和高斯溅射之间的联系是什么，因为我们通常会认为这是传统的摄影测量技术，尽管显然不是。

And, I guess the audience I mean, remind all of us, what the AI connection to Gaussian splats are because we we tend to think of that as traditional photogrammetry even though it's clearly not.

这不一样，它也不是AI。在小组讨论中，有来自Netflix iLine工作室的Paul Debevik，还有Jason Shugart，他现在在英伟达工作，但在视觉特效领域有丰富的经验。

Different, it's also not AI as, so on the panel is Paul Debevik, from Netflix's iLine Studios, and then also Jason Shugart, who's, he works at NVIDIA now but has extensive history in VFX.

对。

Yeah.

我认识杰森。

I know Jason.

他很棒。

And He's great.

他们明确表示，高斯点云并不是直接的AI技术，但确实有关联。

They made clear that Gaussian Splats is not directly AI, but it is Yeah.

在这个领域内有关联。

Related in that field.

所以我认为这里的关联在于，高斯点云并不是生成式AI。

So I think the correlation there is Gaussian splats are not generative AI.

它们并不是从零开始创造新事物。

They're not creating something novel from scratch.

但为了解决点云分布中高斯函数在三维空间中的关联性，使用了AI引擎。

But to solve itself, how the, point splatters, the Gaussians, are, correlated in three-dimensional space is using an AI engine.

嗯。

Mhmm.

而且，是的，顺便简单解释一下这是什么以及它有什么用。

And, yeah, also just to, like, a short explanation of what this is and how it's useful.

基本上，如果我们比较一下重建三维物体的原始方式——这种方法至今仍很常见，那就是使用多边形，用大量小线段连接所有点来构建形状，但这样可能需要数百万甚至数十亿个小点和多边形来构建你想要的三维物体或空间。

Basically, if we kinda compare how we would rebuild things in three d, the original kinda way that is still pretty standard is, polygons, a bunch of little lines connecting everything, you're building out your shapes, but that can use millions or billions of little dots and polygons to build out whatever three d object or space you're building out.

高斯溅射，是的。

Gaussian splats Yep.

一个溅射就是一个包含多种信息的漂浮斑块。

A splat is a floating blob with a variety of information inside it.

如果你有数百万个这样的包含信息的斑块，就可以重建出，是的。

And then if you have millions of these blobs with information inside it, that can rebuild Yep.

你的三维空间。

Your three d space.

这种三维物体的优势在于，它的运行速度更快、资源占用更轻，相比用多边形重建同一场景来说。

And the advantage of this or three d object, the advantage of this is it can run much faster, much lighter weight than if you're rebuilding the same scene with polygons.

正确。

Correct.

是的。

Yeah.

我认为高斯溅射仍然使用了一种所谓的实时渲染技术。

I think Gaussian splats are still using a sort of, quote, unquote, real time rendering.

比如，当你移动摄像头时，它会实时计算出新的视角。

Like, the fact that when you are moving the camera, it's calculating that novel view in real time.

嗯。

Mhmm.

但与其说是在渲染，比如通过光线追踪或像传统计算机图形学那样让光线在表面反弹，它实际上是将这些带有颜色的点块放置在对应于该物体的三维空间中。

But instead of rendering per se, like, with ray trace or, you know, bouncing light off of surfaces the way traditional computer graphics does, what it's doing is it's placing these blobs of color in the three space that corresponds to that object.

由于它们是高斯溅射，你可以将它们紧密排列在一起，从而融合成一个无缝的物体。

And because they have a Gaussian splat, you can actually put them right next to each other, and they blend in like a seamless object.

嗯。

Mhmm.

对。

Yeah.

在这之间，还有NeRFs，它们曾短暂地引起过一些关注，也就是神经辐射场。

And in between this too, there were Nerfs, which kinda had a a little bit of a moment, neural, radiance fields.

但NeRF的问题在于，你必须先处理数据，然后基本上把它固化下来。

But the NERF the thing with NERFs was you had to process the data and sort of it sort of got baked in.

这样理解对吗？

Is that is that kinda correct?

而高斯溅射之所以能实时运行，是因为数据就存储在溅射点中，是这样吗？

Where, like, Gaussian splats can kinda be run-in real time because the data is in the splat?

这超出了我的理解范围，但我认为你说得对，NeRFs已经是过时的技术，目前这个领域大部分开发工作都集中在3D GS上。

Again, that's beyond my understanding, but I I think you're right in that nerfs are an outdated technology, and most of the development work that's happening in this realm is now happening in three d GS.

嗯。

Mhmm.

有趣的是，高斯溅射并不是什么新技术。

And the interesting thing was that Gaussian splats are not new.

它基于一篇大约十五到二十年前的论文，但人们只是最近才重新发现它在三维和四维空间中的应用，这一点我们也稍微提到过。

It was, like, based on a paper, I I think, fifteen, twenty years old, but they sort of just rediscovered that it has this application in three d space and four d space, which we also touched on a bit too.

所以，我的意思是，四维高斯溅射的概念是，你捕捉的不仅是三维环境，还有三维环境中的运动。

So, I mean, the idea of four d Gaussian splats is you are capturing not just the three d environment, but motion in a three d environment.

比如人们跳舞或移动，你能重建出这个场景的四维形态——即在一段时间内的三维空间，并可以在该三维空间中重新定位或移动你的摄像机。

So people dancing or moving, and you're able to reconstruct that scene in four dimensions, a three d space over a period of time, and reposition your camera, move your camera in that three d space.

是的。

Yeah.

所以当你提到四维时，你并不是在说《星际穿越》里那种手部场景吧？

So when you say four d, you're not talking about Interstellar, like the hand scene with

我说的四维是指《木偶奇遇记》四维那种。

I'm talking about four d as in Muppet Vision four d.

这是三维空间随时间的变化。

It's three dimension over time.

对。

Yeah.

没错。

And so right.

就像那个演示视频，

Like and and there was that demo clip.

我想我们几个月前在播客里聊过，还记得那个男人坐下的片段吗？

I think we talked about on the podcast, like, months and months ago of remember that clip of the guy sitting down?

我觉得那是个中国YouTube博主，他坐着，然后用的是四维高斯溅射技术，摄像头围绕着他移动。

I think it was, like, a a Chinese YouTuber, and he was sort of sitting down, and then it was, a four d Gaussian splat, and the camera was sort of moving around him.

然后在我们这些极客圈子里稍微火了一把，因为人们都说，哦，这居然是从一个摄像机角度生成的。

And then it kinda went a little bit viral in our nerdy sphere because people were saying, oh, was generated from, like, one video angle.

然后是的。

And then Yeah.

后来真相大白，我说其实他们用了20台摄像机才完成的，但即便如此，这依然很酷，只是确实用了20台摄像机。

It came out where I was like, no, they needed 20 cameras to pull that off, but it was still, you know, was still cool, but they needed 20 cameras.

是的。

Yeah.

我们从来没看到过幕后制作过程，我记得我朋友吉姆·加迪拉说过，是的。

We we never caught the BTS, and I think I was, what my friend Jim Gadilda Yeah.

他指出了这一点。

Kinda pointed that out.

意思是，你知道的，这和使用50台相机的传统摄影测量一样复杂。

It's like, just so you know, this is as involved as traditional photogrammetry with, like, 50 cameras.

不管怎样，这项技术背后的公司也在Infinity Fest上做了演示。

Well, anyways, the company behind that technology, they were demoing at Infinity Fest as well.

我想是叫FourD View之类的，他们设置了设备和那些演示片段。

I think it was four four d view or something, and they had the rig and that those demo clips set up.

但他们想出了一个非常巧妙的解决方案，用多种PCZ相机搭建设备，用来捕捉4D高斯溅射，而且这个

But they've been doing a really clever solved solution for building out a rig with a variety of PCZ cameras where they can capture four d Gaussian splats, and that, like

所以他们并不是在构建所谓的超立方体。

So they're not building the tesseract per se.

那是什么？

What was that?

就是《星际穿越》里的那个东西。

That's the thing from the Interstellar

我很久没看到那个了。

I haven't seen that in a long time.

那个超立方体是什么？

What was the tesseract?

我不确定这是否

Don't know if this is

卡尔·萨根。

Carl I Sagan.

我不确定这是否真实

I don't know if this is real

卡尔·萨根用一个叫超立方体的物体来描述四维空间。

Carl Sagan described four dimensions using an object called the tesseract.

哦，对了。

Oh, that's right.

那本关于暗物质的书里也有提到。

That was in that, black matter, dark matter book as well.

是的。

Yeah.

所以，就像，你知道的，一个立方体的影子是正方形，嗯。

So, like, you know, like, a a cube shadow is a square Mhmm.

而四维超立方体的影子是

And a tesseract's shadow is a

立方体。

cube.

好的。

Okay.

这让你大脑崩溃了吗？

Does that break your brain?

我正要

I'm to

你想象不出来。

You can't picture it.

我正在努力理解。

I'm trying to wrap.

我们的大脑无法

Our brain can't

想象它。

picture it.

不行。

No.

是的。

Yeah.

好的。

Okay.

但是，是的。

But, yeah.

是的。

Yeah.

当我们谈论四维时，事情就发生在那个层面。

When we talk four dimensions, that's where it goes.

所以让我们回到为什么高斯点云对现代电影制作很重要。

So loop us back into why Gaussian splats are important for modern filmmaking.

好的。

Okay.

那么，让我们回到实际的好处，以及这与虚拟制作和其他几位嘉宾的关系。

So bringing this back into, like, tangible benefits and also, some of the other panelists and what this had to do with virtual production.

在小组讨论中，我们还有像费尔南多·里瓦斯这样的公司或人士，他是Valinga的首席执行官。

Also on the panel, we have companies or people, so, like, Fernando, Rivas, who's the CEO of Valinga.

我在NAB上对他进行了一次深入采访，有关这方面的内容非常丰富。

I did a big interview with him at NAB, so there's extensive stuff on that.

但他基本上创建了Valinga，这是一个Unreal Engine的插件。

But he basically built Valinga, which is an Unreal Engine plug in.

你可以将你做的高斯点云扫描，通过多种技术转化为模型，但我们还谈到了XGrid的PortalCam，这是一种最新的设备，能一体化完成这些工作，让操作变得简单。

You can take a Gaussian splat scan that you do, and you can make these through a variety of techniques, but we also talked about PortalCam from XGrid, which is sort of the latest device that kinda does this all in one and makes it easy to do.

你可以扫描一个房间。

You could scan a room.

我们已经讲过这一点了。

We covered that Yeah.

我们已经讲过这一点了。

We covered that.

是的。

Yeah.

PortalCam 是大约一个月或两个月前发布的。

PortalCamp launch, about a month or two ago.

你可以将房间的扫描数据导入到 Unreal Engine 中。

You can take your scan of a room, bring it into Unreal Engine.

通过 Olinda，他们可以加载高斯点云并将其导入 Unreal，同时还支持 ACES 色彩管理。

With Olinda, they can load Gaussian splats and load it into Unreal, and they also have ACES color management.

然后你可以将它投射到墙上，基本上就能用真实环境的扫描数据来构建你的场景，无论是房间还是布景都可以。

And then you can push that into a wall, and basically, you can have your environment based on a real life environment, scan a room, scan a set.

他们还经常将这种技术用作备份，比如在拍摄期间扫描布景，如果之后需要补拍，或者第二季被续订而他们原本没预料到，或者有各种其他原因，你都可以扫描任何想要的空间，然后把它投射到墙上。

They've also been, you know, using that a lot as backup, so you scan a set from when from production, if you have to do pickups later, or if, like, season two gets picked up and they weren't quite counting on season two, or variety of reasons, you can take a scan of whatever space you want and put it on a wall.

所以这是一种非常实用的方法。

And so that's, a very practical way.

他们还做了一个案例研究，Valinga扫描了奥斯维辛集中营，而那里是严禁任何人进行拍摄的，无论是什么类型的制作。

Also, had a case study too of, that Valinga did where they scanned, Auschwitz, the concentration camp, and that one was they just did not allow anyone to film at Auschwitz if they're trying to do any type of production.

他们做了扫描，现在我们把这个扫描数据提供给那些正在制作与奥斯维辛相关项目的团队，如果他们想使用这个地点，就可以把它投射到墙上，进行数字化拍摄，而不会干扰实际遗址。

They did a scan, and then now we're making that scan available to productions if they're doing a project around Auschwitz or that wants to use the location, they can put it on a wall and have that space, digitally so they can film there digitally, but not disturb the actual site with production.

世界上有太多地方，把它们引入虚拟拍摄棚或视觉特效中都会带来巨大好处。

There's so many places in the world where bringing that place into a volume or into VFX would be super beneficial.

我的意思是，奥斯维辛就是其中之一。

I mean, Auschwitz is one of them.

甚至像金字塔这样的地方，那里有各种许可限制和拍摄时间限制。

Even something like the Pyramids where there's all types of permits and time just of

游客。

tourists.

游客的不便。

Tourist inconvenience.

切尔诺贝利。

Chernobyl.

你知道的。

You know?

是的。

Yeah.

就像那部剧。

Like the show.

对。

Yeah.

如果你要在虚拟影棚里重现那个场景，那可能是最好的方法。

Like, if you were to recreate that in a volume, that's that's probably the best way to do it.

而且我认为，相比传统的摄影测量法——你需要拍摄数小时的视频或成千上万张图片——这种方法的优势在于，传统方法需要一个复杂的处理过程，而且很多步骤都是手工完成的。

And and I think it the advantage over traditional photogrammetry where you take, you know, hours of video or thousands of images is that there's a big solve process, and a lot of it is hand hand done.

传统上使用的是实景捕捉技术，而这项技术现在已被虚幻引擎收购。

Traditionally uses, reality capture, which is now acquired by Unreal Engine.

你所说的不仅仅是运行在配备强大GPU的超级计算机上，而是需要艺术家亲自去清理、对齐各种元素，这是一项非常繁琐的工作。

And you're talking about not just running it on a superfast computer with a lot of GPU power, but an artist manually going there, cleaning stuff up, aligning things, and it's tedious work.

而高斯点溅技术只需要上传图像。

Versus a Gaussian splat, you upload the images.

如果你拍摄得当，它基本上就能自动完成处理。

And if you shot them correctly, it should more or less, quote, unquote, solve for itself.

然后你会得到一个非常轻量的文件，大约只有100兆字节，而不是像10太字节那样的庞大数据。

And then you get this sort of super lightweight file back, which is, like, a 100 megabytes, not, like, 10 terabytes or whatever.

是的。

Yeah.

然后当你的摄像机移动时，它就能实时地大致渲染出画面。

And that could then, in real time, kinda sorta render it as your camera moves.

对。

Yeah.

我的意思是，其他实际应用，比如你看到的Lightcraft Jet Set，他们的价值主张是你可以用手机作为摄像机追踪器，并将你的3D场景加载到手机里。

I mean, other practical use, you know, seen with light Lightcraft Jet Set, you know, their their value prop is you can use your phone as a camera tracker, and you can load in your three d scene into your phone.

这样在拍摄时，你可以大致合成并看到你的3D空间。

So when you're filming, you can roughly comp in and see your three d space.

你可以导入Blender和Unreal的场景，但它们会比较沉重。

You could bring in Blender and Unreal scenes, but they're gonna be kind of heavy.

但你也可以把高斯溅射数据导入到手机上的Lightcraft Jet Set，并在手机上实时查看。

But you could also bring in Gaussian splats into Lightcraft Jet Set on your phone and see this in real time on your phone.

由于溅射数据轻得多，你在拍摄时能获得更好的体验。

And because the splats are much more lighter weight, you get a better, experience when you're filming.

我很想知道Paul de Bebek在小组讨论中说了什么，因为你知道，他是……

I'm curious to know what Paul de Bebek said at the panel, because as you know, he's Yeah.

当今VFX领域最受尊敬的人物之一，所以我只是好奇他具体说了什么。

One of the most respected folks in VFX today, so I'm just curious what he said.

所以，是的，Paul大致谈到了我们刚才讨论过的那些基础内容，比如什么是高斯溅射，包括3D和4D的。

So, yeah, Paul, he kinda talked about a lot of the basics that we sort of just covered now of, like, what is Gaussian splatting, both three d and four d.

他还隐约提到了一篇即将在SIGGRAPH亚洲会议上发表的论文，内容是关于超分辨率技术，用于从大场景捕捉中实现极致特写。

He also sort of alluded to a paper that they've got coming out at SIGGRAPH Asia on super resolution techniques to achieve extreme close ups from large volume capture.

我们没有深入讨论更多细节，但这篇论文我很快就会发布。

So we didn't get into more detail about that, but that's a paper that I've got, coming out soon.

他们还谈到了高斯泼溅的一些局限性。

And they also sort of talked about some of the limitations of Gaussian splat.

目前用于编辑高斯泼溅（比如你扫描得到的数据）的工具仍然很有限，当然远不如已经发展了二三十年的多边形工作流程成熟。

So the tools to edit a Gaussian splat, like a scan that you make, they're still limited, you know, definitely not as much as the existing twenty, thirty year pipeline of working with polygons.

是的。

Yeah.

高斯泼溅的另一个局限性在于它们是辐射场表示，也就是说，扫描过程中捕捉到的光线基本上被固化在其中了，因此重新打光高斯泼溅会更困难。

The other thing the other limitation with Gaussian splats is they are ratings field representation, so basically the light that you captured during your scan is sort of baked into it, and so also relighting Gaussian splats is trickier.

目前，如果需要具备重新打光的灵活性，通常需要结合高斯泼溅、LiDAR技术和传统摄影测量来重建场景。

That's something where currently right now, something like a combo of Gaussian splats, LiDAR technology, and traditional photogrammetry comes into rebuilding a space if you need to have that flexibility of relighting.

Global Object 专长于此，他们综合运用所有这些技术，以便创建一个既属于Unreal Engine三维场景、又具备重新打光灵活性的空间。

That's something that Global Object specializes in, where they combine all of these techniques if you need to have a space where it's an Unreal Engine scene, three d, but has the flexibility of relighting.

这目前就是高斯溅射的局限性之一。

That's just one of the limitations right now with Gaussian splats.

当然。

For sure.

但烘焙光照的优势在于，许多反射效果都非常精准。

But the advantage to having baked lighting is a lot of the reflections are super accurate.

嗯。

Mhmm.

而且很多时候，摄影测量在玻璃或金属物体上会完全出错，而高斯溅射在这方面表现很好，但问题是这些效果是被固化进去的。

And it just looks like photogrammetry, a lot of times, totally messes up, like, glass or metal objects, whereas Gaussian splats are really good at that stuff, but then again, it's baked in.

他还隐约提到了另一点，但我们没有深入讨论，也许将来我们可以试着拆解一下：我们已经从多边形发展到NeRF，现在又转向了高斯溅射，那之后会是什么？

And then the other thing he sort of alluded to, but we didn't get into details, and so maybe we'll try to break this down in the future, was, you know, we've gone from polygons, we've gone to Nerfs, we've gone, now we're looking at Gaussian splats, what's after that?

他提到的是三角形高斯溅射，但我没有深入探讨，也没去查过，不过我注意到已经有一些相关论文出现了。

And triangle Gaussian splats are what was alluded to, but I didn't really get into details about that and I have not looked it up, but I noticed some papers out about that.

尝试。

Tries.

我猜是多边形和高斯点云的组合。

Guessing it is a combo of polygons and Gaussian splats.

是的。

Yeah.

我觉得三角形对GPU来说计算效率更高，因为GPU原本就是为处理三角形和四边形设计的，一次性并行计算百万甚至十亿个三角形的数学运算，相比高斯点云（本质上是球体）要容易得多。

I would imagine tries would be more computationally efficient for GPUs because GPUs are really built for tries and quads, like the math behind just computing a million or a billion tries at a time in parallel versus a Gaussian splat, I think, is a sphere.

所以从几何上看，它更复杂。

So geometrically, it's more complex.

哦，明白了。

Oh, okay.

这说得通。

That makes sense.

好的。

Okay.

所以转向三角形会带来无限更好的性能

So moving to tri's would give you infinitely better So

它更像是，数据存储方式不是球形的斑点，而是三角形的斑点？

it'd be, like, sort of the data stored in sort of instead of, like, a sphere y blob, more of a triangle y blob?

三角形的斑点斑点。

Triangly blob blob.

但这个三角形是平面的，这是现代计算机图形学中的自然基本单元。

But that triangle is a flat triangle, which is, a natural unit for modern computer graphics.

好的。

Alright.

好吧。

Okay.

明白了。

Okay.

有意思。

Interesting.

我们以后得深入研究一下这个。

We'll have to dig into this in the future.

好的。

Okay.

然后为了全面介绍一下 panel 上的其他成员及其讨论内容，我们有本·阿弗格尔，他与 ETC 合作，是一位参与 ETC 项目的创意技术专家。

And then he also just to round out who else was on the panel and what they covered, we had Ben Avergel, who is works with ETC and is a creative technologist involved in ETC.

我们会稍后谈到这些乐队，但他参与了 Pathways 以及其他 ETC 正在制作的项目。

And he we'll talk about the bands in a bit, but he worked on Pathways and other project that ETC is producing.

他们最初尝试用几部 iPhone 实现某种四维高斯点云。

And they were initially trying to do some form of four d Gaussian splats with a few iPhones.

我认为他们用四部 iPhone 录制了一个场景，但数据量远远不够，无法生成一个可以自由移动且保持高质量的高斯点云。

I believe they recorded a scene with, like, four iPhones, but it just was not enough data needed to create a Gaussian splat that you could, like, move around and still have, good quality.

因此他们调整了方案，转向了我们之前讨论过的一种工作流程：使用 iPhone、Lightcraft Jet Set、BeBull、真实演员、绿幕空间，然后用类似拼贴方式的 Unreal Engine 场景替换背景，再通过 AI 视频到视频技术进行后期风格化。

So they shifted and they kind of shifted to a workflow that was more something we've talked about before of like iPhones, Lightcraft Jet Set, BeBull, real actors, green screen space, and then restyling the background with like a kit bashed kind of Unreal Engine scene, and then restyling that with video to video with AI later on.

回到他们最初用 iPhone 实现 4D 高斯点云的初衷，我问过，到底需要多少部 iPhone 才能成功实现？

The thing, going back to the original 40 Gaussian splats that they're trying to do with the iPhone, I did ask, like, how many iPhones would you actually need to successfully pull that off?

他认为大概需要 20 到 30 部，和其它高斯点云视频所用的数量差不多。

And he thinks probably around 20 to 30, the same number that, the other Gaussian Splat videos are using.

最后是艾拉·罗伯茨。

And then lastly, Ella Roberts.

她是《绿野仙踪》在球形影院中进行上采样工作的AI监督。

She was the AI supervisor for the Wizard of Oz upscaling at the Sphere or upscaling.

我认为，马克斯·菲利普斯负责了所有内容。

Max Phillips, I believe, did all the Yeah.

她与他们合作。

She works with them.

她是创意AI监督。

She's the creative AI supervisor.

总的来说，我们讨论了他们所采用的工作流程和制作管线，但显然我无法深入太多细节。

And, yeah, in broad sense, we kind of talked about the workflow and pipeline of what they did, but obviously, I couldn't get into too many details.

但话说回来，这完全是另一个项目：他们将现有格式从原电影中拆解，然后为球形影院16K的宏大景观重新构建每一个场景。

But, yeah, I mean, that was just sort of a whole different project in taking existing formats and sort of breaking them down from the original film and then rebuilding out each scene for the 16 k massive landscape of the sphere.

是的。

Yeah.

那个小组讨论涵盖了行业的多个层面。

It's a nice cross section of the industry on that panel.

我的意思是，从像保罗这样资历深厚、堪称顶尖的专家，到像你这样站在AI前沿的人，应有尽有，嗯。

I mean, you have everybody from, like, tenured, you know, one of the foremost experts like Paul all the way up to, like, you on the AI frontier side Mhmm.

加布里埃尔代表的是Sphere和沉浸式娱乐领域，同时你还有NVIDIA的代表。

Gabriel on the, you know, sphere and location entertainment side, and then you also have, NVIDIA.

你有来自NVIDIA的杰森。

You have Jason from NVIDIA.

不是。

No.

那场讨论非常好。

It was really good.

真的非常棒，这是一个

It was a really good It's a

非常精彩的小组讨论。

really cool panel.

我错过了，真遗憾。

I'm bummed out that I missed it.

是的。

Yeah.

对。

Yeah.

也要感谢科琳娜组织了这个小组讨论。

And a shout out to Corinne for assembling that panel.

不过，是的。

But, yeah.

另外，趁着我们还在聊无限节，我想谈谈另一场非常有趣的演讲，关于如何将普通的AI输出——通常是8位RGB 720p或1080p的图像——提升为4K影院级、16位EXR格式的文件。我们之前讨论过，Luma的ORAY 3很厉害，它能在生成时直接输出这样的高质量结果。

Also wanna talk about one other thing while we're on Infinite Fest because there's another really interesting presentation about upscaling regular AI outputs, are usually, eight bit RGB seven twenty, maybe ten eighty output to four k cinema quality, 16 bit EXR files, which we've talked about how like ORAY three from Luma, their cool thing is they can do that in the output and the generation.

但如果你不想用ORAY 3，或者你有其他输出源呢？

But what if you don't want to use RAY three or you have other outputs?

这个工作流程是怎样的？

What is this workflow?

所以

And so

是的

Yeah.

如果你只是想放大呢？

What if you just want to upscale?

你已经完成生成了？

You already have the generation done?

没错。

Exactly.

所以这个流程非常有趣，是ETC的另一个项目《Benz》，这是一个可爱的动画短片，讲述一条深海鱼在水下旅行并努力游向水面的故事。

And so this is a really interesting workflow with the that ETC's other project, the Benz, which is sort of this kinda cute animated short with like a fish, deep sea fish underwater kind of traveling around and making its way to the surface.

他们构建了一个使用多种Topaz模型的流水线，这个本地模型与Starlight无关。

They built out a pipeline using a variety of Topaz's models, the local model has nothing to do with Starlight.

基本上，他们会先获取原始素材，通过Nick的去噪模型去除噪点，然后再用他们另一个模型Gaia Uprights进行处理。

And so basically they would take their source footage, they would run it through Nick's denoise, kind of strip out the noise from it, then run it through, Gaia Uprights, which is one of their other models.

然后这个流程中还有几个其他步骤，但我拍的截图里没有包含这些。

And then there were a couple other steps in the process, which I don't have in the screenshot that I took.

但基本上，他们用了三个不同的Topaz模型，之后得到了更丰富的细节、更高的质量，输出了16位的XR文件，还有色彩分级等所有内容，他们拥有更大的调整空间来匹配镜头，这就像你用电影级摄像机拍摄时习惯的那样。

But basically, it was like three different Topaz models, and then out of that, they were able to get, much more detail, much higher quality, the XR file, 16 bit, that and the color grade and everything else, they could just they just had a lot more latitude to adjust the shots, match the shots, something that you would, you know, be used to if you're shooting with a cinema quality camera.

是的。

Yeah.

这真是很有趣的工作。

It's really interesting work.

让我更着迷的是分辨率提升之外的位深提升。

One the thing that fascinates me more about the resolution upscale is the bit depth upscale.

你知道，通常8位是0到255。

So, you know, when you talk about typically eight bit is zero to two fifty five.

对吧？

Right?

所以如果你的画面中有太阳，太阳的最高值是255，但现实世界的动态范围，就像我们眼睛看到的那样，远不止255。

So if you have, you know, your sun in the frame, the highest value that sun will have is two fifty five versus real world dynamic range, like what our eyes see, you know, is it won't be two fifty five.

它可能会达到一百万单位。

It'll be, like, 1,000,000 units.

对吧？

Right?

是的。

Yeah.

那么，你是怎么获得如此极端的动态范围的？又如何仅从原始素材中人工插值生成这些数据？

So, like, how do you get that crazy of a dynamic range, and how do you create it artificially interpolate it just from

我知道。

I know.

对。

Right.

你居然能从这种质量很差的原始帧中提取并生成出这么多数据。

From having this, like, kind of crappy source frame that you're able to, like, pull and generate all this data.

对。

Yeah.

真的很令人印象深刻。

Was really impressive.

我真希望当时拍下了波形监视器的画面，因为那里有处理前后的对比，你才能真正看到所获得的动态范围。

And I I wish I had a shot of the waveform monitors because they had before and after the waveform, and that's where you could really see the latitude you get.

而且，另一个优势是，它能大大减少色带现象。

And also, the other advantage is, like, it reduces banding a lot.

你知道，这就是色带问题。

You know, it's a banding.

这是因为如果你使用的是8位，颜色空间是有限的。

It's like just because there's limited if you're in eight bit, it's limited amounts of color space.

对。

So Right.

特别是这部短片发生在深海，所以有很多暗景。

Especially because this short film took place in, like, the deep sea, so there's a lot of dark shots.

因此，即使在渐变和阴影部分，也能看到那些色带线条。

And so even with the kind of gradients and some of the shadows in the dark, would see the, like, banding lines.

这个上采样过程在波形监视器上清晰可见，波形从非常僵硬的阶梯状变成了更平缓的曲线，这正是你希望看到的正常效果，能获得更自然的渐变效果，真的吗？

And this upscale process, and you could see it in the waveform monitors where it went from, very rigid steps in the waveform monitor to a much more gradual curve, which is what you would want to see or normally see where you get more of a a nice gradient with with Oh, really?

色彩空间。

Color space.

我认为很多人对暗场景、光线较少的假设都有误区，比如我想到了《权力的游戏》，我管那些场景叫‘权游场景’，对吧？

And I think the a lot of the sort of assumptions about having darker scenes with less light, You know, I'm I'm thinking Game of Thrones, so I call the scenes the game right?

你可能会觉得，因为画面很暗，就不需要那么高的位深，但实际上，我们的眼睛对阴影和暗部区域的细微差别更加敏感，如果你给暗部区域分配更高的位深，反而能提取出更多细节。

It's just, like, you think that you don't need that big of a bit depth because things are dark, when in reality, our eyes are actually much more sensitive to shadows and differences in the darker lights and the regions that if you throw more bit depth at the darker regions, you actually get more detail out of it.

所以，即使场景中没有太阳，不是白天的场景，它依然能从中受益。

So it still benefits the scene even though there is not a sun in it, for example, it's not a daylight scene.

即使是一个暗场景，你仍然需要在位深上保留更多的信息量。

If it's a dark scene, you still need more bandwidth in the bit depth area.

是的。

Yeah.

而且在调色时，你也能获得更大的控制力和灵活性。

And also just when you're the color grade too, and you have that that control and flexibility.

而且他们正在进行的实验是，试图在专业的好莱坞电影制作流程中构建一套AI生成工作流。

And and this and what the experiment they're doing too is they're trying to really build out an AI generative workflow, but in a professional Hollywood filmmaking pipeline.

整个过程由ASC摄影师罗伯托·沙弗监督，他用自己的眼睛分析场景，试图从这些AI输出中获得他习惯用阿莱Alexa或索尼Venice拍摄的效果。

So, like, this entire process was supervised by an ASC cinematographer, Roberto Schafer, and it, you know, he he's using his eye to analyze the scenes and, like, try to get what he's used to with shooting with an Arri Alexa or Sony Venice out of these AI outputs.

他们做的另一件事是通过胶片化学方法添加颗粒效果，并展示了一些带有颗粒和不带颗粒的对比效果。

And then the other thing they did was they did a grain film pass with photochem, and they showed some kind of comparisons of, like, with grain or without grain.

只是，添加颗粒后，画面就变得

And just, you know, adding the grain just makes it

好太多了。

Way better.

对吧？

Right?

好太多了。

Way better.

是的。

Yeah.

用一个更贴切的词来说吧。

For lack of a better word.

好太多了。

Way better.

颗粒感真正帮助到的一点是色带问题。

One thing grain really helps with is banding.

比如，有一个叫做抖动的过程，嗯。

Like, you know, there's a process called dithering, which Mhmm.

可以减少色带。

Reduces banding.

所以这本质上是在画面中添加噪点，而颗粒感就是一种噪点。

So it's essentially adding noise into the frame, and grain is kind of noise.

因此，它确实有助于克服AI因8位色深和低动态范围带来的诸多限制。

So it does help overcome a lot of the limitations with AI having, you know, eight bit and low dynamic range.

是的。

Yeah.

我还觉得他说了一些特别有意思的话，因为当我想到给AI图像或者数字图像添加颗粒感时，我内心其实有点疑惑：这算不算作弊？

And I thought he said something really interesting too because, like, when I thought about adding grain to an AI image, you know, or even just a digital image, part of it is felt like, is this cheating?

或者这算不算在造假？毕竟，这根本不是用胶片拍的，我们却硬加了胶片颗粒。

Or is this, you know, is this doing something, like, you know, faking it because, like, it, you know, wasn't actually shot on film, but we're adding film grain to it.

但他提到一个很有趣的观点，他说，你用什么设备拍摄其实并不重要。

But he said something interesting where he's just like, it it doesn't matter what you shoot with.

即使我们用的是数码相机，传感器本身也有自己的噪点特性。

Even if we're shooting this with digital cameras, like, sensor has a noise profile.

比如，

Like, they

所有设备拍出来的画面都不会是那种超清晰、毫无瑕疵的。

all like, nothing is shooting super crisp and clear.

太死板了。

Clinical.

是的。

Yeah.

是的。

Yeah.

比如，如果你在制作动画CG电影，或者生成图像时，如果画面太干净了，因为这可能是唯一能生成完全无噪点或无颗粒感的方式，但看起来会很奇怪、太干净了。

Like, and if you're doing, you know, even an, an animated CG film or if you're doing outputs and it's just like too clean because, like, that's the only way you could get generate something that, like, had absolutely zero noise or grain, and it just looks too weird and too clean.

在这个过程中加入颗粒感有助于增强真实感，这并不是说你在添加虚假的胶片效果。

And adding the grain in that process, you know, helps sell the illusion, and it's not like a it's not like, oh, you're adding fake film effect.

你只是在添加一些能模拟即使你用数字电影摄像机拍摄时也会有的效果。

It's just you're adding something that replicates even if you shot it with a digital cinema camera.

是的。

Yeah.

你知道，电影中的颗粒感非常重要，以至于现在很多摄像机都内置了颗粒感调节功能。

You know, the the grain part of cinema is so important that a lot of cameras now have built in grain adjustment.

我不确定RED摄像机是否已经具备这个功能。

I don't know if the RED cameras have it just yet.

也许V Raptor可能有，但Arri Alexa肯定可以，你不仅能调节颗粒感的强度，从0到100，还能选择颗粒的纹理。

Perhaps the V Raptor might, but the Arri Alexa's for for sure, you can not only pick the level of grain like a zero to a 100, but also texture of the grain.

好的。

Okay.

这和选择ISO设置之类的有区别吗？

And this is separate from, like, picking your ISO setting or whatever?

是的。

Yep.

没错。

Exactly.

所以这是相机在拍摄后添加的。

So this is something that the the camera body will add after acquisition.

所以它会像叠加一样加进去。

So it'll, like, kinda overlay it.

我认为如果你使用RE RAW格式，之后也可以调整它。

And I believe if you're in RE raw format, you can tweak it down the road as well.

这说得通。

That that would make sense.

好的。

Alright.

所以，是的。

So, yeah.

这些就是我在Infinity Fest上觉得最值得关注的亮点。

Those are the highlights for me from, Infinity Fest.

我的意思是，有很多精彩的讲座，但我知道我们已经聊过这些了。

I mean, a lot of the great panels, but, I know we've talked about this for a bit.

对我来说，这两个是最突出的。

Those were the two standouts for for me.

不过，这确实是一场不错的会议，有很多非常有趣的演讲。

But, yeah, it was a a good good conference, a lot of a lot of really interesting talks.

我们来聊聊人们正在使用的工具吧。

Let's talk about some of the tools people are using.

好的。

Alright.

你在用什么？

What are you using?

最近你最常用的工具是什么？

What have your been your go to tools lately?

我更专注于图像生成，而不是视频生成。

So I tend to focus more on image generation stuff over video generation.

所以我对图像生成的每一个细节和局限性都非常敏感。

So I'm, like, hyper aware of all of the nuances and the limitations of image generation.

比如，我用的是Nano Banana。

For example, I use Nano Banana.

我的意思是，我不确定。

Like, I don't know.

我每天在Nano Banana上生成大约50次，只是为了弄清楚它能做什么、不能做什么。

I make maybe 50 generations a day on Nano Banana just to kind of figure out what it can and can't do.

你用Nano Banana的时候，是直接用文字生成图像，还是会提供一些输入内容？

Are you doing when you're using Nanobanana, are you base are you doing, like, just straight text to image, or are you giving it any input stuff?