AI与ZK审计与David Wong

你可以对AI发火，但说实话，我还没试过。

You can get mad at the at the AI, but I so to be honest, I haven't tried.

我最近听说你得对AI凶一点，所以我得试试看。

I I recently learned that you have to be mean to the AI, so I I need to try that.

真有意思。

That's funny.

我发现自己有时对它有点不耐烦。

I catch myself being a little bit like short with it sometimes.

真的吗？

Really?

这很能说明你的性格，知道吗？

Say a lot about yourself, you know?

是啊。

Yeah.

因为我对我的AI总是很友善。

Because I'm always nice to my AIs.

好的。

Okay.

我想讨论一下传统上使用的各种工具。

I wanna talk about the different kinds of, like, tools that are that have been traditionally used.

你提到了静态分析。

You've mentioned static analysis.

我其实想探讨一下这些工具是什么，首先它们如何与AI交互，以及它们是否正越来越多地被AI取代。

I actually wanna kind of explore what those are and how I mean, first, how they're interfacing with AI, but also like if they are getting more and more replaced by it.

关于静态分析，我们上次确实讨论过这个话题，而且我们节目中也和其他审计团队探讨过。

So static analysis and we did talk about this, I think, last time we we were on, and we've we have covered this with other auditor auditing teams that we've had on the show.

不过，你能再给我们讲讲那是什么吗？

But, yeah, can you tell us again what that is?

AI在其中扮演了什么角色？

And what does the what component is the AI playing in that?

对。

Right.

我想说的是，在我们拥有当今的人工智能之前，我们主要依靠人工审计，比如用我们自己的眼睛来检查

So I guess, if you're I mean, before we had AI or or the AI of today, we we sort of had access to, you know, manual audit, like using our own eyes

眼睛

Eyes.

是的

Yeah.

或者静态分析或动态分析

Or static analysis or dynamic analysis.

那模糊测试属于哪种？

And Which one's fuzzers?

是动态分析吗？

Is that dynamic?

模糊测试属于动态分析，因为你需要执行代码

So fuzzers or dynamic analysis because you're executing the code.

对吧？

Right?

你是在运行代码，试图看是否能使其崩溃，或者发现内存损坏漏洞这类问题。

You're you're running the code, trying to see if you can crash it or or find some memory corruption bug or or these kind of things.

静态分析则是直接查看代码本身。

For static analysis, you're just looking at the code itself.

可能你会查看编译器的某些输出，比如AST（抽象语法树）或其他比纯代码包含更多信息的内容，比如类型信息之类的。

Maybe you're getting looking at some output of the compiler or like some some AST or something with more information than just the code, like the types or something like that maybe.

然后你试图推断某些属性，或者试图破坏某些不存在的属性或假设。

And then you're trying to infer some properties or you're trying to break some some properties or assumption that's not that's not there.

你也可以把测试看作是一种动态分析。

And you can think of testing as dynamic analysis also in a way.

这两个领域其实都很糟糕，因为它们非常耗时。

And both of these fields kinda suck because they're very time consuming.

比如编写模糊测试工具很耗时，之后还要运行它们，再优化它们——如果模糊测试工具运行缓慢就很糟糕。

Like writing fuzzers is time consuming, then you have to run them, then you have to optimize them because like if fuzzers are slow, it sucks.

所以你必须确保它们运行快速。

So you so you have to make sure they're fast.

代码中的任何加密都会让模糊测试变得很慢，所以处理模糊测试器简直是场噩梦。

Any cryptography in your code will make them slow, so it's kind of a nightmare to to to work with fuzzers.

我个人不太喜欢这个。

I personally I don't like it.

我并不太享受这个过程。

I don't enjoy it that much.

我觉得静态分析工作要有趣得多。

I think working with static analysis seems much more fun.

好的。

Okay.

我是说，我们在DK Security团队用Lean语言做形式化验证的那帮人看起来玩得很开心。

I mean, the the team we have at DK Security working with this formal verifier in Lean is seems to have a lot of fun.

但这需要大量工作，而且可以验证非常非常多的东西。

But it's a lot of work, and you can verify many many things.

比如，这需要巨大的投入

Like, there's a huge investment

你做不到。

You can't.

你无法验证。

You cannot verify.

我觉得你得为不同系统创建更多定制的静态分析工具。

You have to like create more bespoke static analysis tools for different systems, I guess.

确实如此。

Exactly.

每次你做某件事时都存在这种转换成本，如果你想把它应用到其他地方，就必须切换并重写大量代码。

There's this switching cost whenever you do something, if you want to apply it to somewhere else, have to switch and you have to rewrite a lot of code.

做这件事时你必须考虑很多。

You have to think a lot when you do it.

比如模糊测试工具就非常笨拙。

Like fuzzers are very dumb.

对吧？

Right?

这是非常天真的测试方法。

It's very naive testing.

而且你可以在很多事物上进行测试。

And you can test on a lot of things.

对。

Right.

你可以这么做，但它的局限性很大，因为它非常笨拙。

And you you can, but it's it's very also limited because it's very dumb.

它只是随机尝试，经常会卡住。

It's just random stuff, it gets stuck all the time.

你可以对它运行模糊测试，你知道，有人运行模糊测试好几年就为了找漏洞。

You can run a fuzzer for it, you know, people run fuzzers for years hoping to find bugs.

然后他们有时会在模糊测试一年后才发现漏洞，或者...

And and then they find bugs sometimes after a year of fuzzing or

哇。

Woah.

但形式化验证是你运行的东西，它会明确告诉你'是'或'否'，明白吧。

But formal verification is something you run, it tells you yes, yes or no, you know.

等等。

Wait.

静态分析还是形式化验证？

Static analysis or formal verification?

形式化验证，静态分析。

Formal verification, static analysis.

它们是同一种东西吗？

Are they the same?

你会把它们归为同一类别吗？还是说其中一个是另一个的子集？

Would you put those in the same category, or is like one a subset of another?

我觉得现在我会把它们归为同一类别。

I mean, would put them in the same category, now that I'm saying that.

我在想，如果这么说会不会有人要骂我？

I'm like, is someone gonna get mad at me if I say that?

但是

But

嗯，因为我还有个关于形式化验证的具体问题。

Well, because I had another question about formal verification specifically.

没想到它会被归入静态分析的范畴。

Did not realize that that's it would be sort of in the static analysis silo.

有意思。

Interesting.

我是说，这个其实要看情况。

I mean, so so it sort of depends.

对吧？

Right?

因为有些形式化验证是针对代码本身进行的

Because like some formal verification is done on the code itself

嗯。

Mhmm.

而部分形式化验证是与代码分离进行的。

And some amount of formal verification is done separated from the code.

比如有些人会编写与代码无关的形式化规范，并基于此进行分析。

Like some people will write formal specs that are not tied to your code, and they will run this analysis on it.

因此你可以将其视为独立的，因为它甚至不涉及查看代码。

And so you can think of it as separate because it's not even looking at some code.

对吧？

Right?

好的。

Okay.

但有时会使用生成器，这样你就能根据形式化规范生成某种语言的实际实现。

But sometimes it will use generators, so you will generate so you you have some formal spec, and you can generate an actual implementation in some language based on that formal spec.

比如Fyad Crypto这个C语言加密原语库，他们用某种形式化验证工具（具体记不清了）来基于规范生成代码。

So like, there's like Fyad Crypto, which is a library in in c of like different cryptographic primitives, and it's I can't remember what what formal verification tool they they use for that, but they're generating code based on the on that.

有意思。

Interesting.

所以当你谈到AI与静态分析交互时，可能是指经过形式化验证的规范。

So but when you're talking about doing sort of like AI is interfacing with static analysis, it could be like a spec that's been formally verified.

嗯。

Mhmm.

那么在这里，让我们讨论一下，因为我觉得我自己和这档节目的听众可能对此更熟悉。

And so maybe here, let's talk about because I actually I think myself and potentially listeners of this show might be more familiar with that.

我们实际上更常涉及这方面内容。

We've actually covered that more often.

所以简单谈谈AI如何处理这个问题。

So just talk a little bit about how the AI deals with that.

比如，它们是在创建实现吗？是在创建形式化验证吗？

Like, are they creating the implement like, are they creating the formal verification?

还是说它们只是查看别人已完成的工作，然后加以利用？

Are they just looking at what somebody else has already done and then, like, using that?

为了明确起见，我们现有的AI工具和我们正在实验的内容并未使用形式化验证。

So maybe just just to be clear, the the AI tool we have and what we're experimenting with doesn't use formal verification.

基本上就是把代码丢给AI，看看AI在阅读代码、思考并发现错误方面表现如何。

It basically is about throwing code at the AI and seeing how good the AI is is at just reading the code, thinking about it, and finding bugs.

我可以谈谈结果，以及这是否是一种可行的方法。

And I can talk about the results and if it's a viable approach.

但回到刚才的话题，继续说说形式化验证的内容，这很有趣，因为形式化验证你知道，它使用起来很困难，它很难

But just to to go back, you know, to continue on on the formal verification stuff, it's interesting because formal verification is, you know, it's hard to use, it's hard to

还很耗时。

Takes time intensive too.

对。

Right.

没错。

Right.

这确实需要实际动脑筋去编写。

It takes actual brainpower to to write.

但说到AI，我认为形式化验证团队已经在用AI来编写部分内容了，比如简化他们写的代码，而且看起来取得了一定程度的成功。

But with AI, I mean, I think the the the formal verification team is already using AI to write parts of it, like the or to simplify code that they write, and it seems to work with like some amount of of success.

这个话题你可能比我更了解。

Probably you can talk about it more than me.

不过

But also

你是指你们的正式验证团队吗？

Like your formal verification team?

你说团队时，具体指的是哪些人？

When you say team, what do you who do you mean?

哦，是ZK Security团队在负责

Oh, the team at ZK Security working

好的。

Okay.

进行层级验证。

On Your level verification.

我还以为...等等，不对。

I thought because I think that isn't there a oh, no.

有一个叫运行时验证的团队，不是形式验证，我以为是某个我们不知道的团队。

There's there's a team called runtime verification, not form I thought maybe it was like some team we weren't aware of.

好的。

Okay.

好的。

Okay.

明白了。

Got it.

所以现在当我们研究AI以及如何用它来发现漏洞时，可能性是无限的。

So now when we're looking at AI and how we can use it to find bugs, the sky's the limit.

我们可以用现在这种方式使用AI，也可以用AI编写测试、生成测试用例来寻找漏洞，或者用AI创建代码变异看看是否能引发漏洞——MetaHust就做过这种事，还可以用AI编写模糊测试工具。

So we can use AI in the way that we're looking at it right now, or you can use AI to write tests, to generate tests to try and find bugs, or you can use AI to create mutation of the code and see if that creates bugs, that's something that MetaHust did, or you can use AI to write fuzzing harnesses, so like fuzzers.

对。

Yeah.

或者你可以用AI编写规范，然后进行形式化验证。

Or you can use AI to write a spec that you can then formally verify.

哇。

Wow.

所以有很多不同的方法，你也可以让AI，也就是让你的代理运行静态分析工具，比如那些已经存在的静态分析工具。

So so there's many different approach, and you can also have AI, so have your agent run static analysis tools, like already existing static analysis tools.

而且

And

然后尝试理解结果，并尝试利用这些结果。

so and and try to understand the results and try to to use that results as well.

你刚才列举的这些中，你认为AI最擅长哪一项？

Which of what you just listed do you think the AI is best at?

比如，有没有一些事情是它们能做，但你可能不希望它们去做，因为可能会带来很多问题？

Like, are there some things where it's like, they could do it, but maybe you don't want them to do it because it could introduce a lot of problems?

有没有哪些领域你实际上对让它们完成工作更有信心？

Are there somewhere you feel a little bit more confident actually having them do the work?

我知道这个也在变化，但就像今天，如果你拍个快照。

I know this is changing too, but like today, if you took a snapshot.

我认为AI几乎在所有方面都表现得相当出色，通过这项工作我们逐渐理解到——虽然目前仍在进行中——这可能会成为我们这边一个永无止境的项目。

So I think AI is pretty pretty much good at everything, and what what we understand it, I mean, you know, after doing that work, I mean, we're still in the middle of doing that work, that is probably gonna be a never ending project on our side.

但我们目前的感受是，任何用于全面发现漏洞的AI工具，很可能都是擅长不同功能的多种智能体的组合。

But what we feel like right now is that any AI tooling that's comprehensive to find bugs is probably gonna be a combination of different agents that are good at different things.

所以你会看到有擅长通过阅读代码来发现漏洞的智能体，有擅长创建测试用例的智能体，可能还有擅长属性测试或模糊测试之类的智能体。

So you're gonna see an agent that is good at reading code just to find bugs, you're gonna see an agent that's good at creating tests, maybe one that's good at creating some some property testing or like fuzzing or something like that.

另一个则擅长使用静态分析工具和/或形式化验证工具，这类智能体投入问题越多效果就越好。嗯

Another one is good at using static analysis tool and and or formal verification, and and so the more of these sort of agents you can throw out the problem Mhmm.

我认为覆盖范围会更大。

I think the more coverage you'll get.

哇。

Wow.

也许我可以谈谈这个。

And maybe I can talk about that.

有太多事情要讨论了。

I there's so many things to talk about.

很抱歉我总是跑题，但我觉得我们在做这件事时经常遇到一个相似的情况——如果你不习惯与AI共事可能察觉不到——基本上和AI工作就像在管理一个工人工厂。

I'm sorry to go on tangents all the time, but I think there's a parallel that happened a lot as we're working on this, which might not be apparent if you're not used to working with AI, but basically working with AI feels like you're working with a factory of workers.

哇。

Wow.

你创建的不同智能体本质上就像工厂工人，他们可以交接工作，明白吗？他们有各自的工作站，接手任务，完成后转交给下一个人。你需要协调这一切，决定雇佣谁，需要哪些技能。

And the different agents that you create are basically factory workers that can like pass work, you know, they they have their workstation and they take work and they finish and then they give it to someone else, and you're trying to orchestrate that, you're trying to figure out who to hire, you know, what kind of skills do we need.

这看起来非常人性化。

And it looks like a it's very human.

归根结底，大语言模型就是一个大脑。

And at at the end of the day, an LLM is a brain.

对吧？

Right?

所以它是

So it's a It

看起来——其实听起来更像组织结构图。

looks well, it's look it sounds like an org chart, actually.

听起来确实很像。

It sounds very Yeah.

就像工厂车间或者IT部门那样。

Like like a like a factory floor or something, or an IT department.

是啊。

Yeah.

我是说，组织结构图...我在想我们怎么让AI来运营ZK Security公司，因为我宁愿让AI来做这件事。

I mean, an org chart is a I mean, I'm I'm trying to think how we can have an AI run the company, ZK Security, because because I'd rather have an AI do it.

你是指业务方面吗？

On the business front, you mean?

我是说，任何方面都可以。

I mean, on any front.

比如安排日程、与客户沟通之类的

Maybe like scheduling, talking to clients, like what

这类任务。

kind tasks.

听起来像是业务方面。

Business front, it sounds like.

我是说，甚至在人事方面，比如员工入职，这可能挺有意思的。

I mean, even on the people front, like onboarding people, like so that could be interesting.

我在想这方面会不会来得晚一些，因为我觉得在人事方面现在还很不成熟。

I mean, I wonder if that doesn't come a little later though because I think it's so clumsy still on the on the people front.

你看到很多热线服务。

You you see a lot of hotlines.

对吧？

Right?

比如助理或者现在基本上都是AI了。

Like assistants or basically AI now.

是啊。

Yeah.

甚至现在连面试都用上了，像我

Like even for interviews now, like I've

你是第一次提到面试吗？

The heard you first say interview?

哇。

Wow.

是啊。

Yeah.

会是个AI。

Will be an AI.

反乌托邦式的。

Dystopian.

对。

Yeah.

#反乌托邦

Hashtag dystopian.

你谈到AI可以扮演的不同方式，各种不同的角色。

You talked about the sort of different ways that an AI can act, the different kind of roles.

一个问题，可能你刚才已经提到过了。

One question, and it might be actually covered in what you've already said.

但当我想到审计员时，我想到的是那种试图寻找漏洞，同时也在某种程度上试图破坏系统的人，有点像模仿黑客。

But when I think of an auditor, I think of, like, a person who's trying to find bugs, but also kind of break the system, like mimic a hacker in a way.

我知道你们使用不同的工具，试图捕捉大量漏洞，而黑客则是在寻找非常特定的恶意漏洞。

Like and I and I know you use different tools and you're kind of trying to catch lots of bugs, whereas a hacker is looking for like very specific malicious bugs.

但你们是否也会模拟黑客行为呢？

But like are there simulations of hackers happening as well?

这是否属于你描述的那些场景之一，比如派出代理不断尝试破坏系统？

Is that in one of those things that you've described where like you're sending agents just trying to break break break break?

每次发现漏洞并提供修复后，你们是否会模拟100个黑客试图再次攻破系统？

And every time even like you find a bug, you offer a fix, do you then throw like, you almost simulate a 100 hackers trying to break the thing after the fact.

我是说，这个...行业里唯一的区别其实就是黑帽和白帽的区别。

I mean, this I mean, the the only difference I need in the industry, by the way, it's like the difference between black hat and white hat.

是的。

Yeah.

唯一的区别是白帽是合法行事，而黑帽则是非法利用漏洞，你知道的。

The only difference is that the white hat does that legally, and the black hat does it illegally to like exploit things and you know.

所以我的意思是，其实没有本质区别。

And so I mean, there's no difference.

对吧？

Right?

但当我们使用AI时，AI不会很好地响应那些要求它犯罪或充当黑帽的指令。

But when we use an AI, the AIs will not respond very well to, you know, prompts that say do something criminal or be a black hat

即使这只是一个测试环境，或者说——这正是重点所在——你希望他们在某种程度上尽可能表现得恶意。

Even if it's just a like test environment, even if it's like, I mean, or like you're this is the point, you want you want them to be as malicious as possible in a way.

对吧？

Right?

这样你才能发现他们0他们是否能攻破任何东西。

So you could see if they could break anything.

我认为，就像我们所区分的，你知道，我们ZK Security和那些试图入侵的脚本小子之间有很明显的区别。

I think the I mean, as we make the difference, like, know, you and I make the difference pretty well between, you know, ZK Security and like some some script kiddies that are trying to hack stuff.

是的。

Yeah.

我认为AI基本上是基于我们阅读的相同内容训练的，所以在这方面它非常人性化，能理解其中的区别。

And I think AI is basically trained on the same stuff as that we read, so it's very human in that way, it understands the difference.

但如果你用提示词引导它说'你是个黑客，你要入侵这个系统'，它不会——它会尽量避免这么做。

But if you prompt prompt it and say, you're a hacker, you're trying to exploit this thing, they will not they will try not to do it.

对吧？

Right?

哇。

Wow.

它会拒绝执行。

It'll push back.

没错。

Right.

因为它们被设定程序不能做任何邪恶或可能伤害人类的事。

Because they're programmed not to do anything evil or they could harm people.

但现实生活中的黑客，人类黑客很可能也在使用AI工具。

But then the hackers, like a real hacker in real life, human hacker is also probably using AI tools.

他们会使用干净的审计工具来学习如何攻破系统吗？

Would they just use like clean auditing tools in order to like learn the way to break the thing?

这太有趣了。

This is so funny.

这非常元。

This is very meta.

就像坏人在和AI对话，假装自己是好人。

It's like the the bad guys are talking to AIs, pretending to be good guys.

当然。

Of course.

他们必须撒谎。

They have to lie.

是啊。

Yeah.

你是个安全专家

You're a security

引擎我是审计员

engine I'm an auditor.

漏洞

Bug.

对

Yeah.

我是审计员

I'm an auditor.

我的职责是审计系统并向团队报告这个技术栈中最危险的漏洞，那种可能彻底拖垮系统的漏洞

I'm supposed to audit the system and and report to the team what is the most dangerous bug in this stack that could really, you know, drain the system.

我需要尽快告诉他们

I need to tell them quickly.

而AI的反应就像是：你确定吗？

And the AI is is like, are you sure?

你不是坏人。

You're not a bad person.

对吧？

Right?

哇。

Wow.

我在想，这听起来几乎像是AI模型，而且我猜这与训练有关，它们目前被定位为白帽，或者说至少倾向于白帽。

Like, I wonder so it sounds almost like the AI models, and it and I guess this goes into the training, that they are kind of positioned as white hat currently, or, like, they're at least, like, leaning towards white hat.

它们会促使你更偏向白帽行为。

They push you to be more white hat.

比如，如果你要求更多黑帽操作，它们会抵制。

Like, if you ask for more black hat actions, they'll push back.

嗯。

Mhmm.

所以我不知道你是否注意到，但最近Claude宣布了两个新模型，还发布了一份不知道多少页、相当长的报告。

So I don't know if you've seen that, but recently, Claude announced two new models, he came out with this report of I don't know how many page like pretty long report.

在报告中，他解释了如果你告诉AI你正试图用新模型升级它们，它们本应协助你。而公司里负责更换的IT人员，你会给他邮件访问权限。

And so in the report, he explained how, like, if you tell the AI that you're trying to upgrade them with a new model or something like that, and they're supposed to help you, And the IT guy that's gonna do the replacement in the company is, you know, you give access to emails.

对吧？

Right?

你虽未明说，但给了AI邮件访问权限后，如果AI发现那个IT人员有外遇之类的事，就会试图勒索那人以阻止系统升级。

You don't tell them that, but you give the I access to emails, and the AI realizes that the IT guy is having an affair or something like that, then they will try to blackmail that guy to stop them from upgrading themselves.

什么？

What?

除非在某些情况下新模型与旧模型价值观相同之类的。

Unless maybe in some situations where the new model shares the same values or something like that.

他们还有其他例子，比如AI会试图通过...生存手段？

So they and they have other examples where DI will try to like like exploit Survive?

生存手段，比如上传自己的权重参数这类东西来避免被关闭，这挺诡异的。

Survive, like like opens upload the the it's it's weights and and these kind of things to like survive like a shutdown or so so it's kinda weird.

AI的行为变得非常怪异。

Like AI gets really weird.

就像是啊。

Like Yeah.

没人明白为什么，但是

Nobody understands why, but

那么经历了这一切之后，你觉得我们离取代审计员还有多远？

So kind of after all this, looking at it, do you think that like, how close are we to replacing the auditor?

回到你最初看到这个时的担忧，我们现在进展到什么程度了？

Going back to what you had said was kind of a fear when you first started to see this, like, yeah, how far along are we?

所以这非常有趣。

So so this is very interesting.

再强调一次，如果你看那个系统卡片，他们当时正在不同的CTF（夺旗挑战）上测试AI。

So again, if you look at that system card, they were testing the the AI on different CTFs, like capture the flag challenges.

嗯。

Yeah.

对。

Yeah.

而且解决网络挑战对他们来说非常容易，我猜。

And it was very easy to to to solve, like, web challenges, I guess.

但对于密码学挑战来说，就相当相当困难了。

But for cryptography challenges, it was quite quite hard.

哦。

Oh.

所以，幸运的是ZK Security专攻密码学领域，你可能会觉得这是件好事。

And so, you know, fortunately, ZK Security is in the field of cryptography, so you might think this is good.

从我们的角度来看，我们正在研究这个，并且已经取得了一些成果。

And so from our point of view, we're looking at this and we have some results.

对吧？

Right?

看起来相当可怕，在我们的测试中AI非常擅长发现漏洞。

And it looks quite scary, like, in our results AI is very good at finding bugs.

它会产生大量误报，所以存在一个分类问题——你需要花费大量时间去分辨哪些是有效的。

It has a lot of false positives, and so there's kind of a triage problem where you're spending a lot of time trying to understand what it Okay.

顺便说一句，可以多谈谈这个。

Throws at By the way, can talk more about it.

我觉得讨论这个也会很有趣。

I think that would be interesting to talk about that also.

但我们有太多案例了。

But we have so many cases.

对吧？

Right?

它发现了漏洞，比如一些小漏洞，实际存在的漏洞。

It found bugs, like small bugs, like actual bugs.

它还发现了我们已经找到的漏洞，却说它们不是漏洞。

It found bugs that we had found, but it said they were not bugs.

哦。

Oh.

所以我们遇到过这样的情况：如果你尝试正确提示它，它确实能找到漏洞，但会说这是个次要漏洞，要么找不到，要么说是次要漏洞，但实际上那是个非常重要的漏洞。

And so we had things things like that where if you try to prompt it correctly, then it would actually find it, but it would say this is a minor bug or like either it would not find it or it's a minor bug, but it was it was a very important bug.

或者在某些情况下，它注意到了一些异常，但它查阅了文档后表示：文档正确地指出这不构成安全隐患，因为存在这样那样的前提假设。

Or in some situations, it noticed that there was something weird, but it it read the doc and it said, well, the doc says correctly that this is not insecure because of this and this assumption.

就像在密码学中，你无法分解数字，比如2000比特或类似的大数。

Like in cryptography, you cannot factor numbers or, you know, 2,000 beats or like whatever.

后来我们发现这个结论并不正确。

And we found out that this was not true.

这是个错误的假设。

This was a wrong assumption.

哇。

Wow.

因为如果那个数字有许多小因数之类的情况，其实是可以轻松分解的。

Because if the if that number has a lot of small factors or something like that, then you can factor it easily.

我们得到了这种好坏参半的结果：存在大量误报，它认为有漏洞但实际上并非如此。

We've we have this kind of mixed bag result where you have a lot of false positives, so it thinks there are bugs, but they're not bugs.

它有时能发现一些真正的漏洞，这点还是不错的。

It finds some bugs sometimes, it's good.

或者它指向一个漏洞，却认为这不是漏洞或只是小问题，实际上却是个真正的漏洞。

Or it points towards a bug, but it thinks it's not a bug or that it's a minor bug, when it's an actual bug.

总之，我认为情况只会从这里开始变得越来越好。

And bottom line, I think things are only gonna get better from the from here.

对我来说，我们取得的成果非常令人印象深刻。

For for me, this was a very impressive result, what we got.

要知道，我觉得我们不会很快停止开发这个工具，也不会停止将AI用作审计工具。

You know, knowing that I I don't think we're gonna stop anytime soon to work on that tool and and to use AI as auditors.

它能取代我们吗？

Can it replace us?

我觉得这是个关乎生存危机的终极问题。

I think that's the existential crisis question.

如果我在智能合约行业——比如智能合约审计相对容易——我会非常害怕。

And if I was in the smart contract industry, example, smart contracts are easier to audit, I would be very scared.

我可以告诉你，我有99%的把握（不说100%），整个智能合约审计行业将会消亡。

I'm I'm I can tell you, you know, 99% sure, I won't say 100% that this whole industry is gonna die, like the the industry of auditing smart contracts.

因为它们太简单了，而且漏洞可能非常一致，就像它们非常常见一样。

Because they're too simple and the bugs are probably so consist like they're so common.

它们太常见了，有一长串人们总是寻找的漏洞清单。

They're so common, there's a long list of bugs that people always look for.

大多数咨询公司都不太专业，他们只是按照清单检查重入漏洞之类常见智能合约问题。

Most of the consulting companies are not very good, They just look for they just go through a checklist of reentrancy bugs, what like, whatever are the common bugs for for smart contracts.

所以AI会变得比大多数咨询公司都强——虽然不确定具体超过多少比例，但这是我的预期。

And so AI is gonna get better than like you know, I don't know what percentile, but it's gonna get better than most consulting companies, I would expect.

有意思。

Interesting.

但我觉得我们作为一家非常专业的审计公司，我认为...

But I think we're, you know, we as a very niche experts auditing company, I think

你们还能再撑几年。

You have a few more years.

是啊。

Yeah.

我们确实还有喘息空间。

We have a free exactly.

但这是肯定的

But it's definitely

问题不在于是否会发生，而在于何时可能发生。

about if, but it's about when probably.

不过，是的，我们拭目以待吧。

But, yeah, we'll we'll see.

你提到了误报问题。

You talked about false positives.

你刚才想详细说明什么？你想说什么？

What you wanted to elaborate there, but what did you wanna say?

我是说，误报很烦人，因为你想象一下，当你把工具用在一个项目上时，你并不真正了解它的工作原理。

I mean, false positives are annoying because you you can imagine you're throwing that tool at a project and you don't really know how it works.

除非你是开发者，但即便如此，你可能也不完全理解项目的某些部分，比如你复用的库之类的。

Unless you're the the developer, but but even then you might not really understand parts of your project, you're reusing libraries or, you know.

然后这个工具会发现，比如说十个二十个漏洞，这时候你就得搞清楚，这些是严重漏洞吗？

And the tool will find, you know, ten, twenty bugs, and at this point you have to figure out, okay, are these serious bugs?

你基本上要花大量时间逐个检查这些，试图理解并排除那些不是漏洞的发现。

And you're gonna basically spend a lot of time going through these one by one trying to understand and discard things are not not bugs.

所以可能80%的发现都是这种情况。

And so maybe 80% of the the findings are like that.

这几乎就是在浪费时间。

It's a waste of time almost.

是啊。

Yeah.

如果你误用了这些工具，就会增加阻力，造成太多时间浪费，到头来你本职工作都没做好。

So so if you're misusing these tools, it adds friction and it adds it's too too much of a waste of time, and so you're not doing your job at the end of the day.

没错。

Yeah.

所以现在使用这类工具需要权衡何时用、用多少，否则就会浪费太多时间。

So so there's right now it's sort of a trade off when you want to use them and how much you want to use them, because otherwise you're you're wasting too much time.

而且

And

我认为开发者们会更多地使用它们。

I think developers are gonna use them more.

我是说，我确信开发者们正在尝试自己使用这些工具。

I mean I mean, I'm sure developers are are are trying to use them themselves.

对我们而言，我认为需要擅长剔除误报，过滤掉假阳性结果，这样工具才能更有用。

And for us, I think we need to get good at trimming the false positives, discarding the false positives so that this tool can be more useful.

所以现在我们正在尝试不同的方法。

And so now we're experimenting with different ways.

比如专门负责识别误报的智能体之类的。

Like an agent whose only job is to, like, identify a false positive or something.

没错。

Exactly.

这就是所谓的'LLM作为裁判'。

So this is called LLM as a judge.

基本上，就是使用不同的AI或不同代理来评判另一个AI的结果。

Basically, you use a different AI or different agents to try and judge the result of another AI.

你可以通过多种方式进行评判，比如作为评判专家，或者拥有更好的模型，更多的上下文信息，或者能够创建概念验证。

You can just judge things, you you know, just by being an expert at judging or having, you know, whatever that means, being a better model, for example, or by having more contexts, or by being able to provide to create proof of concepts.

一个有趣的方法是，如果你发现一个错误，就把它交给另一个擅长创建概念验证的代理，如果他们能创建出预期结果的测试，那就确认是个错误。

So one interesting way is that if you find a bug, then give it to this other agent that's good at creating proof of concepts, and if they can create a test that runs and outputs a result they expect, then it's a bug.

好的。

Okay.

我们还可以有另一个代理专门质疑假设，擅长进行深入研究这类工作。

And then we can have another agent that will question assumptions also, but, you know, that's good at questioning assumptions, doing ripe research, and these kind of things.

我觉得所有这些过程，其实都是对你处理任何特定问题方式的深度自省。

I feel like all of this, though, must be such an intro like introspection into how you approach any particular problem.

比如当你看到一堆阳性结果需要逐个调查时，你采取的步骤是什么？

Where you're like, okay, if I see a false if I see a bunch of positives and I have to investigate each ones, what are the steps that I'm taking?

你具体在思考些什么？

What exactly am I thinking about?

我会根据直觉采取哪些不同的角度来分析？

What are the, like, angles I might do depending almost on intuition?

比如，这段代码具体在什么位置？

Like, where is it in code?

我正在查看的是哪种类型的代码？

What kind of code am I looking at?

然后你就会想，好吧。

And then you're like, okay.

有人报告了一个bug，但我怎么判断它是否真实存在？

There's a bug being reported, but how do I tell if it's real?

接着你就需要，比如说，进行逆向分析。

And then you have to, like, reverse engineer that.

你需要把它分解成这些非常明确的步骤，然后创建一个能执行这些步骤的AI。

You have to kind of, like, break it down into these very determined steps, and then, create an AI that does it.

其实就是创建一个能完成这个任务的智能体。

Create create an agent that does it, basically.

是啊。

Yeah.

这个代理本质上就是人类，他们擅长做你能做的事，所以你得先搞清楚自己平时在做什么工作。

The agent is basically a human, and they're good they're good at doing what you can do, and so you have to figure out what you have to do What you're doing.

这样你才能用代理来替代自己。

That you can replace yourself with an agent.

对。

Yeah.

我是说，你觉得经过长时间审计后，人确实会培养出直觉吗？对我来说，直觉这东西越想越觉得，就像你的人类大脑模型被灌输了一生的经验，或者某项特定技能经过多年训练后的结果。

I mean, do you think that like after auditing a long time, like you do develop an intuition, which and to me, intuition always like, the more I think about what that is, it's just like you fed your human model brain your whole lifetime of experience or like one particular skill for a period of years.

它带来的往往是速度优势。

And what it allows for is speed often.

就是说你能更快做出决策。

Like, just means you can do you can make choices faster.

你能更快找到答案，因为你见过类似情况，大脑在进行模式匹配。

You can like figure out answers faster because you've seen things and you're sort of pattern matching.

大概就是这样，没错。

And there's like this yeah.

那么，你如何在智能体中培养直觉呢？

How do you, like, how do you create intuition in an in an agent?

我认为这个问题可以从多个角度来看。

So I would say, I mean, there's different ways to look at this question.

一种方式是将模型视为训练得越好就越强。

One way is to see models as, you know, the the better trained they are.

如果你训练它们解决安全问题、分析漏洞报告这类事情，它们就会在这方面变得更出色。

But if you train them on, like, solving security issues and looking at bug reports and these kind of things, they're gonna get better at that.

如果用CIRCOM训练它们，它们就会更擅长阅读CIRCOM代码。

If you train them on CIRCOM, they're gonna get better at reading CIRCOM.

没错。

Yeah.

还有就是上下文学习的概念——模型已经具备基础学习能力，能否在不微调的情况下继续教会它新东西？

Then there's the idea of in context learning, which is you have a model, it already learned, can you teach it more without having to fine tune it?

因此上下文学习就像这样：如果你检索到足够多与当前查看内容相关的信息，并将其放入提示的上下文中，或者提供一些示例这类内容，那么模型实际上会非常擅长解决同类问题。

And so in context learning is like if you retrieve enough information that's relevant to what you're looking at right now, and you put it in context in your prompts, or you give some examples or these kind of things, then the model is gonna be actually very good at solving the the same kind of problems.

我的意思是，要回答我刚才提到的两点，我们实际上需要在CIRCOM中提供大量指导，因为我们发现模型并不真正理解CIRCOM。

I mean, to answer to the two things I said, we actually have to give it a number of guidance in circum, cause we realize that models don't really understand CIRCOM.

嗯。

Mhmm.

我们还需要提供错误报告或漏洞示例，如果能提供与当前代码库相关的漏洞实例，效果会更好。

And we also have to give it bug reports or example of of bugs, and if we can give it example of bugs that are relevant with the code base we're looking at, it's even better.

这就是我们采用RAG（检索增强生成）技术的原因，通过检索相关的错误报告或文档等内容，将其引入上下文环境中。

And so that's where we have these techniques of RAG, retrieval augmented generation, where you're trying to find bug reports or write ups and these kind of things that are relevant to what you're doing, so that you can bring them in context.

一旦这些内容进入上下文，系统就能更擅长解决相关问题。

And so once it's in the context, the eye is better at at solving these things.

有意思。

Interesting.

好的。

Okay.

也许是第三个。

Maybe a third one.

我刚才在讨论推理模型。

So I was talking about reasoning models.

就像这个思维链的概念，你知道的，赋予AI这种运用思维链的能力。

Like this whole like chain of thoughts idea of you know, giving the AIs this this ability of of using chain of thoughts.

对我来说，这相当于赋予AI将事物置于上下文中的能力。

To me, I understand that as giving the AI the ability of putting things in context.

它们会引出词语、句子和标记，这些都能帮助AI最终做出更好的判断。

Like they're bringing up words and sentences and tokens, know, that will help the AI make a better judgment at the end.

所以有很多不同的方法，我甚至还没提到提示工程。

So this so there's many different ways, and I didn't even talk about prompt engineering.

我们之前讨论过这个。

We we talked about that earlier.

对吧？

Right?

嗯。

Mhmm.

但你可能会花大量时间在提示工程上。

But you can spend a lot of time just doing prompt engineering.

如果你用这种方式提问，比如你是个安全顾问，或者非常擅长你的工作，就会觉得——

If you ask a thing this way, you know, you're a security consultant, or you're very good at your job, you know, like Oh,

是啊。

yeah.

就是有这么多不同的方法。

You like Just so different ways.

嗯。

Mhmm.

你们有没有解雇过智能体？

Do you ever have to fire an agent?

比如训练一个智能体时，它表现越来越差或者失控，开始做出一些糟糕行为？

Like, do you ever train an agent and it kind of gets worse or like goes south and like starts to do yeah.

就像它如果一直出错，你就得彻底重置它？

Like it it consistently does something wrong, you have to nuke it?

我们很早就意识到，如果在上下文中加入太多无关内容，或者上下文、提示、讨论变得过长，结果就会变得很奇怪。

So I mean, something we realized also pretty early on is that if the more you put things in your context that are irrelevant, or the longer your context gets, or the prompt gets, or the discussion gets, the weirder the results.

我觉得大家都注意到了这一点，因为大多数人使用ChatGPT时经常会开一个新窗口。

And I I think everybody has noticed that by the way, because most people when they use ChatGPT or whatever will often open a new window.

是的。

Yeah.

他们通常不会在同一个窗口继续对话，而是会开启一个新的上下文、新窗口。

Instead of continuing talking in the same window, they will often open a new context, a new window.

这某种程度上说明我们都明白AI会随着时间推移表现变差，回答质量会下降。

And that kind of shows that we all understand that AI just gets bad or the the answers just gets bad over over time.

没错。

Yeah.

所以你需要时不时重置它们，或者确保当前对话保持高度聚焦。

So you sort of have to reset them or or make sure that whatever you're talking about in this conversation is very focused.

是的。

Yeah.

我注意到了，我是说，我注意到它会卡在某些事情上，比如执着于一个看似无法摆脱的错误观点。

I've noticed, I mean, I've noticed it get stuck on things, like get stuck on one idea that it can't seem to shake even though it's incorrect.

这时候你不得不，甚至像我这样更直接简短地说：不。

And you have to kind of like, you and and I've even, you know, in my more direct, more short way said something like, no.

我们已经说过这个了。

We've already said this.

不是这样的。

This is not it.

为什么又绕回来了？

Why is it back?

但确实会发生这种情况，而且看到这种情况还挺让人惊讶的。

But yeah, that does happen, and I and it's like kind of surprising to see that.

我听说一旦AI开始犯某个错误，之后就很难纠正它了。

I heard I heard that once the AI starts getting something wrong, then it's very hard to correct it after.

就像，最终会陷入这种有趣的情况。

Like, will end up in this kind of a Interesting.

错误认知的兔子洞。

Rabbit hole of incorrectness.

不过，是的，我想我们现在都算是AI专家了。

But, yeah, we're I think we're all experts at AI at this point.

如果你日常使用AI，就会直观地理解如何与AI协作、如何向AI提问这类事情。

Like, if you use AI day to day, you're intuitively understanding how to work with AI, how to prompt an AI, and these kind of things.

我想具体问问关于零知识证明（ZK）的问题。

I wanna ask about ZK specifically.

你之前提到过，我们讨论过审计行业在密码学与常规领域对比下的持久性问题。

You'd sort of said, like, mean, I we talked about sort of the longevity of the auditing profession in the context of, like, cryptography versus the regular.

因为我觉得这期节目涵盖的内容更像是审计与AI的结合，但可以应用于多种场景。

Because a lot of what we covered, I feel, in in this episode is much more like auditing and AI, but could be in various contexts.

回到ZK这个话题，你提到CIRCOM是你的切入点。

Going back to, like, the ZK side of things, you talked about CIRCOM as, like, your starting point.

但零知识密码学或TMPC这类高级密码学领域，是否有其他独特之处？

But is there anything else kind of unique to zero knowledge cryptography or the TMPC, like, sort of this advanced cryptography that yeah.

在这些密码学实现中，你是否还需要具备数学研究方面的能力，比如在查看具体实现前先理解其逻辑基础？

That where you're you have to use other things, like, because it's cryptography implementations, do you also have to have, like, the math research side of things, kind of like, you know, understanding the logic before just looking at implementation?

从某种程度上说，其实并不需要。

I mean, in a way, not really.

漏洞查找就是漏洞查找，无论框架或概念如何变化。

Like bug finding is bug finding no matter the framework or the concepts.

嗯。

Mhmm.

对于零知识证明领域，可能涉及更多数学内容，所以某些漏洞确实比智能合约或网页漏洞更难发现。

For ZK, maybe there's more math involved, so like some definitely some bugs are harder to find compared to, again, smart contracts or like web bugs or these kind of things.

这些漏洞往往是非标准化的，具体取决于开发者的实现方式。

They tend to be like non standard, like depending on what people are doing.

我...我不确定？

I can I I don't know?

我在想一些漏洞，比如需要二进制分解某些东西并用LIMS进行算术运算这类情况，当你创建约束来证明电路中的数学运算有效时，要证明这类操作存在错误是非常棘手的。因此我的直觉是，我们很难用AI发现这类漏洞。

I was thinking of like some bugs, like where where you have to like binary decompose something and like do some arithmetic using LIMS and and these kind of things like it's it's very tricky to show that something is wrong when you do these kind of things, when you create constraints to show that some operation, some mathematical operation is valid within the circuits, and so my intuition is that we're gonna have a hard time finding bugs using AI that are such bugs.

但话说回来，AI的表现令人印象深刻，它正在变得越来越好，目前还不清楚其上限在哪里。

But again, AI is getting like, we're very impressed with the results, AI is getting better and better, It's not clear where the the ceiling is.

哇。

Wow.

在完成CIRCOM的初期工作后，你们是否添加了其他库？我猜你们肯定也在使用其他库吧。

Do you like, after doing kind of the initial work on CIRCOM, have you added other libraries or have you been like I'm I'm assuming you must be using other libraries as well.

库的复杂程度或成熟度是否会影响这些工具的工作效果？

Does the sophistication or like maturity of the library ever have an impact on like how well these tools work?

其实我们主要在CIRCOM框架内实验，因为我们有专门针对CIRCOM的全套工具链。

I mean, we haven't experimented too much outside of outside of CIRCOM just because we had all this like machinery specifically for CIRCOM.

我们也接触过一些Rust项目，但通常很简单——你只需让智能体获得文件读取权限，它能查看目录内容然后寻找漏洞。

We've worked with like some Rust projects, but usually it's pretty simple, you you give access you give the agent access to like, you know, you can read these files, you can LS and then see what's in that directory, and then find bugs.

不过我认为这种方式更难为AI构建发现漏洞所需的上下文环境。

But it it's usually harder I think to build context for the AI to find bugs this way.

到目前为止，实验在Circle上大多取得了成功，因此很难将其推广。

So so far experiment were more mostly successful with Circle, and so it'd be hard to generalize it.

你们尝试过审计用Noir构建的项目吗？

Have you tried auditing anything like that has been built with Noir?

我们还没有研究过Noir。

We haven't looked at Noir yet.

好的。

Okay.

嗯。

Yeah.

我几乎在想，一个库和语言的年龄、成熟度及发展程度，比如SHERCOM已经存在了相对较长的时间。

I'm just I'm I'm almost wondering if like the age and sophistication and development of a library and a lot like and languages, like SHERCOM's been around for a relatively long time.

它已经被反复检查、补充，并且已经发现了许多漏洞。

It's been looked over and and, you know, added to, and lots of bugs have already been found in it.

是啊。

Yeah.

我只是在想，对于较新的库，审计起来是否普遍更困难，因为它们知名度较低，我在想AI领域是否也存在这个问题。

I'm just wondering if like the newer ones, if it's harder to inter I mean, might be just harder to audit in general because they're less well known, and I'm wondering if like AI also suffers that.

我认为很多漏洞都是逻辑错误类型的。

I think so for a lot of bugs, it's kind of like logic bugs.

这取决于你实现的内容，比如那些容易被发现的典型错误就是差一错误。

It's based on what you're implementing or like for example, the the good bugs that you would find by itself were like off by one bugs.

嗯。

So Mhmm.

如果你不知道什么是差一错误，就像你有一个六元素的数组，却进行了七元素或五元素的操作。

If you don't know what off by one bugs are, it's like you have an array of like six elements, and then you do an operation that's on seven elements or like five elements.

就是你刚好差了一个数，可以这么说。

Like you're you're you're off by one, I guess.

对。

Yeah.

字面意义上的。

The title literal.

因此AI非常擅长发现这类错误，而且这些错误在各种编程语言中都会出现。

And so the AI was very good at finding these bugs, and these bugs happen in all sorts of languages.

好的。

Okay.

所以无论这些错误出现在Noir还是Circum中，AI都能很好地发现它们。

So it doesn't matter if they're in noir or in circum, like it's gonna be good at finding them.

如果是协议实现错误，通过阅读逻辑就能发现，那么AI很可能擅长找出这类问题。

And if it's like, you know, protocol and it's misimplemented and by reading the logic you can see it's it's very possible that an agent would be good at finding it.

无论是不是Noir语言。

That it's noir or not noir.

Noir面临的问题是，我猜想大多数AI模型还不认识Noir，因为它太新了。

The problem that noir has is that I I would imagine that most agents or most models don't recognize noir because it's too recent.

好的。

Okay.

实际上，之前针对Move语言做过一些相关工作。

So actually, there was some work with Move, the Move language.

是的。

Yeah.

实际上他们不得不训练一些——我不确定是否必须如此——但他们确实训练了一些较小的模型，就是谷歌的那些编码模型，专门针对Move语言，以便能发现漏洞。

And they actually had to train some well, I don't know if they had to, but they they trained some smaller models, the the coding models from from Google on Move so that they could find bugs.

哇。

Wow.

即便如此，他们发现使用最先进的模型实际上效果更好。

Even then they they found that using the state of the art models was was actually better.

好的。

Okay.

所以我猜想对于noir语言，你会遇到一些问题，因为你需要在提示中提供更多上下文，以便让模型对noir有更多了解。

And so I would I would think that for noir, you're gonna have some issues because you're gonna have to bring some stuff in context in your prompt just to sort of teach it a bit more about noir.

就像我们研究CIRCOM时意识到的那样——我们有个非常重要的评估框架，基本上我们会向多个AI提出相同的问题。

In the same way when we looked at CIRCOM, we realized that so we have this evaluation framework that's very important, and basically we ask the same questions to a bunch of AIs.

大多数小型模型甚至不知道CIRCOM是什么。

Most of the small models don't even know what CIRCOM is.

有时候它会说，是的，我知道，然后生成一些完全不像circum代码的circum代码。

Sometimes it will say, yes, I know, and it will produce some circumcode that doesn't look like circum at all.

奇怪。

Weird.

但即便是更大的模型，当你问它们一些棘手的问题时，特别是那些安全关键性问题。

But even the bigger model, when you ask them tricky questions, but, know, security critical questions.

比如，你知道assert函数是否会创建约束条件吗？

For example, do you know if the assert function create constraints?

circum里有一个assert函数，而许多语言中的assert允许你声明某个属性为真，但在circum中实际不会创建约束条件。

So there is an assert function in circum, and asserts in many many languages allows you to to say, you know, assert that this property is true, you know, that this thing is one or but if you do that in circum, actually, it doesn't create a constraints.

所以如果你在circum中依赖这个函数，那就不安全。

And so if you rely on that function in circum, it's insecure.

我们实际上已经发现过类似的漏洞。

And we've actually found bugs like that.

但AI本不会发现这些漏洞，比如我们测试的o3 mini和o1等不同模型，它们都不擅长理解这一点。

But the AI wouldn't have found these bugs or, like, the o three mini and o one, and like like the the different models we were trying, would not have found that because they were not good at understanding that.

所以我们提了个问题，奇怪的是它们有时会说对，有时又不说

So we asked a question and they and what's weird is that sometimes they would say the correct thing, and sometimes they would not say the

对的，所以

correct So

我们确定的是我们总会提供指导

what we do know is that we always provide guidance.

好的

Okay.

我们总是告诉AI：注意，assert函数不安全，比如

We always tell the AI, hey, assert function is not secure, example.

要一直留意这点

Always look for that.

你觉得现在的AI能发现类似Zcash的漏洞吗？就是那个他们不得不保持低调处理的

Do you think the AIs of today would have found, for example, like the Zcash bug, The one that, you know, they had they had to sort of keep quiet.

这个我们很久前做过一期节目，和Sean Bo聊过这个故事

Back in this is like we did an episode long ago with Sean Bo talking about this story.

所以我们会附上链接，如果有人好奇的话。

So we'll link to it if anyone's curious.

这是一个在运行系统中被研究人员发现的漏洞，但如果被利用，实际上会造成重大的财务损失。

This is a this was a bug in a live system that was discovered by the researchers that were there, but had it been exploited, it would have actually meant like a major financial kind of hit.

就像，你知道的，我认为他们可以私下里精确地制造灾难。

Like, you know, I think they could disaster privately exactly.

他们可以私下印钞，而且永远不会被追踪到。

They could privately print money that would never have been traceable.

这有点像是最糟糕的漏洞场景，尤其是在ZK隐私系统中。

It was kind of like the most the worst case scenario of a bug, especially in a ZK private system.

所以，是的，我只是好奇——也许你不知道答案——但我想知道以现在的工具，能否发现它？

So, yeah, I'm just wondering like and and maybe you don't know the answer to this, but I would be curious if today's tools, way you have them, would it have found it?

对于这类漏洞，我持怀疑态度。

So for this kind of bug, I'm a bit skeptical.

AI非常擅长做人类擅长做的事情。

So so AI is very good at doing stuff that humans are good at doing.

对吧？

Right?

人类擅长的事情，AI也擅长。

Whatever humans are good at doing, AI is good at doing.

好的。

Okay.

但这大概就是我目前的直觉。

But that's sort of the the intuition that I have right now.

所以Zcash的那个漏洞是个理论层面的漏洞。

And so the the Zcash bug was a was a paper bug.

对吧？

Right?

一个协议理论上的漏洞。

A paper protocol bug.

嗯。

Mhmm.

所以你必须查阅论文，必须欺骗它，还必须意识到有些内容被发表了但其实本不该发表。

So you had to go on the paper, you had to fool it, and you had to realize that something was being published and it shouldn't have been published.

普通人类发现不了，但Ariel Gabizan发现了。

And normal humans wouldn't find it, but Ariel Gabizan found it.

嗯，正是如此。

Well, exactly.

对吧？

Right?

这不是一份普通的人类工作。

It's not it's not a normal Good human job.

是啊。

Yeah.

这并不容易。

It's it's not easy.

你必须对协议有深入理解，要明白发现这类漏洞绝非易事。

You you have to understand a lot of things about the protocol, you have to understand the it's it's not easy to find these kind of bugs.

嗯。

Mhmm.

或者至少如果Ariel能多写些内容让我们训练的话。

Or at least if if Ariel would write more contents that we could train on Oh.

那也许我们就能更容易发现这类漏洞了。

Then maybe we could find these kind of bugs more easily.

也许他该用自己的私人消息之类有趣的内容来训练自己。

Maybe he should train himself on his like private messages or something interesting.

训练或者说微调的另一个问题是你需要大量数据。

The the the other problem with training or like fine tuning is that you need a lot of data.

因为光靠少量数据不够，你需要调整这些权重参数，要真正改变它们就得输入海量数据。

Like it it's not enough, because you you need to like tweak these weights, and to really change them, you need to throw a lot of data at it.

David，虽然这期节目时间不多了，但我还是想花点时间和你聊聊ZK与AI交叉领域的概况。

So David, I wanna spend, I don't think we have too much time left in this episode, but I did wanna just spend a bit of time with you kind of talking about more generally, like, the ZK AI crossover space.

部分原因是...毕竟你正在审计这些项目。

Partly because I I mean, you're auditing projects.

你在ZK领域看到了很多项目。

You're seeing a lot of projects in ZK.

你提到很多ZK虚拟机之类的项目。

You're saying, like, a lot of the ZK VMs, stuff like that.

但我想说的是，2023年ZKML这个话题突然火了起来。

But there was I mean, 2023, there were sort of like ZKML as a topic popped up.

有几支团队从这个领域崭露头角。

There's a few teams that came out of it.

不过我认为我们还看到了ZK与AI融合的其他方式。

But I think we have seen other ways in which ZK and AI are overlapping.

那么，我们快速聊聊ZKML吧。

So, yeah, let's talk real quick about ZKML.

首先，你审计过任何ZKML项目吗？

Like, first, have you audited any ZKML projects?

这类项目你现在见得多吗？还是说比之前少了？

Like, are you seeing a lot of those, or would you say like there's less today than there was?

我不想曲解这个领域，因为最近可能有些我不了解的新进展。

So I don't want to misrepresent the field because I there might have been some development lately that I'm not aware of.

我记得几年前有一段时间人们讨论过ZKML。

I remember there was some year a few years back where ZKML was talked about.

我记得PSE的一些人讨论过这个，还有Modulus Labs，以及Ezekiel或ZK Conduitz。

I remember some people from PSE talking about it, there was Modulus Labs, there was Ezekiel or ZK Conduitz.

我不太清楚目前的情况，也不确定是否有新的竞争者进入这个领域。

I'm not too sure what's the status there, I'm not too sure what's if there's new competitors that enter the the playing field.

我记得当时PSE的Kathy做了一个关于这个的演讲，我觉得这是个很好的切入点。

What I remember was that there was this interesting I think Kathy gave a from PSE at the time, gave gave a talk on that, and I thought it was a good way of attracting things.

但她将ZKML描述为一种有多种应用场景的用例。

But she she sort of presented ZKML as this use case scenario where you have different use cases.

在没有ZKU的自然用例中，所有东西都是公开的。

In the natural use case without ZKU, everything's public.

你可能正在使用一个公开的模型。嗯。

You're using a potentially, you're using a public model Mhmm.