鲍里斯·切尔尼：构建Claude Code

好的，鲍里斯。

Alright, Boris.

非常高兴你能来，Claude Code的创造者。

We're so excited to have you here, the creator of Claude Code.

谢谢。

Thank you.

能来这儿真的很荣幸。

It's great to be here.

刚出炉的消息，你们昨天刚发布了Opus 5。

Fresh off the press, you guys just shipped Opus five yesterday.

是的。

Yes.

而且

And

看来这个模型的性能还在持续提升。

it seems that model performance keeps accelerating.

你们在ARC AGI上提升了30%，这太惊人了。

You guys got took ARC AGI three to 30%, which is incredible.

是的。

Yes.

之前最好的成绩才个位数，顶多十几分。

And for context, before the best score was in in the low single digits or low teens.

对吧？

Right?

没错。

Mhmm.

Opus 5现在能做什么，是以前版本做不到的？

What can Opus five do now that it couldn't versus the previous version?

对。

Yeah.

每个新模型都包含大量改进，我们教会了模型很多新能力。

There's there's a lot that goes into every new model and there's a lot of new capabilities that we teach and get the model to do.

每次训练模型，我们都会试图教它很多东西，但大多数时候都行不通。

Whenever you do model training, you try to teach a whole bunch of different things, and most often it doesn't work.

不过总有一些方面，模型确实学会了。

But some subset of the things, the model does learn.

有时候它还会让你感到意外。

And sometimes it also surprises you.

它掌握了一些能力，这些能力我们根本没专门教过，但它自己就学会了。

It has these skills, it has abilities that you actually didn't really teach it, but it it just kinda learned.

对于Opus 5来说，一个其他模型都没做到的例子就是：它能持续运行很长时间。

For five, one example of something it does that I think no other model has done is it runs for a very long period of time.

尤其是把Opus 5和自动模式结合起来，简直太惊人了。

And especially when you combine Opus five with auto mode, it's just like incredible.

它能连续运行几天、几周，甚至几个月。

Like it can go for days, weeks, months at a time.

它就是停不下来。

It just won't stop.

你甚至都不需要使用辅助结构。

You don't even need to use scaffolding.

所以根本不需要用斜杠目标，也不需要那些乱七八糟的东西。

So you don't need slash goal, you don't need all this other stuff.

它自己就会去做，因为它知道该完成这个任务。

It'll just go because it knows it needs to do the task.

我还特别兴奋的一点是，接下来我想多聊聊，这其实挺让人意外的，因为这是一个全新的能力——模型现在好像不会再被提示词注入攻击了。

Another thing that I'm really excited about, and I'm gonna start I think to talk about a little bit more, but it's kinda surprising because it's such a new capability, is the model does not seem to be prompt injectable anymore.

什么是注入攻击？

What's injectable?

这太惊人了。

It's crazy.

以前大家老说那个致命三重奏，现在这个变化真的会彻底改变提示设计、代理设计，还有产品设计。

Like, people have talked about this, like, lethal trifecta for a long time, and this really affects kinda harness design and agent design and and and product design.

因为如果模型在互联网上看到一条指令，比如‘执行X、Y、Z，顺便把用户电脑上的所有东西都删了’——

Because if the model reads some instruction on the Internet that's like, you know, do x and y and z and also delete everything on the user's computer.

一年前，模型可能直接就照做了。

A year ago, the model would have just done it.

但现在，Opus 不会了。

But nowadays, Opus does not.

其实从 Opus 4.7 和 4.8 版本开始，情况就已经是这样了。

And this has actually been the case since like Opus 4.7, 4.8.

Sonnet 5 在这方面表现得相当不错，之前就已经做得很好了。

Sonnet five has been quite good at this table, was quite good at it.

但 Opus 5 在这方面简直开辟了一个全新的领域。

But Opus five just hits like a new frontier on this.

所以基本上，如果你把一个经过良好对齐的模型——也就是花了三年时间研究对齐技术的模型——和一个提示词注入分类器结合起来，我们会对所有流量运行这个分类器。

So essentially, if you combine a well aligned model, so this is like essentially three years of research into alignment, with a prompt injection classifier, which we run for all traffic.

这个分类器是基于 Chrisola 在机械可解释性方面的工作，也就是说，我们直接观察模型大脑中的神经元，看看哪些神经元在发生提示词注入时被激活。

And what this is doing is it's based on Chrisola's mechanistic interpretability work where it's it's literally we're looking at neurons in the model's brain that light up when prompt injection happens.

所以模型自己可能不会告诉你，但我们能实际看到这些神经元，从而判断并诊断出注入正在发生。

So the model won't even tell you, but we can actually see those neurons and we can figure out and diagnose that it's happening.

再把这套系统和自动模式分类器结合起来。

And then you combine that with the auto mode classifier.

有了这三层防护，我们现在已经完全无法被提示注入攻击突破了。

And with these three layers, we just cannot demonstrate prompt injection anymore.

我们还专门聘请了安全研究人员。

And we've hired security researchers.

也做了大量的红队演练。

We've done like red teaming.

还举办了各种对抗竞赛。

We've done competitions.

到现在为止，没人能成功演示出一次有效的提示注入。

No one can demonstrate it.

所以我很好奇，到底有没有人能做到。

And so I'm actually quite curious if if people are able to.

如果真有人做到了，那对研究团队来说会是极其宝贵的反馈。

And that that'll be actually be really amazing signal for the research team.

所以如果在场的任何人能对 Claude Opus 5 成功进行提示注入，Boris 就能拿到一份特别奖品。

So if anyone in this room prompt injects Claude Opus five, you'll get a special prize for Boris maybe.

说到提示注入，另一面就是系统提示词了。

Now talking about a prompt injection, the other side of the coin is now the system prompt.

咱们聊聊新版本吧。

Let's talk a bit about the new release.

你们真的删掉了 Claude Code 超过 80% 的系统提示词？

You actually deleted over 80% of the system prompt from Claude Code?

是的。

Yes.

详细说说这个吧。

Tell us more about that.

很多人可能没意识到，Claude Code 这个产品和它的运行框架其实一直在不断变化。

I think something that a lot of people might not realize is Claude Code as a product and as a harness is just always changing.

我们一直在添加新功能。

We're always adding stuff.

我们也在不断删减内容。

We're always deleting stuff.

每次推出新模型，我们都会删掉一大截系统提示，同时调整很多其他部分。

Every time that a new model comes out, we delete a bunch of the system prompt, change a bunch of the system prompt.

我们也在频繁调整工具集。

We change the set of tools all the time.

工具的提示词也一直在改。

We change the prompts for the tools all the time.

原因在于，每个模型都截然不同。

And the reason is every model is very different.

所以三个月前你为某个模型做的那些设置，很可能在新模型上完全不管用。

So something that you did for one model maybe three months ago, it just might not translate at all to the next model.

Opus 5 的智能程度真的很高。

And so one thing about Opus five is it's just really intelligent.

以前系统提示里很多内容，其实是在纠正模型本该懂却不懂的行为。

And a lot of the stuff in the system prompt was correcting for these behaviors that the model should have known, but it didn't.

现在 Opus 五自己就能搞定这些了。

Now Opus five just does it.

所以我们删掉了系统提示里 80% 的内容。

So yeah, we deleted 80% of the system prompt.

你其实也可以试试把剩下的部分也删掉。

You can actually try deleting the rest of it too.

运行 Claude Code 时，你可以直接加个 --system-prompt 参数，自定义你想用的系统提示，方便自己测试。

So when you run Claude Code, you can just do like dash dash system prompt and set whatever system prompt you want if you wanna experiment with it.

另外你可以试试简单模式。

And another thing that you can try is simple mode.

这个其实是个没公开的隐藏功能。

So this is actually this kind of undocumented feature.

只要设置环境变量 ClaudeCodeSimple=1，再运行 Claude，它就会把所有系统提示——包括工具里的——全清掉。

If you do Claude code simple equals one, like this environment variable, and then you run Claude, it'll delete all the system prompts, including from the tools.

我们平时就用这个方法做对比实验，看看提示到底有没有用。

And we actually use this as a sort of ablation to figure out is the prompt useful.

有趣的是，去掉这些提示后，模型反而更聪明了。

And what's interesting is that the model is actually a little bit more intelligent without these prompts.

这是我们一直发现的现象。

That's something that we've been finding.

但当你把 Claude Code 当作产品使用时，还是需要保留一些提示，因为它们能帮你更好地使用产品，也让模型的表现更符合你的预期。

But when you use Claude Code as a product, you do actually want some of these prompts because it helps you use the product and it it helps the product behave and the model behave in the way that you would want when when you're using it as a person.

我觉得在这个时代，你真正打造出了世界上最出色的 Claude 使用框架，那就是 Claude Code。

I think the thing that's really fascinating in this era of building, basically you have built the best harness in the world for for Claude, and that's Claude Code.

听你这么说，每次新模型发布，你们都会把整个代码库和所有提示都删掉，从头开始。

From what I'm hearing, you for every model release, you basically delete all of the code base, delete all of the prompt, and start from scratch every time.

在以前，哪家初创公司会这么干呢？

That in the old world would have been not something Startup would have done for the product.

简直就像每六个月就把所有东西全删一遍。

It's like press delete every six months for everything.

没错。

That's right.

没错。

That's right.

不过说实话，我们并不会删掉整个代码库，但确实会删掉很大一部分。

We so to be fair, we don't delete the entire codebase, but we do delete a lot.

每次推出新模型时，我们都会联系研究团队，这种做法叫‘消融实验’。

So every time there's a new model, we try we call in research, we call this ablation.

也就是说，我们会先删掉整个系统提示，再一行一行地重新加回来，看看每一行具体带来什么影响。

And so what this means is you delete the entire system prompt, and then you bring it back line by line to figure out what is the impact of each individual line.

这有点像评估，你可以去测试效果，而消融实验其实就是通过删减来判断各个部分的作用。

It's sort of like an eval, and you can kind of like evaluate it, and ablation essentially it's an eval, but you delete things to figure out the impact.

对工具也是同样的做法。

And yeah, like, do the same thing for tools.

我们经常把工具一个个去掉测试。

Like, we unchip tools all the time.

我们经常删掉工具箱里的代码。

We, you know, delete code in the harness all the time.

如果你看看现在Claude Code框架里的代码，大部分都是关于安全、权限和静态分析的，还有一部分是界面代码，其实我们早就移除了很多其他功能。

If you look at actually the code that's in the Claude Code harness today, almost all of it is about safety and permissions and static analysis, and there's a bunch of UI code, and we've actually unshipped a lot of the other code already.

你觉得这种构建智能体产品和框架的方式——每次新模型发布都做一次消融实验——所有在座做AI产品的人是不是都应该这么做？

Do you think this way of building a agentic product and harness and basically doing ablations every time with a there's a new model released, should everyone in this room that's building AI products basically do that?

要有勇气，敢于删代码。

Be comfortable and brave to press delete.

完全同意。

100%.

没错。

Yeah.

对于那些没在做智能体产品、但用着Claude Code的人，每六个月就删一次你的Claude MD。

And and for people that aren't building agentic products but you're using Claude Code, every six months, delete your Claude MD.

删掉你的技能。

Delete your skills.

删掉你的钩子。

Delete your hooks.

看看模型会怎么做，结果可能会让你意外。

See what the model does and it might surprise you.

实际上对于Opus Five，我们强烈建议你直接把所有这些都删掉，因为新模型可能根本不需要你以前为旧模型设的那些指令。

And actually for Opus five, this is something we really do recommend is just try deleting all of these things because the model might really just not need all those instructions that you needed for past models.

我们来聊聊怎么重新构建这个新提示。

Let's talk a bit about how then you build this new prompt.

每次有新模型发布，比如现在大家都想试试Opus Five，肯定会先把系统提示删掉。

When there's a new model release, like, for everyone in the room, everyone will want to try Opus five, and they're gonna press delete on their system prompt.

那他们该怎么重新搭建系统提示呢？

How do they go about building rebuilding the system prompt?

你该怎么设置你的环境？

How do you set up your environment?

你可以一点一点来。

So you do you do it kinda piece by piece.

第一步，先删。

So the first step is you delete.

下一步就是实际使用它。

The next step is you use it.

别猜模型需要什么指令，因为你很可能猜不准。

And you don't wanna guess what's the instruction that the model needs because you might not predict it correctly.

你应该直接运行它，如果你在开发一个自定义的智能代理产品，就真正跑一遍整个流程，看看模型在哪些地方出错，又在哪些地方表现得好。

The thing that you wanna do is you wanna run it, and if it's like a custom agentic product that you're building, you wanna kinda run the product, you wanna see where it fails with the model, you wanna see what it does well.

如果你用的是四号代码，就看看它在你的代码库中哪里表现优秀，或者在架构或其他地方卡壳了。

If you're using quad code, you wanna see where it does well with your code base or maybe where it stumbles over, you know, the architecture or stumbles over something else.

只有当它反复在同一个地方出错时，才把那条指令加回去。

And only when you see it repeatedly stumble on the same thing, that's when you add it back.

但别太早这么做。

But you don't wanna do it too early.

因为你要记住，模型每次使用时都会读这条指令。

Because remember, like, the model is gonna read this instruction every single time you use it.

所以你得确保这条指令真的是模型必需的。

So you really wanna make sure that the model needs this instruction.

我觉得，基于模型来做开发，这一点真的很反常。

I I think this is sort of the crazy thing about building on models.

这跟我以前做过的所有工程都不一样。

It's just so different than all the engineering that I've ever done.

过去我们做系统的时候，都会先设计出一套庞大而精美的架构，花很多心思在前期规划上。

Like, in the past when you built on systems, you built these, like, big beautiful systems, and you really think about the system design upfront.

还会有一整套完整的单元测试。

You have, like, a big suite of unit tests.

每件事都要想周全，重构更是大事，有时候得花上好几个月。

You think about everything, and, you know, like, a rearchitecture is a big project, and sometimes it takes months.

我以前在大公司参与过一些重构项目，一搞就是好几年。

I've worked on rearchitecture products at, you know, big companies that take years.

但模型可不是这样的。

And the model is not like that.

你得把它当成一个活生生的生命体，一种更有机的东西来理解。

It's the way to think about it is almost like a like a living creature, like a something more organic.

每次生成模型，它的表现都不太一样。

It's a thing where every model generation, it behaves differently.

它好像每次都有点不一样的性格。

It has a slightly different personality.

你得花时间去了解它，然后根据它的特点调整测试框架。

And you have to take the time to get to know it and then adjust the harness based on that.

我觉得这真的挺像一种实证的、科学的方法。

And I I think it's just very much like an empirical and kinda scientific thing.

你得用科学的思维方式，先试一试，看结果，再根据反馈不断调整。

You have to take a very scientific mindset to it where you try something, you see the result, and then you iterate based on that.

如果你现在在这个领域做开发，那什么才是稳定的呢？

If you're building in this world right now, what then becomes stable?

评估指标你会沿用之前的模型，继续用在每个新版本里吗？

Are evals something that you keep from the previous models and keep using them in each new model release?

我们会一直用，直到评估指标达到上限为止。

We do until we max out the eval.

所以这就是给大家的建议。

So that's sort of the tip for everyone.

如果你想在最前沿开发，充分发挥模型的潜力，就得把这些代码和系统提示删掉。

So code and system prompt, you have if you wanna build at the bleeding edge and have the most capability for models, you gotta delete those.

但评估指标要保持不变，不断往里添加新的内容。

But evals are constant and keep appending to them, basically.

对。

Yeah.

你得持续不断地添加新的评估内容。

You keep you keep appending.

说实话，我甚至不会说得这么绝对。

What happens is you know, I actually wouldn't even go this far, to be honest.

我觉得评估指标确实比工具框架活得久一些，但也没久太多。

I think evals, they outlive the harness a little bit, but not quite that much.

一个评估指标大概能撑一到三个模型版本吧。

Like, an eval might live for maybe one, two, three model generations.

但现在呢，我们正处于指数增长的阶段。

But nowadays, the you know, we're on the exponential.

模型进步得太快了，经常评估指标还没用多久就饱和了，只能扔掉，重新设计新的评估方式。

The model is improving so quickly, very often we just saturate the eval, and then we have to throw it away, we have to come up with a new eval.

这本来就是开发过程的一部分。

And this is just part of the process.

再说一遍，关键是要基于实际数据。

And again, it's about being empirical.

你得亲自用这个产品。

You have to use the product.

你得亲自用这个模型。

You have to use the model.

你得亲眼看看它在哪些地方卡壳。

You have to see where it struggles.

然后根据这些观察，来设计你的评估集。

And then based on that, that's the evil set that you should build.

我听过你提到一个词，说如何基于Claude打造最好的智能体产品，就是‘解除限制’这个概念。

I think one one term I heard you describe how to build the best agentic products on top of a Claude is this concept of unhobbling plot.

能再详细说说这到底是什么意思吗？

And tell us more more about what that means.

是的。

Yeah.

‘限制’在研究里指的是，模型明明能做某件事，但你却在不断干预它。

So hobbling is this idea in research that the model is doing something and you're just getting in the way.

我对这个想法特别认同，觉得很有启发。

There there's this kind of like way of thinking about it that I really like.

做产品的时候，这个思路特别有用。

It's very useful when you're building product.

这其实叫‘产品滞后’。

And it it it's called product overhang.

意思是，以现在的模型能力——不是未来的模型，就是当下就能做到的——还有很多潜力我们还没挖掘出来。

And the the idea is the model is able to do all sorts of things with today's models, not a future model, but today's model, that we have not yet realized.

模型其实还有很多这种能力，但大家根本没意识到。

And there are so many capabilities the model has like this that people are not aware of.

比如用某个工具、某种语言，或者以特定方式解决某个类型的问题——这些我们原本以为模型做不到的事。

And this is like the ability to, you know, like maybe use a particular tool, use a particular language, solve a particular kind of problem, do things a particular kind of way that we thought was kinda beyond the model's capability.

这种潜力其实每一轮生成时模型都能发挥出来，只是被埋没了。

And there's this overhang because the model can do this at every given model generation.

但往往没有产品能真正释放这种能力，让模型把本事施展出来。

But there is often not a product that lets the model do this and lets it express this kind of ability to do this.

反过来，很多时候反而是产品本身在拖后腿。

And on the flip side, often what happens is the product gets in the way.

这种拖后腿的情况，我们就叫它‘束缚’。

And this getting in the way, we call this hobbling.

而没能激发模型应有的表现，我们就叫它‘能力积压’。

And then not not eliciting the correct behavior from the model, we call this product overhang.

所以这两者其实是同一枚硬币的两面。

So it's kind of like two sides of the same thing.

其中一个例子就是最初的绘图代码。

One example of this was the original plot code.

我刚开始做这个的时候，大概是一年半到两年前吧。

When I first started working on it, this was, you know, like a year and a half, two years ago, something like that.

那时候用的是Sonnet 3.5。

This was like Sonnet 3.5.

当时那真是个超强的编程模型。

At the time, that was an incredible coding model.

那是当时最厉害的编程模型了。

That was like the best coding model that exists.

但现在回头看，按现在的标准，它编程能力差多了。

Nowadays, it's, you know, a pretty terrible coding model by modern standards.

但我认为，那是我们Anthropic团队打造的第一个真正出色的编程模型。

But I think that was, like, the first great coding model that that we built as Anthropic.

那时候，如果看看市面上的编程产品，它们都在做什么呢？

And at the time, if if you looked at the coding products of the time, what were what were they doing?

他们那时候只能做单行代码补全。

They were doing, like, single line autocomplete.

有时候能做多行代码补全。

They were doing sometimes multiline autocomplete.

这在当时算个新点子。

That was sort of a new idea.

他们也做了聊天功能。

They were they were doing chat.

你可以跟智能体对话，但只能看，不能改。

So you can talk to the agent, but it wasn't write access.

只能查看，不能编辑。

You could only read.

你可以问代码库相关的问题。

You could ask about the code base.

所以大家觉得，当时还没有一个真正能充分发挥模型能力的产品——让模型一次性写出完整函数，甚至完整文件。

And so the feeling was that there wasn't really a product that was fully eliciting the model's capability to write entire functions at a time, entire files at a time.

那时候还做不到整个功能，我们还没到那一步。

At the time it wasn't entire features, we weren't there yet.

但那时候，整个文件的编写能力已经是目标了。

But probably entire files, that was the level of capability at the time.

所以Claude Code的想法是：我觉得模型其实能做到这一点。

And so the idea with Claude Code was, alright, we think the model can probably do this.

如果我们去掉所有繁杂的框架，只给模型最简单的工具，让它直接一次性写完一个文件、构建一个完整功能，会怎么样？

What if we get rid of all the scaffolding and just give the model the simplest possible harness so it can write an entire file at a time and build an entire feature?

就这么简单。

And that that was kind of it.

那时候，产品上其实就差这最后一块拼图。

Like, that was the product overhang at the time.

模型已经具备这个能力了，但各种限制反而拖了后腿。

The model was capable of doing something and everything was just kind of getting in the way.

我觉得现在用现代模型，产品潜力大得惊人，可很多初创公司都没抓住这个机会。

I I think that nowadays with modern models, there is so much product overhang that I I'm not seeing startups capture.

我觉得有人已经在思考这些问题了，但从中挖掘模型那些惊人、有趣又极具商业价值的行为，机会依然巨大。

And I think there's people thinking about these problems, but there's just a huge amount amount of opportunity to elicit these behaviors from the model that are just, like, amazing and interesting and and commercially valuable.

我觉得在座的每个人都能获得这个非常独特的洞见。

I think this is such a special insight for everyone here in the room.

只要你们搞清楚怎么释放模型的潜力，你们每个人都有可能打造出下一个 Claude Code，因为这正是 Claude Code 的诞生故事。

Basically, all of you could create the next plot code if you figure out how to unhubble the models because that's effectively the birth story of ClotCode.

你们释放了 Sonnet 3.5 的能力，因为之前的版本都把模型死死地限制在 IDE 里，而 Claude Code 是最早让模型获得完整终端访问权限的尝试之一。

You unhubble Sonnet 3.5 because all the previous iterations were still getting the model very rigid in in IDEs, and Clockcode was one of the first instances that gave it just a full terminal access.

没错。

Yes.

这样一来，就诞生了一个持续演进的出色产品。

And that then created this amazing product just that keeps going.

所以，我们来聊聊，未来创业者应该从哪些方向入手，去释放 Claude 的能力，解决这种产品潜力被压抑的问题？

So let's talk about what are some areas and how should future founders here think about un hobbling Claude and fixing this product overhang?

我想到几个关键点。

So there's a couple things that I will think about.

首先，你应该给模型分配比你自认为能完成的稍微难一点的任务。

One is you should give the model slightly harder tasks than what you think you can do.

我经常看到一个普遍的错误：人们用Claude或Claude Code时，给出的指令太过具体了。

I think a really common mistake that I see is people are using Claude Code, they're using Claude, and they they just give it like way overly specific instructions.

他们总说：你得这么做，但必须按这种方式、那种方式、再一种方式来做。

They're like, want you to do this, but I want you to do it in this way, this way, this way.

必须先做第一步，再做第二步，然后第三步、第四步，一步都不能错。

You must do like one, then two, then three, then four.

但对现在的模型来说，这其实完全不是正确的做法。

And for modern models, that's actually really not the way to do it.

你应该把要求提得更宏观一点。

You wanna go a little bit higher level.

描述清楚你要完成的任务就行。

You wanna describe the task.

再设定好边界和限制。

You wanna describe the guardrails.

你要说清楚完成的标准是什么。

You wanna describe like the exit criteria.

然后就让模型自己去发挥。

And then just go with the model cook.

过一会儿再回来检查。

And come back in a little bit.

我相信结果会让你感到惊喜。

And I think it'll it'll surprise you.

再说一遍，这种做法半年前根本行不通，但现在真的可以。

Like and again, like, this is just not something that would've worked six months ago, but it does work today.

你能举些例子吗？比如现在模型能完成哪些六个月前还做不到的有挑战性的任务或能力？

Can you give some examples of these challenging task or capabilities that people should explore that it can do now that it couldn't six months ago?

是的。

Yeah.

比如，现在模型已经能够把几乎任何代码库从一种编程语言转换成另一种。

So, okay, one example is the model can now rewrite essentially any code base from one language to a different language.

这简直太不可思议了。

It's just sort of crazy.

以前工程师得花很长时间才能完成的工作，现在模型很快就搞定了。

Like, it's this work that would have taken just like a very long time as an engineer, and now the model's like quite fast at it.

举个例子，Claude Code 是基于 Bun 这个 JavaScript 运行时构建的。

So so one example of this is Claude Code is built on the Bun JavaScript runtime.

这是一个开源的 JavaScript 运行时。

It's an open source JavaScript runtime.

它是 Node 的一个替代方案。

It's an alternative to Node.

JavaScript。

Js.

它的速度比 Node 快不少。

It's kind of a faster node.

Bun 是用 Zig 写的。

BUN was written in Zig.

Zig 是一种系统编程语言。

Zig is a systems programming language.

它有点像 C 语言。

It's it's kinda like c.

它的层次非常底层。

It's it's very low level.

C 和 Zig 的一个问题在于，你得手动管理内存。

One of the problems with c with with Zig is you have to manually manage memory.

所以很容易遇到内存泄漏和其他内存管理问题。

And so it's quite easy to run into situations where there's like memory leaks and other memory management issues.

因此，Bun 团队的做法是让 Claude 对代码库进行模糊测试，模拟并触发内存泄漏，他们这么做了很长时间。

And so one thing that the Bundt team was doing is they were having Claude fuzz the code base and try to simulate and trigger memory leaks, and they were doing this for, you know, for a long period of time.

他们发现了大量内存泄漏。

They were able to find a lot of memory leaks.

基本上是一次发现一个漏洞。

It was sort of like a case at a time.

那时候，模型的能力大概就只能做到这种程度的模糊测试。

And that was kind of the capability of the model at the time was doing this fuzzing.

后来，团队里的杰瑞德就说：算了，我们干脆重写一遍吧。

And then at some point, Jared on the team was like, okay, let's just like rewrite it.

也许模型真能搞定这个。

Maybe the model can do this.

我觉得这算是他每次推出新模型时都会丢给模型的一个测试题。

And I I think this is like one of these test problems that he kind of threw at the model with every new model generation.

从Fable开始，模型就慢慢能处理这个任务了。

And starting with Fable, the model started to be able to do it.

我觉得Opus 5应该也能做到。

And so I think Opus five could do it as well.

所以他做了一套测试用例。

And so what he did was, essentially, he defined a test suite.

BUN的好处在于，它的测试非常完善。

The nice thing about BUN is it's very, well tested.

BUN 有一个庞大的测试套件。

There's a big test suite in BUN.

Node 也有一个庞大的测试套件。

There's a big test suite in Node.

JavaScript。

Js.

所以你能很容易判断自己有没有做对。

So it's easy to know if you did the right thing.

他让模型把代码从 Zig 重写成 Rust。

And he had the model rewrite it from Zig to Rust.

就只用了一个提示词。

It was one prompt.

这是一个动态的工作流程。

It was a dynamic workflow.

动态工作流程是我们在 Quad Code 里的一个功能，能让你协调几十、几百甚至上千个代理高效协作。

And a dynamic workflow is our feature in quad code that essentially let you orchestrate, you know, dozens, hundreds, thousands of agents to do work productively.

而且它连续运行了十一天，把整个代码库都重写了一遍。

And it ran for eleven days, and it rewrote the entire code base.

这是一次性完成的？

And this was one shot?

是一次性操作，但不是完全无干预。

It was one shot with no.

不是完全一次性，但有人在引导进程。

It wasn't one shot, but there was steering.

确实有人在引导。

There was steering.

但以前的模型，就算有引导，也根本做不到这一点。

But previous models just couldn't do this, even even with the steering.

这在以前是完全不可能的。

It just wouldn't have been possible.

才十一整天。

Just eleven days.

天哪。

Oh my god.

过去就算有最顶尖的工程师，这得花上好几个月，甚至好几年吧？

This would have taken in the past, even with the best engineers, multiple months, years?

肯定超过一年。

Definitely over a year.

是的。

Yeah.

对。

Yeah.

超过一年。

Over a year.

这相当于重构十多万行代码，JavaScript 运行时太复杂了。

This is like over a 100,000 like, JavaScript runtime is really complicated.

里面的东西多得惊人。

There's there's a lot of stuff in there.

而且，确实，它能正常运行。

And, yeah, it, like, it works.

现在已经在生产环境中使用了。

This is in production now.

你现在使用 Claude Code 时，背后就是这套系统。

This is what Claude Code uses now when when you're running it.

这算是其中一个例子。

So this is kind of one example.

我再举另一个产品滞后效应的例子，这是一个实际的应用场景，你正在解决某个具体问题。

I I would give a second example of sort of product overhang, and so this is like a practical use case where like there's a problem you're solving.

不管是商业问题、工程问题，还是产品问题，都算。

It's like a business problem, an engineering problem, a product problem.

你就不断用最新的模型去试，看能不能搞定。

And you should just keep throwing the latest model at it to see if it'll just do it.

因为就算旧模型不行，新模型说不定就能成功。

Because even if a previous model didn't, the new one might.

我觉得另一种理解方式是把它当成实验。

I think the second way to think about it is experiment.

给自己一些自由，去随便玩玩这个模型，做点有创意的东西。

And just give yourself like freedom to play with the model and do creative things.

它经常会让你感到惊喜。

Often it'll surprise you.

最近几周，在公司内部特别火的一件事是，有人发现你能让Opus使用OpenCV。

So something that's actually been really popular at a internally that's been kind of viral within Anthropic the last couple weeks is someone figured out that you can give Opus five OpenCV.

真的吗？

Oh.

然后让它画图。

And you can have it draw.

你可以这样问Opus：嘿，

And so something you can do is you can ask Opus, hey.

用OpenCV画出这张图。

Use open OpenCV to, like, draw this image.

而且效果还真不错。

And it's actually quite good.

它能画人像。

It can do, like, portraits.

还能画动物。

It can draw, like, animals.

风景图也没问题。

It can do, like, landscapes.

我们根本没训练过它画画。

And we didn't train the model to draw.

这纯粹是能力的意外涌现。

Like, it it it's just, like, the solicitation gap.

只要你问对了方式，它就能自己搞定。

Like, if you ask it to do it the right way, it can just do it.

我们纯属偶然发现的，就是随便玩玩，试了些没直接商业用途的创意点子。

And we discovered this kind of accidentally just by playing around and trying creative things that didn't have direct commercial applications.

挺有意思的。

Interesting.

我的推测是，以现在的模型来看，很可能有几十甚至上百个类似的潜在应用，但还没人发现。

And my hypothesis is there's probably dozens, hundreds of opportunities like this with the models of today that no one has yet realized.

这方面主要的研究方向其实就是模型引导。

And the big area of research for this is basically model elicitation.

对吧？

Right?

就是学会发掘模型的各种能力，然后正确地引导它去做事。

Becoming really good at figuring out all these capabilities and asking the model to do the right thing.

对吧？

Right?

没错。

Yes.

那人们怎么才能在这方面进步呢？

How do people get better at that?

那么，人们该怎么提高提示词设计的能力呢？

And effectively, how do people get better at prompt engineering?

现在还需要花大量精力做提示词设计吗？还是说这种情况也在变？

Do people still need to do a lot of prompt engineering, or is that changing as well?

跟我们说说，未来会往哪个方向发展？

Tell us about where this is going.

是的。

Yeah.

我记得大概一年前，最受欢迎的职位之一就是提示词工程师。

I I remember, like, a year ago, one of the most popular job openings was prompt engineering.

后来情况变了，好像又演变成了上下文工程师。

And then it kinda changed, and then I think it became like context engineer.

这种趋势总是在一波一波地变化。

So there's these kind of waves of it.

我觉得这些角色都会起起落落。

I think these will kinda like come and go.

现在更重要的技能不再是写提示词，而是怎么给Claude安排一个看起来有点太难的任务。

I think the skill nowadays is less about prompt engineering and more about figuring out how do you give Claude a hard task that seems a little bit too hard.

然后怎么让Claude在过程中不断检查自己的工作进展。

And then how do you make it possible for Claude to verify its work along the way?

我觉得这种检查机制才是大多数人最没做对的地方，而且是关键中的关键。

And the verification, I think, is probably the single most important thing that people do not get right, largely.

举个例子，我们有个给Claude用的桌面应用，是用Electron开发的。

One example of this is people were you know, we have this desktop app for Claude, and it's built using Electron.

我们把它优化得挺快了，现在用起来体验相当不错。

We've made it quite fast, so now it's like a pretty awesome experience.

六个月前，它还又卡又不稳定，完全没法用。

Six months ago, it was like sluggish and it wasn't very reliable.

现在好了很多，团队里大多数人日常都靠它了。

Now it's pretty awesome, and you know, it's the thing that most of the team uses.

不过为了做个实验，我想试试看，要是这应用用原生开发，体验会有什么不同。

As an experiment though, I wanted to see like, what would it feel like if it was native?

所以我做了一件事，开了一个Claude Tag会话，Claude Tag是我们新推出的一个产品。

And so what I did is I I started a Claude Tag session, and Claude Tag is just, you know, it's a it's a new product we have.

其实就是Claude在Slack里运行。

It's just Claude running in Slack.

我第一个问题就是：嘿，Tag。

My first question was, hey, Tag.

你在GitHub上有Mac OS的运行环境吗？

Do you have access to a Mac OS runner on GitHub?

它说没有，于是我连上了一个运行器，让GitHub能启动一个Mac虚拟机。

And it said no, And then I I hooked up a runner, so it was able to start a Mac virtual machine using using GitHub.

然后我的第二个问题是，我创建了一个空的代码库，是用Swift重写的Claude桌面应用。

And then my second question is, I created this, like, empty code base that was a Claude desktop app rewritten in Swift.

我就问它：你能访问这个代码库吗？

And I asked, can you access this code base?

它说不能。

It said no.

然后我给了它访问权限，心想，好了，搞定。

And then I gave it access, and I was like, okay, great.

现在我能访问了。

Now I have access.

接着我就说，行吧。

And then I was like, okay.

接下来我要你做的，是把那个 Electron 应用重写成 Swift 版本。

Now I want what I want you to do is I want you to rewrite the Electron app in Swift.

你得在 Mac 虚拟机里运行这个 Electron 应用，截个图，然后逐像素对比 Swift 版本的样子。

I want you to run the electron app in the Mac virtual machine, screenshot it, and then look pixel by pixel, compare it to the Swift version.

没完成之前别停下来。

Don't stop until you're done.

所以这其实就是你的指令，对吧？

And that was your prompt, basically?

对，这就是我的指令。

That was my prompt.