Liz Fong-Jones：PR数量翻倍，事故增加1.5倍

三个月内，从30个到70个拉取请求。

30 to 70 pull requests in just three months.

这是一个很大的跳跃，对开发者来说，要掌控这么多，确实不容易。

That's a big jump and a lot for developers to be in control of, let's say.

一开始，我们似乎看到与之相关的事故增长是次线性的，对吧？

Initially, it looked like we were seeing a sublinear increase in outages correlated with it, right?

比如，我们的变更失败率一开始似乎在下降。

Like our change failure rate appeared initially to be going down.

但后来长期来看，平均下来，我们提交的拉取请求数量翻倍，而事故数量增加了1.5倍。

And then it turned out over the long term it averaged out so that, you know, we were shipping twice as many pull requests and we are seeing, you know, 1.5 times as many incidents.

而且，你能让一个人处理的事故数量是有限的。

And there is a limit to how many incidents you can ask a human to work.

人的代码审查注意力是有限的。

Your human code review attention is limited.

我们如何让开发者不仅提高写代码的能力，还能提高审查代码的能力呢？

How do we enable our developers to scale up not just their ability to produce code, but also their ability to review code?

《AI原生开发者》是一档面向开发者和工程负责人的播客，聚焦AI和智能体编程的前沿领域。

The AI native dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.

每周，主持人Guy Pajani和我Simon Maple，都会与AI领域最激动人心的声音对话，探讨当今开发者面临的最大问题。

Join your hosts, Guy Pajani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.

这里是《AI原生开发者》。

This is the AI native dev.

昨天，你还在写代码。

Yesterday, you were writing code.

今天，你在给智能体下指令。

Today, you're prompting agents.

明天，你将运营一个软件工厂。

Tomorrow, you'll be running a software factory.

智能体编程的发展速度快得让人头晕目眩。

Agenci coding is moving fast enough to give anyone whiplash.

今年六月，AI开发者大会伦敦站的门票已经售罄。

This June, AI dev con London sold out.

数百名开发者、工程主管和副总裁齐聚一堂，聆听来自Anthropic、OpenAI、Netlify和GitHub的演讲者，他们拨开迷雾，直击真正有效的方法。

Hundreds of developers, engineering leads, VPs packed in to hear speakers from Anthropic, OpenAI, Netlify, and GitHub cut through the hype and get to what actually works.

今年十一月，我们将把它带回纽约，规模更大、更大胆，而且满是真正在构建智能体未来的人。

This November, we're bringing it back to New York, bigger, bolder, and full of people actually building the agentic future.

三天时间，全是实实在在的演讲、实实在在的工具、实实在在的答案，无论你是在指挥智能体、构建循环本身、带领团队，还是制定战略。

Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams, or setting the strategy.

作为播客听众，你可以使用促销代码pod 15享受门票15%的折扣。

As a podcast listener, you can get 15% off your ticket using promo code pod 15.

我们准备了很多内容，非常期待在纽约见到你。

We have so much planned, and we can't wait to have you join us in New York.

大家好，欢迎收听《AI原生开发者》的又一期节目。

Hello, and welcome to another episode of the AI native dev.

我是Simon Maple。

My name is Simon Maple.

我是本期节目的主持人。

I'm the host for this episode.

和我一起的是Liz Fong-Jones，Honeycomb的技术研究员。

And joining me is Liz Fong-Jones, Technical Fellow at Honeycomb.

热烈欢迎Liz。

A massive welcome to Liz.

嘿，你好吗？

Hey, how are you going?

嘿，我很好。

Hey, doing very well.

谢谢。

Thank you.

我想这是你第一次来AI原生开发者节目。

And I think this is your first time on the AI native dev.

Liz，你现在是Honeycomb的技术研究员，但你最初是从SRE那边开始的，然后转到了开发者关系，又做了一段时间的现场CTO，现在成了技术研究员。

Now, Liz, you're a Technical Fellow at Honeycomb, but you actually started more on the SRE side, and then moved into DevRel, little bit into field CTO, and now a Technical Fellow.

跟我们聊聊你这一路走来的经历吧。

Tell us a little bit about the journey that you've been on.

是的，我觉得在我的职业生涯中，我一直专注于做可靠性领域的通才，并且致力于提升我所在团队的能力。

Yeah, I think over the course of my career, I focused on being a generalist in the field of reliability, and I focused on up leveling the teams I work with.

所以我和所有初级开发人员一样，从个人贡献者做起，然后转向为谷歌的SRE团队和Suite团队提供建议，告诉他们实施可靠性实践的最佳方式。

So I started as an individual contributor as all junior devs do, and then I moved to kind of advising Google SRE teams and Suite teams on what the best way was to implement reliability practices.

后来我尝试了工程经理的角色，管理谷歌的Bigtable SRE团队，手下大约有12个人，干了几年。

And then I took a swing at the engineer manager pendulum and managed the big table SRE team at Google, about 12 people under me, for a couple of years.

我意识到，如果只在一个特定团队里，和一群特定的人一起工作，我确实能产生很大的影响。

And I realized that I could have all of the impact in the world working on one specific team, working with one specific set of people.

但我真的想对整个行业产生更广泛的影响，因为我看到很多人在他们的组织中实施SRE实践时遇到了困难。

But I really wanted to have a broader impact on the industry because I could see so many people that were struggling to implement SRE practices across their organizations.

我觉得确实有一个角色可以提升整个行业的水平，帮助人们获得更多的满足感，而不是让运维变成一件苦差事，你明白吗？

I thought, you know, there's a real role to up level the industry and to help people get more satisfaction and not just have ops be this drudge thing, you know?

所以，正是这些想法让我走上了DevRel和现场CTO的道路，也就是思考如何推动行业前进，以及什么是最好的杠杆。

So that's really what led led me down this path of DevRel and and field CTO work is kind of thinking about how do we move the industry forward and what's the best lever to do that.

而有时候，最好的杠杆并不是管理一个工程团队。

And sometimes the best lever to do that is not, you know, managing one engineering team.

有意思，其实我从开发者关系转到现场CTO的时候，也挺享受的。

It's funny, actually, because when I also moved from developer relations into the field CTO, really enjoyed it, actually.

在Sneak的时候。

Was at sneak.

我可能跟CSO和CTO聊得更多。

I probably talked more with CSOs and CTOs.

我觉得这有点像开发者关系的角色，但更像是升级到了领导层的领域。

And I kind of felt like it's a little bit like a DevRel kind of role, but it's more leveled up into the into the into the leadership space.

所以……

So...

对。

Exactly.

你在做领导力辅导。

You're doing leadership coaching.

就像，你在和其他高管平级合作，他们正在努力推动整个组织的变革。

Like, you're working peer to peer with other executives who are struggling to implement changes across their org.

其实我们的相似之处也就到此为止了，因为我后来又回到了开发者关系领域，毕竟我稍微有点想念那些开发者。

And that's actually where our similarity ends, because I moved back into DevRel because I kind of missed the developers a little bit more.

不过跟我们简单讲讲 Technical Fellow 这个职位吧。

But tell us a little bit about the Technical Fellow.

对，作为 Technical Fellow，我可以在各个工程团队中从事高级别的技术工作，同时继续做一些推广、教学和撰写关于我们所做工作的内容。

Yeah, so as a Technical Fellow, I get to do kind of distinguished engineering work across all of our various engineering teams, as well as still continuing to hold on to some of that work to evangelize and teach and to write about what we're doing.

说实话，主要就是写文章、上播客，代表 Honeycomb 工程部成为那个对外发声的公众形象，这样我们的工程师就能主要埋头搞技术，当然也要培养那些愿意公开演讲的人。

Really honestly, it'd be writing and appearing on podcasts and being able to be that public face of Honeycomb Engineering so that our engineers can primarily focus on being heads down except for, of course, nurture the ones who want to speak publicly.

作为这种大使，去传达我们在做什么、看到了什么，真的非常有帮助，尤其是 Honeycomb 的品牌定位是服务于具有前瞻性的工程团队。

Being that ambassador for what are we doing, what are we seeing is really, really helpful, especially given Honeycomb's brand is we are for forward thinking engineering teams.

所以帮助试点看看这具体是什么样子的，并且处于一个能够给他人提供建议的位置，因为我们已经见过这条路该怎么走了。

So helping pilot what does that look like and being in a position to give people advice where we've already seen what the path looks like.

Honeycomb 非常公开地谈论过，这种角色确实能极大地帮助技术工程师和技术人员推进他们的职业生涯，而不需要不得不滑入那条我称之为可怕的斜坡——也就是管理岗位的那条路。

And Honeycomb very openly talked about how this kind of a role, it really it really helps technical engineers and technical folks, you know, progress their careers without needing to drop into that slippery that slippery slope, I call it a horrible slope of this management thing.

我记得我记得我的父亲，他是个工程师，实际上是电气工程师，后来被提拔到了……

I remember I remember my my my dad, who was an engineer, actually electrical engineer, kinda was promoted into an...

很多很多年前，他升到了一个管理岗位，当时他觉得那是他的晋升。

A managerial role many, many, many years ago, and that felt like his promotion.

他之所以成功，是因为他转去做管理……

He succeeded because he went into a manager...

管理岗位。

Managerial role.

但当然，这其实并不是大多数工程师特别擅长或者真正想做的事情。

But, of course, that's not actually what most engineers are particularly good at or actually want to do.

所以我觉得这是一个非常有效的方式，可以让人们晋升，而不用……

So I think this is a really strong way of of of leveling folks up without without

我同意。

I agree.

而且，我也觉得，至少在职业生涯中尝试一次管理，会让工程师变得更好。

And, also, I think it does make engineers better engineers to take a turn at management at least once in their careers.

即使他们不打算长期全职做管理，我觉得这也能让你更清楚地看到工程工作是如何运作的，以及作为高级工程师如何产生影响。

Even if they don't decide to do it full time, longer term, I think it gives you a level of visibility into how engineering work, how to have impact as a senior engineer.

所以我不一定会说……

So I I I wouldn't say necessarily that...

我同意你的观点，当经理并不是晋升，但我也要说，工程师们不应该回避它，说，天哪。

You know, I I agree with your premise that being a manager is not the promotion, but, also, I would say that it shouldn't be something where engineers should, you know, steer away from it and say, oh god.

不。

No.

我永远不会那样做。

I'm never going to do that.

因为我觉得至少做一次，你能学到很多东西。

Because I think you have a lot to learn from doing it at least once.

是的。

Yes.

哦，那真是太对了。

Oh, that is so true.

那正是我的经历。

That was my experience.

我完全同意这一点，真的。

I totally agree with that, actually.

我觉得，呃……

Think I think the the...

而且我觉得现在比以往任何时候都更重要的是，人际互动的个人技能，与AI以及他人的沟通，能够把这些事情表达出来，这太重要了。

And I think today more than ever, you know, the the the personal skills of, you know, interaction, the communication with AI as well as other people is so important to be able to to to vocalize to vocalize those things.

莉兹，在这个播客里，我喜欢做的一件事就是找一些人的名言，然后把它甩回给他们，说，我们来聊聊这个。

Now one of the things I love doing, Liz, on on this podcast is finding quotes from people and and and and throwing it back at them and saying, let's chat about this.

看看这话现在还成不成立。

Let's see if this still holds up.

这里有一点我其实完全同意，所以它……

Now one of the things that I actually completely agree with here, actually, so it...

就是……

It's...

你，你……

You you...

你之前说过，AI会放大你现有的做法。

Your line was AI amplifies your existing practices.

它会让一个功能失调的组织更加失调，也会让一个高自主权的组织运转得更快。

It makes a dysfunctional org more dysfunctional and a high ownership org faster.

现在让我们……

Now let's...

我完全同意这一点。

I I totally agree with that.

但我们先来定义一下，什么是功能失调的组织？

But let's define what is a dysfunctional org?

你会看到哪些特征或迹象，然后说，没错。

What are the traits or the things that you would see and say, yes.

如果在那里引入AI，它实际上会让你更快地陷入那种糟糕的状态。

If you add AI there, it'll actually make you get to that bad state faster.

你会把哪些看作这些警示信号？

What would you see as those flags?

哎呀，其实我更想聊聊反过来的情况，也就是功能健全的组织有哪些特点，而不是去讨论那些 dysfunctional 组织的问题。因为 dysfunction 的种类实在太多了，很难一一列举清楚。

Oh man, I actually prefer to talk about the inverse of that and talk about the traits of the functional orgs rather than talk about the traits of the dysfunctional orgs because I think there's just so many different varieties of dysfunction out there that it's hard to enumerate them all.

不过，我还是觉得我们可以围绕你刚才提到的‘ownership’（主人翁意识）来展开讨论。

But I I think, you know, let's anchor on what you said about ownership, though.

我认为 ownership 是决定一个组织是高绩效还是低绩效的最核心因素。

Think ownership is really that primary force that dictates whether an org is high or low functioning.

具体来说，就是开发人员是否把自己的工作成果视为值得骄傲的事情，是否愿意对端到端的全生命周期负责，并且乐意为自己的产出署名。

It's the extent to which developers view their contributions as something that they take a sense of pride in, they want to own the life cycle of end to end, that they are responsible for, and that they they are happy to sign their name to.

我觉得其他问题都是由此衍生出来的。因为当我们谈论可靠性实践时，如果一个组织的可靠性做得不好，通常意味着他们缺乏强烈的责任感。不然的话，为什么你会允许有问题的代码上线，还让它一直带着bug在生产环境里运行呢？

And I think everything else kind of flows out of that because when we talk about reliability practices, an organization that has poor reliability practices likely does not have a strong sense of ownership because otherwise, why would you let broken code ship to production and leave it broken in prod?

所以我们可以聊聊各种让有问题的代码留在生产环境的情况，我认为归根结底……

So we can talk about all the ways that you can leave broken code in prod, I think it find...

核心问题在于，开发人员能不能跟着代码一起进入生产环境进行排查？

It fundamentally boils down to are the developers allowed to follow it into production?

他们能不能直接这么做，而不是每次都得去申请五套不同的权限？

Can they do so, right, like, without having to go and ask for five sets of permissions?

他们真的会跟进到底吗？

And are they actually following through?

所以，是的。

So Yeah.

我们听说……

You know, we hear...

我觉得这最近上了新闻。

I I think this was in the news recently.

Shopify的CEO托比·利特克说，十二个月前，我们是一家AI优先的公司。

Toby Litke, the CEO of Shopify, said, you know, twelve months ago, we are now an AI first org.

每个人都应该使用AI。

Everyone should be using AI.

然后就在一周前，他抱怨工程师在生产环境中解锁了“垃圾炸弹”，并且这些行为超出了他们的团队范围。

And then literally a week ago, he lamented slop bombs being unlocked by engineers at at production and out of their teams.

我就想，天哪，这其实是可以预见的后果，如果你告诉人们没有主人翁精神地行动，只用AI，加快速度，不在乎后果，现在他们突然要承担这些后果了。

And I'm like, man, that was kind of a foreseeable consequence if you were telling people to move without ownership and just, you know, use AI and move faster and, you know, who cares about the consequences, and now suddenly they're weeping the consequences.

所以我认为，不扔垃圾炸弹的关键在于，你要对自己发布的东西负责。

So I think, you know, ownership of not lobbing slop bombs because you are responsible and accountable for what you ship.

我觉得这才是最重要的。

Like, I think I think that's that's the important thing.

所以，

So so

然后

And

你是说，方向比速度更重要，如果你……

then You're saying that the bearing here is more important than the speed in terms of if if you're...

你实际上会更快到达目的地，而如果你在做坏事，

You will you will actually accelerate to the destination faster and if you're doing something bad.

而且我觉得重要的是……

And and I think the the important thing...

而且如果……

And if the...

如果你没有朝着任何特定方向前进，对吧，就像，给一个没有转向或导航的火箭加燃料，那它只会绕圈直到爆炸。

And if you are not steering in any particular direction, right, like, know, you add rocket, you know, rocket fuel to a rocket that has no steering or guidance, and those just go in circles until it blows up.

我觉得这就是没有方向却加速的典型例子。

I I I think that's the classic example of, you know, of of acceleration without direction.

对。

Yes.

它只是在绕圈。

You know, it's just going in circles.

对。

Yes.

我觉得这里的一个要点是，AI不会解决你现有的任何问题。

And and and I I guess, you know, a a takeaway there is AI is not gonna fix any existing problems that you have.

它只会放大这些问题。

It's gonna magnify them.

所以如果你有问题，AI不会帮你解决。

So if you have a problem, AI won't fix it.

你得先解决问题，然后再引入AI来加速你正在走的这条路。

You need to fix the problem and then introduce AI to to accelerate the the the path that you're on to.

如果你对应该发布什么内容没有共识，那你只会发布更多随意的工程改动。

If you're not in alignment about what you should be shipping, then you're just going to ship more random acts of engineering.

这对谁都不好，如果……

Like, that's not good for anyone if...

当然，我知道客户对他们允许UI变化的容忍度是有限的。

Certainly, I know that customers have a very limited patience for the amount that they allow their UI to change.

如果Riverside播客录制软件每次登录都变个样，每周都不同，我很快就会非常恼火。

If the Riverside podcast reporting software looked different every time logged in every single week, I'd become very frustrated very quickly.

你能引入的变化是有限度的，你必须仔细考虑如何利用这些变化。

There are limits to how much churn you can introduce, and you have to think consciously about how you're spending that churn.

所以……

So

我很喜欢你刚才关于所有权的例子，因为我觉得……

I really like the example that you gave about ownership there, because I think...

而且我觉得员工、开发者几乎都有那种意愿，会说，对，我确实想把这个事情一直做到上线。

And I think there's almost like that willingness for employees, for developers to say, actually, yeah, I want to see this through to production.

除了所有权之外，还有没有其他代码仓库的特征，会让你一看就觉得，哦，对，这些人是在用……

Beyond ownership, are there other other traits of a repo where you would look at something and say, oh, actually, yeah, these people are using...

这些开发者是在很好地利用自主权，或者有很强的模式。

These folks are developers using autonomy well or or, you know, strong patterns.

而且其实很容易把AI加进去。

And it's actually very easy to add AI in.

也许是DevOps实践。

Maybe it's DevOps practices.

还有没有其他东西，让你一看某个代码仓库就能说，这个已经准备好用AI了？

Are there other things that you can look at a repo and say, this is AI ready?

我觉得一个主要的点是，AI会复制你代码库里已有的模式。

So I think one primary thing is AI copies existing patterns that exist in your code base.

如果你做同一件事有五种不同的方式，AI就会变得非常混乱。

If you have five different ways of doing something, AI will become very confusing.

它会创造出六种或七种方式。

It will create six ways or seven ways

做事情的方式。

Ways of doing it.

所以我认为，要确保你的工程组织在做事方式上保持一致，就像你说的，要有共同的模式，然后确保把这些模式编码给AI，也给你的新员工。

So I think making sure that you have alignment in your engineering organization about how do you do things, what are the common patterns as you say, and then making sure to encode for AI and also for your new hires.

这些模式在哪里找？

Where do you find these patterns?

怎么用它们？

How do you use them?

我觉得，接着你刚才说的，

And I think, you know, take...

拥有强大的CI/CD模式也很重要，这样你就能用五分钟的努力去验证事情，而不是花六十分钟或三个小时。

Taking your nudge there, right, like, it is important as well to have strong CICD patterns so that you can actually validate things with five minutes of effort and not sixty minutes or three hours of effort.

确保你有良好的可观测性，确保你有良好的可测试性，而且不是空洞的测试，而是真正有意义的测试。

Making sure that you have good observability, making sure you have good testability, and not just, you know, empty calorie tests but actual real meaningful tests.

再说，机器会复制，如果它们复制的是坏习惯，那你最后就会制造出一堆没用的东西。

Again, machines copy and if they are copying bad practices then you're going to wind up creating fluff bombs.

如果你复制的是好习惯，那你测试和验证自己产出的东西就会更容易。

If you are copying good practices then you are going to have an easier time testing and validating what you're producing.

这要从测试开始，从高质量的注释开始。

That starts with testing, starts with high quality comments.

即使在我们代码库里注释质量相对较高的情况下，我们也发现AI不幸地把我们拉回到了平均水平。

Even in our code base where we have relatively high quality comments, we found that AI unfortunately is regressing us to the mean there.

我们有很多AI生成的、价值很低的注释。

We're having lots of very low low value comments that AI is creating.

比如……

Like...

那你如何防止AI让你的代码库变得更糟呢？

So how are you guarding against AI making your code base worse?

你实际上是怎么推广好的测试模式、好的可观测性模式，确保我们以更容易部署到生产环境的方式进行工程开发的？

How are you actually propagating good patterns around testing, good patterns around observability, kind of making sure that we are engineering things in a way that it's going to be easier to roll them out to production.

我觉得我们之前在节目外简单聊过这个话题。

And I think we talked about this briefly off air.

但我们在这里给出的建议，其实和AI出现之前我们给出的建议是一样的，就是什么样的功能是好的，什么样的测试是好的，可靠性要好，所有这些。

But the advice that we kind of like giving here is actually the same advice that we would give pre AI in terms of what functional looks like, what good looks like, you know, good testing, good reliability, all those things.

这其实就是

This is just

告诉人们要吃蔬菜。

good telling people to eat their vegetables.

确实是这样。

It is.

问题就在于，没人愿意吃蔬菜，直到他们得了维生素缺乏症。

And the and the trouble is no one wants to eat their vegetables until they wind up with a vitamin deficiency.

对，没错。

Like Yeah.

但问题是，AI让吃蔬菜这件事变得容易多了，只要你告诉AI去做，而不是让AI随便乱写一堆新功能塞进你的代码库。

Except for the thing is, AI has made it a lot easier to do the eat your vegetables part as long as you tell AI to do it and not just have AI, you know, slop coding a bunch of new features into your code base.

对。

Like Yeah.

你有了很多新的工程资源。

You have a bunch of new engineering bandwidth.

怎么用这些资源取决于你，而且因为机器永远不会觉得无聊，让它们给代码库加遥测、加丰富的事件、加属性，是件很好的事。

How you choose to spend it is up to you and because the machines never get bored, it's a great thing to do to have them add telemetry to your codebase, to add rich y events, to add attributes.

这些以前都是我们得告诉开发人员去做的，就像让他们吃蔬菜或者刷牙一样。

These are things that we used to have to tell developers, telling them to eat their vegetables or brush their teeth.

现在这些事机器人其实挺擅长的，而且只要你指示它们，它们也乐意去做。

Now it's something that the robots are actually pretty competent at doing and they're happy to do it as long as you instruct them to.

我觉得一方面，做这些事的必要性比以往任何时候都高，因为我们要处理的代码量在不断增加。

I think it's both that the necessity of doing it is higher than ever because we're having this increased volume of code that we're trying to win.

另一方面，做这些事的难度也大大降低了。

And also the ease of doing it is much better as well.

不。

No.

完全正确。

Absolutely.

那我们聊聊职能型组织吧。

So let's talk about a functional organization.

聊聊Honeycomb。

Let's talk about Honeycomb.

聊聊Honeycomb机器人……

Let's talk about the Honeycomb bot...

抱歉，是Autobot，它负责合并和创建拉取请求。

Autobot, sorry, that merges pull requests and, you know, creates pull requests.

我记得三个月内，每天合并的拉取请求从30个增加到了70个。

I believe that went from 30 pull requests merged per day to 70 pull requests merged per day in three months.

是这样吗……

Is that...

我记的数字对吗？

Did I get those numbers right?

数字是对的，但我觉得归因不太准确。

You got the numbers right, but I think the attribution is wrong.

没错，我们确实有个内部编程机器人叫 Autobot。

So, yes, we do have a internal coding bot called Autobot.

不过，Autobot 每天大概只生成五个拉取请求，所以它并不是拉取请求的主要来源。

However, the Autobot is producing maybe five of those pull requests per day, so it's not the majority creator of the pull requests.

它是这些拉取请求百分之百的审查者，而且它做得非常好，我们的开发人员不再需要去查找那些小改动里的琐碎 bug 了。

It is the 100% reviewer of those pull requests and it has gotten so good that our developers are no longer needing to look for trivial bugs in changes.

所以我觉得，当我们讨论把拉取请求的数量翻倍甚至更多时，关键在于人类代码审查的精力是有限的。

So I think, you know, when we're talking about increasing your number of pull requests by a factor of two or more, right, like, human code review attention is limited.

我们该怎么让开发人员不仅能提升写代码的能力，还能提升审查代码的能力呢？

How do we enable our developers to scale up not just their ability to produce code, but also their ability to review code?

我觉得这时候有个像 Autobot 这样的助手很有帮助，它能指出问题，这样开发人员就不用亲自去当人肉检查器或者人肉 Linter 了。

And I think it really helps there to have a helpful assistant like Autobot, you know, pointing out problems so that the developer doesn't have to be a human linter or a human bug checker.

这点很有意思，因为现在大家写代码的速度和交付代码的意愿并不是瓶颈所在。

And this is really interesting, because the speed at which people can write code and trying to deliver code is not the problem here.

问题在于，我们人类验证生成的代码、进来的拉取请求是否有效、是否正确、是否经过充分测试的速度。

It's the the the problem is the speed at which us as humans can validate that the code that is being generated, the pull requests that are coming in are are valid, are correct, are well tested and

完全正确。

exactly right.

我觉得，任何真正的生产系统，至少目前，都不应该是一个无人值守的工厂。

Like, I would argue any real production system should probably not be a dark factory, at least at this time.

我认为需要有人类监督、人类审查、人类问责，就像我之前说的。

I think there is a need for human oversight, human review, human accountability as I was talking about.

比如，我们如何确保把学到的经验反馈回去？

Like, you know, How are we making sure that we're feeding back the lessons that we learn?

我们如何确保维持我们的标准？

How are we making sure that we are maintaining our standards?

而且我认为这些标准必须来自人类。

And I think those standards have to come from humans.

我不认为机器人能够自己制定并执行这些标准，对吧？

I don't think the robots are capable of coming up with enforcing the standards on their Right?

就是说，我们得……

Like, we have to...

就像你车上有自适应巡航控制，不代表你可以完全放开方向盘。

Like, just because you have adaptive cruise control in your car doesn't mean that you can fully let go of the wheel.

对，没错。

Like Yeah.

你还是得保持注意力。

You still have to pay attention.

那你能跟我们讲讲Autobot是怎么……

So so so can you tell us a little bit how the Autobot...

我很喜欢Autobot这个名字。

And I love the I love the name Autobot.

听起来很像变形金刚，感觉我们这边力量很强。

It does sound very transformer like, so it feels like we've got a lot of power on our side.

跟我们说说它内部是怎么运作的吧。

Tell us a little bit about how that works under the covers.

为什么它这么可靠？

Why is it so reliable?

对，最初我们构想Autobot，是为了自动化创建小型的、分块式的拉取请求。

Yeah, so originally we conceived of Autobots as this mechanism to automate the creation of small bite sized pull requests.

我们花了不少时间，基本上就是把Claude code封装起来，专门处理单个线性任务。

We invested a fair amount of time in essentially doing Claude code in a box to work on a single linear ticket.

现在它仍然在做这件事，不过我们正在逐步淘汰这个功能，因为现在这已经变得很普遍了，你可以找到专门的服务来帮你完成。

It still does that, although we're in the process of sunsetting that because it turns out that now it's commoditized and now you can have bespoke services that do that in a box for you.

但我觉得，在我们四五月开始这个项目的时候，这效果非常显著，因为第一次让那些不太熟悉Claude code的开发者也能说：你知道吗？

But I think at the time when we started this effort in April or May, this was phenomenally effective in terms of for the first time having even developers who are not super fluent in Claude code be able to say, You know what?

这个任务看起来挺简单的。

This ticket looks relatively simple.

不入虎穴，焉得虎子。

You know, nothing ventured, nothing gained.

比如，把这个任务交给Autobot，看看它能生成什么样的拉取请求。

Like, you know, let's assign that ticket to Autobot and see what pull request it produces.

所以，Autobot 就是这么开始的。

So that's that's kind of how Autobot started.

是不是……

Was it...

这个努力是不是为了处理那些比较简单的任务，不需要开发人员一直盯着、指导机器人，对吧？就是那种“我希望它往这个方向设计”的感觉。

Was this effort to kind of clear off the lower hanging fruit that doesn't require a developer to supervise and tell the robot consistently and steer it, right, like, you know, this is the kind of design direction I want this to take.

理论是，如果你把每件事都拆解成一个清晰的线性任务，让机器人能独立完成，那么你的 Autobot 就能把它变成拉取请求，然后作为发起 Autobot 的人，你可以先自己审查一遍，再交给同事审查。

The theory was if you decomposed everything into a linear ticket well enough so that it would be able to, you know, complete the linear ticket on its own, that then your Autobot could take that and turn it into a pull request and then you could, as a human who requested the Autobot to run, you could go ahead and review that self review first and then after that hand that to your peer reviewer to review as well.

所以，就是这么开始的。

So that's how it started.

然后我们意识到，这些小任务只占一小部分，而且同样的功能也可以用来在封闭环境中复现今年四月或三月 Anthropic 发布的 Claude 代码审查功能。

And then we realized that there were a relatively small percentage of these smaller tickets, and also that same functionality could also be used to reproduce kind of in a box the functionality that I think in April or March this year, Claude...

Anthropic 发布了这个 Claude 代码审查功能。

Anthropic released this Claude code reviews functionality.

而且它非常昂贵。

And it's phenomenally expensive.

每次运行大概要20到30美元，而且得花上十几二十分钟。

Was, like, 20 or $30 per run, and it took, like, ten or twenty minutes to run.

我们看了之后觉得，其实我们也想要类似的功能，但我们更希望它能……

And we looked at it and we said, actually, we want something like this, but we would prefer that it be...

能在两三分钟内跑完，而且每次拉取请求的成本只要几美元，而不是几十美元。

That it run within less than two or three minutes, that it costs dollars per pull request and not tens of dollars.

所以我们把Autobot项目的重心转向了代码审查。因为我们意识到，不管有没有Autobot，我们的开发人员本来就能处理大量拉取请求，所以我们需要把重点放在如何减轻人工审查人员的工作负担上。

So we pivoted the Autobots project towards handling reviews because we realized that our human developers were more than capable of creating this mass volume of pull requests anyways, Autobots or not, and that we needed to focus that instead on how do we make the review load of our human reviewers lighter.

那……它是怎样运作的？

And and how does...

所以，显然 Claude 作为一个通用大模型，

So, obviously, you know, Claude as a as a...

作为通用大模型，这类任务通常成本都很高，运行时间也特别长。

As an LLM or a general LLM, these types of things will always be kinda, like, very expensive and and long to run.

这周新闻里都在讨论这个，毕竟录制的时候 Jev 正火得不得了。

In the news this week, so this will tell folks when we recorded it, obviously, Jev is is making a lot of waves right now.

它现在在很多地方都登上了头条。

It's hitting the it's hitting the headlines in most places.

对于像Jev这样的LLM，它特别擅长给出是或否的答案或者选择题，那这会对像代码审查这样的特定任务或领域产生什么影响呢？

With with an LLM, which is very specific to kind of being able to provide, you know, yes, no answers or multiple choice, does that how will that now impact places or, you know, specific tasks like code review?

所以我觉得我们现在主要考虑用Jev的地方，是把拉取请求分类成两类：一类是安全合并的，只需要自动审查；另一类是不安全合并的，需要人工审查。

So I think the primary place where we're thinking about deploying Jev right now is classifying pull requests into safe to merge with just the automatic review versus not safe to merge requires a human review.

以前我们靠人工自己判断，我们其实在做实验，试图让大约20%的拉取请求自动审查并合并，不需要人工审查。

Previously we had been relying on humans to self identify and we were doing this experiment actually to try to have about 20% of our pull requests automatically reviewed and landed without having a human review.

再说一次，把人力集中在关键的地方，因为如果你把拉取请求的数量翻倍，要么你花在人工审查上的时间也要翻倍，甚至更多。

Again, focusing that human attention because if you double the number of pull requests, either you double the amount of...

要么你就得减少工作量。

More than double the amount of time that you're spending as a human reviewing or you have to load shed.

你得搞清楚哪些审查不那么重要，哪些更重要。

You have to figure out what's less important to review and what's more important to review.

之前我们一直在做实验，让人类和审查者来标记。

Previously, we had been doing this experiment of having humans and the reviewers labeling.

我觉得这个拉取请求需要花很多精力去审查吗？

Do I think that this is a pull request that requires high effort to review or not?

现在我觉得可以让Jev来做这件事。

Now I think we can have Jev do that.

我们可以让Jev作为那个直觉检查，判断我能不能相信审查者关于这个是否安全合并的意见。

We can have Jev be that gut check of, can I trust the reviewer's opinion on whether or not this is safe to land?

如果可以，就直接自动合并，不需要人工干预。

And if so, go ahead and land it without human intervention.

所以仍然有语言模型层在做代码审查。

So there still is the language model layer doing the code review.

但我认为分类器是在做判断代码变更复杂程度的工作。

But I think the classifier is then doing that work of how complex is the code change?

审查者有没有提出什么重要的担忧？

Did the reviewer raise any significant concerns?

我该不该让……

Should I let...

我该不该合并呢？

Should I merge or not?

我觉得Jev这里在新闻上主要有三个关键点。

And I think there are three kind of, like, key things that are hitting the news here with with Jev.

一个是成本。

One is the cost.

一个是速度，但另一个是置信度分数。

One is the speed, but the other is the confidence score.

我猜速度和成本可能是锦上添花的东西，但置信度分数才是关键。

And I presume it's the the the the speed and the and the cost are probably the nice to haves here, but it's the confidence score, I guess.

完全正确。

Exactly.

如果我已经花了五分钟和十美元在token上来审查这个拉取请求，那么真正重要的是得到一个自动化的置信度分数，来判断我能否安全地合并它。

If I'm already spending five minutes and 10 and and $10 in tokens to review to review this pull request, the thing that's actually important here is getting a automated confidence score on can I land this safely?

所以，是的，我完全同意。

So, yeah, 100% agree.

就像……

Like...

但对我们来说，再次强调，我们并不是想减少在代码审查上花费的时间。

But for us, again, this is not that we're intending to spend less time doing code reviews than we did before.

而是希望把花在代码审查上的时间集中在设计模式上，而不是琐碎的bug上；集中在更复杂的PR上，而不是简单的PR上。

It's to focus the time that we're spending doing code reviews on the design patterns and not on trivial bugs and on the more complex PRs and not on the more simple PRs.

如果我只是在调整一个配置项，大多数时候这没什么大不了的，但有时候它确实有可能导致生产环境崩溃。

If I'm twiddling a single config knob, most of the time that's not a big deal except for some of the time it actually has the potential to bring down production.

所以我觉得这就是那种权衡取舍。

Like, so I think that's that's kind of that that trade off there.

当然，人类可以完全选择退出自动代码审查，说嘿，这个需要特别小心。

Like, eye human can of course opt out of automatic code review entirely and say, hey, this requires special caution.

说实话，大多数开发者都很认真负责，至少在高度自主的组织里是这样。

Honestly, most developers are very conscientious, at least in a high autonomy org.

我们不想在生产环境造成故障。

We don't want to cause outages in production.

从定义上来说，故障发生是因为出现了意外情况，是我们没预料到的，原本以为是个简单的改动，结果却不是。

By definition, outages happen when something happens that was unexpected, that we didn't foresee, that we thought it was going to be a simple change and was not.

有了这些额外的保障措施，不管是让语言模型来审查，还是让 Jev 来做一次 sanity check，看看这改动合不合理、能不能直接上线。

Having these additional guardrails, whether it be a language model doing a review or it being Jev doing a, you know, sanity check of does this make sense or not to to immediately land.

我觉得这些才是我们需要采取的措施，这样才能真正突破两倍的效率瓶颈。

Like, think those are the levers that we need to do to really go beyond beyond two x.

因为我们已经触到了天花板……

Because we've hit the seat...

说实话，我们已经达到了能构思和审查代码的上限。

Candidly, we have hit the ceiling of how much code we can conceive of and review.

这挺有意思的，因为每当触及上限时，我们往往会在意想不到的情况下突破它……

And this is interesting because whenever we hit the ceiling, we always tend to go beyond it when...

以一种我们完全没预料到的方式。

In in a way we kinda, like, least expect.

不过，仅仅三个月就有30到70个拉取请求，这个增幅很大，对开发者来说要掌控这么多内容，确实不少，对吧？

But, you know, 30 to 70 pull requests in just three months, that's a that's a big jump and a lot for developers to be in control of, let's say?

是的。

Yes.

而且我们一开始确实看到，与之相关的故障数量增长是低于预期的，对吧？

And we definitely initially, it looked like we were seeing a sublinear increase in outages correlated with it, right?

比如我们的变更失败率一开始似乎是在下降的。

Like our change failure rate appeared initially to be going down.

但后来长期来看，它平均下来其实差不多，你知道，我们提交的拉取请求数量翻了一倍，而故障数量增加了1.5倍。

Then it turned out over the long term it averaged out so that you know, we were shipping twice as many pull requests, and we are seeing, you know, 1.5 times as many incidents.

而且你能让一个人处理多少故障是有限度的。

And there is a limit to how many incidents you can ask a human to work.

所以，

Like, so

对。

Yes.

所以，我们得开始考虑怎么降低其他方面的风险……

Like, we we have to start looking at how do we derisk the other thing...

就是这些其他的事情。

These other things.

那你觉得，经过审查和真正被理解之间，界限在哪里？

So where's the line, would you say, between something that's been reviewed and something that's been understood?

因为几乎就像，我提交一个500行的拉取请求和一个10行的拉取请求，其中一个肯定会比另一个受到更多的仔细审查。

Because it's almost like, you know, when I deliver a 500 line pull request versus a, you know, 10 line pull request, one will definitely be scrutinized more than the other.

同样，如果我给你500个拉取请求而不是一个，那会有很多请求可能因为人类关注度降低而直接通过。

In the same way, if I was to provide you with 500 pull requests versus one, there are gonna be many more that just, you know, go through with maybe, you know, reduced visibility from the human.

我们怎么确保作为人类，我们不会变得很随意，只是把拉取请求推过去，而实际上并不理解发生了什么？

How do we how do we make sure that as humans, we don't just become very blase and just, you know, push pull requests along and actually understand what happens?

这是不是又是那个杰文斯的角度？

Is this, again, another maybe it is the Jev the Jev angle

代码审查是不是讨论这个问题的合适地方？

and the code review the right place to be having that discussion.

通常，我发现，在项目和工单层面讨论这个问题，比在拉取请求层面讨论更好。

Often, find that that is a better place to be talking about it at the project and at the ticket level than at the pull request level.

到了那个时候……

By the times...

尤其是当生成代码的成本很低，你可以轻松生成五遍的时候，对吧？我们为什么还那么执着于具体某段代码的实现方式，而不是更关注整体的设计模式呢？

Especially if generating the code is cheap and you could do it five times over, right, why are we that attached to the way the specific code works and not to the broader design pattern?

我们应该对齐的重点应该是方向对不对，而不是这段实现是否分毫不差。

Like, that's the thing that we should be aligning on is is the direction correct and not, you know, is this implementation exactly correct?

比如，语言模型可以帮你理解每一行代码的具体作用。

Like, language model can help you understand what the individual lines of code do.

不过，更宏观的模式是需要我们把它记在脑子里的。作为监督这些系统的人类，我们必须对它在宏观层面应该如何运作有一个清晰、一致的整体认知。

Like, but the broader pattern, that's something that you have to hold in your head and that we as humans that are supervising these systems need to have a consistent picture of how is this supposed to behave at the macro level.

那咱们再往前推进一步。

Let's then take it one step further.

假设系统出现了故障或者宕机，我们显然得去排查问题。这就需要采集遥测数据，并且具备可观测性。

Let's say, you know, there's an outage or some system fault, we obviously need to have a look at what we need telemetry data, and we need observability.

你在这里把遥测数据和可观测性做了非常明确的区分。

Now, you draw quite a hard distinction here between what telemetry is and what observability is.

确实，数据跟真正理解系统是两回事。

Really, data versus the ability to truly understand the system.

怎么...

How does...

数据跟理解，百分之百，每天都是这样。

Data versus understanding, 100% every day.

那什么是...

Well, what is...

如果代码不是人写的，那AI怎么改变我们对理解系统的定义呢？

How does AI change what understanding a system means if a human actually doesn't write that code?

我觉得人不需要写代码，也不需要自己去查询。

I don't think the human has to write the code, and, also, I don't think the human has to do the the querying by themselves.

但是，检查查询结果，这是人必须做的，这样才能真正内化并理解这次故障是怎么发生的，还有哪些类似的问题我们应该关注，以及怎么防止这种故障再次发生。

However, examining the outputs of the queries, that's something that a human has to do to really kind of internalize and understand how did this outage happen, what are the other things that we should be looking at that are similar, how do we prevent this outage from happening again.

我觉得有遥测数据，并且能让人类和AI都能查询和理解，这对建立反馈循环很有帮助。

Think having telemetry data, having it queryable and legible both to the humans and AI is helpful for then kind of establishing that feedback loop.

而且我觉得不只是故障发生的那一刻。

And I don't think it's just at the moment of outage.

我觉得在开发周期里也一样，对吧，你应该让语言模型通过查询开发环境来测试它们产生的遥测数据，对吧，确保如果我加了一个属性，那个属性是否存在？

I think also in your development cycle, right, you should be having your language models test the telemetry that they're producing by querying a dev environment, right, to make sure if if I added an attribute, is that attribute present?

我能不能检测到某些东西积压了，对吧，如果某些东西开始排队了？

Do I have the ability to detect if something backlogs, right, if if something starts queuing?

所以我觉得答案是肯定的，而且……

So I think I think the answer is yes and.

我觉得，是的，我们必须有遥测数据。

I think, yes, we have to have telemetry.

是的，我们必须有能力分析这些遥测数据。

Yes, we have to have the ability to analyze that telemetry.

而且这不只是人类的工作，也不只是机器人的工作。

Also it's not just the human's job and it's not just the robot's job.

把遥测数据转化为理解，是双方共同的责任。

It's a joint responsibility to turn the telemetry data into understanding.

对。

Yes.

而且我觉得，作为在SRE领域有丰富经验的人，你之前提到过你对人们应该如何在生产环境中使用Dart工厂的看法。

And and I guess as someone who has a lot of history in the world of SRE, you talked a little bit about how, you know, your opinion on how people should be using Dart factories in production.

在事故处理中，我们应该在多大程度上信任AI来处理数据？

How much should we be trusting AI to do with data in an incident?

如果能早上醒来，点一下按钮，然后说，好的，我看到AI代理已经识别出了问题，那该多好啊。

It would be lovely to think that we can wake up in the morning and and kinda, you know, click a button to say, yes, I can see the AI agents have identified the problem.

它们已经完成了所有的后台根因分析。

They've done all the the background root cause analysis.

它们已经提出了问题……

They've suggested the issue...

抱歉，它们已经提出了解决方案。

Sorry, they've suggested a solution.

它们已经审查过了。

They've reviewed it.

我只需要自己审核一下，然后把它推上去。

I just need to review it myself and push that through.

我们现在离这个目标有多近？

How how close are we to that today?

你觉得有没有可能……

And do you see a world to...

还是那句话，这取决于你过去十年有没有好好落实那些实践。

Again, depends on how well you are implementing the practices of the last ten years.

对。

Yes.

我们做功能开关已经有十到十五年了，具体看你怎么算Netflix早期的那些东西。

We have had feature flagging for ten or fifteen years, depends on how, you know, you count Netflix's early stuff.

功能开关我们用了十五年了。

We've had feature flags for 15 for fifteen years.

如果你功能开关用得好，那确实，一个能访问MCP的机器人就能识别出哪个功能开关能解决你的问题，并且能修复……

If you were implementing feature flags well, then, yes, a robot with access to MCP can identify which feature flag, will solve your problem and can fix...