AI 能写代码，它能运行系统吗？| Grepr CEO Jad Naous

好的

Okay

……别人的PR，然后大家就像PR满天飞一样。

... people's PRs, and everybody's like just PRs flying around.

被PR淹没了。

Drowning in PRs.

对。

Yes.

但审查流程中变化很大的一个点是，我们开始更加关注测试——

But the thing that has changed a lot in the review cycle is that we started to focus much more on the tests-

当然

Sure

……比如做黄金预期，以及它们是否以不同方式变化等等。

... and doing things like golden expectations, and are they shifting in different ways and so on.

但Opus 5，以及我最近学到的一个技巧是，不用子代理，而是让Claude启动命令行子代理，因为它们能保持持久的会话状态。

But Opus 5, and the trick that I've recently learned about is that instead of using sub-agents, you basically tell Claude to spin up command line sub-agents because those maintain their persistent session over time.

所以你可以让它分别启动这些代理。

And so you can have it spin up these agents separately.

你还可以从其他会话中运行它们，等等。

You can actually run them from other sessions and so on.

这样从上下文的角度来看，效率会高很多。

And that allows you to be much more efficient from a context perspective.

你运行多少个代理？

How many agents do you run?

多少个？

How many?

多少个？

How many?

多少个什么？

How many what?

代理。

Agents.

智能体？

Agents?

通常，我会同时开三到四个会话窗口，因为我发现如果超过这个数量，我就会开始丢失上下文……

Typically, I work on three or four session windows at the same time because I found that more than that, I just start losing context-

……在不同的任务之间，因为它们完成得更快，然后就会等待。

... between the different things because they start finishing faster, and then they wait.

那它们是通过PR来交互的吗？

And do they interact via the PR?

是这样运作的吗？

Is that how it works?

子智能体……

The sub-agents-

……是一起协作的吗？

... like together?

不，它们直接通过命令行交互。

No, they interact directly through the command line.

哦，哇，好的。

Oh, wow, okay.

所以我让主要负责实现的那个主代理找一个技术负责人来审查，他们基本上就是向它提问。

So I asked the main thing that does the implementation to have a tech lead reviewer, and they basically ask it questions.

“这样合理吗？

"Does this make sense?

这样不合理吗？

Does this not make sense?

帮我审查一下我的PR，”等等，然后他们就在这些PR上一起反复修改。

Review my PR," et cetera, and so they iterate on these PRs together.

然后他们有一个PR文档，基本上就是他们一起做的，里面包含设计文档和所有内容。

And then they have a PR doc that basically they work on with a design doc and everything in it.

我做的第一件事就是让那个代理去做，让技术负责人审查PR文档。

The first thing that I do is I ask that agent to do it, ask the tech leads to review the PR doc.

我会过一遍PRD。

I go through the PRD.

我会仔细看设计文档，确保它合理，逐项详细检查，确保所有算法和各个部分都能正常工作，分工合理，PR之间的划分和关系也没问题，然后我就告诉它直接开始做。

I go through the design doc, make sure it makes sense, go through each piece in detail, make sure that all the algorithms work and all the pieces work, the division of labor works, the division between PRs and their relationships work, and then I tell it to just go start doing it.

看看能不能并行处理，然后启动其他代理去实际执行。

Figure out if there's parallelization that you can do, and then spin up other agents to actually go do them.

但最终，你还是得把所有东西整合到一个堆栈里，作为人类逐个过一遍，否则就会变得混乱、分支太多，根本没法处理。

But ultimately, you have to kind of put it all back in one stack and be able to go through it as a human one by one, because otherwise, it just gets messy and bushy, and you can't really tackle it.

那你现在做的项目和六年前相比，时间上有什么差别吗？

So can you give a sense of a project you do now versus, I don't know, six years ago, like time differences?

我最近交付的项目大概是——

So the latest project that I shipped was something like-

顺便说一下，要明确的是，这是生产代码。

And by the way, to be clear, this is production code.

这不是——

This isn't-

这是生产代码。

This is production code

……演示或者原型之类的。

... demo or mock-ups, whatever.

所以你要说的重点就是，把生产环境代码上线后能带来哪些收益。

So the statement you're about to make is the gains that you get for shipping production.

那我拿最近参与的一个PR，或者说最近一组PR来举例吧，我们管它叫数据湖里的自动升级。

So I'll give you the example of the last PR that I worked on, or the last set of PRs, was what we call auto promotion in a data lake.

在数据湖里，我们会存储所有数据，通常这些数据是按列存储的，比如Parquet文件和Iceberg格式等等。

So in a data lake, you store all the data, and typically, the data is column-oriented, Parquet files and Iceberg and so on.

当你从日志里获取数据时，这些数据通常是半结构化的，列也是任意的等等。

And when you get data from logs, they're semi-structured, they're arbitrary columns, and so on.

为了能让数据布局最优化，并且能够高效、快速地查询，理想的情况是日志事件里的每个字段都变成一列，这样你就可以单独对它们进行过滤——

In order to have the best layout of the data and to be able to query it efficiently and fast, the ideal situation is that every field in the log event becomes a column that you can filter on-

好的

Sure

……而且这些列可以独立于其他行进行操作。

... independently of the rest of the rows.

所以我们称之为自动提升或数据拆分。

And so we call that auto promotion or shredding.

明白。

Sure.

做这种事情非常复杂，对吧？

And doing something like that is incredibly complicated, right?

你得去搞清楚元数据系统是怎么运作的，元数据系统的提交是怎么工作的，怎么协调不同的写入者，等等。

You have to go and figure out how does the metadata system work, how do the commits of the metadata system work, how do you coordinate across different writers, and so on.

这大概花了十天时间。

And that took maybe about 10 days.

以前的话，这可能需要大约六个月——

In previous days, that would have taken maybe about six months-

哇。

Wow

——而且是一个人来做。

... for a single person.

哇。

Wow.

这是你一个人做的吗？

And was this just you?

对，就我一个人。

That was just me.

简直不可思议。

It is insane.

差不多是十倍、二十倍的提升。

Sort of factor of 10, factor of 20 improvement.

而且是用于新产品的生产代码。

For production code for new products.

是的。

Yes.

我一直有个疑问，我明白这些模型是在所有代码上训练的，可以说是所有写过的代码，对吧？

One question I always have is, I can very much understand these models are trained on all the code ever written, let's call it, right?

所以我理解它们为什么能生成代码，这部分是语法层面的。

So I can understand why they're able to generate code, which is the syntax piece.

但你现在做的是核心基础设施。

But then, you're doing core infrastructure.

比如说你在写分布式算法。

Let's say you're doing distributed algorithms.

它的运行特性和自我管理方式，跟普通代码差别非常大。

The properties, how it handles itself when it's running is very, very different.

很多时候你光看语法根本猜不到会发生什么。

Often you just can't even look at the syntax to know what's going to happen.

比如你在写一个分布式系统，它的性能、可用性，甚至正确性，都取决于工作负载是什么样的。

Let's say you're writing some distributed system, and then the way that it will perform, the availability, the correctness even, is going to be dependent on what the workload looks like.

访问模式是什么？

What are the access patterns?

流量是什么样的？

What is the traffic?

你说的是它的性能方面吗？

Are you talking about the performance aspects of it, though?

还是说语义方面——

Or are you talking about the semantic-

嗯，我说的是可用性。

Well, I'm talking availability.

你可能需要加锁，这样在一段时间内它不可用。

You may have to lock things so that it's not available for a period of time.

对。

Sure.

我说的是正确性。

I'm talking about correctness.

也许你决定采用最终一致性，所以如果你读数据，它不会是一致的，因为没有锁。

Maybe you've decided to do eventual consistency, and so if you do a read, it's not going to be consistent because there isn't a lock.

然后是性能，也许你缓存，也许不缓存。

And then performance, maybe you cache, maybe you don't cache.

所以这些都是设计空间里的权衡，对吧？

So these are all kind of trade-offs in the design space, right?

而且除非你知道访问模式，否则你不会知道它在这些方面会表现如何。

And unless you know the access patterns, you won't know how it's going to perform in these things.

所以我就在想，好吧，我有一个AI。

And so I just think to myself, okay, I've got an AI.

我能理解它如何为一个通用的分布式数据存储写代码，比如说。

I can understand how it will write code for a generic distributed data store, for example.

我不明白的是，它怎么知道需要做哪些权衡，因为它甚至不知道访问模式是什么。

What I don't understand is how it will know to make the trade-offs needed because it doesn't even know what the access patterns are.

而且这些东西也不是说能……

And it's not like these things can...

连学习如何构建一个理解所有操作的系统所需的数据都不知道在哪里，对吧？

Where is even the data to learn how to build a system that understands all of the operations, right?

今天，我们依赖人类来管理集群和优化数据库等等。

Today, we rely on human beings to manage clusters and to optimize databases, et cetera.

那你能不能谈谈你是怎么想的？我明白写代码这部分，但你们还有这些深层的运维系统。

And so can you maybe talk about how you think of, I get the writing the code, but you have these deep operational systems.

你怎么把AI应用到那个问题上？

How do you apply AI to that problem?

而且，第一，我们有没有取得任何进展，还是这仍然是个未解决的问题？

And A, have we made any progress at all, or is this still an outstanding problem?

我觉得这个问题可能关乎复杂性，以及你能在多大程度上把你描述的那些运维方面的事情——

The question, I think, is maybe about complexity, and to what extent can you take the things that you've described that are operational-

——那些在生产环境中发生的事情——

... and that happen in production-

——转化成AI能够推理的东西？

... and convert them into something that an AI can reason about?

而且——

And-

对。

Exactly.

AI怎么能理解这个系统呢？

How can an AI reason about the system?

首先，你觉得它在写代码的时候，能多大程度预测接下来会发生什么？

Well, A, first off, to what extent do you see that it writing the code that it can actually predict what's going to happen?

它在这方面做得怎么样？

How good of a job does it do at that?

第二个问题是，我们怎么让它去理解这个系统？

And the second one is, okay, now how do we make it reason about this?

那我们先来问第一个问题。

So let's ask the first one.

如果我用你的方法，就是Jad那种写代码的方式，去搭建一个大型分布式系统，

If I use your method, the Jad method of writing code, and I build some big distributed system-

从性能和正确性的角度来看，它实际做对的频率有多高？

How often does it actually get it from a performance correctness standpoint?

它在实际操作中，合理的频率又有多高？

How often is it actually operationally pretty sensible?

顺便说一下，我在做这个分布式高斯溅射运行器，我觉得它做得挺烂的。

And by the way, I work on this distributed Gaussian splat runner, and I would say it's shit.

简直糟透了。

It's horrible.

因为你有几百万个高斯溅射需要分页进出和流式传输，这完全取决于运行系统。

Because you've got millions of Gaussian splats that you've got to page in and out and stream, and it just totally depends on the running system.

所以，没有一个模型能做好这件事。

And so, none of the models do a good job.

所以在我们这个领域，我发现我们基本上把性能基准定义得很清楚，

So in our world, what I've found is that we've defined basically really well what performance benchmarks look like-

以及系统应该如何运行，还有在我们的流处理器中构建时面临的约束，才能达到

... and how the system should run, and what the constraints of actually building within our stream processor look like to get-

正确的性能。

... the right performance.

明白了。

I see.

但偶尔它也会……

But every once in a while, it'll do...

我最近的项目主要是关于查询优化和查询性能的。

My latest project has been all about query optimization and query performance.

然后我就想，嘿——

And so I was like, hey-

这就像那种典型的难题。

It's like the classic example of something that's hard

我就想，我们基本上是在生成对应SQL的代码，然后想在管道里运行它。

I was like, we are basically generating code that corresponds to SQL, and we want to run it in a pipeline.

然后我们就说，好吧，你生成的代码很烂。

And we're like, okay, well, you're generating shit code.

修好它。

Fix it.

然后它就开始运行了，但它没意识到的是，它一开始其实并没有对代码做任何优化。

And it started basically going through, and what it didn't realize was that it wasn't actually doing initially any optimization on the code.

就像，哦，看，那没问题。

It's like, oh, look, that's fine.

有什么问题？

What's the problem?

然后我说，"你看，这里有这些连接词。"

And I said, "Well, look, there's these conjuncts over here.

你在做 AND 操作，这个特定的项重复了，应该被去掉。"

You're doing and, and this particular term is repeated, and it should've been taken away."

然后它就说，"啊，好的，让我想想办法，" 之类的。

And then it's like, "Ah, okay, let me figure this out," and so on.

然后它基本上就沿着一条思路去尝试找出所有——它建立了一堆假设，关于所有它可以去优化以改善情况的地方。

And then it basically went on a thread to try to figure out what are all-- It built a bunch of hypotheses around what are all the things that it can go and optimize to make this better.

但最终，因为它能够基于性能基准来验证，我就说，"你看，这里有一组查询数据和用户提问等。"

But ultimately, because it was able to ground that in performance benchmarks, and I was like, "Look, here's a dataset of the queries and what the users are asking and so on."

"去处理吧。"

"Go work on it."

明白了。

I see.

所以它基本上能做得不错。

And so it can basically do a relatively good job that way.

但要是问它该给这个系统分配多少内存——

But when it comes to saying something around how much memory should I give this system-

……

...

它就完全没概念了。

it has no idea.

这必须得靠实际生产环境中的实验来得出。

And that has to come from actual experiments that run in production.

所以我觉得，这些系统在运维方面目前做得好的地方在于，它知道，嘿，这个问题我可能答不上来。

And so what I think, actually, these systems seem to be doing well at this point in terms of operational stuff is that it knows that, hey, I probably won't be able to answer this question.

让我想办法做个实验——

Let me run an experiment somehow-

……然后得出那个信息。

... and come up with that information.

但当然，在运维方面，生产系统的表现往往出乎你的意料。

But of course, when it comes to operations, production systems don't behave the way that you expect.

而最终，这就是事故发生的根本原因。

And ultimately, that's why incidents happen.

那你有没有发现，或者你自己有没有在做一些AI SRE相关的事情，就是有一个正在运行的系统，有日志，然后你把日志喂给AI，让它帮你调整系统？

So have you found, or are you doing any of the AI SRE stuff where you actually have a running system that has logs and you feed it to an AI and that kind of helps you tweak it?

有的。

Yes.

那效果怎么样？

And how well does that work?

因为从我的经验来看，其他人做这个的效果并不太好。

Because anecdotally, my experience with other people doing this has not been that great.

我听到的反馈都不太乐观。

The reports I've been getting anecdotally have not been great.

所以有几件事要说。

So a couple of things.

第一，我们既在做运维方面，也在做安全方面。

The first one is we're doing that for both operational pieces and for security.

但我们处理的方式和别人从AI SRE角度做的很不一样。

But the way that we're tackling it is not the way that other people are doing it from an AI SRE perspective.

很多这类系统基本上都是从滞后指标开始的，比如“嘿，这个指标飙升了”或者“这个警报触发了”。

A lot of those systems basically start from a lagging indicator, like, "Hey, this metric spiked," or, "This alert happened."

然后回溯一个非常复杂的系统，试图找出为什么会发生这种情况。

Go back through a very complex system and try to determine why that happened.

对，确实。

Yeah, sure.

这就是为什么那些方法不管用，因为回溯到根本原因其实非常困难。

That's why those don't work, because backtracking into a root cause is actually very problematic.

所以我们反其道而行之，先看看你的环境里有没有什么变化，有什么不同的地方。

And so instead, what we're doing is doing it the other way around, which is, can we actually start by figuring out if there's something different in your environment that has changed?

我们能不能做大规模聚类，大规模地找出大海捞针，找出那个不同的东西？

Can we do a large-scale clustering, a large scale, kind of figure out a needle in a haystack, something that is different?

然后——

And then-

哦，所以你假设之前一切正常，现在不正常了，所以肯定发生了什么变化？

Oh, so you're assuming that it was working fine before and now it's not, so something happened?

但这也可能只是正常流量模式的变化，对吧？

But it could be just a change in normal traffic patterns, right?

那也没关系。

That's fine, too.

我们关注的是，当事情出错时，系统通常会以某种方式告诉你出了问题。

And so the thing that we're looking at is when things go wrong, usually the system will tell you something went wrong somehow.

如果一切正常，扩展也正常，那就不会有什么动静。

If it's working fine and scaling as normal, nothing will happen.

但有时候你会看到一条消息，写着“错误，连接用完了”。

But sometimes you'll see some message which says, "Error, connections ran out."

或者“内存超限”，或者“资源缓冲区已清空”，或者——

Or, "Memory exceeded," or, "Resource buffer is emptied," or-

……某种迹象，某种非常早期的预警，说明出事了。

... some kind of indication, some kind of really early warning, something happened.

而我们要捕捉的就是那条消息，然后从那里入手，看看这是否是一个真正需要调查和处理的问题。

And that's the one message that we want to capture and then go from there and see, is that an actual problem that we need to investigate and deal with?

那它表现怎么样？

So how does it do?

到目前为止，表现相当不错。

So far, it's doing pretty well.

我们已经在自己的系统里运行它了。

We are running it internally in our systems.

我们也在几个客户的环境里跑过。

We ran it in a couple of our customers' environments.

而且有几次我们忽略了它——

And there was a couple of cases where we ignored it-

结果那些变成了生产事故。

... and those turned into production incidents.

真的吗？

Really?

哦，哇。

Oh, wow.

从直觉上讲，我能理解这些模型怎么理解语法，因为就像我说的，它们是在所有代码上训练的，结果就是覆盖了很多有用的场景。

Just from an intuitive perspective, I can understand how these models understand syntax because, again, like I said, they're trained on all of the code ever written, and it just turns out that covers a lot of useful cases.

我甚至很难想象软件操作的实际边界，对吧？

It's hard for me to actually even think of a bound on the actual operations of software, right?

这话听起来有点夸张，但其实有点道理：每次给程序加一位，它的操作状态空间就翻倍。

This is a ridiculous statement, but it's actually kind of true, is every time you add a bit to a program, the state space of what it can do operationally increases by a factor of two.

所以很快就能达到比宇宙中原子还多的状态。

So you very quickly get to- ... more states than atoms in the universe.

可以说，这就是为什么产品化的运维方法一直不太管用的原因，对吧？

And arguably, this is why product approaches to operations have never really worked, right?

混沌工程那套其实一直没真正奏效，我们长期以来都是靠人肉来管理分布式集群的，这其实是个公开的秘密。

The chaos engineering stuff never really worked, and we've always kind of relied... Actually, the dirty secret for the longest time of running a distributed cluster was just human beings managing it.

所以我在想，要训练AI来做这件事，你得有足够的数据和配置才行。

And so I'm just wondering, and then in order to even train an AI to do it, you have to have enough data and enough configuration.

所以我挺好奇的，你觉得我们现在能用AI来运行分布式集群或者管理这些服务吗，还是说这还得再等等？

So I'm just kind of curious, do you think we're at a spot where you could use AI to run a distributed cluster or run one of these services, or do you think that's still on the come?

我觉得已经有足够的数据和配置来启动这个循环了。

I think that there's enough to be able to have it start that loop.

最终，我同意你的看法。

Ultimately, I think that I agree with you.

这里面的空间是无限的。

There's infinite space.

有很多不同的东西。

There's a lot of different things.

但说到底，人类也是按照操作手册来运行的。

But ultimately, humans also operate on runbooks.

当然，生产环境里有很多事情最终还是需要人来处理，比如，“嘿，这太复杂了。”

And sure, there's a lot of things that happen in production that you ultimately need some human to go and like, "Hey, this is too complex.

有些隐藏的状态我知道，是因为前几天我在饮水机旁跟另一个同事聊过。”

There's some state that's hidden that I know of because I talked to this other guy at the water fountain the other day."

“他告诉我他部署的那个东西。”

"And he told me about this thing he deployed."

系统状态里可能有很多东西是看不到的。

There's a lot of stuff that potentially isn't visible in the state of the system.

但我要说的是，我们的目标应该是获取足够的遥测数据和理解，让一切变得透明，比如，“嘿，这就是系统的样子。”

But I'll argue that the goal of what we need to be able to do is to get to have enough telemetry and enough understanding to make everything really visible from a, "Hey, here's what the system looks like.

这是它如何运作的。

Here's how it's working.

这就是各种情况。”

Here's whatever."

然后我觉得，如果系统之前运行正常，那么运营中真正出现的大多数问题，通常都是因为某种变更引起的。

And then I think ultimately, most of the issues that end up really showing up in operations, assuming you have the system previously running okay, is due to some kind of change.

如果你能专注于弄清楚变化是什么，然后去排查——

And if you're able to just focus on the goal of figuring out what the change was and kind of going through-

抱歉，我能问个傻问题吗？我从来不觉得这些生成模型擅长预测东西。

Sorry, can I just ask a stupid question on this, which is, I never think of these generative models as good at predicting stuff.

听起来你讲了很多预测建模，比如预测变化，或者建立基线然后检测异常。

And it sounds like you're talking a lot about predictive modeling, like you predict a change, or you baseline and you detect an anomaly.

我是不是理解错了？

Am I thinking about this wrong?

不，你没有。

No, you're not.

好的。

Okay.

因为目标不是让代理去预测，而是让它得到一个信号，知道某些东西确定性地发生了变化。

Because the goal is not actually having the agent try to predict, but have it get a signal that something changed deterministically.

哦，是发生了什么事。

Oh, that something happened.

明白了。

I see.

等等，等等。

Wait, wait.

抱歉。

Sorry.

所以是有一个单独的模型来做预测，然后它会提醒AI？

So there is a separate model that does the predictive stuff, and then that will alert the AI?

是的。

Yes.

好的。

Okay.

对。

Yes.

有道理。

That makes sense.

所以我们做的事情基本上就是，你每天给我们一PB的日志，

So what we do is basically, you give us your petabyte of logs per day-

然后我们扫描所有这些日志，每次发现新的模式出现，也就是以前没见过的新东西，我们就会把它提出来交给AI，然后说，

... and we scan all of them, and every time we see a new kind of pattern emerge in your logs, a new thing that we haven't seen before, we lift it up to the AI and say-

明白了。

I see.

所以那是一个预测模型，它做的就是预测模型擅长的事情，也就是找出异常情况之类的。

So that's a predictive model that actually does what predictive models are good at, which is determining out-of-norm things or whatever, whatever.

没错。

Exactly.

然后这些信息会传给AI，AI就利用这些信息来决定该怎么做。

And then that goes to the AI, and the AI kind of uses that to figure out what to do.

对。

Exactly.

这样AI基本上就能进行推理并驱动后续操作了。

So the AI basically can then reason and drive that

明白了。

I see

……处理下去。

... that down.

所以它可以这样说：‘哦，这条消息，让我看看代码，找出它来自哪里，理解代码在做什么。’

So it can say, "Oh, this message, let me actually look at the code and figure out where it comes from and understand what the code does.

让我看看这种情况以前是否发生过。

Let me see if it's actually happened before.

让我看看。’

Let me see."

所以它基本上会运行各种假设和启发式方法，试图理解：这合理吗？

So it basically runs all these hypotheses and heuristics to try to understand, does it make sense?

它在某种程度上是否有意义？

Is it meaningful in some kind of way?

然后它就能决定该怎么处理。

And then it can determine what to do with it.

那你觉得我们离……咱们先回到查询优化，不说日志那些，还有多远？

So how far away do you think we are for... Let's just go back to query optimization, not logs or something.

那你觉得，在查询优化方面，AI基本上能替代人类做那些工作，还有多远？

So how far away do you think we are from having an AI basically do the job that a human used to do when it comes to query optimization?

我觉得非常近了。

I think very close.

真的吗？

Really?

是的。

Yes.

哇。

Wow.

我觉得——

I think-

比如运行一个分布式集群。

Like running a distributed cluster.

我就想，比如你有个MIT博士学位的James Cowling，他负责管理分布式集群，如果集群挂了，公司就没法运转，如果集群正常，那就一直跑着。

I just think of like, okay, you've got James Cowling, PhD from MIT, who runs a distributed cluster, and if the cluster's down, the company doesn't work, and then if it's up, it's running.

我也不太清楚。

I don't know.

你觉得现在到了可以说AI能胜任这个工作的地步了吗？

Are we at the point where you would say that an AI can do that?

不，还没有。

No, not yet.

哦，这样啊。

Oh, okay.

我觉得，很简单，当我们刚开始用AI做编程的时候，它只是自动补全。

I think, very simply, when we started doing AI for coding, it was auto-complete.

而且当时没人真正知道用AI到底能构建多复杂的软件。

And nobody really knew to what extent you can actually build complex software using AI.

我们当时看的时候，就觉得，"哎呀，AI永远不可能构建出——

And when we were looking at it, we were like, "Ah, AI is never going to be able to build-

……X或Y或Z之类的。

... X or Y or Z or whatever."

然后它基本上突破了预期，因为我们能够在系统中建模越来越多的复杂性。

And then it basically broke through the expectations because we were able to model more and more complexity in the systems.

我觉得自己在这方面的想法很简单，对我来说，这些模型就是输入分布，输出分布。

I just feel like I'm a very simple thinker on this stuff, and to me, these models are distribution in, distribution out.

事实证明，你确实可以在所有已有的代码上进行训练，而且那个流形其实挺容易处理的。

And it turns out you can actually train on all the code ever created, and it's actually like that manifold is pretty manageable.

我只是不确定运维是否也受到同样的限制。

And I just don't know if operations is constrained in the same way.

对我来说，运维更像是宇宙，重尾分布、基于异常、幂律分布，这些都是这些模型难以学习的东西。

That feels to me more like the universe, heavy-tailed and exception-based and power laws and all the tough stuff for these models to learn.

但我不确定。

But I don't know.

你是这个领域的专家，所以也许情况并非如此。

You're the one that's in the field, and so maybe that's not the case.

也许它们是可以管理的，但是——

Maybe they're manageable, but-

所以——

So-

但我要告诉你，系统的历史，你也知道，就是总想构建一个聪明的算法来做高效的事情。

But I will tell you, the history of systems, as you know, is you always want to build a smart algorithm that does the efficient thing.

是的

Yes

聪明的虚拟内存管理、聪明的拥塞控制、聪明的路由。

The smart virtual memory management, the smart congestion control, the smart routing.

但我们每次实际做的是什么呢？

And what do we do instead every single time?

我们构建一个更复杂的东西？

We build a more complex thing?

不，我们每次都过度配置。

No, we over-provision every time.

不，真的，每次都是这样。

No, literally, every single time.

还记得交换内存吗？就是交换到磁盘上的那个。

Remember swap memory, where you swap to disk?

兄弟，你要是那么干，那就完蛋了。

Dude, if you did that, you were fucked.

就是这样。

That's it.

你的电脑就废了。

Your computer was dead.

所以我们干脆就多配点内存。

So we literally over-provision RAM.

还有路由，你知道有多少自适应路由算法吗？

And routing, you know how many adaptive routing algorithms there are?

我们会根据流量来应对。

We will respond to traffic.

但是会有振荡，我们就说，'算了，我们过度配置。'

But you have oscillations, we're like, "Fuck it, we over-provision.

我们就用ECMP。

We just use ECMP."

所以我感觉系统的历史就是，你不去预测，而是过度配置，因为你预测不了。

So I just feel like the history of systems is that you don't predict stuff, you over-provision because you can't.

所以我觉得在运维方面，要么我们提前构建过度配置的系统，这样你就不需要做复杂的运维工作，要么也许AI能像人一样做，但我觉得这和代码很不一样，因为你处理的是异常情况。

So I feel like in the operations side, so either we'll build the systems up front that are over-provisioned, so you don't have to do complex operational stuff, or maybe AI can do it the way that a human being, but I think it's very different than code because you're dealing with exceptions.

数据也不多，等等。

There's not a lot of data, et cetera.

我觉得这完全是另一个领域。

I just feel like a different domain.

但我不确定。

But I don't know.

我是做风投的。

I'm a VC.

我就觉得……

I'm just like...

我之所以更乐观，是因为我所在的领域，我们处理的就是可观测的世界。

The reason that I'm more optimistic here is because I live in the world where what we do is actually deal with the observable world.

我正想说，你才是这方面的专家，我问你这些问题真是有点傻。

I was going to say, you're the literal expert on this, so it's silly for me to even ask you these questions.

不，完全不会。

No, not at all.

因为我在乎的只是——

Because all I care about is very-

因为在我的领域里，我的假设是我们拥有所有数据，我们确切知道什么时候有新情况发生、什么时候出现异常、工作负载何时变化等等。

Because in my world, my assumption is that we have all the data, is that we know exactly when a new thing happens or an exception emerges or a workload changes or whatever.

在我的领域里，所有这些数据都已经收集并流式传输过来了，我们可以——

In my world, all of that has already been collected and streamed through, and we can-

对，但是那个模型并没有基于这些数据的真实来源进行训练。

Yeah, but the model that is not trained on a grounded source for that stuff.

不是。

No.

那确实很难。

That would be really hard.

但这里的情况不是像，“嘿，克劳德先生，这里有一堆日志。”

But the story here isn't like, "Hey, Mr. Claude, here's a bunch of logs.

“是不是有什么问题？”

Is something going on?"

它必须更能运行实验，深入探究，看看可能存在的不同因果链形态——

And it has to be much more able to run experiments and dive in and see what are the different shapes of possible causal-

明白了。

I see

——这些因果链要真正符合现有的环境。

... chains that actually fit in the environment that exists.

明白了。

I see.

对，这就是你之前说的。

So yeah, this is what you said before.

现在我明白你的意思了。

Now, I see what you're saying.

你是说，宇宙是重尾分布的，可能发生的坏事数量是异常庞大的，但用来排查问题的步骤却是可控的。

You're saying that the universe is heavy-tailed, the number of bad things that can happen is probably an exception thing, but the set of steps used to debug it is a manageable thing.

没错。

Exactly.

而这正是你可以教给这些模型的东西，也是可能发生的事情。

And that's what you can teach one of these models, and that's what can happen.

没错。

Exactly.

因此，如果你给它正确的信号，它就能通过一系列步骤来真正限制这个问题。

Therefore, if you give it the right signals, it can go through a number of steps to actually constrain this problem.

没错。

Exactly.

有意思。

Interesting.

没错。

Exactly.

而且它还能——

And it can-

所以它确实是在扮演人类的角色，因为记得——

So it really is taking the role of the human because remember-

没错。

Exactly

……整个混沌工程浪潮，就像……

... the whole chaos engineering wave where like-

对。

Yes

……而它没能取得进展的原因在于，你试图极其迅速地缩小一个由宇宙原子构成的状态空间，并真正使其变得可控。

... and the reason it didn't go anywhere is you're very quickly trying to reduce a state space that was atoms in the universe, super quickly, and actually make that manageable.

而且之前是人工一步步去调试，现在你说的是让AI来做这件事。

And it was the human being that would go through steps that would do that on the actual debugging, and this is what you're saying the AI would do.

对。

Yes.

所以目标不是让模型只是说“这就是发生的情况”。

So the goal isn't to have the model just be like, "This is what happened."

明白了。

I see.

而是说“嘿，按照这个流程去排查，看看发生了什么”。

It's, "Hey, run through this workflow to determine what happened."

我们在安全领域也在尝试这个，比如在安全方面，你的统计数据里有些东西变了，实体存在的情况、各种数量关系上有些行为差异——

And we're trying that also on the security side, for example, where, hey, in security, you have something different changed in your statistics, some behavioral difference in terms of what entities exist, the cardinalities of various things-

——就是那些存在的实体。

... that exist.

所以这就变成了系统中的一个扰动——

And so that becomes a perturbation in the-

……随便什么，整个宇宙。

... whatever, the universe.

所以我们拿到那个信号，然后把它喂给AI，然后我们说：“嘿，AI，想出一堆可能的因果链，然后去验证一下，看哪些符合我们实际看到的数据模式。”

And so we take that signal, and then we feed it into an AI, and we're like, "Hey, AI, come up with a bunch of possible causal chains that exist, and then go through and verify if any of those fit the pattern of the data that we actually see."

所以这就更像是在说：“嘿，把这个方块塞进那个孔里，或者找到适合这个方块的孔。”

And so that becomes a much more like, "Hey, here's fit this square into find the peg or find the hole that fits this peg."

我觉得这对我来说更难，因为操作方面的问题是，系统可能会进入非常罕见的故障状态，而且这种罕见状态有很多。

That seems even harder to me because the operations one, the argument is The system can get into broken states that are very rare, and there's a lot of those rare states.

所以这根本没法训练模型，因为它可能进入的坏状态太多了。

And so it's just not something that you could train a model on, just because there's so many rare states it can get into that are bad.

也许你可以通过一步步操作来减少这种情况。

And so maybe you can reduce that by doing the steps.

安全方面的问题看起来更难，因为这不仅仅是罕见状态的自然顺序。

The security one seems even harder because this isn't just the natural order of rare states.

实际上有一个对手在试图——

You actually have an adversary that is trying to-

对

Yes

……故意去找那些罕见的状态——

... intentionally find rare states-

对

Yes

……而且故意去找那些——

... and intentionally find stuff-

对

Yes

……你做不到的东西。

... that you can't do.

所以有可能代码问题会被解决。

And so it may be the case that code becomes solved.

操作方面需要更长时间，而安全方面则需要更久——

It takes a little while longer to do operations, and it takes a lot longer to-

对，但你也想想，我也不知道。

Yeah, but also think about, I don't know.

对我来说，整个世界在某种程度上都是量子化的。

To me, the whole world is sort of quantized to a certain extent.

到普朗克长度。

To Planck length.

物理学家是这么说的吗？

Is that what the physicists say?

我不知道。

I don't know.

我又不是物理学家。

I'm not a physicist.

普朗克常数。

The Planck constant.

不，我觉得有像……其实，我现在完全不知道自己在说什么，但我真的觉得时空是离散的，分成普朗克长度。

No, I think there's like a... Actually, I literally don't know what I'm talking about right now, but I literally actually think that space-time is discretized into a Planck length.

不过话说回来。

But anyway.

对，我同意。

But yes, I agree.

某种程度上，你说得对，但就一般的操作和系统而言——

To a certain extent, what you just said, but for operations and systems in general-

……

...

可能的状态以及它们之间的可行路径是离散的。

there is a discretization of the possible states and the paths through them that are possible.

我只是觉得，当你处理——

I just think when you're dealing with-

可能有很多。

There might be a ton of them.

可能有很多。

There might be a ton of them.

我跟你说。

I'm going to tell you.

你还记得咱们研究生小组里的桑达尔吗？

So remember Sundar, who was in our grad school group?

艾尔？

Eyre?

桑达尔·艾尔？

Sundar Eyre?

对，艾尔。

Eyre, yeah.

是的。

Yes.

我记得他有一次来找我，看起来特别难过。

So I remember he came to me once, and he was looking very sad.

他说，‘马丁，我刚有一个特别难过的发现，就是如果你拿任何一首MP3……’你还记得MP3吗，就是那些音乐文件？

He's like, "Martin, I just had this really sad realization that if you take any MP3..." Remember MP3s, the music files?

是的。

Yes.

他说MP3的时长是有限的，比如说四分钟。

He said MP3s have a finite, let's say, four minutes.

大概就是四兆字节。

It's like four megabytes.

那就意味着宇宙中歌曲的数量是有限的。

And that means there's a finite number of songs in the universe.

那可以——

That can-

不，因为音乐都是由比特组成的，对吧？

No, because it's all made of bits, right?

而且就是四兆字节，那就意味着你可以把每一个这样的组合都列出来。

And it's four megabytes, and that means you could just enumerate every one of those constants.

我对音乐有限性这个概念感到非常难过，对吧？

I'm so sad in this notion of the finiteness of music, right?

说实话，我们当时正走在那个碟形天线上，他就在那儿说这个，我心里想：“我去，哥们儿，你知道你这么快就把宇宙里的原子都算完了吗？”

Literally, we were walking on the dish when he had this conversation, and I'm like, "Motherfucker, dude you know how quickly you've overcome the atoms in the universe?

你连K都还没到呢，对吧？”

You haven't even gotten to a K, right?"

这就像2的2次方的2次方的2次方。

It's like two to the two to the two to the two.

我说：“哥们儿，你到底在说啥呢？”

Like, "What are you talking about here, man?

开心点吧。”

Be happy."

2的2次方的2次方。

Two to the two to the two.

四兆字节。

Four megabytes.

四兆字节。

Four megabytes.

你在说什么呢……

What are you talking...

状态太多了。

There's so much state.

开心点吧。

Be happy.

这几乎是无限的，对吧？

That's effectively infinite, right?

就像，我们到底在讨论什么呢？

It's like, what are we talking about here?

所以我确实觉得，在某种程度上，我们低估了系统可能进入的状态数量。

So I do think at some level, we do underestimate the number of states that systems can get into.

而且我作为风投，看过这些公司，那些试图解决运营问题的公司，日子都不太好过。

And I will tell you from the perspective of a VC that looks at these companies, the ones that are trying to tackle operations are having a tough go at it.

这看起来比那些解决代码问题的公司要难得多，因为我觉得这个问题本身就更难。

It seems much, much tougher than those that are tackling the code, because I think it's a much tougher problem.

完全同意，我也是这么想的。

100%, I agree.

这确实是个更难的问题。

It is a much tougher problem.

而且不只是因为系统本身的问题，还有外部负载是随机的，可能突然发生，或者有人刚发布了一个新东西，直接部署到生产环境，也没告诉任何人，或者当这个发生和那个发生时，就会出现某种状态，最后导致死锁。

And it's not just because the systems themselves are, there's external loads that are random that kind of happen to you or that somebody just shipped a new thing and rolled it out into production, didn't tell anybody, or there's some state that actually happens when this thing happens and this thing happens, and then you end up in a deadlock.

还因为这个问题涉及更多人的参与。

It's also because it involves humans a lot more.

而且不只是写代码的那个人，实际上还涉及——

And not just one human who's the coder, it actually also involves-

哦，对。

Oh, right.

知识也是分散在不同人手里的。

The pools of knowledge are distributed too.

没错。

Exactly.

还有连接各个部分并把它们组合起来所需要的那些交互。

And the interactions between what it takes to connect the pieces and put them together.

但归根结底，我在想AI现在处于什么位置，未来有哪些可能性，以及所有这些事情的深度和前瞻性。

But ultimately, I think about where AI is and what are the possible futures and the depth and the forward-looking capacity of all the things.

我觉得代码对我来说，它就是一个缩影，也许我只是太宿命论了。

I think code is just, to me, it's a microcosm of, maybe I'm just too fatalistic.

也许我对世界运作的方式太悲观了，我觉得所有事情都遵循某种概率路径。

Maybe I'm just too pessimistic about how the world works, and I think of everything as following some kind of probabilistic path.

所以对我来说，如果我们能让代码运转起来，我们就能一点一点地扩展那种复杂度，一层一层地叠加，直到我们能处理那些运营上的问题。

And so to me, if we can make it work in code, we can continue expanding that level of complexity bit by bit and kind of adding layers and layers and layers until we can handle those problems of operations.

很多人说这不是宿命论，恰恰相反。

A lot of people say it's not fatalistic, it's the opposite.

人们会说这是非常乐观的。

It's very optimistic is what people would say.

对我来说，我不认为宇宙会轻易交出它的秘密，但走着瞧吧。

For me, I don't think the universe gives up its secrets that easily, but we'll see.

这是个关键问题，我得说，目前还没有答案。

This is kind of the big question, and I would say, as yet, it's not an answered question.

对。

No.

对，但我也不确定。

No. But I don't know.

很多我们人类觉得随机的事情，其实是没法解决的。

A lot of things that, as humans, we think of as being random are impossible to tackle.

这会影响架构。

It impacts the architecture.

是的。

Yes.

这就是我们在编程中看到的。

And that's what we saw with coding.

一开始，人们觉得能用代码实现的东西是有限的，你没法让一个系统真正理解人脑所需的全部复杂性。

Initially, people thought the possible world of all these things that you can implement in code are kind of limited, and you can't really get a system to potentially understand the full complexity of a human brain that's required.

我觉得随着时间的推移，一切都会……

I think over time, it'll all...

真有意思，我居然在跟你说这个，而你是个看未来的VC，你会说——

It's so funny that I'm actually telling you this, and you're the VC who's looking into the future and being like-

不，好吧，我来告诉你我的看法。

No, well, I'll tell you my view.

我的看法是，研究这些模型，你是在数据上训练，要么就是……

My view is the study of these models are you train on data and either that...

我在想怎么措辞。

I'm trying to phrase it.

所以你在数据上训练，这代表了一个分布。

So you train on data, and that represents a distribution.

这个分布有用或者没用。

That distribution is useful or not.

当然。

Sure.

所以当你用语言训练模型时，它之所以有用，在我看来，这说明人类非常了不起，因为他们观察了宇宙这个高维系统，并将其简化成了一个对模型真正有用的流形，对吧？

And so when you train on language, the fact that it's useful, that to me says that humans are so remarkable because they've looked at the universe, which is this very high-dimensional system, and they've reduced it to a manifold that's actually useful for a model, right?

是的。

Yes.

但这和宇宙本身是不同的。

But that's different than the universe.

当然。

Sure.

从高维到语言，这是一个降维的过程，语言的维度要低得多。

There's a reduction from this high dimensionality to the language, which is much less dimensional.

而且我认为，代码是机器所有运行状态的一种简化，它把机器运行所需的结构提炼了出来。

And I think that code is a reduction of all the operational states of a machine into the structures that make it run.

所以在我看来，指令或描述与实际宇宙的运行、实际系统的运行之间，仍然存在脱节。

And so to me, there's still a disconnect between the instructions or the description and the actual running of the universe and the actual running the system.

我觉得我们可以在这方面取得进展，但我要说的是——你有点在向模拟的方向靠拢了。

I think we can make gains toward it, but what I will say is- You're kind of converging on simulation.

有些事情在计算上是不可简化的。

There are things that are computationally irreducible.

你得实际运行它才能得到答案。

You kind of have to run it to get the answer.

对。

Yes.

你不能只是——

You can't just-

……预测它。

... predict it.

所以也许我们正处在一个事件视界上，模型能解决任何你不需要实际模拟就能得到答案的问题。

And so maybe we just are at an event horizon where models will solve anything that you don't have to actually simulate to get the answer from.

但为什么？

But why?

为什么你非得从它必须现在回答问题这个角度来想，而不是能够模拟然后——抱歉。

Why do you have to just think about it in terms of, hey, it'll have to answer the question now as opposed to being able to simulate and-- Sorry.

能够规划自己的执行步骤，去获取它需要的信息，以便能够——

Being able to plan its own execution to gain the information that it needs in order to be able to-

也许这就是我们前进的方向，是的。

Maybe that's where we're going, yeah.

这正是我——

That's kind of what I'm-

有道理。

Fair enough.

而且我认为存在一个低维空间，也就是你将要采取的步骤，我们可以用AI大幅减少这些步骤，然后对于它无法完成的步骤，它可以编写代码来完成，或者通过模拟来完成。

And I think there's a lower dimensional space which is the step that you would take that we can reduce a whole bunch with AI, and then whatever steps that you need that that can't do, it can create the code to do or it can simulate and do it.

但你要说的重点是——

And the point that you're making, though-

对

Fair

……就是，你看，模型实际上无法去探索我们需要发现的世界额外维度，以便将宇宙建模到它的世界模型中。

... is that, hey, look, the model can't actually go and figure out the additional dimensions of the world that we need to go and find to model the universe into its world.

这很可能是真的。

That's probably true.

但它真的能模拟一个科学家，达到那个程度吗？

But can it actually simulate a scientist sufficiently to be able to get to that point?

也许吧。

Maybe.

也许吧。

Maybe.

好。

Cool.

我能问你一个问题吗？

Can I ask you a question?

你觉得风投工作中有多少是AI可以主导的？

How much of the VC work do you think is something that an AI can drive?

我对风投的理解是，它非常依赖人际关系。

So my model for VC is it's very relationship driven.

所以我就是相信市场其实挺有效的。

So I just have a belief that markets are actually pretty efficient.

所以关键不是去找别人都没发现的公司。

And so it's not about finding the company that nobody else finds.

我觉得大家都能找到好公司。

I think people all find the right company.

我觉得市场是有效的。

I think they're efficient.

然后我对VC的看法是，科技是积极的，我对所有科技都挺乐观的，我也不担心市场规模。

And then I also have the model of VC that tech is positive and I'm kind of optimistic about all tech, and I don't worry about TAM.

所以我觉得关键就是能不能接触到这些机会。

And so I think the key is access.

就像融资，靠的是人脉，是能不能拿下项目，还有拿下之后能不能帮上忙。

It's like raising money, it's access, it's ability to win deals, and it's ability to add value after that happens.

而很多这些事，初创公司自己根本做不来，比如招大牌高管或者在这方面帮忙。

And a lot of that is stuff that startups just can't naturally do, like recruit big execs or help with that.

所以我觉得做风投这件事，你得是个真正的人才能干。

And so the way that I think about venture, you kind of have to be a human being to do.

因为那才是真正的活儿。

Because that's the actual job.

所以在这方面，我觉得AI永远做不到——就算你给AI一个模拟宇宙，让它去搞什么奖励机制、建立信任关系，那也行。

So for that, I don't think an AI can ever- If you gave an AI a simulated universe where it could, whatever, reward hack, building trust relationships, fine.

但我觉得那也得——不过话说回来，我确实见过AI做的分析，跟任何人比都不差，甚至更好——

But I think that that would have to be- That said, I will say, I have seen analysis done by AI that are as good or better than any-

比我合作过的任何天才都强。

... genius I've ever worked with.

所以在分析这块，它们很厉害。

So for the analysis piece, they're great.

但我觉得这更多说明的是风投的分析能力不行，而不是模型有多强大。

But I think this is more of an indictment on VC's ability to analyze than it is how powerful the models are.

我就是觉得那全是花架子，都是扯淡。

I just think it's all a kabuki show and bullshit.

我觉得真正好的风投会认为，科技是净正面的。

I just think the right VCs are like, tech is net positive.

科技总能创造价值。

Tech ways always add value.

找到最好的公司，然后投资它。

Let's find the best company and invest in it.

市场是非常非常高效的。

The markets are very, very efficient.

所以我觉得这个问题的答案几乎有点像是钻牛角尖。

And so I just feel like the answer to this question is almost kind of a technicality.

我的答案几乎就是钻牛角尖。

My answer's almost a technicality.

所以有件事，我来这里，其实想问你一个真实的——

So one of the things, I came here, I wanted to ask you this real-

——这个问题，就是我们开始具体聊运营的时候。

... this question, which is, we started talking specifically about ops.

但对我来说，AI写代码的世界，就像是一个抽象的模拟世界，它存在于其中。

But to me, there's this world of AI writing code, which is kind of like this abstract simulation world that it lives in.

然后还有现实世界的其他部分。

And then there's the rest of the world.

对，没错。

Yeah, exactly.

而且——

And-

整个讨论就是这个。

The entire discussion is this.

AI所在的世界，全是数字化的，不管是什么，训练数据、分布内的世界，然后还有现实世界。

There's the world that the AI lives in, which is all digital and whatever it is, the training data, the in-distribution world, then there's the actual world.

这两者不一定是一样的。

And those things are not necessarily the same.

很多时候它们并不一样。

They're quite often not.

所以问题是，你需要训练AI到什么程度，它才能在一个世界里生存？

And so the question is, to what extent do you need to train an AI to be able to survive in a world?

还有它真正需要的是什么——我不想回答那个问题，你需要训练什么？

And what are the things that it actually-- I don't want to answer that question, what do you need to train?

但我对开放空间很好奇。

But I'm curious about the open spaces.

我们之前聊了一点运营方面的事。

So we talked a little bit about operations.

你说："哦，我不知道这能不能解决。"

You said, "Oh, I don't know if it's going to be solved or not."

我就说："对，马丁，这肯定能解决。"

I'm like, "Yeah, Martin, it's going to be solved."

在我看来，还有很多其他事情。

And in my mind, there's so many other things.

比如，自动驾驶汽车。

For example, self-driving cars.