为何传统基准测试无法评估现代 AI 模型——OpenAI 研究科学家 Noam Brown

用 GBT 三的时候，你没法通过增加测试计算资源来提升性能。

With GBT three, you couldn't scale test time compute.

比如，你给它一千万美元的预算，说好吧。

Like, if you gave it a budget of $10,000,000 and said, okay.

那我们就看看 GBT 三到底能干点什么。

Well, let's see what GBT three can do.

但它实际上能做的非常有限。

It really can't do that much.

那些脆弱的框架和负责任的扩展策略，根本没考虑测试阶段的计算资源投入。

The precarious frameworks and responsible scaling policies, they don't really account for the amount of test time compute.

它们只是说，好吧。

They just say, okay.

那这个模型的能力到底怎么样？

Well, what's the capability of the model?

问题是，现在这个世界里，模型的能力基本上就取决于你投了多少钱。

The problem is we're in a world now where the capability of the model is a function of how much money you put into it, basically.

如果给它一万美元的预算，它能做的事情远比只给一万美元多得多。

If you give it a budget of $10,000, it can do a lot more than what it can do with a budget of $10.

要是给它一千万美元的预算，它能做的就更多了——但到底该用多少预算来评估这些模型呢？

Give it a budget of $10,000,000, it can do even At what budget should you evaluate these models?

如今的评估政策根本没回答这个问题。

The policies that exist today don't really address that question.

大家好，各位听众。

Hi, listeners.

我是莎拉·郭，欢迎回到《无界》节目。

I'm Sarah Guo, and welcome back to NoPriers.

今天我请到了诺姆·布朗，他是AI推理领域的奠基人之一。

Today, I'm here with Noam Brown, one of our godfathers of AI reasoning.

我们会聊聊当前评估体系的缺陷、大规模测试计算资源的问题，他如何看待递归式自我提升，以及前沿领域未来竞争的方向。

We talk about the broken state of evaluations, very large scale test time compute, how he thinks about recursive self improvement, and what's next on the horizon for competition at the frontier.

欢迎。

Welcome.

Noam，很高兴你再次来做客。

Noam, I'm so excited to have you back.

能回来我也很开心。

That's great to be back.

是的。

Yeah.

你是我们节目的第一位嘉宾。

You are our first guest.

能请到你这样的朋友和研究者，我真的很自豪，毕竟推理时的算力扩展现在对整个行业都这么重要。

I'm very proud of my taste in friends and researchers for the pod, given, you know, how important, you know, inference time scaling has become to the industry.

你也该感到自豪，毕竟你确实是这一领域的开拓者。

You should be proud too, having actually pioneered

嗯。

it.

和许多其他人一样，我也参与其中。

Played apart, yeah, among many others.

你刚写了一篇关于大规模测试阶段计算资源的文章，特别有共鸣，讲的是为什么行业对这些模型的评估还不够严谨。

You just wrote this essay that really resonated about large scale test time compute and why the industry is not evaluating these models as robustly as it should be.

你写这篇文章的初衷是什么？

What was the motivation for it?

是的。

Yeah.

初衷是因为我们发布了5.5版本，一开始大家的反应是有点怀疑，觉得它没好到哪里去。

The motivation was we released 5.5, and the initial reaction was kinda skepticism that it was a substantially better model.

说实在的，这种怀疑只持续了几个小时，等大家实际去用、去试了之后，马上就发现它确实强了不少。

This to be fair, that only lasted for a few hours before people had some time to play around with it and and try it out themselves, and they saw that it was actually substantially better.

但我认为，很多怀疑其实来自那个公开的基准测试表格。

But I think a lot of the skepticism came from the benchmark grid that was published.

一般来说，每次发布新模型，都会出一张基准测试表，横轴是各种测试项目，纵轴是不同模型的表现，大家一眼就能对比出来。

Basically, whenever a new model is released, there's this benchmark grid where they show all these different benchmarks on on the x axis and then the performance of different models on the y axis, and you can just, like, compare different models.

就是用一个数字来代表一个模型在某个测试里的表现。

It's like a single number for a model on a single benchmark.

如果你光看纸面上的数据，比如5.5和5.4或者其他版本之间的差距，确实有提升，但没大到哪儿去。

And if you look on paper at the difference between, like, five point five and five point four or or other models, it wasn't it was an improvement, but it wasn't a huge improvement.

在一些基准测试里，只高出几个百分点而已。

It was only a few percentage points on some benchmarks.

所以人们一看这数据，就觉得这模型也没强到哪儿去。

So people looked at that, and they were skeptical that it was actually a better model.

可一旦真上手试了，看法就完全不一样了。

Once they played around with it, the story changed.

我觉得这些基准测试没反映出真实优势，是因为测试结果的呈现方式有问题。

I think the reason why it doesn't show up as so much better on the benchmarks is because the benchmarks are being presented the benchmark results are being presented in the wrong way.

它们没控制每次提问所用的计算资源量。

They're not controlling for the amount of test on compute that is being used on that benchmark question.

结果发现，5.5在思考效率上高得多。

It turned out that 5.5 is just much more efficient with its thinking.

如果开到最高配置，5.4的思考时间要长得多。

If you run it at max settings, 5.4 is thinking for a lot longer.

它返回结果的时间比5.5要长得多。

It takes longer to get back a response than 5.5.

一旦把思考时间控制住，你就会发现5.5相比5.4有明显提升。

And once you control for the amount of thinking time, actually, you can see that 5.5 is a substantial jump over 5.4.

我觉得这正是大家日常使用时的真实感受。

That is, I think, people's day to day experience with it.

当我跟别人提起这点时，最常见的反应就是：哦，这样啊。

And then when I mention this to people, the reaction the typical question I get is like, okay.

那为什么不干脆让5.5也像5.4那样想得久一点呢？

Well, why not just have 5.5 think for as long as 5.4?

问题来了：那到底该让它想多久才合适？

And the question is like, well, how long should they think for?

我通常得到的回答是：直到性能不再提升为止。

Typically, the response I get is, well, until the performance plateaus.

对吧？

Right?

到了某个时候，模型在基准测试上的表现就会趋于稳定，我们就直接在那个点上评估结果。

This is at some point where the performance on the benchmark is gonna plateau, and you just evaluate to that point.

问题是，现在这个稳定点其实已经远远靠后了。

The thing is, the point at which it plateaus is actually really far out these days.

要是放在2022年GPT-3那会儿，模型根本没法长时间有效思考，所以直接跑直到稳定就行。

I mean, if it's true in g p d three land back in 2022, models couldn't really think productively for that long, and so you could just run them until they plateau.

那时候还没那么远。

It's not that far away.

但现在的先进模型，比如5.5，只要给它合理的引导，能连续思考好几个星期，才在某些基准测试上达到表现稳定。

But what we're seeing today with the modern models is that 5.5 and other models can think for if you scaffold them reasonably well, can think for weeks even, before having performance plateau on some of these benchmarks.

所以，稳定点已经远到根本没法实际测试了。

And so the point at which they plateau is simply too far out to reasonably test.

我们现在真的得设定一个耐心上限，或者从token角度设定预算限制，这在几年前根本不是问题。

We all need to actually reinforce either, like, a patience limit or a budget limit from a token perspective now, and that wasn't true a few years ago.

没错。

Exactly.

所以我认为，现在评估模型的正确方式是，要么为基准测试设定一个预算限制，不管是按令牌数、成本还是时间，要么就把性能作为输入计算量的函数画出来。

And so I think the proper way to and so my claim is the proper way to evaluate the models now is you either have some kind of budget for the benchmark, whether it's tokens or cost or time or whatever, or you plot the performance as a function of the amount of test time compute that's going into the model.

这样一来，不同模型之间的性能对比就会清晰多了。

And then it becomes much more clear how to compare the performance between these different models.

考虑到模型评估的周期，以及很多任务的性能在很长一段时间内都不会趋于稳定，你该怎么处理这个问题？毕竟有些评估任务所需的预算或时间，已经超出了当前模型发布周期所能承受的范围。

Given the model evaluation cycle and the fact that performance does not asymptote for many tasks over quite a long period of time, what do you do about that issue, the fact that some of the evals that you would want to run are both beyond the scope of budget or time that's reasonable given the current model release cycle?

就拿网络安全这类任务来说，我们已经看到，AISI的评估结果表明，模型在处理一亿个令牌时仍在持续提升。

I mean, I think for things like cyber, we've seen, and actually the AISI in their evaluations has shown that the models continue to improve at a 100,000,000 tokens.

如果你让模型运行一亿个令牌，它在那之后还在继续进步。

You know, if you run them for a 100,000,000 tokens, they're still improving at beyond that point.

对。

Mhmm.

而运行这么长时间会花掉非常多的时间。

And that can take a very long time to run.

但你也确实会发现，性能的提升并不是突然跳变的。

But you also do see that, like, the performance is is it's not just, like, a discontinuous jump.

实际上，你能看到在这1亿个token内，模型性能提升的趋势。

It's actually, like, you can see the slope of improvement over those 100,000,000 tokens.

所以你可以在某个预算范围内做一次评估，然后直接说：好吧，

And so you you could probably do some kind of evaluation up to a certain budget and then just say, okay.

这就是我们预测的性能表现。

Well, this is what we project the performance to look like.

我觉得这方面目前还没有太多研究。

And I think this this there hasn't been a lot of research on this yet.

如果真有学者在找研究方向，我觉得这绝对是个很棒的论文选题。

I actually think this would be a great paper to publish if there's any academics out there looking for something to research.

你能根据不超过10美元、100美元这样的推理预算，预测出在1万美元预算下的性能表现吗？

Can you predict what the performance looks like at an inference budget of, let's say, $10,000, only using inference budgets up to 10 or 100 or 10 or $100?

我有个相关但不同的问题想问你。

So maybe an orthogonal question for you.

你认为用户在使用模型时，是不是普遍没有花足够长的时间去深入思考问题？

Do you think users are systematically, like, not thinking long enough with their models about problems?

你说的‘想得不够久’是指什么？

What do you mean by not thinking long enough?

如果你能构建一个智能体，或者控制推理时的计算资源，那就有两部分：一部分是模型自己做的，另一部分是用户能做的。

If you can build an agent or control the amount of test time compute being used, like, there's what is done by the model itself, and there's what the user can do.

你觉得行业目前用的推理计算量是刚刚好、远远不够，还是说问题出在模型本身，它们只是需要更快地完成思考？

Do you think that, you know, the industry is using test time compute an optimal amount, way undershooting it, or it's, you know, it's a problem in the models where they just need to be able to do that thinking faster?

我觉得这得看具体是什么问题。

I I think it depends on the problem.

有人觉得模型只要让它想上一星期，再回复就行。

I think this idea that the models, you just let them think for a week or whatever, then they respond.

听起来挺美，基准测试结果也好看，但实际用起来根本不现实——你问模型一个问题，然后干等一星期等它回复。

It's it sounds nice, and, yes, the benchmarks look great, but it's not very practical when working because, like, okay, you ask the model a question, then you sit there for a week waiting for it to come back to you.

对。

Mhmm.

我觉得大家发现最有效的方法，其实是跟模型快速地反复迭代。

I think what people have have found most effective is to kind of, like, iterate quickly with the models.

所以我觉得，思考的时间应该是灵活的。

And so the thinking time, I think, needs to be flexible.

当需要快速回应用户时，就应该迅速给出答案。

When it makes sense to respond quickly to the user, it should respond quickly.

而当情况需要深入思考，用户也希望它多想想时，那就该花时间好好琢磨。

And then when it makes sense to think for a long time and the user wants it to think for a long time, then it makes sense to think for a long time.

我认为，基于当前的实际情况，人们已经找到了比较合适的平衡点。

I think people have been striking the right balance given what they have to deal with right now.

关于刷分现象，大家现在谈得很多，说能提升各种基准测试的分数。

How would you characterize know, there's a lot of talk about benchmark maxing and the ability to gain different benchmarks.

你觉得现在的基准测试整体情况是怎样的？

What would you characterize the landscape of benchmarks as today?

你有没有特别看重的那些测试，觉得它们更能真实反映模型的能力？

And then do you have favorites that you think are more indicative of capability than others?

刷分这件事其实也推动了我写那篇论文，因为确实很容易通过把多个模型组合起来，就让分数看起来比以前的模型好很多。

So the benchmark maxing thing is also motivation for for writing an essay that I think it's really easy to show you can do much better than previous benchmarks or or previous previous models on benchmarks by just, for example, scaffolding a bunch of models together.

所以如果你说，好吧。

So if you say, okay.

我们不是只运行一次这个模型，而是运行五次，然后挑出五个回答里最好的那个，或者找个人来评判哪个更好，这样得分就能高很多。

Well, we're going to instead of just running this model once, we're gonna run it five times and take the best of the five responses or, like, ask a judge which one it it thinks is best, then you can get much higher scores than that model.

所以只要稍微动点手脚，表面上看模型表现好多了，但其实只要控制住测试时的计算开销，根本没实质提升。

And so it's really easy to make something that looks a lot better on paper but is actually not better once you control for the amount of test time compute.

这正是我对基准测试刷分现象感到担忧的一点。

That is one thing that I'm worried about when it comes to benchmark maxing.

我的担心其实就一点：这种做法有点误导人。

I mean, it, like it's it's a little misleading is the only concern that I have.

至于基准测试本身，总存在一个风险，就是大家只顾着针对测试题来优化。

And then as far as, like, the benchmarks themselves, I think there is always a risk of, like, just optimizing for the benchmark.

我确实一直提醒我的团队，我觉得OpenAI在这方面也做得不错，不会刻意去迎合某个具体基准。

And I've I've certainly encouraged my team, and I think at OpenAI, we're pretty good about not trying to optimize for for specific benchmarks.

但一旦发布了基准测试，就难免会被拿来专门优化。

But once you put out a benchmark, it's there's it's always at risk of just being optimized for.

我觉得解决这个问题的一个办法是保留一些不对外公开的私有数据集。

And I think one way to one way to address that is to keep held out private sets that isn't publicly available.

大家通常用来判断模型是否真正更好的办法，就是自己多试用一段时间。

The most popular fallback advice for, you know, figure out if a model is significantly better or not is to just play with it for a while.

你有没有比这更高级一点的建议，可以推荐给大家？

Do you have anything more sophisticated than that that you suggest people do?

比如，除了你们在OpenAI用的私有保留集之外，每次都会自己设计新的评估方式吗？

Like, do you create your own set of new evaluations each time besides private holdback at OpenAI?

我觉得每个人都有自己的一套问题，每次新模型出来都会问一遍。

I think everybody has their own set of questions that they like to ask the model whenever it comes out.

对。

Mhmm.

最近我常用它来开发扑克机器人，看看它能做出多强的扑克机器人。

For me lately, it's been I I use them to make poker bots and see how good they can make a poker bot.

我觉得这是个不错的测试方法，因为市面上公开的扑克机器人代码非常少。

I think it's a nice eval because there is very little open source code for making poker bots.

虽然有很多已发表的文章和论文，但你真的得自己仔细推敲每一件事。

And there's a lot of published essay there's a lot of published papers on it, but you really have to reason through everything.

这需要大量的思考、反复尝试，还有很多细微的坑，我都已经自己踩过一遍了，所以能看出来模型在哪些地方会出错。

And it's, like it requires a lot of just reasoning and iteration and, like, a lot of small gotchas that I can kind of I've already worked through myself, so I can see where the models fail along the way.

现在它们在这方面已经变得非常厉害了。

They've gotten really good at it now.

你能举个例子吗？比如在你开发扑克机器人时，模型在几个版本迭代中，它的推理能力是怎么逐步提升的？

Can you describe perhaps, like, with your poker bot creation, like, how reasoning might have progressed in model releases for you guys over a few releases?

是的。

Yeah.

早期的模型在这方面特别差。

When the early models were really bad at it.

基本上什么正经事都做不了。

Like, they could not basically do anything.

到了5.2版本，我就能用它来开发一个反向求解器了。

And then 5.2, I was able to work with it to make a reverse solver.

所以这算是最终阶段了，没错。

So that's, like, the final stage Mhmm.

扑克的最终阶段。

Of poker.

我觉得这一点真的非常出色。

And that itself was, I thought, really impressive.

我确实得花点功夫和它配合，但真的挺惊讶的，因为我用它把逆向求解器的速度提升到了差不多五倍，这要是我自己干可没这效率。

I I had to work with it a little bit, but I was actually really impressed because I was able to make the reverse solver probably about five times faster than I would have alone.

中间有几处地方我被卡住了。

There were a couple things that I got tripped up on.

阻断牌这个问题一直特别棘手。

Blockers was always a big, big issue.

但总的来说，只要稍微引导一下，它就像个研究生一样——虽然会遇到问题，但至少我能看懂问题在哪，也知道怎么解决。

But overall, like, you know, with a with a bit of gentle steering, it just kind of like I I kinda felt like a grad student where, okay, they would run into issues, but at least, like, I would know what those issues were and know how to fix it.

我只要提个建议，它就会自己去执行。

And I could just make suggestions, and it would go off and then do it.

然后很快就回给我一个特别棒的方案。

And then pretty quickly, would actually come back with something really good.

对。

Mhmm.

尤其是优化方面，真的让人印象深刻。

And then especially the optimization, thought was very impressive.

它能把速度提升十倍，因为它把代码优化得实在太好了。

It was able to make it, like, 10 times faster than what I was able to do because it was just able to optimize the code so well.

5.2 的问题在于，我总觉得它在误导我，得时刻小心核对，确认清楚。

The downsides with 5.2 is I felt like it was gaslighting me a lot, and I always had to be very careful checking it and making sure, like, okay.

它真正在做它说的那些事吗？

Is it actually doing what it said it did?

有没有什么明显的问题，它却没发现，或者干脆装作没看见？

Are there any things that are, like, glaring issues that it's not recognizing or it's just pretending aren't issues?

我记得有一次，我拿另一个模型做测试——不是5.2——当时我干脆把它当单元测试，跟它说：好了。

I remember there was, like, one point where for one of the models I was playing around with it, not 5.2, I kind of, like, as a as a unit test, I told it, okay.

假设底池里有100块，我选择弃牌。

Well, let's say I have a 100 in the pot, and I fold.

那我输了多少？

How much am I losing?

模型给出的答案是92块。

And the model said $92.

我当时就觉得这太离谱了。

And I was like, that's crazy.

如果底池里只有一百美元，我就直接弃牌。

I I have a $100 in the pot, I just fold it.

我怎么能避免输掉这一百美元呢？

How do I not lose a $100?

它回答说，哦，你知道的，是92。

And it said, oh, you know, it's 92.

这已经很接近100了。

It's close to a 100.

没什么大不了的。

It's fine.

不碍事。

It's no big deal.

我当时就觉得，这明显是个问题。

And I was like, clearly, this is a problem.

对吧？

Right?

所以这些模型确实有个问题，老是让你怀疑自己的判断。

So the models did have this problem where they would gaslight you a lot.

对。

Mhmm.

但等到5.5版本时，我觉得进步大得多。

But once we got to 5.5, I actually thought it was way better.

它基本上能直接零样本处理了。

It was able to basically do a zero shot.

事实上，我一直在做一个完整的扑克求解器，基本上只要我稍微引导一下，它就能独立完成整个任务。

And in fact, I've been working on just doing a full scale poker solver, and it it's basically able to do the whole thing with some gentle steering from me.

我一点都不奇怪，再过六个月到一年，模型就能一次性独立完成整个扑克求解器——也就是我整个博士论文的工作。

And I wouldn't be surprised if, six months or a year from now, the model is able to do zero shot an entire poker solver, basically my entire PhD thesis, in one go.

我们来聊聊评估这些模型时更宏观的问题，比如推理速度、效率，跟令牌数量、预算或者其他衡量标准之间的关系，对吧？

Let's talk about the larger implications of needing to evaluate these models relative to, let's say, speed of their reasoning or efficiency versus token volume, right, or dollar budget or whatever your scaler is.

你能谈谈你文章里提到的一些更深层的影响吗？特别是关于安全评估的部分？

Can you describe some of the larger implications in your essay, including around safety evaluations?

是的。

Yeah.

关于安全评估，这其实是个有点让人不舒服的真相——好吧。

The safety evaluations thing, it's it's a bit of an inconvenient truth thing where okay.

简单说一下背景，目前几乎所有实验室都有类似的东西，叫负责任扩展政策、准备框架之类的。

So I guess for background, a lot of the all of the labs have these things called either responsible scaling policies, preparedness frameworks.

它们的名字各有不同。

They go by various names.

但他们的想法是，每当发布一个模型，都要经过一系列评估，看看它有没有危险的能力。

But the idea is that whenever a model's released, they go through a series of evaluations to measure, are there dangerous capabilities?

这些模型会不会做出一些我们不希望坏人去做的事？

Could these models do things that we're we we wouldn't want, a bad actor to do?

如果模型能力不强，那也就没什么大不了的。

And if the model isn't very capable, then it's no big deal.

但如果它能力很强，比如能用来制造生物武器，那就得采取措施来防范了。

But if it is very capable, if it could be used, for example, to make bioweapons, then you want to put in mitigations against that.

对。

Mhmm.

但问题来了，

But the question is, okay.

你怎么判断这个模型到底有没有这种能力？

Well, how do you evaluate whether the model is capable of that?

他们有一套不同的方法来开展这些评估。

And they have, like, various protocols about, like, how they do these evaluations.

但很多这类框架都是在ChatGPT问世前后开发的，那时候测试阶段的计算资源扩展还没那么重要。

But a lot of these frameworks were developed around the era of ChatGPT, either before or after, when test time compute scaling was not really as much of a thing.

对。

Mhmm.

那时候这么做是合理的。

And it made sense.

比如GPT-3，你再怎么增加测试时的计算资源，也发挥不出太大作用。

Like, with GPT three, you couldn't scale test time compute.

就算你给它一千万美元的预算，说：‘来，看看GPT-3能干点啥’，

Like, if you gave it a budget of $10,000,000 and said, okay.

它能做的事，其实跟花十块、一块钱差别也不大。

Well, let's see what g p t three can do, it really can't do that much more than what you could do with, like, $10 or $1.

现有的准备框架和负责任的扩展政策，都没怎么考虑测试阶段的计算资源投入。

The preparedness frameworks and responsible scaling policies, they don't really account for the amount of test time compute.

它们只是简单地说：‘好吧。'

They just say, okay.

那这个模型的能力到底怎么样？

Well, what's the capability of the model?

问题是，现在模型的能力基本上取决于你投了多少钱。

The problem is we're in a world now where the capability of the model is a function of how much money you put into it, basically.

对。

Mhmm.

如果给它一万美元的预算，它能做的事情远比只给十美元多得多。

If you give it a budget of $10,000, it can do a lot more than what it can do with a budget of $10.

如果给它一千万美元的预算，它能做的事情还会更多。

If you give it a budget of $10,000,000, it could do even more.

那到底该用多少预算来评估这些模型呢？

And so at what budget should you evaluate these models?

现有的政策根本没回答这个问题。

The policies that exist today don't really address that question.

对。

Mhmm.

有些模型表现好一些，有些差一点，但总的来说，这个因素并没有被认真考虑。

Some do some do better than others, but for the most part, this is not really a factor that's being heavily considered.

至于要不要发布，我也不想深入讨论这个问题。

Now whether it should be released anyway, I don't I don't wanna wade into this question.

我觉得两边都有道理，这事儿本来就挺复杂的。

I think there's, you know, there's arguments on both sides.

但关键在于，我们现在就像视而不见，假装这问题根本不存在，其实不管怎么选，我们都得正视它。

But I think the important thing to recognize is that this is a question that is not being we're we're just kind of, like, you know, pretending that this issue doesn't exist, and I think it's important to just, you know, one way or the other account for it.

是的。

Yeah.

这就跟能力问题是一体两面：如果模型在高预算下能不断突破，那它在我们不希望它做的那些事上，也会同样变得更强。

It was the mirror image of the capability question of if the models can continue to do more and more without asymptoting on some tasks at very large budgets, then they should also be able to do so for tasks we don't want them to do as a society.

对吧？

Right?

所以，测试这些能力时用的预算，跟模型发布的时间点根本对不上。

And so testing for that and what budget is allocated, it also seems out of sync from the model release cycle itself.

对吧？

Right?

现在的情况是，新模型的发布速度越来越快，有时候几天或几周就出一个，而不是以前的六个月。

There's been this acceleration of, you know, you get a new model every sometimes few days and weeks at this point versus six months.

你在文章里提到，要真正评估一个智能体在长期任务上的表现，可能得让它运行一整年，这对有用的任务和有害的任务都一样。

And you have a line in the essay where you say, like, the the only way to truly evaluate an agent on some very long running task might be to run it for a year, and that's gonna be true of both, like, useful and negative tasks.

对吧？

Right?

那你怎么看这种评估方式和模型发布周期之间的矛盾？

And so how do you think about that versus the model release cycle?

是的。

Yeah.

这也是个挺有意思的现象：随着模型能力变强，它们在更长时间跨度上的操作能力也提升了。

This this is also an interesting dynamic where, basically, as the models have become stronger, they've they're more they're better able to operate over longer horizons.

对。

Mhmm.

所以，拿 GPT-3 来说，如果你真想让它跑上一星期，其实很难把它拆解成能持续运行一周的有用任务。

So, again, with g p d three, if you wanted to run it for, you know, a week, there's really not much you could do to scaffold it into something useful that could actually run for a week.

但现在的最新模型就不一样了，比如 5.5，你已经能让它连续执行一系列实验，持续几周甚至几个月。

But we're seeing now with the most recent models that you can actually scaffold, for example, 5.5 into doing a series of experiments that can run for weeks, for months.

你有没有给你的扑克求解器任务无限的资源？

Have you given your poker solver task, like, infinite budget yet?

我还没真把任务搭起来，直接跟它说：‘你去跑几周吧。’

I haven't really scaffolded something together where I just tell it, like, okay, just run this for for weeks.

我觉得我能做，只要给它一个目标，然后让它放手去干就行。

I think I could it I I could probably give it slash goal and just, like, yeah, tell it to go nuts.

但要是现在就给它一个目标，让它去完成反向求解器，它百分之百能做到。

But I I think at this point, it could 100% do the reverse solver if I just give it slash goal.

不过，要让它光靠一个目标就自己跑一个月，做出完整的扑克求解器，它现在还达不到这个水平。

I don't think it's at the level yet where it could do, like, the full poker solver if I gave it just slash goal and told it, yeah, go go run for a month.

但再过不久，我们大概就能直接跟它说：‘你去搞一个月，回来交我一个顶尖的扑克求解器。’

But we're going to pretty soon be at that point where I I probably could just tell it, like, yeah, go work on this for a month, and then come back to me with a a full, complete poker solver that's state of the art.

问题是，如果你想评估一个模型运行一个月后的能力，唯一能完全确定的方法就是真的让它跑上一个月。

And the problem is if you want to evaluate the capabilities of a model, what it can do after running for a month, the only way to be fully sure is to actually run it for a month.

如果你想了解它六个月后的表现，唯一可靠的办法就是让它跑六个月。

And if you wanna know after six months, the only way to know fully is to run it for six months.

后面我会提到一些我们可以用来解决这个问题的方法，但首先得明白，现在的模型发布周期就是这样的。

Now there I'll I'll get to, like, things we could do to address that a little bit later, but, like, it's important to recognize that the model release cycle is look.

目前，我们差不多每两三个月就推出一个新模型。

We're releasing new models, like, every two or three months at this point.

所以一个模型刚发布，得花两三个月才能把它的潜力榨干，结果这时候下一个模型又出来了。

And so a model comes out, it takes two or three months to push it to its limits, and then you have another model come out.

因此，没人真正知道这些模型的能力上限在哪里，因为没人有耐心把它们跑够足够长的时间来验证。

And so nobody actually knows what the ceiling of capabilities are for these models because nobody's actually run them for long enough to really tell.

比如，当 Slash Goal 刚发布的时候，有人开始用它跑任务，结果花了整整一周才跑完，可大家直到一周后才意识到这有多厉害。

When slash goal came out, for example, I mean, people started running things that it took over a week for it to finish, and so people actually didn't realize that this was a big deal until after a week, until a week after it was released.

对。

Mhmm.

我觉得这种情况会越来越普遍。

I think that's gonna be more and more true.

这背后的影响其实挺有意思的，问题是，实验室在发布模型之前，到底该怎么彻底评估它们的能力呢？

You know, the implications of that are, I think, pretty interesting because what do the labs do to, like, fully evaluate their models before they're released?

这真的很难，因为要真正做好评估，唯一的办法就是推迟模型的发布周期。

It's actually very difficult because, yeah, you would have to the only way to to really do the evaluations is then delay the model release cycle.

但现在市场竞争这么激烈，大家都不愿意这么做。

And, you know, there's a lot of competitive pressure right now to not do that.

你认为那些已经发布的模型里，是不是还藏着一些没人来得及发掘的潜力？

Do you think there's, like, exciting latent capability in the models that are already released that people have not fully explored given timeline?

我绝对这么认为。

I think absolutely.

其实有个特别好的例子，就是埃爾德什单位距离问题。

I think actually a really great example is the Erdos unit distance problem.

对不了解的观众来说，几周前，我们用OpenAI的一个内部模型推翻了埃爾德什单位距离猜想。

So for the viewers that don't know, like, we used an internal model at OpenAI a few weeks ago to disprove the unit Erdos unit distance conjecture.

我不是数学家，但这件事显然非同小可。

Now I'm not a mathematician, but this seems like it was a a pretty big deal.

在数学界，这可是第一个许多专家长期钻研却始终没解决的问题，而模型不仅解决了，还给出了一个让数学家都觉得有趣又有用的方法。

The in the math community, it was, like, the first first problem that a lot of mathematicians had really spent a lot of time on, and the model was able to do something that they weren't able to do and do it in a way that was actually interesting and useful for mathematicians.

说实话，整个过程花的钱少得惊人。

Honestly, it did it at a budget that was dirt cheap.

我们其实没花多少心思在这上面。

I mean, we didn't put a lot of effort into this.

我们只是训练了一个新模型，纯粹好奇它能干点啥，就拿几个问题试试看。

We just we trained a new model, and we were just curious what it could do, and we ran it through some problems.

结果这个低预算的尝试，居然一下就出来了：哦，对了，找到了。

And this one, at a pretty low budget, it was like, oh, yeah.

我好像有个反例，后来我们验证了，没错，这个反例是成立的。

I think I have a disproof, and then we were able to verify that, yeah, that disproof is correct.

我们公布结果后，不少人发现用5.5也能得出同样的答案。

After we announced the results, a bunch of people found that you could get the answer out of 5.5 as well.

现在可没那么简单了，不能光去问5.5。

If now it's not as simple as just asking 5.5, hey.

这是神经元距离猜想。

Here's the neural unit distance conjecture.

怎么反驳这个猜想？

What's the disprove?

你得给它搭个架子。

You had to scaffold it a bit.

你得稍微引导一下它。

You had to, like, steer it a bit.

后来有人发现，其实可以这样：

And so somebody found, okay.

你问了5.5。

You asked 5.5.

列出一些解决这个问题的方法。

List a bunch of ways that you could tackle this problem.

然后它会列出一条真正有希望证伪这条猜想的路径。

And then for it it lists one of the paths that are actually promising to get to the disprove.

接着你跟它说，好吧。

And then it you tell it, like, okay.

再深入探索一下这条路径。

Explore this some more.

如果你这么反复操作足够多次，它最终还真能找到证伪的方法。

And then if you do this enough times, it actually ends up arriving at the disprove.

这意味着，原则上你可以让5.5作为一个通用的辅助工具，先列出一堆不同的策略，然后对每个策略都让它去深入研究。

Now what this means is you could, in principle, ask 5.5 to you know, as as a general purpose scaffold, list a bunch of different strategies, and then for each strategy, tell it to investigate that strategy.

这样它很可能就能靠着这个通用辅助工具找到证伪的答案。

And then it would probably be able to arrive at the disprove with a general purpose scaffold.

不过这个辅助过程会非常昂贵。

Now that scaffold would be very expensive.

我的意思是，粗略估计，可能要花一千到十万美金不等。

I mean, it would probably cost I I just ballpark, like, a thousand to 10 to a $100,000.

但这确实是有可能的，早在我们之前，就有人可以用通用模型来推翻埃尔德什-希奇迪斯距离猜想。

But it would be possible, and it would have been possible for somebody to disprove the Erdosheated disc distance conjecture before we did using a general purpose model.

但没人去深入探索过：如果我花十万美金的算力去用5.5，会发生什么？

And nobody had explored sufficiently, well, happens if I put a $100,000 worth of compute into 5.5?

它到底能做什么？

What could it do?

答案是，没错，你很可能真能从它那儿得到这样的结果。

And the answer is, yeah, you probably could get stuff like that out of it.

所以人们应该更多地去尝试当前这一代模型，

So people should be experimenting more with the current generation in terms of

不过这引出一个有趣的问题：值不值得去试？因为模型的更新周期每几个月就一次，每次都会推出更强大的新版本。

Well, this is, I think, an interesting question of, is it worth it to experiment with because, again, the model release cycle is every every couple months, we put out a new model that's even more powerful.

所以每次模型更新，推翻埃尔德什-希奇迪斯距离猜想所需的代价，大概会下降十倍甚至一百倍，有时候更多。

And so the cost of disproving the air to ocean at distance congestion drops by, like, 10 or a 100 x with every model release cycle, probably, in some cases, more.

你肯定看过那个梗，说：好吧，

So You've seen the meme that's like, oh, alright.

既然这样，干嘛还要费劲做工程呢，不如直接等下一个版本好了。

Like, why why bother doing any engineering work when I should just wait for the next

模型发布？

model release?

去度假，两个月后回来，发现解决问题的成本已经便宜了一千倍。

On vacation and come back two months later, and then it's, you know, a thousand times cheaper.

你同意这个说法吗？

So agree with that?

我是说，这

I Is that

你们现在在OpenAI，就是等着下一个模型发布吗？

what you're doing right now at OpenAI, just waiting for the next model release?

我觉得吧，现在确实处在技术飞速进步的阶段，模型的能力也在不断提升。

I think I I mean, I will say we're in we're in a period where progress is very fast, and, like, yeah, the models are becoming more capable.

我可以明确说，OpenAI目前有一件事我们是刻意不做的，你看——

I I can say, like, at OpenAI, one of the things that we're we're actively not doing look.

我们有很多数学家。

We have a lot of mathematicians.

我们也有大量物理学家。

We have a lot of physicists.

大家都对这些模型目前的能力感到非常兴奋。

People are very excited about what these models can do right now Mhmm.

尤其是那些内部模型。

Especially, you know, the internal models.

我们正在鼓励大家别把所有时间都花在解决数学和物理的开放性问题上，别一味地去测试模型的极限，看它们能证明或推翻什么。我们真正认为，重点应该放在如何打造更强大的模型，如何更快、更安全地把它们推向世界，让全球的科学家都能用这些模型自己去解决问题。

We are trying to encourage people to not spend all their time just, like, going through all the mathematical open problems, physics problems, and just seeing pushing the models to their limits to see what they can prove or disprove Because we really think the focus should be on how do we make even more capable models, how can we get them get them out safely to the world as quickly as possible so that all the scientists in the world can use these models to solve the problems themselves.

所以某种程度上，我们确实觉得，现在把所有精力都投入到扩大模型规模、看看它们在极限下能做什么，这很有诱惑力，但真正的重点应该是：如何利用这些模型去创造更强大、更高效的下一代模型，让它们能以更低的成本完成更多任务。

So, yeah, in some sense, we are thinking about this that, yes, it's really tempting to just put all of our efforts into scaling up these models and see what they can do with their limits right now, but really the focus should be on how do we use these models to make even more powerful models, even more capable models that can do everything much more cost effectively.

基于你对大规模推理计算影响的看法，你认为研究方向或资源分配上有什么变化吗？

What is changing about the direction or allocation of resources for research in your mind given your beliefs about this very large scale the impact of very large scale test time compute?

这和递归自改进的概念有什么关系？比如，这一直是各个实验室追求顶尖模型能力的核心思路。

How does this interact with the idea of recursive self improvement, for example, where, you know, it's a dominant idea for how, you know, any lab gets to the best capability model.

我得澄清一点，我们现在还没到那种地步——只要给模型一个极高的推理预算，它就能在各方面都变得超级智能。

So one thing I should clarify, I don't think we're at the point where, okay, you just give it an arbitrary an an extremely high inference budget, and it's just it's just super intelligent across the board.

或者说目标

Slash goal

是的

Yeah.

基础水平

Baseline.

明白了

Okay.

GPT-7也好，其他什么模型也好，然后就放手去试吧。

G p d seven or whatever and then, like, yeah, just go nuts.

那我们和那个目标之间还差什么？

What's between us and there then?

我觉得我用过这些模型，所以有点体会。

I think having played around with the model so okay.

首先，有些基准测试中，模型就算增加推理预算，表现也不会提升。

So first of all, there are some benchmarks where the models will just not improve if they have more inference budget.

所以我觉得，很多事实性检索类的问题都属于这一类。比如你问一个人亚伯拉罕·林肯是哪年出生的，如果他不知道，就算坐那儿想上一星期，

So I think a lot of factual retrieval kind of questions fall into this category of if you ask a person when was Abraham Lincoln born and they don't know the date, they could sit there.

他想上一星期，

They could think about it for a week.

但如果他没有电脑或网络，想得再久，答案也不会比五秒内给出的更好。

If they if they don't have access to a computer or something, they're not gonna be able to do better answering that question if they thought about it for a week compared to five seconds.

模型也是一样。

Same with the model.

有趣的是，如果你给模型这类事实性问题，稍微给点思考时间，它确实能表现得更好一点；但要是真给它一星期，它也不会突然就记住了更多日期。

If you actually, interestingly enough, if you give the model these kinds of, like, factual retrieval questions and you give them a little bit of time to think, they do actually do But if you give them a week, they're not suddenly gonna do better at remembering dates.

有些基准测试中，模型明显能靠增加推理计算量提升表现，但也有一些怎么加都提升不了。

There are so there's some benchmarks where they clearly improve with more test time compute, and there's some where they don't.

而在另一端，也有一些测试，模型只要给更多推理时间，表现就能持续提升，几乎没有上限。

I think on the other extreme, there are benchmarks where they kind of obviously will keep improving limit without limit with more test time compute.

我常举的例子是数独。

So the example I like to point to is sudoku.

解数独其实有个很简单的方法：随便试一堆数字，看看能不能满足所有规则。

If you it's there's a really simple strategy to solving sudoku, which is just try a bunch of different random numbers and then see if it fits the criteria, if it if it matches all the constraints.

如果不行，就换一组数字再试。

And if it doesn't, just try a different random combination of numbers.

只要给足时间，用这个方法一定能解出任何数独谜题。

And clearly, with enough time, you will be able to solve any sudoku puzzle with this strategy.

很明显，只要给模型更多计算时间，它就能不断做得更好。

You can kind of trivially see, like, okay, any model could keep doing better and better if it was just given more test compute.

所有的评测基准其实都处在这两个极端之间的某个位置。

So you have and and all the benchmarks kind of exist somewhere between these two extremes.

模型还没到那种地步——只要给足计算时间，就能替我们做完所有工作，因为确实有些任务它怎么算也提升不了。

The models are not at the level where if you just give them enough test time compute, they will be able to do all of our jobs just because, yeah, there's some benchmarks where they will not improve.

有些事情，它就是提升不了。

There are some things where they they will not improve.

我在研究领域特别注意到一点，就是这些模型现在还缺乏良好的研究判断力，所以我觉得它们其实和研究人员是很好的互补组合。我自己就发现，用了这些模型之后，我的工作效率高了很多，但它们还远不能完全取代整个研究流程。

One thing I see for research in particular is they don't have very good research taste right now, and so I think they're actually a very good complement to researchers, especially, you know, I've found, like, I've found that I've been much more effective by using these models, but they're not able to fully replace the whole research cycle.

那这种情况会随着时间改变吗？

Now does that change with time?

大概会吧。

Probably.

我的看法是，这些模型整体上确实在不断进步。

I mean, I think the models are getting better across the board.

有些方面进步得快，有些慢，但目前还没到光靠增加计算时间就能完全取代研究人员的地步。

Some some things are getting better faster than others, but they're not at the point where they're fully replacing researchers with just enough test time compute.

你能举一两个例子吗？比如让模型直接帮你做研究任务，像这样的情况？

Can you give an example or two of, like, asking the model to do a research task for just like this?

那可太糟糕了。

Is a terrible idea.

我还是拿我那个扑克求解器的例子来说吧，当时我对模型优化我博士期间开发的算法的能力印象深刻。

I mean, I I think going back to my poker solver example, I was really impressed with the model's ability to optimize the algorithms that I had developed in my in my PhD.

说实话，回头看看，我发现自己以前效率低得惊人，而它们能把速度提升一千倍。

It was honestly it it was it was shocking to see how inefficient I was, in retrospect, and they were able to make it, like, you know, 1,000 x faster.

然后我就想，好吧。

And then I was like, okay.

你能设计出一个比我和别人都想出来的算法更好的新算法吗？

Can you come up with an algorithm that is better than the algorithms that I came up with or that anybody else came up with?

你去把所有已发表的研究都翻一遍，整合起来，然后试着搞出点新东西——可它就是做不到。

You and go ahead and, like, look at all the published work and synthesize that and then try to come up with something novel, and it it's not able to do it.

哪怕我给它再多时间，它还是做不到。

And I can give it a lot of time, it's it's still not able to do it.

也许如果我给它一些引导，稍微限制一下范围，它最终能想出更好的方案，但这需要大量工作，远不是简单说一句‘你给我弄个更好的算法’就行的。

Now it's possible that if I scaffolded something and, like, kind of constrained it a bit more, that maybe it could eventually come up with something better, but it would take a lot of it's not just as simple as saying, okay.

你给我设计一个更好的算法。

Please come up with a better algorithm.

你觉得这个问题该怎么解决？

And how do you think that that gets improved?

我注意到，每次模型更新，它在这类任务上的表现确实都在进步。

What I've seen is with every model release cycle, it does get better at this sort of thing.

在我看来，它还是不够好，但比以前强多了。

It's still it's still bad in my opinion, but it it's not as bad as it used to be.

我不觉得奇怪，总有一天，编程和数学也会出现这样的转折点——突然之间，它变得足够实用了。

And I wouldn't be surprised if at some point same thing with coding, same thing with math, where there's just, like, this inflection point where suddenly it's actually good enough to be useful.

我觉得，科研领域也可能迎来类似的转折点。

I wouldn't be surprised if we encounter that point for research tastes as well.

说到这个，你平时是怎么看待RSI的？

Even that, what do you like, what is your framing of RSI today?

你认为我们应该怎么理解它？

Like, how should we think about it?

这些模型确实大大提升了研究人员在实验室里的工作效率。

The models are definitely accelerating what researchers can do inside the labs.

但我觉得，它们只加速了某些方面，而不是所有方面。

But I think they are accelerating some things and not other things.

现在的情况是，就算某些环节快了一百倍，还是会卡在那些没快这么多的地方。

And currently, we're at the point where, okay, if something goes a 100 x faster, you get bottlenecked by the things that don't go a 100 x faster.

随着时间推移，拖后腿的环节会越来越少，我觉得这方面会慢慢迎来一个稳步提升的阶段。

Over time, the things that we're getting bottlenecked on are going to shrink, and and there will be, I think, a kind of a gradual takeoff in that respect.

但眼下更重要的是转变研究者的工作方式，而不是彻底取代他们。

But it's more about transforming right now, it's more about transforming what researchers do rather than fully replacing the researchers.

这么说的话，你并不认为我们马上就要迎来一场极速的飞跃了？

So that actually implies that you don't think we're close to a very fast takeoff right now.

所谓的极速飞跃，其实要看怎么定义。

I think fast takeoff is relative.

现在进展确实很快，但我总觉得有一种假设，认为模型可能会一夜之间找到某种突破，让自己变得更聪明，然后立刻引发更多突破，持续自我升级。

Things are moving very fast, but I think there is this hypothesis that you could have basically an overnight intelligence explosion where the models discover some kind of breakthrough to make themselves smarter, and then that leads to more breakthroughs that make themselves even smarter immediately.

结果就是，模型在极短的时间内，全面超越人类水平。

And you have, basically, in an instance, the model's just, you know, becoming very superhuman across the board in in moments.

但我不觉得我们会走向那样的未来，主要因为模型要达到最高智能，极度依赖大规模的计算和数据。

And I don't think we're headed to that world largely because of the fact that the models rely so much on large scale test on compute in order to achieve their greatest intelligence.