第17集——当人工智能擅长数学后，接下来会发生什么？

大家好，我是安德鲁·梅恩，欢迎收听这个OpenAI播客。

Hello, I'm Andrew Mayne, this is the OpenAI podcast.

今天我们的嘉宾是两位研究员，塞巴斯蒂安·布贝克和柳镇。

Today, our guests are researchers, Sebastian Bubeck and Ernest Ryu.

我们会聊聊数学——它怎么从几乎被嘲笑的东西，一路飙升到奥赛水平，以及为什么想实现通用人工智能，数学是绕不开的。

And we're going to talk about math, how it went from almost laughable to Olympiad level and why you need math to reach AGI.

过去几年的进展，

The progress of the

简直堪称奇迹。

last few years has been nothing short of miraculous.

我们很快就能让大模型解决需要五十多页推理过程的问题。

We will be able to have LLMs be able to solve problems that require more than 50 pages of thinking.

数学只是

Mathematics was just the

过去四年里，检验模型进步的完美标尺。

perfect benchmark to see the model making progress during the last four years.

Sebastian，Ernest，我很想多了解你们两位。

Sebastian, Ernest, I'd love to know more about you.

你们能说说自己的工作是怎样的吗？

So how would you explain your roles?

当然可以。

Yeah, sure.

我搞数学已经快二十年了。

So I have been working in mathematics for almost twenty years now.

以前主要做优化和机器学习相关的工作。

I used to work in optimization and series of machine learning.

在去微软之前，我在普林斯顿当过几年教授，现在是OpenAI的研究员。

I was a professor at Princeton for a few years before moving to Microsoft, and now I'm a researcher at OpenAI.

最近几年，我一直在研究AI怎么帮数学家解决问题，也想看看我们到底在用AI攻克难题上走了多远。

And in the last few years, I've been really trying to understand how AI can help mathematics and to really evaluate the progress that we're making in terms of solving difficult math problems with AI.

Ernest，你呢？

Ernest, how about you?

是的。

Yeah.

我最近刚加入OpenAI做研究员，之前一直做应用数学，专攻优化和机器学习理论。

So I recently joined OpenAI as a researcher, but before that I was an applied mathematician working on optimization and machine learning theory.

我之前在加州大学洛杉矶分校数学系当数学教授。

And in my previous job, I worked as a professor of mathematics at the UCLA Math Department.

很多人觉得这些模型根本不擅长数学，毕竟它们叫语言模型嘛。

So I think a lot of people have this perception that these models aren't good at math, literally called language models.

那现在情况是怎么变的？

And how has that changed?

到底发生了什么？

What's gone on?

对，我觉得过去几年的进展简直堪称奇迹。

Yeah, I think, you know, the progress of the last few years has been nothing short of miraculous.

别忘了，两年前我们连能推理的模型都没有，更别提能证明复杂数学定理的模型了。

It's important to remember that two years ago, we didn't even have reasoning models, let alone models that could prove, you know, difficult mathematical theorems.

现在两年过去了，这些模型已经能帮菲尔兹奖得主处理日常研究工作了。

Today, two years later, the models, they are able to help fields medalists in their day to day work.

所以这个进步真的太惊人了。

So really the jump is just simply astounding.

如果让我再补充一点，很重要的一点是，所有人都被这个进展惊到了，包括我们自己。

And maybe if I can build a little bit more on that, something which is important to understand is that everybody has been surprised by this progress, including us.

我跟你们讲个事儿，一年半前，我参加一个学术会议的研讨会，跟其他数学家们讨论了一个问题：扩大大语言模型的规模，能不能帮我们解决重大未解难题。

So to tell you a story, a year and a half ago, I was at a workshop at a conference with other fellow mathematicians, and there was a debate that I participated in on whether LLMs, scaling LLMs will help us resolve major open problems.

那时候是场辩论，一年半前，现场意见严重分裂。

So this was a debate, you know, a year and a half ago, and the room was very divided.

开场做了个投票，我记得有80%的人觉得这根本不可能。

In fact, they did a poll at the beginning and I think it was like 80% said no, impossible that this would happen.

结果辩论进行到一半，到了最后，支持和反对的差不多各占一半。

So then the debate unfolded and, you know, by the end of the debate, it was more like fiftyfifty.

也就是说，就这一个小时，大家的看法就变了这么多。

So, you know, pretty good progress during that hour.

现在回头看，当时的想法简直大错特错——才过了八个月，模型就开始

This obviously was just so wrong in hindsight, like just mere eight months later, the model was starting to

能做研究级别的数学了。

be able to do research level mathematics.

你是什么时候意识到AI和数学之间有这么棒的交集的？

What was the breakthrough moment for you realizing that there was a really good intersection between AI and mathematics?

到了2025年，大新闻是ChatGPT在国际数学奥林匹克竞赛中表现达到了顶尖人类水平，拿了金牌。

So '25, the big news was ChatGPT was able to achieve a top human level performance at the International Math Olympiad, a gold medal performance.

这消息太震撼了，说明至少在竞赛数学这块，模型已经能跟最顶尖的高中生媲美了。

So that was amazing news, that demonstrated that, well, at least for the competition level mathematics, the models are very highly capable, only on par with the top human high school contestants.

但竞赛题都是套路题。

Competition problems are canned problems.

解法都比较短，因为得在几小时内做完，而且也不是新问题——早有人想出来并给出标准答案了。

They have relatively short solutions because they are meant to be solved within a few hours, and they're not novel because somebody came up with it and there's a solution.

所以这跟研究型数学完全不是一回事。

So it's not research level math.

所以我很好奇，很多人也都在问，ChatGPT 能不能做研究级别的数学？

So then I got curious, and a lot of people got curious, can ChatGPT do research level mathematics?

网上吵得挺厉害，我就想，不如我自己动手试试看我的问题。

And there was a lot of debate online, And then I thought to myself, I should try it on my own problems.

反正我也是搞数学的，不如亲自试试，自己下结论，别光听别人怎么说。

Maybe I'll try it for myself and make up my own minds as opposed to, you know, listening to what other people say because I'm a mathematician myself.

我就挑了一个优化理论里的经典开放问题，这正是我平时研究的领域。

So I took a classical open problem in optimization theory, which is a branch of applied mathematics that I work in.

具体来说，有个著名的算法叫 Nesterov 加速梯度法。

And the question specifically is there's a famous algorithm called the Nestrov accelerated gradient method.

这个算法是收敛的呢？还是说在某些极端情况下，它会发散？

And does this have this convergent behavior, or is it possible that in certain bad cases, can there be a certain divergent behavior?

这个问题一直没定论——大家知道大多数情况下算法表现很好，是收敛的，但真有没有那种特别差的情况呢？没人说得准。

This question was generally open in the sense that people know that in most cases the algorithm behaves well, it's convergent, but people really did not know, like, is there a bad instance?

在最坏的情况下，它会不会真的发散？

Does it, in the worst case, could it diverge?

结果答案是肯定的。

The answer turned out to be yes.

我是怎么发现的，我记得特别清楚。

And the way I discovered it is I remember it distinctly.

我儿子的睡觉时间是晚上八点，之后我就尽量不熬到半夜。

So my bedtime for my son is 8PM, and then I try not to stay awake after midnight.

所以每天晚上我大概有四个小时的空闲时间，可以专心做点事。

So I had four hours of usually evening hours to myself if I want to focus on something.

于是我决定，好，我就花几天时间好好研究一下这个问题。

So I decided, okay, I'm going to spend a few days working on this.

接下来三天，总共十二个小时，我一直跟ChatGPT在讨论这个问题。

So over the course of three days, so that's twelve hours total, I interacted with ChatGPT on this question.

这可不是我丢个问题进去，它就直接给我答案那么简单。

It wasn't as simple as me just putting in the prompt and getting a solution.

我扮演的是验证者的角色。

I played the role of the verifier.

只要模型出错，我就马上纠正它。

I told whenever the model made a mistake, I corrected it.

我还试着把对话引向我觉得更有新意的方向。

I also tried to point the conversation into approaches that I felt were novel.

过了一阵子，它给出了一个证明，我仔细核对了。

And after a while, there was a proof and I checked it.

我还让ChatGPT自己再检查了一遍，结果没错。

I also asked ChatGPT to double check it and it was correct.

就这样，这个困扰了数学界四十二年的难题被解开了。

And that's how this forty two year old open problem got resolved.

拿到答案后，我在想：怎么才能让这个发现变得最有意思、最有趣呢？

And once I got this solution, I thought to myself, what would be the most fun way for me to publicize this?

毕竟，我完全可以写篇论文，但那太没劲了。

Because I could just write a paper and that would be less fun.

所以我决定，干脆去Twitter上聊聊这件事。

So I decided, let me go to Twitter and talk about this.

挺危险的，不过值得一试。

Dangerous, but yeah.

但我真的玩得很开心。

But well, I had a lot of fun.

我觉得这可能是AI首次真正解决了一个数学界的长期开放难题。

It was, I think, one of the earliest instances of a genuinely open problem, mathematical open problem being solved by AI.

没错，我全身心投入了，特别过瘾。

And yeah, I mean, ate it up and it was a lot of fun.

你提到这点挺有意思，我们确实常看到有人宣称自己发现了什么酷炫或新颖的东西。

It it is interesting as you brought that up that we we've seen sometimes people said, I found something cool or novel.

结果有时候被扒得体无完肤。

And then sometimes it gets torn apart.

有时候却站得住脚。

Sometimes it stands up.

发到社交媒体上确实有点吓人，但听上去，我们确实需要这样的反馈循环。

And going into social media can be kinda scary, but it sounds like we do need these kind of feedback cycles.

我觉得对我们很多人来说，难点在于听到一些术语，比如‘国际数学奥林匹克’，然后我们就在想：这到底是个什么级别的问题？

I think part of the challenge for a lot of us is we hear terms, you know, we hear like the International Math Olympiad and we're trying to figure out like, okay, what does that mean from like a scale of a problem?

我知道加减乘除这些，这些我都懂。

You know, I can understand addition, subtraction, multiplication.

你能举个例子吗？比如从最早的ChatGPT开始，它那时候只是勉强能用，后来能做数学题、会用工具，再后来，模型居然能隐隐约约自己理解这些了。

Could you give me an example of understanding like where we went from, from like, you know, first ChatGPT, which could kind of sort of use it, then it could do math or could use a tool, but then the model sort of implicitly understanding that.

那时候才刚出来，大概是2023年初吧。

When you know, just entered the scene in early twenty three.

我开始去测试，特别好奇这模型在普通数学题上表现怎么样。

I started testing, I was very curious about how the model would perform fair on common math problems.

这些题包括高中水平的数学题，也包括日常生活中那种有点数学味儿的问题。

So these would include math problems that you would see in the high school level, but also day to day math ish problems.

比如，想象一下咱们仨一起去露营，我付了一笔，你付了一笔，Andrew你付了另一笔，最后咱们想平摊费用，把账清一清。

So for example, imagine a scenario where the three of us went camping together and then I paid for this, I paid for this, and then Andrew, you paid for whatever, and then we might want to clear the ledger out and we want to split things evenly at the end.

ChatGPT能帮我们算清楚这笔账吗？

Can ChatGPT do the calculations for us?

如果有十七项开销，这事儿就有点麻烦了。

And this is moderately complicated if you have like 17 items that we've purchased.

23年到24年，还有25年初，我记得那时候模型还做不到这个。

In 2324, and also in early twenty five, I remember, the models couldn't do this.

再比如，我在韩国，塞布在巴黎，安德鲁你在美国加州，咱们想约个Zoom会议。

Another example would be, I'm in, let's say, Korea, Seb's in Paris, Andrew, you're in California, and want to set up a Zoom meeting.

啥时候开最合适？

What would be a good hour to do so?

同样是25年初，那时候模型还是搞不定这个。

Again, in early twenty five, the models couldn't do this.

但突然之间，情况就变了。

But then just suddenly things just changed.

我当时不在OpenAI，所以也不清楚你们具体做了啥，但模型突然就开始解IMO题了。

And I wasn't in OpenAI at the time, so I'm not at all I don't I'm not quite privy to what exactly you did, but suddenly the models started solving IMO problems.

而且后来，它甚至开始解决科研问题了。

And then furthermore, it started solving research problems.

我现在怎么衡量这个呢？如果你不是专业的数学家，不打算发现新的数学理论，而只是像物理学家或化学家那样，用一些比较复杂的数学，比如微分方程、微分几何之类的——你只是在用数学，不是在创造新数学——那ChatGPT完全能搞定你所有需要的数学计算。

And the way I sort of calibrate this right now is that unless you are a professional mathematician trying to discover new mathematics, if you are somebody who's, let's say, a physicist or a chemist who uses relatively complicated mathematics like differential equations, differential geometry, things like this, you're not inventing new math, then ChatGPT can do all of the math that you would need.

所以，任何从事STEM领域、需要用到高等数学的人，现在都可以让ChatGPT帮你把数学问题全解决了。

So any user of high level mathematics from STEM can now use ChatGPT to basically have their math taken care of.

当然，你还是得留个心眼，核对一下结果对不对，最好再跑个模拟验证一下。

You would want to exercise some degree of caution, check whether things are right, run simulations just to double check.

模型也会出错的。

The models can make mistakes.

但现在，只要你遇到的数学题，99%的人需要解决的，模型都能帮你搞定。

But now, math problem that you would want to solve, most people, for 99% of the population, the models can do it.

我当初做GPT-4发布的时候，就拿排程当过例子。

When I worked on the release of GPT-four, I used scheduling as one of those examples.

我能把三个人的时间安排进去，让系统自动找出合适的时间段，但再往深了推，就特别吃力了。

And I could put three people into a schedule and have it figure out time slots, but pushing it beyond that, that was really hard.

后来有什么变化吗？

Was there a change?

所以欧内斯特刚才提到，我们都注意到自己进步了。

So Ernest just talked about noticing all of us that got better.

现在我们知道，其中一个关键是工具的使用。

Now we know one thing was tool used.

你可以让模型用计算器，但模型本身也发生了别的变化。

You could let the model use a calculator, but something else happened with the models themselves.

回头看看我刚才说的那场争论，焦点其实是：光靠扩大大语言模型的规模，能不能推动数学研究取得突破。

So going back to the debate that I just told you about, like the framing was really about can scaling alone, scaling of LLMs alone, bring you to solving research breakthroughs in mathematics.

这种提法其实是错的。

And this is a wrong framing.

我们在OpenAI做的，是大量前沿研究。

What we do at OpenAI, we do a lot of research, innovative research.

这不光是把模型规模堆上去那么简单。

It's not just about scaling the model.

所以当你问‘到底发生了什么’，或者去年年中突然间模型能解数学题了，到底是怎么回事？——其实发生了好多事。

So when you ask what happened, or, you know, when you're asking what happened middle of last year, when suddenly the model were able to solve mass problems, well, a lot of things happen.

我们做了大量研究，所有这些方面都得同步推进。

We do a lot of research and all of this has to progress at the same time.

所以我没法指出是哪一个单一因素起的作用。

So I can't really point to a single element.

但模型自己确实做到了，根本没用那些工具。

But it was able to do it itself though, without the tools though.

对。

Yeah.

我觉得特别特别重要的是，要继续坚持欧内斯特说的那些进展，比如以前模型根本解决不了的调度问题。

So I think it's really, really important to, you know, just double down on what Ernest was saying about the progress and, you know, the scheduling problems that the model wasn't able to do back then.

两年前我就说过，那时候还没有具备推理能力的模型。

I said that two years ago, we didn't have reasoning models.

再往前推四年，也就是ChatGPT出现之前，你想想看。

Well, think about four years ago, four years ago, so this is pre ChatGPT.

我记得那时候谷歌推出过一个叫Minerva的数学模型。

And I remember Google came out with a mathematics model called Minerva at the time.

我直接从椅子上摔下来了。

And I fell from my chair.

我简直太震惊了。

I was so impressed.

我到底震惊啥了？

What was I impressed by?

那个模型只要给它平面上几个点的坐标，它就能画出一条穿过这些点的直线。

That the model, could give it the coordinates of points in the plane and it would give me a line that goes through those points.

我这么一说，你现在听来可能都觉得这有啥稀奇的？

Like when I say that, you know, now it's almost hard to understand what are you talking about?

现在当然谁都能做到这一点了。

Obviously a model can do that.

我觉得我们早就忘了技术进步得有多快。

So I think we have kind of forgotten how quickly things have happened.

现在呢，就像欧内斯特说的，除非你真想发明新的数学，不然模型的能力已经差不多到头了。

Now, yeah, you know, Ernest was saying that it's basically at the point where, unless you're trying to invent new mathematics, it's kind of at the right level already.

我觉得，就连发明新数学，现在也快能做到啦。

I would say we're already seeing glimmers that even to invent new mathematics, it's getting there.

不过你能说说吗？除了那些想开创数学新领域、或者搞新证明的人，这东西对其他人有什么影响？

Could you break down though, aside from somebody who's interested in developing new fields of mathematics or just making new proofs, what does this affect everything else?

这对科学整体会带来什么改变？

What is the impact of this going to be on science?

你做的其他那些工作，又会对什么产生影响？

What is the impact of the rest of what you're working on?

这到底为什么重要？别光说‘哇，它会算数学’就完事了。

Why is this really important and not just, oh, cool, it does math.

我觉得，‘哇，它会算数学’这部分，

So I think the, oh, cool, it does math part.

在我们开发这些模型的时候，其实是个很好的进度衡量标准。

What did matter as we were developing those models as a good way to benchmark the progress.

数学的好处在于，问题都特别清晰、没歧义，人人都清楚问题在问什么。

The nice thing about mathematics is that the question are very clear, non ambiguous, you know, everybody agrees on what the question is asking.

所以这是第一点。

So that's point number one.

第二点，你可以验证答案对不对。

Point number two, you can verify the answer.

模型给出答案后，大家都能 agreed，这答案是对还是错。

Once So the model can give an answer, everybody will agree, was it correct or was it not correct?

当然这事儿也没那么简单，因为到了研究层面，评估就没那么容易了，但在那之前，评价起来其实挺直接的。

Although you can put a pin on that because we will talk about, you know, in research level, it's not that simple anymore to evaluate, but before research level, it's very easy to evaluate.

所以过去四年，数学简直就是衡量模型进步的完美标尺。

So mathematics was just the perfect benchmark to see the model making progress during the last four years.

现在我们可以说，这方面差不多已经到头了，你可能会问：行吧，那然后呢？

Now we'd say we have kind of saturated that aspect and you can ask, okay, now, fine.

模型确实会做数学了。

The models do mathematics.

这一点我们已经明白了。

We have understood.

那下一步呢？

What about the next steps?

我觉得，模型要是能精通数学，对其他很多领域都会有帮助。

And for the next step, I would say that having our models be good at mathematics is going to be good for many, many other things.

我来解释一下为什么。

And let me explain why.

数学的一个关键特点就是，解决一个问题往往得长时间思考，可能是几天、几周，有时候甚至好几年。

A key feature of mathematics is that to resolve a problem, have to think for a long time, be it days, weeks, sometimes years.

这种长时间的思考，不仅要求你持续想很久，还得一直保持逻辑连贯。

So this long thinking, not only do you have to think for a long time, but you also have to think consistently for a long time.

只要你推理链条里有一处出错，整个论证就全垮了。

If at some point in your chain of reasoning, is a mistake, this will kill the entire argument.

后面再怎么对都没用。

It doesn't matter if everything after that is correct.

只要有一个地方出错，整个论证就彻底废了。

If there is one single failure point, everything, the entire argument is destroyed.

所以，这一点正是我们希望推理模型具备的——一旦出错，它们能自己纠正。

So this property makes it that this is what you want out of reasoning models, that if they make mistakes, they will be able to correct themselves.

我们期待，这种通过数学培养出来的能力，能迁移到其他领域，其实人类也正是这样。

So we are hoping that this property that they acquire through mathematics will generalize to other domain, which by the way, is exactly the same thing with human beings.

我们为什么要训练人类学数学呢？

Why do we train human beings in mathematics?

当然，数学这东西挺有意思的。

I mean, it's a very fun topic.

我个人超爱这个领域。

I love it.

我们以前是把它当职业来做的。

We did it professionally.

现在可能还保留一点，但说到底，我们为什么让人类学数学？

Maybe we still do some of it a little bit, but why do we train humans in mathematics?

原因就在这儿。

Exactly for the same reason.

它能让你养成这种特别严谨的逻辑思维。

It gives you this kind of very logical thinking.

我们是不是得想点新办法，来跟大家讲这些发现？

Do we need to think about new ways to talk about these discoveries?

是的。

Yeah.

我个人觉得，教育研究圈了解最新进展，也是我职责的一部分。毕竟我既有数学背景，现在又在AI前沿工作，算是双料背景吧。

So I personally view it a little bit as part of my role to try to educate the research community about the recent advances, because I have this, you know, dual background of both being a former mathematician and now working on the frontier of AI.

确实，像推特这样的社交媒体，特别适合解释这些进展，毕竟这领域发展得太快了。

And indeed like Twitter and social media is a great place to try to explain what is the progress, in particular, because this progress is so fast.

比如说，咱们可以聊聊埃爾德什问题，还有围绕它闹出的一些争议。

So, you know, for example, maybe we can talk a little bit about the Erdos problems, you know, and some of the controversies that happened around that.

最早有个例子。

So there was a first example.

先是埃爾德什那个问题，后来又解决了几个别的难题。

So there was first, you know, Ernest example, and then there were a few other problems that were solved.

你也可以介绍一下保罗·埃尔德什，大家肯定都想了解他是谁，为什么他的问题这么有意思。

You would explain Paul Erdos though, too, just so I think people would love to know who he is and why his problems are sort of interesting.

当然了。

Yeah, of course.

保罗·埃尔德什是上个世纪最多产的数学家之一。

So Paul Erdos is one of the most prolific mathematician of the last century.

他写了大概一千五百篇论文。

He has written, I think, 1,500 research paper.

他是个特立独行的人。

He was a very iconoclastic figure.

他没有自己的房子或公寓。

You know, he didn't have a house or an apartment.

他总是从一所大学跑到另一所，到处找人合作。

He was just traveling from one university to the next, trying to find new collaborators.

每次到一个地方，他就不停地提问题，特别特别擅长问问题。

Every time he would go to a place and basically ask questions, he was very, very, very gifted at asking questions.

他提的问题也不是个个都有趣。

Not all the questions that he asked were interesting.

我得先说清楚这一点，但不得不说，他确实特别有成效，整个数学圈跟他合写过很多论文。

Let me just say that right away, but still it was very productive and, you know, the research community wrote a lot of papers with him.

甚至还有一个叫‘厄多斯数’的概念，就是你跟厄多斯通过合作者的链条，隔了多远？

There is even this concept of an Erdos number, is, you know, how far away are you in the chain of collaborators from having also a paper with Erdos?

我的厄多斯数是二，我跟一个跟厄多斯合写过论文的人一起发过文章，对，我特自豪。

My Erdos number is two, I've co authored a paper with someone who co authored with Erdos, you know, yeah, I'm pretty happy about that.

我的是三。

My number is three.

有个笑话是说，你要是跟他坐一趟火车，到站的时候，说不定就跟他一起写完一篇论文，名字也上了。

The joke was, you know, you could be on a train ride with him and then by the end of the train ride, you'd maybe work on a paper with him and have your name.

没错。

Absolutely.

没错。

Absolutely.

我觉得二和三这个差距，其实反映的是咱们俩的年龄差。

I think the two versus three basically says something about our respective age.

说白了就是这个意思。

Essentially what it says.

所以呢，Erdos 留下了一大堆问题。

So anyway, so Erdos has, you know, all of this problem.

有个叫托马斯·布卢姆的人搞了个特别棒的网站，专门追踪那些还没解决的 Erdos 问题。

And there is a very nice website by Thomas Bloom who is keeping track of all the Erdos problems that are still open.

那个网站上大概有一千个问题左右吧。

So I think there is like a thousand problem or something like that on that website.

托马斯本人也花了不少功夫去梳理，他是组合数学领域的专家。

And Thomas himself has done the work of trying to find, you know, he's an expert in combinatorics.

所以他能判断：这个还没解出来。

So he can kind of say, okay, this is open.

这个嘛，已经解决了。

This is, you know, resolved.

每个问题的状况都不太一样，挺复杂的。

This has some complicated status, you know, for every problem.

当然，他也不可能知道所有问题的答案。

Of course, he doesn't necessarily know the answer to all of them.

所以，如果一个问题是标为‘未解决’的，不等于没人知道怎么解，但这个网站特别互动，大家都可以去留言，评论每个问题，说明有没有人解决了，等等。

So if there is a paper which is marked open, it is not necessarily true that nobody knows how to solve it, but it is also a very interactive website where people can go on it and, you know, add comments to every problem and explain whether there is a solution, etcetera.

这真是个特别活跃、超棒的网站。

So it's a very dynamic, great website.

自从GPT能解研究型数学题之后，我们就觉得这简直就是个宝藏，正好拿来测试我们的模型。

So of course, once we started to have GPT be able to solve research mass problem, this sounded like a treasure trove of problem to try our models on.

我们试了几个，结果出乎意料——模型居然答出了几个标着‘未解决’的问题。

And we tried a couple and to our great surprise, the model came back with answers to some of them that were marked as open.

我们当时可兴奋了。

So we got really excited about this.

我最早在推特上提到的那个问题，具体啥时候我记不清了，大概是去年十月左右吧。

The first one, you know, that I tweeted about, I don't remember when it was, maybe it was in October or something like that last year.

那是一次深度文献检索的结果。

It was deep literature search result.

让我解释一下这是什么意思。

Let me explain what that means.

意思是，GPT 做了一次大规模的文献搜索，扫描了成千上万篇论文，结果在某个毫不相关的领域里找到了问题的答案。

It means that what GPT did is that it did a vast literature search trying to scan, you know, thousands of papers and it found in some unrelated field, the answer to the question.

但你得明白，那个领域的人根本没说自己在解 Erdos 问题。

Now it's really important to understand that it's not like in that, you know, unrelated field, the person said, okay, I'm solving an Erdos problem.

他们用的是完全不同的表达方式。

It was written in a completely different language.

数学体系也不一样。

It was different mathematics.

得花功夫把这两边连起来，而 GPT 就做到了。

You have to do work to connect the two pieces and GPT did that.

这真的挺了不起的。

So that was kind of amazing.

而且这完全是临时拼凑的，就是我们直接在ChatGPT界面里手动试了试。

And this was very ad hoc, like, you know, we just tried by hand basically in the ChatGPT interface.

看到这结果后，我们团队的马克·塞尔克也决定系统性地把所有问题都跑一遍。

Once we saw that, Mark Selke, who is, you know, in our team also decided to have a more systematic approach of trying all of the problems.

他这么一试，模型居然给出了10个埃尔德什问题的解答。

And he tried that and the model came back with solutions to 10 Erdos problems.

那时候大家还在激烈争论，这些模型到底能不能突破现有水平，甚至创造出新的数学。

And this was, have to remember at that point, there was still, I think, very dynamic discussion about whether, you know, those models could go beyond the state of the art and invent new mathematics.

我看到这结果特别兴奋，就发了条推特，结果这条推特后来成了‘黑历史’——很多人误以为我说模型彻底解决了10个超难的开放性问题，还给出了前所未有的全新解法，但其实根本不是这么回事。

So I got very excited about this result and I tweeted about it and, you know, it's kind of an infamous tweet because people misunderstood it as kind of saying it really found the solution to 10 open problems that are very hard and the solution is completely new and did not exist in the literature, but that's not what happened.

这其实和前面说的那个‘深度文献检索’是同一个原理。

It was connected of course, to the previous case where it is a deep literature search.

当时我们和谷歌、还有恩德米斯那边，也讨论过该怎么准确描述这类成果。

So there was some, you know, with, with Google about, you know, with endemis about whether, you know, this is the right way to talk about such results.

但最精彩的是，几个月后事情有了转折。

But now the punchline is kind of amazing, which is a few months later.

所以我之前说的那十个解，其实都是文献里已经有的。

So again, I said 10 solutions to open problems, and these were solutions in the literature.

那接下来的问题就是：能不能找到文献里根本没有的解？

And then the question is, can you find solutions that are not in the literature?

到现在，我们已经找到了十多个完全原创的解法，完全可以发到组合数学顶刊上，这些解法有的来自ChatGPT，有的来自我们自家的模型。

By now we have more than 10 actual solutions that are completely new, that are publishable in top journal in combinatorics, completely obtained by, you know, some by ChatGPT and some by our internal models.

这短短几个月的变化，真能说明加速有多快。

So just within, again, really speaks to the acceleration.

几个月前，说能解决十个埃尔德什问题，听起来简直荒唐；现在呢？真就发生了，而且进展还在加速。

In the span of just a few months, we went to, it's kind of a ridiculous statement to say that there would be 10 solutions to Erdos problems, to it's actually happening for real and it's accelerating.

是的。

Yeah.

有意思的是，第一步其实是让模型能做非常扎实的文献调研。

It's interesting because it seems like that, you know, step one is have models be able to do really good literature research.

已经有不少重要论文和奖项颁给了那些只靠文献检索，就发现某个问题其实早有解法，还能迁移到其他地方的人。

And there have been major papers and awards done given to people who've just done literature searches and found the solution was solved here and it actually applies elsewhere.

所以这挺厉害的，它先从这一步开始，现在居然能做出原创的东西了。

So it's neat that it does that as a first step, now that it's actually doing original.

说实话，我特别喜欢AI研究的一点是，它逼着我们去直面一些大问题——关于智能、关于研究和进步，还有我们到底怎么发现新东西。

I mean, you know, the one thing that I really like about AI research is that it forces us to confront big questions about intelligence and about, you know, research and progress, and how do we discover new things.

尤其是现在这个疑问：我们看到的科学进展，是不是只是把各种现成的碎片拼在一起，再稍微推理一下？

In particular, there is this question of whether the progress that we're seeing in science, is it just putting together different pieces and, you know, doing a little bit of reasoning on top of it?

还是说，真有那种灵光一现的洞见？

Or are there those brilliant sparks of insight?

大家自然都会提到爱因斯坦的相对论。

Everybody of course points to Einstein's, you know, relativity.

但说实话，我都不确定那算不算真正的灵光一现。

I'm not even sure that really counts, to be honest.

我觉得现在还说不准，这种靠重组和一点点思考的方式，到底能不能无限扩展人类的知识，还是说，你非得靠那种只有人类才有的天才灵感不可？

So I think the jury is still out on whether this process of just recombination and a little bit of thinking, whether you can kind of increase, you know, human knowledge with no limit, or do you really need the sparks of genius that would be somehow only human?

就连爱因斯坦自己也承认，那个类比和可视化的方法，其实不是他想出来的，我忘了是谁了。

Well, even he credited, I forgot who it was, but who came up with analogy, the visualization method.

你知道吗，他说这主意不是他的。

You know, he said it wasn't his.

我们指出了真正的人是谁，而他则把这想法又推进了一大步，这很明显。

We pointed out who did it and he kind of took it to the next step further, obviously.

我觉得我们有时候太爱听这种小故事了，其实事情远比这复杂得多。

And I think that we sometimes, we love these tiny little stories when it's a lot more complex than that.

是的，完全对。

Yeah, Absolutely.

如果我们有了更强大的数学工具和AI，这对科学家整体意味着什么？

What will it mean for scientists in general if we have better mathematical tools and AI?

这会影响到其他方面吗？

How does it affect other things?

比如生物学、材料科学？

Biology, material science?

对。

Yeah.

所以再说一遍，这对其他科学领域会有什么影响？

So again, how how it affects the the rest of science?

关键是，我觉得每个人都得明白这一点。

Well, the point is, I think it's really important for everybody to understand.

我们并不是在为数学搞什么特别玄乎的东西。

It's not like we're doing something very, very special for mathematics.

我们的方法和训练方式其实非常通用。

Our techniques, our training techniques are very general.

这些方法用在方方面面。

They are applied to everything.

所以我们预期，数学领域会看到更多进展。

So our expectation is that we are seeing more progress in mathematics.

原因之一是，数学特别容易做评估。

Well, one reason is because it's very easy to benchmark.

进步一目了然，但我们完全相信，这种变化会发生在所有科学领域。

It's very easy to see that progress, but we have full expectation that this is going to happen in all sciences.

这可不只是限于数学领域。

It's not going to be limited to mathematics.

看起来，如果这个成立，那个也成立，能一路推导出一长串这样的逻辑链条，这本事在别的地方也能大有用武之地。

It seems like something that's very good at going, if this is true, and then this is true, and going through a long sequence of those kinds of statements has a lot of applications elsewhere.

我们听过‘自动研究员’这个词。

We've heard the term auto researcher.

你能展开说说这是啥意思吗？

Do you want unpack that a bit?

我们现在的做法，其实就是欧内斯特说的那样——一种互动。

Now, the way we work is exactly what Ernest described, which is really an interaction.

有点像教授和学生之间的互动：ChatGPT当学生，教授先抛个问题，学生回来回应，然后两人再聊聊。

It's kind of a professor student interaction where ChatGPT is a student and the professor is kind of, you know, giving a first problem and the student comes back and then they talk a little bit.

学生再走开一周，回来再汇报进展。

The student goes away for another week, comes back.

当然，这么一来，整个周期被大大压缩了。

One point of course, is that it's compressing those timelines greatly.

最后呢，这个难题我们花了十二个小时就解决了。

In the end of story, you know, of solving this problem in twelve hours.

我是说，要是没有ChatGPT，你觉得得花多久？

I mean, I don't know, without ChatGPT, how long would it have taken you?

我之前没用AI的时候，光是碰壁就花了四十多个小时，搞不好得一个月吧。

Well, I have spent more than forty hours failing without AI and I don't know, maybe a month.

对啊。

Right.

是的。

Yeah.

没错。

Exactly.

所以说，时间真的被大大压缩了。

So, you know, there is this thing of just compressing timelines.

现在说到自动化研究员，那又是另一种设想——模型或者一组模型能自己持续工作很长一段时间。

Now, when we talk about the automated researcher, that's a slightly different vision where the model or maybe a collection of model would work autonomously for a long period of time.

如果我们想突破现在的水平，这几乎是必须的。

This is kind of needed if we want to go beyond the current level.

现在的互动模式，比如师生之间隔一周才碰一次，靠这种节奏根本没法做出真正的突破，解决那些长期悬而未决的研究难题，或者在生物学这种需要频繁跟实验台打交道、做各种实验的领域取得进展。

The current level of interaction, you know, the professor student interaction, where the student comes back after a week, it's going to be very hard with that mode of interaction to do real breakthroughs, to solve actually long standing research problems or to make progress in, you know, very difficult fields in biology where you need to interact, you know, with the wet lab and do all kinds of experiments.

所以，一旦你想追求真正的突破，就得拉长时间线，这时候自动化研究员就派上用场了。

So once you want to go towards the real breakthrough, we will need to work over longer timelines and this is where the automated researcher comes in.

也许我换种方式说清楚点。

Maybe let me say it in a slightly different way.

我特别认同一个概念，叫‘AGI时间’。

One concept that I'm a big fan of is this concept of AGI time.

你可以有AGI的秒、分、小时、天，等等。

So you can have AGI seconds, minutes, hours, days, and so on.

意思就是，AI能模拟人类的思考，但能持续多久呢？

So that really means you have an AI and for like, it can mimic human thinking, but for how long?

就像欧内斯特说的，两年前，模型可能也就相当于一个高中生，对着一个问题想几分钟。

So as Ernest was saying, you know, two years ago, maybe models were mimicking, you know, a high school student who thinks for a few minutes on a problem.

现在我们能模拟一个能连续思考几小时、甚至几天的研究员。

Now we can mimic a researcher who can think for hours, maybe a few days.

我们真正想往前走——这四年来，进展一直很稳定，从几秒钟一路走到几分钟、几小时、几天，现在差不多到了几天到一周的水平。

We really want to go towards, and this progress has been going on for now, you know, very consistently for four years where we went literally from seconds to minutes, to hours, to days, And now we are roughly at days slash one week.

我们目标是往几周迈进，甚至几个月。

We want to go to weeks, if not months.

这还是开放性研究。

This is open research.

说实话，我觉得全球没人确切知道该怎么做到，但说到底，我们一直在做大量研究和创新。等所有东西拼起来，你就会看到这条清晰的进步轨迹——AGI思考时长在持续突破，而这正是自动化研究员的发展方向。

You know, I don't think anyone on the planet knows exactly how to do it, but this goes back to, we are doing a lot of research, a lot of innovation, I think once everything will be put together, we're just seeing this arc of progress where we keep making progress in AGI time, but this is the direction of the automated researcher.

我接触的那些数学家，他们用AI的方式就是打开ChatGPT，然后在里面跟它聊。

So the people, the other mathematicians that I talk to, their mode of using AI is they open up ChatGPT and then they talk to ChatGPT within that context window.

你可以开多个会话，但每个会话的上下文长度都是有限的。

And you can have multiple sessions, but each session has a finite context length.

大概相当于五十页数学论文的量。

And roughly on the order of 50 pages of a math paper.

但这点上下文根本不够做出真正深刻的数学突破，因为很多数学论文都超过五十页。

And that's not long enough to make true deep math groundbreaking breakthroughs because a lot of math papers are longer than 50 pages.

而且，写一篇十页或三十页论文背后的人类思考过程，通常要比最终成果长好几个数量级。

And also, the human thought that went into to produce, let's say, a 10 or 30 page paper is usually, well, much orders of magnitude longer than the final output.

所以，上下文窗口太短是个硬伤。

So there's a limitation with the limited context window.

但用过代码工具的人应该知道，用这些工具其实可以进行超长的连续工作。

Users, people But who've used codecs will know that you can actually have very long work sessions with codecs.

你可以不断给出指令，告诉它你想写什么样的代码，而你正在开发的代码库——在数学里，这就相当于你写下的那些笔记。

So you just keep giving instructions as to what kind of code you want to write, and then the code itself that you're working on, the repository of your code, which in the math sense, the analogy would be that would be analogous to like math notes that you write down.

这些笔记可以长得不得了。

That can be very, very, very long.

但Codex处理这种长内容还挺在行的。

But Codex is pretty good at dealing with that.

它偶尔会把对话压缩一下，但整体上能变成一个超强的智能体，能在超大的代码库和超长的对话历史里，完成特别复杂的任务。

Once in a while, compactifies conversations and it has its way of becoming this really amazing agent that can do really complex jobs over huge repositories of code over a really long context of conversation.

我觉得数学研究也会迎来同样的变化。

And this, I believe, is going to happen with mathematics research as well.

到时候，大模型就能解决那些需要远超五十页思考深度的问题了。

So we will be able to have LLMs be able to solve problems that are longer than just, you know, that require more than 50 pages of thinking.

这本来就是人类的做法。

And that's what humans do.

真正的数学家就是这么干的。

That's what human mathematicians do.

人们会为一个问题苦思一整天，然后把想法整理成笔记。

People think for a day on a certain problem, and then we kind of summarize our ideas and then put it into notes.

第二天，或者下一周，再回来继续琢磨。

Next day or the next week, we come back to it.

几个月下来，虽然思考的时间很长，但都被有条理地浓缩、整理成了可掌控的形式。

Then over several months, we've thought for so long, but it's sort of summarized, it's sort of organized in a way that becomes manageable.

最后，一份三十页的论文就诞生了，它总结了数月乃至数年的思考。

And in the end, the final output becomes a 30 page paper summarizing the thoughts over many, many months or even years.

所以，我觉得这事肯定会成。

So, yeah, I think that's going to happen.

上周末我搞了个特别搞笑的题目，你们听了肯定觉得好笑，就是想用一个特别小的模型来做数学。

I was working on a very, very laughable problem to you guys over the weekend and using an LLM to try to do it, to figure out like how to use a really small LLM to do math.

做着做着，我需要个评估标准。

In the middle of it, I needed a benchmark.

结果我发现了 EasyMath，这是专门给小型模型用的基准测试。

And I came across easy math, which is a benchmark for small LLMs.

问题就是一篇相关的论文。

And the problem is just a paper on it.

其实没什么现成的数据。

There wasn't really like a lot of data.

我就在 Codex 里琢磨，能不能自己搞个基准，直接生成所需的数据？

And I just, in the middle of Codex ago, can you create my own benchmark here and just generate the data for that?

对。

Yeah.

五分钟后我就搞定了。

And five minutes later I had it.

这让我觉得特别神奇，因为我当时正忙着开发一个工具，结果它突然就派上用场了。

And that was magical to me because I'm in the middle of working on the tool that would have involved me all of a sudden.

我花了几个小时写了个生成器，把这类数据批量弄出来。

Okay, I got to spend a few hours to go do a generator, go produce this sort of stuff.

没错。

Absolutely.

它在后台默默运行。

And it runs in the background.

我真没法想象你们做那些正经科研问题是什么感觉。

I can't imagine what it's like for you guys doing grown up problems.

是的。

Yeah.

你刚才说的，其实正是我们发那篇论文时想做的——论文题目叫《用GPT-5加速科学的早期实验》。

I mean, what you described is really, you know, what we went after when we published the paper whose title was early experiments in science acceleration with GPT-five.

你经历的这种，简直就是真正的加速。

Like we, what you have experienced is literal acceleration.

以前这事儿可能得花你好几天才能搞定，谁知道呢。

Like this is something that would have taken you before, I don't know, maybe a few days of work or something.

我早就放弃了。

I would have given up.

是的。

Yeah.

对啊。

Yeah.

这其实是个特别棒的观点。

So that's actually a great point.

我肯定也早放弃了。

You know, I would have given up.

这真的让全世界的科学家都受益了。

This really enables scientists everywhere.

比如数学家现在能直接用代码了。

Like for example, mathematicians to be able to use code.

我们大多数朋友本来都不写代码，可现在突然间他们都有了代码助手。

Most of our friends, they don't code, you know, and now suddenly they have codecs.

他们现在能自己做所有实验，以前呢，还得到处找苦哈哈的研究生帮他们做。

They can do all the experiments that, you know, before they were trying to find the poor grad student to do the experiment for them.

现在这些实验对他们来说都变得特别简单了。

Now they can do all of these experiments very easily.

反过来，各个领域的科学家也能借助ChatGPT用上更高级的数学了。

The flip side is of course like that scientists in all the disciplines, they can also use more advanced mathematics now, thanks to ChatGPT.

我坐下来教了鲍勃·梅特卡夫怎么用Codecs跑R语言，他当时在做一个项目，R对他来说是新东西，想学一学。

I sat down with Bob Metcalf and showed him how to use Codecs to do R because he was working on a project and R was new to him and he wanted to learn that.

是啊。

Yeah.

帮一个脑子特别聪明的人省掉大把摸索的时间，直接把工具递给他，这感觉还挺棒的。

And that was kind of a fun experience to take somebody who's got a great mind and say, oh, here's instead of spending a lot of time having to figure this out, here's the tool for you.