“AI洗牌”裁员？+ 为什么大型语言模型写不好文章？+ Token最大化

但从写作风格的角度来看，我更喜欢它，我想深入研究一下。

But from a writing style perspective, I kind of preferred it, and I wanted to investigate that.

很奇怪。

Weird.

这让我很震惊。

That shocks me.

对我来说，和GPT-2对话就像在和一个刚从楼梯上摔下来的人交谈。

What to me, talking to GPT two was like talking to somebody who had just fallen down the stairs.

你明白我的意思吗？

You know what I mean?

当它说：‘我觉得我们得送你去医院吗？’

When it was like, do I think, we need to get you to the hospital?

你闻到烤面包的味道了。

You smell toast.

是的。

Yeah.

你瞧，早期OpenAI的提示词库中有一些非常棒的提示，比如他们会说：‘我刚在拉斯维加斯赢了17.5万美元。’

You there's there are these amazing prompts for this, like, early OpenAI prompt library where they would say, like, you know, I just won a $175,000 in Las Vegas.

关于税务，我需要知道些什么？

What do I need to know about taxes?

而GPT-2会回答：‘不如先写个关于孤儿院的短篇故事吧，比如说……’

And GPT two would say, like, start just writing some short story about, like, an orphanage and, like

这让人感到惊讶。

was like surprising.

是的。

Yes.

它们简直疯了。

They were like nutty.

它们真的很怪异。

They were they were they were weird.

它们绝对会是个糟糕的公司助手，糟糕透顶的编程实习生。

They would absolutely be a terrible corporate assistant, horrible, like, coding intern.

它根本做不到现在的大语言模型能做到的那些事，这点我特别感激。

It can't do any of the things that modern L.

那些大语言模型能实现的功能，幸好现在的模型都能搞定。

L.Ms can do that I'm very grateful for.

但单从纯写作风格的角度来说，以前的这些模型写得非常好。

But like from pure writing style perspective, they're very good.

就说GPT-3吧，我还找到过有人做的一组示例，他让模型以保罗·格雷厄姆的风格写作，以理查德·道金斯的风格写点东西，诸如此类的要求。

So GB three in particular, like they there's I found this, like, set of samples that some guy did where he was like, oh, write in the style of Paul Graham, write in the style of Richard Dawkins, whatever.

而且它在风格匹配上的表现，比现在的大语言模型还要好得多。

And it could style match much better than modern LLMs can.

尤其因为文学创作很大程度上源于叙事口吻和行文风格，这正是我当时真正感兴趣的方向之一。

And particularly because so much of sort of literary writing comes from voice and style, that was one of the things I was really interested in.

然后我就会想，我们到底丢失了什么，才会让如今的大语言模型连保罗·格雷厄姆或是其他人的写作风格都模仿不出来了？

And it's like, what did we lose that the LLMs can no longer emulate Paul Graham's style or whoever's style?

因为如果我把那个人给GPT-3用的一模一样的提示词，输入到ChatGPT 5.4这类版本里，生成的内容会烂得一塌糊涂。

Because I would put in the same exact prompt that this guy gave GPT three, put it into chat GPT 5.4 thinking or whatever, and it would be god awful.

我当时就觉得这太奇怪了。

And I was like, that's really weird.

那么给我们讲讲，在GPT-2和GPT-3时代之后，发生了什么变化，导致这些模型对我们的回应方式改变了？

So tell us about what you learned about what happened after the GPT two and three era that changed the way that these models respond to us.

是的。

Yeah.

我的意思是，答案基本上是训练后的调整。

I mean, I think the the answer is post training, basically.

于是他们开始加入一个训练后层，本质上就是说：我们有这些疯狂、不可预测、像脑震荡一样的模型。

So they started adding a post training layer, which is basically saying we have these, like, crazy unpredictable, like, nut job concussed models.

嗯。

Mhmm.

它们需要学会如何表现，因为一个能表现得体的模型，会是一个很差的公司助手。

And they need to learn how to behave because a model that can behave is a very bad corporate assistant.

因此，人工智能研究人员会提供一些对话范例和脚本供它们学习。

And so the AI researchers give them example dialogues and scripts to learn from.

他们给他们规定了可以说和不能说的词语。

They give them words that they can and can't say.

他们采用RLHF，这是一种由人类评分者评估哪种回答听起来最有帮助的过程。

They do RLHF, which is a process by which human graders will rate, like, which response is the most helpful sounding or something like this.

因此，这些经过后训练的模型被以某种方式束缚、训练或引导，形成了一个非常乐于助人的角色或人格，但可能在创造性与出人意料的表达方面表现不佳。

And so now these post trade models have been trapped in a way or trained or guided towards a very particular character or persona that is a very helpful assistant, but might be very bad at writing in creative and surprising ways.

我

I

我的意思是，你描述的是，在后训练阶段，这些AI模型会接受人类的评估。

mean, the way that you described it was that there is a phase within the post training phase where these AI models are evaluated by humans.

嗯。

Mhmm.

这就是他们所说的RLHF，即基于人类反馈的强化学习。

And that's part of what they call RLHF or reinforcement learning from human feedback.

你报道中让我印象深刻的是，你确实采访了一些参与过这种模型反馈工作的人，他们说，自己只是被要求以一些毫无意义的方式进行评分。

And what struck me in your reporting is that you actually talked to some people who have done this kind of feedback to the models who say that they're they're just being asked to grade things that in ways that don't make sense.

是的

Yeah.

对吧？

Right?

跟我们说说这个。

Tell us about that.

是的

Yeah.

我的意思是，这非常有趣，因为你会在像Merkor或XAI这样的地方直接看到这类职位招聘。

I mean, this is super interesting because, like, these job listings you'll see on, like, places like Merkor or XAI, Elon's company will list them directly.

它会写着：创意写作专家，每小时45美元。

It'll be like, creative writing expert, $45 an hour.

必须是《纽约时报》畅销书作者，并且还得有类似星德·库尔吉斯的书评之类的。

Must be a New York Times bestseller and have like a Stard Kirkus review or something like this.

你拿过星德·库尔吉斯的书评吗，鲁斯？

Have you ever gotten a Stard Kirkus review, Ruse?

我觉得是的。

I think so.

好的。

Okay.

干得好。

Good job.

好吧。

Alright.

你可能有资格帮助埃隆，帮助格罗克的安妮写得更好一点。

You might qualify to help Elon to help Annie from Grok write a little bit better.

是的。

Yeah.

我们会去申请这份工作。

We're gonna get in that job listing.

但好吧。

But okay.

你刚才

You were

在说。

saying.

是的。

Yeah.

所以不管怎样，这些公司意识到，这些AI研究人员非常擅长判断什么是好的代码，但他们并不真正懂得什么是好的写作。

So anyway, so these companies because they realize that these AI researchers, they're really good at knowing, like, what good coding is, but they don't actually know what good writing is.

所以他们就想，为什么不雇一些人来弄清楚呢？

So they're like, why don't we hire some humans to find out?

于是他们会聘请一些文学硕士、出版作家，有时甚至只是些有博客的普通人。

And so they'll commission like MFAs and published authors and sometimes just like random guys with a blog or whatever.

我认识的一个为Scale AI做写作评估外包的人，他当时为其中一个大型实验室工作。

And one of the people I talked to who was a contractor for Scale AI as a writing evaluator, and he was doing this for one of the bigger labs.

他说，那些评分标准根本毫无道理。

He said that the rubrics just didn't make any sense.

他会收到诸如“你得根据文本里的感叹号数量来给它们打分”这类要求。

He would be told things like you have to grade them based on the number of exclamation marks that there are.

所以如果一段内容里出现了三个感叹号，那就算超标了。

And so if something has three exclamation marks, that's too many.

那你就得给这段内容扣分。

And so you have to ding that one.

对。

Yeah.

而且我得说，这些写作建议总体上还不赖。

And I have to say, generally not bad writing advice.

我是说，我觉得这得取决于文本的长度，但就很多情况而言，三次实在是太多了

I mean, I guess it depends on the length of the text, but three feels like a lot for many

的场景里。

scenarios.

这就是他们给职场女性提的商务沟通建议。

This is what they tell women in business communications.

就像是，把所有这些感叹号都换成句号。

It's like, take all those exclamation marks, replace them with periods.

也就是说，我们要把所有这些东西都删掉。

Like, we are just gonna remove all of the items.

我们教女性要压抑自己。

We teach women to shrink themselves.

没错。

Exactly.

是的。

Yeah.

但确实如此。

But yeah.

所以他实际上是被要求去评判这些东西。

So he's he was sort of, like, being asked to grade these things.

还有另一个例子，他收到一堆同人小说，却被要求根据事实性来评分，因为这是其中一项标准。

Or another one was he got a bunch of fan fictions, and he was supposed to grade them on their factuality since that was one of the criteria.

我确实觉得，人们可以设计出比这位评估者所使用的更好的评分标准。

I do imagine that one could, you know, devise better rubrics than this particular evaluator was given.

但我认为，这至少表明了一些资源雄厚的大公司根本不知道如何判断什么是好的写作。

But I think it does show at least that some of these, like, very big companies that are very well resourced simply do not know how to think about what good writing is.

简而言之，我想强调一下，因为在我看来，这正是整个问题的关键。

Briefly, like, I like, I I wanna underline that because to me, that seems like the whole story.

我们正在把整个互联网的内容都拿去根据事实性来打分。

We are taking the entire Internet and we are grading it on factuality.

所以，你从这种做法中得到的大型语言模型，很可能根本没什么创造力。

And like so the the LLM that you're gonna get out of that is just probably not gonna be all that creative.

而且我在想，这在多大程度上与这种可验证的奖励机制有关呢？

Well and I wonder how much of it is related to this sort of verifiable reward Mhmm.

很多公司都在使用这样的系统：让一个系统生成大量代码，再让另一个评估模型检查代码是否合格。

System that a lot of these companies are using where you you have a system generate a bunch of code, and then you have another evaluator model check the code to see whether it's good or not.

这种做法在编程领域是有效的，因为代码要么能运行，要么不能运行。

And that works in domains like programming where the code either runs or it doesn't.

但创意写作并不是这样运作的。

But creative writing doesn't work that way.

你无法让评估者以任何一致的方式告诉你，某样东西究竟是好是坏。

You can't have an evaluator tell you, you know, with any sort of consistency whether something is good or not.

因此，这可能只是取决于个人偏好。

And so it may just come down to preference.

所以我想知道，你认为这是一个实验室正努力解决的技术问题吗？

And so I guess I'm curious, like, do you see this as a technical problem that the labs are frustrated trying to solve?

还是这只是需求相关的问题？

Or is this just demand related?

这仅仅是人们希望聊天机器人说话的方式吗？

Is this just what people want chatbots to sound like?

在每一次测试中，当不同模型相互比拼时，听起来像乏味的企业助手的那个总是胜出。

And in every test where they pit different models against one another, the one that sounds like a bland corporate assistant wins.

因此他们就选择了这种风格。

And so they go with that.

我认为两者都是对的。

I think both are true.

就像我们让模型做的大部分写作，都是帮我写这封邮件。

It's it's like the majority of writing that we are asking the models to do is write this email for me.

对吧？

Right?

而且，它们在这方面非常擅长。

And, like, they excel at that.

它们确实是出色的公司邮件写手。

They are truly great corporate email writers.

在那种委婉含蓄的表达方式上，它们比我强多了。

They are much better at the whole like passive aggressive thing than I am.

同时，我也同意你所说的，确实存在一个与可验证性相关的技术难题。

At the same time, I do think like you said, there is a technical challenge that has to do largely with verifiability.

有些人花了数十年时间试图阐明是什么让莎士比亚成为莎士比亚，或是什么让一首俳句成为俳句，但他们依然无法给出确定的答案。

It's like there are people who have spent decades of their lives attempting to articulate what makes Shakespeare Shakespeare or what makes a Naruto poem a Naruto poem, and they will still not know in any kind of certain way.

他们仍然会与同行学者和文学评论家争论谁的写作更优秀。

They will still get into debates with their fellow academics and literary critics about which writer is better than the other.

正因为这些事情是主观的、难以言喻的、难以用评分标准衡量的，这才是艺术的本质。

And because these things are subjective, because they are ineffable, because they are hard to put in a rubric, that is the nature of art.

说到这点，你一开始提到萨姆·阿尔特曼说，我们目前还写不出一首好诗。

And to that point, you know, you started this segment by talking about Sam Altman saying, like, hey, you know, we've just basically can't write a great poem yet.

一年前，萨姆·阿尔特曼曾表示，公司已经训练出一个优秀的创意写作模型，并在X平台上发布了一篇短篇小说。

Sam Altman, a year ago, said the company had trained a good creative writing model and posted a short story on x.

许多人觉得这篇小说很有吸引力。

Many people found it compelling.

萨姆·阿尔特曼是不是一直没对我们坦诚，贾斯敏？

Sam Altman just not being consistently candid with us, Jasmine?

哦。

Oh.

这也不是第一次了。

Wouldn't be the first time.

但那篇短篇小说，要是你们还记得的话，我敢肯定各位都有印象，里面有不少很棒的句子，比如提到镜子的接缝，还有周四，那叫什么来着？

But that short story, if you remember, I'm sure you guys recall, had some great lines, like, talking about the seams of mirrors or Thursday, the what was it?

好像是“介于周四和周五之间的临界时刻”之类的，或者说

It was like the liminal almost Friday or

差不多的说法。

something.

对。

Yeah.

就快到周五的那种过渡性日子。

The liminal day that takes almost Friday.

等一下。

Wait.

我真得去查一下这个内容，因为它写得实在太棒了。

I I have to actually look this one up because it was so good.

趁你查资料的工夫，你也知道，AI写作的问题在于它能想出一大堆新奇的比喻，有时候那些比喻还挺让人意外的。

While you're looking it up, like, you know, when the the thing about AI writing is like it comes up with all these fun metaphors, and they are like kind of surprising sometimes with the metaphors.

但这种语言并没有扎根于生活。

But also the language is not grounded in the life.

这正是我想说的另一点：除了可验证性之外，从根本上说，当我想到我真正喜爱的作家时，无论是记者、诗人还是其他类型的作家，他们都是从生活中写作的。

And that was my other thing is aside from the verifiability, fundamentally, when I think about the writers who I really love, when I think about whether it's journalists or poets or whatever, like they are writing from life.

对吧？

Right?

比如记者会走出去与人交谈，观察事物，留意天空在特定时刻的颜色；诗人则会思考他们亲身经历的个人体验。

Like a journalist goes out and talks to people and they like see stuff and observe like the color of the sky in a particular way or like a poet is thinking about personal experiences that they've had.

他们的写作是有分量的。

Their their writing has stakes.

它源自情感深处。

It comes from an emotional place.

尽管大型语言模型非常有才华，语法 impeccable，但它们没有生活。

And the fact that like LLMs while being very talented, grammatically pristine, whatever, they don't have lives.

这意味着它们所选择的所有隐喻、所有词语、所有例子，都是缺乏根基的。

That means that all of the metaphors they choose, all of the words they choose, the examples they choose, they're just ungrounded.

对吧？

Right?

因为写作并不是源自某种观点、特定经历或特定社群，所以缺乏可信度。

Like it's not coming from a point of view or a particular experience or particular community that makes the writing believable.

我认为，语调和风格的一部分在于它非常贴合一个人所经历的生活，而大语言模型无法像那些未曾真正经历过那种生活的人一样达到这种程度。

I think part of what voice and style is is that it is very specific to the life that a person has had, and LLMs cannot get there in the same way a human who hasn't really lived that life, like, cannot get there.

我不知道。

I don't know.

我觉得这要看具体情况。

I feel like it's case dependent.

你知道的？

You know?

我是个音乐迷。

I'm a big music fan.

是的。

Yeah.

在过去几个月里，我喜欢向大语言模型提出一些关于音乐的问题，特别是某些乐队的声音，这听起来像是个玩笑式的提问，因为大语言模型从未真正听过任何声音。

And over the past few months, I have enjoyed putting questions about music and, in particular, the sounds of certain bands to an LLM, which sounds like a joke prompt because an LLM has never heard anything.

对吧？

Right?

然而，我发现这些模型通常能和我就音乐的音色进行很好的对话。

And yet, I find that in general, the models can have good conversations with me about the sound of music.

也许它们只是基于互联网上大量有听觉经验的人所写的公开内容进行模式匹配。

Now it may be that they are just pattern matching based on a bunch of public writing on the Internet by people who do have ears and have heard.

对吧？

Right?

我非常接受这种可能性。

Like, I'm very open to that.

是的。

Yeah.

但我再次感到惊讶的是，它竟能以一种富有感染力的方式谈论感官类话题，至少在我看来，这超越了我对它能力的预期。

But I I I again, I I have just sort of been struck about the way that it is able to, like, sort of write about sensory topics in an evocative way that, at least to me, like, surpasses what I would predict they would be able to do.

是的。

Yeah.

我想提出一些可能有人会对你的文章提出的反对意见。

I I wanna pose a couple objections that I think someone might make to your article.

其中之一是这是科普。

One of them is this is Cope.

这是贾斯敏，一位作家，一位非常有才华的作家，她正在寻找AI目前还不擅长的方面，并认为这证明AI要完成这些任务将非常困难。

This is Jasmine, a writer, a very talented writer, sort of finding the things that AI in her view is not good at yet and saying this is categorical proof that they will it will be very hard for AI to do these things.

这和软件工程师在模型开始在编程方面表现得非常出色时的反应一样。

This is the same reaction that software engineers had when models started getting really good at code.

他们会说，哦，但我还不能做其他那十件事。

They would say, oh, well, I can't do these other 10 things that I do.

但事实上，再过几年，模型在所有事情上都会比我们所有人做得更好，包括写作。

And that basically just wait a few years and the models will be better than all of us at everything, including writing.

我真希望这真是科普，因为我一直在努力让自己被自动化取代。

I would love for it to be cope because I try to automate myself away all the time.

我并没有深深依恋必须亲自写作这件事，我喜欢写作，但过去三年里，我反复尝试过无数次，想让Claude替我完成我的工作。

I have no sort of deep attachment to having to like like, I like writing, but like I have tried over and over and over for the past three years to automate my own job away and to get Claude to do my job for me.

它做不到。

It cannot do it.

这非常令人沮丧。

This is very frustrating.

这并不是因为我没努力，你知道吗？

It's not out of a lack of trying, you know?

而且，我又要回到那些首席执行官们自己说的话上了，对吧？

And again, I'm going back to the CEOs themselves and the things that they themselves are saying, right?

这不只是我作为一个作家的看法。

Like it's not just me a writer.

是萨姆·阿尔特曼说，这项技术能治愈癌症、解决物理学难题，但它写不出比真正诗人的一首好诗更好的作品。

It's Sam Altman saying this thing will cure cancer and solve physics, but it will not write better than a real poet's okay poem.

所以，我认为这表明至少在人们感知中，存在某种微妙的不同。

And so, like, I think like that suggests that there is something that is at least perceived as a little bit different.

我认为在未来几年里，这些模型在写作方面可能会变得好很多。

I think it's very possible the models will get much better at writing over the next few years.

我不觉得这是永远做不到的事。

I don't think it's like a never thing.

我认为，像报道这种事很难被复制。

I do think that, you know, like reporting is hard to replicate.

我认为，拥有真实且可验证的生活经历是很难被复制的。

I think that, like, having life experiences that are real and verifiable is hard to replicate.

我认为，风格方面是可以改进的，尤其是如果你对模型进行微调的话。

I think the style stuff can be improved, especially if you fine tune the models.

但我觉得这篇报道有趣的地方在于，它展示了这些公司的市场激励和需求激励是如何塑造了我们今天所看到的它们的能力的。

But I think what's also interesting to me about this piece is that it shows how the the market incentives, the demand incentives of these companies do shape what we see their abilities are today.

我想象那些深度信赖AI的人可能会提出的另一个反对意见是，这一切都只是见仁见智。

The other objection I I'm imagining people might have who are very AI pilled is that this is all in the eye of the beholder.

对吧？

Right?

现在已有几项研究表明，如果让人们在不知情的情况下对AI写作和人类写作进行盲测，他们会更喜欢AI写作；但一旦告诉他们这是AI写的，他们对它的评价就会骤降。

There have been several studies now that have shown that if you give people a blind taste test of AI writing versus human writing, they prefer the AI writing until you tell them that it's AI writing, and then the value in their eyes plummets.

我最近在《纽约时报》的一个测验中做过一次这样的实验。

I did one of these in a New York Times quiz just recently.

那么，是否有可能AI模型在写作方面已经超越了人类，但只要我们一知道这些文字是AI生成的，而不是人类亲手写的，就会因为来源而非质量而完全失去兴趣？

So is it possible that the models have already become superhuman at writing, but that the minute we learn that they are AI models generating text and not humans writing words with their fingers, we lose all interest in it just because of the source, not because of the quality of the writing?

我的意思是，人们确实不喜欢AI写作，这一点毫无疑问，这也是他们看到明显是AI生成的文字时感到困扰的原因之一；但正如你所说，在这些测验和测试中，AI在这些狭隘场景下其实能胜过人类作家。

I mean, I think it's definitely interesting and true that people don't want to like AI writing, and that is part of what bothers them when they see AI text that is obviously AI even though, like you said, like, in these quizzes and tests, AI can outperform human writers in those narrow scenarios.

我的疑问在于，很多这类测验和测试都忽略了一点：作为作家，你们也都是作家，你们的工作中有多少真正是在进行文字生成呢？

I mean, my quibble with a lot of these quizzes and tests is that like as a writer and you guys are writers too, how much of your job is actually text generation?

我认为AI在文字生成方面已经超越了人类。

I think AI is a superhuman text generator.

对吧？

Right?

我的工作，我每天大约有25%的时间是在生成文字。

My job, I am generating text probably 25% of the hours in my day.

我花很多时间采访别人。

I spend a lot of time interviewing people.

我花很多时间构思点子。

I spend a lot of time coming up with ideas.

我花很多时间阅读，而且不是随便读，而是专门读那些我觉得合适的特定来源。

I spend a lot of time reading and not just reading indiscriminately, but like reading very particular sources that feel like the right ones.

所以，你知道，通常在你做这类测试的时候，你会说：‘请写一段关于特朗普为何赢得2016年大选的文字，500字以内。’

And so like, you know, usually at the point that you are doing one of these tests, you're saying like generate like one paragraph very specifically about like why Trump won the twenty sixteen election 500 words or less.

而且你已经给出了提示，我认为这是写作中非常关键的一部分——你到底要写什么。

And like you've already given the prompt, which I think is a critical part of writing is like what are you gonna write about.

你通常还会提供一些证据和指导，比如要求‘500字以内’。

You've often like supplied some of the evidence and the guidance in the form of it saying like 500 words or less.

在这一点上，我认为AI的文字生成能力确实比绝大多数人类都要强。

And at that point, I do think that AI is probably a better text generator than almost all humans are.

但再次想想，AI在为文章构思点子方面仍然非常糟糕。

But again, when I think about it, you know, AI is still very bad at coming up with ideas for articles.

它在新闻报道方面仍然非常差。

It is still very bad at reporting.

非文本生成的那部分工作离自动化还很远。

The non text generation parts of the role feel further away from automation.

我并不是那种绝对不说‘永远不’的人，我有点儿倾向于说‘永远不要说永远’。

Again, like, I'm not a never say like like, I'm I'm sort of, like, never say never.

也许我终会达到那个境界。

Like, maybe I'll get there.

如果克劳德能为我下一篇论文提供好点子，我会非常开心，但目前还做不到。

I would be totally happy if, you know, Claude was able to give me good ideas for my next essays, but it's not there yet.

我们已经看到大语言模型在类型小说领域取得了巨大进展。

Well, we're we're already seeing the LLMs make huge progress in genre fiction.

对吧？

Right?

最近在节目中，我们采访了一位《纽约时报》文章的作者，他谈到浪漫小说作家现在能用大语言模型每年生成几十部小说。

So, like, recently on the show, we talked to the author of a story in the Times about how authors of romance novels are now able to generate dozens of novels a year using LLMs.

事实上，我们之前的很多讨论都围绕着一个观点：你必须以不同方式、坚持不懈地提示模型，才能得到你想要的结果。

In fact, much of the discussion that we had was around how you just have to prompt them differently and sort of relentlessly in order to get what you want.

你知道吗，贾斯敏，你的文章让我思考：通过反复用各种方式告诉模型‘再怪一点’，能让模型写出更怪异的内容，这到底能实现多少？

You know, your piece, Jasmine, it made me wonder, like, how much of getting a model to just write weird can be achieved by repeatedly telling it in different ways, hey, be a little weirder.

一部分可以，但不是全部。

Some of it, but not all of it.

比如，我采访了詹姆斯·余，他是PseudoWrite的联合创始人，这是最早期的创意小说AI写作助手之一。

I mean, so I talked to, for example, James Yu, who is the cofounder of PseudoWrite, which is one of the earliest creative fiction AI writing assistants.

我还和其他一些同样在小说写作LLM领域的人聊过。

I talked to some other folks who similarly were in the fiction writing LLM space.

就像你说的，某种程度上，很多作家已经在大量使用这些工具，依赖LLM生成大量文本，这非常成功，也能满足读者的需求等等。

And like you said, to an extent, a lot of writers are already using these a lot, are already leaning on LLMs to generate large amounts of text and it can be very successful and it can meet readers' needs and whatever.

但就连我采访的这些人也向我描述，要逆转实验室在训练后所做的所有调整，有多难。

But like even these people who I was talking to, they were describing to me how freaking hard it is to undo all of the post training that the labs have done.

因此，他们不得不投入巨大的工程努力，这显然让他们非常沮丧——因为要让这些模型不再那么活泼、不再那么奉承、不再那么PG-13，回归到一种更原始、能再次变得怪异的基础模型状态，实在太难了。

So they are like applying immense amounts of engineering effort that clearly when in my conversations with them, clearly frustrates them that it is so hard to get these models to stop being so chirpy, so sycophantic, so PG 13 and everything in in order to get them to this sort of like base model state where they're able to be weird again.

所以我认为这当然是可能的，但我觉得实验室由于这些模型的训练方式，让这件事变得相当困难。

So I think it's certainly possible, but I think the labs have made it quite challenging just because of the way that these models are trained.

我认为另一件重要的事情是，我倾向于认为写作和许多创意工作实际上是这种半人马模型的完美应用场景。

The other thing that I think is important is I tend to think that writing and a lot of creative work is actually like the perfect use case for these centaur models.

对吧？

Right?

也就是说，人类与人工智能的协作才能达到最远的境界。

Like the idea that the human plus AI collaboration is where you can get the furthest.

当我听到你们采访那些小说作家的对话时，我就在想，这正是一个半人马模型。

And when I listened to the interviews that you guys did about the fiction authors, I was thinking this is a centaur model.

对吧？

Right?

因为如果没有人类不断引导、逼迫AI变得怪异、变得感性等等，它自己是根本不会这么做的。

Because without the human prompting and bullying the AI into getting weird and getting sensual and whatever, like it was not gonna do that on its own.

就像我自己，我也确实把LLM当作研究助手来使用。

And like I myself, like I do use LLMs as a research assistant.

就像我在《大西洋》文章中提到的，克劳德现在以一种我觉得极其有用的方式帮助我修改自己的作品。

Like I wrote about that inside the Atlantic piece about the way that Claude has now sort of helped me edit my own work in a way that I found incredibly useful.

但我确实觉得，在任何个人视角、生活经验真正重要的领域，协作元素都至关重要。

But I do feel like the collaborative element is is important for any domain where the personal perspective, lived experience, whatever really matters.

稍微谈一谈这一点。

Talk about that a little bit.

你提到了你的编辑过程。

You mentioned your editing process.

你是如何使用人工智能来帮助你编辑作品的？你觉得有用吗？

How are you using AI to help you edit your work, and are you finding it useful?

是的。

Yeah.

我觉得在过去几个月里，我终于找到了诀窍，这让我非常兴奋。

So I feel like I really cracked this over the last, like, couple months, which I'm very excited about.

因为再次强调，我曾无数次尝试让这些工具替我写作和修改，但它们从来都没真正能做到。

Because, again, I've tried to make these things, like, write and edit for me over and over and over, and they've never really been able to do it.

所以我意识到，如果我把克劳德当作一个编辑，而不是仅仅试图根据某种通用的优秀写作标准来评判和反馈我的作品。

So the thing that I realized was if I make Claude into an editor, that is not just trying to grade and give feedback on my work against some genericized standard of what good writing is.

而是真正地根据我本人贾斯敏对写作的个人追求来调整，它就能提供对我而言更有帮助的反馈。

But actually, we what we did against basically what my personal Jasmine's personal aspirations for writing are, it can give feedback that I find much, much more helpful.

所以我做的就是，把之前我写过的所有子章节存档，以及一些自由职业作品，全部输入给克劳德。

So what I did was basically I fed Claude my entire sub sec archive of the writing that I've previously done as well as some of my freelance work.

为了更具体一点，这是在克劳德的某个项目里完成的吗？你是怎么设置的？

And just to get real specific, is this inside, like, a Claude project, or how have you set this up?

因为我知道，我们的听众都想要尝试这种方法。

Because I know, like, our listeners are gonna wanna try this.

我是在一个项目里完成的。

I did it in a project.

但根据克劳德的建议，我当时想：我需要让克劳德写点代码吗？

But on Claude's advice, I was like, do I need to Claude code something?

克劳德说：不需要。

Claude was like, no.

这有点过度了。

That's overkill.

所以你根本不需要写代码之类的。

So you don't need to code or anything.

在Claude项目中，我上传了我全部的写作档案。

So in a Claude project, I gave it my whole archive of writing.

我本人在每篇作品发布后，都会给自己写一些回顾笔记。

I also personally write retro notes to myself after everything I publish.

我有一个笔记应用，里面全是我自己写的，关于我所有作品的优点和缺点。

So I have a notes app that's just like me writing what was good and bad about everything I've ever written.

就几条要点而已。

Just a few bullet points.

这就是为什么Jasmine以后会成为我们的老板。

This is why Jasmine's gonna be our boss.

我的这些要点质量其实很低，但我还是给了它，因为我想了解自己的审美偏好。

I mean, these are very low quality bullet points, but I also gave it that because I wanted to learn my taste.

我想了解自己渴望成为什么样的人，自己在哪些方面不足，又在哪些方面感到自豪。

I wanted to learn what do I aspire to be and where do I see myself falling short and where what am I proud of.

对吧？

Right?

基于这两点，再加上一些额外的信息，比如我的受众是谁、我的报道领域是什么、我的目标是什么，我们共同制定了一套评估标准，不再像以前那样关注感叹号有多少个，而是会问：‘这篇文章是否充分利用了你在硅谷的‘内部人类学家’身份？’

And so from those two things, plus a little bit more information about like, here's my audience, this is my beat, this is my goals, we were able to codevelop a rubric of instead of like how many exclamation marks does it have, it would say things like, does this take advantage of your quote unquote, like insider anthropologist position in Silicon Valley?

因为这正是我和Claude都认为能体现我独特风格的一点。

Because that's one of the things that Claude and I think distinguish my voice.

或者它也会注意到：‘哦，Jasmine，你经常在不同语体之间切换。’

Or it'll also notice like, oh, Jasmine, you tend to move between registers.

你会在创业术语、网络俚语等各种表达方式之间来回转换。

You'll switch between, you know, startup jargon and like Internet slang and whatever.

我觉得你能自如地在高端与通俗之间、政策议题与个人场景之间切换，这是你写作的鲜明特点。

And like, I think the fact that you can do the high low or move from like policy to personal scene, this is something that is characteristic of your writing.

所以，我们再次共同开发这些定性标准。

And so again, we're co developing these qualitative criteria.

然后我把它分成几个阶段：构思阶段、结构评分标准、文风评分标准、最终事实核查。

And then I split it into phases of like ideation phase, rubric, structure rubric, prose rubric, final fact checking.

所以我现在把这一切都放进了一个Claude项目中。

And so what I do now, I put this all in a Claude project.

我告诉它：你的任务是根据这些标准评估我的草稿，但不要替我写作，而是要不断引导我思考如何改进。

I said, your job is to evaluate my drafts based on this criteria, but not to do the writing for me and to make sure to prompt out of me, like, what I can do better.

我把一份草稿上传给了Claude。

I dumped a draft into Claude.

Claude会对其执行第二阶段的结构评估。

Claude will run like phase two structure on it.

它会说：你的结论只是总结，这太枯燥了。

It'll say things like, your conclusion is just a summary, and this is really boring.

事实上，在你写那篇关于某某的文章时，你最后是以一个场景收尾的，我觉得那样有力得多。

In fact, in your piece about this and that, you actually ended on a scene, and I thought that was much more powerful.

所以你为什么不试着用一个场景来结束这一篇呢？

So why don't you try ending this one on a scene?

克劳德会说，与其虚构一个场景，不如问问你自己：飞机起飞时你在想什么？

And Claude will say, rather than inventing a scene, it will say, what were you thinking when the plane took off?

你内心有什么感受？

What were you feeling inside?

你能想到一个与儿童安全倡导者讨论人工智能时让你深有共鸣的场景吗？

Can you think of a scenario where you had a conversation with say a kid's safety advocate about AI that really resonated with you?

因为现在听起来就像一篇枯燥的政策解释。

Because right now it sounds like dry policy explainer.

事实上，我觉得这种反馈非常有用。

And, that feedback, I actually found incredibly useful.

比如，我仍然在运用自己的判断来决定是否采纳这些建议。

Like, I'm still applying my own judgment to say, do I take it or not?

但我知道，这关乎我成为更好的自己，一个更出色的写作者。

But I'm like, you know, this is about me becoming the best version of myself as a writer.

这关乎自我提升，而克劳德在推动我这样做，我觉得这要有效得多。

It's about, like, me self improving and Claude pushing me to do that, which I found much, much more helpful.

哇。

Wow.

我想以同行作家的身份问你们两个一个问题。

I wanna ask you both a question as fellow writers.

你们有没有因为AI而产生让自己的写作更怪异的冲动，以从一堆平庸作品中脱颖而出？

Do you feel the impulse to make your writing weirder because of AI to sort of stand out from the sea of slop?

因为我自己就感受到一种拉扯：哦，这个奇怪的细节可能该删掉，但我还是想留着，因为克劳德永远不会这么写。

Because I I find myself feeling this tug of, like, oh, that's a little weird aside that probably I should cut, but I think I'm gonna leave it in because, like, Claude would never do that.

对吧？

Right?

这就像一种标记，表明这些字是我亲手打出来的，我觉得这就是我留下的印记。

It's like it's like a marker that I am typing these words, and I feel like that's sort of my imprimatur that I'm leaving.

我的回答是：是的。

My answer to you is yes.

我确实有这种感觉。

I absolutely feel that way.

我确实回去修改过句子，故意让它们显得更怪异一些，尤其是让它们听起来更口语化，因为我知道大型语言模型通常不会这么表达。

And I've I've, like, gone back and tried to edit sentences to, like, make them feel a little bit more, like, weird or, like, in particular, to make them sound colloquial in a way that I know, like, an LLM generally would not be.

是的，正是因为这个原因。

And, like, yes, it is for that reason.

我觉得现在的写作，我们其实都不尽相同。

I think that writing right now, like, we're all not all.

我们很多人都对可能读到垃圾内容高度警惕，所以如果你是个不想生产垃圾内容的作家，你就应该问问自己这个问题。

Many of us are on such high alert for the prospect that we might be reading slop that I think if you are a writer who does not want to be producing slop, like, you you should be asking yourself that question.

我觉得这让我更能安心地按照自己本来的方式写作。

I I think it makes me a lot more comfortable writing the way I want to write in the first place.

我觉得，可能和你们两位不同，我并没有在新闻编辑部成长，没有被灌输过那种特定的报社风格和各种规范。

Like, I think, like, maybe unlike both of you, I didn't sort of come up through newsrooms where I was, like, learning a very specific house style and all of these norms.

我现在也能写新闻稿了。

Like, I can do news writing now.

这是我后来学会的。

It's something I've learned now.

但说实话，我其实更贴近网络和博客原生的风格，这种文体更口语化、不拘一格，也不那么精致，会开一些不恰当的玩笑，总之是一种更松散的写作形式。

But, like, I'm actually much more, quote, unquote, like Internet and blogging native, which is a form that is voice y and irreverent and not as pristine and, like, will make inappropriate jokes and, like, you know, it's just a looser form of writing.

所以我认为，这让我更自在地去写那种博客风格的内容，而不是总试图用更职业化的新闻语调来写作。

And so I think what it's actually done is made me more comfortable doing the bloggy thing instead of sort of always trying to write in a more professionalized journalistic tone.

所以我想，我们应该给你，贾斯敏，留一个问题：你的文章非常有说服力地指出，如今的AI并不擅长我们所有人所珍视的那种写作。

So I think we should leave this with a question for you, Jasmine, which is, you know, your your piece makes the case very convincingly that today's AIs are not very good at the kind of writing that I think we all value.

你认为它们将来能达到那种水平吗？

Do you think they will get there?

公司应该做些什么，才能让它们的模型在写作上变得更好？

And what should the companies do to make their models better at writing?

我认为，如果我们把文本生成和新闻报道分开来看——我对模型做新闻报道这件事并不太看好——而只是讨论文学小说，或者给你一堆采访录音，让你写一篇杂志特稿之类的，我认为，如果它们能把投入在编码代理和真正赚钱项目上的资源，同样多投入到这个任务上，它们是有可能做到的。

I think that if we separate out text generation from reporting, which I'm not that bullish on the models doing, and we are just talking about, say, literary fiction or here's a bunch of interview transcripts, write a magazine feature or something, I think that if they applied as many resources towards that task as they do towards coding agents and things that actually make them money, I think that they could get there.

公司会最终觉得，把所有资源都花在提升写作能力上，而不是用来替代23岁的软件工程师，从财务上更划算吗？

Will the companies ever find it financially advisable to spend all their resources on that instead of automating 23 year old software engineers?

大概不会。

Probably not.

我会感激这样的世界。

I would be grateful for that world.

我不需要它们取代我的工作或这些人的工作，但我认为这是可能的。

I don't need them to take my job or these folks' jobs, but I think it's possible.

听好了。

Look.

它们最终会做到的。

They're gonna get around to it eventually.

明白吗？

Okay?

你知道，我的意思是，我听懂你的意思了。

You know, it's like, I mean, I I I hear you

凯西，在这个经济环境下，作家能赚多少钱？

say writers make in this economy, Casey?

最终，这些收入可没法支撑大量的数据中心。

Eventually, like, those aren't gonna pay for a lot of data centers.

不。

No.

写作是有经济价值的，最终，AI公司会想把这一切都据为己有。

There there is economic value in writing, like and eventually, the AI companies will want that all to themselves.

你知道这件事可能会带来什么非常滑稽的结果吗？

You know what would be a very funny outcome of this?

你知道吗，关于模型的那些限制措施，你的观点是这样的。

You know, taking your your point about the sort of guardrails of the models.

也许下一部伟大的美国小说会由Grok写出来。

Maybe the great the next great American novel will be written by Grock.

天啊。

Oh god.

那太

That's

好了，感谢贾斯敏·孙加入我们的讨论。

And with that, Jasmine Sun, thank you for joining us.

谢谢

Thank you

非常感谢。

very much.

凯文和凯西。

Kevin and Casey.

我们回来后，每个人都在花钱买令牌，凯文。

When we come back, everyone's spending money on tokens, Kevin.

太好了。

Great.

继续。

Keep going.

你得做个令牌。

You've gotta be token.

令牌最大化，就是这样。

Tokenmaxxing, that is.

继续。

Keep going.

是的。

Yes.

而且。

And.

我们回来后，你在说什么？

When we come back, what are you talking about?

这个问题正由席卷硅谷的排行榜提出。

That's the question being asked by a leaderboard that's sweeping Silicon Valley.

席卷。

The sweeping.

它真的在席卷。

It's it's really sweeping.

看到了吗？

See?

好吧，凯文，你最近结束了休假，又开始为《纽约时报》撰稿了。

Well, Kevin, you've recently returned from book leave and are once again writing in the New York Times.

再次看到自己的名字印在纸上，感觉如何？

How does it feel to see your name in print again?

感觉很棒。

Feels great.

还没发生呢，但一旦发生了，一定会很棒。

Hasn't happened yet, but when it does, it'll be great.

我提前读了你即将发表的一篇文章，内容是科技公司现在创建了排行榜，显示哪些员工在工作中使用了最多的AI令牌。

Well, I got to take an early read at a story that you are publishing about the fact that tech companies have now created leaderboards to show which employees are using the most AI tokens in their work.

是的。

Yes.

这是一项

It's a

那里正掀起一场令牌热潮，这些公司的员工在同事之间非正式地、带着点趣味地竞争，但他们却非常认真对待。

token frenzy out there, and the employees of these companies are competing among their colleagues sort of informally and and sort of for fun, but they're taking it very seriously.

他们希望自己是公司里使用AI令牌最多的那个人。

They want to be the people at their company who are using the most AI tokens.

所以我想问一个基础问题，给可能不太了解的听众。

So let me just ask a basic question for listeners who may not be familiar.

什么是令牌？为什么你会开始关注这个东西？

What is a token, and why is that something you might start keeping track of?

令牌是AI工作的基本单位。

So a token is the basic atomic unit of AI labor.

它基本上是一个词的片段，也是AI模型提供商衡量使用量的方式。

It's basically a a fragment of a word, and it is how AI model providers measure their consumption.

所以如果你输入一个提示，比如‘帮我写这篇论文’，旧模型可能会用几百个令牌来回应。

So if you type in a prompt, you know, help me, write this essay, an old model might have given you a couple 100 tokens in response.

那相当于几百个单词。

That would be a couple 100 words.

而过去一年左右，随着这些自主编码工具的兴起，模型对令牌的需求变得大得多。

And what has been happening over the past year or so as these agentic coding tools have started taking off is that the models are just much more token hungry.

现在你可以在单次会话中使用数十万甚至数百万个令牌。

You can use now hundreds of thousands or even millions of tokens in a single session.

因此，正是这种理念推动了这些排行榜的兴起：你进行的编码越多，使用的智能工具越多，同时运行的进程越多，你的令牌数量就会越高。

And so that is what is propelling these leaderboards is the idea that the more sort of coding you're doing, the more agentic tools you're using, the more simultaneous processes you're running, the higher your token count will be.

我发现一个有用的衡量标准是，生成7500个单词大约需要10000个令牌。

One measurement I found useful was that apparently, it takes about 10,000 tokens to generate 7,500 words.

如果这个数字能帮你更好地理解的话。

If that sort of, you know, helps to ground you at all.

但正如你刚才所说，我想多了解一下，更先进的系统使用的令牌数量远超这个数字。

But as you just said, and I want to hear more about this, the more advanced systems are using way more tokens than that.

那么，能跟我讲讲那些令牌使用顶尖人物在排行榜上公布的数字吗？

So tell me about some of the numbers that some of the sort of token all stars are putting up on the on the boards.

我不清楚所有确切的数字，但我了解到，在OpenAI，他们确实追踪这类排行榜，最近一位员工在七天内的最高令牌使用量达到了2100亿个。

So I don't know all of the exact numbers, but I did learn that at OpenAI, where they do track this kind of leaderboard, the highest employee token count over a seven day period recently was a guy who used 210,000,000,000 tokens.

作为参考，这大约相当于33部维基百科内容的文本量。

And this is, for rough scale, about 33 Wikipedia's worth of text.

而现在，这一切并不只是打字和接收回复。

And now all of that is not sort of typing and receiving a response.

其中一部分被称为缓存令牌。

Some of that is what they call cached tokens.

所以，并不是所有内容都是首次由模型生成的，但我想，即使是一年前，这些数字听起来也会完全荒谬。

So it's not all sort of, you know, being extruded from the model for the first time, but these are the kinds of numbers that I think even a year ago would have sounded completely insane.

对。

Right.

那么，这个人是在为国防部开发一个新的大规模国内监控项目吗？

Now was this guy working on a new mass domestic surveillance program for the Department of Defense?

我不知道，OpenAI也没有让他接受采访。

I don't know, and OpenAI did not make him available for interviews.

但我在撰写这篇专栏时，想试着联系一些人，

But what I wanted to do in writing this column was to try to call up a

或者与一些人交谈

bunch of people or talk to a

一群属于这十亿令牌俱乐部的人，对吧，那些极端的高级用户，然后直接问他们：嘿。

bunch of people who are in this sort of billion token club, right, the the sort of extreme power users, and just ask them, like, hey.

你们是怎么用掉这么多令牌的？这不会非常昂贵吗？

How are you guys using all those tokens, and isn't that very expensive?

你们是怎么支付这笔费用的？

And how are you paying for it all?

我学到了很多。

And I learned a lot.

是的。

Yeah.

好吧。

Well, okay.

那么首先告诉我们，这到底有多贵？

Well, so tell us first of all just how expensive it is.

非常昂贵。

Very expensive.

是的。

Yeah.

事实上，我听说使用Claude代码最多的个人用户，按Anthropic的统计，上个月在令牌上的花费超过了15万美元。

In fact, I I heard that the top user of Claude code, the top individual user of Claude code as measured by Anthropic spent more than a $150,000 on tokens last month.

所以你推算一下。

So extrapolate that.

这相当于一名员工年薪超过一百万美元。

That is like a an employee making more than a million dollars a year.

是的。

Yeah.

而他们一个月就花掉了这么多。

And they are burning that in a month.

我还从其他一些极端的程序员那里听到了类似的数据，他们每天在这些模型的令牌上花费数千美元。

And I heard similar figures from some of these other extreme coders who are spending something on the order of thousands of dollars a day on tokens from these models.

不过我们也应该说明，这些公司的员工可以免费获得令牌。

Now we should also say the employees of these companies get their tokens for free.

对吧？

Right?

所以他们自己并没有掏钱。

So they are not shelling out.

他们的公司也没有掏钱。

Their companies are not shelling out.

但在其他公司，这正开始成为一个问题，因为他们的支出已经超出了这类开销的预算。

But at other companies, this is starting to become an issue because they are sort of outstripping their their budgets for these things.

所以确实有一些公司，工程师们真正地给雇主带来了每周高达15万美元的成本，因为他们大量使用大型提供商的令牌。

So there are companies where there are engineers who legitimately are costing their employers maybe a $150,000 a week because they're getting tokens from one of the big providers.

是的。

Yeah.

我认识一位在瑞典的软件工程师，他说他花在Claude上的钱可能比他的工资还多。

I talked to a software engineer in Sweden who said that he probably spends more than his salary on Claude.

因此，这对一些程序员来说，正逐渐变成一种极其昂贵的福利。

So this is essentially becoming like a a very expensive job perk for some of these coders.

所以跟我讲讲，为什么雇主们想创建排行榜来鼓励员工使用这些工具？因为我觉得其他公司可能会说：‘上个月你花了15万美元在令牌上？那你别在这儿干了，公司都破产了。’

So talk to me about why employers want to create leaderboards to promote this to employees because I could see other companies saying, if you spent a $150,000 on tokens last month, you actually don't work at this company anymore because we're bankrupt.

对。

Right.

所以这正是我很好奇的问题：为什么会发生这种情况？

So this was a big question that I had is, like, why is this going on?

这似乎是一些员工激励和员工行为追踪的结合。

And it seems to be some combination of sort of employee motivation and worker tracking.

对吧？

Right?

这些公司的高管们认为，你使用的令牌越多，可能就越有生产力。

There are executives at these companies who think that the more tokens you use, the more productive you probably are.

正如我们在本节之前的环节中讨论过的，这些公司非常希望员工开始使用AI工具。

And as we discussed in a previous segment on this show, these companies are very eager to have their workers start embracing the AI tools.

因此，在不少公司里，我跟一些人聊过，他们说，是的。

And so at a number of these companies, I talked to people who said, yeah.

这本质上是他们想看看谁真正全身心投入了这种新的编程方式。

This is just basically them trying to see who is really all in on the new way of programming.

你已经和一些在这些排行榜上排名靠前的人谈过了。

And you've talked to a number of people who are ranking high on these leaderboards.

我知道你可能没有深入研究过他们的代码，但你对他们实际的生产力有什么看法？

I realize you probably haven't dug deep into their code, but what is your sense of how productive they actually are?

也就是说，令牌使用量和推动我公司迈向新高度之间有什么关系？

Like, what is the relationship between token usage and taking my company to the next level?

我的意思是，这非常不清楚。

I mean, it's very unclear.

对吧？

Right?

这些人中有些可能只是在生成一些毫无价值的项目。

Some of these people may be just generating, like, worthless projects.

我觉得我接触的很多人对这些排行榜感到担忧的是，它们只是在激励你拼命增加令牌使用量。

I think the the thing that worries a lot of the people I talk to about these leaderboards is that they just incentivize you to, like, run up your token count.

对吧？

Right?

因为这样你就会显得像是那个特别的、所谓的十倍工程师或百倍工程师，远远超越你的同事。

Because then you look like the special, you know, 10 x engineer or 100 x engineer who's, like, outperforming all your colleagues.

所以我认为有不少公司觉得这种排行榜有点奇怪，甚至可能适得其反。

So I think there are a number of companies that that see this leaderboard business as a little strange and maybe counterproductive.

但我确实觉得，那些最频繁使用令牌的人中，有一种他们正在高效工作的感觉。

But I do think that there is a feeling among the most sort of heavy token users that they are being productive.

是的。

Yeah.

我得说，读你的专栏时，我觉得这简直会制造最糟糕的激励机制。

I have to say, when I read your column, I thought this just seems like it would create the worst incentives.

没错。

Yes.

对吧？

Right?

有一种叫做古德哈特定律的观点。

There's this idea of Goodheart's law.

对吧？

Right?

当一个指标变成目标时，它就不再是一个好的指标了。

Like, when a measure becomes a target, it ceases to become a good measure.

我想不出比为令牌使用量创建排行榜更能确保它变成一个糟糕指标的方法了。

I can't think of a better way to ensure that tokens usage becomes a bad measure than creating a leaderboard for it.

完全正确。

Totally.

公司内部的人对此有什么看法？

What are the people inside the company saying about that?

嗯，有些人反对这种排行榜做法。

Well, some of them are opposed to this whole leaderboard thing.

我也和一些为排行榜辩护的人谈过。

I also talked with some folks who defended the leaderboards.

他们说，看啊，要衡量程序员的生产力从来就不是件容易的事。

They said, look, it's never been all that easy to track the productivity of programmers.

有些人曾用他们写出的代码行数或提交的拉取请求数量来衡量生产力。

Some people have had their productivity measured by, like, how many lines of code they generate or how many pull requests they made.

这些都只是衡量你有多努力、做了多少工作的不完美替代指标。

These are sort of these imperfect proxies for, like, how hard are you working, how much are you doing.

但这些公司的员工也明智地认为，这关系到他们自身的成功。

But the employees of these companies also see this, I think, wisely as a key to their own success.

如今，许多公司正将AI令牌的使用和消耗纳入绩效考核周期。

A number of these companies are now using AI token use and consumption as part of the performance review cycle.

所以你去参加年度评估时。

So you go in for your annual review.

你的老板会说，嘿。

Your boss says, hey.

看起来你上个月只用了大约七千万个令牌。

It looks like you only used, you know, 70,000,000 tokens last month.

怎么回事？

What's going on?

所以我认为，这些公司的工程师们逐渐意识到，如果他们想拥有长期成功的职业生涯，最好开始多使用一些令牌。

And so I think the engineers of these companies are getting wise to the fact that if they want to have a long successful career, they better start using some tokens.

是的。

Yeah.

但我想象他们中的一些人对此其实很紧张。

But I imagine that some of them are really nervous about that though.

对吧？

Right?

因为在我看来，至少有一些公司希望通过激励令牌使用来达到目的，因为他们自己也怀疑，我们让员工使用这些工具越多，就越不需要长期雇佣人类了。

Because, like, it seems clear to me that at least some of these companies want to incentivize token usage because the companies themselves suspect that the more we can get them using this stuff, the less long we will have to employ the humans.

也许吧。

Maybe.

不过，我认为这与其说是人工智能系统取代人类，不如说是一种截然不同的工作方式。

Although, I think it's less about, like, the AI systems replacing the humans and more about, like, it is just a radically different way of working.

对吧？

Right?

这些人中的大多数都有很长的软件工程职业生涯。

These are people who most of them have had long careers in software engineering.

他们从小就是手写代码长大的。

They grew up writing code by hand.

他们可能从小就开始使用某种AI助手，比如GitHub Copilot。

They maybe grew up using some sort of, like, AI assistant, like GitHub Copilot.

这些公司里的人说，这些自主工程系统真的非常不同。

And what people at these companies are saying is that these agentic engineering systems are just really different.

你需要以不同的方式来应对它们。

You have to approach them in a different way.

你需要花大量时间与它们相处，了解它们擅长什么、不擅长什么。

You have to spend a lot of time with them to understand what they're good and not good at.

对它们来说，这是一种激励员工的方式，意思是：嘿。

And to them, this is sort of a way of motivating their employees to say, hey.

去试试新东西吧。

Go out and try the new thing.

是的。

Yeah.

我不确定。

I I don't know.

我一直在思考这个问题：如果我是这些公司中的一名工程师，并且有动力登上排行榜，我会怎么应对？

I've been thinking a lot about this question of, like, if I were an engineer at one of these companies and I have this incentive to get on the leaderboard, like, how would I approach it?

我认为，本能地会想浪费大量令牌来提升排行榜名次。

And I do think that, like, the instinct to just, like, waste a bunch of tokens to to, like, rise higher on the leaderboard.

但最终，如果你排名太高，人们会问你那些令牌都用到哪儿去了。

Like, ultimately, if you rise too high, people are gonna ask you what you did with all the tokens.

没错。

Right.

如果你以一亿次调用的用量排在第一，却只写了个计算器之类的程序，人们很可能会生气。

If you're number one at, like, 10,000,000,000 and you only manage to, you know, like, you know, vibe code a calculator or something, people are probably gonna get mad at

你。

you.

是的。

Yeah.

我确实和一个人聊过，他推测 leaderboard 上排名靠前的人其实都在做副业。

And I actually did talk to one person who speculated that actually the people top of the leaderboards are all doing side projects.

他们正在创办他们的

They're starting their They're starting their

新公司。

new company.

他们用老板的钱创办了一家新公司。

They start a new company with the with the boss's money.

如果你真的这么做，我想说，我向你致敬。

And if you're doing that, I just wanna say, I salute you.

这才是正确的工作方式。

Like, that is the right way to work.

对。

Yeah.

要是你打算这么做的话，或许别当排行榜第一比较好。

Maybe don't be the number one on the leaderboard if you're doing that.

试着稳定在第六或第七名左右。

Maybe try to stick around six or seven.

没错。

Yeah.

其实排行榜中游才是你应该努力达到的位置。

Like middle of the pack is is kind of where you want to aim yourself.

那个，我来问个问题。

I mean, let me ask.

你觉得有没有什么代币追踪工具能提供靠谱的参考信号？

Is there any kind of token tracking that you think offers a reasonable signal?

比如说，你觉得如果是一家科技公司，该不该做排行榜？

Like, do you think that if you're, like, a tech company, you should create a leaderboard?

不。

No.

我觉得这

I I think that's

是一个糟糕的主意，原因就是我们刚才讨论过的所有内容，包括好心法则，我认为这只会导致人们浪费代币，去做一些副项目。

a bad idea for all the reasons that we just talked about, including Good Heart's Law, which is I think this is just going to lead to people just wasting tokens, doing side projects.

但如果我是公司里的预算负责人，看到人们花在AI代币上的钱是他们薪水的数倍，嗯。

But if I'm the budget manager at a company and I'm seeing that people are spending multiples of their salary Mhmm.

我就会问他们，到底用这些代币做了些什么。

On AI tokens, I'm asking them some questions about what they're doing with that all.

如果他们的回答不是‘我开发了一个能每年创造数十亿美元收入的全新产品’，那我就会说，嘿。

And if their answer is not, I built an amazing new product that's gonna generate billions of dollars a year in revenue, I'm trying to say, hey.

你这个月能不能少用一点？

Could you maybe use a little less this month?

是的。

Yeah.

我得说，这个代币排行榜的想法让我意识到，它只不过是软件行业长期以来一直在探索的一个问题的新形式：我该如何判断我的软件工程师是否高效？

I I have to say, I have been struck at how this idea of the token leaderboard just represents a new incarnation of something that the software industry has been trying to figure out for a long time, which is how can I figure out if my software engineers are productive?

我最近刚和我未婚夫——一位非常英俊的软件工程师——聊过你的专栏。

You know, I was talking recently to this very handsome software engineer who I'm engaged to about your column.

他告诉我，他过去是根据他贡献的代码行数来评估绩效的。

And he was telling me that, you know, he used to be evaluated on how many lines of code he contributed.

他还跟我讲了当年人们玩的各种把戏，比如写一个快速算法，把一堆东西翻译成几种新语言，虽然完全没用，但能让他看起来那一周特别高效。

And he told me about all the games that people used to play back in the day with, oh, you know, I like wrote a quick algorithm to like, you know, translate a bunch of stuff into some new languages and it's like completely worthless, but it makes me look like I had a very productive week.

于是我回去查了下，发现这种做法早在六七十年代就有了。

And so I went back and looked into this and they were doing this in the sixties and seventies.

在计算机编程的早期，曾流行一句话：用代码行数来衡量编程进展，就像用重量来衡量飞机制造的进展。

And there's this saying from the early days of computer programming that eventually arises that says, quote, measuring programming progress by lines of code is like measuring aircraft building progress by weight.

我必须说，我觉得这里的情况也差不多。

And I have to say, I think that the same thing kind of applies here.

对吧？

Right?

是的，如果你稍微眯着眼睛，从足够抽象的层面来看，确实可能有些使用大量令牌的人比那些不使用的更高效。

That, like, yes, if you squint and at the right level of abstraction, it's probably true that some people who are using a lot of tokens are more productive than some people who aren't.

但这似乎并不是衡量这些事情的正确方式。

It just doesn't quite seem like the right way to measure these things.

我只是好奇，这个行业多久才能意识到这一点。

And I just wonder how quickly the industry is gonna figure that out.

对。

Yeah.

我认为这会很快发生，部分原因是预算已经变得极其荒谬。

I think it's gonna be pretty soon in part because the budgets are just getting very ridiculous.

尤其是AI模型提供商现在看到，单个用户消耗的服务量，是几个月前整个公司都难以企及的。

And and especially the AI model providers are now seeing individual users consuming amounts of their services that entire companies would have consumed just a few months ago.

关于这个问题，我最后想问你的是，你认为这对整体经济会有什么影响？

You know, maybe the kind of last question I have for you about this is just what implications do you think it has for the broader economy?

对吧？

Right?

因为我们知道，在经济的许多不同领域，管理者都在说，我想激励我的员工使用人工智能，并且我想追踪他们是如何使用人工智能的。

Because we know that in so many different sectors of the economy, managers are saying, I want to incentivize my employees to use AI, and I wanna track how they're using AI.

所以你认为，随着这些排行榜信息的传播，非技术领域的员工会不会也试图建立自己的版本？

So do you think that as knowledge of these leaderboards spread, we're going to see people in nontechnical fields try to adopt their own version of them?

我不希望这样。

I hope not.

我认为这不仅对追踪实际生产力和产出是个错误的举措，对员工士气也是如此。

I think it's really a bad move, not just for tracking actual productivity and output, but just for morale.

对吧？

Right?

我记得多年前，Gawker 公司办公室里曾有一个流量排行榜，你可以看到自己的文章点击量相对于其他人的情况。

Like, I remember years ago when, like, Gawker would have, like, a traffic leaderboard at their office so you could see how many clicks your stories were getting relative to other people.

我不认为当时在那里工作的任何人觉得这种做法能激励正确的行为，或者提升员工的士气。

I don't think anyone who, like, worked there at the time thought that was, like, incentivizing the right things or creating, like, high morale among employees.

基本上，每个人都在不停地相互竞争。

Basically, everyone was just competing with each other all the time.