01: Applications of Generative AI with NVIDIA's Jim Fan

但现在你还想再进一步优化。

But now you want to refine it even more.

做法是，用同样的指令让模型生成五次回答，它会给出五个不同的版本。

So the way you do it is you query the model, let's say, five times given the same instruction, and the model will give you five different answers.

然后由人工标注员来对模型生成的这五个回答进行排序，而不是亲自写答案。

And you have a human annotator that instead of writing the answers to that instruction, the human annotator just ranks on the the five answers that the model generated.

这样任务简单多了，标注成本也低得多，省了不少人力。

So that is a much easier task and much less expensive in annotating, much less human effort.

只需要给个排序就行。

It's just to give you a ranking.

比如说，我觉得 B 比 C 好，C 比 D 好，D 比 A 好，A 比 E 好。

Like, I prefer, b to c to d to a to e, for example.

通过这种偏好排序，就能训练出一个奖励模型。

And from this, preference ranking, you'll be able to learn a reward model.

这个奖励模型的作用是，根据指令和回答，判断这个回答有多好。

And the reward model essentially tells you that given this instruction and an answer, how good this answer is.

现在这个奖励模型就像一个自动化的标注人员。

And now this reward model kinda it's kind of acting as an autonomous human annotator.

对吧？

Right?

你不需要再雇更多的人来打分了，因为模型已经学会怎么做了。

You don't need to hire more human annotators to give you more reward because the model already learns how to do it.

然后你可以用这个模型来做强化学习，也就是说，模型会通过不断尝试和错误，自己学会提高奖励分数。

And then you can use this model to just do reinforcement learning, which means the model will figure out on its own using trial and error how to improve this reward score.

剩下的部分就完全自动化了。

And then the rest of it is completely autonomous.

是的。

Yeah.

这是个独立的模型。

This is a separate model.

它不是那个预训练模型本身。

This is not the same pretrained model.

这是一个独立的奖励模型。

This is a separate reward model.

对吧？

Is that right?

在最初的InstructGPT论文里，它确实是单独训练的一个模型，扮演评审的角色。

At least in the original InstructGPT paper, it is a separate model, separately trained, and that model is acting as a judge.

明白了。

Okay.

问题是这样的。

Like, these this is the question.

答案是这样的。

This is the answer.

你做得怎么样？

Are you doing well?

是或不是？

Yes or no?

模型就是这么做的。

That that is what the model does.

然后呢，这个GPT模型会通过不断试错，来尽量提高被这个评判者打的分数。

And then, you know, this kind of, this GPT model will try to maximize the score against this judge by doing trial and error.

剩下的部分就完全自动运行了，不需要人工介入。

And then the rest is just, you know, completely autonomous without a human error loop.

所以在强化学习的训练循环里，你其实是在调用这个奖励模型来做评判。

So during the RL training loop, you're actually calling into this reward model

对。

Yep.

去打分。

To judge.

对。

Yep.

我很感兴趣。

I'm interested.

没错。

Exactly.

正是如此。

Exactly.

然后你可以用强化学习这类方法，让模型根据这个评判标准不断优化生成内容。

And then you can use techniques like reinforcement learning to improve what a model generates against this judge.

剩下的部分，模型就能自己慢慢改进了。

And the rest, you know, the model can can just improve on its own.

对。

Yeah.

这就是人类反馈强化学习的基本原理。

So that is kind of how reinforcement learning from human feedback works.

阿姆贾德，你想聊聊宪法式AI吗？

And and, Amjad, do you wanna talk about constitutional AI?

或者

Or

我对这个其实不太精通。

I I don't have, you know, good expertise in that.

不过我确实关注过自指令这个方法，没错。

But, yeah, I've been looking at the self instruct, which is Mhmm.

它和你刚才说的指令微调很像，但不用强化学习。

Instruct tuning similar to what you described, but it doesn't use RL.

自指令是一种更偏向微调的技术。

Self instruct is what's used it's it's more like a fine tuning technique.

Alpaca的做法是，用GPT-3这样的大模型生成大量指令和对应的回答，造出成千上万组数据。

And, basically, what they've done with Alpaca is they would generate, basically, pairs of instructions and answers from g p t three from a larger model, and you generate thousands of examples.

然后用这些数据作为标准答案，去微调一个小一点的模型。

And then you would fine tune a smaller model based on these, like, pairs, basically, ground truths.

你可以拿一个预训练好的模型，比如LLaMA 70亿参数的，用这个自指令数据集再训练一遍。

And you can, like, you can take a pretrained model, like a LLMA 7,000,000,000, and, like, train it on the, you know, self construct dataset.

最后得到的效果，其实跟RLHF很接近，模型真的会听懂你的指令。

And you end up with something, you know, very similar to RLHF where it is actually listening to your commands.

它现在能更清楚地理解和响应你的指令和问题。

It's actually, like, able to, you know, able to follow your orders and your questions more clearly.

你要是还记得 GPT-3 2020 年之前的样子，就知道那时候它经常对你说的话置之不理，自顾自地干别的。

Because, you know, anyone who remembers c p t three twenty twenty, you know, pre all these algorithmic innovations, it it sometimes just ignores what you're saying, and it goes and does something else.

那时候它也有自己的魅力，但现在大家已经习惯了能认真听你说话的模型。

And and it had its its own charm, but now people are just used to models that actually listen to them.

宪法AI是另一个方法。

And Constitution AI is is another is another one.

那我可能得把话筒还给你，你来解释一下。

So may maybe I'll pass it back to you to to explain it.

好的。

Yeah.

我大概讲讲宪法AI背后的基本思路。

I'll I'll roughly explain kind of the intuition behind Constitutional AI.

论文里有更多技术细节。

So and the paper has a lot more technical details.

这是Anthropic公司的一项研究成果。

It is a work from from Anthropic.

如果你对具体的实现细节感兴趣，我建议大家去读一读这篇论文。

I encourage everyone to read it if you're interested in the particular implementation details.

简单来说，宪法式AI就是让模型扮演我们刚才提到的指令范式中的人类角色。

So, like, roughly speaking, constitutional AI is fine to have a model kind of play the part of of the human in the instruct paradigm that we just discussed.

很多人可能听说过阿西莫夫的机器人三定律。

So, you know, many of you might have heard, Asimov's, three laws of robotics.

比如第一条：绝对不能伤害人类。

Like, the first law is, you you can never hurt human.

第二条是：要保护自己，除非这会伤害到人类，诸如此类。

The second law is, like, you should protect yourself unless you are hurting a human, blah blah.

总共有这么三条定律。

So there are, like, three laws.

那我们能不能让AI遵守这些规则呢？

And can we kind of have an AI observe these laws?

与其给它们一万个例子，不如直接告诉它们这些规则，绝对不能违反？

Instead of giving them, you know, 10,000 examples, can we just specify these are the rules and never to violate them?

所以我开发了一种方法，让另一个语言模型根据这些规则、问题和答案，判断这个回答是否符合规则。

So I developed a way where they actually use another language model that given these laws and then given a question and given an answer, tell the model if your answer is following these laws or not.

他们还发现，在一定规模下，大型语言模型会自然涌现出出色的判断能力。

And they also show that at certain scale, large language models have an emergent ability to be a really good judge.

其实，做判断比生成文本还要简单。

And it's actually easier to be a judge than to generate a text.

我觉得这很合理。

I think that's intuitive.

你明白吧？

You know?

做判断只需要读一段文字，然后给出是或不是就行。

For to be a judge, you just need to read a text and then give me a b c d.

你是在遵守规则，还是在违反规则？

Are you doing well against the law, or are you breaking the law?

这相对容易得多。

It is relatively easy.

这更像一个分类问题。

It is more like a classification problem.

但要真正生成内容，需要更多的推理和更强的认知能力。

But to actually generate, you you need a lot more reasoning and a lot more cognitive capacity.

因此，利用这种不对称性，你可以再次构建一个强化学习循环。

So kind of by exploiting this asymmetry, you can construct, again, a reinforcement learning loop.

到这个时候，你就不再是依靠人类反馈来训练了。

By this time, you are not doing a reinforcement learning from human feedback.

而是依靠人工智能的反馈来进行强化学习。

You are doing reinforcement learning from AI feedback.

所以这就叫 RL-AIF。

So kind of r l a I f.

对。

Yeah.

这些缩写词都快用滥了。

These acronyms going going out of time.

但别管这些缩写，关键是记住背后的直觉。

But, you know, you know, just keeping in mind the intuition, you know, regardless of the acronym.

你现在是让AI裁判来评判，它会读宪法，然后判断你的回答是否合规。

So you are now learning from an AI judge, and the AI judge reads the constitution and tell you kind of classify your response.

这样，你就可以让这两个模型相互对抗。

And in this way, you can pitch these two models kind of adversarially.

我觉得这就像新一代的GAN，生成器是语言模型，判别器就是这个裁判。

So I see this as a new generation of GAN or generative adversary generative adversarial network where the generator is the language model and the discriminator is the judge.

不过现在它们玩的游戏要复杂得多。

But now they're playing a much more complicated game.

通过这种来回互动，你可以让语言模型朝着你设定的损失目标对齐。

And by doing this, you know, back and forth and back and forth, you can align a language model, against the loss that you wrote down.

所以我觉得，当我用GPT-4的时候，明显感觉它更符合预期，也更遵守安全规范，比ChatGPT强多了。

So, I I think, you know, when I'm, trying out g p t four, I find it much more aligned and also much more observant to safety rules than chat g p t.

所以这算是一个附带的成果。

So this is kind of a side product.

你可以让它更精准地给出有用的回答。

You can align it to provide more useful answers.

你也可以训练它拒绝回答某些有害或危险的问题，而GPT-4在这方面要保守得多。

You can also align it to refuse to answer certain toxic or dangerous queries, and g v d four does that a lot more conservatively.

所以，确实有这么一种现象。

So, yeah, there's that phenomenon.

对。

Yeah.

我觉得这里的关键在于，生成式预训练变换模型蕴含了大量智能，但这种智能是无方向的。

I I think the sort of meta point here is that generative pre trained transformer models pack a lot of intelligence, but in an undirected way.

想让它们按我们想要的方式做事非常困难，因为它们几乎像是有自己的想法。

It's really hard to get them to do the thing that we want them to do just because they are they almost have a mind of their own.

所以我们现在正在不断发现各种方法，让这些模型表现得更可控，更适合我们想构建的应用场景。

And so I think we're just discovering all these techniques to to get these models to behave in a way that makes them useful and for applications we wanna we wanna build with them.

我觉得阿姆贾德其实提了一个特别有意思的观点。

I I think Amjad actually has raised a really interesting point.

我觉得与其说语言模型有自己的想法，不如说它们有太多想法了。

I just think instead of language model having a mind of their own, I think they have way too many minds.

没错。

Yes.

对。

Yeah.

因为它们是在整个互联网上训练的，而网上的人情绪、思维方式和世界观千差万别，模型同时从所有这些内容里学到了东西。

Because they are training on the entire Internet, and people have just the wildest distribution of emotions and reasoning and worldviews online and language model learned from all of them at the same time.

所以我觉得像原始的GPT-3这样的在线语言模型，就像是我自己的各种想法混杂在一起。

So I I see the online language models like the original GD three as kind of a juxtaposition of mine.

它可能汇聚了上百万种不同的思维模式。

It has maybe, like, a million different minds juxtaposed together.

但RLHF和RLAIF这些宪法式AI，正是试图把这些思维统一起来。

But ROHF and RO AIF, these constitutional AI, is trying to collapse these mines.

更像是

It's more like

就是这么回事。

the only exactly.

没错。

Exactly.

这有点像量子力学里的量子纠缠，只不过你强行让它们坍缩到某个确定的状态。

It's kind of like, you know, in in quantum quantum mechanics, you have quantum entanglement, but you kinda force them to collapse to a certain state.

这才是真正有用的AI助手。

That is actually a useful AI assistant.

这就引出一个问题了。

So that begs the question.

对吧？

Right?

那其他那些想法怎么办？

Like, how about the rest of the minds?

我觉得其他那些思维在别的场景下依然很有用，比如在创意类的环境中。

I I do think the rest of minds are still useful in another setting, maybe in a creative setting.

举个例子，如果你希望GPT-4扮演一个脾气暴躁的老人，它会直接拒绝，因为它已经被固化成一个AI系统了。

So, for example, if you want g p d four to play a role of, let's say, a grumpy old man, it it it will refuse to do so because it's kind of collapsed to the AI system.

但在讲故事、拍电影、写剧本或者导演这类工作里，你可能恰恰需要那些被ROI忽略或淘汰掉的其他思维模式。

But you could imagine that in storytelling in, you know, like, cinema or scripting or all of these or, like, movie directing, you may need all the other minds that ROI track or ROI kind of eliminated.

对。

Mhmm.

用这个角度去想，真的很有趣，谢谢你提到这一点。

So it's very interesting to to to think in this lens, and I'm glad you brought this up.

对。

Mhmm.

是的。

Yeah.

如果非要说一个缺点，至少在我看来，就是我们总是从一个极端跳到另一个极端。

Maybe if I have to bring up a shortcoming, at least, you know, in my humble opinion, is the fact that we jump from one extreme to the other.

所以我们从多种思维并存的状态，变成了一个非常僵化的系统。

So we go from this juxtaposition of several different minds into a very inflexible system.

如果你留意过最近几个月它的变化，就会发现，虽然我们错过了很多推特截图，但它变得越来越受限。

So if you especially followed, you know, how it evolved in the past few months, and there are a lot of Twitter screenshots that we missed, it has become more close and close.

越来越多的话题不能再谈了，但它又特别爱聊天。

There have been more topics that can't be covered anymore, but but but it's very chatty.

如果你不用它来闲聊，而是想干点别的，比如争论，它就没什么用处了。

So if you try to use it in any other way apart from the chats, you know, arguing, it doesn't really serve a purpose.

我觉得我们还得摸索出，到底该给大语言模型的思维设定多大的弹性空间，既要让它更接近人类的行为方式，又要保留所有必要的安全机制，才能真正投入生产环境。

So we I think we still have to find out what is the right level of flexibility in terms of confining the brain of the LLM to, you know, map more similar to what how a human would behave, and at the same time, you know, still putting all these safety guards that are needed to put such LLM in production.

这注定是一段漫长的旅程。

It it it's gonna be a journey.

在我看来，宪法式AI比我们之前的LHF是进步了，但离理想状态还差得远。

I think constitutional I AI, in my opinion, is a step forward compared to our LHF, but it's, you know, it's it's still not there.

而且我觉得，现在没人敢说它的代码能力比以前强了多少。

And I I I don't think anyone would dare to say that code is much better than at this point.

我觉得我们听到的基本上都是对这两个系统的负面反馈。

I think we heard mostly only the opposite feedback about those two systems.

是的。

Yeah.

如果你最近关注过，OpenAI 曾想弃用 Codex，结果遭到了研究社区的强烈反对。

I mean, the if you've seen recently, OpenAI wanted to deprecate Codecs, but they they got a lot of pushback from the research community.

他们对此很意外，因为 DaVinci 3 在编程方面其实比 Codex 更强。

And they were surprised by it because DaVinci three is actually better at code than Codex.

人们想要 Codex 的原因在于它不带指令，大家反而喜欢它这种无拘无束的风格。

The reason people wanted it is because it's not instructed, and they actually like the fact that it's not instructed.

DaVinci 3 的问题在于它太有主见了，而原来的 Codex 更像一个纯粹的、老实的代码补全工具。

So the shortcomings of da Vinci three is that, you know, it is too opinionated, whereas the original Codex was just, like you know, was just was just more honest completion engine.

对。

Yeah.

有意思的是，底层技术其实完全一样。

It's interesting because the underlying tech is exactly the same.

大部分训练数据其实是一样的。

The vast majority of the training data is the same.

你只是稍微调了几个参数，结果就完全不一样了。

You tweak just a few knobs, basically, and it is in a in a completely different way.

对啊。

Mhmm.

这跟普通软件工程师的预期完全不符，不管系统在某天有多大的限制。

So, like, that's not what you would expect from a standard software engineer, for example, no matter how rampant it will cap a certain day.

没错。

Right.

其实我们做过这个实验。

Actually, we did this

我们做了一个有趣的尝试，你知道的，米歇尔，既然你在帮忙，我们其实一直在自建一些代码模型，用于CodeComplete。

we did this fun experiment with so so as as you know, Michele, since you're helping, we're we're actually training self hosting a lot of our our sort of code models for CodeComplete.

但为了好玩，我们把ChatGPT直接集成到了IDE编辑器里，当代码补全引擎用。

But just for fun, we hooked up ChatGPT inside the IDE, inside the editor as a a completion engine.

结果表现得很差，因为它太执着于聊天模式了。

And it sucked really bad because of how so, you know, adamant about chatting it is.

不管你怎么引导它，它就是只想当个聊天机器人。

No matter how much you prompt it, it's it just really wants to be a chatbot.

所以我们考虑到速度和成本，就觉得也许它当补全引擎还挺有潜力的。

And so we're like because we're looking at the speed and economics, we're like, oh, maybe this is actually useful as, like, a completion engine.

结果发现，一旦你给这些模型设定明确指令，灵活性就大打折扣了。

Turns out that you're losing a lot of flexibility once you, you know, instruct these models.

是的。

Yeah.

对。

Yeah.

没错。

Yeah.

确实如此。

Yeah.

它逼着你把大量自然语言和代码混在一起，因为它总想解释自己生成的代码。

It it forces you tries to interleave a lot of natural language and code because it tries to explain the code that it's actually generating.

对。

Yes.

在聊天界面里，这种做法还挺管用的。

That that works well in a, you know, chat interface.

当然，这明显是个外来设计。

It's, of course, a visible foreign ID.

所以我们

So we

除非你是在做唐纳德·诺思那种文学编程。

Unless it's you're doing, you know, Donald Newth literary programming.

你知道那是什么吗？

Do you know that?

是的。

Yeah.

是的。

Yeah.

对。

Yeah.

所以除非你在做文学编程，否则通常没什么用。

So unless you're doing literary programming, yeah, typically, it's not useful.

也许这就是我们和你目前这种奇怪语法之间的区别。

Maybe that's where we're going apart from the crazy syntax you have at this stage.

我们不如直接让它模仿一下。

We should just condition it to, like, you know, mimic.

没错。

Exactly.

也许只需要

Maybe that's

我们只需要

all we need is

需要一些提示来提问。

some prompting to ask.

到那时候，如果丹尼真能模仿出来，那他简直就是超人类智能了。

Danny will really be superhuman intelligence at that point, yeah, if he's gonna mimic it.

没错。

Yep.

好的。

Alright.

既然我们在聊编程，不如咱们直接切入这个话题吧？

Given that we're talking about coding, you know, why don't we jump on on that?

咱们也聊聊人工智能在代码方面的应用吧？

And why won't why don't we cover a bit AI for code?

说真的，当年GPT-3在生成代码方面表现得特别惊艳，还有Codex带来的种种突破，都是我们合作中最让人兴奋的成果。

So to be fair enough, it was the most exciting applications of our alliance back in the days and the fact that g p d three was so powerful at generating code and, you know, and everything that we've we've been seeing with Codex.

过去这几年，这方面的进展可真不小。

A lot has happened in the in the past, you know, couple of years at this point.

模型在多种编程语言上的能力越来越强，不仅能生成代码，还能接手软件工程师每天浪费大量时间的那些琐碎工作。

Models are becoming more and more powerful across several different programming languages, not only in generating code, but also in taking over some of the ability to staff the software engineer waste hours a day on.

我们昨天或者今天早上看到了 Copilot 的新公告。

We have seen this Copilot announcements yesterday or this morning.

我都有点数不清现在冒出多少新东西了。

I'm I'm losing track of how many things are happening.

吉姆，咱们能不能简单聊聊这个？

Jim, maybe we can have a quick chat about it.

给咱们说说，他们这次发布最让人兴奋的地方是什么？

You know, give us the highlights of what they're excited about for their announcement.

是的。

Yes.

这是 GitHub 发布的公告。

So it is an announcement from GitHub.

叫 GitHub Copilot X，他们基于 GPT-4 加入了几项新功能。

It's called GitHub for pilot x, and they implemented a few kind of new components based on g v v four.

不过我想指出，这些功能其实都不算新，之前在 Replit 或者很多社区开发的 VS Code 插件里，这些想法早就出现过了。

And and I do want to point out that, like, none of these features are particularly new because we have seen, like, either in Replit or in many community built, like, Versus code plugins that many of these ideas have already floated around.

只是因为 GitHub 和微软能优先使用 GPT-4，而 GPT-4 在写代码方面特别出色。

Just that because GitHub and Microsoft, they have privileged access to g p d four, and g p four is exceptionally good at coding.

所以他们的工具用起来感觉更顺，性能更好，背后靠的是更强的代码语言模型。

So that's why, like, their their announcement, their tools, may may feel better in terms of performance because they are backed by a better language modeling coding.

我简单说说这几个新功能，Copilot 主要有四个重要更新。

So I will quickly kind of, talk about the features, and there are four kind of major features major new features to Copilot.

第一个是你可以直接和代码聊天。

The first one is that, you can chat with a code.

不过我觉得这谁都不觉得意外。

But I I think that is no surprise to anybody.

自从 ChatGPT 出来以后，任何文本、数据库都能聊天了，代码库当然也不会例外。

Like, when ChatGPT came out, you can like, any piece of text database will be chatable, and the code base is not an exception.

现在你可以直接跟代码对话，问它问题，不用再一头雾水地翻代码，尤其是面对不熟悉的项目时。

You'll be able to chat with a code, ask questions about it instead of reading through it and scratching your head, especially if the code base you are not familiar with.

第二个功能是拉取请求。

And the second feature is pull request.

Copilot X 能够查看拉取请求，给出评论，检查漏洞或者单元测试覆盖率之类的。

So the Copilot x can actually, like, look at pull request and maybe give you comments and check bug, check check for bugs or unit test coverage, things like that.

这挺有意思的，因为现在的代码模型不只是提升单个开发者的效率，还在改善人与人之间的协作。

I think this is interesting because now we're seeing these code models, code language models improving not just a single developer, but human collaboration.

对吧？

Right?

甚至能助力整个开源社区的运作。

Like, powering entire operate open source communities.

以前，一个仓库的维护者每天得审阅上百个拉取请求，决定哪些该合并，哪些不该，特别繁琐。

Because previously, like a human, a maintainer of the repo will need to go through hundreds of poll requests every day and then decide, you which one to add and all of that.

任何维护过大型开源项目的人，都知道这有多耗神。

And it's it is just anyone who has maintained huge, open source projects know, like, how mentally taxing that is.

如果你回复得慢，别人就会抱怨。

And if you don't get back to people quickly, they will complain.

对吧？

Right?

这就是 GitHub 上人们的工作方式。

That that's how people on GitHub work.

但现在我们有了这种增强的能力。

But now we can have this augmented capability.

也许一位维护者可以借助这些工具，每天快速浏览所有拉取请求，筛掉那些明显有 bug 或者跟当前任务清单不一致的请求。

Maybe a single maintainer will be able to use these tools to kind of get a quick daily summary of all the pull requests and, you know, rule out some pull requests that have obvious bugs or are obviously not aligned with the current, you know, to do list.

这样一来，人类在维护社区贡献项目时就能高效多了。

And, a human can be a lot more productive in maintaining these community contributor projects.

所以这是第二个功能。

So that's the second one.

第三个是全新的命令行界面，可以接入 Copilot。

And the third one is a new command line interface that can use Copilot.

像 Bash 这样的工具，用起来其实挺别扭的。

So, things like Bash is can be pretty awkward.

尤其是某些命令行工具，有些参数简直太冷门了。

And especially for some of the command line tools, some flags are really obscure.

你得上 Stack Overflow 搜半天才能找到答案。

You need to go to Stack Overflow and search for a long time to find the answers.

比如 FFmpeg 就是个典型例子。

An example is FFmpeg.

FFmpeg 的任何高级功能都得用特别生僻的参数，花的时间可太长了。

Any advanced capability of FFmpeg will require esoteric flags, and it's gonna take you a long time.

但现在你直接在命令行里问 GPT-4 就行，根本不用离开终端。

But now you can just ask g p t four in the command line without ever leaving a terminal.

最后还有一个功能：现在 GPT-4 支持的上下文长度长多了，能达到三万两千个标记，几乎是以前模型的十倍。

And the final feature is now, now that g p t four supports a much longer context length, which is up to 32 k tokens, that is, like, a lot more, like, order almost order of magnitude more than previous models, you can now fit entire documentations in content.

这太关键了，因为现在你不用做任何检索或搜索，直接把整个文档塞进上下文里，GPT-4 就能立刻回答关于这个工具的任何问题。

And that is huge because now, you know, without any, like, retrieval, like, retrieval or search or anything, you just fit in the documentation in context, and g p t four will immediately be able to answer questions about this particular, tool that is well documented.

这又是一个软件工程师必备的实用工具。

So that is another very useful tool to be added to a software engineer's toolbox.

所以，这些是我最期待的一些主要功能。

So, yeah, these are some of the major features I'm most excited about.

对。

Mhmm.

你知道这些功能是独立的产品，收费方式不同，还是只是Copilot的一些实验性功能吗？

Do you know if it's, like, a separate product that's gonna be charged differently, or is it is it, like, is it, like, more experimental Copilot features?

我不太清楚。

I'm not sure.

所以我把所有等待名单都申请了。

So I I tried to join all the waitlist.

顺便说一下，这些完整功能居然需要申请四个不同的等待名单。

And by the way, the full feature somehow require four different waitlist.

我不明白为什么。

I I don't know why.

不过就是这样。

But Yeah.

不行。

No.

如果你想用全所有功能，记得四个都报名。

Remember to sign up for all four if you want all of these.

光点第一个链接是不够的。

It's not, you know, sufficient for the first link.

对。

Yeah.

所以我也不太确定。

So I'm I'm not sure.

也许它们的收费方式会不一样。

Maybe they will be charged differently.

是的。

Yeah.

因为三万两千个token，基本上能把整个代码库都塞进提示里，这简直太惊人了。

Because 32 k tokens, you can, like, really fit the entire repo in the prompt, which is pretty pretty freaking amazing.

但这样成本会非常高。

But, also, it would be very expensive.

确实如此。

Indeed.

对。

Yeah.

对。

Yeah.

每次查询都要算一笔账。

Each single query would be math.

对。

Yeah.

我们是按三十次来收费的。

We charge on 30 to get on it.

所以这

So it's

是的。

Yeah.

但能够一次性调用整个代码库，主要是因为我们在Ghostrider里做的大部分工作其实就是写提示词。

But but being able to prompt the entire repo is just because most of the work we do in Ghostrider is prompting, really.

基本上全是提示词工程和上下文压缩。

It's like a lot of prompt engineering, a lot of context collapse.

我们的Ghostwriter聊天功能，经常要来回切换不同的上下文内容。

Our Ghostwriter chat, we do a lot of, like, paging out of different context.

我们会用很多嵌入技术，来判断你需要哪部分上下文。

We use a lot of embeddings to figure out what context do you need.

还会用不少经验规则，比如你当时在哪个文件里操作之类的。

We use a lot of heuristics, what file you were on, etcetera.

所以，我们做的这些事，让我想起以前——虽然那时候我还不是程序员，但我是九十年代计算机开始变得强大又便宜的时候才入行的。

So most of what we do it it sort of reminds me of like, I I wasn't programmer then, but, like, I, you know, I became a programmer when computers, like, were starting to become really powerful and cheap in the nineties.

不过我觉得，在七八十年代，你要是给Atari之类的老机器做游戏，就得使出各种招数，就为了把程序塞进32KB的内存里。

But I think in the eighties and seventies, when you're, like, making game for Atari or whatever, you you do all these hacks to fit fit into, like, you know, a 32 k memory or whatever.

对吧？

Right?

我觉得卡马克早期成名，就是因为他用特别高效的方式实现了滚动效果，做出了流畅的卷轴游戏。

And I think Carmack was really you know, one of his first claims to fame was building scrolling in such an efficient way to build scroll games.

那时候最厉害的程序员，其实就是靠在极有限的资源里折腾出成果。

And that's basically what, like, the best programmers were doing was, like, managing very little amount of resources.

所以今天这个上下文的问题，感觉就跟当年一模一样。

So the context issue today just feels like that.

现在大部分应用的瓶颈，都卡在能给大模型输入多少上下文。

Like, most of the applications are bottlenecked on how much context you can sort of put into the LLM.

还有成本问题，

And price I

我觉得我们确实认同。

think we Yeah.

我们同意，大模型就是新一代的计算机。

We agree that LLMs are the new computer of the Yes.

就是这十年。

Of this decade.

对吧？

Right?

我们现在正经历着和上世纪七八十年代一样的限制，也在学着怎么更好地使用它们。

And we're seeing all the limitations that we were facing in the '17 and the eighties, and we're learning how to program them correctly.

我很喜欢你的这个比喻，它让我想起了自己小时候刚开始写 BASIC 程序的时候。

So I love your analogy because it made me think when I started to write basic in my very young years.

我当时遇到的困难，完全一模一样。

I I face exactly the same struggles.

是的。

Yeah.

不好意思。

Sorry.

抱歉打断你，兄弟。

Sorry to interrupt you, bud.

不。

No.

当然。

Absolutely.

我的意思是，这正是问题的难点所在。

I mean, that's that's basically the the hard thing about it.

我的意思是，微软确实能优先接触到这些模型。

I mean, yeah, Microsoft has a privileged access to these models.

他们能优先访问模型的权重。

They have privileged access to the to the weights.

很明显，他们能以其他OpenAI用户做不到的方式进行优化。

Obviously, they can optimize it in ways, you know, other OpenAI customers can't really optimize.

所以对他们来说，可能更经济一些。

So maybe it's, like, more economical for them.

但作为应用开发者，棘手的地方在于：到底该优化到什么程度？

But the the tricky part as an application developer is, like, how much how much do you optimize?

因为你知道，价格和性能都会越来越好。

Because you know the price performance is gonna get better.

从商业角度看，你是不是得先亏点钱，让用户体验好起来，但心里清楚，以后利润自然会提升？

And and and, basically, like, maybe from a business perspective, do you, like, lose money on it in the short term in order to make sure your users are happy, but you know that the margins will improve in the future?

阿姆贾德，让我问你个关键问题——我们现在和即将推出的那些功能，对开发者来说简直太惊艳了。

Amjad, let let me ask you the right question because we we cover some amazing features that software developers are gonna have in the rents now or very soon.

我觉得我们都羡慕现在刚开始学编程的人，跟我们当年比，他们手里的工具多太多了，学习过程也远没那么挫败。

I think we all envy people that are starting to learn how to code today because compared to when we started, they have much more in their ends, and the experience will be much less frustrating.

你能不能预测一下，对那些一边学编程一边用AI的人，未来会怎样？像我们这样有经验的开发者又会面临什么？

Why don't you give me a prediction of what do you think is gonna happen for people that are learning to code together with AI and also what's gonna happen to more seasoned developers like us?

他们的生产力到底能提升多少？

How more productive are they gonna become?

我听过你的播客，那个‘十万倍开发者’的那期。

I I I heard your podcast, the 100,000 x developer.

我特别想听听你对这个的看法。

And what I would love to hear your take about this.

是的。

Yeah.

我觉得，设想出现千倍效率的开发者，并不夸张。

I I I think, like, it's not unreasonable to imagine a thousand x developers.

但在当前的大型语言模型和AI软件里，我还看不到这种可能。

And I don't see it in the current generation of LLMs and AI software.

我觉得，到这个十年末，我们可能会看到更多能替你自主行动的智能代理，而你则能站在更高的层面去掌控一切。

I think maybe by the end of the decade, we're gonna have more sort of autonomous agents that are acting on your behalf, and you end up sort of being at a much higher level.

所以我认为，程序员这个概念本身会发生变化。

So I think, like, the concept of the programmer will change.

比如现在，你确实还是得学编程。

Like, today, yes, you still have to learn how to code.

你可能更多时候是在审代码，而不是亲自敲代码。

You still maybe you're reviewing code a lot more than you're actually typing code.

但对顶尖程序员来说，亲手敲代码反而成了效率的瓶颈。

But, you know, for the best programmers, typing code is actually somewhat of a bottleneck.

比如卡马克这样的人，有人开玩笑说，写代码根本不是瓶颈，之类的。

Like, if you're Carmack, like, typing code like, you know, there's this meme that, you know, typing code is not the bottleneck or or whatever.

但事实上，那些最有生产力的人，想法多、目标多，真正限制他们的反而是敲代码的速度。

But, actually, some of the most productive people, like, they have a lot more ideas, and they have a lot more productivity and and ways things that they wanna achieve that actually typing code ends up becoming a bottleneck.

所以短期内，把Copilot、Ghostwriter这些工具看作打字助手，虽然听起来有点简化，但其实非常重要。

So in the short term, I think thinking about Copilot, Ghostwriter, etcetera, as typing aids are kind of you know, might seem reductionist, but it's actually very, very important.

对吧？

Right?

我觉得我们现在正进入一个新阶段。

I think we're entering a new phase now.

从打字助手开始，

So you go from typing aids.

下一步就是聊天式交互。

The next phase is the chat.

对吧？

Right?

现在你有个助手，能帮你找代码、回答关于代码库的问题，还能激发你的创意。

So now you have sort of an assistant that is that is helping you find code, that is helping you answer questions about your repo, that is helping you be creative.

比如，你可以跟Ghostwriter聊聊，说嘿。

Like, actually, like, being able to talk to Ghostragger and say, like, hey.

你觉得这个函数怎么样？

What do you think this function?

你觉得我该怎么处理它？

What do you think I should do with it?

和它一起头脑风暴，真的特别有意思。

And brainstorm with it, it's actually a very fun experience.

所以现在，它已经更像一个真正的助手了。

And so now you have something that's acting more like an assistant.

我觉得下一代可能会更进一步，谁知道呢？

I think the next generation and who knows?

也许今年就会出现。

Maybe it's this year.

也许再过五年吧。

Maybe it's in five years.

到时候会是一群AI在协同工作。

It's gonna be more a team of AIs.

我觉得它们会更像真人，出现在你的编辑器里，跟你聊天似的。

Like, I think they're gonna feel more like humans that you're gonna see them in your editor, more like, you know, talking to you.

你给它们分配任务，它们会看拉取请求、审查代码，甚至直接提交完整的PR。

You're giving them tasks, them looking at PRs and reviewing PRs, submitting entire PRs.

到那时候，你可能根本不用再写代码了。

And maybe at that point, you're not actually coding anymore.

你可能只是在审阅代码。

Maybe you're reviewing code.

你或许更多是在整体设计系统，但指挥着一支AI团队，替你处理那些繁琐的活儿。

Maybe you're sort of designing the system as a whole, but you are commanding an army of AIs that are doing a lot of the menial type work for you.

说到学编程，我觉得现在这体验有趣多了，因为你能很快做出点东西来。

In terms of, like, learning how to code, I think today, it's like, you know, the experience of learning how to code is so much more fun because you could do something really quickly.

我们现在看到各种讨论都在热传。

We're seeing all these threads.

推特上突然冒出大量关于Replit的讨论，大家都以为我们在搞什么营销活动。

There was such a explosion of Replit mentions on Twitter that people thought we were doing a campaign.

其实，Replit就是尝试这些AI技术最自然的方式。

That, basically, Replit is, like, the natural way to to try a lot of these AI technologies.

很多刚开始学编程的人，也能快速做出打砖块、小鸟飞飞，还有好多其他有趣的小游戏，以前这些可得花上好几年。

And you had people that were just learning how to code, able to build Pong, able to build Flappy Birds, able to build a lot of these interesting games that would have taken you years.

从想法到能摸得着、感觉得到、还能分享给别人的实际成果，这个时间正在急剧缩短。

And so the time from idea to something tangible that you can sense and you feel and share with someone else is collapsing.

这对人的动力来说特别重要。

And that's a very important thing for people's motivation.

以前学编程特别容易没劲，因为从写第一行代码到看到真正能用的程序，中间得等上几个月甚至好几年。

Part of the reason learning how to code was very demotivating because the the reward for seeing, you know, a real application in the world was, like, months to years away from the your first line of code.

所以在Replit，我们一直很看重一个问题：怎么把从想法到可分享的产品之间的时间缩短？

So at Replit, we always value this idea of, like, how do you reduce the time from an idea to a product or something that you can share?

而有了人工智能，这个时间几乎直接缩短到零了。

And with AI, it's really collapsing as in total to zero.

我觉得会有更多人开始接触编程。

And I think we're gonna see a lot more people getting into programming.

确实有些人对这项技术很抵触，有的是出于理念，有的就是不想改变自己的习惯。

I I will say that there are people that are very resistant to this technology, somewhat ideologically, somewhat just because they don't wanna change their ways.

我觉得他们真该重新考虑一下，不然迟早会被甩在后面。

And I think they're really like, they should reconsider because otherwise, they're gonna be left out.

我觉得这就像一股上涨的潮水，会带动所有船只前进。

I think a lot of it's it's a it's a it's a rising tide that will lift all boats.

但如果你的船不愿意随波逐流，那你就只能被落下。

But if if your boat is not willing to be lifted, you're gonna be you're gonna be left out.

是的。

Yeah.

我觉得我们经常听到生成式AI对繁荣带来的影响。

And I think we hear a lot of the impact on prosperity that the GI will have.

也许今天还看不出来，但我相信短期内，由于开发者效率大幅提升，变化会非常明显。

Maybe that gets from today, but I do think in the short term, what's gonna happen, thanks to developer becoming way more productive.

你知道，这种趋势是没法忽视的。

You know, it it can be ignored.

我们已经看到过去几十年软件和机器学习带来了多大的影响。

The already seen how much software and ML have done in the past few decades.

想象一下，未来十年超级开发者们能创造出多少东西，又能为世界带来多少财富。

Imagine what's gonna happen in the next ten years of superpower developers, how much they're gonna be able to create, how much wealth they're gonna, you know, be able to generate in the world.

这真是一个令人兴奋的前景。

So it's extremely exciting scenario.

吉姆，我最后问你一个问题，好让我们赶紧进入问答环节。

Jim, let me ask you the last question of the panel so that we can quickly jump to the q and a.

这会是那个老生常谈的百万美元问题。

It's gonna be the usual million dollar question.

假设一年后的今天，我们还坐在这里开同样的对话，又在讨论大语言模型。

So say we're gonna be here one year from today having the same panel, and we're gonna be talking about LLMs once again.

真让人意外。

What a surprise.

你觉得那时候我们会走到哪一步？

What where do you think we're gonna be at that point?

大语言模型会强大到什么地步？

How powerful LLMs would be?

它们会具备哪些能力？

What kind of capabilities would they have?

你觉得在未来十二个月里，它们会怎么发展？

And how do you see them evolving, you know, in the next twelve months?

这个问题太有意思了，我每天晚上都在想。

I think that's a fascinating question that I dream about every night.

好吧。

And okay.

我来谈谈我的预测。

So here's my prediction.

我觉得我们会看到更多模态更自然地融入大语言模型。

I think we'll see a lot more modalities weaved more naturally into LLMs.

GPT-4的视觉API只是个开始。

And g d four, the Vision API is just the first.

我提到过，视频和三维内容很快就会整合进来，其实NVIDIA那边已经有不少令人兴奋的进展了。

I mentioned that we'll have a video and three d integrated very soon, and I, like, there are already exciting works at NVIDIA doing this.

我敢肯定，其他地方也在考虑同样的事情。

I'm pretty sure, like, other places are thinking about the same thing.

整个研究圈都在往文本以外的模态靠拢。

The whole research community is gravitating towards modalities beyond text.

所以这件事几乎肯定会发生。

So that is almost for sure gonna happen.

接下来的一批模态会是行动类的，比如像记者型智能体，而不仅仅是语言模型。

And the next thing kind of the next set of modalities would be action modalities, kind of like journalist agents rather than just language models.

我说的记者型智能体，是指能配备执行器的具身AI。

And what I mean by journalist agent are embodied AI that can have actuators.

它们能以多模态方式感知世界，比如图像、三维视频，然后采取行动去探索环境、引发变化，最终完成任务。

Like, they can perceive the world in multimodality, you know, in image, three d video, and then they can take actions to explore the environment, enact changes, and then complete tasks.

在我看来，这完全涵盖了 ChatGPT、GPT-4 或 Claude 的所有能力。

And to me, that is a strict superset of char g b t or g b t four or Claude.

对吧？

Right?

根据《思考，快与慢》这本书，我们知道人类有两种思维模式：系统一和系统二。

Like, from the book, thinking fast and slow, we know that humans have both system one and system two thinking.

GPT-4 主要处理的是系统二，也就是那种有条理、深思熟虑的推理。

And what g b t four addresses is system two thinking, which is systematic, deliberate reasoning.

对吧？

Right?

比如写邮件、考试、编程，这些 GPT-4 已经让你惊艳的场景，都属于系统二。

Like, examples are writing emails, taking exams, coding, whatever g v four has already impressed you.

但系统一更像是一种本能反应，不需要太多有意识的思考。

But what system one does is more like a reflex, which doesn't require a lot of conscious thinking.

比如走路、跳跃，或者用五根手指抓起东西，这些都属于机器人技术。

Examples are like walking, jumping, or picking up an object using five fingers, and that is robotics.

实际上，现在的机器人技术比写代码难多了。

And, actually, robotics is somehow a much harder problem right now than coding.

这让我既高兴又有点失落。

And it's both, you know, happy and sad for me.

高兴是因为我还能再干一阵子。

Happy because I'll keep my job for a while.

我目前正在研究通用型智能体，比如让AI玩游戏，或者控制AI完成机器人任务，操纵机械臂和机械腿。

I am actively doing research on generalist agents, either, you know, controlling AI to play games or controlling AI to do robot tasks to control robotic arms and legs.

所以我挺开心自己还能继续干下去，但同时也有点遗憾，因为我们还是远远落后于语言模型。

So I'm happy I'm gonna keep the job for a while, but I'm also kind of sad that, you know, we're still lagging behind language models.

我们还没见过能当记者的机器人，比如从冰箱里随便拿点食材，就能给你煮一盘意大利面。

We have not seen journalist agents yet, like a robot that can go ahead and cook a spaghetti for you just from the ingredients in the fridge.

这事儿就是难得多，这正是经典的莫拉维克悖论。

It is just so much harder somehow, and that is the classical example of the Moravix paradox.

我们以为最难的事情，结果对人工智能来说反而更容易。

The things that we think are hardest somehow turns out to be easier for AI.

那些我们根本没意识到的直觉性思维，反而比理性思考更难复制。

Somehow system one that we don't even think about is harder than system two.

我觉得，GVD4接下来的能力和模态，会往这些直觉性功能上拓展。

And I feel that, you know, g v d four, the next set of capabilities and modalities will be adding things from system one.

但在进入直觉层面之前，我们还有很多理性思维的问题没解决，就像今天讨论会上提到的对齐问题、代码优化问题。

But before system one, we still have a lot of system two problems not yet solved as we discussed in the panel today about alignment, about better coding.

对吧？

Right?

我们能不能先让那些纯粹写代码的智能体，别老想着跟你闲聊？

Like, can we ask agents that are just pure coding agents, not trying to chat with you all the time?

能不能先把Replit提升到千倍效率？

Can we first make make Replit a thousand expat?

对吧？

Right?

这些本来就是特别难的问题。

Like, these are already very difficult problems.

接下来，它能不能处理来自全世界、来自YouTube这些地方的海量视频？

And next, can it process so much videos from the world, from YouTube, from all of these places?

它能处理所有的视频吗？

Can it process all the videos?

再往后，我们就要问了：它能不能控制实体身体？

And then after that, we'll start to answer, can it control physical bodies?

它能拥有实体吗？

Can it be truding bodied?

我就想要一碗香喷喷的肉丸意大利面。

And all I want is a delicious meatball spaghetti.

我同意。

I'm with that.

吉姆，我很好奇。

Jim, I'm curious.

是的。

Yeah.

有没有人试过直接对机器人做大规模无监督学习，就是把感知转化成标记，再把动作也转成标记，然后全扔进一个超大的模型里，看看会怎么样？

Like, has anyone tried to just, like, do massive unsupervised learning on robotics just like translating perception into tokens and and actions and just, like, stick it into, like, a big ass model and see what happens.

有人试过吗？

Was that tried?

当然试过。

Absolutely.

很多研究团队，包括我们自己，都在做这类尝试，比如谷歌机器人、英伟达，还有伯克利、斯坦福这些地方，现在都开始这么做了。

Like, a lot of researchers, including my own team, are trying these approaches, you know, like Google Robotics and NVIDIA and many Berkeley, Stanford, many many places are starting to do this.

但我觉得现在的瓶颈在于数据——网上文字和视频多得是，但机器人数据，比如控制信号、触觉感应这些，根本找不到。

But I think the bottleneck here is data because there are a bunch just way too much text and videos online, but there aren't any robot data, like control signals or touch sensing or all of these data online.

这些数据在网上根本找不到。

It's not readily available online.

所以我们得自己去收集。

That means we need to collect it ourselves.

但这样就很难规模化了。

But then you don't scale up.

机器人硬件太贵了。

The robot hardware are so expensive.

对吧？

Right?

就连工业界和研究机构，也买不起超过十几二十台机械臂。

Like, even the industry labs, research labs cannot afford beyond maybe a dozen robot arms or two dozens.

实在是太贵了。

It's just so expensive.

而且，物理世界本身就有速度限制。

And, also, you have a rate limit in the physical world.

像文本或者ROHF这类东西，只要你买更多显卡，就能一直扩展。

Like text or, like, ROHF, you can do it as long as you have you you as long as you buy more GPUs.

但在物理世界里，这根本行不通。

You know, you can't really do that in the physical world.

其实解决这个问题有很多方法。

And there are many ways kind of to address this.

一种方式是做模拟，不过我们今天可能没时间深入讲了。

One way is to build simulations, I suppose we don't have time to get into that.

但这本身是个很有趣的话题，关于怎么应对系统一的问题。

But that's a whole other interesting topic on how we address system one.

不过我觉得，今天大概会以一堆未解决的问题和对系统二的期待来收尾，因为我们还没达到那个阶段。

But I think, you know, we'll probably close today with many problem unsolved problems and aspirations on system two because we're not there yet.

最近在系统二里，增加更多模态、提升编程能力，其实蕴含着巨大的经济价值和实用潜力。

And I think there are so much economic value and utility just to growing more modalities and doing coding better in the system two recently.

是的。

Yeah.

谢谢你的提问。

Thanks for the question.

很好。

Great.

没错。

Absolutely.

我觉得我们这个小组真是太棒了。

And I think we're we're an amazing panel.

我们连续聊了一个小时，尤其是最后这部分。

We we talked one hour in a row, especially the the closing.

你懂的。

You know?

还有很多事情要做。

So much to be done.

是啊。

I know.

我们让更多人对人工智能产生了兴趣，未来会积极投身这个领域。

You know, we got more people excited to dive head first in AI and contribute to the field in the in the years to follow.

我们还剩几分钟。

So we have a few more minutes left.

所以呢，大家还可以多留一会儿，因为咱们收到了社区里好几个特别棒的问题。

So, you know, attendees can stick around for a while longer because we received several amazing questions from the community.

我挑几个说说，不按顺序，也不分轻重，就挑我觉得有意思的。

I will cherry pick a few of them, not in order or importance of how valuable they are.

主要是想感谢大家的积极参与，提的问题都很有启发。

I'm just trying to pick some of them because I found them fascinating, but thanks everyone for contributing.

有一个是来自Tinker的提问。

We have one from Tinker.

我觉得这个问题，还是直接问你吧。

I I think, you know, I I will address this to you.

Amjad问的是，AI的出现，会怎么改变我们开发和设计应用的方式？

Amjad is asking, how do we see the the process of developing and architecting your application changing because of AI?

你刚才聊那些聊天机器人的时候，其实已经简单提到过了。

And I think you briefly covered that when you were talking about those chatter chats.

像这种应用，一年前我们根本想不到会有人去做的。

So that's an kind of application that we we wouldn't have even thought about one year ago.

但确实如此。

But Yeah.

现在，我们的设计方式已经从底层发生了变化。

Right now, the the way we design has changed from the ground up.

对。

Yeah.

但我最近一直在思考的是，大型语言模型不仅帮助我们编写软件，还在帮助我们构建软件内部的各个组件。

But another thing that I've been really thinking a lot about is how large language models are not just helping us write the software, but also our component pieces inside the software itself.

对吧？

Right?

我来举个例子。

So, you know, I'll give you an example.

写代码的时候，我们经常会遇到解析问题。

A lot of times when you're writing software, you hit into a parsing issue.

这其实是个很常见的任务。

You know, that's a common task.

GPT-3 只要花十五分钟的提示，就能学会解析。

GPT three can learn parsing with, like, a fifteen minute prompting session.

要是你自己写一个高质量的解析器，就算你是资深工程师，也得花上至少一周时间。

To write, like, a really good parser, you know, it's like a week's work at least if you're a good good engineer.

所以很明显，软件工程师的铁律就是大家都爱偷懒。

So it just feels obvious that people are gonna just like the iron law of software engineers is that we're all lazy.

因此，大家自然会把这些经过特定提示的大型语言模型，嵌入到软件的调用链里。

And so it's just obvious that people are gonna plug in these LLMs prompted for a specific task as part of the software sort of call graph.

对吧？

Right?

但问题来了，这种做法的最佳实践到底是什么？

And so then but the problem is, like, what are the best practices there?

你怎么测试它？

How do you test it?

怎么让它变得可预测？

How do you make it predictable?

你怎么发布它？

How do you ship it?

你怎么开发它？

How do you develop it?

你怎么保证它可复现？

How do you make it reproducible?

你怎么围绕它协作？

How do you collaborate around it?

这一点，我们在Replit已经开始深入思考了。

And that's something that we're starting to think really deeply about at Replit.

我觉得很明显，未来每一段软件都会内置某种大型语言模型，来完成各种不同的任务。

And, you know, it it feels very obvious to me that in the future, every piece of software will have embedded in it some kind of large language model that's doing a lot of different tasks.

关键问题是，围绕这个该有哪些最佳实践？

The question is, like, what are the best practices around that?

那还要求软件必须一直连着互联网。

That would also require the software to be connected to Internet all the time.

如果你是在调用一个 API 的话。

Well, if you're if you're calling an API.

对吧？

Right?

没错。

Right.

是的。

Right.

如果你是本地运行，那肯定得在本地跑这个模型。

If you're running it locally, then that for sure if you're running the model locally.

没错。

Yeah.

我挺好奇你想怎么看待这个问题的。

It feels it feels like be curious to hear your thoughts about this.

我只是觉得，我现在每台电脑上都装了个小号的 Llama 或 Alpaca，纯粹玩玩。

But just seeing, like, I have now a a small llama alpaca on every computer that that I own just for fun.

我觉得用不了多久，这些模型就会无处不在了。

It just feels like pretty soon, like, we're gonna just gonna have these models everywhere.

但你得有显卡才能运行任何软件啊。

They still because you will need GPUs to run any software.

真的。

Please.

你不会觉得……

You you don't think Yeah.

你觉得大模型在CPU上运行不可能吗？

You don't think LMs on CPU is gonna be possible?

我觉得这会很困难。

I think it's gonna be difficult.

如果推理速度不是关键问题，那或许我们可以放宽这个限制。

Or if, I mean, if inference time is not a big bottleneck, then maybe we can relax that that kind of constraint.

但如果你对推理效率有要求，那还是得靠GPU或者类似的加速器。

But if you do require some kinda inference efficiency, then I think GPUs or equivalent accelerators will be required.

明白了。

Gotcha.

对。

Yeah.

好消息是，现在大部分消费级设备都配备了一定性能的加速器，就连M1、M2这些芯片也一样。

The good news is most of the consumer order at this point has some level of good accelerators and even the m one, m twos or, you know Yeah.

NVIDIA的显卡已经内置在笔记本里了。

NVIDIA GPUs are embedded in in Yeah.

笔记本电脑。

Laptops.

当然没那么强，不像H100那种级别，但跑小模型还是够用的。

Not as powerful, of course, a a 100 h, one hundreds, but, you know, still good enough to run small models.

没错。

Right.

我也完全同意，迟早这些大模型——不如叫轻量模型吧，因为越来越这样了。

And I totally agree that eventually those LLMs or let's call it an SLM because more as Exactly.

我们

We

是的

Yeah.

它们会在我们的本地应用中找到自己的位置，我们能为它们注入强大的能力。

They will find out their spots, you know, in our local applications, and we're gonna be able bring in superpowers to them.

吉姆，我有个问题想问你。

Jim, I have a question for you.

结合之前提到的《哈利·波特》的例子，我觉得这简直再合适不过了。

I think it's gonna be a perfect fit given the Harry Potter example from before.

当然。

So Sure.

我们之前聊了很多关于提示工程的事。

We have been talking a lot about prompt engineering.

现在这已经成了一门专业技能。

It has become a professional at this point.