第528期：利用LLM构建模块开发Python应用

然后就像是

And it was just like

UV。

UV.

UV。

UV.

怎么

How

你能说多少次UV？

many times can you say UV?

是啊。

Yeah.

那些反馈评论。

The feedback comments.

我是说，那是个不确定的新人。

I mean, it was a new someone new who wasn't not sure.

对吧？

Right?

他们已经看过了所有不同的版本，确实如此。

They've they've seen all the different ones and yeah.

我是说，我见过最酷的对比是在一条推特上，有人发帖说，假设你现在身处数据领域，十年前的工具是什么？

So I mean, the coolest comparison I've seen I think it was a tweet someone posted, but like, it it definitely like, suppose you're in the data field right now, like, what what are the tools ten years ago?

现在的工具又是什么？

What are the tools now?

确实感觉像是这样，好吧。

It definitely does feel like, okay.

以前我们用PIP，现在用UV。

Before we had PIP, now we have UV.

以前用PANDAS，现在用POLARS。

Before we had PANDAS, now we have POLARS.

以前用MATLAB，现在用Alteran。

Before we had MATLAB, now we have Alteran.

以前我们用Jupiter，现在有了Marimo。

Before we had Jupiter, now we've got Marimo.

你可以看到一种代际更替，不仅体现在笔记本工具上，还包括包管理器和数据框库这些方面。

You you can kinda see a generational shift, not just on the notebook side of things, but, like, on the package manager, on the data frame library.

我们都在从前人那里学习，这感觉就像是...

We're all learning from the previous generation, and it kinda feels like.

是啊。

Yeah.

完全同意。

Absolutely.

这是个令人惊叹的时代。

It's, it's an amazing time.

我在Python领域的每一天都比前一天更令人兴奋。

I every single day in Python is just more exciting than the day before.

没错。

Yeah.

不过，我也得提一下，特别是在波兰这件事上，Rust语言也占了不小的比重。

Although, I should also mention, like, especially with the Polish thing, there's a fair bit of rust too.

这也带来了变化。

That also makes a difference.

是啊。

Yeah.

没错。

Yeah.

当然。

Of course.

所以在Flyby之前，我是说，你提到过你的YouTube频道。

So before Flyby, I mean, you talked a bit about your YouTube channel.

你评测人体工学键盘。

You review the ergonomic keyboards.

给那些现在才收听或观看的人介绍一下，你的背景布置非常棒。

You have for those people who are just listening now, watching, your background is super nice.

你在这里的资历相当扎实。

Your bonafis are solid here.

是的。

Yep.

是的。

Yep.

但你有很多不同的键盘。

But you got a bunch of different ones.

我个人觉得，你甚至可能会把我踢出俱乐部，如果我...

And I personally also I don't you might even just kick me out of the club if I

那应该是微软Sculpt键盘。

should do That's the Microsoft Sculpt.

对吧？

Right?

还有很多其他款式。

There's a lot one.

Sculpt人体工学键盘

Sculpt Ergonomic.

我特别喜欢它是因为可以取下这个小巧的现代部件，反向旋转后塞进背包随身携带，薄得跟剃刀似的。

And I like it so much because I can take this little modern thing off, and it's, like, razor thin if I could rotate in reverse and jam it in my backpack and take it with me.

但我已经饱受重复性劳损问题困扰二十五年左右，情况曾非常严重。

But I've also had RSI issues for twenty five years or something, and it was really bad.

以前我用笔记本键盘打字，后来换成了这种专用键盘。

And I switched I used to type them on like laptops and stuff, and I switched to just something like this.

现在每天打十小时字都没问题。

And I could type ten hours a day, and it's no problem.

而以前如果强迫自己在笔记本上高强度打字，可能不到一周就会严重到无法工作。

Whereas before, if I if I were to force to type full on on a laptop, I would probably be, like, unable to work in weeks, if not a week.

非常糟糕。

So bad.

我们有个

We had an

两周前在旧金山有个场外活动。

off-site in San Francisco, so was there, like, two weeks ago.

虽然很有趣，但机场安检看到这个可能会觉得是什么奇怪的外星设备。

A lot of fun, but, like, I'm not gonna bring like, airport security is gonna look at this and wonder what ex what like, what kind of weird alien device this is.

对吧？

Right?

所以我想，好吧。

So I figured, okay.

我就把它们留在家里。

I'll leave those at home.

只带我的普通笔记本电脑。

Just bring my normal laptop.

一周后，我的手腕又开始不舒服了。

And after a week, I'm feeling it in my wrist again.

就是那种

Like, it's

不太好。

not good.

是啊。

Yeah.

这确实需要认真对待。

It's it's something to take serious.

不过我也得说，因为孩子和妻子都生病了，我在家没能做太多运动。

Although, I will also say, like, because the kid was sick and the wife was sick, so I wasn't able to do a whole lot of exercise at home.

也要做运动。

Also do exercise.

所以那也确实很有帮助。

So that that also totally helps.

如果我必须的话，当然。

If I had to, like, Sure.

如果你喜欢，可以买个符合人体工学的键盘。

Buy an ergonomic keyboard if you like.

可编程键盘确实很棒。

Programmatic keyboards, they're great.

但如果你以此为借口不锻炼，那就大错特错了。

But if you're gonna do that as an excuse not to do exercise, you're doing it wrong.

这是我想补充的一点。

That's the the one thing I do wanna add to that.

有很多拉伸运动你可以做。

There's lots of stretches you can do.

注意休息很重要。

Taking breaks matters.

锻炼很重要。

Exercise matters.

但我认为那些键盘确实是必备功能。

But I think those keyboards are an essential feature.

它们确实很有帮助，而且有时候不完全是治疗手段，更多是预防措施。

They they they do they do totally help and also it's not sometimes it's not so much a healing measure, it's more of a preventative measure.

是的。

Yes.

百分之百赞同。

100%.

我想对所有正在收听、年轻且自认为坚不可摧的朋友们传达这个信息。

And I would like to put this message out just to everyone listening who is young, absolutely indestructible.

拜托了。

Please.

我知道这些带有弧度设计的键盘用起来有点麻烦。

I know they're a bit of a pain in the butt with the warrior curviness.

但绝对值得骄傲地说：我从未有过任何重复性劳损问题。

It is so worth it to just say, I've never had any RSI issues.

虽然我用着这种奇怪的键盘被人取笑，但我根本不在乎。

I just use this weird keyboard and people make fun of me, but I don't care.

对吧？

Right?

你知道，这总比说'我正在努力'要好得多。

You know, that's that is a better thing than, like, I'm working on it.

是啊。

Yeah.

哦，就像疼得厉害，你懂吧？

Oh, like hurts up, you know?

嗯，另外现在尤其是我们有了像Claude这样的工具。

Well, another thing is also especially now we've got stuff like Claude.

对吧？

Right?

我只想指出这方面的一个要点。

I just want if I can point out, like, one thing on that front.

这些人体工学键盘通常可以按需编程。

So these ergonomic keyboards, you can often program them as you see fit.

所以你可以设置比如按这个键就输入字母k。

So you can say, like, if I hit this key, it's k.

但如果我长按它，它就会变成command键或任何你想要的指令。

But if I hold it down, it becomes command or whatever you like.

这也意味着你可以自定义宏或快捷键映射。

But it also means that you can map macros or shortcuts.

所以每当我按下这个按钮，一个叫MacWhisper的应用就会启动。

So whenever I hit this button, an app called MacWhisper boots up.

它会开始录制我的声音。

That's gonna record my voice.

我爱MacWhisper。

Love MacWhisper.

超爱它。

Love it.

而且它还有各种替代版本。

And then there's alternative variants for it.

我相信Linux平台肯定也有类似工具。

I'm sure you've got stuff for Linux as well.

但它主要的功能就是提供非常出色的语音转文字服务。

But the main thing it does is just really good speech to text.

然后每当我讲完话，它会立即粘贴到我当前所在的任何地方。

And then whenever I'm done talking, it's just immediately gonna paste whatever I'm in.

所以，哦，这实际上非常棒，因为现在的键盘快捷键只需要我按住大拇指，而不是要做出某种奇怪的、像爪子一样的手势去够键盘。

So oh, that's actually very nice because now the keyboard shortcut is just me holding my thumb down instead of some sort of weird, like, claw I gotta eject to my keyboard.

突然间，这不一定只是为了方便。

And suddenly, doesn't necessarily become a convenience thing.

它也变成了一种高级用户的功能。

It also becomes kind of a a power user thing.

就像，如果你能把所有操作都按照你喜欢的方式映射到键盘上，你就能真正定制你的电脑使用体验。

Like, you can really customize your computer experience if you can really map everything to your keyboard just the way that you like.

这就像是拥有VIM，但是适用于所有应用程序。

It's like having VIM, but for all the apps out there.

好的。

Okay.

这挺酷的。

That's pretty neat.

是啊。

Yeah.

我把大写锁定键重设为启动MacWhisper的快捷键。

I've remapped caps lock to my activate MacWhisper.

如果我按住大写锁定键，就能进行语音输入。

If I hit caps lock and hold this down, then I can dictate.

我知道电脑早就有听写功能了，但以前真的很糟糕。

And I know computers have had dictation for a long time, but it's been really bad.

对吧？

Alright?

Mac OS或Windows自带的听写功能并不好用，尤其是当你试图说缩写词时，比如PyPI。

The built in dictation to Mac OS or Windows isn't great, especially when you try to talk acronyms like, hey, do PyPI.

然后它就...

And it's like

是啊。

Yeah.

MacWhisperer虽然还没做到尽善尽美，但不得不说它的质量确实有几次让我感到惊喜，真的。

Mac But MacWhisperer is it's it's not, like, all the way there, but I will say it's surprise it surprised me in quality a few times, definitely.

对。

Yeah.

没错。

Yeah.

而且那个应用里你还能手动设置替换规则。

And that app, you can go in and actually set replacements.

比如说，如果你觉得以后会输入这个，就让它自动替换成那个。

Like, if you write if you ever think you're gonna write this, write that instead.

所以我给'TalkPython'设置了规则，因为它老是被识别成驼峰式连写。

So I've done that with, like, TalkPython because it always just turns it into CamelCase combined.

我就想，这可不行。

I'm like, no.

那些是两个分开的，你知道的，随便啦。

Those are two separate you know, whatever.

你可以对它进行一定程度的自定义

You can sort of customize it a

是啊。

Yeah.

我觉得我最喜欢的一个例子，MacWhisper到现在还是会把scikit learn翻译错。

And I think my favorite one, I think MacWhisper actually gets this one wrong still.

每当我写scikit learn这个非常流行的数据科学包时，它总是翻译成'心理学家学习'。

But whenever I write scikit learn, which is a very popular site like data science package, it always translates it to psychologists learn.

这感觉就有点奇怪

Which, you know, feels kind of like weird in

就像，我永远不希望你打出这一串词，因为如果真需要，我宁愿每次手动输入。

there, like, I never want you to ever write this this series of words because if you're if I have to, I'll just type it the one time.

这不是我常用的

It's not my common

嗯，其实我现在处于一个尴尬境地——我经常输入‘psychic learn’，多到它几乎要变成一个问题了。

It's well, so I'm not it's it's it's one of these moments where actually I type psychic learn often enough where it's like almost becoming the issue.

所以我几乎要手动为所有我接触的这些奇怪的开源包添加规则了。

So I'm like at the verge of adding these rules manually for all these weird open source packages that I interact with.

没错，认真对待人体工程学吧各位。

Yeah, it's Take ergonomics serious people.

这是我想强调的一点。

That's the one thing I wanna say.

你不必总是买超级昂贵的键盘。

You don't always have to buy a super expensive keyboard.

如果你想探索可编程键盘，因为它们支持自定义功能，那是个绝佳的理由。

If you wanna explore programmatic keyboards because you can customize things, that's an excellent reason.

但要记得休息、做做运动，保持健康。

But take a break and do exercises and just be healthy.

这才是赢得马拉松的方式。

That's how you win a marathon.

确实如此。

Yes, that's for sure.

虽然这不是文森特的键盘评测马拉松，但让我们以一个问题收尾：如果让你从墙上挂着的键盘中选一把，你会用哪款？

Now this is not the Vincent's keyboard review marathon, but let's wrap it up with what if if you could take any keyboard hanging around on your wall there, which one would you use?

嗯，这里有四对这样的键盘，所以可能是这一把。

Well, so there's four so I there's four pairs of these, so it's probably this one.

我这儿还有几块其他也很棒的键盘。

So in my there's a couple of other boards that are great too.

这块键盘不是我推荐给所有人的款式。

This board is not the board I would recommend to everyone.

但如果你有严重的手部劳损问题，我认为Glove 80确实是最佳选择。

But if you have serious RSI issues, I do think the Glove 80 is your best bet.

它的造型简约，可能是最适合大多数手型的工学设计。

It's it's simple in terms of the shape, it is probably the most ergonomic shape for most hand sizes.

是啊。

Yeah.

好的。

Okay.

太棒了。

Awesome.

好吧。

Alright.

那么，我们切换到讨论编程话题吧

Well, let's switch over to talking about programming

是的。

Yes.

关于大语言模型（LLMs），不是编程LLMs。

LLMs with LLMs, not programming LLMs.

我想正如你已经提到的，你写了这门名为《Python的LLM构建模块》的课程。

And I guess like you already called out, you wrote this course called LLM Building Blocks for Python.

非常有趣的课程。

Super fun course.

真的很棒。

Really neat.

这门课程相当简短精炼，主要探讨如何可靠地将某种LLM集成到你的代码中。

It's pretty short and concise, and it really talks about how can you reliably add some kind of LLM into your code.

我想你在课程中讨论的内容，无论是自托管模型还是OpenAI或Anthropic的模型，都同样适用。

I guess what you're what you're talking about in this course really applies regardless of whether it's a self hosted one or it's OpenAI or Anthropic.

对吧？

Right?

你可以根据需求选择使用哪种LLM。

You can have there's some some choices you can make on which LLM to use.

对吧？

Right?

所以，我设计这门课程的核心想法是：LLM某种程度上是个构建模块，但它与编程中常规的构建模块截然不同。

So so the, like, the main idea I have with the course was, like, an LLM is a building block at some point, but it's very unlike a normal building block when you're dealing with code.

因为通常编程时，你向函数输入内容，就会得到一个明确的输出。

Because normally with code, you put something into a function and one thing comes out.

但在这种特殊情况下，你向函数输入一个东西，却完全无法预知会得到什么输出。

But in this particular case, you put a thing into a function, you have no idea upfront what's going to come out.

不仅如此，即使你两次输入相同的内容，也可能得到不同的输出结果。

And not only that, but you put the same thing in twice and something else might come out as well.

这意味着你需要以更谨慎的态度来考虑这个工具的使用。

So that means that you're gonna wanna think about this tool a bit more defensively.

这是个奇怪的构建模块。

It's a weird building block.

就像你必须给它筑起护城河，否则这个构建模块很可能会做出你意想不到的事情。

It's like you have to put a moat around it because otherwise the building block's gonna do stuff you don't want it to do almost.

其中有些涉及语法问题，有些关乎如何理解这些乐高积木般的组件，还有些则是关于如何通过统计测试来验证它是否大致符合预期的方法论。

And like some of that is syntax, some of that is how do you think about these LEGO bricks, and some of it is also just what is good methodology in general to like statistically test if the thing is doing roughly what you want it to do.

不错。

Nice.

这就是核心要点了。

That that's the that's the gist of it.

不过确实。

But Yeah.

不错。

Cool.

是的。

Yeah.

我从课程中提炼了你谈到的几个观点、概念和工具。

I pulled out a couple of ideas, concepts, and tools that you talked about throughout the course.

即便没上过这门课，这些内容也会很有用。

And you don't have to have taken the course to have these things be useful.

我只是觉得聊聊你提到的这些话题会很有趣。

I just thought it might be fun to riff on some of the things you touched on here.

我觉得最有趣的是——这大概有半年了吧。

The main thing I think will be fun is it's been half a year, I think.

看看其中有多少内容仍然站得住脚会很有意思。

It will be fun how much of it is still intact.

我认为大部分内容肯定仍然适用，但看看能否找到任何可能过时的部分会很有趣，这样就能知道世界是否变化得很快。

I think most of it still definitely is, but it might be fun to sort of see if we can find anything that might be dated just to see if the world has moved on quickly.

我想可能只有一点，但我们能...我只是有点好奇？

I can I think there's only one thing, but we can I'm just kinda curious?

让我们...来看看吧。

Let's let's see.

是啊。

Yeah.

好的。

Alright.

那我们就对此保持警惕吧。

Well, let's keep our our radar up for for that.

这绝对是变化更快、更容易过时的内容，但我认为它整体上还是站得住脚的。

It's definitely something that's more changing quicker and and has a higher likelihood of being dated, but I think it holds up pretty well.

没错。

Yeah.

好的。

Like, okay.

我记得我们强调过的一点是，你需要做些像缓存这样的处理。

So we like, one of the things I remember emphasizing is you wanna do some stuff like caching.

比如说，如果你有一个函数，并且你用了LLM来处理它。

So, like, if you let's say you've got a function and you use an LLM for it.

我们尽量简化这个问题。

And let's let's keep it simple.

假设我们只是做摘要。

Let's say we're just making summaries.

比如输入一段Python播客内容，输出一个单句摘要。

So talk Python episode paragraph goes in, single sentence is supposed to come out.

类似这样的。

Something like that.

嗯。

Mhmm.

好的。

Okay.

嗯，你可能会有个循环，比如先试一次，用一种设置跑一个LLM，再用不同设置试另一个LLM来生成所有数据。

Well, you might have a loop and you're gonna do like maybe one pass, try one LLM with one type of setting, try another LLM with different type of settings to generate all this data.

如果使用LLM这种计算成本较高的工具时，出现这种情况就太遗憾了。

It would be a shame that you're gonna use an LLM, which is kind of an expensive compute thing.

如果你不小心输入了相同内容，结果要支付双倍费用，那可就太糟心了。

If you put the same input in by accident, and then you incur the cost twice, like that would really stink.

所以有件事你始终要考虑，就是缓存策略。

One of the So things you always wanna do is think a little bit about caching.

有个叫Disk Cache的Python库我一直爱用，强烈推荐大家了解一下。

And there's a Python library called Disk Cache that I've always loved to use, and I highly recommend people have a look at it.

Michael，我记得你之前的课程里也用过的。

I think Michael, you've also used it in one of your courses before.

关键在于

The trick

我们得好好聊聊这个。

is We have to talk about this.

这太棒了。

It's so good.

确实非常棒。

It is so good.

它是SQLite，而且超级好用。

It is SQLite, and it's so good.

它甚至比SQLite还要好。

It is even better than SQLite.

简直好得难以置信。

It is unbelievably good.

这还得感谢你。

And I have you to think.

我虽然知道它，但，嗯，也就那样吧。

I I knew about it, but, like, yeah, whatever.

然后我看到你用了它之后，我就觉得，太天才了。

And then after I saw you use it, I'm like, genius.

这确实是天才之作。

It is it is genius.

没错。

Yes.

所以这就像是拥有了磁盘缓存，但它同时也存储在磁盘上。

So it's like having disk the cache, but it's also on disk.

因此，即使你重启Python，基本上缓存中的所有内容都还在。

So if you were to restart Python, you still have everything in cache basically.

而且它是一个类似SQL的数据库，所以你随时可以检查里面存储的所有内容。

And it's a SQL like database, so you can always inspect all the stuff that's in there.

如果你愿意，还可以做些高级操作，比如为每个对象设置生存时间，这在Docker容器的Web应用中也能实现。

If you wanted to, you can also do fancy things like add a time to live to every single objects, and this is something you could do in a Docker container for a web app.

但处理LLM时最棒的是，你总能事后比较不同LLM的表现，比如这个LLM和那个相比如何？

But the main thing that's always nice when you're dealing with LLMs, is you always want to be able to say in hindsight, like, okay, how did this LLM compare to that one?

你需要比较输出结果。

You want to compare outputs.

然后只需在函数上添加一点装饰器，就能将其存入SQLite，这样你就彻底解决了这个顾虑。

And then just writing a little bit of a decorator on top of a function is the way to put it in SQLite, and you're just done with that concern.

这简直太棒了。

That is just amazing.

而我们正在直接使用这个缓存。

And we're using this cache directly.

如果你在命令行使用Simon Willison的LLM工具，那里也有一个机制可以让你按需将数据存入SQLite。

If you're using LLM by Simon Willison from the command line, there's also a mechanism there so that you can get it into SQLite if you wanted to.

所以这也是个可以考虑的功能。

So that's also a feature you could consider.

当然，

Of course,

他肯定会往SQLite里塞点东西。

he's gonna put something in SQLite.

就像，他不可能写出一个不往SQLite里塞东西的库，他的数据集项目就是这样的。

Like, he couldn't write a library that doesn't put something in SQLite, his dataset projects.

那可是西蒙·威利森啊。

It's Simon Willison.

就算是周日，他也会把SQLite塞进SQLite里。

He'll put SQLite in SQLite if it's a Sunday.

没错。

Exactly.

但Discache——如果你还没用过它的话——绝对是那种让你觉得‘因为兜里装着这个工具，就能解决更多问题’的库。

But Discache if you but if you haven't used Discache before, it definitely feels like one of these libraries that because I have it in my back pocket, it just feels like I can tackle more problems.

简而言之就是这样。

That's the short story of it.

再说一次，课程里只是很基础地使用了它，但要知道你在LLM里做的每件事其实都只需要执行一次。

And again, the course uses it in a very sort of basic fashion, but knowing that everything you do in an LLM can only only needs to happen once.

如果你只想用一次，这能省下不少钱。

If you're interested in using it once, that just saves you so much money.

是啊。

Yeah.

而且它真的非常实用。

And it's it's so useful.

你在课程中做的一件事是，你说过，好吧，我们要存储的值的键将由模型、模型设置和提示组成。

So one of the things you did in the course is you said, alright, the key for the value that we're gonna store is gonna be the model, the model settings, and the prompt Yes.

作为一个元组。

As a tuple.

对吧？

Right?

大致是这么个意思。

Something along those lines.

然后你就用这个作为键。

And then you use that as the key.

所以如果其中任何一项有变化，模型会变吗？

So if any of those variations change, does the model change?

设置改变了吗？

Do the settings change?

或者说，任何变化都意味着那是一个完全不同的请求。

Or like, anything, it's that's a totally different request.

然后你只需存储响应，就这么简单。

And then you just store the response, and then boom.

如果你用相同的模型、相同的提示和相同的设置提出完全相同的问题，为什么还要花十秒钟等待、浪费金钱并造成环境负担，而实际上你可以在微秒内得到答案呢？

If you ask the exact same question of the same model with the same prompt with the same settings, why do you need to go and wait ten seconds and burn money and environmental badness when you could literally within a microsecond get the answer back?

是的。

Yeah.

而且在此基础上还有一个巧妙的技巧可以用。

And there's also a fancy and there's a trick you can do on top.

有时候你会想说，这里还存在一个统计层面的问题。

So sometimes you wanna say, well, there's a statistical thing also happening.

所以有时候我希望针对一个输入存储大约10个输出，这样我就可以查看所有这些输出，或许之后还能进行标注。

So sometimes I wanna have one input and actually store maybe 10 outputs so I can look at all these outputs, maybe annotate that later.

解决这个问题的方法就是在元组里加个整数，这样你就能存储多个输出结果了，如果你真想这么做的话。

And the way you solve that is you just add an integer to the the tuple, basically, and then you're also able to store many outputs if you really like.

对。

Right.

天空才是极限。

Sky's the one.

第八个。

The eighth one.

随便吧。

Whatever.

嗯哼。

Uh-huh.

是的。

Yeah.

不。

No.

这很酷。

It's cool.

而且很灵活。

And Flexible.

是啊。

Yeah.

它还能做一大堆非常非常神奇的事情。

And it does a whole bunch of neat things that are really, really wild.

比如看起来就像，嘿，我把这个值放进去，它就存起来了。

Like so it looks like just, hey, I put this value into it, right, and it stores it.

它非常强大，因为可以跨应用执行。

It's really powerful because across application executions.

所以，比如你在笔记本里缓存那个响应，然后做些测试，如果之后回来重新启动笔记本或内核之类的，它不像LRU缓存那样。

So, like, maybe if you're caching that response in your notebook and what are you doing some testing, if you come back later, you've start the notebook back up or restart the kernel or whatever, like, it's not like a LRU cache.

它能记住是因为数据存储在本地SQLite文件的临时存储里，这太棒了。

It remembers because it's stored somewhere in, like, temporary storage in a local SQLite file, which is amazing.

它还有一些有趣的设计理念。

It also has interesting ideas.

我不太确定具体在哪里，但它确实有不同类型的缓存机制。

I'm not sure really where they are, but, like it has different kinds of caches as well.

所以你可能在里面存储了大量数据。

So maybe you're storing a ton of stuff in there.

因此它基本上会实现内置的分片功能。

And so it'll do basically built in sharding.

哦，是啊。

Oh, yeah.

你可以看到

You can see the

轻量级文件，而且效果非常非常好。

light files and it's really really good.

这是个非常深奥的库。

It's it's this is a deep library.

这不仅仅是，哦，是的。

This is not just, oh, yeah.

就像是，哦，你在办公桌前吗？

It's like, oh, are you at the desk?

所以这个库最酷的地方在于它确实功能深入，但如果你只想把它当字典用，也完全可以。

So so the fun the cool thing about that library is it really does go deep, but if you really just wanna use it as a dictionary, you can.

这就是我真正喜欢它的地方。

Like, that's the thing that I really love about it.

从各方面来说，你完全可以把它当作字典来用，完全没问题。

It for all intents and purposes, you just treat it as a dictionary and you're good.

或者把它当作函数装饰器使用，同样简单好用。

Or use it as a decorator on a function, and again, you're good.

所以，这又是一个小技巧——只要你了解这个库，就能在常规Python开发中事半功倍。

So, like, again, it's it's one of those little hacks where, oh, if you just know about the library, just become more productive at typical Python stuff.

是啊。

Yeah.

我来告诉你一个我正在实际使用它的场景。

I'll give you a place where I'm using it actually.

我在一些LLM相关的项目中用到它，就像你提到的原因那样，当我针对语言模型编程时。

I I use it on some LLM stuff, like when I'm programming against an LM for exactly the reason you did.

因为如果输入相同，就没必要重复提问。

Because if you have the same inputs, don't ask the question again.

直接给你答案就行。

You just here's the answer.

这真的非常快。

It's real real fast.

但如果你转到，比如说，Talk Python这边来看看。

But if you go over to, like, talk Python let's see here.

以嘉宾部分为例。

Go over the guests, for example.

对吧？

Right?

所以这里有一大堆嘉宾。

So there's a bunch of guests.

这位是文森特。

And here we have Vincent.

不是那个文森特。

Not that Vincent.

也不是那个文森特。

Not that Vincent.

这才是你。

There you are.

那个文森特。

That Vincent.

这些文森特都是谁啊？

Who these Vincents?

他们到处都是。

They're all over the.

是啊。

Yeah.

我大概有560位客人左右。

I have like 560 guests or something.

数量相当多。

There's a lot.

但在这里你会注意到，如果你查看这个东西的源代码，到处都是，只要有图片，末尾就会带这个小缓存ID。

But in here, you'll notice that if you go into the view the source on this thing, like, all over the place, anytime there's a picture, it'll have this little cash busting ID on the end.

嗯哼。

Mhmm.

本地加载时没问题，因为你可以直接看文件，知道它只是个文件，所以启动时缓存是一样的。

And that's fine when it's served locally because you can just look at the file and go, it's just a file, so it was the same cache at startup.

但如果它来自类似S3 blob存储，应用重启后，我怎么知道那是什么？

But if it comes from like a s three blob storage, you know, and the app restarts, how do I know what that is?

就是说，它必须重新下载整个...

Like, it has to go it would have to redownload the entire

过程

process of

博客

the blog.

快看这个

It's so check this out.

对

Yeah.

所以我刚添加了 对

So I just added Yeah.

对

Yeah.

对

Yeah.

对

Yeah.

这感觉像是在正规CDN和从S3获取之间还有某种中间状态。

So this feels like there there's something between the proper CDN and then getting it from s three.

就是说，有时候你需要一个介于两者之间的解决方案，而这个缓存可能正好适用。

Like, there are these moments when you wanna have something that's kind of in between, and then this cache could actually be a good solution for it.

没错。

Exactly.

这个网站的工作原理是它有一个磁盘缓存。

So what the site does is it has a disk cache.

每当它需要引用外部资源时，就会下载一次，计算哈希值，然后存入磁盘缓存——除非背后的资源有变动。

And anytime it has it says, hey, I wanna, like, refer to a resource that's external, it'll download it once, compute the hash, and then store it in the disk cache unless you change something behind it.

这样就能自动利用这个机制，确保资源永不陈旧，即使是从类似S3这样的远程服务获取，也能瞬间加载。

And so it's, like, automatically using this, and it makes everything, like there's never stale resources, and it's instantly fast even if they're served out of, like, something remote, like s three equivalent.

那你们...

And do you Yeah.

你们还做直播吗？

Do you also do a So to live?

存活时间也是，比如每天允许刷新一次之类的，我猜？

It is also time to live is also, like, every day is allowed to refresh once or something like that, I suppose?

对，是的。

For yeah.

对于S3的内容，我没有设置，因为我配置了所有管理功能，只要我通过管理界面更改了内容，它就会从缓存中删除。

For the s three stuff, I don't because I've set up all the admin functions that if I ever change one through the admin, it deletes it out of the cache.

明白了。

Gotcha.

明白了。

Gotcha.

所以它是内部一致的。

So it's like, it's internally consistent.

但对于其他内容，比如从字典中解析出的描述信息，它们会有一个大约一天的存活时间。

But for, like, other things, like, it re if it parses something out of the, say, the description which sits set in the dictionary, that stuff, it's just got a time to live of, like, a day or something.

有很多这样的设置。

It's got, like, a there's a bunch of those.

所以我正在使用所有这些不同的地方。

So the I'm using all these different places.

哇哦。

And wow.

这太棒了。

It's so good.

我只是想道声谢谢，因为我知道我们今天会讨论这个，因为这实在太棒了。

I just wanted to say thank you because I knew we're gonna talk about it today because this is so good.

是啊。

Yeah.

我知道。

I know.

这简直——如果它是标准库的一部分，我一点都不会感到惊讶。

It's it's it like, it's it's one of the if it was part of the standard library, I would honestly not be surprised.

就像，这也是关于它的故事。

Like, that's also the the the story with that.

不过，SQLite确实很棒。

But, yeah, SQLite is great.

磁盘缓存也很出色。

Disc cache is great.

它用起来像字典，但功能却强大得多。

It's it feels like a dictionary, but it gives you so much more.

这太棒了。

It's great.

这就是我能说的全部了。

That's the the only thing I could say about it.

观众可能在想：等等，

So people are probably like, wait.

我以为我们在讨论大语言模型？

I thought we were talking about LLMs.

是啊。

Yeah.

有趣之处在于，有这么多有意思的工具，甚至不需要用到智能代理AI，也不仅仅是那些专为LLM问题设计的酷炫库和工具，它们同样能应用于其他领域。

We interesting things is because there's all these interesting tools, and it's not even about using agentic AI, and it's not not like, there's really cool libraries and tools that you can just apply to this LLM problem, but also apply everywhere else.

对吧？

Right?

这是我在使用LLMs开发软件时发现的一个普遍现象。

It's one of those so it's one thing I have found with LLMs in general if you're building software with it.

一方面，我认为如果你是个资深工程师，它会非常有用，因为你会知道类似‘我需要一个缓存’这样的概念。

On the one end, I think it can be very helpful if you're like a proper senior engineer kind of a person, because then you know about things like, oh, I want a cache.

什么样的缓存最实用？

What's a pragmatic cache?

顺便说一句，你也可以选择Redis。

And you can also pick Redis, by the way.

如果你当时用了Redis，同样也能解决问题。

If you would have used Redis for this, that could have also worked.

只是我觉得确实如此。

Just that I think Sure.

本来也没问题。

It would been fine.

是啊。

Yeah.

本来也没问题的。

Would have been fine.

我喜欢的是不需要服务器。

What I like is there's no servers.

我不需要操心服务器。

I don't have to deal with the servers.

我喜欢这样，它就是个文件。

I like, it's just it's a file.

确实如此。

It's very true.

这样更简单。

It just keeps it simpler.

这完全正确。

It's totally true.

但我想在这里强调的是，你之前作为专业工程师的经验仍然有助于编写优秀的LLM软件。

But I the the point I wanted to get it here, like, your previous experience as a proper engineer will still help you write good LLM software.

不过，从类似的角度来看，我也认为我们确实有这一代的数据科学家，可能还有数据工程师、数据分析师这类人才。

However, from a similar perspective, I also think that we do have this generation of data scientists and maybe data engineer, data analyst kind of person.

以分析的方式思考，对算法输出保持相当批判性的态度。

Thinking analytically, being quite critical of the output of an algorithm.

我想说，在当今这个时代，具备这种素质也很重要。

That's also a good bone to have in this day and age, I would say.

因为在我那个年代，我确实构建过几个推荐系统。

Because if I've built a couple of recommenders back in my day.

到现在差不多有十年了。

Like, it's been a decade now.

但当你构建推荐系统时，确实会学到一点：你会面临这样的问题——'嘿，我给用户推荐了这个，他们点击了'。

But like, one of the things you do learn when you're building a recommender is that you're stuck with this problem of, hey, I gave this user this recommendation and they clicked it.

如果我向他们推荐了其他东西，他们会点击那个吗？

Would they have clicked this other thing if I would have recommended it to them?

突然间你面对的是一个高度随机且难以预测的系统，你必须对测试和比较这些不同算法的方式保持严格态度，你会想再三考虑如何进行AB测试。实际上，就像磁盘缓存作为工具很有用一样，脑海中掌握一些统计学方法论也会对你有所帮助，因为比较LLMs很大程度上就是在进行评估工作。

And suddenly you're dealing with a system that's like really stochastic and hard to predict and you have to be kind of strict about the way you test and compare these different algorithms and you wanna think twice about the way you AB test these things and oh, actually, just like Disk Cache is useful as a tool, having a little bit of methodology statistically in your mind will also help you because comparing LLMs, a lot of it is doing evaluations.

就是要在这方面保持严格。

Like being being kind of strict about that.

这也是我在课程中努力展示的——如果你对评估工作非常严格，你就会发现对于某些问题，使用scikit learn仍然更合适，因为你通过评估会发现，在scikit learn那边的数值表现更好。

And that's also what I try to do in the course, I try to just show you that if you're really strict about the evaluations, then you can also learn that for some problems, you're still better off using scikit learn because you just evaluate it and then you learn that like, the number is better on the scikit learn side of things.

是啊。

Yeah.

这时候你就会觉得，'哦，也许我做错了'，或者你花了一大笔钱运行昂贵又缓慢的LLM，结果发现还不如用一个简单的预测模型。

That's that's when you feel like, oh, maybe I did it wrong or you you paid a bunch of money to run expensive slow LLMs and you're like, I would just use a predicament.

哇。

Wow.

你这么说真有意思。

So it's funny you say that.

其实我最近一直在和荷兰一些公司里从事大语言模型工作的人交流。

So I'm I'm I've actually been talking to a bunch of people that do LLMs at companies here in The Netherlands.

你知道，去参加个PI数据会议，给他们灌点啤酒让他们酒后吐真言。

And you know, you you go to a PI Data, you go to a conference, and you give them just enough beer so they're honest.

就是那种情况。

You know, that that kind of a situation.

保密协议的门帘稍微掀开了一点，诸如此类。

The NDA curtain opens just a little or whatever.

再说了，这个游戏的关键就在于模棱两可。

Plus, moldeniability is the name of the game in this one.

没错。

Yes.

但听到的故事还挺鼓舞人心的——他们都说现在有AI项目预算了，所以我们会尝试各种大语言模型，或者投资评估之类的工作。

But then the stories you hear, which I did find somewhat encouraging, is they do all kind of go, well, there's budget now to do AI stuff, which means that we we try out all the different LLMs or we'd be like invest in evals and that sort of a thing.

有趣的是，我们还会放些基础模型进去作为必须超越的标准基准。

And funnily enough, we also put some base models in there, like as a standard benchmark that we should beat.

我最近多次听到这样的故事：由于LLM和AI的热潮，在实施并完成所有基准测试后，人们发现AI反而让Scikit Learn在许多地方投入了实际应用。

And it kind of, I've heard a story a bunch of times now that because of the hype around LLMs and AI, after it was implemented, after they did all the benchmarks, it turns out that AI is the reason that Scikit Learn is now in production in a bunch of places.

spaCy的情况也类似，因为很多人发现：当你提供足够训练数据后（这些数据无论如何你都需要用于评估），spaCy模型或轻量级模型的表现与LLM模型相差无几。

And it's also the same thing with spaCy because what a lot of people do learn is that, hey, if the spaCy model or the lightweight model, so to say, is somewhat equivalent to the LLM model after you give it enough training data, which you do want to have either anyway because you need that for evaluations.

通常这些模型更加轻量化，而且总能保持输出的一致性。

Well, typically those models are more lightweight and they will always produce the same output.

相同的输入永远会产生相同的预测结果。

Same thing goes in, same prediction will always come out.

你可以轻松地将其隔离部署，放在普通虚拟机上运行也完全没问题。

And you can really fence it off, have it on like a normal VM, that's also totally fine.

哦还有另一个优势，它的运行资源消耗极低。

Oh, and another benefit, it's super lightweight to run.

你只需要一个Lambda函数就能搞定。

You just need a Lambda function and you're good.

完全不需要GPU或者支付高昂的云服务账单之类。

As opposed to like a GPU or like a huge cloud bill or something like that.

所以我听到的一些故事确实表明，这些语言模型也在帮助推动标准的数据科学工作。

So some of the stories that I'm hearing do suggest that, okay, these LMs are also helping out like standard data science work, if it were.

仅仅因为管理层现在确实想认真对待这项投资，并真正希望进行实验。

If only because management now really does want to be serious about the investment and really wants to do the experiments.

本段TalkPython内容由Nordstellar为您呈现。

This portion of TalkPython to me is brought to you by Nordstellar.

Nordstellar是Nord Security家族推出的威胁暴露管理平台，该团队也是NordVPN的开发者，平台整合了暗网情报、会话劫持防护、品牌滥用检测及外部攻击服务管理功能。

Nordstellar is a threat exposure management platform from the Nord Security family, the folks behind NordVPN, that combines dark web intelligence, session hijacking prevention, brand abuse detection, and external attack service management.

保障团队和公司安全是一项艰巨挑战。

Keeping your team and your company secure is a daunting challenge.

这就是为什么你需要Nordstellar的支持。

That's why you need Nordstellar on your side.

它提供全面的服务组合、监控和警报系统，以降低您遭遇数据泄露和攻击的风险，并在事件发生时立即采取行动。

It's a comprehensive set of services, monitoring, and alerts to limit your exposure to breaches and attacks, and act instantly if something does happen.

其运作机制如下：Nordstellar能检测到员工和消费者凭证的泄露情况。

Here's how it works: Nordstellar detects compromised employee and consumer credentials.

它能检测到信息窃取日志和暗网来源中被盗的认证cookie，标记受入侵设备，无需在应用中添加额外代码即可减少多因素认证绕过导致的账户接管攻击。

It detects stolen authentication cookies found in Infostealer logs and dark web sources, and flags compromised devices, reducing MFA bypass ATOs without extra code in your app.

Nordstellar会扫描暗网上针对您公司的网络威胁。

Nordstellar scans the dark web for cyber threats targeting your company.

它监控论坛、市场、勒索软件博客以及超过25,000个网络犯罪电报频道，提供可预警且可搜索的上下文信息，您可以将这些信息路由到Slack或事件响应工具中。

It monitors forums, markets, ransomware blogs, and over 25,000 cybercrime telegram channels with alerting and searchable context you can route to Slack or your IR tool.

Nordstellar还提供品牌和域名保护功能。

Nordstellar adds brand and domain protection.

它通过视觉相似性、内容相似性和搜索透明度日志检测域名抢注和仿冒网站，并在全网范围（包括社交媒体和应用商店）执行更广泛的品牌滥用下架，从而降低用户遭受钓鱼攻击的风险。

It detects cyber squats and look alikes via visual, content similarity and search transparency logs, plus broader brand abuse takedowns across the web, social and app stores to cut the phishing risk for your users.

他们不仅会就冒充行为向您发出警报，还会处理并管理下架流程。

They don't just alert you about impersonation, they file and manage the removals.

最后，Nordstellar对开发者非常友好。

Finally, Nordstellar is developer friendly.

它提供平台和API两种使用方式。

It's available as a platform and an API.

无需安装任何代理程序。

No agents to install.

如果安全对您和您的组织至关重要，不妨了解一下Nord Stellar。

If security is important to you and your organization, check out Nord Stellar.

请访问talkpython.fmnordstellar。

Visit talkpython.fmnordstellar.

链接可在播客播放器的节目说明和剧集页面中找到。

The link is in your podcast player show notes and on the episode page.

请使用我们的专属链接talkpython.fm/nordstellar，这样他们就知道您是通过我们了解到他们的服务。

Please use our link talkpython.fm/nordstellar so that they know that you heard about their service from us.

您知道现在是一年中的什么时候吗？

And you know what time of year it is?

现在是深秋时节。

It's late fall.

这意味着黑色星期五也即将到来。

That means Black Friday is in play as well.

Nordstellar团队给了我们一个优惠码：Black Friday twenty。

So the folks at Nordstellar gave us a coupon, Black Friday twenty.

就是Black Friday（全大写连写）加上twenty two zero。

That's Black Friday, all one word, all caps, twenty two zero.

使用该码可享受8折优惠。

That grants you 20% off.

如果你近期准备注册他们的服务，不妨使用优惠码Black Friday twenty，省下20%的费用。

So if you're gonna sign up for them soon, go ahead and use Black Friday twenty as a code, and you might as well save 20%.

优惠有效期至2025年12月10日。

It's good until 12/10/2025.

感谢Nord安全团队对《TalkPython》节目的支持。

Thank you to the whole Nord security team for supporting TalkPython to me.

可以说我们的计划是引入AI功能，用户会想先看看效果如何，所以我们进行了测试，然后

You could say we're gonna use your mandate is to add AI and then you go user like, well, we got to see how well it's working so we tested it and then

我们需要将其与基准指标进行对比，而实际上这个基准...

We have to compare it to something of a benchmark, and then the benchmark actually

这个基准测试又快又几乎免费，效果也差不多好。

The benchmark's fast and effectively free, and about as good.

所以，我们没问题。

So, we're good.

我们只是

We're just

打算这么做，你懂吧？

gonna do that, you know?

我觉得是的。

I think it's Yeah.

不过没关系。

But it's fine.

有些组织需要来自上层的强制指令才能推动事情。

Some organizations need, like, mandate from above in order to get something done.

而这次LLM热潮，至少确实引发了自上而下的强制指令。

And this LLM craze, if nothing else, does seem to have caused the mandate from above.

我确信如此。

I'm sure it has.

所以这是件好事。

So that that's a that's a good thing.

亲爱的听众，这也是你需要牢记在心的一点。

Something I would keep in the back of your mind as well, dear listener.

有时候这种授权也能让你有权限去做其他事情。

Like, sometimes that mandate can also give you permission to do other things.

是啊。

Yeah.

我已经很久没在大型企业类公司工作了。

Not not working for a large enterprise type company for quite a while.

我猜想自己对这些来自高层的指令如何转变相当迟钝。

I've I imagine I'm pretty blind to how that that is that is transforming, you know, directives from the top.

但我敢肯定老板们都在说：'我在用ChatGPT，它比我们的软件好太多了'。

But I'm sure bosses are like, I'm using ChatGPT and it's so much better than our software.

我们该怎么应对这个问题呢？

What are we gonna do about that?

你知道吗？

You know?

是啊。

Yeah.

嗯，我在开发者工具领域工作，既会与大公司也会与小公司打交道。

It's well, I mean, I'm in the developer tooling space, I still talk to big companies and small companies as well.

你会注意到大公司的运作方式确实与小公司不同。

And you do notice that big companies, they work differently than small companies.

这无疑是事实，无论好坏。

That is certainly true, for better or worse.

比如，大公司也有其合理之处——如果我是银行，遵守某些规则确实是必要的，也确实如此。

Like, there's also good there's also good reasons why bigger com like, if I'm a bank, like, I'm I'm pretty sure it's a good idea to also have some rules that I have to abide by, which Oh, it is.

没错。

Yeah.

我更多是在想，高层管理人员与产品之间有多脱节，以及他们多大程度上只是在发号施令。

I'm just thinking more of, like, how disconnected is the higher level management from the product and, like, how much do they just, like, dictate a thing.

好的。

Okay.

那我们来谈谈一些工具吧。

So let's talk about some of the tools.

你主要关注的事情之一就是使用Simon Willison的LLM

One of the things you primarily focused on was using Simon Willison's LLM

是的。

Yes.

库。

Library.

给我们讲讲这个东西。

Tell us about this thing.

关于这个库的有趣之处在于，它实际上更像是作为一个命令行工具设计的。

So the funny thing about that library is that it's actually kind of more meant as a command line utility.

我认为这基本上就是他设计的入口点。

I think that's kind of the the the entry point that he he made.

但他也确保围绕它提供了一些Python API接口。

But then he also just made sure there was some sort of a Python API around it.

在看过那个库并尝试了其他所有库之后，我很高兴这么说——这是句赞美——我发现Simon Willison的LLM库是目前为止最‘无聊’的一个。

And after looking at that library and also after playing around with all these other libraries, and I'm glad to say this, but this is a compliment, I just found the LLM library by Simon Willison by far to be the most boring.

我这么说完全是褒义的。

And I mean that really in a good way.

它毫不花哨，只专注于少数功能。

Just unsurprising, only does a few things.

而这些少数功能的表现都非常稳定可靠。

The few things that it does is just in a very predictable way.

特别是在快速原型开发时，我感觉很棒的是它确实提供了能让你快速搭建的积木块，但又不会让你陷入可能最终束缚你的抽象层中。

And especially if you're doing rapid prototyping, what I felt was just kinda nice is that it does feel like you have building blocks that allow you to build very quickly, but it doesn't necessarily feel like you're dealing with abstractions that can sort of wall you in at some point.

所以对于入门级的'Hello World'来说，它就是用非常Pythonic的方式完成一些事情。

So for a hello world getting started, just do some things in a very Pythonic way.

这个‘无聊’的库确实很管用。

This boring library really did the trick.

而且我很高兴地告诉大家，它至今仍是我常用的库。

And I'm also happy to report it's still a library that I use.

它绝对是我工具库中的常备选项。

Like, it's still definitely a library in my tool belt.

每当我想快速实现某些功能时，这绝对是我的首选工具。

Whenever I wanna do something real quick, that is definitely a tool that I refer to.

其中有一点

And one of the things

那么它算是一种对不同LLM的抽象封装吗？

that So is it kind of a is it kind of an abstraction across the different LLMs and that kind of stuff?

比如我想在Anthropic和OpenAI之间切换，这会有影响吗？

Like, I wanna talk to Anthropic versus OpenAI, does that matter?

我认为这个库的构建方式也体现了良好的设计品味。

So the the way that the and the way that this is built is also, I think, with good taste.

作为一个库，你不想做的是声称要支持世界上所有的库。

So what you don't want to do as a library is say like, I'm gonna basically support every library under the sun.

因为作为一个单独的维护者，比如Simon Willison，你会被这些不同的供应商淹没。

Because as a single maintainer, in the case of Simon Willison, you're just gonna drown in like all these different providers.

所以他选择的是建立一个插件生态系统。

So what he went for instead was a plug ecosystem.

插件生态系统的缺点是，你将维护特定来源插件的责任转嫁给了其他维护者。

At a downside of a plug in ecosystem is that then you defer the responsibility of, like, maintaining a plug in for a specific source to another maintainer.

但可能会有Mistral的员工来开发Mistral插件。

But you might get someone who works at Mistral to make the Mistral plug in.

也可能会有OpenRouter的员工来开发OpenRouter插件，等等。

And you might get someone who works at OpenRouter to make the OpenRouter plug in, etcetera.

这样你就能以一种不错的方式分散工作量。

So you do distribute the workload in kind of a nice way.

而且所有主流模型都得到了支持。

And all the bigger models are definitely supported.