Pieter Wuille 解释集群内存池 | SLP730

我甚至可以说，这就是比特币采用工作量证明的全部原因。

I would go as far as saying this is the entire reason why Bitcoin has proof of work.

拥有工作量证明的意义在于，任何人都可以加入挖矿市场。

The point of having proof of work is that anyone can join the mining market.

比特币的去中心化和抗审查性，最终在情况严重恶化时才真正体现出来。

Bitcoin's decentralization and censorship resistance ultimately boils down if things go really wrong.

最终的保护措施是，你可以成为自己的矿工，即使亏损，也希望不会亏得太惨。

Final protection is you can become your own miner, maybe even at a loss, but hopefully not at a big one.

是什么激励着现有的大型矿工接受每一笔交易？

What is it that incentivizes existing large miners to basically accept every transaction?

因为他们知道，如果他们开始可靠地审查某些有经济需求的交易，总会有人冒出来继续挖这些交易。

It is the knowledge that if they start reliably censoring some transactions that have economic demand, someone will just pop up and mind them anyway.

大家好，欢迎回到斯蒂芬·利维拉播客。

Hi, everyone, and welcome back to Stephan Livera podcast.

今天，我邀请到了皮特·武耶。

Today, I'm joined by Pieter Wuille.

他是一位资深的比特币开发者和研究员，目前在Chaincode Labs工作。

He is a long time Bitcoin developer, researcher, and he is working at Chaincode Labs.

我们今天将讨论一下Cluster Mempool，这是一个即将登陆比特币核心v31新版本的项目，我认为是这样。

And we're gonna be talking a bit about Cluster Mempool, which is a a new project that is landing in the new version of Bitcoin Core v 31, I believe.

没错。

Correct.

所以，Pieter，欢迎来到节目，我们先从为什么开始吧。

So, Pieter, welcome to the show, and let's start with start with why.

你为什么会对Cluster Mempool这个项目感兴趣？

Why were you interested in this project of Cluster Mempool?

是的。

Yeah.

你称它为一个新项目。

So you call it a new project.

我们大约三年前就开始了这个项目，当时直接的动机是Suhas在Chaincode Labs做的一场演讲，谈到了当前内存池设计中存在的某些问题。

We started this, just about three years ago and the immediate motivation for this project was a talk that Suhas gave here at Cinco Labs about some of the existing issues with the current mempool design.

直接的动机是这样一个例子：在当前的内存池中，你实际上可能会陷入一种状态。

The immediate motivation was this example of actually you can end up in a state today in a mempool.

显然，这种情况不太可能发生在正常情况下，而更可能出现在一种病态的、恶意构造的场景中——当内存池填满时，最先被驱逐的交易，恰恰是你最希望打包的那笔交易。

Obviously not likely would need a pathological, adversarially constructed situation, but where the very first transactions that would be evicted if the mempool fills up, the first thing that gets evicted is actually the very first transaction you'd want to mine.

这显然是不可取的。

And this is obviously undesirable.

这对矿工来说是不利的，但更糟糕的是，当网络无法合理判断交易的盈利能力时，问题会更加严重。

It's undesirable for miners, but it's actually far worse than that when the network is sort of unable to reason about proper profitability of transactions.

因为显然，无论是不是矿工节点，你都希望被驱逐的交易大致对应于矿工最后才会包含的交易。

And because obviously you'd want eviction and mining, even if you're not a mining node, like you would want the things you evict to roughly correspond to the very last things a miner would want to include.

因此，我们最初的动机是想说明：当前的设计在许多方面都存在缺陷。

And so that was the motivation to show like actually the current design we have is broken in many ways.

这并不一定意味着可以被利用，但它确实使得我们很难对系统行为进行合理推理。

That doesn't necessarily mean exploitable, but it sort of makes it hard to reason about things.

我们希望解决这个问题，于是设计了一个很棒的方案，最终它演变成了一种彻底取代内存池内部所有机制的方案，同时也改变了相关的中继策略。

And we wanted to address that, came up with a cool design, and it grew into something that ended up replacing everything in the mempool, basically, internally touching on some relay policies that change along with it.

是的，所以现在已经合并了。

And yeah, and so it's been merged now.

它将被包含在 Bitcoin Core 31 中，预计下个月发布，我想。

It will be in Bitcoin Core 31, which is slated to be released next month, I think.

所以非常令人兴奋

So pretty excited

对此。

about that.

太好了。

Great.

为了帮助听众理解，这个项目对普通比特币用户有什么意义？

And so just to motivate this for listeners, why does this project matter for everyday Bitcoin users?

他们可能会看到什么影响，或者说，嗯，是什么影响。

What is the impact that they might see or the know, what is yeah.

简而言之，普通比特币用户会在这里感受到什么影响？

Basically, is the impact that an everyday Bitcoin user will see here?

希望什么都不会发生。

Hopefully nothing.

我认为这个举措的动机更多是出于网络长期健康的问题，而不是直接改善用户的体验。

I I think the the motivation is is more a long term health of the network question than it is directly improving things for users.

现在，他们可能会看到一些变化。

Now, they might see a few changes.

最明显的变化是，如今，在当前的比特币核心版本中，我们有包含祖先限制和后代限制的中继策略。

The most noticeable, I think, is how today, so up to the current Bitcoin Core release, we have relay policies that include ancestor limits and descendant limits.

这意味着任何交易及其所有未确认的祖先交易合计不得超过25笔。

That means any transaction together with any and all of its unconfirmed ancestors cannot be more than 25.

同样，任何交易及其所有未确认的后代交易合计也不能超过25笔。

And similarly, a transaction together with all its unconfirmed descendants cannot be more than 25 transactions.

有时候你会看到这种‘剥离链’：有人发起一笔交易，把找零发回给自己，再用这笔钱进行另一次支付，又发回给自己。

And this sometimes you see these peeling chains of someone makes a transaction, sends a change back to themselves, use the chain to do another payout, sends it back to themselves.

希望他们会把这些交易合并成一笔，但如果他们没有这么做，这种链条在任何时刻都会被限制在25笔交易以内。

Hopefully they would batch this and turn it into a single transaction, but if they don't, this this sort of chain is limited to 25 transactions at any given time.

并不是说无法继续拆分，而是你需要等待其中一些交易确认后，后续的交易才能传播开来，而Cluster Mempool改变了这一点。

It's not that it's impossible to peel further, but you'll need to wait for some of them to confirm before the later ones propagate across And this changes with Cluster Mempool.

不再有祖先或后代的限制。

Are no more ancestor or descendant limits.

取而代之的是集群限制。

Instead, are cluster limits.

可以把这想象成继续沿用家族树中祖先和后代的类比。

Think of it as a Continue with the analogy of ancestors and descendant in a family tree.

集群就是最广泛的扩展家庭概念。

Cluster is just the widest possible extended family thing.

它就像你的父母和孩子、祖父母和孙子女，还有叔伯姑姨、侄子侄女，以及他们的父母和子女。

It's like your parents and children, your grandparents and grandchildren, also aunts and uncles, niece and nephew, their parents, their children.

任何通过‘是父节点’或‘是子节点’关系相连的交易都算在内。

Anything that is related through any combination of is parent of or is child of.

你有时会看到一些相当复杂的情况，比如一个用户向多个其他用户付款，然后将这些币与从另一方获得的另一个输出一起花费。

And you can and sometimes do see fairly complicated things where you have one user that pays a number of other users and then they spend those coins together with another output that they got from yet another party.

所有交易都会在同一个簇中变得相关。

All become related in a cluster.

因此，在簇内存池中，我们对一个簇的交易数量限制为64笔。

And so in Cluster Mempool, we have a limit of 64 transactions in a cluster.

这个数字是我们根据内部算法的性能特征选定的，就像过去祖先和后代限制设为25一样，那是基于我们仍能高效推理、而不会让节点计算时间失控的考量。

That's a number we pick based on performance characteristics of the algorithms inside, similar to how the ancestor and descendant limits of 25 in the past were picked based on Well, these are the things we can sort of efficiently reason about still without the computation time for nodes blowing out of the water.

是的。

Yeah.

因此，最直接的影响可能是，你可以构建长达64笔而非25笔的剥离链。

So from the most obvious direct impact is probably you could build these peeling chains up to 64 rather than 25.

但这并不是简单地增加数字，因为你还要算上随之计入的整个扩展家族关系。

But it's not like just increasing the number because you also have this extended family thing that gets counted along with it.

好的。

Okay.

我们能简单概述一下当前内存池的工作原理吗？

And so can we just overview a little bit on how the mempool works today?

那么，您认为这其中主要的问题是什么？

And then what are the main issues that you see in that?

我想您之前提到过一些关于祖先和后代限制的问题。

I think you were touching on some of that around the ancestor and descendant limits.

然后我们可以进一步介绍一些其他概念，帮助我们更好地理解。

And then we can sort of go into some other concepts to help make this understandable for us.

所以，先简单概述一下当前的内存池以及存在的问题。

So just an overview on the mempool today and the issues.

比特币核心采用了一种设计，每个节点——而不仅仅是矿工——都会维护一组它预期在短期到中期被挖出的交易，按照默认配置，大约涵盖一天内的交易量。

So Bitcoin Core uses a design where it maintains every node, not just miners, maintains a set of transactions that it expects to be mined in the short to medium term, up to roughly a day worth of transactions with default configurations.

它将这组交易保持为一致的状态，因此内存池中永远不会出现两个相互冲突的交易。

It maintains this set as a consistent set, so we'll never have two conflicting transactions in the mempool.

如果你试图去推理这种情况，会变得极其困难。

Just becomes awfully hard if you try to reason about that.

因此，它的用途有很多。

So its purpose is that there are many uses for this.

一方面，矿工可以从这里选择交易，但或许更重要的是，它能让网络中的节点了解当时交易市场的状况。

One is obviously for miners to pick their transactions from, but maybe more importantly, it's for nodes in the network to get an idea of what the transaction market at that point is like.

它用于手续费估算。

It's used for fee estimation.

它用于决定是否进行交易替换。

It's used for deciding what replacements to make.

如果你看到两个相互冲突的交易，就需要决定是否替换另一个。

If you see two conflicting transactions, will need to replace the other or not.

你会保留旧的那笔交易。

You stick with the old one.

你如何判断，这实际上是一种改进吗？

How do you reason about, well, is this actually an improvement?

如果我们能确定这笔新交易对矿工来说确实更好，我们就应该假设它会传达到那些矿工，从而让我们能够预测未来的行为。

If we can decide that this new transaction actually looks better to miners, we should assume it will make their way to those miners and thus for us to be able to predict the future behavior.

我们也希望它们出现在我们的内存池中。

We want them in our mempool to.

这基本上是对当前时刻区块空间市场状况的一种建模。

It's basically a modeling of what we think the block space market is like at this very instance.

而今天，这种建模是通过一系列启发式算法来引导的。

And the way this is guided by a whole number of heuristical algorithms today.

例如，在Cluster Mempool出现之前，挖矿使用的是基于祖先集的挖矿方式，即对每一笔交易，计算其所有祖先交易的集合——正如我之前解释的，过去这个集合被限制在25笔以内，然后查看整个集合的平均费率。

So before Cluster Mempool, for example, mining used something called ancestor set based mining, which is for every transaction you compute the set of all its ancestors, which, as I explained, was limited to 25 in the past, And look at the average fee rate of that whole set.

举个例子，我付给你钱，而你现在想花掉这些钱，但我支付的手续费非常低。

And the reasoning is like, say I pay you and now you take the but I attach a very low fee only.

你因为某种原因想尽快花掉这些币，于是你用这些币支付给其他人。

And you want to spend those coins for some reason, and you want to do so urgently, so you take those coins, pay someone else with it.

但我的交易还没确认。

But my transaction hasn't confirmed yet.

所以你想要做的是提高手续费。

So what you want to do is bump the fee.

这被称为子交易支付父交易，即你附加一个比正常更高的手续费，来弥补我所欠的那部分费用。

This is called child pays for parent, where you attach a higher fee than you normally would to sort of pay for the missing fee I have.

因为矿工在没有包含我的交易的情况下，是不允许包含你的交易的。

Because miners are not allowed to include your transaction without including mine as well.

因此，通过附加更高的手续费，你激励矿工同时包含这两笔交易。

And so, but by attaching a higher fee, you incentivize them to include both of the transactions at the same time.

这种机制通过基于祖先集的挖矿方式建模，而挖矿逻辑或区块模板生成逻辑——在实际矿工介入之前——会将这两笔交易作为一个整体来看待，因为它们构成了一个祖先集，并计算整个集合的平均手续费率。

And this is sort of modeled using this ancestor set based mining where the mining logic or the block template building logic this is long before there's an actual miner involved will look at the two set transaction combined because they form an ancestor set and sees this is the average fee in this whole set.

我打算把它当作一个整体一次性打包挖矿。

I'm gonna try treating it as a package that gets mined in one go.

这就是今天儿童支付父交易（CPFP）能够运作的原因。

So this is what makes child pays for parent work today.

这种机制自2015年以来一直如此。

And it has since since 2015.

实际运作方式是：我们预先为内存池中的每笔交易计算其所有祖先交易的集合，计算这些祖先集合的平均手续费率，并根据平均祖先集手续费率对交易进行排序。

Practically how it works, we pre compute for every transaction in the mempool the sets of all its ancestors, compute the average fee rate of all of those ancestor sets and sort the transactions by their average ancestor set fee rates.

我们几乎会优先打包手续费率最高的祖先集，然后重新开始，直到区块填满。

And we pretty much eagerly include the highest ancestor set fee rates package and then start over until the block fills out.

这是一个相当不错的近似方法。

This is a pretty good approximation.

但恰恰相反，我们希望做完全相反的事情，因为网络中的每个节点并不都是矿工，但每个节点都有理由试图预测这种行为——即驱逐。

But the opposite, we would like to do the very opposite because again not every node in the network is a miner, but every node has reasons to try to predict that behavior is eviction.

而在驱逐时，我们要做完全相反的事：为每笔交易计算其后代集合，因为当你想要驱逐一笔交易时，必须同时驱逐它的所有后代。

And in eviction, do the exact opposite, which is we compute the descendants sets for every transaction, because whenever you want to evict the transaction, you must also evict all their descendants.

它们会变得无效。

They become invalid.

我付了你钱，你又付给了别人。

I paid you, you paid someone else.

如果我的交易被驱逐，你就拿不到钱了，所以你的交易也必须消失。

If my transaction gets evicted you don't get your money anymore, so your transactions need to disappear as well.

因此，我们会为每笔交易预先计算其后代集合及其平均费率。

So we do we pre compute for every transaction the descendant sets and the average fee rate of that descendant sets.

在驱逐时，我们会选择平均费率最低的后代集合。

And when evicting we pick the descendant set with the lowest fee rate.

这感觉非常像挖矿算法的反面。

And this feels very much like the opposite of what a mining algorithm is.

但事实证明，你可以构造出涉及菱形依赖关系的奇怪边界情况。

But it turns out you can construct weird edge cases that involve diamond shaped dependencies.

你有一个父交易和两个子交易，而某种机制会导致这两个子交易被重复计算。

You have one parent and two children and something that spells those two children that causes things to be counted double.

因此，这两种排序方式——优先处理祖先集合最高的和优先处理后代集合最低的——并不是彼此的对立面。

As a result, these two orderings, like highest ancestor set first and lowest descendant set first, are not opposites of each other.

但还涉及许多其他启发式方法。

But there are many more heuristics involved.

例如，对于RBF，我们长期以来一直遵循BIP 125规则，这些规则规定了在满足某些条件时，你的交易将被接受替换。

For example, for RBF, we've had for the longest time the BIP 125 rules that sort of give a number of conditions on if you satisfy this and this and this, your transaction will be accepted for replacement.

它试图启发式地回答：这是否真的能提高矿工的手续费收入？

And it tries to heuristically answer, does this actually make things better for miners in terms of fee income?

因为如果确实如此，矿工会希望如此，而我们不希望人为制造理由，让人们将交易提交给矿工，从而使区块空间市场变得近乎私有化。

Because if it is, they'll want it and we don't want to introduce a reason for people to submit their transactions to miners and make the block space market basically private.

为了避免这种情况，我们需要能够分析这种盈利能力。

So to to avoid that, we we we need to be able to to reason about this profitability.

BIP 125 是在 Cluster Mempool 之前对此的一种尝试。

And bit one twenty five was an attempt at that in pre Cluster Mempool.

还有像费用估算这样的功能会使用内存池。

There's more things like fee estimation uses mempool.

比如，如果你突然同时收到大量交易，我们不希望压垮我们的对等节点，因此我们会逐步发送。

There's decisions like if you suddenly get a flood of transactions at the same time, we don't want to overload our peers, so we trickle them out.

我们应该优先发送哪些交易？

Which ones do we send first?

当然是我们认为更好的交易。

Well, the transactions that we think are better.

所以我们优先发送手续费率更高的交易，以此类推。

So we try to send the ones with a higher fee rate first, and so forth.

如果要总结今天的内容，我认为内存池本质上是试图近似当前区块空间市场的需求状况，它有一整套启发式规则来决定保留哪些交易、剔除哪些交易、发送哪些交易以及接收哪些交易，以尽可能贴近这种经济现实。

That's sort of how I think if I had to summarize today is like we have mempool is an attempt at approximating the current view of demand for the block space markets, and it has a whole bunch of heuristical rules to decide what to keep and what to evict, and what to send and what to receive that try to approximate this economic reality.

我认为Cluster Mempool正在用一个真正能合理处理这些事务的框架取代所有这些，它能考虑依赖关系的任意性，摒弃所有启发式规则，只保留一条准则：这样做会让情况变得更好吗？

I'd say Cluster Mempool is replacing all of that with a framework that can actually reason about things properly, taking dependent arbitrary dependencies into account and do away with all the heuristics and replace it with one rule is does this make things better or not?

好的。

Okay.

所以按照我的理解，简单来说，我们有一个mempool，人们把他们的交易放进去。

And so as I'm reading you then, it's like in simple terms, okay, we've this mempool, people put their transaction in this in there.

但正如你所说，围绕祖先和后代以及驱逐机制存在一些启发式规则，而这些启发式规则并不总是正确的，因为它们本质上只是启发式方法。

But as you said, there are certain heuristics around ancestors and descendants and eviction, what we're saying, and some of these heuristics aren't necessarily always correct because they're heuristics.

也许作为一条经验法则，它们大部分时候是有效的，但有时这些启发式规则也会出错或失效。

And maybe as a quote unquote rule of thumb, they might kind of mostly work, but then there are some times where maybe the heuristic goes awry or goes wrong.

有些情况下，矿工本应包含某个交易及其后代交易，但由于这些启发式规则，它们却被剔除了。

And there are examples where maybe a miner would have or should have wanted to include a particular transaction and the descendants of that transaction, but actually because of the heuristics, they would cut them out.

是的。

Yep.

你想要表达的就是这个意思吗？

Is that what you're getting at?

是的。

Yep.

对。

Yeah.

对。

Yeah.

而这并不理想，因为我们需要在这里做一个说明：最终，只有人们实际做的事情才重要。

And and this is undesirable because we do have to make a caveat here is that, ultimately only the things that people actually do matter.

比如，如果没有人创建极其复杂的交易依赖图，那就没有真正的经济需求。

Like if nobody creates super complicated dependency graphs of transactions, there's no real economic demand for that.

它并不重要我们如何处理这些情况，因为只有经济现实才是关键。

It doesn't really matter how things like it doesn't matter that we treat those sub optimally because it's only an economic reality that matters.

但问题不仅在于我们的决策可能是次优的，还可能是不一致的。

But the problem becomes not just that our decisions may be suboptimal but they may be inconsistent.

不一致性可能更糟糕，因为现在不仅仅是矿工在某个没人真正使用的边缘情况下少赚了一点钱。

Inconsistency can be a lot worse because now you it's not just okay, miners make in this weird edge case that nobody really does make a tiny bit less money.

谁在乎？

Who cares?

但如果我们讨论的是驱逐规则，也许有人希望发起这些交易，但它们并没有传播开来。

But if there's If we're talking about eviction rules, maybe you you get some well, someone may wants to make those transactions and they're not propagating.

所以现在你是在激励市场参与者再次转向私有内存池。

So now you're incentivizing market players to move towards private mempools again.

没错。

Exactly.

我认为这正是核心问题：我们试图让公共点对点透明内存池市场尽可能完善，但这种情况非常脆弱。

And I think that is really the core issue here is like it's a very tenuous situation how we're trying to make the public peer to peer transparent mempool market as good as possible.

而我们的期望是，如果它足够完善，人们就没有动力去绕过它。

And the hope is that if it is sufficiently good, there's just no incentive for people to bypass it.

它。

It.

我们看到了绕过它的尝试，那到底是什么？

We see attempts at bypassing it, There are what is it?

加速

Acceleration.

没错

Exactly.

很多矿池多年来一直在使用不同的加速机制。

Different accelerators that a lot of the pools have had for years.

这并不是什么新东西。

It's not a new thing.

是的

Yeah.

我觉得这令人担忧。

And I think it's worrying.

我不认为今天这一定是个问题。

I don't think it's necessarily a problem today.

我认为这些服务的总收入大部分可以忽略不计，除非在某些特定时刻。

Think that the total income from those services is mostly negligible, except maybe at certain times.

但如果我们陷入这样一种情况，即大量经济活动通过这些私有交易通道进行，而网络完全无法看到这些活动，最终会使得进入挖矿市场变得更加困难。

But if we were to end up in a situation where just that there is so much economic activity going through these private transaction rails where the network just doesn't have visibility into it, it ultimately makes it harder to enter the mining markets.

因为假设只有三家大公司拥有私有内存池，你可以向它们提交交易，而作为矿工，你实际上别无选择，只能与其中一家或多家签订合同，否则你就会错失大量手续费收入，无法与那些这样做的人竞争。

Because say there's three big companies that, you know, have a private mempool that you can submit things to, and as a miner you really have no choice but to contract with one or most of these because otherwise you just miss out on a substantial portion of fee income and are unable to compete with those who do.

这使得尤其是匿名参与变得更加困难，而我们非常希望世界上任何一个人都是可能做到的。

It just makes it harder to especially anonymously, like we would very much like it to be possible for any one in the world.

当然，你需要接入互联网、电力和硬件，但如果你认为有利可图，或者对其他人审查的交易感到不满，你并不需要获得任何人的许可就能进入挖矿市场。

Obviously, you need access to Internet, electricity and hardware, but you don't need permission from someone to enter the mining market if you are if you think there's there's money to be made or if you are unhappy with the set of transactions that are being censored by others.

现在我们来到了核心问题：抗审查性。

Now we get to the core issue, censorship resistance.

这真的太重要了。

And it like so much.

因此，我甚至会说，这正是比特币采用工作量证明的全部原因。

So I would go as far as saying this is the entire reason why Bitcoin has proof of work.

工作量证明的意义就在于，任何人都可以加入挖矿市场。

The point of having proof of work is that anyone can join the mining market.

比特币的去中心化和抗审查性最终在情况严重恶化时才真正体现出来。

Bitcoin's decentralization and censorship resistance ultimately boils down if things go really wrong.

最后的保障是你自己可以成为矿工，即使亏损也无所谓，但希望不要亏太多。

Like the final protection is you can become your own miner, maybe even at a loss, but hopefully not as a big one.

那么，是什么激励现有大型矿工接受所有交易呢？

And so what is it that incentivizes existing large miners to basically accept every transaction?

这是因为，如果他们开始可靠地审查某些有经济需求的交易，就会有人冒出来继续挖这些交易。

It is the knowledge that if they start reliably censoring some transactions that have economic demand someone will just pop up and mind them anyway.

但这一切只有在用户创建并具有经济需求的交易流是公开的情况下才有效。

And but this all of this only works when access to the stream of transactions that users create and have economic demand for is public.

你不需要参加私人聚会才能获得这些经济活动的访问权。

You don't need to go to a private party to get access to that economic activity.

因为如果你这么做了，他们就会成为守门人。

Because if you do that, they become the gatekeepers.

他们可以制定规则说：你是个匿名矿工。

They could enforce a rule and say, well, you're anonymous miner.

你无法获取我们的交易流。

You don't get our stream.

因此，我认为这正是所有这些工作的根本动机。

And so I think this is ultimately the motivation for all of this work.

我们希望确保公共网络能够可靠地传播用户希望创建、矿工希望挖掘的交易，让所有人都能访问这些交易。

We want to make it so that the public network is able to reliably relay transactions that users want to create and miners want to mine out into the public so everyone has access to them.

不幸的是，在去中心化系统中，抵御拒绝服务攻击是一个非常困难的问题。

Unfortunately in a decentralized system denial of service resistance is a very hard problem.

我们不能仅仅转发每笔交易。

We can't just relay every transaction.

我会把我的视频流通过它传输，并将其编码成相互冲突的交易。

I'd stream my video over it and code it into transactions that they're all conflicting with one another.

确实存在一个网络会免费转发每笔交易。

There's a network out there that will free relay every transaction.

但这行不通。

That doesn't work.

因此，长期以来，比特币核心中内存池设计的哲学是：假设我们可以可靠地预测哪些交易会被挖出，因为矿工是经济主体，那么我们的拒绝服务防护机制基本上就是：我们内存池中的交易会被确认，并且它们支付了手续费。

So what is and this has for a long time been sort of the philosophy behind the mempool design in Bitcoin Core is under the assumption that we can reliably predict what will be mined because miners are economic actors, then our denial of service protection is basically well, transactions in our mempool will get confirmed and they pay a fee.

这就是防护机制。

This is the protection.

即使它们没有支付给我们中继节点，它们也支付给了其他人，因此它们必须消耗有限的资源。

They pay a fee even if it's not to us as a relaying node, they're paying the fee to someone, so there's a finite resource they have to spend.

所以我们需要启发式方法来决定哪些交易应该中继，哪些不应该，这些方法本质上都是预测交易的盈利能力。

So we need heuristics to decide what will be relayed, what won't be, and those boil down to predicting profitability of those transactions.

我之前提到的所有关于挖矿、驱逐和排序的例子，都是试图推断盈利能力的有时不准确的启发式方法。

And all the examples I gave about mining and eviction and deciding the order in which things go are sometimes faulty heuristics that try to reason about profitability.

而通过集群内存池，我们可以更可靠、更一致地做到这一点。

And so with Cluster Mempool, we can just do that much more reliably and consistently.

好的。

Okay.

是的。

Yeah.

所以让我复述一下，我从中理解到的是，与其使用这些启发式方法，不如转向新的集群内存池模型，这能让节点运行者或矿工更准确地判断哪些交易更有可能被确认。

So just replaying some of that, the understanding I'm taking from it is instead of using these heuristics, moving to a new model of Cluster Mempool, which allows, let's say, that that node runner or that miner to more accurately, let's say, get an idea of what transactions are likely to be Yep.

最有利可图的。

The most profitable.

正如你所说，这不仅仅是优化利润，更是为了让挖矿保持开放性，让新矿工能够

And as you said, it's not just about optimizing the profit, it's about making it so that it's an, mining remains an open thing that a new Making miner can

可预测，即使面对可能复杂的交易组合也能保持可预测。

it predictable, but even predictable in the face of possibly complicated constellations of transactions.

举个非常简单的例子，这不是一种攻击：我之前提到过‘子交易支付父交易’的机制，即一个子交易可以通过提高手续费来提升父交易的手续费，因为父交易的手续费太低。

So as a very simple example and not an attack or anything is to So I've talked about child pays per parent where you can have a single child that bumps the fee of a parent because the parent paid too little fee.

你也可以让一个子交易同时提升多个父交易的手续费，但你不能让多个子交易同时提升同一个父交易的手续费，因为祖先集框架下，每个子交易都有自己的祖先集，它们永远不会被当作一个整体来处理。

And you can have one child that bumps multiple parents at the same time too, but you cannot have multiple children that bump the same parents because of this ancestor set framework, each of the children has its own ancestor sets, and they're never treated as a single set.

我认为这并不是集群内存池的原始目标，因为我还没真正讲清楚它是什么。

And I think this was not the original goal of Cluster Mempool, because I haven't really talked about what it is yet.

但目标并不是为了改进或增加更多复杂的使用场景，比如子交易为父交易支付手续费。

But the goal wasn't so much improving or adding more crazy use cases that someone could construct, like children pay for parent.

只是为了让事情保持一致。

It was just making things consistent.

但在这个过程中，我们发现实际上出现了更优秀的算法，这些算法能够考虑到这些更复杂的结构，并几乎以最优方式处理它们。

But then along the way we discovered that there were actually much nicer algorithms that became available that that can take these much more complicated constructions into account and still deal with them almost optimally.

是的。

Yeah.

好的。

Okay.

当我试图研究和理解这将产生哪些影响时，正如你所说，挖矿显然是其中一个方面。

And so as I was trying to research and understand where this will impact, so as you said, mining is is an obvious one.

据我理解，闪电网络和Layer 2也可能受到一些影响。

Lightning and l twos, as I understand it, it might have some impact to them.

然后举个例子，交易所的提币场景。

And then maybe as an example, the exchange withdraw case.

据我了解，一些交易所可能会这样做，比如说

So as I'm understanding some exchanges, they might do, like, they might let's say

你只能

You can only

比如，我是交易所，我有大约一百个客户，我可能进行一笔大额提现，但随后我会持续使用 CPFP 或 RBF 来不断向这笔交易中添加新的输出，分别支付给这些客户。

link let's say, I'm the exchange and I've got like a 100 customers, I might be doing a big withdraw, but then I might be continually doing either CPFP or RBF to continually add new outputs to that transaction to go out to the the customers.

我推测，集群内存池在这种情况下也能提供帮助，因为它们能更可靠地处理这种情况。

And I presume then Cluster Mempool might help in this context also because they can more reliably do that.

是的。

Yeah.

所以我看到过交易所做的一些操作。

So the there were there's a number of things that I've seen exchanges do.

比如，它们可能会进行批量处理，用一笔交易同时支付给大量用户。

So they might do batching where you have one transaction that pays out tons of people at the same time.

我认为这不会受到太大影响，或者你可以使用这些找零。

I don't think that that will be affected too much, or you can have these change.

你谈的更多是持续的替换。

You're talking more about continual replacements.

所以我不知道它们是否会受到影响。

So I don't know if they will be affected.

但规则变了，因为在Vipone 25之前，某些明显是改进的操作是被允许的。

But the rules change there, because with Vipone 25, used to be the case that there are certain things that are obvious improvements.

比如，创建一个支付更高手续费的交易来替换另一个交易，并做出一些更改。

Like, make a transaction that just pays a higher fee, it replaces another one, it makes some changes.

但之前是不被接受的。

But wouldn't be accepted.

而反过来，有些明显不是改进的操作，却反而会被接受。

And the other way around, where you have something that's clearly not an improvement and yet it would be accepted.

我认为从宏观上看，情况变得更简单了，你可以采用一种策略：直接提高手续费。

I think at a high level things become simpler in the sense that you can adopt a strategy of I'll just bump the fee.

比如，我做一次替换，如果没成功通过，我就再稍微提高一点手续费。

Like, I do a replacement, it doesn't make it its way through, I just bump the fee a bit more.

如果还是不行，我就再提高手续费，用另一个交易替换它。

And if it still doesn't work, I bump the the fee more, replace it with another transaction.

这样就能正常工作了。

And this will just work.

也就是说，只要交易真的更好了，它就会被处理。

Like, the transaction will go the as soon as it's actually better.

过去你可能会尝试遵循BIP25规则，但据我所知，实际上没人这么做。

Where in the past you could try to follow one twenty five rules, but as far as I know nobody actually does.

实际上，人们早就采用了这种试错的方式。

In practice people already adopt this, you know, try and see approach.

比如你提高手续费，如果没成功，我就再提高一点。

Like you bump the fee, it doesn't work, I'll I'll bump it some more.

现在，只要你真正改善了情况，交易就会通过。

Now it'll go through whenever you've actually improved the situation.

对。

Right.

所以实际上，人们本来就在用一种试错的方式，是的。

So it's kind of like in practice, people were just doing sort of a trial and error approach anyway Yeah.

关于这些

On these

我认为，从理论上讲，这种新方法最大的缺点是我不能再给你一套简单的规则了，比如‘如果你满足这些条件，这个交易的费率就高于那个’。

And I think this is In theory, I think this is the biggest downside to the new approach is I cannot give you a simple set of rules anymore that are of the form like, if you satisfy these rules, the fee rate of this is bigger than the fee rate of that.

还有这个的大小小于那个之类的规则。

And the size of this is less than that and whatever.

我保证替换机制。

I guarantee replacement.

这种情况不再成立了。

That's no longer the case.

没有简单的规则了。

There's no simple rules.

你必须让事情变得更好。

Is you have to make things better basically.

是的。

Yeah.

好的。

Okay.

所以，我的意思是，我们已经设定了一些背景，我认为这对听众来说是有帮助的。

So, I mean, we've set a bit of a context and I think it was useful for listeners.

现在让我们实际来看看Cluster Mempool到底是怎么运作的？你能给我们做一个概述吗？

Let's actually go now to, you know, how does Cluster Mempool actually So can you take us through an overview on that?

是的。

Yeah.

主要的观察是，为了可靠地预测在当前算法下，这些交易是否比那些更好，哪一个会先被挖出？

So the main observation is in order to reliably predict how, you know, make judgments about are these transactions better than these, will this one get mined before this one today with the algorithm we have?

我想实际弄清楚，让我们谈谈驱逐的情况。

I wanted to actually figure out, let's talk about the eviction case.

如果我想可靠地确定今天最后一个会被挖出的交易是哪一个？

If I wanted to actually reliably figure out today what is the last transaction that would be mined?

想象一下，没有新的交易进来，不管怎样。

Imagine no new transactions come in, whatever.

我给你一个包含4000笔交易的内存池，你想要找出最后会被打包的那笔交易，因为如果我资源受限，我就想剔除这一笔。

I give you a mempool of 4,000 transactions, 4,000 and you want to figure out what is the very last one that would be mined because that's the one I want to evict if I'm somehow resource constrained.

如今唯一的方法是运行挖矿算法，但不是针对一个1MB大小的区块，而是针对一个包含10万或9999笔交易的区块，然后看看哪一笔交易没有被包含进去。

Well, the only way you have today is you run the mining algorithm but not for a one mega V byte block but for a 100,000 or 9,999 transaction block and you see what's the one transaction that doesn't go in.

那就是我想剔除的那一笔。

That's the one I would want to evict.

可惜的是，这种方法在计算上根本不可行。

Sadly, that's just computationally not feasible.

我们不能每次想剔除一笔交易时，都重新运行整个区块模板构建流程来确定哪一笔会最后被包含。

Cannot anytime we want to evict a single transaction run this entire block template building thing to see what goes in last.

那么，作为计算机科学家，显而易见的解决方案是什么？

So what is the obvious answer as a computer scientist?

我们需要预先计算这些内容。

Well we want to pre compute things.

我们需要缓存它们。

We want to cache it.

不幸的是，事实证明，在这里的白板上画出来对听众帮助不大。

Unfortunately, turns out that Yeah, I don't think drawing it on a whiteboard here is helpful for listeners.

但我已经提供过一些例子和书面材料，说明你可以构建一些病态的内存池，其中有一连串交易，每个父交易都有两个子交易，每个子交易又有两个父交易，它们以网格形式连接，最优排序是从左到右。

But I've given examples and write ups about this where you can construct pathological mempools where you have like a whole sequence of transactions where they're all like every parent has two children and every child has two parents and they're sort of connected as a trellis where the optimal ordering is going from left to right.

最好的交易在左边。

The best transactions are in the left.

最差的交易在右边。

The worst transactions are in the right.

现在，一个新交易突然进入，并在右侧附着，支付了极高的手续费。

And now a single new transaction comes in that pays a huge huge fee and attaches on the right.

于是，最优排序突然变成了从右到左。

And now suddenly the best order becomes you go from right to left.

这看起来很反直觉，但如果你仔细推导，就会发现这是真的。

And this is pretty unintuitive, but it's true if you work it out.

因此，这里的观察是：一个单独的交易附加到一个庞大的交易集群上，可能会彻底改变该集群内所有交易的优劣排序。

So the observation here is that a single transaction attaching to a huge cluster of transactions can basically completely overhaul how good things are within that cluster.

因此，我们以为可以预先计算好顺序，然后只做些小调整，这种想法行不通，因为一个新交易的加入就可能要求我们重新计算全部内容。

So this notion of we can just pre compute what the order is and and make small changes to it doesn't work because a single thing can come in that requires us to recompute everything.

我明白了。

I see.

另外，出于好奇，像 mempool.space 这样的服务是怎么处理的呢？

And just, I mean, out of curiosity, how does something like mempool dot space do it?

我的意思是，它们似乎会预测区块可能会包含哪些交易。

I mean, they they kind of project out what they think the blocks will be.

这真的计算成本太高，还是根本不可行？

Is that just, really computationally expensive or, like, just not feasible?

我不知道它们后台运行的是什么。

So I don't know what they run-in the background.

我猜它们可能使用了比特币核心节点来获取数据，这些节点会做些处理，但结果可能不是最优的。

I assume they have Bitcoin Core nodes that they pull things from, which do something, but it may be it will be suboptimal.

我不太清楚，比如那个，是的。

I don't know, like the Yeah.

实际上，你会对所有事情进行近似处理，然后得到一些输出。

In practice, you make an approximation for for everything, anything, and and you'll get some output.

它可能并不对应于真正最优的方案，从这个观察来看，在一个假设的对抗性构造的内存池中，你不会看到90%的内存池是由一两个交易簇组成的。

It just may not correspond to what is the actual best And thing to from this observation that a single transaction in a hypothetical adversarially constructed mempool, you won't see like, know, 90 of the mempool is one or two transaction clusters.

这里没有问题。

There's no problem here.

但在最坏的情况下，可能会有一个交易进来，迫使你重新计算所有内容。

But in the worst case, you can have a single transaction comes in that requires you to recompute everything.

我要明确一下，当我说到重新计算所有内容时，这可能只是几十毫秒的时间。

And to be clear when I say recompute everything this may be tens of milliseconds or something.

但我们无法对每一个进入的交易都这样做。

It's just not something we can do for every single transaction that comes in.

对的。

Right.

我不是在说这会花费数小时来计算。

It's not like I'm claiming this will take hours to compute.

这可能是几十或几百毫秒。

It's like tens or hundreds of milliseconds.

但每次有变化时都这样做，代价太高了。

But that is too much to do all the time every time a change comes in.

因此，聚类的想法就出现了，因为事实上我们并不需要整个内存池的完整排序。

And so that's where the idea of the clustering comes in, because it turns out we don't actually need the ordering of the entire mempool.

我们只需要每个交易簇内部的排序。

We just need the ordering within every cluster of transactions.

如果我预先计算，比如有一组相关的、有依赖关系的交易，但这是一个小群体，我知道如何对它们排序。

If I pre compute, like I have a bunch of related transactions with dependencies and whatever, but it's a small group and I know how to order those.

然后我有另一个簇，我知道如何对它们排序，还有另一个。

And then I have another cluster and I know how to order those and yet another one.

我知道如何对它们排序。

And I know how to order those.

你可以非常高效地将它们合并成一个对内存池的统一视图。

You can very efficiently sort of merge them into a single view of the mempool.

这就是集群内存池概念的由来。

And this is where the idea of Cluster Mempool comes in.

如果我们能限制这些集群的大小，那么任何新进入的交易都只会影响其所在集群的顺序。

If we were to just able to be limit how big those clusters can get then any individual transaction that comes in will just affect the order of that one cluster.

我们最终选定64笔交易作为提案数值，因为对于这种规模的集群来说——它们并不算太小——64笔交易远大于当前系统所支持的数量。

And 64 transactions is the number we ended up with as our proposal because for such small clusters and they're not tiny, 64 transactions is significantly larger than what things are supported today.

是的。

Yeah.

当一笔新交易进入时，你只需要重新计算该小部分集群的顺序。

A new transaction comes in, you you will just need to do the recomputation for that small group of clusters.

我们不需要担心单笔交易会带来影响。

And we don't have to worry about a single transaction, you know.

比如颠覆整个区块的排序

Like upending the whole block kind

这样的情况。

of thing.

没错。

Exactly.

相反，你只是重新考虑那个特定的集群。

Instead, you're just reconsidering that particular cluster.

是不是

Is it not

在我之前那个假设性的例子中，我谈论的不只是一个区块。

in in my previous, like, hypothetical example, I wasn't just talking about one block.

我谈的是整个内存池，成百上千个区块的交易在假设情况下都可能需要被追加。

I was talking about the entire mempool could could, hundreds of blocks worth of transactions could need to be appended in a hypothetical situation.

举例来说。

Example.

在这里就变得有限制了。

Here it becomes limited.

因此，集群内存池转向了一种策略规则，根据我们的直觉、信念和观察，限制集群的大小，因为用户实际上并不关心超大的集群。

And so Cluster Mempool is switching to a policy rule that imposes a limit on how big the clusters can get based on our intuition, belief, observation that users don't really care about huge clusters.

然后就可以预先计算每个集群内的挖矿顺序，由此可以隐式地定义整个内存池的排序，现在你需要做出决定。

And then it becomes possible to pre compute the mining order within every cluster, and from that you can implicitly define a whole mempool ordering, and now you you to decide.

这比那个更好吗？

Is this better than this?

这是一种改进吗？

Is this an improvement?

最佳的选择是什么？

What's the best thing to pick?

我们最后应该剔除什么？等等。

What's the last thing we should be evicting and so forth?

然后你的想法是，这些类别，我想是分块吗？

And then the idea is that these classes you'll have, I guess, is it chunks?

然后你将一个分块与另一个进行比较，比如，集群或分块B的手续费率高于分块A，因此我会倾向于选择B，等等。

And then you assess one chunk against another to say, oh, the fee rate in cluster or chunk b is higher than chunk a, so I'm gonna, you know, preference that in favor of, you know, etcetera.

是的。

Yeah.

这些块实际上是CPFP交易组的泛化形式。

So the chunks are sort of the generalization of CPFP groups of transactions.

每当有一些交易为其他交易支付费用时。

Whenever you have some transactions paying for other transactions.

因为，比如说，我们有一个单独的父交易同时支付了我们两个人。

And because, say, we we have a a single parent transaction that pays both of us.

你和我都得到了一个输出。

You and me both get an output.

但这是一个手续费很低的交易，我想加速它。

But it's a fairly low fee transaction and I want to accelerate it.

所以我花掉了我的输出，但用了一个很高的手续费。

So I spend my output, but I spend it with a huge fee.

你花掉了你的输出，但你并不在意，你用的是很低的手续费。

You spend yours but you don't really care you spend it with a low fee.

你希望的是我的追加交易能提升父交易的优先级，但你的交易不需要，因为你的手续费本来就低，你根本不在乎。

What you want is my bumping transaction to bump the parent but yours doesn't in the sense that yours is low you don't care.

所以，尽管这是一个集群，它会变成两个块。

So even though this is one cluster, it will become two chunks.

第一个块是同时吸引我们两人的父交易以及我用来提升它的交易。

The first chunk will be the parent that bait both of us and my transaction that bumps it.

第二个块则是你的交易，它依赖于父交易。

And then the second chunk is yours that becomes dependent on it.

这与交易的排序密切相关。

And it's very related to the ordering.

我不认为我应该在这里深入太多细节。

I don't think I should go into the details too much here.

但如果我给你一个集群中交易的排序，你就能找出哪些是块，哪些内容需要合并。

But if I give you an ordering of transactions for a cluster, you can find what the chunks are like, which things need to be combined.

事实上，找到最优排序问题与找到块的问题是同一个问题。

And actually finding the optimal ordering is the same problem as finding the chunks, it turns out.

好的。

Okay.

我认为这是描述什么是区块最简单的方式。

It's I think the simplest way of describing what a chunk is.

取你的集群，它是一组相互依赖的交易，不管怎样，可能有几十笔。

Take your cluster, which is a whole group of dependent transactions, whatever, can be dozens.

你称一个拓扑集合为包含其所有祖先的交易集合。

You call a a topological set is a set of transaction that includes all its own ancestors.

所以在我们的例子中，仅父交易就是一个拓扑集合，父交易加上我的交易是一个，父交易加上你的交易也是一个，或者三者一起也是一个拓扑集合。

So in our example just a parent would be a topological set, the parent together with mine would be one, the parent together with you would be, or all three would be topological.

然后在所有的拓扑集合中，选择费率最高的那个。

And then among all the topological sets pick the one with the highest fee rates.

这个就成为你的第一个区块。

And that becomes your first chunk.

从集群中移除它。

Remove it from the cluster.

然后对集群剩余部分重复相同的计算。

And now you redo that same computation on what remains of the cluster.