alicexbt [ARCHIVE] / is on Nostr

📅 Original date posted:2022-09-27
📝 Original message:Hi Eric,
> If by "range" you mean a connected tx subgraph, I don't see why not. But note that nodes only operate over signed txs. PSBT is a wallet workflow.
Matt Corallo mentioned that pre-signed transactions created with low fee rate become an issue when they are broadcasted after a long time and there is a high demand for block space at that moment.
Example:
Bob created PSBT1 in a multi party contract with fee rate 5 sat/vbyte however its taking hours/days to confirm the transaction with such low fee rate.
Carol created PSBT1 (5 sat/vbyte), PSBT2 (10 sat/vbyte) and PSBT3 (20 sat/vbyte) for spending same inputs. She broadcasted PSBT3 which got confirmed in a few minutes.
> Always. Only signed transactions are accepted. But assuming you are referring to a transaction that has been produced by a pre-signing workflow, I'm not sure how this would be distinct from any other tx.
`minfeefilter` for all peers of my node was 0.00001000 at the time of writing this email. I am assuming nobody creates pre-signed transaction with fee rate below 1 sat/vbyte. How often does it happen that `minfeefilter` is above this default value?
> I'm not sure I follow this, maybe you could reword it. But it seems that you are saying that CPFP fee-bumping is a problem scenario and the complexity of the proposed solutions are not justified by such scenarios.
Sorry that sentence was confusing. Yes complexity isn't justified for CPFP fee-bumping txs below minimum fee rate.
> There are many node implementations used presently. And of course these are protocol proposals, which presumes more than one implementation.
Yes, a few implementations exist (knots, libbitcoin, btcd, bcoin etc.) however they aren't used by lot of nodes. Based on this [chart][1] 98% nodes use bitcoin core. Lot of bitcoin protocol proposals are influenced by bitcoin core contributors and things could be different if even 30% nodes used other implementations.
> I don't consider this relevant to any protocol considerations. Miners should always be expected to select the most optimal set of txs available in the time available to do so.
Agree, miners should be expected to select most optimal set of txs. However, according to one [comment][2] by Pieter Wuille, miners could affect the security of some bitcoin projects with MEV.
> Over time we are likely to see that the only policies that remain in widespread application are those that are necessary for DOS protection (fee rate), as other restrictions are not economically rational and cannot be enforced. We've seen recent debate regarding dust policy, and op_return policy. "non-standard" txs are perfectly valid but get stuck very easily. I'll reiterate, any policy beyond what is published via the protocol will cause the above problems.
I completely agree with this.
[1]: https://luke.dashjr.org/programs/bitcoin/files/charts/software.html
[2]: https://bitcoin.stackexchange.com/questions/107787/front-running-in-bitcoin#comment123441_107796
/dev/fd0
Sent with Proton Mail secure email.
------- Original Message -------
On Tuesday, September 27th, 2022 at 2:49 AM, <eric at voskuil.org> wrote:
> > Hi Eric,
> >
> > This email wasn't answered by anyone on mailing list however I did some
> > research about packages yesterday including this email and below are my
> > observations, questions etc.
>
>
> Hello, thanks for the reply.
>
> > > The sole objective, as expressed in the OP proposal, is to:
> > >
> > > "Propagate transactions that are incentive-compatible to mine, even if they
> > > don't meet minimum feerate alone."
> >
> > According to bitcoinops: Without package relay, it’s not possible to
> > effectively CPFP fee bump a transaction that’s below the minimum feerate
> > nodes accept.
>
>
> Yes, the problem statement is not in question, just the mechanism of resolution. The problem of stuck txs arises from minimum fee rate policy, which is a necessary DOS guard.
>
> A secondary issue is that of orphan relay. As a node must allow receipt of orphans, it has no means to differentiate a flood of unconfirmable txs from those that are confirmable.
>
> > Matt Corallo's thoughts in a bitcoin core issue:
> >
> > "Matt Corallo recently wrote about an example on the bitcoin-dev mailing list
> > involving lightning transactions, where pre-signed transactions might be
> > broadcast to the blockchain long after they were generated, and thus not
> > have been created with a fee that is sufficient to be confirmed quickly (or
> > even be accepted to node mempools). In such situations, channel
> > participants may need to use chained transactions (CPFP) in order to increase
> > the confirmation speed of such transactions, and that implies we may need
> > to introduce a mechanism for those parent transactions to be relayed along
> > with their higher feerate children, even if the parent transaction would be
> > rejected by itself."
>
>
> While this is a valid scenario, the problems directly affect Bitcoin. Those problems propagate to layers, but are not unique to layering.
>
> > 1)Is it possible to have multiple pre-signed transactions with different fee
> > rates in a range? Example: PSBT1: 5 sat/vbyte, PSBT2: 10 sat/vbyte, PSBT3: 20
> > sat/vbyte and PSBT4: 100 sat/vbyte
>
>
> If by "range" you mean a connected tx subgraph, I don't see why not. But note that nodes only operate over signed txs. PSBT is a wallet workflow.
>
> > 2)How would covenants affect this problem?
>
>
> There are a good number of covenant proposals, though I assume they are all implemented within script. If a tx is confirmable and satisfies fee rate (for DOS protection), it is relayable. Covenants affect confirmability and should not have any unique impact on relay.
>
> > 3)How often does it happen that a pre-signed tx gets rejected by nodes
> > because it did not meet the minimum fee rate? Is it predictable and could be
> > managed in a different way?
>
>
> Always. Only signed transactions are accepted. But assuming you are referring to a transaction that has been produced by a pre-signing workflow, I'm not sure how this would be distinct from any other tx.
>
> > After reading several links related to packages and bitcoin core pull requests,
> > I found it anti-bitcoin to introduce so much complexity because its not
> > possible to CPFP fee bump a tx below minimum fee rate.
>
>
> I'm not sure I follow this, maybe you could reword it. But it seems that you are saying that CPFP fee-bumping is a problem scenario and the complexity of the proposed solutions are not justified by such scenarios.
>
> I would say that the problem is real, and that the least complex option is generally preferred. There are always tradeoffs, and balancing these is part of protocol development. But as a rule, complexity within a protocol (communication) is to be avoided where possible.
>
> > > Furthermore any tx that is "stuck" can be freed by simply sending another
> > > tx. The nodes at which the tx has become stuck will just package it up and
> > > relay it to peers. In other words, there is no impact on wallet implementation
> > > apart from raising the aggregate fee using a descendant transaction.
> >
> > It is easy to send another tx if there is only one user involved however
> > packages are trying to fix issues in which multiple users and transaction pre-
> > signed between them are involved. So, it will be difficult to coordinate and
> > create new pre-signed transactions in some cases although it is possible for
> > some use cases.
>
>
> Given that nodes do not deal in presigned txs, this coordination difficulty could not be increased in any scenario.
>
> A node produces sets of txs ("packages") dynamically to satisfy its peer's feerate. When a wallet broadcasts a tx/package to a node, it is operating as a peer on the p2p network. The wallet simply implements the same dynamic packaging algorithm as any peer - because it is a peer.
>
> > > This is barely a protocol change - it's primarily implementation. All that
> > > should be required is an additional INV element type, such as
> > > MSG_TX_PACKAGE.
> >
> > > * All elements of MSG_TX_PACKAGE in one INV message MUST to be of
> > > the same package.
> > > * A package MUST must define a set that can be mined into one block
> > > (size/sigops constraint).
> > > * A package SHOULD not contain confirmed txs (a race may cause this).
> > > * A package MUST minimally satisfy peer.feerate.
> > > * A partial tx order, as in the manner of the block.txs ordering, MUST be
> > > imposed.
> > > * A node SHOULD drop a peer that sends a package (or tx) below
> > > node.feerate.
> > > * A node MAY drop a peer that sends a non-minimal package according to
> > > node.feerate.
> >
> > This makes sense particularly if multiple node implementations are used in
> > future.
>
>
> There are many node implementations used presently. And of course these are protocol proposals, which presumes more than one implementation.
>
> > My other questions:
> >
> > a)If a package has tx1, tx2, tx3, tx4 and tx5 and miner just include tx1 and tx2
> > in the block, how does this affect the projects considered for packages
> > proposal?
>
>
> I will leave that to authors of such proposals to answer. However in what I have proposed it just means tx3/4/5 get considered for subsequent block inclusion to the extent that fee rate policy is satisfied.
>
> One of the several problems with static construction of packages is that they can still get stuck by fee rate policy. This is just kicking the can down the road while complicating the protocol.
>
> > b)How does changing the order of txs in a package affect these transactions?
>
>
> There is no impact. I proposed the partial ordering to facilitate fail fast.
>
> The partial ordering in block txs is unnecessary (given the PoW DOS guard). This is a consequence of the order imposed by Satoshi's implementation and only serves to slow validation (order constrains concurrency).
>
> > c)Do packages introduce more attack vectors in bitcoin for front running or
> > MEV? MEV in bitcoin currently only affects the projects that are considered
> > in packages proposal.
>
>
> I don't consider this relevant to any protocol considerations. Miners should always be expected to select the most optimal set of txs available in the time available to do so.
>
> > d)What if the package contains a transactions with sanctioned address?
>
>
> One can consider this a policy, much like fee rate. Any policy that is applied to transactions and not known to its peers will result in the node receiving orphans. As such the node either must allow orphans or drop peers sending orphans under the assumption that the peer is expected to have implemented the same policy.
>
> > e)Why would miners use packages if the existing scenario in terms of fees
> > per block is beneficial for them?
>
>
> The presumption is that the miner is only ever seeing txs that satisfy its fee rate policy, so this is just more opportunity.
>
> I'd add that the problem of "pinning" is related, but exacerbated by opaque policy (internal to certain implementations). Any node that ejects txs from its pool of valid but unconfirmed txs that satisfy fee rate policy is going to see orphans and going to cause txs to get stuck. This is one of the many problems with placing an arbitrary bound on the size of this pool.
>
> A subset of this problem is RBF policy. It is nice to see some movement toward generalizing RBF. The term is really a misnomer. Conflicting txs and subgraphs of txs are only problematic in the case of DOS, which is also resolved through advertised fee policy. Any node that imposes policy beyond this will also see orphans and cause txs to get stuck.
>
> The scenario and therefore complexity consequences of an implementation-specific memory-constrained tx pool are becoming increasingly apparent. These are implementation issues, not protocol issues. This can be observed in a recent thread: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
>
> Over time we are likely to see that the only policies that remain in widespread application are those that are necessary for DOS protection (fee rate), as other restrictions are not economically rational and cannot be enforced. We've seen recent debate regarding dust policy, and op_return policy. "non-standard" txs are perfectly valid but get stuck very easily. I'll reiterate, any policy beyond what is published via the protocol will cause the above problems.
>
> e
>
> > /dev/fd0
> >
> > Sent with Proton Mail secure email.
> >
> > ------- Original Message -------
> > On Thursday, June 9th, 2022 at 4:13 AM, Eric Voskuil via bitcoin-dev <bitcoin-
> > dev at lists.linuxfoundation.org> wrote:
> >
> > > Hi Suhas/Gloria,
> > >
> > > Good questions. I've started a new thread because it became something
> > > else...
> > >
> > > Various ideas about packaging seem to be focused on the idea of an atomic
> > > message that is gossiped around the network like a transaction or block.
> > > From my perspective that seems to create a set of problems without good
> > > solutions, and it is not a proper analogy to those atomic structures. It may be
> > > worth taking the time to step back and take a close look at the underlying
> > > objective.
> > >
> > > The sole objective, as expressed in the OP proposal, is to:
> > >
> > > "Propagate transactions that are incentive-compatible to mine, even if they
> > > don't meet minimum feerate alone."
> > >
> > > Effectively producing this outcome with an atomic packaging approach
> > > while at the same time maintaining network invariants seems unlikely, if not
> > > impossible.
> > >
> > > Fees:
> > >
> > > A node knows what fee rate a peer will accept, and announces individual
> > > txs that satisfy peer.feerate. Similarly a node knows its own feerate, and
> > > SHOULD drop any peer that announces txs that do not satisfy node.feerate.
> > >
> > > Orphans:
> > >
> > > A node MAY drop a peer that announces txs that the node sees as orphans
> > > against its DAG. It SHOULD drop the orphan tx and MAY request missing
> > > ancestors. Presumably after some amount of time connected to peer, node
> > > does not expect to see any more orphans from that peer, so these choices
> > > could evolve with the channel. However, the design that can only consider
> > > each tx in isolation will continue to cause orphan announcements on the
> > > channel. A below peer.feerate tx does not get announced to peer, and later
> > > a descendant high peer.feerate does get announced to the peer - as an
> > > orphan.
> > >
> > > BIP133 (feefilter):
> > >
> > > "There could be a small number of edge cases where a node's mempool
> > > min fee is actually less than the filter value a peer is aware of and
> > > transactions with fee rates between these values will now be newly
> > > inhibited."
> > >
> > > https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki
> > >
> > > Whether the problem is "small" or not depends on the disparity between
> > > node fee rates, which is not a matter of protocol. This is an existing problem
> > > that can and should be dealt with in packaging, as part of the above
> > > objective.
> > >
> > > Packaged Transaction Relay:
> > >
> > > One might instead think of packaging as a per-connection function,
> > > operating over its transaction (input->output) DAG and the feerate of its
> > > own node and that of the peer. Logically a "package" is nothing more than a
> > > set of transactions (optimized by announcement). Only a node can
> > > effectively determine the packaging required by each of its peers, since only
> > > the node is aware of peer.feerate.
> > >
> > > The only way to avoid dead-ending packages (including individual
> > > transactions, as is the objective) is for a node to package txs for each peer.
> > > The origination of any package is then just a wallet peer doing what a node
> > > does - packaging transactions that satisfy peer.feerate (i.e. that of its node).
> > >
> > > Current transaction relay (txB->txA):
> > >
> > > ===============================
> > > Node0
> > > txA.feerate > node.feerate, and not orphaned (accept txA)
> > >
> > > txA.feerate > peer1.feerate (announce txA to peer1)
> > >
> > > txA.feerate < peer2.feerate (do not announce txA to peer2)
> > > -----
> > > txB.feerate > node.feerate (accept txB)
> > >
> > > txB.feerate > peer1.feerate (announce txB to peer1)
> > >
> > > txB.feerate > peer2.feerate (announce txB to peer2)
> > >
> > > Node1
> > > Sees/accepts txA and txB.
> > >
> > > Node2
> > > Never sees txA, sees/rejects txB (as an orphan).
> > >
> > > Packaged transaction relay (txB->txA):
> > >
> > > ===============================
> > > Node0
> > > txA.feerate > node.feerate, and not orphaned (accept txA)
> > >
> > > txA.feerate > peer1.feerate (announce txA to peer1)
> > >
> > > txA.feerate < peer2.feerate (do not announce txA to peer2)
> > > -----
> > > txB.feerate > node1.feerate (accept txB)
> > >
> > > txB.feerate > peer1.feerate (announce txB to peer1)
> > >
> > > txB.feerate > peer2.feerate (do not announce txB to peer2) <== avoid
> > > predictable orphan
> > >
> > > txA.feerate + txB.feerate > peer2.feerate (announce pkg(A, B) to
> > > peer2) <= create minimal package
> > >
> > > Node1
> > > Sees/accepts txA and txB.
> > >
> > > Node2
> > > pkg(A, B) > node2.feerate (accept txA, txB)
> > >
> > > txA.feerate > peer3.feerate (announce txA to peer3)
> > >
> > > txB.feerate > peer3.feerate (announce txB to peer3)
> > >
> > > Sees/accepts pkg(A, B).
> > >
> > > Node3
> > > Sees/accepts txA and txB. <= avoided unnecessary packaging
> > >
> > > Summary:
> > >
> > > In this design, any node that receives an announcement for a pkg (or tx)
> > > later determined to be less than node.feerate SHOULD drop the announcing
> > > peer. Unlike with existing tx relay, a node can become "current" and
> > > subsequently see few if any tx or pkg orphans, and MAY at some point
> > > decide to drop any peer that announces one. Notice that packages are
> > > created dynamically, and any package that doesn't need to be grouped gets
> > > trimmed down to individual transactions. Furthermore any tx that is "stuck"
> > > can be freed by simply sending another tx. The nodes at which the tx has
> > > become stuck will just package it up and relay it to peers. In other words,
> > > there is no impact on wallet implementation apart from raising the aggregate
> > > fee using a descendant transaction.
> > >
> > > This is barely a protocol change - it's primarily implementation. All that
> > > should be required is an additional INV element type, such as
> > > MSG_TX_PACKAGE.
> > >
> > > Additional constraints:
> > >
> > > * All elements of MSG_TX_PACKAGE in one INV message MUST to be of
> > > the same package.
> > > * A package MUST must define a set that can be mined into one block
> > > (size/sigops constraint).
> > > * A package SHOULD not contain confirmed txs (a race may cause this).
> > > * A package MUST minimally satisfy peer.feerate.
> > > * A partial tx order, as in the manner of the block.txs ordering, MUST be
> > > imposed.
> > > * A node SHOULD drop a peer that sends a package (or tx) below
> > > node.feerate.
> > > * A node MAY drop a peer that sends a non-minimal package according to
> > > node.feerate.
> > >
> > > The partial ordering of block.txs introduces an ordering constraint that
> > > precludes full parallelism in validating input attachment. This is an
> > > implementation artifact that made its way into consensus. However in the
> > > case of packaging, the set of txs is not presumed to be valid under the proof
> > > of work DoS guard. As such constraints should minimize the work/traffic
> > > required to invalidate the message. The partial order constraint ensures that
> > > the DAG can be built incrementally, dropping the attempt (and peer as
> > > desired) as soon as the first orphan is discovered. As a result the network
> > > traffic and work required is not materially different than with tx relay, with
> > > two exceptions.
> > >
> > > These are the two central aspects of this approach (Avoiding Predictable
> > > Orphans and Creating Minimal Packages). These are graph search algorithms,
> > > some basic computer science. Minimality requires only that the package does
> > > not introduce txs that are not necessary to reach the peer.feerate (as these
> > > can always be packaged separately). It does not require that nodes all
> > > generate the same packages. It does not require negotiation, package
> > > identity, cryptography, or hashing. As a graph search it should be O(n) where
> > > n is the unconfirmed ancestry of the package, but should typically be much
> > > lower, if not a single step.
> > >
> > > Sufficiently-low-fee nodes will see only single txs. Moderate-fee
> > > nodes may cause partial breakup of packages. Sufficiently high fee
> > > nodes will cause peers (having received and completed the acceptance
> > > of a tx/pkg with pkg.feerate < peer.feerate) to navigate from each
> > > tx/package external input until reaching txs above peer.feerate, or
> > > confirmed (both of which the peer is presumed to already have). If the
> > > pkg.feerate is sufficiently high to connect all external inputs to the
> > > intervening txs, they are added to the package and it is announced to
> > > the high fee peer. Note that the individual tx.feerate > peer.feerate
> > > is insufficient to ensure that the peer should have the tx, as there
> > > may be ancestor txs that do not, and for which the tx was insufficient
> > > to cause them to be packaged. So a non-caching algorithm must be able
> > > to chase each package external input to a confirmed tx (or cache the
> > > unconfirmed ancestry fee rate at each tx). Note that fee rates are not
> > > directly additive, both size/
> > >
> > > weight and fee are required for summation (and aggregate sigops should
> > > be considered).
> > >
> > > This makes no assumptions about current implementations. The design
> > > would call for maintenance of a transaction (input->output) DAG with
> > > tx.feerate on each tx. This could be the unconfirmed tx graph (i.e. "memory
> > > pool") though it does not require maintenance of anything more than the
> > > parameters necessary to confirm a set of validated txs within a block. It is
> > > very reasonable to require this of any participating node. A simple version
> > > negotiation can identify a package-accepting/sending nodes.
> > >
> > > I have thought about this for some time, but have not implemented either
> > > the graph search, source code, or BIP. Just wrote this off the top of my head.
> > > So I am sure there are some things I have incorrect or failed to consider. But I
> > > think it's worth discussing it at this point.
> > >
> > > e
> > >
> > > > -----Original Message-----
> > > > From: bitcoin-dev bitcoin-dev-bounces at lists.linuxfoundation.org On
> > > > Behalf Of Suhas Daftuar via bitcoin-dev
> > > > Sent: Wednesday, June 8, 2022 8:59 AM
> > > > To: Bitcoin Protocol Discussion
> > > > bitcoin-dev at lists.linuxfoundation.org
> > > > Subject: Re: [bitcoin-dev] Package Relay Proposal
> > > >
> > > > Hi,
> > > >
> > > > Thanks again for your work on this!
> > > >
> > > > One question I have is about potential bandwidth waste in the case
> > > > of nodes running with different policy rules. Here's my
> > > > understanding of a scenario I think could happen:
> > > >
> > > > 1) Transaction A is both low-fee and non-standard to some nodes on
> > > > the network.
> > > > 2) Whenever a transaction T that spends A is relayed, new nodes will
> > > > send INV(PKGINFO1, T) to all package-relay peers.
> > > > 3) Nodes on the network that have implemented package relay, but do
> > > > not accept A, will send getdata(PKGINFO1, T) and learn all of T's
> > > > unconfirmed parents (~32 bytes * number of parents(T)).
> > > > 4) Such nodes will reject T. But because of transaction
> > > > malleability, and to avoid being blinded to a transaction
> > > > unnecessarily, these nodes will likely still send getdata(PKGINFO1,
> > > > T) to every node that announces T, in case someone has a transaction
> > > > that includes an alternate set of parent transactions that would pass
> > > > policy checks.
> > > >
> > > > Is that understanding correct? I think a good design goal would be
> > > > to not waste bandwidth in non-adversarial situations. In this case,
> > > > there would be bandwidth waste from downloading duplicate data from
> > > > all your peers, just because the announcement doesn't commit to the
> > > > set of parent wtxids that we'd get from the peer (and so we are
> > > > unable to determine that all our peers would be telling us the same thing,
> > > > just based on the announcement).
> > > >
> > > > Some ways to mitigate this might be to: (a) include a hash (maybe
> > > > even just a 20-byte hash -- is that enough security?) of the package
> > > > wtxids (in some canonical ordering) along with the wtxid of the
> > > > child in the initial announcement; (b) limit the use of v1 packages
> > > > to transactions with very few parents (I don't know if this is reasonable
> > > > for the use cases we have in mind).
> > > >
> > > > Another point I wanted to bring up is about the rules around v1
> > > > package validation generally, and the use of a blockhash in
> > > > transaction relay specifically. My first observation is that it
> > > > won't always be the case that a v1 package relay node will be able
> > > > to validate that a set of package transactions is fully sorted
> > > > topologically, because there may be (non-parent) ancestors that are
> > > > missing from the package and the best a peer can validate is
> > > > topology within the package -- this means that a peer can validly
> > > > (under this
> > > > BIP) relay transaction packages out of the true topological sort (if
> > > > all ancestors were included).
> > > >
> > > > This makes me wonder how useful this topological rule is. I suppose
> > > > there is some value in preventing completely broken implementations
> > > > from staying connected and so there is no harm in having the rule,
> > > > but perhaps it would be helpful to add that nodes SHOULD order
> > > > transactions based on topological sort in the complete transaction
> > > > graph, so that if missing-from-package ancestors are already known
> > > > by a peer (which is the expected case when using v1 package relay on
> > > > transactions that have more than one generation of unconfirmed
> > > > ancestor) then the remaining transactions are already properly ordered,
> > > > and this is helpful even if unenforceable in general.
> > > >
> > > > The other observation I wanted to make was that having transaction
> > > > relay gated on whether two nodes agree on chain tip seems like an
> > > > overly restrictive criteria. I think an important design principle
> > > > is that we want to minimize disruption from network splits -- if
> > > > there are competing blocks found in a small window of time, it's
> > > > likely that the utxo set is not materially different on the two
> > > > chains (assuming miners are selecting from roughly the same sets of
> > > > transactions when this happens, which is typical). Having
> > > > transaction relay bifurcate on the two network halves would seem to
> > > > exacerbate the difference between the two sides of the split --
> > > > users ought to be agnostic about how benign splits are resolved and
> > > > would likely want their transactions to relay across the whole network.
> > > >
> > > > Additionally, use of a chain tip might impose a larger burden than
> > > > is necessary on software that would seek to participate in
> > > > transaction relay without implementing headers sync/validation. I
> > > > don't know what software exists on the network, but I imagine there
> > > > are a lot of scripts out there for transaction submission to the
> > > > public p2p network, and in thinking about modifying such a script to
> > > > utilize package relay it seems like an unnecessary added burden to first
> > > > learn a node's tip before trying to relay a transaction.
> > > >
> > > > Could you explain again what the benefit of including the blockhash
> > > > is? It seems like it is just so that a node could prioritize
> > > > transaction relay from peers with the same chain tip to maximize the
> > > > likelihood of transaction acceptance, but in the common case this
> > > > seems like a pretty negligible concern, and in the case of a chain
> > > > fork that persists for many minutes it seems better to me that we
> > > > not partition the network into package-relay regimes and just risk a
> > > > little extra bandwidth in one direction or the other. If we solve
> > > > the problem I brought up at the beginning (of de-duplicating package
> > > > data across peers with a package-wtxid-commitment in the
> > > > announcement), I think this is just some wasted pkginfo bandwidth on
> > > > a single-link, and not across links (as we could cache validation failure for a
> > > > package-hash to avoid re-requesting duplicate pkginfo1 messages).
> > > >
> > > > Best,
> > > > Suhas
> > > >
> > > > On Tue, Jun 7, 2022 at 1:57 PM Gloria Zhao via bitcoin-dev <bitcoin-
> > > > dev at lists.linuxfoundation.org <mailto:bitcoin-
> > > > dev at lists.linuxfoundation.org> > wrote:
> > > >
> > > > Hi Eric, aj, all,
> > > >
> > > > Sorry for the delayed response. @aj I'm including some paraphrased
> > > > points from our offline discussion (thanks).
> > > >
> > > > > Other idea: what if you encode the parent txs as a short hash of
> > > > > the wtxid (something like bip152 short ids? perhaps seeded per
> > > > > peer so collisions will be different per peer?) and include that in the inv
> > > > > announcement?
> > > > > Would that work to avoid a round trip almost all of the time,
> > > > > while still giving you enough info to save bw by deduping parents?
> > > >
> > > > > As I suggested earlier, a package is fundamentally a compact block
> > > > > (or
> > > > > block) announcement without the header. Compact block (BIP152)
> > > > > announcement is already well-defined and widely implemented...
> > > >
> > > > > Let us not reinvent the wheel and/or introduce accidental
> > > > > complexity. I see no reason why packaging is not simply BIP152
> > > > > without the 'header'
> > > > > field, an
> > > > > updated protocol version, and the following sort of changes to
> > > > > names
> > > >
> > > > Interestingly, "why not use BIP 152 shortids to save bandwidth?" is
> > > > by far the most common suggestion I hear (including offline feedback).
> > > > Here's a full explanation:
> > > >
> > > > BIP 152 shortens transaction hashes (32 bytes) to shortids (6 bytes)
> > > > to save a significant amount of network bandwidth, which is
> > > > extremely important in block relay. However, this comes at the
> > > > expense of computational complexity. There is no way to directly
> > > > calculate a transaction hash from a shortid; upon receipt of a
> > > > compact block, a node is expected to calculate the shortids of every
> > > > unconfirmed transaction it knows about to find the matches (BIP 152:
> > > > 1, Bitcoin Core: 2). This is expensive but appropriate for block
> > > > relay, since the block must have a valid Proof of Work and new
> > > > blocks only come every ~10 minutes. On the other hand, if we require
> > > > nodes to calculate shortids for every transaction in their mempools every
> > > > time they receive a package, we are creating a DoS vector.
> > > > Unconfirmed transactions don't need PoW and, to have a live
> > > > transaction relay network, we should expect nodes to handle
> > > > transactions at a high-ish rate (i.e. at least 1000s of times more
> > > > transactions than blocks). We can't pre- calculate or cache shortids
> > > > for mempool transactions, since the SipHash key depends on the block
> > > > hash and a per-connection salt.
> > > >
> > > > Additionally, shortid calculation is not designed to prevent
> > > > intentional individual collisions. If we were to use these shortids
> > > > to deduplicate transactions we've supposedly already seen, we may
> > > > have a censorship vector. Again, these tradeoffs make sense for
> > > > compact block relay (see shortid section in BIP 152 [3]), but not package
> > > > relay.
> > > >
> > > > TLDR: DoSy if we calculate shortids on every package and censorship
> > > > vector if we use shortids for deduplication.
> > > >
> > > > > Given this message there is no reason to send a (potentially
> > > > > bogus) fee rate with every package. It can only be validated by
> > > > > obtaining the full set of txs, and the only recourse is dropping
> > > > > (etc.) the peer, as is the case with single txs.
> > > >
> > > > Yeah, I agree with this. Combined with the previous discussion with
> > > > aj (i.e. we can't accurately communicate the incentive compatibility
> > > > of a package without sending the full graph, and this whole dance is
> > > > to avoid downloading a few low-fee transactions in uncommon edge
> > > > cases), I've realized I should remove the fee + weight information
> > > > from pkginfo. Yay for less complexity!
> > > >
> > > > Also, this might be pedantic, but I said something incorrect earlier
> > > > and would like to correct myself:
> > > >
> > > > > > In theory, yes, but maybe it was announced earlier (while our
> > > > > > node was down?) or had dropped from our mempool or similar,
> > > > > > either way we don't have those txs yet.
> > > >
> > > > I said "It's fine if they have Erlay, since a sender would know in
> > > > advance that B is missing and announce it as a package." But this
> > > > isn't true since we're only using reconciliation in place of
> > > > flooding to announce transactions as they arrive, not for
> > > > rebroadcast, and we're not doing full mempool set reconciliation. In
> > > > any case, making sure a node receives the transactions announced
> > > > when it was offline is not something we guarantee, not an intended use
> > > > case for package relay, and not worsened by this.
> > > >
> > > > Thanks for your feedback!
> > > >
> > > > Best,
> > > >
> > > > Gloria
> > > >
> > > > 0152.mediawiki#cmpctblock
> > > > 2:
> > > > https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.cp
> > > > p#L49
> > > > [3]: https://github.com/bitcoin/bips/blob/master/bip-
> > > > 0152.mediawiki#short-transaction-id-calculation
> > > >
> > > > On Thu, May 26, 2022 at 3:59 AM <eric at voskuil.org
> > > > mailto:eric at voskuil.org > wrote:
> > > >
> > > > Given that packages have no header, the package requires identity in
> > > > a
> > > > BIP152 scheme. For example 'header' and 'blockhash' fields can be
> > > > replaced with a Merkle root (e.g. "identity" field) for the package,
> > > > uniquely identifying the partially-ordered set of txs. And use of
> > > > 'getdata' (to obtain a package by hash) can be eliminated (not a use
> > > > case).
> > > >
> > > > e
> > > >
> > > > > -----Original Message-----
> > > > > From: eric at voskuil.org mailto:eric at voskuil.org
> > > > > <eric at voskuil.org mailto:eric at voskuil.org >
> > > > > Sent: Wednesday, May 25, 2022 1:52 PM
> > > > > To: 'Anthony Towns' <aj at erisian.com.au
> > > > > mailto:aj at erisian.com.au >; 'Bitcoin Protocol Discussion'
> > > > > <bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-
> > > > > dev at lists.linuxfoundation.org> >; 'Gloria Zhao'
> > > > > <gloriajzhao at gmail.com mailto:gloriajzhao at gmail.com >
> > > > > Subject: RE: [bitcoin-dev] Package Relay Proposal
> > > > >
> > > > > > From: bitcoin-dev <bitcoin-dev-
> > > > > > bounces at lists.linuxfoundation.org <mailto:bitcoin-dev-
> > > > > > bounces at lists.linuxfoundation.org> > On
> > > > > > Behalf
> > > > > > Of Anthony Towns via bitcoin-dev
> > > > > > Sent: Wednesday, May 25, 2022 11:56 AM
> > > > >
> > > > > > So the other thing is what happens if the peer
> > > > > > announcing packages to us
> > > > > > is
> > > > > > dishonest?
> > > > > >
> > > > > > They announce pkg X, say X has parents A B C and the fee
> > > > > > rate is
> > > > > > garbage.
> > > > > > But
> > > > > > actually X has parent D and the fee rate is excellent. Do
> > > > > > we request the
> > > > > > package from another peer, or every peer, to double
> > > > > > check? Otherwise
> > > > > > we're
> > > > > > allowing the first peer we ask about a package to censor
> > > > > > that tx from
> > > > > > us?
> > > > > >
> > > > > > I think the fix for that is just to provide the fee and weight
> > > > > > when
> > > > > > announcing
> > > > > > the package rather than only being asked for its info?
> > > > > > Then if one peer
> > > > > > makes
> > > > > > it sound like a good deal you ask for the parent txids from
> > > > > > them,
> > > > > > dedupe,
> > > > > > request, and verify they were honest about the parents.
> > > > >
> > > > > Single tx broadcasts do not carry an advertised fee rate,
> > > > > however the'
> > > > > feefilter' message (BIP133) provides this distinction. This
> > > > > should be
> > > > > interpreted as applicable to packages. Given this message
> > > > > there is no
> > > > > reason
> > > > > to send a (potentially bogus) fee rate with every package. It
> > > > > can only be
> > > > > validated by obtaining the full set of txs, and the only
> > > > > recourse is
> > > > > dropping (etc.) the peer, as is the case with single txs.
> > > > > Relying on the
> > > > > existing message is simpler, more consistent, and more
> > > > > efficient.
> > > > >
> > > > > > > > Is it plausible to add the graph in?
> > > > > >
> > > > > > Likewise, I think you'd have to have the graph info from
> > > > > > many nodes if
> > > > > > you're
> > > > > > going to make decisions based on it and don't want
> > > > > > hostile peers to be
> > > > > > able to
> > > > > > trick you into ignoring txs.
> > > > > >
> > > > > > Other idea: what if you encode the parent txs as a short
> > > > > > hash of the
> > > > > > wtxid
> > > > > > (something like bip152 short ids? perhaps seeded per
> > > > > > peer so collisions
> > > > > > will
> > > > > > be different per peer?) and include that in the inv
> > > > > > announcement? Would
> > > > > > that work to avoid a round trip almost all of the time,
> > > > > > while still
> > > > > > giving
> > > > > > you
> > > > > > enough info to save bw by deduping parents?
> > > > >
> > > > > As I suggested earlier, a package is fundamentally a
> > > > > compact block (or
> > > > > block) announcement without the header. Compact block
> > > > > (BIP152)
> > > > > announcement
> > > > > is already well-defined and widely implemented. A node
> > > > > should never be
> > > > > required to retain an orphan, and BIP152 ensures this is not
> > > > > required.
> > > > >
> > > > > Once a validated set of txs within the package has been
> > > > > obtained with
> > > > > sufficient fee, a fee-optimal node would accept the largest
> > > > > subgraph of
> > > > > the
> > > > > package that conforms to fee constraints and drop any
> > > > > peer that provides a
> > > > > package for which the full graph does not.
> > > > >
> > > > > Let us not reinvent the wheel and/or introduce accidental
> > > > > complexity. I
> > > > > see
> > > > > no reason why packaging is not simply BIP152 without the
> > > > > 'header' field,
> > > > > an
> > > > > updated protocol version, and the following sort of changes
> > > > > to names:
> > > > >
> > > > > sendpkg
> > > > > MSG_CMPCT_PKG
> > > > > cmpctpkg
> > > > > getpkgtxn
> > > > > pkgtxn
> > > > >
> > > > > > > For a maximum 25 transactions,
> > > > > > > 2324/2 = 276, seems like 36 bytes for a child-with-
> > > > > > > parents package.
> > > > > >
> > > > > > If you're doing short ids that's maybe 254B=100B
> > > > > > already, then the
> > > > > > above
> > > > > > is
> > > > > > up to 36% overhead, I guess. Might be worth thinking
> > > > > > more about, but
> > > > > > maybe
> > > > > > more interesting with ancestors than just parents.
> > > > > >
> > > > > > > Also side note, since there are no size/count params,
> > > > >
> > > > > Size is restricted in the same manner as block and
> > > > > transaction broadcasts,
> > > > > by consensus. If the fee rate is sufficient there would be no
> > > > > reason to
> > > > > preclude any valid size up to what can be mined in one
> > > > > block (packaging
> > > > > across blocks is not economically rational under the
> > > > > assumption that one
> > > > > miner cannot expect to mine multiple blocks in a row).
> > > > > Count is
> > > > > incorporated
> > > > > into BIP152 as 'shortids_length'.
> > > > >
> > > > > > > wondering if we
> > > > > > > should just have "version" in "sendpackages" be a bit
> > > > > > > field instead of
> > > > > > > sending a message for each version. 32 versions should
> > > > > > > be enough right?
> > > > >
> > > > > Adding versioning to individual protocols is just a reflection
> > > > > of the
> > > > > insufficiency of the initial protocol versioning design, and
> > > > > that of the
> > > > > various ad-hoc changes to it (including yet another
> > > > > approach in this
> > > > > proposal) that have been introduced to compensate for it,
> > > > > though I'll
> > > > > address this in an independent post at some point.
> > > > >
> > > > > Best,
> > > > > e
> > > > >
> > > > > > Maybe but a couple of messages per connection doesn't
> > > > > > really seem worth
> > > > > > arguing about?
> > > > > >
> > > > > > Cheers,
> > > > > > aj
> > > > > >
> > > > > > --
> > > > > > Sent from my phone.
> > > >
> > > > _______________________________________________
> > > >
> > > > > > bitcoin-dev mailing list
> > > > > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-
> > > > > > dev at lists.linuxfoundation.org>
> > > >
> > > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> > > >
> > > > _______________________________________________
> > > > bitcoin-dev mailing list
> > > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-
> > > > dev at lists.linuxfoundation.org>
> > > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> > >
> > > _______________________________________________
> > > bitcoin-dev mailing list
> > > bitcoin-dev at lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2022-06-26
📝 Original message:Hi Antoine,
Thanks for sharing the DoS attack example with alternatives.
> - Caroll broadcasts a double-spend of her own input C, the double-spend is attached with a low-fee (1sat/vb) and it does _not_ signal opt-in RBF
> - Alice broadcasts the multi-party transaction, it is rejected by the network mempools because Alice double-spend is already present
I think this affects almost all types of coinjoin transaction including coordinator based implementations. I tried a few things and have already reported details for an example DoS attack to one of the team but there is no response yet.
It was fun playing with RBF, DoS and Coinjoin. Affected projects should share their opinion about full-rbf as it seems it might improve things.
Example:
In Wasabi an attacker can broadcast a transaction spending input used in coinjoin after sending signature in the round. This would result in a coinjoin tx which never gets relayed: https://nitter.net/1440000bytes/status/1540727534093905920
/dev/fd0
Sent with Proton Mail secure email.
------- Original Message -------
On Tuesday, June 21st, 2022 at 11:43 PM, Antoine Riard <antoine.riard at gmail.com> wrote:
> HI alicexbt,
>
> > Lets consider there are 2 users with name Bob (normal LN user) and Carol (attacker running LN node) which I will use in this email for examples. In this case Bob and Carol can manage security of their OS and it is not affected by others using vulnerable systems or OS.
>
> Yes, I believe my argument was the set of components making the security of your LN node is far beyond Bitcoin softwares. Of course, you might review by yourself the millions lines of code entering in the trusted computing base (OS, bootloader, BIOS, device firmwares, essential utilities, ...) on which your cryptocurrency software stack lays out, and as such exercise an extended span of control on your personal computation. Though, while I hope we'll have more LN node operators doing so, I'm not sure it's realistic to expect it will be the behavior standard among them..
>
> > The odds are low as you said, this can be managed by Bob and Carol because they can use a better ISP. Others using ISP with some issues may not affect their LN usage.
>
> Sure, though as I would like to underscore being dependent on a Bitcoin node policy and being dependent on a ISP internet traffic routing policy could be analyzed as logically equivalent, all things are equal. That said, if your personal risk aversion is too high for the Lightning security model, once it's well-understood there is a strong reliance on a censorship-resistant tx-relay network back to economically-rational miners, you're free to not use it and satisfy yourself with the Bitcoin base layer.
>
> > Bob might use full-rbf as its suggested by LN developers for secure LN usage and better for miners. Carol could use a different RBF policy for some nodes and mining. In this case Bob may get affected at some point because of Carol's choice to use a different RBF policy which was not true above.
>
> Indeed, your secure LN usage is going to be dependent of the number of p2p network nodes running an economically-rational policy like full-rbf. That said, I think it's reasonable to assume that the players of the Bitcoin game are economically-rational, and as such incentived to pick up a policy such as full-rbf. I know the term "economically-rational" is poorly defined here, and I think it could be interesting for any academic with an economic background to study the incentives of Bitcoin actors.
>
> > Allowing users to create different mempool policies is great. My thought process is to code for happy path, where X policy is expected for replacement and edge cases where Y policy or no policy would be used. Users can try out different policies and even act as attackers. This is also true for other things in mempool, 'spkreuse=conflict' prevents address reuse in the mempool when using knots. If I assume that address reuse is always relayed, this could become a problem if some users and miners adopt this setting in their mempool.
>
> Agree, I'm all in for people to experiment with mempool policies. Though at the end it's a software engineering resource question. If users are interested in more features, they're always free to implement themselves. Really, the binary distinction developers-vs-users doesn't make sense and if we would like Bitcoin to be successful in the long-term, we should promote high degree of software literacy among bitcoiners.
>
> > This makes sense and I would be interested to follow two things once full-rbf is available in a bitcoin core release: 1. Percentage of transaction getting replaced 2. Miners profit (Fee for replaced Tx - Fee for original Tx)
>
> Yes, I would be interested too to have those metrics once full-rbf is available in a bitcoin core release. I think that's something every full-rbf curious node operator could observe on its own with a few more loggers, at least for the first metric.
>
> > Can you explain how p2p coinjoin is affected with mempool DoS vector with some examples? What is considered a p2p coinjoin? Joinmarket or [Stonewall][1]?
>
> I don't remember the Joinmarket code right now and I don't know the ins and outs of Samourai coinjoin as I'm not sure the code is open source. Though let's say for a p2p coinjoin as one you can build once you have implemented LN's interactive construction protocol [0].
>
> [0] https://github.com/lightning/bolts/pull/851
>
> Here the DoS attack situation :
> - Alice, Bob and Caroll would like to coinjoin 3 inputs to 3 outputs
> - Each of the input is singly controlled by one party, e.g Alice owns input A, Bob owns input B, ...
> - Alice, Bob and Caroll exchanges a PSBT to build a multi-party funded transaction (the coinjoin_tx)
> - Alice is elected as the multi-party transaction broadcaster once the signatures have been exchanged
> - The block feerate is around 10sat/vb
> - One of the transaction input signals opt-in RBF, the transaction is attached a compelling feerate 10sat/vb
> - Caroll broadcasts a double-spend of her own input C, the double-spend is attached with a low-fee (1sat/vb) and it does _not_ signal opt-in RBF
> - Alice broadcasts the multi-party transaction, it is rejected by the network mempools because Alice double-spend is already present
> - Two alternatives are offered to the coinjoin participants :
>
> Alternative A)
> - They estimate the multi-party feerate as not high enough
> - They fee-bump at 20sat/vb
> - Caroll double-spend one of the input of her malicious double-spend to eject it from the network mempools
> - The multi-party transaction is confirmed at a block feerare far above what was necessary
> - Alice, Bob, Caroll have loss fee-bumping value without compensation
> - Note, even if Caroll is attacker and assuming the fee-bumping burden is fairly spread among participants, the economic loss inflicted is asymmetric
>
> Alternative B)
> - They wait until some time-out
> - They double-spend their own inputs, Alice double-spend utxo A, Bob double-spend utxo B
> - They wasted the timevalue of their inputs for the time-out delay
> - Note, even if Caroll is attacker and loss some timevalue too, the economic loss inflicted is asymmetric
>
> Let me know if you see any error or wrong in this DoS scenario exposure. I believe it's fairly simple to execute
> for a medium-skilled attacker.
>
> Antoine
>
> Le ven. 17 juin 2022 à 00:54, alicexbt <alicexbt at protonmail.com> a écrit :
>
> > Hi Antoine,
> >
> >
> >
> > > One could list the operating system on which is running your Lightning process or the compiler toolchain turning out your Lightning source code in a binary artifact. Some weird kernel's memory mapping change could allow access to your channel funding keys, _without_ breaking the Bitcoin consensus rules [0].
> >
> > Lets consider there are 2 users with name Bob (normal LN user) and Carol (attacker running LN node) which I will use in this email for examples. In this case Bob and Carol can manage security of their OS and it is not affected by others using vulnerable systems or OS.
> >
> >
> > > Moreover, your Lightning node is also relying on the existence of a global Internet allowing your HTLC transaction to flow from your physical host to the crowd of transactions confirming in the blockchain. Due to this "protocol assumption" your channel balance would be vulnerable to any change in your ISP routing policy, e.g refusing to accept your IPV4 traffic by a sudden desiderata to impose an IPV6 supremacy. Still _without_ breaking the Bitcoin consensus rules. Of course, the odds of your ISP operator adopting this behavior are really low, mostly because your operator has to bind to social and economic constraints to stay in business.
> >
> > The odds are low as you said, this can be managed by Bob and Carol because they can use a better ISP. Others using ISP with some issues may not affect their LN usage.
> >
> >
> > > And I believe this imperative to stay in business is certainly not absent in the incentives of the Bitcoin node operators. You're free to run any policy on your node, especially one hardening the safety of your operations beyond the default one. However, if you start to a transaction-relay non-compatible with miner incentives, you won't have an efficient view of the blockspace demand, and from then won't be able to offer compelling feerates to execute your business transactions to satisfy your client needs. Or you won't consolidate your wallet UTXOs at times of low-demand. Indeed, a sane visibility of the mempools might not be critical now for your Bitcoin operations, but this is not likely to become true with miner's coinbase reward lowering with time and the system security relying on a fruitful fee market.
> >
> > Bob might use full-rbf as its suggested by LN developers for secure LN usage and better for miners. Carol could use a different RBF policy for some nodes and mining. In this case Bob may get affected at some point because of Carol's choice to use a different RBF policy which was not true above.
> >
> >
> >
> > > So assuming there is a significant number of economically rational entities running p2p nodes, I think it's a reasonable assumption for Lightning developers that a policy maximizing miner's income and economic nodes operations will be widely run on the p2p network, and therefore lay its security model on that. When there is a gap between the economically optimal policy (full-rbf) and the effectively deployed one (optin), and this gap constitutes a flaw for exploitation, I believe it's better to fix it.
> >
> > Agree with the assumption there is nothing wrong in experimenting with a new RBF policy (non-default) if that helps some users and projects.
> >
> >
> > > If you have a different mode of thinking w.r.t how we should design protocol in a trust-minimized, open, adversarial environment such as Bitcoin, I'm curious to listen to it.
> >
> > Allowing users to create different mempool policies is great. My thought process is to code for happy path, where X policy is expected for replacement and edge cases where Y policy or no policy would be used. Users can try out different policies and even act as attackers. This is also true for other things in mempool, 'spkreuse=conflict' prevents address reuse in the mempool when using knots. If I assume that address reuse is always relayed, this could become a problem if some users and miners adopt this setting in their mempool.
> >
> >
> > > Of course not. If you deliver any critical software, you should attach a solid manual explaining all the corner cases and rough edges. Even better would be to enshrine the manual directly in your software API to minimize the footgunish behaviors. E.g, with any ECC library, forbidding to reuse nonces. If your user still ignores or misread the manual and provides an insecure input, there is not that much you can do.
> >
> > Agree with the documentation as it helps users.
> >
> >
> > > Given there are like 17000 public LN nodes, if half of them adopt full-rbf it should give already a good number of full-rbf transaction-relay routes across the p2p network graph. When we're there, we can measure and think more about how to tune the full-rbf sub-topology.
> >
> > Sounds good.
> >
> >
> > > Because it's breaking the reliability and security of their use-cases. Use-cases which didn't exist a few years ago. The mempool DoS vector is described here [4]. To the best of my understanding, it might affect a bunch of use-cases, such as dual-funded channels, on-chain DLCs, p2p coinjoins, batched submarine swaps out. With the attack described, the honest set of users might not have visibility of the network mempools that there is a malicious, low-cost, opt-out double-spend preventing the propagation of their multi-party transaction. With the existence of a full-rbf transaction-relay topology, the multi-party transaction is able to replace the optout.
> >
> > This makes sense and I would be interested to follow two things once full-rbf is available in a bitcoin core release: 1. Percentage of transaction getting replaced 2. Miners profit (Fee for replaced Tx - Fee for original Tx)
> >
> >
> > Can you explain how p2p coinjoin is affected with mempool DoS vector with some examples? What is considered a p2p coinjoin? Joinmarket or [Stonewall][1]?
> >
> >
> > > Selecting a full-node to underpin any serious Bitcoin infrastructure or secure a significant stack of coins should be submitted to a fully-fledged decision-making process. Many factors are likely to matter such as the level of activity of the contributor community, the chain of trust w.r.t dependencies, the security incident track records, the quality of the documentation, the exhaustivity and robustness of the set of features, ...
> >
> > I agree that contributor community and documentation could be improved in Knots.
> >
> >
> > > Developers are also Bitcoin users, and they're modifying the software to suit their use-case needs. And that's exactly the purpose of the 'full-rbf' PR I'm proposing, aiming to propose a "good" policy for a Lightning node, without actually seeking to change the default.
> >
> > I like that default still remains opt-in and cool with different policies being tried out if that helps some users.
> >
> >
> > > If they're parties interested in implementing more RBF policy options in Bitcoin Core, I think they're free to propose such changes and invest the engineering effort to do so. If you're interested in advancing the state of policy options in Bitcoin Core, there are a lot of interesting resources available and communities to encourage you in the learning process to contribute to the codebase [6].
> >
> > Thanks for sharing the link. I would love to see 5 RBF policies available to use in bitcoin core. I have already tried experimenting with a few on regtest and will try to open pull request if there are enough people interested to test it on other chains (testnet3, signet, mainnet)
> >
> >
> > [1]: https://docs.samourai.io/spend-tools
> >
> >
> >
> >
> > /dev/fd0
> >
> >
> >
> > Sent with Proton Mail secure email.
> >
> > ------- Original Message -------
> > On Friday, June 17th, 2022 at 7:04 AM, Antoine Riard <antoine.riard at gmail.com> wrote:
> >
> >
> > > Hi alicexbt,
> > >
> > >
> > >
> > > Thanks for taking time to review the pull request,
> > >
> > >
> > >
> > > > 1)If something relies on a policy which can be changed without breaking consensus rules, how is it secure in any case with or without full rbf?
> > >
> > >
> > >
> > > Your Lightning node software relies on far more software and hardware components than the transaction-relay p2p network. One could list the operating system on which is running your Lightning process or the compiler toolchain turning out your Lightning source code in a binary artifact. Some weird kernel's memory mapping change could allow access to your channel funding keys, _without_ breaking the Bitcoin consensus rules [0]. Moreover, your Lightning node is also relying on the existence of a global Internet allowing your HTLC transaction to flow from your physical host to the crowd of transactions confirming in the blockchain. Due to this "protocol assumption" your channel balance would be vulnerable to any change in your ISP routing policy, e.g refusing to accept your IPV4 traffic by a sudden desiderata to impose an IPV6 supremacy. Still _without_ breaking the Bitcoin consensus rules. Of course, the odds of your ISP operator adopting this behavior are really low, mostly because your operator has to bind to social and economic constraints to stay in business.
> > >
> > >
> > >
> > > And I believe this imperative to stay in business is certainly not absent in the incentives of the Bitcoin node operators. You're free to run any policy on your node, especially one hardening the safety of your operations beyond the default one. However, if you start to a transaction-relay non-compatible with miner incentives, you won't have an efficient view of the blockspace demand, and from then won't be able to offer compelling feerates to execute your business transactions to satisfy your client needs. Or you won't consolidate your wallet UTXOs at times of low-demand. Indeed, a sane visibility of the mempools might not be critical now for your Bitcoin operations, but this is not likely to become true with miner's coinbase reward lowering with time and the system security relying on a fruitful fee market.
> > >
> > >
> > >
> > > So assuming there is a significant number of economically rational entities running p2p nodes, I think it's a reasonable assumption for Lightning developers that a policy maximizing miner's income and economic nodes operations will be widely run on the p2p network, and therefore lay its security model on that. When there is a gap between the economically optimal policy (full-rbf) and the effectively deployed one (optin), and this gap constitutes a flaw for exploitation, I believe it's better to fix it.
> > >
> > >
> > >
> > > If you have a different mode of thinking w.r.t how we should design protocol in a trust-minimized, open, adversarial environment such as Bitcoin, I'm curious to listen to it.
> > >
> > >
> > >
> > > > If I write a python script that expects user to enter char 'a' or 'b' but user can enter 'c' and there is no code to handle exceptions or other chars, will it be secure?
> > >
> > >
> > >
> > > Of course not. If you deliver any critical software, you should attach a solid manual explaining all the corner cases and rough edges. Even better would be to enshrine the manual directly in your software API to minimize the footgunish behaviors. E.g, with any ECC library, forbidding to reuse nonces. If your user still ignores or misread the manual and provides an insecure input, there is not that much you can do.
> > >
> > >
> > >
> > > By analogy, I believe that's the same with Lightning. One recommendation of the deployment manual would be to be always connected to a full-rbf transaction-relay topology. Defaulting to this rule and your node exposes far more surface of attacks. Assuming the manual has been well-written (big assumption!), I don't think the system designer would be to blame.
> > >
> > >
> > >
> > > That said, one issue to confess with current Lightning is our lack of understanding of what should be figured out in the LN user manual for safe operations. I would say that's an active area of research [1] [2] [3]
> > >
> > >
> > >
> > > > 2)full-rbf is not default in the 2 open pull requests, so this experiment still relies on users changing RBF policies manually. If majority of nodes use default opt-in policy, how would this affect vulnerable projects?
> > >
> > >
> > >
> > > If we define the goal as ensuring there is a significant number of transaction-relay routes between the L2s nodes requiring full-rbf and the set of miners supporting this policy, and the set of miners is populated enough, there is no need to convince the majority of nodes operators to switch to full-rbf.
> > >
> > >
> > >
> > > Beyond landing the 'full-rbf' pull request, in pursuit of a partial full-rbf deployment, I'm thinking of reaching out to Lightning vendors to recommend running LN nodes operators run their full-node with the setting enabled. And also to few mining pool operators to advocate the potential increase in their income.
> > >
> > >
> > >
> > > Given there are like 17000 public LN nodes, if half of them adopt full-rbf it should give already a good number of full-rbf transaction-relay routes across the p2p network graph. When we're there, we can measure and think more about how to tune the full-rbf sub-topology.
> > >
> > >
> > >
> > > > 2-3% transactions are replaced with opt-in RBF, if someone did not replace earlier why would they do it with full RBF?
> > >
> > >
> > >
> > > Because it's breaking the reliability and security of their use-cases. Use-cases which didn't exist a few years ago. The mempool DoS vector is described here [4]. To the best of my understanding, it might affect a bunch of use-cases, such as dual-funded channels, on-chain DLCs, p2p coinjoins, batched submarine swaps out. With the attack described, the honest set of users might not have visibility of the network mempools that there is a malicious, low-cost, opt-out double-spend preventing the propagation of their multi-party transaction. With the existence of a full-rbf transaction-relay topology, the multi-party transaction is able to replace the optout.
> > >
> > >
> > >
> > > None of those use-cases were deployed a few years ago, and the understanding of the interactions with the mempool policy is still nascent among their operators. However, if we assume that layering is a way to grow the Bitcoin ecosystem, as I do, it is reasonable to expect they will constitute a notable share of the Bitcoin transaction traffic during the next decade.
> > >
> > >
> > >
> > > > I am not opposed to full-rbf; rather, I am opposed to the notion that full-rbf will solve all problems
> > >
> > >
> > >
> > > I wished we had a magic Silver Bullet (tm) solving all the Bitcoin problems...
> > >
> > >
> > >
> > > I'm only advocating a partial full-rbf deployment to solve a real precise security issue affecting multi-party funded transactions. That said, full-rbf is far from solving the known set of problems affecting the L2s due to interactions with network mempools. E,g, see package relay motivation [5]
> > >
> > >
> > >
> > > > I would suggest users to try Bitcoin Knots instead which already has an option to disable all RBF policies if required, opt-in and full RBF policy.
> > >
> > >
> > >
> > > Selecting a full-node to underpin any serious Bitcoin infrastructure or secure a significant stack of coins should be submitted to a fully-fledged decision-making process. Many factors are likely to matter such as the level of activity of the contributor community, the chain of trust w.r.t dependencies, the security incident track records, the quality of the documentation, the exhaustivity and robustness of the set of features, ...
> > >
> > >
> > >
> > > This process might take tens of hours, to be duplicated by the number of node operators who would have to do the full-node vending switch. If you consider the cognitive cost at the level of the Bitcoin ecosystem, it's far less costly to implement and review a few lines of codes in Bitcoin Core.
> > >
> > >
> > >
> > > > Developers should provide basic RBF policy options rather than attempting to define what constitutes a good policy and removing the ability to disable something when necessary.
> > >
> > >
> > >
> > > Of course, this statement assumes there is a clear line between the developers and the users. Developers are also Bitcoin users, and they're modifying the software to suit their use-case needs. And that's exactly the purpose of the 'full-rbf' PR I'm proposing, aiming to propose a "good" policy for a Lightning node, without actually seeking to change the default. If they're parties interested in implementing more RBF policy options in Bitcoin Core, I think they're free to propose such changes and invest the engineering effort to do so. If you're interested in advancing the state of policy options in Bitcoin Core, there are a lot of interesting resources available and communities to encourage you in the learning process to contribute to the codebase [6].
> > >
> > >
> > >
> > > Antoine
> > >
> > >
> > >
> > > [0] https://dirtycow.ninja
> > >
> > > [1] https://github.com/t-bast/lightning-docs/blob/master/pinning-attacks.md
> > >
> > > [2] https://arxiv.org/pdf/2006.01418.pdf
> > >
> > > [3] https://arxiv.org/pdf/2006.08513.pdf
> > >
> > > [4] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
> > >
> > > [5] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-May/020493.html
> > >
> > > [6] https://www.summerofbitcoin.org
> > >
> > >
> > > Le jeu. 16 juin 2022 à 00:15, alicexbt <alicexbt at protonmail.com> a écrit :
> > >
> > > > Hi Antoine,
> > > >
> > > >
> > > > Thanks for opening the pull request to add support for full-rbf in Bitcoin Core. I have a disagreements with the approach and questions.
> > > >
> > > >
> > > > > Recent discussions among LN devs have brought back on the surface concerns about the security of multi-party funded transactions (coinjoins, dual-funded LN channels, on-chain DLCs, ...). It turns out there is a low-fruit, naive DoS vector playable against the funding flow of any such construction due to the lack of existent full-rbf transaction-relay topology on today's p2p network [0] [1].
> > > >
> > > > 1)If something relies on a policy which can be changed without breaking consensus rules, how is it secure in any case with or without full rbf? If I write a python script that expects user to enter char 'a' or 'b' but user can enter 'c' and there is no code to handle exceptions or other chars, will it be secure?
> > > >
> > > > 2)full-rbf is not default in the 2 open pull requests, so this experiment still relies on users changing RBF policies manually. If majority of nodes use default opt-in policy, how would this affect vulnerable projects?
> > > >
> > > >
> > > > > If you're a mining operator looking to increase your income, you might be interested to experiment with full-rbf as a policy.
> > > >
> > > > Miners can only increase their income if users replace transactions. 2-3% transactions are replaced with opt-in RBF, if someone did not replace earlier why would they do it now even with full RBF? Or even if we add some users in it who could not signal for some reasons, do you think it would be anything above 5%?
> > > >
> > > >
> > > > > If you're a Bitcoin user or business and you don't like full-rbf, please express an opinion on how it might affect your software/operations. I'm always interested to learn more about mempool and transaction-relay interactions with upper-layers and applications and to listen to feedback in those areas, and I guess a lot of other Bitcoin researchers/devs too. I know there have been a lot of concerns about full-rbf in the past, however I believe the Bitcoin ecosystem has matured a lot since then.
> > > >
> > > > I am not opposed to full-rbf; rather, I am opposed to the notion that full-rbf will solve all problems and the lack of basic options in Bitcoin Core to employ/disable different RBF policies. There is also a speculation about making full RBF default in an year which isn't relevant to discuss at this point without trying different RBF policies.
> > > >
> > > > I would suggest users to try Bitcoin Knots instead which already has an option to disable all RBF policies if required, opt-in and full RBF policy. This can also be done using GUI if not familiar with config option `mempoolreplacement`.
> > > >
> > > > The rationale in PR #16171 was insufficient to justify removing it in the first place, had 2 NACKs and was reopened to merge it. Why bother with a few lines of code that may allow someone disable it if required in local mempool since it's only useful when a big percentage of miners utilize it and essentially underused according to the PR author? Developers should provide basic RBF policy options rather than attempting to define what constitutes a good policy and removing the ability to disable something when necessary.
> > > >
> > > >
> > > > /dev/fd0
> > > >
> > > >
> > > > Sent with Proton Mail secure email.
> > > >
> > > > ------- Original Message -------
> > > > On Tuesday, June 14th, 2022 at 5:55 AM, Antoine Riard via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
> > > >
> > > >
> > > > > Hi list,
> > > > >
> > > > > Recent discussions among LN devs have brought back on the surface concerns about the security of multi-party funded transactions (coinjoins, dual-funded LN channels, on-chain DLCs, ...). It turns out there is a low-fruit, naive DoS vector playable against the funding flow of any such construction due to the lack of existent full-rbf transaction-relay topology on today's p2p network [0] [1]. While it does not consist in a direct loss of funds, if exploited well I think it's annoying enough to inflict significant timevalue loss or fee-bumping waste
> > > > > to the future providers or distributed swarm of users doing multi-party funded transactions. Of course, it can be fixed one layer above by introducing either fidelity bonds or a reliable centralized coordinator, though at the price of an overhead per-participant ressources cost and loss in system openness [1].
> > > > >
> > > > > For that reason, I believe it would be beneficial to the flourishing of multi-party funded transactions to fix the Dos vector by seeing a subset of the network running full-rbf and enabling propagation of honest multi-party transactions to the interested miners, replacing potential non-signaling double-spend from a malicious counterparty. Moving towards that direction, I've submitted a small patch against Bitcoin Core enabling it to turn on full-rbf as a policy, still under review [3]. The default setting stays **false**, i.e keeping opt-in RBF as a default replacement policy. I've started to run the patch on a public node at 146.190.224.15.
> > > > >
> > > > > If you're a node operator curious to play with full-rbf, feel free to connect to this node or spawn up a toy, public node yourself. There is a ##uafrbf libera chat if you would like information on the settings or looking for full-rbf friends (though that step could be automated in the future by setting up a dedicated network bit and reserving a few outbound slots for them).
> > > > >
> > > > > If you're a mining operator looking to increase your income, you might be interested to experiment with full-rbf as a policy. Indeed, in the future I believe the multi-party transactions issuers who need full-rbf to secure their funding flow should connect by default to full-rbf peers. One can conjecture that their transactions are likely to be more compelling in their feerate as their liquidity needs are higher than the simple transaction. For today, I think we have really few standards and bitcoin softwares relying on multi-party funded transactions [4].
> > > > >
> > > > > If you're a Bitcoin user or business and you don't like full-rbf, please express an opinion on how it might affect your software/operations. I'm always interested to learn more about mempool and transaction-relay interactions with upper-layers and applications and to listen to feedback in those areas, and I guess a lot of other Bitcoin researchers/devs too. I know there have been a lot of concerns about full-rbf in the past, however I believe the Bitcoin ecosystem has matured a lot since then.
> > > > >
> > > > > Any mistakes or missing context is my own.
> > > > >
> > > > > Cheers,
> > > > > Antoine
> > > > >
> > > > > [0] For more info about replace-by-fee, see https://bitcoinops.org/en/topics/replace-by-fee/
> > > > >
> > > > > [1] For more details about the DoS vector, see https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
> > > > >
> > > > > [2] E.g I think it does not affect the Lightning Pool service, as there is a preliminary step where the participant funds are locked first in a 2-of-2 with the coordinator before being committed in the multi-party batch transaction.
> > > > >
> > > > > [3] https://github.com/bitcoin/bitcoin/pull/25353
> > > > >
> > > > > [4] E.g DLCs : https://github.com/discreetlogcontracts/dlcspecs/blob/master/Transactions.md ; Lightning dual-funded channel : https://github.com/lightning/bolts/pull/851