Kaz Wesley [ARCHIVE] / is on Nostr

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2014-07-17 to 2014-08-08
💬 Posted a total of 14 messages
📚 Participated in the following categories: Bitcoin-development
🔇 Please do not DM this user as it's used for archive purposes only.

Last Notes

2023-06-07 17:24:39

- reply

📅 Original date posted:2014-07-31
📝 Original message:On Thu, Jul 31, 2014 at 6:06 PM, Peter Todd <pete at petertodd.org> wrote:
> Anything that changes the semantics of nLockTime will do harm to
> existing and future applications that make use of nLockTime for things
> like refund transactions.
I think this would be compatible with most uses of nLockTime -- e.g.,
at the time a refund transaction becomes broadcastable, its
beneficiary would usually have no reason to wait for a long time
before broadcasting it; if they did so (probably because they weren't
online to redeem the refund), they'd need to use the
submit-directly-to-miner option, but they wouldn't lose their refund.
> In any case, why do transactions need finite lifespans in mempools? If
> you want to double-spend them with higher fees, then implement
> replace-by-fee.
Perpetuating transactions that have been in mempools for a long time
and are not being confirmed has been cited as a reason for nodes not
to exchange mempools (#3721, #1833, #3722); it's been implied that it
would be desirable for wallets to wait until a transaction had had a
chance to be accepted before double-spending with a higher fee
(#3722); and an unconfirmed transaction-age-based policy for
preventing mempool accumulation has been advocated (#3753, #3722) [I
hope my summarization is not misrepresenting anyone's opinions here;
please see the arguments made in the actual comments on the bugs].
These discussions are mostly fairly old, but I don't know of any
changes that have been made that provide alternative answers to these
concerns mentioned by at least three different developers.
> In any case, lifetimes will never be deterministic as not everyone runs
> the same software.
That's true, but none of the benefits of these changes require the
policy to be unanimous; most of the effect could be provided by most
of the network following these rules.
>> Transactions would stop being relayed and drop out of mempools a fixed
>> number of blocks from their creation; once that window had passed, the
>> sender's wallet could begin to expect the transaction would not be
>> confirmed. In case a reorg displaces a transaction until after its
>> expiry height, a miner can still put it back in the blockchain; the
>> expiry height is just a relay rule. Also, a user who needed to get
>> their original "expired" transaction confirmed could still do so by
>> submitting it directly to a miner with suitable policies.
>
> ...in which case someone will circumvent this IsStandard() rule by
> submitting "expired" transactions directly to miners with suitable
> policies.
Yes, that is a feature. None of the benefits of transaction expiration
rely on expiration being final, and any possible downsides of
expiration are largely mitigated by the option still being available
to get expired transactions mined.

2023-06-07 17:24:38

- reply

📅 Original date posted:2014-07-31
📝 Original message:There is currently little in place for managing transaction lifetime
in the network's mempools (see discussion in github in #3722 "mempool
transaction expiration", and it seems to be a major factor blocking
some mempool exchange, see #1833/1918, #3721). Expiry per-node a
certain amount of wall time after receipt has been proposed, but
that's a fragile mechanism -- a single node could keep all relayable
transactions alive forever by remembering transactions until most
nodes have dropped them and then releasing them back into the wild.
I have a proposal for a way to add finite and predictable lifespans to
transactions in mempools: we d̶e̶s̶t̶r̶o̶y̶ ̶t̶h̶e̶
̶r̶e̶s̶u̶r̶r̶e̶c̶t̶i̶o̶n̶ ̶h̶u̶b̶ use nLockTime and a new standardness
rule. It could be done in stages, would not necessarily require even a
soft fork, and does not cause problems with reorgs like the proposal
in #3509:
1. start setting nLockTime to the current height by default in newly
created transactions (or slightly below the current height, for
reorg-friendliness)
2. once users have had some time to upgrade to clients that set
nLockTime, start discouraging transactions without nLockTime --
possibly with a slightly higher fee required for relay
3. start rate-limiting relay of transactions without an nLockTime
(maybe this alone could be used to achieve [2])
4. add a new IsStandard rule rejecting transactions with an nLockTime
more than N blocks behind the current tip (for some fixed value N, to
be determined)
Transactions would stop being relayed and drop out of mempools a fixed
number of blocks from their creation; once that window had passed, the
sender's wallet could begin to expect the transaction would not be
confirmed. In case a reorg displaces a transaction until after its
expiry height, a miner can still put it back in the blockchain; the
expiry height is just a relay rule. Also, a user who needed to get
their original "expired" transaction confirmed could still do so by
submitting it directly to a miner with suitable policies.

2023-06-07 17:24:09

- reply

📅 Original date posted:2014-07-18
📝 Original message:That's true, but I think it can be balanced with the usefulness of
knowing what messages a node has received. An invack would be sent if
N invs have been received without any resulting getdata; since we're
keeping track of peer inv order, one message can cover an arbitrarily
large series of invs.
On Fri, Jul 18, 2014 at 10:48 AM, Jeff Garzik <jgarzik at bitpay.com> wrote:
> On a flood-fill network, you don't want to create a storm of "I
> already have this" replies.
>
> On Fri, Jul 18, 2014 at 1:39 PM, Kaz Wesley <keziahw at gmail.com> wrote:
>> Peers exchanging mempool priority policies is great; that accomplishes
>> the flexibility in what txes to remember that I was going for with the
>> forget-filters, but much more neatly, with less overhead and some side
>> benefits.
>>
>> Here's what I'm picturing now:
>> - exchange priority policies in peer introductions
>> - assign unique sequential IDs in the order the transactions were
>> inved (per peer)
>> - receiving a getdata for a tx updates last-known-peer-received inv to
>> all invs up to the one referenced
>> - include ID-last-received, last-known-peer-received in sparse block
>> - reference txes in sparse block by index in receiver's
>> prioritiziation with peer's sent invs up to ID-last-received and
>> sender's prior invs up to last-known-peer-received
>>
>> Possible new messages:
>> - sparseblock
>> - invack message a node can send at times when it's received a bunch
>> of invs it already has, so it hasn't acked with a getdata in a while
>> - gettx: getdata, but using new sequential ID to save 28 bytes per tx
>>
>> It seems important for ordering policies to be able to be specified in
>> as much detail as possible. Parameters that should be available:
>> - total inputs
>> - total outputs
>> - bytes
>> - coin days destroyed
>> - net UTXO size change
>> - sigops
>> - is data carrier
>> - is output raw multisig
>> - age in mempool
>> - what else?
>> This parameter set should be extensible to allow for unforeseen future factors.
>>
>> Ordering policies should allow arbitrary algebraic combinations of
>> their parameters, as well as thresholds. Boolean combinations of
>> sub-policies would also be desirable. This could be implemented with a
>> tx-script-like stack-based language, in which each supported tx
>> property is pushed onto the stack by a particular opcode, and
>> +-*//min/max/boolean operators combine them to yield the sort key.
>>
>> Difficult parameters:
>> * Coin-days-destroyed: changes, peers need agreement on when (if?)
>> it's recalculated. Probably can just not recalculate, but peers still
>> need agreement on "time seen" to get CDD.
>> * Age in mempool: seems intractable in terms of time, but could be
>> done easily in terms of "how many txes old is this sequential ID"
>>
>> One potential pitfall: this allows for an environment of completely
>> heterogeneous mempool policies. I think that's a good thing, but we
>> need to avoid a situation where only least-common-denominator
>> transactions make it farther than a hop or two, and we don't want
>> nodes to have a strong preference for connecting to like-minded peers
>> since clustering reduces overall connectivity. It may be worthwhile to
>> add a parallel mechanism for relay policies, to differentiate between
>> what a node would keep in its mempool vs. what it wouldn't even relay
>> and doesn't want to see at all. Relay policies could be specified just
>> like prioritization policies, but with the final stack value evaluated
>> in a boolean context.
>>
>> An interesting additional use of policy-scripts would be a
>> standardized way for miners to include a policy script in a coinbase,
>> allowing miners a mechanism to advertise things like their relative
>> price of sigops vs bytes. Nodes may then choose to take this
>> information into account in order to optimize their mempool policies
>> for likelihood of consistency with future blocks. Since policy scripts
>> provide only relative information on prices of different transaction
>> properties rather than an absolute fee, this should not allow miners
>> to "vote fees up", although care would need to be taken they wouldn't
>> be able to drive up prices by claiming common transaction types are at
>> the high end of the fee scale.
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc. https://bitpay.com/

2023-06-07 17:24:09

- reply

📅 Original date posted:2014-07-18
📝 Original message:I've updated the gist, and added an additional proposal that I think
meshes well:
https://gist.github.com/kazcw/43c97d3924326beca87d#ultra-fast-block-validation
sparseblocks + UFBV would tighten the new-block process to this (when
txes have been received in advance):
- receive block (~2kB for 1000 tx)
- check whether block contains txes known to belong to conflict-sets,
and if so whether more than one tx from a single conflict-set has been
included (a few operations on very small sets)
- relay block (~2kB)
The benefits of these changes only occur when the transactions have
been seen in advance, but incentivizing ahead-of-block transaction
propogation is a plus, as Jeff mentioned; working on a block without
first ensuring peers have its transactions would be very expensive
from a miner's point of view.

2023-06-07 17:24:08

- reply

📅 Original date posted:2014-07-18
📝 Original message:Peers exchanging mempool priority policies is great; that accomplishes
the flexibility in what txes to remember that I was going for with the
forget-filters, but much more neatly, with less overhead and some side
benefits.
Here's what I'm picturing now:
- exchange priority policies in peer introductions
- assign unique sequential IDs in the order the transactions were
inved (per peer)
- receiving a getdata for a tx updates last-known-peer-received inv to
all invs up to the one referenced
- include ID-last-received, last-known-peer-received in sparse block
- reference txes in sparse block by index in receiver's
prioritiziation with peer's sent invs up to ID-last-received and
sender's prior invs up to last-known-peer-received
Possible new messages:
- sparseblock
- invack message a node can send at times when it's received a bunch
of invs it already has, so it hasn't acked with a getdata in a while
- gettx: getdata, but using new sequential ID to save 28 bytes per tx
It seems important for ordering policies to be able to be specified in
as much detail as possible. Parameters that should be available:
- total inputs
- total outputs
- bytes
- coin days destroyed
- net UTXO size change
- sigops
- is data carrier
- is output raw multisig
- age in mempool
- what else?
This parameter set should be extensible to allow for unforeseen future factors.
Ordering policies should allow arbitrary algebraic combinations of
their parameters, as well as thresholds. Boolean combinations of
sub-policies would also be desirable. This could be implemented with a
tx-script-like stack-based language, in which each supported tx
property is pushed onto the stack by a particular opcode, and
+-*//min/max/boolean operators combine them to yield the sort key.
Difficult parameters:
* Coin-days-destroyed: changes, peers need agreement on when (if?)
it's recalculated. Probably can just not recalculate, but peers still
need agreement on "time seen" to get CDD.
* Age in mempool: seems intractable in terms of time, but could be
done easily in terms of "how many txes old is this sequential ID"
One potential pitfall: this allows for an environment of completely
heterogeneous mempool policies. I think that's a good thing, but we
need to avoid a situation where only least-common-denominator
transactions make it farther than a hop or two, and we don't want
nodes to have a strong preference for connecting to like-minded peers
since clustering reduces overall connectivity. It may be worthwhile to
add a parallel mechanism for relay policies, to differentiate between
what a node would keep in its mempool vs. what it wouldn't even relay
and doesn't want to see at all. Relay policies could be specified just
like prioritization policies, but with the final stack value evaluated
in a boolean context.
An interesting additional use of policy-scripts would be a
standardized way for miners to include a policy script in a coinbase,
allowing miners a mechanism to advertise things like their relative
price of sigops vs bytes. Nodes may then choose to take this
information into account in order to optimize their mempool policies
for likelihood of consistency with future blocks. Since policy scripts
provide only relative information on prices of different transaction
properties rather than an absolute fee, this should not allow miners
to "vote fees up", although care would need to be taken they wouldn't
be able to drive up prices by claiming common transaction types are at
the high end of the fee scale.

2023-06-07 17:24:07

- reply

📅 Original date posted:2014-07-17
📝 Original message:I'm moving this design document to a gist so that I can integrate
changes as they come up:
https://gist.github.com/kazcw/43c97d3924326beca87d
One thing that I think is an important improvement over my initial
idea is that the bloom filters don't need to be kept around and built
up, they can just be one-shot and clear any matching entries from the
set of known-knowns upon arrival -- provided a node is careful to
ensure the txes it wants to forget are known-known-known (which isn't
as bad as it sounds) to the peer it's telling it's forgetting them
when the forget-filter arrives.
On Thu, Jul 17, 2014 at 3:46 PM, Gavin Andresen <gavinandresen at gmail.com> wrote:
>
> A couple of half-baked thoughts:
>
> On Thu, Jul 17, 2014 at 5:35 PM, Kaz Wesley <keziahw at gmail.com> wrote:
>>
>> If there's support for this proposal, I can begin working on the specific
>> implementation details, such as the bloom filters, message format, and
>> capability advertisment, and draft a BIP once I have a concrete proposal for
>> what those would look like and a corresponding precise cost/benefit analysis.
>
>
> I'd encourage you to code up a prototype first (or at the same time), in whatever programming language / networking library you're most familiar with.
>
> Maybe not even using the existing p2p protocol; there could be a mining-only very-fast-block-propagation network separate from the existing p2p network.
>
> Combining your optimizations with "broadcast as many near-miss blocks as bandwidth will allow" on a mining backbone network should allow insanely fast propagation of most newly solved blocks.
>
> --
> --
> Gavin Andresen
Thanks Gavin, I am planning on working out the design details as I
work on a prototype. I have the beginnings of a previous shot at
implementing this in bitcoind to start from but my new design has some
important improvements to add to that.
-kaz