Cory Fields [ARCHIVE]

⚠️ This account is not associated with a real person. It's an archival account for mailing list messages.
🗓️ Active from 2014-10-15 to 2020-09-19
💬 Posted a total of 25 messages
📚 Participated in the following categories: Bitcoin-development, bitcoin-dev
🔇 Please do not DM this user as it's used for archive purposes only.

Public Key

npub19r53ufymeycp9wzk3yzru55y5u46e4vva2lj7nak4zrn6yhrna7qsyj32s

Profile Code

nprofile1qqsz36g7yjdujvqjhptgjpp722z2w2av6kxw40e0f7m23peazt3e7lqpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0dslrykul

Publishing to

relay.damus.io

nos.lol

Author Public Key

npub19r53ufymeycp9wzk3yzru55y5u46e4vva2lj7nak4zrn6yhrna7qsyj32s

Show more details

Last Notes

2023-06-07 20:26:53

- reply

📅 Original date posted:2020-09-18
📝 Original message:Conceptually this is so simple and explicit it almost seems like an obvious
primitive.
Glossing over some of the design/policy decisions...
I wonder what the real-world privacy implications are due to the
dependencies now being encoded on-chain rather than requiring some effort
to watch the mempool?
Cory
On Fri, Sep 18, 2020, 20:52 Jeremy via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi Bitcoin Devs,
>
>
> I'd like to share with you a draft proposal for a mechanism to replace CPFP and RBF for
> increasing fees on transactions in the mempool that should be more robust against attacks.
>
> A reference implementation demonstrating these rules is available
> [here](https://github.com/bitcoin/bitcoin/compare/master...JeremyRubin:subsidy-tx) for those who
> prefer to not read specs.
>
> Should the mailing list formatting be bungled, it is also available as a gist [here](https://gist.github.com/JeremyRubin/92a9fc4c6531817f66c2934282e71fdf).
>
> Non-Destructive TXID Dependencies for Fee Sponsoring
> ====================================================
>
> This BIP proposes a general purpose mechanism for expressing non-destructive (i.e., not requiring
> the spending of a coin) dependencies on specific transactions being in the same block that can be
> used to sponsor fees of remote transactions.
>
> Motivation
> ==========
>
> The mempool has a variety of protections and guards in place to ensure that miners are economic and
> to protect the network from denial of service.
>
> The rough surface of these policies has some unintended consequences for second layer protocol
> developers. Applications are either vulnerable to attacks (such as transaction pinning) or must go
> through great amounts of careful protocol engineering to guard against known mempool attacks.
>
> This is insufficient because if new attacks are found, there is limited ability to deploy fixes for
> them against deployed contract instances (such as open lightning channels). What is required is a
> fully abstracted primitive that requires no special structure from an underlying transaction in
> order to increase fees to confirm the transactions.
>
> Consensus Specification
> =======================
>
> If a transaction's last output's scripPubKey is of the form OP_VER followed by n*32 bytes, where
> n>1, it is interpreted as a vector of TXIDs (Sponsor Vector). The Sponsor Vector TXIDs must also be
> in the block the transaction is validated in, with no restriction on order or on specifying a TXID
> more than once. This can be accomplished simply with the following patch:
>
>
> ```diff
> +
> + // Extract all required fee dependencies
> + std::unordered_set<uint256, SaltedTxidHasher> dependencies;
> +
> + const bool dependencies_enabled = VersionBitsState(pindex->pprev, chainparams.GetConsensus(), Consensus::DeploymentPos::DEPLOYMENT_TXID_DEPENDENCY, versionbitscache) == ThresholdState::ACTIVE;
> + if (dependencies_enabled) {
> + for (const auto& tx : block.vtx) {
> + // dependency output is if the last output of a txn is OP_VER followed by a sequence of 32*n
> + // bytes
> + // vout.back() must exist because it is checked in CheckBlock
> + const CScript& dependencies_script = tx->vout.back().scriptPubKey;
> + // empty scripts are valid, so be sure we have at least one byte
> + if (dependencies_script.size() && dependencies_script[0] == OP_VER) {
> + const size_t size = dependencies_script.size() - 1;
> + if (size % 32 == 0 && size > 0) {
> + for (auto start = dependencies_script.begin() +1, stop = start + 32; start < dependencies_script.end(); start = stop, stop += 32) {
> + uint256 txid;
> + std::copy(start, stop, txid.begin());
> + dependencies.emplace(txid);
> + }
> + }
> + // No rules applied otherwise, open for future upgrades
> + }
> + }
> + if (dependencies.size() > block.vtx.size()) {
> + return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-dependencies-too-many-target-txid");
> + }
> + }
> +
> for (unsigned int i = 0; i < block.vtx.size(); i++)
> {
> const CTransaction &tx = *(block.vtx[i]);
> + if (!dependencies.empty()) {
> + dependencies.erase(tx.GetHash());
> + }
>
> nInputs += tx.vin.size();
>
> @@ -2190,6 +2308,9 @@ bool CChainState::ConnectBlock(const CBlock& block, BlockValidationState& state,
> }
> UpdateCoins(tx, view, i == 0 ? undoDummy : blockundo.vtxundo.back(), pindex->nHeight);
> }
> + if (!dependencies.empty()) {
> + return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-dependency-missing-target-txid");
> + }
> ```
>
> ### Design Motivation
> The final output of a transaction is an unambiguous location to attach metadata to a transaction
> such that the data is available for transaction validation. This data could be committed to anywhere,
> with added implementation complexity, or in the case of Taproot annexes, incompatibility with
> non-Taproot addresses (although this is not a concern for sponsoring a transaction that does not use
> Taproot).
>
> A bare scriptPubKey prefixed with OP_VER is defined to be invalid in any context, and is trivially
> provably unspendable and therefore pruneable.
>
> If there is another convenient place to put the TXID vector, that's fine too.
>
> As the output type is non-standard, unupgraded nodes will by default not include Transactions
> containing them in the mempool, limiting risk of an upgrade via this mechanism.
>
> Policy Specification
> ====================
>
> The mechanism proposed above is a general specification for inter-transaction dependencies.
>
> In this BIP, we only care to ensure a subset of behavior sufficient to replace CPFP and RBF for fee
> bumping.
>
> Thus we restrict the mempool policy such that:
>
> 1. No Transaction with a Sponsor Vector may have any child spends; and
> 1. No Transaction with a Sponsor Vector may have any unconfirmed parents; and
> 1. The Sponsor Vector must have exactly 1 entry; and
> 1. The Sponsor Vector's entry must be present in the mempool; and
> 1. Every Transaction may have exactly 1 sponsor in the mempool; except
> 1. Transactions with a Sponsor Vector may not be sponsored.
>
>
> The mempool treats ancestors and descendants limits as follows:
>
> 1. Sponsors are counted as children transactions for descendants; but
> 1. Sponsoring transactions are exempted from any limits saturated at the time of submission.
>
> This ensures that within a given package, every child transaction may have a sponsor, but that the
> mempool prefers to not accept new true children while there are parents that can be cleared.
>
> To prevent garbage sponsors, we also require that:
>
> 1. The Sponsor's feerate must be greater than the Sponsored's ancestor fee rate
>
> We allow one Sponsor to replace another subject to normal replacement policies, they are treated as
> conflicts.
>
>
> ### Design Motivation
>
> There are a few other ways to use OP_VER sponsors that are not included. For instance, one could
> make child chains that are only valid if their parent is in the same block (this is incompatible
> with CTV, exercise left to reader). These use cases are in a sense incidental to the motivation
> of this mechanism, and add a lot of implementation complexity.
>
> What is wanted is a minimal mechanism that allows arbitrary unconnected third parties to attach
> fees to an arbitrary transaction. The set of rules given tightly bounds how much extra work the
> mempool might have to do to account for the new sponsors in the worst case, while providing a "it
> always works" API for end users that is not subject to traditional issues around pinning.
>
> Eventually, rational miners may wish to permit multiple sponsor targets, or multiple sponsoring
> transactions, but they are not required for the mechanism to work. This is a benefit of the
> minimality of the consensus rule, it is compatible with future policy should it be implemented.
>
>
> #### Attack Analysis of new Policy
>
> In the worst case the new policy can lead to a 1/2 reduction in the number of children allowed
> (e.g., if there are 13 children submitted, then 12 sponsors, the 25 child limit will saturate
> before) and a 2x increase in the maximum children (e.g., if there are 25 children submitted, and
> then each are sponsored). Importantly, even in the latter attack scenario, the DoS surface is not
> great because the sponsor transactions have no children nor parents.
>
> #### Package Relay/Orphan Pool
>
> Future policy work might be able to insert sponsors into a special sponsor pool with an eviction
> policy that would enable sponsors to be queried and tracked for transactions that have too low fee
> to enter the mempool in the first place. This is treated as a separate concern, as any strides on
> package relay generally should be able to support sponsors trivially.
>
> Reference Implementation
> ========================
> A reference implementation demonstrating these rules is available
> [here](https://github.com/bitcoin/bitcoin/compare/master...JeremyRubin:subsidy-tx). This is a best
> effort implementation, but has not been carefully audited for correctness and likely diverges from
> this document in ways that should either be reflected in this document or amended in the code.
>
>
> Best,
>
> Jeremy
>
>
>
> --
> @JeremyRubin <https://twitter.com/JeremyRubin>
> <https://twitter.com/JeremyRubin>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200918/c7e52557/attachment-0001.html>

2023-06-07 20:18:39

- reply

📅 Original date posted:2019-06-20
📝 Original message:A belated thanks (as always) for the great notes, Bryan. This is so
valuable!
Cory
On Fri, Jun 7, 2019 at 11:07 AM Bryan Bishop via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi,
>
> The following are some notes from the coredev.tech Amsterdam 2019 meeting.
> Any mistakes are my probably my own.
>
> Here is a conversation about the code review process in Bitcoin Core:
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-05-code-review/
>
> Here is a conversation with some of the maintainers about what problems
> they are seeing:
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-maintainers/
>
> Wallet re-architecture discussion
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-05-wallet-architecture/
>
> Great consensus cleanup
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-great-consensus-cleanup/
>
> SIGHASH_NOINPUT, OP_CHECKSIGFROMSTACK, OP_CHECKOUTPUTSHASHVERIFY,
> OP_SECURETHEBAG
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-noinput-etc/
>
> Taproot discussion
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-taproot/
>
> Utreexo
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-utreexo/
>
> assumeutxo
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-07-assumeutxo/
>
> Hardware wallets and HWI
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-07-hardware-wallets/
>
> bip151, p2p encryption and v2 message format
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-07-p2p-encryption/
>
> Signet for bitcoin test networks
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-07-signet/
>
> Statechains overview
>
> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-07-statechains/
>
> Thanks,
> - Bryan
> https://heybryan.org/
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190620/611c57f3/attachment.html>

2023-06-07 20:13:48

- reply

📅 Original date posted:2018-07-11
📝 Original message:Hi Simon
The poll is intended for regular Bitcoin Core contributors, who meet
once a week to discuss what they're working on. It is a mundane
software meeting, unrelated to the research and discussion of Bitcoin
itself. The meetings are open to all, but it makes sense to constrain
scheduling decisions to the current active contributors. If you'd like
to attend but are unable, meeting transcripts are always made
available online afterwards.
Please see the Contributing doc [0] if you're interested in becoming a
regular contributor to Bitcoin Core :)
Regards,
Cory
0: https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md
On Tue, Jul 10, 2018 at 7:17 PM, Simon Selitsky <simon at coingyft.com> wrote:
> Cory,
>
> Please send me a link to vote.
>
> Thanks !
>
>> On Jul 10, 2018, at 4:24 PM, Cory Fields via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> Hi all
>>
>> This is a bit offtopic for bitcoin-dev, but I'm sending here since
>> many core devs are subscribed.
>>
>> As discussed in last week's meeting, it would be helpful to have an
>> idea of what times devs are generally available to meet online. With
>> better data, we could potentially reschedule the weekly IRC meetings
>> to a time that would exclude fewer people.
>>
>> I've thrown together a quick poll, please watch for a mail from
>> civs at cs.cornell.edu and follow the link to vote. To keep it simple,
>> the day of the week (Thursday) is fixed for now.
>>
>> The initial list of voters is based on previous meeting participation.
>> If you regularly attend the weekly meetings (or can't now but may be
>> able to attend at a different time), but do not receive a link to vote
>> in the next hour, please reply here or ping me on IRC so that I may
>> add you.
>>
>> Polling will conclude at the end of the scheduled IRC meeting on July 19.
>>
>> Regards,
>> Cory
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 20:13:47

- reply

📅 Original date posted:2018-07-10
📝 Original message:Hi all
This is a bit offtopic for bitcoin-dev, but I'm sending here since
many core devs are subscribed.
As discussed in last week's meeting, it would be helpful to have an
idea of what times devs are generally available to meet online. With
better data, we could potentially reschedule the weekly IRC meetings
to a time that would exclude fewer people.
I've thrown together a quick poll, please watch for a mail from
civs at cs.cornell.edu and follow the link to vote. To keep it simple,
the day of the week (Thursday) is fixed for now.
The initial list of voters is based on previous meeting participation.
If you regularly attend the weekly meetings (or can't now but may be
able to attend at a different time), but do not receive a link to vote
in the next hour, please reply here or ping me on IRC so that I may
add you.
Polling will conclude at the end of the scheduled IRC meeting on July 19.
Regards,
Cory

2023-06-07 20:12:08

- reply

📅 Original date posted:2018-05-17
📝 Original message:Matt: That's a good point. I'll do up a chart comparing utxo size at each
block, as well as comparing over-the-wire size for each block. I think the
period of coalescing earlier this year should be a good example of what
you're describing.
Greg: heh, I was wondering how long it would take for someone to point out
that I'm cheating. I avoided using the word "compression", mostly to
side-step having the discussion turning into reworking the wire
serialization. But you're absolutely right, the de-duping benefits are
independent of the UHS use-case.
Cory
On Thu, May 17, 2018, 12:56 PM Gregory Maxwell <greg at xiph.org> wrote:
> On Wed, May 16, 2018 at 4:36 PM, Cory Fields via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
> > Tl;dr: Rather than storing all unspent outputs, store their hashes.
>
> My initial thoughts are it's not _completely_ obvious to me that a 5%
> ongoing bandwidth increase is actually a win to get something like a
> 40% reduction in the size of a pruned node (and less than a 1%
> reduction in an archive node) primarily because I've not seen size of
> a pruned node cited as a usage limiting factor basically anywhere. I
> would assume it is a win but wouldn't be shocked to see a careful
> analysis that concluded it wasn't.
>
> But perhaps more interestingly, I think the overhead is not really 5%,
> but it's 5% measured in the context of the phenomenally inefficient tx
> mechanisms ( https://bitcointalk.org/index.php?topic=1377345.0 ).
> Napkin math on the size of a txn alone tells me it's more like a 25%
> increase if you just consider size of tx vs size of
> tx+scriptpubkeys,amounts. If I'm not missing something there, I think
> that would get in into a very clear not-win range.
>
> On the positive side is that it doesn't change the blockchain
> datastructure, so it's something implementations could do without
> marrying the network to it forever.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180517/84d84610/attachment.html>

2023-06-07 20:12:06

- reply

📅 Original date posted:2018-05-16
📝 Original message:Tl;dr: Rather than storing all unspent outputs, store their hashes. Untrusted
peers can supply the full outputs when needed, with very little overhead.
Any attempt to spoof those outputs would be apparent, as their hashes would not
be present in the hash set. There are many advantages to this, most apparently
in disk and memory savings, as well as a validation speedup. The primary
disadvantage is a small increase in network traffic. I believe that the
advantages outweigh the disadvantages.
--
Bitcoin’s unspent transaction output set (usually referred to as “The UTXO
set”) has two primary roles: providing proof that previous outputs exist to be
spent, and providing the actual previous output data for verification when new
transactions attempts to spend them. These roles are not usually discussed
independently, but as Bram Cohen's TXO Bitfield [0] idea hints, there are
compelling reasons to consider them this way.
To see why, consider running a node with the following changes:
- For each new output, gather all extra data that will be needed for
verification when spending it later as an input: the amount, scriptPubKey,
creation height, coinbaseness, and output type (p2pkh, p2sh, p2wpkh, etc.).
Call this the Dereferenced Prevout data.
- Create a hash from the concatenation of the new outpoint and the dereferenced
prevout data. Call this a Unspent Transaction Output Hash.
- Rather than storing the full dereferenced prevout entries in a UTXO set as is
currently done, instead store their hashes to an Unspent Transaction Output
Hash Set, or UHS.
- When relaying a transaction, append the dereferenced prevout for each input.
Now when a transaction is received, it contains everything needed for
verification, including the input amount, height, and coinbaseness, which would
have otherwise required a lookup the UTXO set.
To verify an input's unspentness, again create a hash from the concatenation of
the referenced outpoint and the provided dereferenced prevout, and check for
its presence in the UHS. The hash will only be present if a hash of the exact
same data was previously added to (and not since removed from) the UHS. As
such, we are protected from a peer attempting to lie about the dereferenced
prevout data.
### Some benefits of the UHS model
- Requires no consensus changes, purely a p2p/implementation change.
- UHS is substantially smaller than a full UTXO set (just over half for the
main chain, see below). In-memory caching can be much more effective as a
result.
- A block’s transactions can be fully verified before doing a potentially
expensive database lookup for the previous output data. The UHS can be
queried afterwards (or in parallel) to verify previous output inclusion.
- Entire blocks could potentially be verified out-of-order because all input
data is provided; only the inclusion checks have to be in-order. Admittedly
this is likely too complicated to be realistic.
- pay-to-pubkey outputs are less burdensome on full nodes, since they use no
more space on-disk than pay-to-pubkey-hash or pay-to-script-hash. Taproot and
Graftroot outputs may share the same benefits.
- The burden of holding UTXO data is technically shifted from the verifiers to
the spender. In reality, full nodes will likely always have a copy as well,
but conceptually it's a slight improvement to the incentive model.
- Block data from peers can also be used to roll backwards during a reorg. This
potentially enables an even more aggressive pruning mode.
- UTXO storage size grows exactly linearly with UTXO count, as opposed to
growing linearly with UTXO data size. This may be relevant when considering
new larger output types which would otherwise cause the UTXO Set size to
increase more quickly.
- The UHS is a simple set, no need for a key-value database. LevelDB could
potentially be dropped as a dependency in some distant future.
- Potentially integrates nicely with Pieter Wuille's Rolling UTXO set hashes
[1]. Unspent Transaction Output Hashes would simply be mapped to points on a
curve before adding them to the set.
- With the help of inclusion proofs and rolling hashes, libbitcoinconsensus
could potentially safely verify entire blocks. The size of the required
proofs would be largely irrelevant as they would be consumed locally.
- Others?
### TxIn De-duplication
Setting aside the potential benefits, the obvious drawback of using a UHS is a
significant network traffic increase. Fortunately, some properties of
transactions can be exploited to offset most of the difference.
For quick reference:
p2pkh scriptPubKey: DUP HASH160 [pubkey hash] EQUALVERIFY CHECKSIG
p2pkh scriptSig: [signature] [pubkey]
p2sh scriptPubKey: HASH160 [script hash] EQUAL
p2sh scriptSig: [signature(s)] [script]
Notice that if a peer is sending a scriptPubKey and a scriptSig together, as
they would when using a UHS, there would likely be some redundancy. Using a
p2sh output for example, the scriptPubKey contains the script hash, and the
scriptSig contains the script itself. Therefore when sending dereferenced
prevouts over the wire, any hash which can be computed can be omitted and only
the preimages sent.
Non-standard output scripts must be sent in-full, though because they account
for only ~1% of all current UTXOs, they are rare enough to be ignored here.
### Intra-block Script De-duplication
When transactions are chained together in the same block, dereferenced prevout
data for these inputs would be redundant, as the full output data is already
present. For that reason, these dereferenced prevouts can be omitted when
sending over the wire.
The downside would be a new reconstruction pass requirement prior to
validation.
### Data
Here's some preliminary testing with a naive POC implementation patched into
Bitcoin Core. Note that the final sizes will depend on optimization of the
serialization format. The format used for these tests is suboptimal for sure.
Syncing mainnet to block 516346:
UTXO Node UHS Node
IBD Network Data: 153G 157G
Block disk space: 175G 157G
UTXO disk space : 2.8G 1.6G
Total disk space: 177.8G 158.6G
The on-disk block-space reduction comes from the elimination of the Undo data
that Bitcoin Core uses to roll back orphaned blocks. For UHS Nodes, this data
is merged into to the block data and de-duplicated.
Compared to the UXTO model, using a UHS reduces disk space by ~12%, yet only
requires ~5% more data over the wire.
Experimentation shows intra-block de-duplication to be of little help in
practice, as it only reduces overhead by ~0.2% on mainnet. It could become more
useful if, for example, CPFP usage increases substantially in the future.
### Safety
Assuming sha256 for the UHS's hash function, I don't believe any fundamental
changes to Bitcoin's security model are introduced. Because the unspent
transaction output hashes commit to all necessary data, including output types,
it should not be possible to be tricked into validating using mutated or forged
inputs.
### Compatibility
Transitioning from the current UTXO model would be annoying, but not
particularly painful. I'll briefly describe my current preferred approach, but
it makes sense to largely ignore this until there's been some discussion about
UHS in general.
A new service-bit should be allocated to indicate that a node is willing to
serve blocks and transactions with dereferenced prevout data appended. Once
connected to a node advertising this feature, nodes would use a new getdata
flag, creating MSG_PREVDATA_BLOCK and MSG_PREVDATA_TX.
Because current full nodes already have this data readily available in the
block-undo files, it is trivial to append on-the-fly. For that reason, it would
be easy to backport patches to the current stable version of Bitcoin Core in
order to enable serving these blocks even before they could be consumed. This
would avoid an awkward bootstrapping phase where there may only be a few nodes
available to serve to all new nodes.
Admittedly I haven't put much thought into the on-disk format, I'd rather leave
that to a database person. Though it does seem like a reasonable excuse to
consider moving away from LevelDB.
Wallets would begin holding full prevout data for their unspent outputs, though
they could probably back-into the data as-is.
### Serialization
I would prefer to delay this discussion until a more high-level discussion has
been had, otherwise this would be a magnet for nits. The format used to gather
the data above can be seen in the implementation below.
It should be noted, though, that the size of a UHS is directly dependent on the
chosen hash function. Smaller hashes could potentially be used, but I believe
that to be unwise.
### Drawbacks
The primary drawback of this approach is the ~5% network ovhead.
Additionally, there is the possibility that a few "bridge nodes" may be needed
for some time. In a future with a network of only pruned UHS nodes, an old
wallet with no knowledge of its dereferenced prevout data would need to
broadcast an old-style transaction, and have a bridge node append the extra
data before forwarding it along the network.
I won't speculate further there, except to say that I can't imagine a
transition problem that wouldn't have a straightforward solution.
Migration issues aside, am I missing any obvious drawbacks?
### Implementation
This code [2] was a quick hack-job, just enough to gather some initial data. It
builds a UHS in memory and never flushes to disk. Only a single run works,
nasty things will happen upon restart. It should only be viewed in order to get
an idea of what changes are needed. Only enough for IBD is implemented,
mempool/wallet/rpc are likely all broken. It is definitely not consensus-safe.
### Acknowledgement
I consider the UHS concept to be an evolution of Bram Cohen's TXO bitfield
idea. Bram also provided invaluable input when initially walking through the
feasibility of a UHS.
Pieter Wuille's work on Rolling UTXO set hashes served as a catalyst for
rethinking how the UTXO set may be represented.
Additional thanks to those at at Financial Crypto and the CoreDev event
afterwards who helped to flesh out the idea:
Tadge Dryja
Greg Sanders
John Newbery
Neha Narula
Jeremy Rubin
Jim Posen
...and everyone else who has chimed in.
[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013928.html
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014337.html
[2] https://github.com/theuni/bitcoin/tree/utxo-set-hash3

2023-06-07 20:10:48

- reply

📅 Original date posted:2018-02-13
📝 Original message:I agree that this is a bad idea. When trying to work around a social
issue for a highly technical project, a legal hack is certainly not
the answer. As Daniel pointed out, the result of such a change would
simply be that 100% of all Bitcoin companies would be told by their
legal teams to use the last MIT-licensed version of Bitcoin Core as
they would have no idea how to prove that they're not in violation. So
I think it would succeed in exactly the _opposite_ of its intended
purpose.
As Patrick said:
> This software is meant to be free and open for anyone to use, unfortunately that means some people will sometimes do things you disagree with.
Bitcoin is a Kleenex, a Q-Tip, a Xerox in the crypto world. I think we
should just accept that as a feature at this point. Let other projects
faff about with copyright litigation and trademark dilution concerns
:)
Besides, I assume many/most developers would be unwilling to accept
such a change. Speaking for only myself at least, I would not
contribute under that license.
I must admit, though, that it would be fun to read codified
No-True-Scotsman appeals in a software license :p.
Cory
On Tue, Feb 13, 2018 at 12:28 PM, Felix Wolfsteller via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> I'd call the license change an attack on bitcoin if its code license
> prohibited me to play around with it and call it whatever I the fud I want.
> Other entities like companies, goverments and whoknowswhat might
> prohibit that (in some countries of the world), but the nature of the
> source and protocoll shall be Free (as in free speech).
>
> Even if my code changes are compatible with the current blockchain as
> per bitcoin core I would have the lifetime "threat" that one day my code
> wouldnt anymore because of changes in bitcoin core, and I wouldnt like
> to get letters from lawyers earning their money by sending out letters.
>
> Besides I am not fully sure if I could sign the main assumption that the
> forks "... [are] exacerbating the confusion about the very nature of the
> project, and harming it in many ways."
> Or at least I am not sure that the "harm done" __in the end__ is bigger
> than the gains and the proof-of-spirit as well as all the insights
> gained through what happens here, regarding Free (well, MIT) Software
> out in the world. Yes, its not always pleasant but I think its worth it.
>
> -f
>
>
> On 13.02.2018 16:47, Bedri Ozgur Guler via bitcoin-dev wrote:
>> Hello,
>> The use of name Bitcoin cannot be avoided due to it's nature of being a
>> Protocol. Prohibition of usage of it as a "brand name" is just like
>> prohibiting the word "Linux", which is the name of the kernel, being used
>> as a brand name or part of a brand name. If that had happened, systems
>> based on Linux kernel couldn't have used Linux word in their brands. The
>> licence in the Linux example is GPL but it does not really differ so much.
>> Making a protocol name a Trademark(TM) name and prohibiting it's use may
>> solve some confusions and bad reputation causing actions but it also
>> prohibits the protocol to be used widely so damages the credibility of the
>> protocol itself which was born to be an independent, freedom-based,
>> government-free, boundaries-free etc. approach to the current corrupted
>> monetary system.
>>
>> If precautions should be taken to control the usage of Bitcoin word in
>> various positions and cases, it should be done in such a way that it should
>> not contradict with the philosophy of the Bitcoin itself. Social
>> /marketing-based approaches proposed by Jameson Lopp will be more logical
>> and freedom based. Trademarking and in some sense Cartel-ing the Bitcoin
>> Protocol who arose against trademarks and cartels on "money" will destroy
>> it's own roots and birth-right of existence in my opinion.
>>
>> Bedri Özgür Güler
>>
>> On Tue, Feb 13, 2018 at 6:24 PM, Jameson Lopp via bitcoin-dev <
>> bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>>> If I'm understanding the problem being stated correctly:
>>>
>>> "Bitcoin is under a branding attack by fork coins."
>>>
>>> The proposed solution is to disincentivize fork coins from using the word
>>> Bitcoin by altering the license terms. I'm not a lawyer, but it seems to me
>>> that the words of the license are basically useless unless there is an
>>> entity that intends to make use of court systems to threaten noncompliant
>>> projects into submission.
>>>
>>> In my opinion, the perceived attack on Bitcoin here is social /
>>> marketing-based, thus it makes sense that any defense against said attack
>>> should also be social / marketing-based. I don't think that Bitcoin should
>>> be reliant upon courts or governments to defend itself against attacks of
>>> any form.
>>>
>>> On Tue, Feb 13, 2018 at 9:25 AM, Natanael via bitcoin-dev <
>>> bitcoin-dev at lists.linuxfoundation.org> wrote:
>>>
>>>>
>>>>
>>>> Den 13 feb. 2018 15:07 skrev "JOSE FEMENIAS CAÑUELO via bitcoin-dev" <
>>>> bitcoin-dev at lists.linuxfoundation.org>:
>>>>
>>>> ***
>>>> NO PART OF THIS SOFTWARE CAN BE INCLUDED IN ANY OTHER PROJECT THAT USES
>>>> THE NAME BITCOIN AS PART OF ITS NAME AND/OR ITS MARKETING MATERIAL UNLESS
>>>> THE SOFTWARE PRODUCED BY THAT PROJECT IS FULLY COMPATIBLE WITH THE BITCOIN
>>>> (CORE) BLOCKCHAIN
>>>> ***
>>>>
>>>>
>>>> That's better solved with trademarks. (whoever would be the trademark
>>>> holder - Satoshi?)
>>>>
>>>> This would also prohibit any reimplementation that's not formally
>>>> verified to be perfectly compatible from using the name.
>>>>
>>>> It also adds legal uncertainty.
>>>>
>>>> Another major problem is that it neither affects anybody forking older
>>>> versions of Bitcoin, not people using existing independent blockchain
>>>> implementations and renaming them Bitcoin-Whatsoever.
>>>>
>>>> And what happens when an old version is technically incompatible with a
>>>> future version by the Core team due to not understanding various new
>>>> softforks? Which version wins the right to the name?
>>>>
>>>> Also, being unable to even mention Bitcoin is overkill.
>>>>
>>>> The software license also don't affect the blockchain data.
>>>>
>>>>
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev at lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 20:10:27

- reply

📅 Original date posted:2018-01-31
📝 Original message:A public key was published recently for future macOS releases. Sadly,
that key was created the wrong way (iPhone OS instead of macOS), so
another had to be generated.
The new, working pubkey for Bitcoin Core releases starting with
0.16.0rc1 is included in the message below. That message is signed
with the key mentioned in the previous mail.
It can be verified with: openssl smime -verify -noverify -in msg.pem
Sorry for the noise.
-----BEGIN PKCS7-----
MIIPbQYJKoZIhvcNAQcCoIIPXjCCD1oCAQExCzAJBgUrDgMCGgUAMIIC5gYJKoZI
hvcNAQcBoIIC1wSCAtNBIHB1YmxpYyBrZXkgd2FzIHB1Ymxpc2hlZCByZWNlbnRs
eSBmb3IgZnV0dXJlIG1hY09TIHJlbGVhc2VzLg0KDQpTYWRseSwgdGhlIHB1Ymxp
c2hlZCBrZXkgd2FzIGNyZWF0ZWQgdGhlIHdyb25nIHdheSAoaVBob25lIE9TIGlu
c3RlYWQgb2YgbWFjT1MpLCBzbyBhbm90aGVyIGhhZCB0byBiZSByZXF1ZXN0ZWQu
DQoNClRoZSBuZXcsIHdvcmtpbmcgcHVia2V5IGZvciBCaXRjb2luIENvcmUgcmVs
ZWFzZXMgc3RhcnRpbmcgd2l0aCAwLjE2LjByYzEgaXM6DQoNCi0tLS0tQkVHSU4g
UFVCTElDIEtFWS0tLS0tDQpNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4
QU1JSUJDZ0tDQVFFQXF4aWJEZ2pBT09WVXBTY3pVMnBqDQp0UEVpQ0lZeXl2V21E
N2VidGhQbzI5WG9xMUJqYWJGNDlCZ3diNkZFaU1haFN5UTY4ZklMSUhDanJ5SUo4
RUN1DQpROFJWbVF3cGdhKzV0OTZiMEM5emN5WTFhcSsrRzIyMVNqNmFpUmVveXZw
cHIrZ2poNmNPbktEc1B0Z2pUcGdiDQovOUhuMmtwYzFmZ000ZkRFMlQ2VXZHVHMw
d3d5dWNvL21ya0s1LzEySCtqZUE3QXVNcjBLQTBVSktSS1VOenFhDQo4QjlLalFF
ektaRGVVVHRYak9vSmIyNkRQU3hCbXBGd25zWSs2aHBjeFZSSmphNG1FYzRFYnIy
b2gxSmVORU5uDQp4WXR3MHRWVWczTUwvWlI2WU9qQVpMY0V0cW5IR2ZOZXVRazJX
Vm1pYy9JY3d4VEM0cUk4MnFROGgxQnFpY3pRDQo4UUlEQVFBQg0KLS0tLS1FTkQg
UFVCTElDIEtFWS0tLS0tDQqgggnZMIIFzTCCBLWgAwIBAgIId5kUM+xSbWMwDQYJ
KoZIhvcNAQELBQAwgZYxCzAJBgNVBAYTAlVTMRMwEQYDVQQKDApBcHBsZSBJbmMu
MSwwKgYDVQQLDCNBcHBsZSBXb3JsZHdpZGUgRGV2ZWxvcGVyIFJlbGF0aW9uczFE
MEIGA1UEAww7QXBwbGUgV29ybGR3aWRlIERldmVsb3BlciBSZWxhdGlvbnMgQ2Vy
dGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTgwMTEwMjAyNTA1WhcNMTkwMTEwMjAy
NTA1WjCBwDEaMBgGCgmSJomT8ixkAQEMCllaQzdXSDNNUlUxUDBOBgNVBAMMR2lQ
aG9uZSBEaXN0cmlidXRpb246IEJpdGNvaW4gQ29yZSBDb2RlIFNpZ25pbmcgQXNz
b2NpYXRpb24gKFlaQzdXSDNNUlUpMRMwEQYDVQQLDApZWkM3V0gzTVJVMS4wLAYD
VQQKDCVCaXRjb2luIENvcmUgQ29kZSBTaWduaW5nIEFzc29jaWF0aW9uMQswCQYD
VQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKsYmw4IwDjl
VKUnM1NqY7TxIgiGMsr1pg+3m7YT6NvV6KtQY2mxePQYMG+hRIjGoUskOvHyCyBw
o68iCfBArkPEVZkMKYGvubfem9Avc3MmNWqvvhtttUo+mokXqMr6aa/oI4enDpyg
7D7YI06YG//R59pKXNX4DOHwxNk+lLxk7NMMMrnKP5q5Cuf9dh/o3gOwLjK9CgNF
CSkSlDc6mvAfSo0BMymQ3lE7V4zqCW9ugz0sQZqRcJ7GPuoaXMVUSY2uJhHOBG69
qIdSXjRDZ8WLcNLVVINzC/2UemDowGS3BLapxxnzXrkJNllZonPyHMMUwuKiPNqk
PIdQaonM0PECAwEAAaOCAfEwggHtMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcw
AYYjaHR0cDovL29jc3AuYXBwbGUuY29tL29jc3AwMy13d2RyMTEwHQYDVR0OBBYE
FNOBKRRpuWarZwT6owhUtiOP6lbSMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAU
iCcXCam2GGCL7Ou69kdZxVJUo7cwggEdBgNVHSAEggEUMIIBEDCCAQwGCSqGSIb3
Y2QFATCB/jCBwwYIKwYBBQUHAgIwgbYMgbNSZWxpYW5jZSBvbiB0aGlzIGNlcnRp
ZmljYXRlIGJ5IGFueSBwYXJ0eSBhc3N1bWVzIGFjY2VwdGFuY2Ugb2YgdGhlIHRo
ZW4gYXBwbGljYWJsZSBzdGFuZGFyZCB0ZXJtcyBhbmQgY29uZGl0aW9ucyBvZiB1
c2UsIGNlcnRpZmljYXRlIHBvbGljeSBhbmQgY2VydGlmaWNhdGlvbiBwcmFjdGlj
ZSBzdGF0ZW1lbnRzLjA2BggrBgEFBQcCARYqaHR0cDovL3d3dy5hcHBsZS5jb20v
Y2VydGlmaWNhdGVhdXRob3JpdHkvMA4GA1UdDwEB/wQEAwIHgDAWBgNVHSUBAf8E
DDAKBggrBgEFBQcDAzATBgoqhkiG92NkBgEEAQH/BAIFADANBgkqhkiG9w0BAQsF
AAOCAQEARvNgy5mhFqZsI5JGgn6HSR/eQIXjuoGyOivOa6+uCb5qcrSjSR+PSj7D
K/SBxrz+sVgKvwQ3buhv3BJnURmbYtEmqRr60G+yZE6xNpDMEyZyEM7aT6R9zBMX
++5mwqq5Ip57Mq8yB+pGTzSCBUAat6qiMBUkJBa+F/fk+vXZxgKAfKGMEfALLR5j
Rnwadg2CoTng47Mt4gzuGqjRSJH2vlB44GzRiFoJjXJOJGZ0hdagXl1ARTKul1NF
QukGMeJa1xlXzEk2K1sT7inGHEHTO5KD4RyyVFaDTnhWtvDfmDZt5R/Ipfc7KMmc
dObDKqWe/TGoKM5noj3dvafhNFZ9mDCCBAQwggLsoAMCAQICCBh6qajCliEMMA0G
CSqGSIb3DQEBCwUAMGIxCzAJBgNVBAYTAlVTMRMwEQYDVQQKEwpBcHBsZSBJbmMu
MSYwJAYDVQQLEx1BcHBsZSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTEWMBQGA1UE
AxMNQXBwbGUgUm9vdCBDQTAeFw0xMjAyMDEyMjEyMTVaFw0yNzAyMDEyMjEyMTVa
MHkxLTArBgNVBAMMJERldmVsb3BlciBJRCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0
eTEmMCQGA1UECwwdQXBwbGUgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxEzARBgNV
BAoMCkFwcGxlIEluYy4xCzAJBgNVBAYTAlVTMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAiXZPBluaQe6lIysCo1/Xcz/ANbCLhAo/BiR/p5U/608Ok6+0
DtDIPuVtGLMf6IlHv9cJCOT/VpgpFeeUnbk1owrNtMDh4mD0yuwpeEVpaWBrX4qS
/J4j5jrCIrMxTxy68rY0WULusKkCAxiRBLazeC4zH4BFDUVvuw5aW38659gI1wsO
Mm37hjbkbKvEEYpwhCaqn0TR8bjGe5QXm0j3C1gWuiPFnxU5fspdwzJfD+BSf0Dq
vqwIZJVbyRqc5YDKH2pEHGw+xLAmHx3se69eoGo9R6lYEjE/IHYobR0csMJOEWkm
i8vW0BGCyU4P8VZ00NkIS2Z4oqusp+LSTIdZyQIDAQABo4GmMIGjMB0GA1UdDgQW
BBRXF+2iz9x8mKEQ4Py+hy0s8uMXVDAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQY
MBaAFCvQaUeUdgn+9GuNLkCm90dNfwheMC4GA1UdHwQnMCUwI6AhoB+GHWh0dHA6
Ly9jcmwuYXBwbGUuY29tL3Jvb3QuY3JsMA4GA1UdDwEB/wQEAwIBhjAQBgoqhkiG
92NkBgIGBAIFADANBgkqhkiG9w0BAQsFAAOCAQEAQjl0a6HcxqSPNyqMsx0KRLyV
LH+8WbisYfsHkJIyudS/O8FQOWpEdKLsWx9w5ardS2wcI3EtX9HFk77um4pwZYKd
FuMaEBeJLajN/Qx4WEkMKH8z7gB6G7R2rLa1u0/fqBudyBmXSgtWZy/CPrazxIM6
8HdtdMQuI1HumqUDb2D0pUinBsK7WuIfH0ZFfuSX9ScQtyAicm9y2sZQdcU9JY9d
owDpnzaMSDmPszvqkIAulZpg9HjO9A4KUz6i+k/YHq6ElY0yvFZNiel4GOCsmkK6
ekYbhKKJzhToiNFYi/auVsQsBSpFrwvZS6kCDzSsiMdhVYlEySdzB+6C5U71cDGC
An8wggJ7AgEBMIGjMIGWMQswCQYDVQQGEwJVUzETMBEGA1UECgwKQXBwbGUgSW5j
LjEsMCoGA1UECwwjQXBwbGUgV29ybGR3aWRlIERldmVsb3BlciBSZWxhdGlvbnMx
RDBCBgNVBAMMO0FwcGxlIFdvcmxkd2lkZSBEZXZlbG9wZXIgUmVsYXRpb25zIENl
cnRpZmljYXRpb24gQXV0aG9yaXR5Agh3mRQz7FJtYzAJBgUrDgMCGgUAoIGxMBgG
CSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE4MDIwMTAx
MTExNFowIwYJKoZIhvcNAQkEMRYEFNKi/xYPqnN6zp/RogVZBZ3ICGOBMFIGCSqG
SIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3
DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMA0GCSqGSIb3DQEBAQUABIIB
AJQtrcrRd/3PLS9rhey0RyU1ZRnuB4Ib+y/wAan3k+fRNpA70F9kaxxcme78eqho
HH5rvizY4InvrG1wjtpYeickMHp+s0E51j1AbVxOgZ/UiEgjLRq9Dv5OCPgKoLaB
lsyCj41baXvlqzXZ8RaP7Li2SPLpdksqLE5yegiN+yMIiEPfNAtmaRLN3CNnbbMf
X1bF4ifgyhy3P1VGPPk+WTiQyu0VqySrlhz0Ux9+acB/TFUrymFEKxJ/7bM//4nL
UpEQVlnj9rl3OYzhYgDsQgz0kGU+6UG7Iw6gB9xFAMeE/1Y5Xrs2UdjBVC9hkSy8
r1+2rPF1yixiWjiORNk4kyU=
-----END PKCS7-----
Regards,
Cory
On Fri, Jan 12, 2018 at 5:14 AM, nullius via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> On 2018-01-12 at 08:54:12 +0000, Peter Todd <pete at petertodd.org> wrote:
>>
>> While a clunky way to do it, you can use the `-signer` option to tell
>> OpenSSL to write the signer's certificate to a file. That certificate can
>> then be compared to the one from the repo, which was still in the repo as of
>> the (signed!) v0.15.1 tag.
>>
>>
>> Fun fact: OpenTimestamps has git integration, which means you can extract
>> a OTS proof from 2016 for that certificate from the repo:
>>
>> $ git checkout v0.15.1
>> $ ots git-extract share/certs/BitcoinFoundation_Apple_Cert.pem
>> share/certs/BitcoinFoundation_Apple_Cert.pem.ots
>> 36f60a5d5b1bc9a12b87d6475e3245b8236775e4
>> $ ots verify share/certs/BitcoinFoundation_Apple_Cert.pem.ots
>> Assuming target filename is
>> 'share/certs/BitcoinFoundation_Apple_Cert.pem'
>> Success! Bitcoin attests data existed as of Thu Oct 13 14:08:59 2016
>> EDT
>>
>> Homework problem: write a paragraph explaining how the proof generated by
>> the above three commands are crypto snakeoil that proved little. :)
>
>
> It says, “Bitcoin attests data existed”. Within the scope of those three
> commands, I don’t see any proof of who put it there. Does OTS check the PGP
> signatures on *commits* when it does that `git-extract`? The signature on
> the v0.15.1 tag is irrelevant to that question; and FWIW, I don’t see *that*
> signature being verified here, either.
> Second paragraph: Moreover, with the breaking of SHA-1, it *may* be
> feasible for some scenario to play out involving two different PEMs with the
> same hash, but different public keys (and thus different corresponding
> private keys). I don’t know off the top of my head if somewhere could be
> found to stash the magic bits; and the overall scenario would need to be a
> bit convoluted. I think a malicious committer who lacked access to the
> signing key *may* be able to create a collision between the real
> certificate, and a certificate as for which he has the private key—then
> switch them, later. Maybe. I would not discount the possibility off-hand.
> OTS would prove nothing, if he had the foresight to obtain timestamps
> proving that both certificates existed at the appropriate time (which they
> would need to anyway; it is not a post facto preimage attack).
>
>> [...]
>>
>> What's nice about OpenPGP's "clearsigned" format is how it ignores
>> whitespace; a replica of that might be a nice thing for OTS to be able to do
>> too. Though that's on low priority, as there's some tricky design choices(1)
>> to be made about how to nicely nest clearsigned PGP within OTS.
>>
>>
>> 1) For example, I recently found a security hole related to clearsigned
>> PGP recently. Basically the issue was that gpg --verify will return true on
>> a file that looks like the following:
>>
>> 1d7a363ce12430881ec56c9cf1409c49c491043618e598c356e2959040872f5a
>> foo-v2.0.tar.gz
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
>> foo-v1.0.tar.gz
>> -----BEGIN PGP SIGNATURE-----
>>
>> <snip pgp stuff>
>> -----END PGP SIGNATURE-----
>>
>> The system I was auditing then did something like this to verify that the
>> file was signed:
>>
>> set -e # exit immediately on error
>> gpg --verify SHA256SUMS.asc
>> cat SHA256SUMS.asc | grep foo-v2.0.tar.gz
>> <do installation>
>>
>> While it makes it a bit less user friendly, the fact that PKCS7's encoding
>> made it impossible to see the message you signed until it's been properly
>> verified is a good thing re: security.
>
>
> Potential solutions using PGP:
>
> 0. Don’t use clearsigning.
>
> 1. Use a detached signature.
>
> 2. Use `gpg --verify -o -` and pipe that to `grep`, rather than illogically
> separating verification from use of data. (By the way, where is the *hash*
> verified? Was `grep` piped to `sha256sum -c`?)
>
> 3. Have shell scripts written by somebody who knows how to think about
> security, and/or who knows how to RTFM; quoting gpg(1):
>
>> Note: When verifying a cleartext signature, gpg verifies only what makes
>> up the cleartext signed data and not any extra data outside of the cleartext
>> signature or the header lines directly following the dash marker line. The
>> option --output may be used to write out the actual signed data, but there
>> are other pitfalls with this format as well. It is suggested to avoid
>> cleartext signatures in favor of detached signatures.
>
>
> 4. Obtain an audit from Peter Todd.
>
>> And yes, I checked: Bitcoin Core's contrib/verifybinaries/verify.sh isn't
>> vulnerable to this mistake. :)
>
>
> P.S., oh my! *Unsigned data:*
>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> --
> nullius at nym.zone | PGP ECC: 0xC2E91CD74A4C57A105F6C21B5A00591B2F307E0C
> Bitcoin: bc1qcash96s5jqppzsp8hy8swkggf7f6agex98an7h | (Segwit nested:
> 3NULL3ZCUXr7RDLxXeLPDMZDZYxuaYkCnG) (PGP RSA: 0x36EBB4AB699A10EE)
> “‘If you’re not doing anything wrong, you have nothing to hide.’
> No! Because I do nothing wrong, I have nothing to show.” — nullius
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

2023-06-07 20:09:41

- reply

📅 Original date posted:2018-01-12
📝 Original message:Hi all
As discussed in a few of the last weekly meetings, Bitcoin Core's
macOS code signing certificate expired today.
We are (Greg is ;) in the process of establishing a new threshold
signing scheme that will allow us to handle code signing without any
single point of failure. But until then, releases will be signed as
before, just with a new certificate.
As a matter of record, I used the old code-signing key/certificate to
sign a message containing the pubkey that matches the new
key/certificate. It's attached at the end of this message.
The pkcs7 format is rather clunky, but I wanted to include the current
signing certificate to make verification easier. I'll leave it to the
reader to extract the certificate from a previous release in order to
make sure that they match. It was also in the Core git repo until it
was removed recently.
To verify, you can use something like:
openssl smime -verify -in sig.pkcs7 -inform pem -ignore_critical -purpose any
- "ignore_critical" setting tells openssl to ignore the Apple-specific
critical extensions that it doesn't understand.
- "-purpose any" allows the "purpose == smimesign" check to be
skipped. This would otherwise fail because this certificate is only
authorized to sign code, not arbitrary messages.
By now, the signature will probably fail to validate because the
certificate has expired.
The signed message below is timestamped on the Bitcoin blockchain
using OpenTimestamps. See the attached ots file containing the
timestamp proof. If the attachment gets scrubbed and doesn't make it
to the list, don't be afraid to nag Peter Todd about a mail-friendly
format for these proofs :)
Regards,
Cory
expire.txt.sig:
-----BEGIN PKCS7-----
MIILTwYJKoZIhvcNAQcCoIILQDCCCzwCAQExCzAJBgUrDgMCGgUAMIIDNAYJKoZI
hvcNAQcBoIIDJQSCAyFUaGUgY3VycmVudCBCaXRjb2luIENvcmUgbWFjT1MgY29k
ZSBzaWduaW5nIGNlcnRpZmljYXRlIGV4cGlyZXMNCmxhdGVyIHRvZGF5LCBKYW51
YXJ5IDExLCAyMDE4Lg0KDQpJbiB0aGUgZnV0dXJlLCBhIHRocmVzaG9sZCBzaWdu
YXR1cmUgd2lsbCBiZSB1c2VkIHRvIHNpZ24gbWFjT1MNCnJlbGVhc2VzLCBidXQg
c2luY2UgdGhpcyB3YXMgbm90IHJlYWR5IGluIHRpbWUsIGEgdGVtcG9yYXJ5DQpj
ZXJ0aWZpY2F0ZSB3aWxsIGxpa2VseSBiZSB1c2VkIGZvciB0aGUgMC4xNiByZWxl
YXNlLg0KDQpUaGUgcHVibGljIGtleSB0byBiZSB1c2VkIHdpdGggdGhpcyBuZXcg
Y2VydGlmaWNhdGUgaXM6DQoNCi0tLS0tQkVHSU4gUFVCTElDIEtFWS0tLS0tDQpN
SUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXF4aWJE
Z2pBT09WVXBTY3pVMnBqDQp0UEVpQ0lZeXl2V21EN2VidGhQbzI5WG9xMUJqYWJG
NDlCZ3diNkZFaU1haFN5UTY4ZklMSUhDanJ5SUo4RUN1DQpROFJWbVF3cGdhKzV0
OTZiMEM5emN5WTFhcSsrRzIyMVNqNmFpUmVveXZwcHIrZ2poNmNPbktEc1B0Z2pU
cGdiDQovOUhuMmtwYzFmZ000ZkRFMlQ2VXZHVHMwd3d5dWNvL21ya0s1LzEySCtq
ZUE3QXVNcjBLQTBVSktSS1VOenFhDQo4QjlLalFFektaRGVVVHRYak9vSmIyNkRQ
U3hCbXBGd25zWSs2aHBjeFZSSmphNG1FYzRFYnIyb2gxSmVORU5uDQp4WXR3MHRW
VWczTUwvWlI2WU9qQVpMY0V0cW5IR2ZOZXVRazJXVm1pYy9JY3d4VEM0cUk4MnFR
OGgxQnFpY3pRDQo4UUlEQVFBQg0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tDQqg
ggWLMIIFhzCCBG+gAwIBAgIIJ0r1rumyfZAwDQYJKoZIhvcNAQELBQAweTEtMCsG
A1UEAwwkRGV2ZWxvcGVyIElEIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MSYwJAYD
VQQLDB1BcHBsZSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTETMBEGA1UECgwKQXBw
bGUgSW5jLjELMAkGA1UEBhMCVVMwHhcNMTMwMTEwMjIzOTAxWhcNMTgwMTExMjIz
OTAxWjCBqDEaMBgGCgmSJomT8ixkAQEMClBCVjRHTFM5SjQxQDA+BgNVBAMMN0Rl
dmVsb3BlciBJRCBBcHBsaWNhdGlvbjogQklUQ09JTiBGT1VOREFUSU9OLCBJTkMu
LCBUSEUxEzARBgNVBAsMClBCVjRHTFM5SjQxJjAkBgNVBAoMHUJJVENPSU4gRk9V
TkRBVElPTiwgSU5DLiwgVEhFMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBALTd5zURuZVoJviusr119aktXksenb9IN9vq6kBbq38v
xEk79wkKMES2XfBRh0HxcEizGzhMNy5OCXuTLMaNMihYdfwYSoBoR2foEU+6kjPU
nyJ4dQBFLJZJr5/QeQmALmYHEgZ6lwXFD2lU8t92340zeJ4y5LZw5pcEHtH9Iumm
YDutOGCkCGXDcjL+5nHhNScJiXHhswM+62o6XXsQiP6EWbM1CsgrGTNLtaa0U/Uv
VDwE79YKklSC5Bog2LD0jBcTuveI66mFzqu++L9X9u+ZArtebwCl7BPNQ+uboYy5
uV2dzf8lpNNZLfXCFjoLe9bLICKfZ7ub9V5aC8+GhckCAwEAAaOCAeEwggHdMD4G
CCsGAQUFBwEBBDIwMDAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AuYXBwbGUuY29t
L29jc3AtZGV2aWQwMTAdBgNVHQ4EFgQUa5xsqKVzcHDiV6NJ2GL7l8elXV4wDAYD
VR0TAQH/BAIwADAfBgNVHSMEGDAWgBRXF+2iz9x8mKEQ4Py+hy0s8uMXVDCCAQ4G
A1UdIASCAQUwggEBMIH+BgkqhkiG92NkBQEwgfAwKAYIKwYBBQUHAgEWHGh0dHA6
Ly93d3cuYXBwbGUuY29tL2FwcGxlY2EwgcMGCCsGAQUFBwICMIG2DIGzUmVsaWFu
Y2Ugb24gdGhpcyBjZXJ0aWZpY2F0ZSBieSBhbnkgcGFydHkgYXNzdW1lcyBhY2Nl
cHRhbmNlIG9mIHRoZSB0aGVuIGFwcGxpY2FibGUgc3RhbmRhcmQgdGVybXMgYW5k
IGNvbmRpdGlvbnMgb2YgdXNlLCBjZXJ0aWZpY2F0ZSBwb2xpY3kgYW5kIGNlcnRp
ZmljYXRpb24gcHJhY3RpY2Ugc3RhdGVtZW50cy4wDgYDVR0PAQH/BAQDAgeAMBYG
A1UdJQEB/wQMMAoGCCsGAQUFBwMDMBMGCiqGSIb3Y2QGAQ0BAf8EAgUAMA0GCSqG
SIb3DQEBCwUAA4IBAQAfJ0BjID/1dS2aEeVyhAzPzCBjG8vm0gDf+/qfwRn3+yWe
L9vSnMdbilwM48IyQWTagjGGcojbsAd/vE4N7NhQyHInoCllNoeor1I5xx+blTaG
RBK+dDhJbbdlGCjsLnH/BczGZi5fyEJds9lUIrp1hJidRcUKO76qb/9gc6qNZpl1
vH5klDUuJYt7YhAs+L6rTXDyqcK9maeQr0gaOPsRRAQLLwiQCorPeMTUNsbVMdMw
ZYJsR+PxiAnk+nyi7rfiFvPoASAYUuI6OzYL/Fa6QU4/gYyPgic944QYVkaQBnc0
vEP1nXq6LGKwgVGcqJnkr/E2kui5gJoV5C3qll3eMYICYTCCAl0CAQEwgYUweTEt
MCsGA1UEAwwkRGV2ZWxvcGVyIElEIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MSYw
JAYDVQQLDB1BcHBsZSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTETMBEGA1UECgwK
QXBwbGUgSW5jLjELMAkGA1UEBhMCVVMCCCdK9a7psn2QMAkGBSsOAwIaBQCggbEw
GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTgwMTEx
MTgxMDUwWjAjBgkqhkiG9w0BCQQxFgQUvqCmkSFwZTLWSNhIddUfdxBPQSswUgYJ
KoZIhvcNAQkPMUUwQzAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI
hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAE
ggEAQadtQ5qePkjvB3xqLeSvN3e6SpoGQGn6Oo57IiUs/9zP3LAziS2pLbOxSlrS
WWJ5byt7qHdxg9Hi+8IRK5ppps3TxX49ZtN9xHR0BQECspHhbad++JnLuCVjoW88
tgX6NylWb16xekpKA9D1xsLOaVlxFJry4S9k3wz53ajg7J83jlA5K1j9rcS8dVhZ
WjIl12I2AalQ//PXVyu1soF7ieKgyFKeOefGaAOT3ybji1ibYoPfsS/IdnBz7hbn
EmHUHDdl2R+TWDf0ADXMqV3qjMuG5osFRUJbeWm5CUne1/w2BdcIkmkvfmzU+Bmh
jixGT1Xg83O4e3LL4Bww0rRY6w==
-----END PKCS7-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: expire.txt.sig.ots
Type: application/vnd.oasis.opendocument.spreadsheet-template
Size: 1740 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180112/6cf33db9/attachment.ots>

2023-06-07 20:06:14

- reply

📅 Original date posted:2017-09-27
📝 Original message:Hi Mark
Thank you very much for posting the findings. I took a look through our
repository and I think I can provide a bit of context.
I'll go through each one, annotating what I've found.
> Apache License 2.0
This is used by a few java files in the libsecp256k1 project. That library
(which lives here: https://github.com/bitcoin-core/secp256k1) is a
sub-module created and maintained by Bitcoin Core developers. The files in
question are bindings that allow other applications to use libsecp256k1
from Java. Bitcoin Core makes no use of them.
> Boost Software License 1.0
This comes from tinyformat.h. Bitcoin Core indeed uses it.
> BSD 2-clause "Simplified" License
I'm unable to find any 2-clause BSD licensed files.
> BSD 3-clause "New" or "Revised" License
This comes from leveldb, which is database software used by Bitcoin Core.
Because database software version inconsistencies can cause accidental
forks (this actually happened in 2013), we include these files in our
repository and use them rather than linking to arbitrary versions at
runtime.
There are a few non-upstream files we use in our leveldb tree to provide
windows support. Quoting from src/leveldb/util/env_win.cc:
This file contains source that originates from:
http://code.google.com/p/leveldbwin/source/browse/trunk/win32_impl_src/env_win32.h
http://code.google.com/p/leveldbwin/source/browse/trunk/win32_impl_src/port_win32.cc
Those files don't have any explicit license headers but the
project (http://code.google.com/p/leveldbwin/) lists the 'New BSD License'
> Creative Commons Attribution Share Alike 3.0
I didn't manage to find any CC-licensed files. The match probably comes
from our gui svg icons, which contain an xml tag with a link to
creativecommons.org. This seems to be the default behavior of inkscape,
which was used to create those icons. Any icons that we have not created
ourselves are listed in contrib/debian/copyright (they're all expat/public
domain).
> Expat License
See MIT.
> GNU General Public License v2.0 or later
The debian folder, which holds the files used to create debian/ubuntu
packages is licensed gplv2+. These are packaging resources only,
unnecessary for use of the code.
Additionally, some gplv2 m4 macros are used to bootstrap the code that is
used to build the Bitcoin code. These contain the additional exception:
As a special exception, the respective Autoconf Macro's copyright owner
gives unlimited permission to copy, distribute and modify the configure
scripts that are the output of Autoconf when processing the Macro. You
need not follow the terms of the GNU General Public License when using
or distributing such scripts, even though portions of the text of the
Macro appear in them. The GNU General Public License (GPL) does govern
all other use of the material that constitutes the Autoconf Macro.
> GNU General Public License v3.0 or later
The macdeploy script, useful for creating DMG files for macOS is gplv3. It
is not necessary for any other platform, and is only used during the build
process. Additionally, it is not the only way to create DMG files (apple's
native tools can be used as well).
Additionally, config.guess and config.sub are gplv3 scripts used to build
our buildsystem.
> GNU Lesser General Public License v2.1 or later
authproxy.py, A python script used in our test suite is licensed lgpl
v2.1+. It is only necessary for running optional tests during development.
> License for A fast alternative to the modulo reduction
This references a comment cuckoocache.h. No code from the site is used. The
link to the site was added after the code, as the site provides a helpful
explanation for the technique used.
> License for atomic by Timm Kosse
Another m4 file. As explained with the others above, this is a macro which
builds code which builds code. It is used in the build process only.
> MIT License
The primary and default license for all contributions.
> Public Domain
> University of Illinois/NCSA Open Source License
clang-format-diff.py, a python script optionally used by developers to
clean up code changes.
tl;dr: Best I can tell, all source files that comprise Bitcoin Core
binaries are licensed (excluding the public domain ones) as MIT, BSD, or
Boost.
It's also worth repeating Omar's point that many of the files in the
Bitcoin Core repository are used for optional programs/libraries. None of
the artwork, for example, is needed for the primary bitcoin daemon.
Hope that helps!
Regards,
Cory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170927/af791133/attachment.html>

2023-06-07 19:48:20

- reply

📅 Original date posted:2016-02-01
📝 Original message:On Mon, Feb 1, 2016 at 6:08 PM, Luke Dashjr <luke at dashjr.org> wrote:
> On Monday, February 01, 2016 9:43:33 PM Cory Fields wrote:
>> On Mon, Feb 1, 2016 at 2:46 PM, Luke Dashjr <luke at dashjr.org> wrote:
>> > Allowing for simpler cases both encourages the lazy case, and enables
>> > pools to require miners use it. It also complicates the server-side
>> > implementation somewhat, and could in some cases make it more vulnerable
>> > to DoS attacks. Keep in mind that GBT is not merely a bitcoind protocol,
>> > but is used between pool<->miner as well... For now, it makes sense to
>> > leave
>> > "default_witness_commitment" as a bitcoind-specific extension to
>> > encourage adoption, but it seems better to leave it out of the standard
>> > protocol. Let me know if this makes sense or if I'm overlooking
>> > something.
>>
>> I think that's a bit of a loaded answer. What's to keep a pool from
>> building its own commitment and requiring miners to use that? I don't
>> see how providing the known-working commitment for the
>> passed-in-hashes allows the pool/miner to do anything they couldn't
>> already, with the exception of skipping some complexity. Please don't
>> confuse encouraging with enabling.
>
> Making it simpler to do a centralised implementation than a decentralised one,
> is both enabling and encouraging. GBT has always been designed to make it
> difficult to do in a centralised manner.
>
But your suggestion is "use libblkmaker" which will build the trees
for me. By that logic, isn't libblkmaker making a centralized
implementation easier? Shouldn't that usage be discouraged as well?
And along those lines, shouldn't the fact that it's used as a pool <->
miner protocol be discouraged rather than touted as a feature?
I don't wish to sound hostile, I'm just trying follow the logic. I
can't rationalize why GBT shouldn't expose the commitment that it
knows to be correct (when paired with the transactions it provides),
purely to make things difficult.
>> What's the DoS vector here?
>
> It's more work for the pool to provide it, similar to the "midstate" field was
> with getwork. Someone performing a DoS needs to do less work to force the pool
> to do complex calculations (unless the same transaction tree / commitment is
> used for all miners, which would be an unfortunate limitation).
It's being provided to them. And if they're using a modified set of
tx's, they'll need to re-calculate it in order to verify the result
anyway. I suspect I'm not understanding this argument.
>
>> >> The issue in particular here is that a non-trivial burden is thrust
>> >> upon mining software, increasing the odds of bugs in the process.
>> >
>> > It can always use libblkmaker to handle the "heavy lifting"... In any
>> > case, the calculation for the commitment isn't significantly more than
>> > what it must already do for the stripped merkle tree.
>>
>> Agreed. However for the sake of initial adoption, it's much easier to
>> have a known-correct value to use. Even if it's just for the sake of
>> checking against.
>
> Sure, I'm not suggesting we remove this from bitcoind (probably the only place
> that makes initial adoption easier).
>
How about exposing it as a feature/capability, then? That way pools
can expect it from bitcoind, but won't be required to expose it
downstream.
>> >> [4]:
>> >> https://github.com/theuni/ckpool/commit/7d84b1d76b39591cc1c1ef495ebec513
>> >> cb 19a08e
>> >
>> > I'm pretty sure this commit is actually /introducing/ a bug in working
>> > (albeit ugly) code. The height, while always positive, is serialised as
>> > a signed number, so 0x80 needs to be two bytes: 80 00.
>>
>> You're right, thanks. The current code breaks on heights of (for ex)
>> 16513. I'll fix up my changes to take the sign bit into account.
>
> I'm curious what bug it was fixing? Was it overwriting data beyond the number?
Using 16513 as an example:
serialized by bitcoind: 0x028140
serialized by ckpool: 0x03814000
ckpool works because blocks after 32768 end up looking the same, but
it will break again at 2113664.
>
> Luke

2023-06-07 19:48:19

- reply

📅 Original date posted:2016-02-01
📝 Original message:Thanks for getting this started, Luke.
Noticeably absent here is the "default_witness_commitment" key, as
added by the current reference implementation[0].
I assume (please correct me if I'm wrong) that this has been omitted
for the sake of having clients create the commitment themselves as
opposed to having it provided to them.
I don't think that the two approaches (providing the default
commitment for the complete tx set as well as the ability to create it
from chosen transactions) are at odds with each-other, rather it
merely allows for a simpler approach for those who are taking tx's
as-is from bitcoind. It's obviously important for the clients to be
able to chose tx's and create commitments as they desire, but it's
equally important to allow for simpler use-cases.
The issue in particular here is that a non-trivial burden is thrust
upon mining software, increasing the odds of bugs in the process. I'd
like to point out that this is not a theoretical argument. I've
already fixed a handful of bugs relating to serialization or
commitment creation in the mining/pool software that I've worked on
for segwit [1][2][3][4]. Asking them to handle more serialization and
calculation of complex structures needlessly increases the complexity
for zero benefit in the case where the tx's are to be used as-is.
I'll PR this change to the BIP, as I can't really come up with an
argument against. At worst, it can simply be ignored.
[0]: https://github.com/sipa/bitcoin/blob/segwit/src/rpcmining.cpp#L590
[1]: https://github.com/bitcoin/libblkmaker/commit/22f6e42844aa14ed0037ebf12a734f07e63533d7
[2]: https://github.com/bitcoin/libblkmaker/commit/15e2c35bf69c997488e37147cf062dfa925b4912
[3]: https://github.com/bitcoin/libblkmaker/commit/9a5799891e0f3590779b8e5a993a7b306088e2fa
[4]: https://github.com/theuni/ckpool/commit/7d84b1d76b39591cc1c1ef495ebec513cb19a08e
Regards,
Cory
On Sat, Jan 30, 2016 at 1:50 PM, Luke Dashjr via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> I've completed an initial draft of a BIP for updating getblocktemplate for
> segregated witness here:
> https://github.com/luke-jr/bips/blob/segwit_gbt/bip-segwit-gbt.mediawiki
>
> Please review and comment (especially with regard to the changes in the
> sigoplimits handling).
>
> (Note: libblkmaker's reference implementation is at this time incompatible
> with the "last output" rule in this BIP.)
>
> Thanks,
>
> Luke
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 19:48:19

- reply

📅 Original date posted:2016-02-01
📝 Original message:On Mon, Feb 1, 2016 at 2:46 PM, Luke Dashjr <luke at dashjr.org> wrote:
> On Monday, February 01, 2016 6:41:06 PM Cory Fields wrote:
>> Noticeably absent here is the "default_witness_commitment" key, as
>> added by the current reference implementation[0].
>>
>> I assume (please correct me if I'm wrong) that this has been omitted
>> for the sake of having clients create the commitment themselves as
>> opposed to having it provided to them.
>>
>> I don't think that the two approaches (providing the default
>> commitment for the complete tx set as well as the ability to create it
>> from chosen transactions) are at odds with each-other, rather it
>> merely allows for a simpler approach for those who are taking tx's
>> as-is from bitcoind. It's obviously important for the clients to be
>> able to chose tx's and create commitments as they desire, but it's
>> equally important to allow for simpler use-cases.
>
> Allowing for simpler cases both encourages the lazy case, and enables pools to
> require miners use it. It also complicates the server-side implementation
> somewhat, and could in some cases make it more vulnerable to DoS attacks. Keep
> in mind that GBT is not merely a bitcoind protocol, but is used between
> pool<->miner as well... For now, it makes sense to leave
> "default_witness_commitment" as a bitcoind-specific extension to encourage
> adoption, but it seems better to leave it out of the standard protocol. Let me
> know if this makes sense or if I'm overlooking something.
>
I think that's a bit of a loaded answer. What's to keep a pool from
building its own commitment and requiring miners to use that? I don't
see how providing the known-working commitment for the
passed-in-hashes allows the pool/miner to do anything they couldn't
already, with the exception of skipping some complexity. Please don't
confuse encouraging with enabling.
What's the DoS vector here?
>> The issue in particular here is that a non-trivial burden is thrust
>> upon mining software, increasing the odds of bugs in the process.
>
> It can always use libblkmaker to handle the "heavy lifting"... In any case,
> the calculation for the commitment isn't significantly more than what it must
> already do for the stripped merkle tree.
Agreed. However for the sake of initial adoption, it's much easier to
have a known-correct value to use. Even if it's just for the sake of
checking against.
>
>> I'd like to point out that this is not a theoretical argument. I've
>> already fixed a handful of bugs relating to serialization or
>> commitment creation in the mining/pool software that I've worked on
>> for segwit [1][2][3][4].
>
> That's not really fair IMO. I wrote the libblkmaker branch prior to even
> reading the SegWit BIPs or code, and without a way to test it. It's only to be
> expected there are bugs that get fixed in first-try testing.
I didn't mean this as an insult/attack, quite the opposite actually.
Thanks for doing the integration :)
I was merely pointing out how easy it is to introduce subtle bugs here.
>
>> [4]:
>> https://github.com/theuni/ckpool/commit/7d84b1d76b39591cc1c1ef495ebec513cb
>> 19a08e
>
> I'm pretty sure this commit is actually /introducing/ a bug in working (albeit
> ugly) code. The height, while always positive, is serialised as a signed
> number, so 0x80 needs to be two bytes: 80 00.
You're right, thanks. The current code breaks on heights of (for ex)
16513. I'll fix up my changes to take the sign bit into account.
Heh, that only reinforces my point above about introducing bugs :p
>
> Luke

2023-06-07 19:44:31

- reply

📅 Original date posted:2015-11-12
📝 Original message:Hi all
As some of you may recall, I tried to throw together a small
code-sprint after the Scaling Bitcoin event in Montreal. There were
several problems stemming from the last-minute-ness and some confusion
about the scope and goals, but I think it was a good thing overall.
Most importantly, it pointed out that many devs are interested in
face-time for technical/code discussions.
For those of you attending the Hong Kong conference next month, I'd
like to invite you to stay an extra day or two and join in on some
in-person hacking, code-review, and technical discussion.
If there was one take-away from the Montreal sprint, it was this:
trying to stick to a pre-defined set of goals with so many people is
counter-productive. The plan was to review a few long-standing Bitcoin
Core pull-requests in-person in order to knock them out quickly, but I
think it was the organic tangents and ad-hoc discussions that proved
to be more interesting. So let's encourage that!
The plan:
Thanks to Pindar and Cyberport, we have two rooms available the two
days after the conference. These will be treated as general meeting
rooms for technical discussion; anything goes as long as it's
technical and Bitcoin-related. Personally, I'll be bringing my laptop
and demoing some recent code to others who might be interested (or
anyone who will listen!). It's also a great opportunity for nascent
devs to ask veterans questions questions about development processes,
hard-to-understand code, etc. Miners are encouraged to come as well,
for discussing any technical hurdles or questions that may benefit
from some real-time technical discussion or debugging.
Attendees are encouraged to self-organize and huddle up as necessary.
Topics are by no means limited to Bitcoin Core, so feel free to
discuss/learn about projects outside of your usual bubble. If you find
yourself saying "I'd like to look at that code with you later" at the
conference, plan a time to meet and do it! While this isn't associated
with Scaling Bitcoin or its organizers, it's obviously meant to
piggy-back off of the event. If it becomes too chaotic, we may throw
together a sign-up sheet, but the intent is to let things happen
organically.
What it's not:
This is a venue for technical discussion. It should not be treated as
a place for discussing politics, agendas, plans for world domination,
etc. Let your code speak for you!
The location:
Video Conferencing Rooms 2 and 3, Level 3, Core C
Cyberport 3
100 Cyberport Rd,
Telegraph Bay,
Hong Kong
Room 2 seats 20 people around a conference table, room 3 seats 12.
The time:
Tuesday December 8 at 8:00am - Wednesday December 9 at midnight.
Extras:
Coffee/tea/water will be provided, but food is not arranged. Likely
some herds will form and venture out for food, but we can also order
in. Wifi/whiteboards provided as well.
See you all in Hong Kong!
Cory

2023-06-07 19:35:09

- reply

📅 Original date posted:2015-08-18
📝 Original message:Pull request submitted: https://github.com/bitcoin/bitcoin/pull/6571
Regards,
Cory
On Tue, Aug 18, 2015 at 1:25 PM, Cory Fields <lists at coryfields.com> wrote:
> See responses inline.
>
> On Tue, Aug 18, 2015 at 6:31 AM, Tamas Blummer <tamas at bitsofproof.com> wrote:
>> Thanks a lot Cory for following through the test case and producing a patch.
>>
>> I confirm that libconsensus is now running stable within the Bits of Proof
>> stack,
>> in-line with test cases we use to verify the java implementation of the
>> script engine,
>> that are BTW borrowed from Bitcoin Core.
>>
>> The performance of libconsensus is surprisingly close to the java one.
>> Validating a 2-of-2 a multi-sig transaction runs at 1021 ops/sec with java
>> and 1135 ops/sec
>> in libconsensus. This is on a 2.2GH i7 laptop (4 hyper threading cores used
>> by 8 threads).
>> Another nice demonstration why one should not trade in advances
>> of languages for the last decades for a marginal gain of performance with
>> C/C++,
>> I assume thereby that Bouncy Castle’ EC lib s not superior to OpenSSL's.
>
> A few points there. First, Core is switching to libsecp256k1 for
> several reasons, and one of them is speed. I seem to recall it being
> up to 8x faster than OpenSSL.
>
> Also, it can depend heavily on compiler switches and optimization
> levels. For example, In playing with my test-case for hitting the
> OpenSSL race issue, I managed to get a ~100% speedup by simply using
> -O3 and lto.
>
>>
>> I disagree that the problem was rare in the real-world, it should affect any
>> modern
>> implementation that validates transactions parallel in multiple threads.
>>
>
> Well I'd say you're a bit biased in this case ;)
>
> It's only those using ancient (0.98 or 1.00) versions of OpenSSL who
> are affected, or those with OPENSSL_BN_ASM_MONT support disabled or
> missing. Note that official releases of libbitcoinconsensus are
> compiled against a much newer version and shouldn't have any issues.
>
> The earlier patches for locking callbacks should be unnecessary.
>
>> Aborting also does not make the problem less severe in my opinion.
>
> Well it's not a good thing by any means, but it's certainly better
> than incorrect results! In any undefined/error condition for the
> consensus library, aborting is the right thing to do. If we can't
> explain how we've reached a certain "unreachable" condition as is the
> case here, the only reasonable recourse is to shut down. Otherwise we
> risk network forks, DOS, etc.
>
>> Therefore hope the pull will be included into Core with next release.
>>
>
> It will likely be unnecessary for the next release, but I do think
> it's worth backporting to the 0.10 and 0.9 series.
>
>> I can’t assign a timeline to “near future" secp256k1 integration. Can you?
>
> I believe the libsecp256k1 guys are generally happy with the lib these
> days, but I'll avoid guessing at a timeline. We can discuss that on
> the PR for this fix, which I'll do today.
>
> Regards,
> Cory

2023-06-07 19:35:08

- reply

📅 Original date posted:2015-08-18
📝 Original message:See responses inline.
On Tue, Aug 18, 2015 at 6:31 AM, Tamas Blummer <tamas at bitsofproof.com> wrote:
> Thanks a lot Cory for following through the test case and producing a patch.
>
> I confirm that libconsensus is now running stable within the Bits of Proof
> stack,
> in-line with test cases we use to verify the java implementation of the
> script engine,
> that are BTW borrowed from Bitcoin Core.
>
> The performance of libconsensus is surprisingly close to the java one.
> Validating a 2-of-2 a multi-sig transaction runs at 1021 ops/sec with java
> and 1135 ops/sec
> in libconsensus. This is on a 2.2GH i7 laptop (4 hyper threading cores used
> by 8 threads).
> Another nice demonstration why one should not trade in advances
> of languages for the last decades for a marginal gain of performance with
> C/C++,
> I assume thereby that Bouncy Castle’ EC lib s not superior to OpenSSL's.
A few points there. First, Core is switching to libsecp256k1 for
several reasons, and one of them is speed. I seem to recall it being
up to 8x faster than OpenSSL.
Also, it can depend heavily on compiler switches and optimization
levels. For example, In playing with my test-case for hitting the
OpenSSL race issue, I managed to get a ~100% speedup by simply using
-O3 and lto.
>
> I disagree that the problem was rare in the real-world, it should affect any
> modern
> implementation that validates transactions parallel in multiple threads.
>
Well I'd say you're a bit biased in this case ;)
It's only those using ancient (0.98 or 1.00) versions of OpenSSL who
are affected, or those with OPENSSL_BN_ASM_MONT support disabled or
missing. Note that official releases of libbitcoinconsensus are
compiled against a much newer version and shouldn't have any issues.
The earlier patches for locking callbacks should be unnecessary.
> Aborting also does not make the problem less severe in my opinion.
Well it's not a good thing by any means, but it's certainly better
than incorrect results! In any undefined/error condition for the
consensus library, aborting is the right thing to do. If we can't
explain how we've reached a certain "unreachable" condition as is the
case here, the only reasonable recourse is to shut down. Otherwise we
risk network forks, DOS, etc.
> Therefore hope the pull will be included into Core with next release.
>
It will likely be unnecessary for the next release, but I do think
it's worth backporting to the 0.10 and 0.9 series.
> I can’t assign a timeline to “near future" secp256k1 integration. Can you?
I believe the libsecp256k1 guys are generally happy with the lib these
days, but I'll avoid guessing at a timeline. We can discuss that on
the PR for this fix, which I'll do today.
Regards,
Cory

2023-06-07 19:35:07

- reply

📅 Original date posted:2015-08-18
📝 Original message:Back to the list (from github) in case anyone finds this via Google.
The patch that I posted here a few days ago did not fix the issue for Tamas.
I spent some time tracking down this edge-case because
libbitcoinconsensus needs to be as bullet-proof as possible. Thanks to
Tamas for creating a bare-bones test case after some discussion.
I finally managed to reproduce the issue on OSX. It's subtle and
likely rare in the real-world, though obviously not impossible given
the report here. For posterity, here's a rundown (braindump) of the
issue.
When calling EC_KEY_new_by_curve_name(), openssl internally checks to
see how to setup the curve's EC_METHOD (simple, montgomery, or nist).
Unfortunately, in all released OpenSSL versions (as far as I can tell
master is the only branch that has fixed this issue), it's tested like
so:
- Try a method. If it fails, set a global error and return.
- If the global error is set, try a different method.
Prior to OpenSSL 1.0.0, these were tested in the order:
EC_GFp_nist_method -> EC_GFp_mont_method. The secp256k1 curve fails
the ec_GFp_nist_group_set_curve test and sets the global error. That
error is then checked for failure, and EC_GFp_mont_method is tried
(and succeeds).
Obviously that global error usage is dangerous, especially since it
happens for _each_ transaction verification in libbitcoinconsensus. In
a multi-threaded environment, a crash is guaranteed within a few
seconds.
However, OpenSSL 1.0.1 reversed the order, trying EC_GFp_mont_method
first, so that the global error doesn't end up being used:
https://github.com/openssl/openssl/commit/17674bfdf75bffa4e225f8328b9d42cb74504005
This was backported from master back to 1.0.1, but not to 1.0.0 or 0.9.8.
So that change (accidentally) "solved" the problem. As you can see,
it's still possible to hit the reversed order in the
!defined(OPENSSL_BN_ASM_MONT) case. That's easily tested by building
OpenSSL with the -no-asm config option. It's probably also the case
for obscure architectures and OSs, but I haven't looked deeply into
that. In that case, it's reasonable to assume that this crash would
likely occur on such platforms.
Also, OSX, even the latest version (10.10 as of now), still ships with
OpenSSL 0.9.8. Which is how Tamas ran into it.
Since Bitcoin Core and libbitcoinconsensus are switching away from
OpenSSL for verification in the near future, I don't think this is
much of an issue. Especially since the problem manifests as a
controlled assertion failure/abort. However, I've prepared a patch for
anyone who may run into the issue in the short-term:
https://github.com/theuni/bitcoin/commit/adf0a691ee1c2f02e26828f976cfe5b78896b507
I'll open a pull-request for Bitcoin Core to discuss whether it's
worth merging or not.
Regards,
Cory
On Fri, Aug 14, 2015 at 5:10 PM, Cory Fields <lists at coryfields.com> wrote:
> Ugh, what an unfortunate oversight!
>
> The good news is that this issue should be solved in future versions
> when we switch to the new libsecp256k1 lib for validation.
>
> For now, I've thrown together a quick hack to allow a user-specifiable
> callback for libbitcoinconsensus. I think it's not worth messing with
> the official API since it will be fixed soon, but rather hacked in as
> a temporary work-around as needed. It _should_ be documented as an
> issue with the current version, though.
>
> Please see here for a work-around to try:
> https://github.com/theuni/bitcoin/commits/openssl-consensus-threads
> Unfortunately it's not pretty, but it works fine here. Note that you
> should give this some _serious_ testing before deploying in any real
> way. It should mimic the way we do it in Core, though.
>
> That's on top of current master, but it should be trivial to apply to
> release tags.
>
> Please let me know how it works out.
>
> Regards,
> Cory
>
> On Fri, Aug 14, 2015 at 12:37 PM, Tamas Blummer via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> We integrated libconsensus into bits of proof. It works well, in-line for all test cases with our Java engine and is about 50% faster on a single thread.
>>
>> The performance advantage unfortunatelly reverses if libconsensus is executed on several threads simultaneously as we do with the Java engine, since an error:
>>
>> Assertion failed: (pkey != NULL), function CECKey, file ecwrapper.cpp, line 96.
>>
>> arises under that stress.
>>
>> I guess that the cause is that thread callbacks as advised for OpenSSL on https://www.openssl.org/docs/crypto/threads.html are not registered.
>> Registering those however would require access to OpenSSL functions, not exported from the lib.
>>
>> I’d be thankful for a pointer to a workaround.
>>
>> Tamas Blummer
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>

2023-06-07 19:35:07

- reply

📅 Original date posted:2015-08-14
📝 Original message:Ugh, what an unfortunate oversight!
The good news is that this issue should be solved in future versions
when we switch to the new libsecp256k1 lib for validation.
For now, I've thrown together a quick hack to allow a user-specifiable
callback for libbitcoinconsensus. I think it's not worth messing with
the official API since it will be fixed soon, but rather hacked in as
a temporary work-around as needed. It _should_ be documented as an
issue with the current version, though.
Please see here for a work-around to try:
https://github.com/theuni/bitcoin/commits/openssl-consensus-threads
Unfortunately it's not pretty, but it works fine here. Note that you
should give this some _serious_ testing before deploying in any real
way. It should mimic the way we do it in Core, though.
That's on top of current master, but it should be trivial to apply to
release tags.
Please let me know how it works out.
Regards,
Cory
On Fri, Aug 14, 2015 at 12:37 PM, Tamas Blummer via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
> We integrated libconsensus into bits of proof. It works well, in-line for all test cases with our Java engine and is about 50% faster on a single thread.
>
> The performance advantage unfortunatelly reverses if libconsensus is executed on several threads simultaneously as we do with the Java engine, since an error:
>
> Assertion failed: (pkey != NULL), function CECKey, file ecwrapper.cpp, line 96.
>
> arises under that stress.
>
> I guess that the cause is that thread callbacks as advised for OpenSSL on https://www.openssl.org/docs/crypto/threads.html are not registered.
> Registering those however would require access to OpenSSL functions, not exported from the lib.
>
> I’d be thankful for a pointer to a workaround.
>
> Tamas Blummer
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

- reply

- reply

- reply

- reply

- reply

📅 Original date posted:2015-07-22
📝 Original message:On Wed, Jul 22, 2015 at 8:13 PM, Eric Lombrozo <elombrozo at gmail.com> wrote:
>
>> On Jul 22, 2015, at 5:05 PM, Cory Fields <lists at coryfields.com> wrote:
>>
>> On Wed, Jul 22, 2015 at 7:53 PM, Eric Lombrozo <elombrozo at gmail.com> wrote:
>>> FWIW, I had worked on something similar a while back:
>>> https://github.com/CodeShark/bitcoin/tree/coinparams_new/altconf
>>>
>>> I like the idea in principle…but we should require a new genesis block,
>>> different magic bytes, and a different network port at the very least. :)
>>>
>>
>> Not sure if serious, so I'll assume you are :)
>
> Only being partly serious - I strongly am in favor of a sufficiently modularized codebase that swapping out consensus rules is fairly straightforward and easy to test. I’m not in favor of encouraging forking an existing blockchain without having mechanisms in place to gracefully merge back without significant network disruptions. We do not have this yet.
>
Again, why? If someone wants to create a scamcoin, they can. If
someone wants to burn money on a scamcoin, equally, they can. I'm not
sure how this is any different. If someone manages to garner realistic
support for a hard-fork, I don't see the benefit in forcing them to
use forked software.. that only leaves Core in the middle because it's
forced to choose a side (not choosing is unfortunately a side as
well). It doesn't remove the reality of the split.
>> Why? The idea in this case would be to allow the user to decide
>> between (say) "./bitcoind -1mbchain" and "./bitcoind -2mbchain" at
>> runtime rather than the likely alternative of "./bitcoind" vs
>> "./bitcoin-fork”.
>
> That’s exactly what my coinparams_new branch does. Adding a parameter for maximum block size would be straightforward.
>
>> Chain params may be identical other than the value of some future
>> event (miner vote for example), in which case the configs would run
>> identically until that point.
>
> Yes, indeed - this would be a special case.
>
>> If your concern is about nodes with different configs communicating
>> with eachother, I'd like to reiterate: the idea really is no different
>> than suggesting that someone fork the codebase and implement their own
>> changes, it just cuts out most of the work required.
>
> I do not encourage anyone to try to fork an existing blockchain without first securing overwhelming (near unanimous) consensus…or without having yet built a mechanism that can merge divergent chains gracefully.
Well of course. It would be a terrible idea. People would try it and
fail, and lose money. But for those crying foul at Core for being the
consensus/policy gatekeeper, it seems to me that user-selectable
params is the only logical solution.

2023-06-07 17:43:07

- reply

📅 Original date posted:2015-07-22
📝 Original message:I'm not sure why Bitcoin Core and the rules and policies that it
enforces are being conflated in this thread. There's nothing stopping
us from adding the ability for the user to decide what their consensus
parameters should be at runtime. In fact, that's already in use:
./bitcoind -testnet. As mentioned in another thread, the chain params
could even come from a config file that the user could edit without
touching the code.
I realize that it'd be opening Pandora's Box, and likely met with very
loud and reasonable arguments about the obvious terrible implications,
but it's at least an alternative to the current status quo of Core's
conflation with the consensus rules. The idea really is no different
than suggesting that someone fork the codebase and implement their own
changes, it just cuts out most of the work required.
With that in place, consensus changes would be more about lobbying and
coalitions, and less about pull requests.
Cory
On Wed, Jul 22, 2015 at 6:40 PM, Raystonn via bitcoin-dev
<bitcoin-dev at lists.linuxfoundation.org> wrote:
>> If the developers fail to reflect user consensus, the network will let us
>> know.
>
> This is true with the caveat that there must be more than one option present
> for the network to show it's preference. If developers discourage anything
> that forks from the rules enforced by Bitcoin Core, they harm the network's
> ability to inform us of a failure to reflect user consensus.
>
> On 22 Jul 2015 3:31 pm, Jeff Garzik via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> I wouldn't go quite that far. The reality is somewhere in the middle, as
> Bryan Cheng noted in this thread:
>
> Quoting BC,
>> Upgrading to a version of Bitcoin Core that is incompatible with your
>> ideals is in no way a forced choice, as you have stated in your email;
>> forks, alternative clients, or staying on an older version are all valid
>> choices. If the majority of the network chooses not to endorse a specific
>> change, then the majority of the network will continue to operate just fine
>> without it, and properly structured consensus rules will pull the minority
>> along as well.
>
> The developers propose a new version, by publishing a new release. The
> individual network nodes choose to accept or reject that.
>
> So I respectfully disagree with "core devs don't control the network" and
> "core devs control the network" both.
>
> There are checks-and-balances that make the system work. Consensus is most
> strongly measured by user actions after software release. If the developers
> fail to reflect user consensus, the network will let us know.
>
>
>
>
>
>
>
>
>
>
>
> On Wed, Jul 22, 2015 at 2:43 PM, Mike Hearn via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Hi Pieter,
>
> I think a core area of disagreement is this:
>
> Bitcoin Core is not running the Bitcoin economy, and its developers have no
> authority to set its rules.
>
> In fact Bitcoin Core is running the Bitcoin economy, and its developers do
> have the authority to set its rules. This is enforced by the reality of
> ~100% market share and limited github commit access.
>
> You may not like this situation, but it is what it is. By refusing to make a
> release with different rules, people who disagree are faced with only two
> options:
>
> 1. Swallow it even if they hate it
> 2. Fork the project and fork the block chain with it (XT)
>
> There are no alternatives. People who object to (2) are inherently
> suggesting (1) is the only acceptable path, which not surprisingly, makes a
> lot of people very angry.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

2023-06-07 17:43:07

- reply

📅 Original date posted:2015-07-22
📝 Original message:On Wed, Jul 22, 2015 at 7:53 PM, Eric Lombrozo <elombrozo at gmail.com> wrote:
> FWIW, I had worked on something similar a while back:
> https://github.com/CodeShark/bitcoin/tree/coinparams_new/altconf
>
> I like the idea in principle…but we should require a new genesis block,
> different magic bytes, and a different network port at the very least. :)
>
Not sure if serious, so I'll assume you are :)
Why? The idea in this case would be to allow the user to decide
between (say) "./bitcoind -1mbchain" and "./bitcoind -2mbchain" at
runtime rather than the likely alternative of "./bitcoind" vs
"./bitcoin-fork".
Chain params may be identical other than the value of some future
event (miner vote for example), in which case the configs would run
identically until that point.
If your concern is about nodes with different configs communicating
with eachother, I'd like to reiterate: the idea really is no different
than suggesting that someone fork the codebase and implement their own
changes, it just cuts out most of the work required.
Cory
> On Jul 22, 2015, at 4:42 PM, Cory Fields via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> I'm not sure why Bitcoin Core and the rules and policies that it
> enforces are being conflated in this thread. There's nothing stopping
> us from adding the ability for the user to decide what their consensus
> parameters should be at runtime. In fact, that's already in use:
> ./bitcoind -testnet. As mentioned in another thread, the chain params
> could even come from a config file that the user could edit without
> touching the code.
>
> I realize that it'd be opening Pandora's Box, and likely met with very
> loud and reasonable arguments about the obvious terrible implications,
> but it's at least an alternative to the current status quo of Core's
> conflation with the consensus rules. The idea really is no different
> than suggesting that someone fork the codebase and implement their own
> changes, it just cuts out most of the work required.
>
> With that in place, consensus changes would be more about lobbying and
> coalitions, and less about pull requests.
>
> Cory
>
> On Wed, Jul 22, 2015 at 6:40 PM, Raystonn via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> If the developers fail to reflect user consensus, the network will let us
> know.
>
>
> This is true with the caveat that there must be more than one option present
> for the network to show it's preference. If developers discourage anything
> that forks from the rules enforced by Bitcoin Core, they harm the network's
> ability to inform us of a failure to reflect user consensus.
>
> On 22 Jul 2015 3:31 pm, Jeff Garzik via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> I wouldn't go quite that far. The reality is somewhere in the middle, as
> Bryan Cheng noted in this thread:
>
> Quoting BC,
>
> Upgrading to a version of Bitcoin Core that is incompatible with your
> ideals is in no way a forced choice, as you have stated in your email;
> forks, alternative clients, or staying on an older version are all valid
> choices. If the majority of the network chooses not to endorse a specific
> change, then the majority of the network will continue to operate just fine
> without it, and properly structured consensus rules will pull the minority
> along as well.
>
>
> The developers propose a new version, by publishing a new release. The
> individual network nodes choose to accept or reject that.
>
> So I respectfully disagree with "core devs don't control the network" and
> "core devs control the network" both.
>
> There are checks-and-balances that make the system work. Consensus is most
> strongly measured by user actions after software release. If the developers
> fail to reflect user consensus, the network will let us know.
>
>
>
>
>
>
>
>
>
>
>
> On Wed, Jul 22, 2015 at 2:43 PM, Mike Hearn via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Hi Pieter,
>
> I think a core area of disagreement is this:
>
> Bitcoin Core is not running the Bitcoin economy, and its developers have no
> authority to set its rules.
>
> In fact Bitcoin Core is running the Bitcoin economy, and its developers do
> have the authority to set its rules. This is enforced by the reality of
> ~100% market share and limited github commit access.
>
> You may not like this situation, but it is what it is. By refusing to make a
> release with different rules, people who disagree are faced with only two
> options:
>
> 1. Swallow it even if they hate it
> 2. Fork the project and fork the block chain with it (XT)
>
> There are no alternatives. People who object to (2) are inherently
> suggesting (1) is the only acceptable path, which not surprisingly, makes a
> lot of people very angry.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

2023-06-07 17:28:06

- reply

📅 Original date posted:2014-12-15
📝 Original message:On Mon, Dec 15, 2014 at 7:47 AM, Peter Todd <pete at petertodd.org> wrote:
> BtcDrak was working on rebasing my CHECKLOCKTIMEVERIFY¹ patch to master a few
> days ago and found a fairly large design change that makes merging it currently
> impossible. Pull-req #4890², specifically commit c7829ea7, changed the
> EvalScript() function to take an abstract SignatureChecker object, removing the
> txTo and nIn arguments that used to contain the transaction the script was in
> and the txin # respectively. CHECKLOCKTIMEVERIFY needs txTo to obtain the
> nLockTime field of the transaction, and it needs nIn to obtain the nSequence of
> the txin.
>
> We need to fix this if CHECKLOCKTIMEVERIFY is to be merged.
>
> Secondly, that this change was made, and the manner in which is was made, is I
> think indicative of a development process that has been taking significant
> risks with regard to refactoring the consensus critical codebase. I know I
> personally have had a hard time keeping up with the very large volume of code
> being moved and changed for the v0.10 release, and I know BtcDrak - who is
> keeping Viacoin up to date with v0.10 - has also had a hard time giving the
> changes reasonable review. The #4890 pull-req in question had no ACKs at all,
> and only two untested utACKS, which I find worrying for something that made
> significant consensus critical code changes.
>
> While it would be nice to have a library encapsulating the consensus code, this
> shouldn't come at the cost of safety, especially when the actual users of that
> library or their needs is still uncertain. This is after all a multi-billion
> project where a simple fork will cost miners alone tens of thousands of dollars
> an hour; easily much more if it results in users being defrauded. That's also
> not taking into account the significant negative PR impact and loss of trust. I
> personally would recommend *not* upgrading to v0.10 due to these issues.
>
> A much safer approach would be to keep the code changes required for a
> consensus library to only simple movements of code for this release, accept
> that the interface to that library won't be ideal, and wait until we have
> feedback from multiple opensource projects with publicly evaluatable code on
> where to go next with the API.
>
> 1) https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki
> 2) https://github.com/bitcoin/bitcoin/pull/4890
>
> --
> 'peter'[:-1]@petertodd.org
> 00000000000000001b18a596ecadd07c0e49620fb71b16f9e41131df9fc52fa6
It would appear as though you're trying to drum up controversy here,
but the argument is quite a stretch, and contrary to some other
arguments you're making in parallel. There seem to be three themes in
your above complaint, so I'd like to address them individually.
1. Pr #4890 specifically. The argument seems to be that this was not
properly reviewed/tested, and that it is an unnecessary risk to the
consensus codebase.
Looking at the PR at github, while I certainly don't agree with those
conclusions, I suppose I can understand where they're coming from.
There's plenty of context missing, as well as sidebar discussions on
IRC and other PRs. To an outside observer, these changes may look
under-tested and unnecessary.
The context that's missing is the flurry of work that was going on in
parallel to modularize this (and surrounding code). #4890 was one of
the first pieces necessary for that, so some of the discussion about
it was happening in dependent pull requests.
You can point to a lack ACKs in one place for that PR, but that
doesn't mean that the changes weren't tested/reviewed/necessary. You
could also argue that ACKs should've been mirrored on the PR in
question for posterity, which would be a perfectly reasonable argument
that I would agree with.
2. These changes conflict with a rebased version of your
CHECKLOCKTIMEVERIFY changes. OK? You have a tree that's a few months
old, and you find that you have conflicts when rebasing to master. It
happens to all of us. Do as the rest of us do and update your changes
to fit. If you missed the review of #4890 and think it should be
reverted, then call for a revert. But please give a concrete reason
other than "I've picked this commit series for a crusade because it
gave me merge conflicts".
What is the conspiracy here? There's a signature cache that is
implementation-specific, and in a parallel universe, you might be
arguing that we should rip it out because it adds unnecessary
complexity to the consensus code. The PR provides a path around that
complexity. For some reason, your reaction is to cry foul months later
because you missed reviewing it at the time, rather than cheering for
the reduced complexity.
3. You seem to think that 1. and 2. seem to point to a systemic
failure of the review process because modularization "shouldn't come
at the cost of safety". I agree that it shouldn't come at the cost of
safety, but I see no failure here. There has been a HUGE effort to
modularize the code with a combination of pure-code-movement and small
interface reworks. Please take a moment to grep the git logs for
"MOVEONLY" in the 0.10 branch.
You'll notice that script verification is now 100% free of bitcoind
state, threading, and third-party libraries (other than openssl for
now). That constitutes a massive reduction in code complexity, future
review overhead, etc. I'll point out here that those were my reasons
for my contributions to the libbitcoinconsensus effort. I have no
interest in altcoins or sidechains.
Those milestones were thanks to an effort which included #4890. If you
have issues with these changes and/or how they were made, please call
out individual failures and proposed solutions _in context_. That they
conflict with CHECKLOCKTIMEVERIFY may be a valid concern, and it may
be worth evaluating how separate coding efforts may more effectively
parallelized.
Without pointers to specific failures or solutions, I'm not sure what
you were trying to communicate here, other than maybe stirring the
social networks with: "I
personally would recommend *not* upgrading to v0.10 due to these
issues." That's fine I suppose, but it does nothing to solve whatever
issue you're trying to call out here.
Cory

2023-06-07 17:28:04

- reply

📅 Original date posted:2014-12-15
📝 Original message:On Mon, Dec 15, 2014 at 10:20 AM, Jeff Garzik <jgarzik at bitpay.com> wrote:
> On Mon, Dec 15, 2014 at 9:57 AM, Btc Drak <btcdrak at gmail.com> wrote:
>>
>> We all want to see more modular code, but the first steps should just be
>> to relocate blocks of code so everything is more logically organised in
>> smaller files (especially for consensus critical code). Refactoring should
>> come in a second wave preferably after a stable release.
>
>
> This is my opinion as well. In the Linux kernel, we often were faced with a
> situation where you have a One Big File driver with > 1MB of source code.
> The first step was -always- raw code movement, a brain-dead breaking up of
> code into logical source code files.
>
> Refactoring of data structures comes after that.
>
> While not always money-critical, these drivers Had To Keep Working. We had
> several situations where we had active users, but zero hardware access for
> debugging, and zero access to the vendor knowledge (hardware documentation,
> engineers). Failure was not an option. ;p
>
> Performing the dumb Break Up Files step first means that future, more
> invasive data structures are easier to review, logically segregated, and not
> obscured by further code movement changes down the line. In code such as
> Bitcoin Core, it is important to think about the _patch stream_ and how to
> optimize for reviewer bandwidth.
>
> The current stream of refactoring is really a turn-off in terms of
> reviewing, sapping reviewer bandwidth by IMO being reviewer-unfriendly. It
> is a seemingly never-ending series of tiny
> refactor-and-then-stuff-in-a-class-and-make-it-pretty-and-do-all-the-work.
> Some change is in order, gentlemen.
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc. https://bitpay.com/
That's exactly what happened during the modularization process, with
the exception that the code movement and refactors happened in
parallel rather than in series. But they _were_ done in separate
logical chunks for the sake of easier review. The commit tag
"MOVEONLY" developed organically out of this process, and a grep of
the 0.10 branch for "MOVEONLY" is a testament to exactly how much code
moved 1:1 out of huge files and into logically separated places and/or
new files.
Perhaps it's worth making "MOVEONLY" (which as the name implies, means
that code has been copied 1:1 to a new location) use an official dev
guideline for use in future refactors.
Cory

2023-06-07 17:28:04

- reply

📅 Original date posted:2014-12-15
📝 Original message:On Mon, Dec 15, 2014 at 2:35 PM, Jeff Garzik <jgarzik at bitpay.com> wrote:
> On Mon, Dec 15, 2014 at 1:42 PM, Cory Fields <lists at coryfields.com> wrote:
>>
>> That's exactly what happened during the modularization process, with
>> the exception that the code movement and refactors happened in
>> parallel rather than in series. But they _were_ done in separate
>> logical chunks for the sake of easier review.
>
>
> "That's exactly what was done except it wasn't"
>
> Yes, in micro, at the pull request level, this happened
> * Code movement
> * Refactor
>
> At a macro level, that cycle was repeated many times, leading to the
> opposite end result: a lot of tiny movement/refactor/movement/refactor
> producing the review and patch annoyances described.
>
> It produces a blizzard of new files and new data structures, breaking a
> bunch of out-of-tree patches, complicating review quite a bit. If the vast
> majority of code movement is up front, followed by algebraic
> simplifications, followed by data structure work, further patches are easy
> to review/apply with less impact on unrelated code.
>
I won't argue that at all because it's perfectly logical, but in
practice that doesn't translate from the macro level to the micro
level very well. At the micro level, minor code changes are almost
always needed to accommodate useful code movement. Even if they're not
required, it's often hard to justify code movement for the sake of
code movement with the promise that it will be useful later.
Rather than arguing hypotheticals, let's use a real example:
https://github.com/bitcoin/bitcoin/pull/5118 . That one's pretty
simple. The point of the PR was to unchain our openssl wrapper so that
key operations could be performed by the consensus lib without
dragging in bitcoind's structures. The first commit severs the
dependencies. The second commit does the code movement from the
now-freed wrapper.
I'm having a hard time coming up with a workflow that would handle
these two changes as _separate_ events, while making review easier.
Note that I'm not attempting to argue with you here, rather I'm
genuinely curious as to how you'd rather see this specific example
(which is representative of most of my other code movement for the
libbitcoinconsensus work, i believe) handled.
Using your model above, I suppose we'd do the code movement first with
the dependencies still intact as a pull request. At some later date,
we'd sever the dependencies in the new files. I suppose you'd also
prefer that I group a bunch of code-movement changes together into a
single PR which needs little scrutiny, only verification that it's
move-only. Once the code-movement PRs are merged, I can begin the
cleanups which actually fix something.
In practice, though, that'd be a massive headache for different
reasons. Lots in flux with seemingly no benefits until some later
date. My PR's can't depend on eachother because they don't actually
fix issues in a linear fashion. That means that other devs can't
depend on my PRs either for the same reason. And what have we gained?
Do you find that assessment unreasonable?
Cory

2023-06-07 17:26:29

- reply

📅 Original date posted:2014-10-15
📝 Original message:Sounds like this is what you're after, it's a fairly new feature:
https://github.com/blog/1375%0A-task-lists-in-gfm-issues-pulls-comments
I've been meaning to use it in a PR to try it out.
Cory
On Wed, Oct 15, 2014 at 5:36 AM, Wladimir <laanwj at gmail.com> wrote:
>> This all makes a lot of sense to me, and would help a lot with the
>> workflow. Unfortunately github pulls and issues really have nothing
>> to faciltate a multistage workflow... e.g. where something can go
>> through several steps.
>
> Indeed, pull requests don't have a "status".
> It would be possible to (ab)use labels for this.
>
> The drawback of labels is that only the repository team can set these,
> there is no way to delegate. But I suppose it'd be possible to build
> something on top of the github API that handles this.
>
>> We're also having problems with people failing to comment on things,
>> not even "I looked at this and have no opinion", which is really
>> obstructing things.
>
> Well - the only way to avoid that is to set a reasonable deadline,
> after which there is a default decision. You'd hope this would
> motivate people to get involved in time.
>
> Wladimir
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development