Nostr notes by Richard Myers [ARCHIVE]

📅 Original date posted:2020-05-07 📝 Original message: ...

2023-06-09T13:00:03Z

In reply to nevent1q…ehxp
_________________________

📅 Original date posted:2020-05-07
📝 Original message:
Thanks Antoine for starting this thread, I've also been curious about the
current thinking on light clients and incentivized full node services after
seeing the LSAT work.

On Wed, May 6, 2020 at 11:40 AM Antoine Riard <antoine.riard at gmail.com>
wrote:

>
> Yeah, I hadn't time to read the spec yet but that was clearly something
> like LSATs I meaned speaking about monetary compensation to price
> resources. I just hope it isn't too much tie to HTTP because you may want
> to read/write over other communication channels like
> tx-broadcast-over-radio to solve first-hop privacy.
>

I think we should consider alternative peer-to-peer communication channels
both in the context of supporting light client users and encouraging full
node diversity.

Because block headers and block filters can be cached and forwarded we can
use pure broadcast systems like geostationary satellites or radio systems
with various trade-offs between range and bandwidth such as amateur radio,
unlicensed UHF and community WiFi. Protocol features that support
low-bandwidth broadcast and local peer-to-peer networks add resilience to
the Bitcoin network because they can not be as easily sybilled, censored or
surveilled en mass, as centralized ISPs can be.

Bandwidth is the primary limitation of these alternative last-mile networks
compared to nodes using wired internet connections. We would like all
Bitcoin nodes to operate as equal peers (flat network), but the reality is
that potential Bitcoin users do not have equal access to affordable
bandwidth from centralized providers. A system that lets light clients
access full nodes over local peer-to-peer networks could make self-custody
more accessible to light-client users and more local full nodes
incentivized to provide services over local peer-to-peer networks could
help increase the geographic diversity of full nodes.

For example, a light client could have inbound connections for block
headers and filters from 1) other light client peers via bluetooth, 2) an
ISP connected full node via a community WiFi network and 3) a blocksat
connected full node via a UHF mesh network. These scenarios in more detail:

1) two nearby light clients can exchange cached block headers, block
filters and full blocks over bluetooth or shared local WiFi - either tit
for tat or altruistically. Full blocks can be used to spot check block
filters and new block headers can help detect forks. There is no
communication cost and when the local internet is censored or down can help
(slowly) relay network state from any small set of operating links to the
outside internet.

2) a light client can query an ISP connected full node on the same
unmetered local WiFi network and exchange differences in block headers
opportunistically or pay for large missing ranges of headers, filters or
full blocks using a payment channel. Cost is reduced and privacy is
enhanced for the light client by not using a centralized ISP. Bandwidth for
running the full node can be amortized and subsidized by payments from
light clients who they resell data to.

3) an off-grid validating full node can receive block information from the
blocksat, but cross validate block headers from nearby full nodes connected
to ISPs and light clients with cached information via low-bandwidth
long-range UHF mesh radio. This full node has no fixed bandwidth costs and
can also earn LN payments from light clients to help subsidize their
initial hardware purchase.

I also believe a light-client could confirm specific transactions by
querying for Merkle proofs instead of full blocks when using a
low-bandwidth long distance and/or multi-hop radio link without the same
privacy linking concerns these queries would have if made using an internet
address tied to their identity or more specific physical location. I
wouldn't argue to add this to the core protocol, but like Watchtowers it's
a service that can monetize and support the operation of a full node.

-- Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200507/f6246a76/attachment.html>;

📅 Original date posted:2020-05-12 📝 Original message: ...

2023-06-09T13:00:03Z

In reply to nevent1q…gufj
_________________________

📅 Original date posted:2020-05-12
📝 Original message:
Thanks for sharing your thoughts ZmnSCPxj. I think I can summarize your
concern as: A node without direct internet connectivity can not rely on an
opportunistically incentivized local network peer for blockchain
information because the off-grid node's direct LN peers could collude to
not forward the payment.

On Mon, May 11, 2020 at 7:44 AM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:

> > 2) a light client can query an ISP connected full node on the same
> unmetered local WiFi network and exchange differences in block headers
> opportunistically or pay for large missing ranges of headers, filters or
> full blocks using a payment channel. Cost is reduced and privacy is
> enhanced for the light client by not using a centralized ISP. Bandwidth for
> running the full node can be amortized and subsidized by payments from
> light clients who they resell data to.
>
> A relatively pointless observation, but it seems to me that:
>
> * The light client is requesting for validation information, because...
> * ...its direct peers might be defrauding it, leading to...
> * ...the money it *thinks* it has in its channels being valueless.
>
> Thus, if the light client opportunistically pays for validation
> information (whether full blocks, headers, or filters), the direct peers it
> has could just as easily not forward any payments, thus preventing the
> light client from paying for the validation information.
>
> Indeed, if the direct peer *is* defrauding the light client, the direct
> peer has no real incentive to actually forward *any* payments --- to do so
> would be to reduce the possible earnings it gets from defrauding the light
> client.
> ("Simulating" the payments so that the light client will not suspect
> anything runs the risk that the light client will be able to forward all
> its money out of the channel, and the cheating peer is still potentially
> liable for any funds it originally had in the channel if it gets caught.)
>

One question I had is: how can a malicious direct payment peer "simulate" a
successful payment made by an off-grid victim peer to an information
source? The censoring peer wouldn't be able to return the preimage for a
payment they failed to forward. Also, since the information provider and
off-grid node can presumably communicate via their local network
connection, it would be obvious if all of the victims LN peers were failing
to forward payments (whether maliciously or due to routing failures) to an
information provider they could otherwise communicate with.

Any LN payments not monitored by a watchtower that are received by the
eclipsed off-grid victim node would be at risk in this attack scenario.
Likewise any layer 1 payments they received should be buried under
sufficient valid block headers before being relied on.

I don't see how a LN node one-step removed from a direct internet
connection is at more risk than an internet connected node eclipsed by
their ISP, for example. In both cases, failure to get timely blockchain
info should trigger warnings to stop accepting payments.

> What would work would be to use a system similar to watchtowers, wherein
> the validation-information-provider is prepaid and issues tokens that can
> be redeemed later.
> But this is not suitable for opportunistic on-same-WiFi where, say, a
> laptop is running a validation-information-provider-for-payment program on
> the same WiFi as a light-client mobile phone, if we consider that the
> laptop and mobile may have never met before and may never meet again.
> It would work if the laptop altruistically serves the blocks, but not if
> it were for (on-Lightning) payment.
>

There's another problem if we can't rely on a recurring relationship with
an information provider besides not being able to prepay for validation
information doesn't make sense. We don't want an information provider to
collect payments for serving invalid information. Maybe for very small
payments this isn't a problem, but ideally validity could be coded into the
HTLC.

For example, an alternative HTLC construct that only paid for valid 81 B
headers that hash to 32 B values with a number of leading zeros committed
to by the HTLC. It would make more economic sense for an internet gateway
node to serve valid mined header to nodes on their local WiFi network than
to create bogus ones with the same (high) amount of work.

> So it seems to me that this kind of service is best ridden on top of
> watchtower service providers.
>

Public watchtowers or some sort of HTTP proxy data cache similar to a
watchtower makes the most sense to me because they would be expected to be
economically motivated and LN payment aware. Full nodes could potentially
be incentivized to exchange new data with other nodes in a tit-for-tat way,
but I don't expect them to be incentivized by light clients using LN
micropayments in a server-client arrangement.

Network agents that monetize full node information services beyond channel
monitoring would be more than just a "Watchtower" for light clients. Would
they be more like incentivized Electrum servers? Are there still privacy
concerns when they serve generic/un-personalized headers/filters/blocks to
light clients? A personal, altruistic or friends and family watchtower is
also possible, but I'm thinking about how light clients without this
possibility can be served.

Happy new epoch,

-- Richard

--
Richard Myers
Decentralized Applications Engineer, goTenna
gotenna.com
@gotenna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200512/ea1a456e/attachment.html>;

📅 Original date posted:2019-10-01 📝 Original message: ...

2023-06-09T12:56:23Z

In reply to nevent1q…7wry
_________________________

📅 Original date posted:2019-10-01
📝 Original message:
Thanks Christian for pulling together this concise summary.

1. General agreement on the usefulness of noinput / anyprevoutanyscript /
> anyprevout.
>

I certainly support the unification and adoption of the sighash_noinput and
anyprevoutput* proposals to enable eltoo, but also to make possible better
off-chain protocol designs generally.

Among the various advantages previously discussed, the particular use case
benefits from eltoo I want to take advantage of is less interactive payment
channel negotiation.

In talking with people about eltoo this summer, I found most people
generally support adding this as an option to Lightning. The only general
concern I heard, if any, was the vague idea that rebindable transactions
could be somehow misused or abused.

I believe when these concerns are made more concrete they can be classified
and addressed.

I don't find too compelling the potential problem of a 'bad wallet
designer', whether lazy or dogmatic, misusing noinput. I think there are
simpler ways to cut corners and there will always be plenty of good wallet
options people can choose.

Because scripts signed with no_input signatures are only really exchanged
and used for off-chain negotiations, very few should ever appear on chain.
Those that do should represent non-cooperative situations that involve
signing parties who know not to reuse or share scripts with these public
keys again. No third party has any reason to spend value to a
multisignature script they don't control, whether or not a no_input
signature exists for it.

2. Is there strong support or opposition to the chaperone signatures
> introduced in anyprevout / anyprevoutanyscript? I think it'd be best to
> formulate a concrete set of pros and contras, rather than talk about
> abstract dangers or advantages.
>

As I mentioned before, I don't think the lazy wallet designer advantage is
enough to justify the downsides of chaperone signatures. One additional
downside is the additional code complexity required to flag whether or not
a chaperone output is included. By comparison, the code changes for
creating a no_input digest that skips the prevout and prevscript parts of a
tx is much less intrusive and easier to maintain.

3. The same for output tagging / explicit opt-in. What are the advantages
> and
> disadvantages?
>

I see the point ZmnSCPxj makes about tagged outputs negatively impacting
the anonymity set of taproot transactions. The suggested work around would
impose a cost to unilateral closes of an additional translation transaction
and not using the work around would cause a hit to anonymity for off-chain
script users. I feel both costs are too high relative to the benefit gained
of preventing sloppy reuse of public keys.

4. Shall we merge BIP-118 and bip-anyprevout. This would likely reduce the
> confusion and make for simpler discussions in the end.

I believe they should be merged. I also think both chaperone signatures and
output tagging should become part of the discussion of security
alternatives, but not part of the initial specification.

I understand the desire to be conservative with protocol changes that could
be misused. However, with just taproot and taproot public key types the
anyprevout functionality is already very opt-in and not something that
might accidentally get used. Belt-and-suspender protections like chaperone
signatures and tagged outputs have their own impacts on code complexity,
on-chain transaction sizes and transaction anonymity that also must be
considered.

I believe efforts like descriptors will help people follow best practices
when working with complex scripts without pushing extra complexity for
safety into the consensus layer of bitcoin. Anywhere we can make core code
simpler, and handle foot-guns in higher level non-consensus code, the
better.

_______________________________________________

> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20191001/f57d3131/attachment-0001.html>;

📅 Original date posted:2019-09-16 📝 Original message: ...

2023-06-09T12:56:04Z

In reply to nevent1q…5k4f
_________________________

📅 Original date posted:2019-09-16
📝 Original message:
Thanks for the feedback ZmnSCPxj.

I imagine a future where most people do not typically have single-signer
> ownership of coins onchain, but are instead share-owners of coins, with
> single-signer ownership occurring onchain only in the case of dispute or
> for long-term cold storage.
>

The change-in-membership ritual you describe seems like a good start for
elaborating on this idea.

Some aspects of multi-party Decker-Russell-Osuntokun channels have analogs
to a signet blockchain that use a n-of-n federation of signers. But other
places, like change-in-membership, do not have direct analogs.

For example, some signet concepts with multi-party channel analogs:

block script:
* the first 'update' and 'settle' transactions, aka 'setup' and 'refund'
transactions, define the set of signers that must sign subsequent channel
updates

genesis block:
* the initial 'funding' transaction, aka outpoint of the commitment
transaction, which establishes the funded channel

utxo set:
* the specific set of on-chain outputs from the 'settlement' transaction
that spends the balance of the latest 'update' transaction signed by the
complete set of channel parties.

mempool:
* the set of proposals for specific changes to the set of outputs from the
latest 'settlement' transaction (similar to update_add_htlc,
update_fail_htlc, etc)

Concepts where layer two channels do not have an obvious analog to a layer
one signet blockchain:

cooperative close:
* when all parties mutually agree to close the channel
* close the channel with a layer one transaction which finalizes the
outputs from the most recent channel output state
* should be optimized for privacy and low on-chain fees

membership change (ZmnSCPxj ritual):
* when channel parties want to leave or add new members to the channel
* close and reopen a new channel via something like a channel splicing
transaction to the layer one blockchain
* should be optimized for privacy and low on-chain fees paid for by parties
entering and leaving the channel

balance change (similar to membership change):
* when channel parties want to add or remove some of the finalized value in
the channel
* close and reopen a new channel via something like a channel splicing
transaction to the layer one blockchain
* should be optimized for privacy and low on-chain fees paid for by parties
adding and removing value from the channel

uncooperative close:
* when one or more nodes fails to sign the next channel state update
* use a layer one transaction to commit both finalized and un-finalized
outputs from the most recent channel output state
* script timeouts determine when channel parties should uncooperatively
close the channel if not all parties have signed the next 'update' and
'settlement' transaction

uncooperative membership change:
* a subset of channel parties might want to cooperatively sign a channel
splicing transaction to 'splice out' uncooperative parties

mining, mining reward and difficulty adjustment
* no equivalent concept for multi-party channels

transaction fees:
* updates to layer two channels do not incur transactions fees
* invalid updates dropped to layer one should be paid by cheating node
* splice in/out transactions should be paid by requesting signers only
* do transaction fees prevent 'griefing' attacks?

privacy:
* disassociate a particular update from signer(s)
* disassociate IP address of signers from signature
* using SIGHASH_ALL for cooperative closes

liveness:
* if signers know they will be offline, can they pre-sign updates that just
commit their own outputs, rather then splice out?
* contingent tap-leafs to splice out non-responsive signers

The "block" that would need to be signed by the participants would actually
> be a Decker-Russell-Osuntokun update+state transaction, and would commit to
> the UTXO set rather than the transaction set.
> Unless I misunderstand your meaning here.
>

Oops, ya, I did mean a "block" to be a particular Decker-Russell-Osuntokun
update+state transaction.

I think it will be useful to have these ideas in the back of my mind as I
work on making eltoo scripts that support multi-party channels.

--
Richard Myers
Decentralized Applications Engineer, goTenna
gotenna.com
@gotenna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190916/afc6b87e/attachment.html>;

📅 Original date posted:2019-09-09 📝 Original message: I ...

2023-06-09T12:56:03Z

In reply to nevent1q…yvmn
_________________________

📅 Original date posted:2019-09-09
📝 Original message:
I believe using the eltoo update scheme as a way to consolidate blocks of
off-chain transactions is an interesting idea worth exploring.

ZmnSCPxj brings up some limitations on arbitrary outputs scripts in eltoo.
Although using CSV is more complicated and outputs must also use
SIGHASH_NOINPUT [1], the ability to have multiple party channels and the
most used types of scripts makes eltoo compelling compared to LN-Penalty
for this kind of application.

The multiple party aspect in particular introduces an interesting way to
unify concepts from different second layer protocols like federated
sidechains and statechains (ht. aakselrod [2]).

Though the Statechains proposal relies on eltoo [3], I think what Christian
suggested does not try to solve the dynamic membership problem. That's why
I think of this as more an evolution of the channel factory paper towards
something like a federated sidechain.

I think this reconciliation between the off-chain model and the on-chain
> model, with many concepts cleanly mapping from one context to another
> (state outputs = UTXO, off-chain update = on-chain transactions,
> cut-through = confirmation, operation batching = block creation) is
> rather nice :-)

One additional concept that could be new to this off-chain blockchain model
would be something like batched multi-party loop-in/out. In a
Schnorr/Taproot world you could add signers/inputs and remove
signers/outputs with a single multi-signature negotiated off-chain. You'd
still like to limit these onchain txs, even if they are small, but updating
channels periodically seems like a straight forward way to address the
dynamic membership problem.

I guess this all gets back to how to design an off-chain protocol for
managing these negotiations. Ultimately I can imagine a sort of multi-party
eltoo based 'signet' with the same RPC interface, but different transaction
validation and block creation logic. Perhaps there would be a new message
where the channel parties would add their signature before forwarding a
valid block, and the block wouldn't be built on until all parties had
signed.

[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-August/001383.html
[2] https://twitter.com/stile65/status/1171030423394078720
[3]
https://medium.com/@RubenSomsen/statechains-non-custodial-off-chain-bitcoin-transfer-1ae4845a4a39
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190909/bb60fee0/attachment.html>;

📅 Original date posted:2022-04-24 📝 Original message:Hi ...

2023-06-07T23:08:00Z

In reply to nevent1q…w7qe
_________________________

📅 Original date posted:2022-04-24
📝 Original message:Hi darosior,

Thanks for sharing your thoughts on this.

> I would like to know people's sentiment about doing (a very slightly tweaked version of) BIP118 in place of
> (or before doing) BIP119.

Sounds good to me. Although from an activation perspective it may not be either/or, both proposals do compete for scarce reviewer time so their ordering will necessarily be driven by reviewer's priorities. My priority is eltoo which is why I focus on BIP-118.

> SIGHASH_ANYPREVOUTANYSCRIPT, if its "ANYONECANPAY" behaviour is made optional [0], can emulate CTV just fine.

For someone not as versed in CTV, why is it necessary that ANYONECANPAY be optional to emulate CTV? Is there a write-up that explains how APO-AS w/out ANYONECANPAY approximates CTV?

In the case of eltoo commit txs, we use bring-your-own-fee (BYOF) to late-bind fees; that means ANYONECANPAY will always be paired with APO-AS for eltoo. Settlement txs in eltoo use just APO and do not necessarily need to be paired with ANYONECANPAY.

I would guess making ANYONECANPAY the default for APO-AS was a way to squeeze in one more sighash flag. Perhaps there's another way to do it?

Including SIGHASH_GROUP with APO for eltoo is also tempting. Specifically so the counter-party who commits a settlement tx can use for fees their settled to_self balance. How to rejigger the sighash flags to accommodate both APO and GROUP may be worth some discussion.

The BIP-118 proposal will certainly benefit from having input from reviewers looking at other protocols than eltoo.

-- Richard

📅 Original date posted:2021-08-17 📝 Original message:I ...

2023-06-07T22:58:13Z

In reply to nevent1q…zqeq
_________________________

📅 Original date posted:2021-08-17
📝 Original message:I implemented some basic functional test scripts for the eltoo channel update scheme to better understand BIP-118/Anyprevout (APO). My tests are based on the rough Taproot eltoo scripts AJ Towns outlined on this list back in 2019. These tests also require AJ's APO branch of core. This is just the start of a real eltoo implementation, but I wanted to share it to get feedback and to help others interested in understanding APO and eltoo.

If you want to take a quick look, my functional tests are implemented in the function "test_tapscript_eltoo":
https://github.com/remyers/bitcoin/blob/eltoo-anyprevout/test/functional/simulate_eltoo.py#L1623

I've also written a blog post that describes in more detail how to run the tests and includes notes that I hope will be helpful to anyone else interested in APO and eltoo:
https://yakshaver.org/2021/07/26/first.html

I'd love to work with anyone else interested in eltoo and APO who wants to help extend and elaborate these tests. Any PRs for the eltoo tests or blog post would be very welcome.

Many thanks to AJ and Christian Decker who have both been very generous with their time in helping me understand the finer details of APO and eltoo.

All the best,

-- Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210817/3302d5cf/attachment.html>;

📅 Original date posted:2020-08-07 📝 Original message:When ...

2023-06-07T18:26:10Z

In reply to nevent1q…cqft
_________________________

📅 Original date posted:2020-08-07
📝 Original message:When you say that a special relay network might be more "smart about
replacement" in the context of ANYPREVOUT*, do you mean these nodes could
RBF parts of a package like this:

Given:
- Package A = UpdateTx_A(n=1): txin: AnchorTx, txout: SettlementTx_A(n=1)
-> HtlcTxs(n=1)_A -> .chain of transactions that pin UpdateTx_A(n=1) with
high total fee, etc.

And a new package with higher fee rate versions of ANYPREVOUT* transactions
in the package, but otherwise lower total fee:

- Package B = UpdateTx_B(n=1): txin: AnchorTx, txout: SettlementTx_B(n=1)
-> HtlcTxs(n=1)_B -> low total fee package

Relay just the higher up-front fee-rate transactions from package B which
get spent by the high absolute fee child transactions from package A:

- Package A' = UpdateTx_B(n=1): txin: AnchorTx, txout: SettlementTx_B(n=1)
-> HtlcTxs(n=1)_A -> ...chain of up to 25 txs that pin UpdateTx(n=1) with
high total fee, etc.

On Thu, Aug 6, 2020 at 5:59 PM Matt Corallo via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:

> In general, SIGHASH_NOINPUT makes these issues much, much simpler to
> address, but only if we assume that nodes can
> somehow be "smart" about replacement when they see a SIGHASH_NOINPUT spend
> which can spend an output that something else
> in the mempool already spends (potentially a different input than the
> relaying node thinks the transaction should
> spend). While ideally we'd be able to shove that (significant) complexity
> into the Bitcoin P2P network, that may not be
> feasible, but we could imagine a relay network of lightning nodes doing
> that calculation and then passing the
> transactions to their local full nodes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200807/79ab9202/attachment.html>;

📅 Original date posted:2020-05-20 📝 Original ...

2023-06-07T18:24:35Z

In reply to nevent1q…s0ls
_________________________

📅 Original date posted:2020-05-20
📝 Original message:I've been looking at using compressed block headers from the perspective of
a node that wants to use SMS messages to sync block headers. I realized
that it would be helpful if sendheaders2 took a parameter for how often to
send compact blockheaders. For example, in the case of an SMS transport
layer, it would make sense to request 4 headers at a time to optimally fill
a 160 byte SMS message.

Although using SMS messages would be about the most expensive way to
receive block headers, it is also the most universally available to
smartphone users, even where mobile data might be expensive or unavailable.
If we assume an SMS costs a bulk sender 0.025 USD per SMS, then sending 4
at a time (4 x 39 + 1 byte < 160 byte max SMS) reduces costs 1/4 to 252
instead of 1008 messages at a total weekly cost of ~7 instead of 26 USD.
Still not ideal, but a huge savings.

Since you're proposing a new message anyway, it doesn't break compatibility
to add a parameter for how often sendheaders2 chunks up recent headers.

On Fri, May 8, 2020 at 3:34 PM Will Clark via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:

> Hello list,
>
> I would like to propose a compressed block header scheme for IBD and block
> announcements. This proposal is derivative of previous proposals found on
> this list (see links in spec below) with some modifications and
> clarifications.
>
> The below specification (also found at
> https://github.com/willcl-ark/compressed-block-headers/blob/v1.0/compressed-block-headers.adoc
> ) details the compression recommended along with the generated bandwidth
> savings in the best-case scenario.
>
> I look forward to any feedback anyone has to offer on the specification
> itself, as well as any additions or objections to the motivation.
>
> Cheers,
> Will
>
>
> = Compressed block headers
> Will Clark <will8clark at gmail.com>
> v1.0, May 2020:
> :toc: preamble
> :toclevels: 4
>
>
> This work is a derivation of these mailing list posts:
>
> 1.
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-August/014876.html[bitcoin-dev:
> "Compressed" headers stream - 2017] (with resurrection
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015385.html[here]
> )
>
> 2.
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015851.html[bitcoin-dev:
> Optimized Header Sync]
>
> '''
>
> == Motivation
>
> Block headers as exchanged by nodes over the p2p network are currently 81
> bytes each.
>
> For low bandwidth nodes who are doing a headers-only sync, reducing the
> size of the headers can provide a significant bandwidth saving. Also, nodes
> can support more header-only peers for IBD and protection against eclipse
> attacks if header bandwidth is reduced.
>
> === Background
>
> Currently headers are sent over the p2p network as a vector of
> `block_headers`, which are composed of the following sized fields:
>
> [cols="<,>"]
> |===
> |Field |Size
>
> |Version |4 bytes
> |Previous block hash |32 bytes
> |Merkle root hash |32 bytes
> |Time |4 bytes
> |nBits |4 bytes
> |nonce |4 bytes
> |txn_count |1 byte
> |*Total* |81 bytes
> |===
>
> Some fields can be removed completely, others can be compressed under
> certain conditions.
>
> == Proposed specification
>
> === block_header2 data type
>
> The following table illustrates the proposed `block_header2` data type
> specification.
>
> [cols="<,>,>"]
> |===
> |Field |Size |Compressed
>
> |Bitfield |1 byte | 1 byte
> |Version |4 bytes |0 \| 4 bytes
> |Previous block hash |32 bytes |0 \| 32 bytes
> |Merkle root hash |32 bytes |32 bytes
> |Time |4 bytes |2 \| 4 bytes
> |nBits |4 bytes |0 \| 4 bytes
> |nonce |4 bytes |4 bytes
> |*Total* |81 bytes |range: 39 - 81 bytes
> |===
>
> This compression results in a maximum reduction from an 81 byte header to
> best-case 39 byte header. With 629,474 blocks in the current blockchain, a
> continuous header sync from genesis (requiring a single full 81 byte header
> followed by only compressed `block_header2`) has been tested to have its
> required bandwidth reduced from 50.98MB down to 25.86MB, a saving of 49%.
>
> ==== Bitfield
>
> To make parsing of header messages easier and further increase header
> compression, a single byte bitfield was suggested by gmaxwell footnote:[
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-December/015397.html].
> We propose the following amended bitfield meanings (bits re-ordered to
> match `headers2` field order):
>
> [cols="<,<"]
> |===
> |Bit |Meaning + field size to read
>
> |0 +
> 1 +
> 2 |version: same as the last *distinct* value 1st ... 7th (0 byte
> field) or a new 32bit distinct value (4 byte field).
> |3 |prev_block_hash: is omitted (0 byte field) or included (32 byte
> field)
> |4 |timestamp: as small offset (2 byte field) or full (4 byte field).
> |5 |nbits: same as last header (0 byte field) or new (4 byte field).
> |6 |possibly to signal "more headers follow" to make the encoding
> self-delimiting.
> |7 |currently undefined
> |===
>
> This bitfield adds 1 byte for every block in the chain, for a current
> total increase of 629,474B.
>
> ==== Version
>
> In most cases the Version field will be identical to one referenced in one
> of the previous 7 unique versions, as indicated by bits 0,1,2 of the
> Bitfield.
>
> To block 629,474 there were 616,137 blocks whose version was in the
> previous 7 distinct versions, and only 13,338 blocks whose version was not,
> this includes any version bit manipulation done via overt ASIC boost.
>
> [cols=">,>,>,>"]
> |===
> |Genesis to block |Current (B) |Compressed (B) |Saving (%)
>
> |629,474 |2,517,896 |53,352 |98
> |===
>
> ==== Previous block hash
>
> The previous block hash will always be the
> `SHA256(SHA256(<previous_header>))` so is redundant, presuming you have
> the previous header in the chain.
>
> [cols=">,>,>,>"]
> |===
> |Genesis to block |Current (B) |Compressed (B) |Saving (%)
>
> |629,474 |20,143,168 |0 |100
> |===
>
> ==== Time
>
> The timestamp (in seconds) is consensus bound, based both on the time in
> the previous
> header: `MAX_FUTURE_BLOCK_TIME = 2 * 60 * 60 = 7200`, and being greater
> than the `MedianTimePast` of the previous 11 blocks. Therefore this can be
> safely represented as an offset from the previous headers' timestamp using
> a 2 byte `signed short int`.
>
> [cols=">,>,>,>"]
> |===
> |Genesis to block |Current (B) |Compressed (B) |Saving (%)
>
> |629,474 |2,517,896 |1,258,952 |50
> |===
>
> ==== nBits
>
> nBits currently changes once every 2016 blocks. It could be entirely
> calculated by the client from the timestamps of the previous 2015 blocks
> footnote:[2015 blocks are used in the adjustment calculation due to an
> off-by-one error:
> https://bitcointalk.org/index.php?topic=43692.msg521772#msg521772";].
>
> To simplify 'light' client implementations which would otherwise require
> consensus-valid calculation of the adjustments, we propose to transmit this
> according to the <<Bitfield>> specification above.
>
> To block 629,474 there have been 298 nBits adjustments (vs an expected 311
> -- there was none before block 32,256).
>
> [cols=">,>,>,>"]
> |===
> |Genesis to block |Current (B) |Compressed (B) |Saving (%)
>
> |629,474 |2,517,896 |1,196 |99.6
> |===
>
> ==== txn_count
>
> txn_count is included to make parsing of these messages compatible with
> parsing of `block` messages footnote:[
> https://bitcoin.stackexchange.com/questions/2104/why-is-the-block-header-txn-count-field-always-zero].
> Therefore this field and its associated byte can be removed for
> transmission of compact headers.
>
> [cols=">,>,>,>"]
> |===
> |Genesis to block |Current (B) |Compressed (B) |Saving (%)
>
> |629,474 |629,474 |0 |100
> |===
>
> === Service Bit
>
> A new service bit would be required so that the nodes can advertise their
> ability to supply compact headers.
>
> === P2P Messages
>
> Three new messages would be used by nodes that enable compact block header
> support, two query messages: `getheaders2` and `sendheaders2` and one
> response: `headers2`.
>
> ==== `getheaders2` -- Requesting compact headers
>
> The new p2p message required to request compact block headers would
> require the same fields as the current `getheaders` message:
>
> [cols=">,<,<,<"]
> |===
> |Field Size |Description |Data type |Comments
>
> |4 |version |uint32_t |the protocol version
> |1+ |hash count |var_int |number of block locator hash
> entries
> |32+ |block locator hashes |char[32] |block locator object; newest
> back to genesis block (dense to start, but then sparse)
> |32 |hash_stop |char[32] |hash of the last desired
> block header; set to zero to get as many blocks as possible (2000)
> |===
>
> ==== `sendheaders2` -- Request compact header announcements
>
> Since
> https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki[BIP-130],
> nodes have been able to request to receive new headers directly in
> `headers` messages, rather than via an `inv` of the new block hash and
> subsequent `getheader` request and `headers` response (followed by a final
> `getdata` to get the tip block itself, if desired). This is requested by
> transmitting an empty `sendheaders` message after the version handshake is
> complete.]
>
> Upon receipt of this message, the node is permitted, but not required, to
> preemptively announce new headers with the `headers2` message (instead of
> `inv`). Preemptive header announcement is supported by the protocol version
> ≥ 70012 | Bitcoin Core version ≥ 0.12.0.
>
> For the motivational use-case it makes sense to also update this mechanism
> to support sending header updates using compact headers using a new message.
>
> ==== `headers2` -- Receiving compact headers
>
> A `headers2` message is returned in response to `getheaders2` or at new
> header announcement following a `sendheaders2` request. It contains both
> `length` and `headers` fields. The `headers` field contains a variable
> length vector of `block_header2`:
>
> |===
> |Field Size |Description |Data type |Comments
>
> |1+ |length |var_int |Length of `headers`
> |39-81x? |headers |block_header2[] |Compressed block headers in
> <<block_header2 data type>> format
> |===
>
> === Implementation
>
> * The first header in the first `block_header2[]` vector to a
> newly-connected client MUST contain the full nBits`, `timestamp`, `version`
> and `prev_block_hash` fields, along with a correctly populated `bitfield`
> byte.
> * Subsequent headers in a contiguous vector SHOULD follow the compressed
> <<block_header2 data type>> format.
> * Subsequent compressed headers supplied to an already-connected client
> (requesting compressed headers), SHOULD follow the compressed
> <<block_header2 data type>> format.
>
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

--
Richard Myers
Decentralized Applications Engineer, goTenna
gotenna.com
@gotenna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200520/f9b6f40d/attachment-0001.html>;

📅 Original date posted:2020-05-12 📝 Original message:Thanks ...

2023-06-07T18:24:27Z

In reply to nevent1q…2y8z
_________________________

📅 Original date posted:2020-05-12
📝 Original message:Thanks for sharing your thoughts ZmnSCPxj. I think I can summarize your
concern as: A node without direct internet connectivity can not rely on an
opportunistically incentivized local network peer for blockchain
information because the off-grid node's direct LN peers could collude to
not forward the payment.

On Mon, May 11, 2020 at 7:44 AM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:

> > 2) a light client can query an ISP connected full node on the same
> unmetered local WiFi network and exchange differences in block headers
> opportunistically or pay for large missing ranges of headers, filters or
> full blocks using a payment channel. Cost is reduced and privacy is
> enhanced for the light client by not using a centralized ISP. Bandwidth for
> running the full node can be amortized and subsidized by payments from
> light clients who they resell data to.
>
> A relatively pointless observation, but it seems to me that:
>
> * The light client is requesting for validation information, because...
> * ...its direct peers might be defrauding it, leading to...
> * ...the money it *thinks* it has in its channels being valueless.
>
> Thus, if the light client opportunistically pays for validation
> information (whether full blocks, headers, or filters), the direct peers it
> has could just as easily not forward any payments, thus preventing the
> light client from paying for the validation information.
>
> Indeed, if the direct peer *is* defrauding the light client, the direct
> peer has no real incentive to actually forward *any* payments --- to do so
> would be to reduce the possible earnings it gets from defrauding the light
> client.
> ("Simulating" the payments so that the light client will not suspect
> anything runs the risk that the light client will be able to forward all
> its money out of the channel, and the cheating peer is still potentially
> liable for any funds it originally had in the channel if it gets caught.)
>

One question I had is: how can a malicious direct payment peer "simulate" a
successful payment made by an off-grid victim peer to an information
source? The censoring peer wouldn't be able to return the preimage for a
payment they failed to forward. Also, since the information provider and
off-grid node can presumably communicate via their local network
connection, it would be obvious if all of the victims LN peers were failing
to forward payments (whether maliciously or due to routing failures) to an
information provider they could otherwise communicate with.

Any LN payments not monitored by a watchtower that are received by the
eclipsed off-grid victim node would be at risk in this attack scenario.
Likewise any layer 1 payments they received should be buried under
sufficient valid block headers before being relied on.

I don't see how a LN node one-step removed from a direct internet
connection is at more risk than an internet connected node eclipsed by
their ISP, for example. In both cases, failure to get timely blockchain
info should trigger warnings to stop accepting payments.

> What would work would be to use a system similar to watchtowers, wherein
> the validation-information-provider is prepaid and issues tokens that can
> be redeemed later.
> But this is not suitable for opportunistic on-same-WiFi where, say, a
> laptop is running a validation-information-provider-for-payment program on
> the same WiFi as a light-client mobile phone, if we consider that the
> laptop and mobile may have never met before and may never meet again.
> It would work if the laptop altruistically serves the blocks, but not if
> it were for (on-Lightning) payment.
>

There's another problem if we can't rely on a recurring relationship with
an information provider besides not being able to prepay for validation
information doesn't make sense. We don't want an information provider to
collect payments for serving invalid information. Maybe for very small
payments this isn't a problem, but ideally validity could be coded into the
HTLC.

For example, an alternative HTLC construct that only paid for valid 81 B
headers that hash to 32 B values with a number of leading zeros committed
to by the HTLC. It would make more economic sense for an internet gateway
node to serve valid mined header to nodes on their local WiFi network than
to create bogus ones with the same (high) amount of work.

> So it seems to me that this kind of service is best ridden on top of
> watchtower service providers.
>

Public watchtowers or some sort of HTTP proxy data cache similar to a
watchtower makes the most sense to me because they would be expected to be
economically motivated and LN payment aware. Full nodes could potentially
be incentivized to exchange new data with other nodes in a tit-for-tat way,
but I don't expect them to be incentivized by light clients using LN
micropayments in a server-client arrangement.

Network agents that monetize full node information services beyond channel
monitoring would be more than just a "Watchtower" for light clients. Would
they be more like incentivized Electrum servers? Are there still privacy
concerns when they serve generic/un-personalized headers/filters/blocks to
light clients? A personal, altruistic or friends and family watchtower is
also possible, but I'm thinking about how light clients without this
possibility can be served.

Happy new epoch,

-- Richard

--
Richard Myers
Decentralized Applications Engineer, goTenna
gotenna.com
@gotenna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20200512/ea1a456e/attachment-0001.html>;